Computer vision and behavioral phenotyping: an autism case study

Despite significant recent advances in molecular genetics and neuroscience, behavioral ratings based on clinical observations are still the gold standard for screening, diagnosing, and assessing outcomes in neurodevelopmental disorders, including autism spectrum disorder. Such behavioral ratings are subjective, require significant clinician expertise and training, typically do not capture data from the children in their natural environments such as homes or schools, and are not scalable for large population screening, low-income communities, or longitudinal monitoring, all of which are critical for outcome evaluation in multisite studies and for understanding and evaluating symptoms in the general population. The development of computational approaches to standardized objective behavioral assessment is, thus, a significant unmet need in autism spectrum disorder in particular and developmental and neurodegenerative disorders in general. Here, we discuss how computer vision, and machine learning, can develop scalable low-cost mobile health methods for automatically and consistently assessing existing biomarkers, from eye tracking to movement patterns and affect, while also providing tools and big data for novel discovery.