Software increases precision when assessing children with autism

A program developed by Brazilian and US researchers measures visual attention of children through videotaped assessment sessions

By Elton Alisson

Agência FAPESP – Psychology professionals may soon have a computer tool to aid them in the process of triage of children with Autism Spectrum Disorder (ASD).

A group of researchers from the University of Minnesota and Duke University, in the United States, together with colleagues from the Computer Sciences Institute of the University of Campinas (Unicamp), have developed software to automatically analyze videotaped autism assessment tests.

Several of the findings from the test analyses performed by the software were described in the June issue of the journal Autism Research and Treatment.

“The idea is that the software could help increase the accuracy of assessing children with autism,” Thiago Vallin Spina, doctoral candidate at the Computer Sciences Institute of Unicamp and one of the authors of the journal article, told Agência FAPESP.

“Our goal is to have a version of the software that could be used in pre-schools, for example, to more precisely assess children suspected of having autism and send them for further diagnostic testing with specialists as soon as possible,” affirmed Spina, FAPESP fellowship recipient, whose research advisor is Prof. Alexandre Xavier Falcão.

According to Spina, recent studies indicate that many children with ASD present behavioral markers of autism in their first year, such as difficulty in disengaging attention from a particular point to track a visual stimulus.

In an attempt at earlier detection of these disturbances in child development with the aim of beginning intensive clinical intervention, three types of behavioral tests are usually administered, based on the Autism Observation Scale for Infants (AOSI), to assess a child’s visual attention.

The first test consists of shaking a noisy toy on the left side of the infant and then shaking a second noisy toy on the right side to assess the time it takes for the child to respond to the second stimulus by shifting his eyes.

In the second test, a toy is moved horizontally near the face in the infant’s field of vision to determine if there is any tendency to track the object’s movement.

In the third test, a ball is rolled towards the child to determine whether the child grabs the ball and establishes visual contact and social interaction with the specialist.

However, diagnosis has been complicated because these tests occur in real time. The problem is that while they are taking place, the practitioner needs to not only control the stimulus but also calculate how long the child takes to respond. These features make the diagnosis imprecise, according to Spina. “The time it takes for the child to react to the stimuli considered in the measurements of visual attention is from one to two seconds,” he said.

“The ASD diagnosis obtained with these tests depends largely on the experience and accuracy of the specialist in precisely identifying the time it takes for the child to respond to the stimulus,” Spina stated.

Automatic measurements

To try to increase the precision of the results, the researchers have developed image processing and computer vision algorithms (sequences of commands) that automatically measure the visual attention of children during the behavioral tests to triage ASD from videotaped assessment sessions.

To do this, they utilized videotaped recordings of behavioral tests administered during ASD assessment sessions conducted by Amy Esler, pediatrics professor at the University of Minnesota, with a group of 12 children between the ages of 5 and 18 months, identified to undergo the tests. The recordings were made during Spina’s research internship at the U.S. university in the group led by Prof. Guillermo Sapiro.

“We placed two high-resolution conventional cameras in the room where the assessment sessions take place. One camera was positioned on the center of Prof. Esler’s desk and pointed toward the child’s side. The other was located in a corner of the room to obtain a view of both the clinician and the participant during the session,” Spina explained.

The software was able to track the direction of the visual attention based on information about the face of the child undergoing the behavioral tests. To do this, the computer system initially identified the direction of the eyes and nose of the children in the first frame of the videotape relative to the object that was presented to the children.

Based on computer vision algorithms, the software assessed whether the direction of the eyes and nose of the child repeated the same pattern or moved during the later frames of the video.

With this information, the software was able to establish vectors of the child’s eye and nose movement from one frame to the next. Then, using geometric measurements, the software estimated the direction in which the child was looking relative to the object during the tests –toward the object or not toward the object.

“Because we knew what direction the child was looking in the first frame of the video and what position the object was in, the software was able to track the child’s eye movement and indicate whether it correlated with the direction of the toy,” Spina explained.

The results of the video analyses made using the software were compared with the clinical assessment conducted by Esler based on the real-time observation of the tests and the videos themselves – without having undergone software analysis – and with assessments by two undergraduate psychology students and a psychologist who was not an expert in autism.

The comparison showed that the program and the specialist were equally able to detect behavioral signs indicative of autism. The comparison further showed that the program performed better than the nonexpert psychologist and psychology students.

“The program enables us to record the child’s reaction time to a visual stimulus within tenths of seconds because each second of a video has 30 frames,” Spina explained.

Possible contributions

The software represents an initial stage in a long-term project developed by a multidisciplinary group of researchers in the field of psychology, computer vision and machine learning who are seeking to develop low-cost, automatic tools for data analysis that could be useful in achieving earlier identification of children with ASD.

Although autism symptoms often appear early and the behavioral disturbances can be diagnosed during the early years of life, the average age for ASD diagnosis in Brazil, as in countries such as the United States, is closer to age 5, say the article’s authors.

“The software could help psychology professionals and ASD researchers identify autism risk factors by analyzing large amounts of natural behavior videos of children at home or at school or from sessions of clinical analysis,” Spina said.

“Furthermore, it could open doors to improving current assessment protocols and discover new behavioral characteristics of children with ASD, increasing the granularity of the analyses and providing data on a finer scale,” he added.

In his doctoral research, Spina is using algorithms to conduct video analyses of motor behavior involving arm position and movement identified as a possible new sign characteristic of autism.

Called arm asymmetry, the behavior has been identified during studies conducted in recent years on children with autism between the ages of 18 and 24 months.

The authors of the study have identified that, unlike the gait of children without autism, whose arms tend to stay alongside their bodies in a symmetrical position with a slight swing, children with autism present an asymmetrical arm position, with one arm extended and the other flexed horizontally forward.

“We’ve developed software to measure this specific motor behavior. The idea is to expand its application to measure other movements that are also quite characteristic of children with ASD, such as rocking the torso back and forth,” Spina explained.

The group of researchers at Duke University is developing an application for tablets designed to replace the visual attention tests are that performed today. The objective is to imitate the same types of interactions as those measured by tests with toys and balls but not to use actual objects.

“They are discussing which types of behaviors indicative of autism could be identified by this tablet application,” explained Spina, who is not directly participating in the project. “We plan to continue to closely cooperate with Sapiro at Duke University on a joint project once my doctorate is complete.”

The article, Computer vision tools for low-cost and noninvasive measurement of autism-related behaviors in infants (doi: 10.1155/2014/935686), by Spina and colleagues, may be read in the journal Autism Research and Treatment at: www.hindawi.com/journals/aurt/2014/935686.