Software identifies expressions of pain in newborns

Computer program developed by Brazilian researchers monitors babies in neonatal ICUs and can assist caregivers.

By Diego Freire

Agência FAPESP – Researchers at the São Paulo School of Medicine (EPM), which is part of the Federal University of São Paulo (UNIFESP), have developed a computer program that detects pain-related facial expressions in newborn babies and can therefore be of assistance in care management, enabling more agile and precise interventions.

The software design resulted from a research project conducted by Ruth Guinsburg with support from FAPESP.

According to Guinsburg, the initiative arose in response to the experience of healthcare professionals working in neonatal intensive care units (ICUs), many of whom say they have difficulty recognizing and interpreting signs of pain in infants.

“This subjectivity hinders intervention when needed, since a number of factors may lead a newborn baby to express discomfort that’s not necessarily pain-related. The research enabled us to create a useful tool for monitoring pain in infants as a routine activity for neonatal units,” Guinsburg told Agência FAPESP.

Pain in preverbal infants is typically recognized using behavioral and physiological indicators, such as simple motor responses, facial expressions, and crying. The software created at UNIFESP is based on the Neonatal Facial Coding System (NFCS), a widely used scale for recognizing pain-related facial movements. The researchers converted the scale into computer language in collaboration with EPM’s Health Informatics Department (DIS) and professionals at the University of Mogi das Cruzes, also in São Paulo State.

Design work on the software mechanisms began in 2009, after UNIFESP’s Research Ethics Committee approved the project. Thirty newborns were filmed at Hospital São Paulo, EPM’s teaching hospital, between June and August 2013.

“We had to take great care to prepare the families so that there were no misunderstandings about the filming, which was done during the performance of painful procedures prescribed by physicians, such as capillary, venous or arterial blood collection and intramuscular or subcutaneous injections, necessary for care of neonates and not performed for research purposes,” Guinsburg said.

The babies selected were between 24 and 168 hours of age, without any need for ventilator support or a gastric tube and without congenital malformations. Their parents or legal guardians signed a waiver authorizing capture of the images used in the research.

During the monitoring period, the babies’ facial expressions were photographed in real time by three cameras positioned to the left and right of the subjects and above them.

The software, based on biometric identification, mapped and detected 66 points on each baby’s face. These were then narrowed down to 16 main nodal points, from which were selected those that moved most when acute pain caused by a medical procedure was expressed.

The distances between the points served as a basis for detecting facial expressions that evidenced signs of pain according to the scale adopted by the researchers: bulging brow, eyes squeezed tightly shut, deepening of the nasolabial furrow, open lips, and pursed lips.

The researchers recorded 5,644 images, averaging 188 per newborn. They then tested the analysis performed by the software to see how well it matched assessments by six healthcare professionals who had experience in recognizing neonatal pain and specialized in neonatology. Three images were compared for each baby: two recorded in a resting state in the absence of pain and one during a painful procedure.

“We observed that the software didn’t detect pain-related expressions in 85% of the images captured while the babies were resting and not being subjected to painful procedures,” Guinsberg said. “However, the program detected pain-related expressions in 100% of the images taken during the performance of painful procedures, whereas some professionals detected them in only 77%.”

Sick babies

For Guinsburg, the accuracy with which the software detected expressions makes it a potentially important tool for assisting healthcare professionals in neonatal ICUs.

“The pain felt by a newborn must always be seen by someone else. This perception depends on decoding by the carer and depends on the time available for monitoring as well as on subjective factors such as empathy. Automation of this surveillance can contribute to the baby’s well-being and assist carers by leading to more assertive intervention,” Guinsburg said.

The researchers are now working on a special version of the software for use in monitoring sick babies, by upgrading the cameras and adapting the system for bedside use.

The research findings were published in an article in the journal Acta Paediatrica by Tatiany Marcondes Heiderich, Ana Teresa Stochero Leslie and Guinsburg, all of whom are affiliated with UNIFESP’s Pediatrics Department. The article is available at onlinelibrary.wiley.com/doi/10.1111/apa.12861/full.