Changes in infant alertness and neurological status may indicate serious underlying conditions, yet these are typically evaluated through physical examinations that occur intermittently and involve subjective judgment. There is a clear need for dependable, ongoing monitoring techniques. We proposed that a computer vision technique using pose estimation artificial intelligence (AI) to monitor movement could help forecast neurological alterations in the neonatal intensive care unit (NICU). Video recordings synchronized with electroencephalography (video-EEG) were gathered from infants under 1 year corrected age at Mount Sinai Hospital, a level IV urban NICU in New York City, spanning February 1, 2021, to December 31, 2022. A deep learning pose estimation model was trained on these video recordings, with 14 key anatomical landmarks annotated across 25 frames per infant. Classifiers were subsequently developed using these landmark positions to forecast cerebral dysfunction (identified via EEG interpretation by a neurologist specializing in epilepsy) and sedation status (determined by the delivery of sedative drugs). The study assembled the most extensive video-EEG dataset currently available, comprising 282,301 video minutes from 115 infants drawn from a varied demographic. Infant pose estimation proved accurate in cross-validation, unseen frames, and unseen infants, yielding receiver operating characteristic area under the curve (ROC-AUC) values of 0.94, 0.83, and 0.89, respectively. Median movement levels rose with advancing age; after adjusting for age, movement decreased in association with sedative administration and among infants exhibiting cerebral dysfunction (all P < 5 × 10-3, based on 10,000 permutations). The model demonstrated strong performance in predicting sedation across cross-validation, held-out time segments, and held-out infants (ROC-AUCs 0.90, 0.91, 0.87). Similarly, it effectively predicted cerebral dysfunction (ROC-AUCs 0.91, 0.90, 0.76). These results establish that pose AI can be effectively deployed in an intensive care environment and that an EEG-based diagnosis of cerebral dysfunction can be inferred solely from video recordings. Deep learning combined with pose AI represents a promising, scalable, and minimally intrusive tool for neurological remote monitoring in the NICU.
