An Intelligent Augmented Lifelike Avatar App for Virtual Physical Examination of Suspected Strokes

An intelligent-augmented lifelike avatar mobile app (iLAMA) that integrates computer vision and sensor readings to automate and streamline the NIH Stroke Scale (NIHSS) physical examination is presented. The user interface design is optimized for elderly patients while the app showcases an animated lifelike 3D model of a friendly physician who walks the user through the exam. The standardized NIHSS examination included in iLAMA consists of five core tasks. The first two tasks involve rolling the eyes to the left and then to the right, and then smiling as wide as the user can. The app determines facial landmarks and analyzes the palsy of the face. The next task is to extend the arm and hold the phone at the shoulder level, and the smart phone gyroscope is used to detect acceleration to determine possible weakness in the arm. Next, the app tracks the location of the hand keypoints and determines possible ataxia based on the precision and accuracy of the locations of the touches. Finally, the app determines the user's forward acceleration in walking and possible imbalances using the accelerometer. The app then sends analyzed results of these tasks to the neurologist or stroke specialist for review and decisions.Clinical Relevance- The physical examination of a stroke patient is a time consuming and repetitive process, and there is a lack of infrastructure and resource to monitor patient in post-stroke recovery after they leave the hospital for home or rehabilitation facilities. iLAMA app aims to automate a subset of the NIHSS physical examinations in measuring motor function recovery and also allows individual patients to track their performance over time. It will be an essential component in monitoring rehabilitation recovery and therapy effectiveness after hospitalization and can easily scaled to lo help millions of patients at a fraction of the cost.