ReViSe: An End-to-End Framework for Remote Measurement of Vital Signs

Abstract

Photoplethysmography (PPG) is a fast, inexpensive and convenient method of collecting biometric data from fingertip videos. Remote Photoplethysmography (rPPG) is a contactless method to remotely calculate vital signs from facial videos. Post COVID-19 pandemic there has been a surge in online medical advising that requires monitoring of vitals at the remote location of the patient. This can be facilitated with the PPG and rPPG techniques which involves processing video frames to obtain skin pixels, extracting the cardiac data from it and applying signal processing filters to extract the Blood Volume Pulse (BVP) signal. Different algorithms are applied to the BVP signal to estimate the various vital signs. We implemented an end-to-end framework to measure a person's Heart Rate (HR), Heart Rate Variability (HRV), Oxygen Saturation (SpO2), Respiration Rate (RR), Blood Pressure (BP), and stress level from the face video using rPPG. The accuracy and robustness of the framework was validated with the help of volunteers whose vitals were also measured with medical devices. To protect user's privacy a face masking technique was employed. The rPPG technique is highly sensitive to illumination and motion variation which can lead to inaccurate results due to these factors. We analyzed the various factors that affect the results such as physical displacement of the person, facial movements, non-uniform illumination of the face regions, changes in light, and low light conditions. By displaying necessary messages to guide the user to reduce the noise and thereby the anomalies, a robust framework has been built which yields a cleaner BVP signal. We enhanced the existing PPG and rPPG models with an additional deep learning based BP estimation model. Due to the unavailability of a public dataset with videos and BP readings, this model was first trained on a public PPG dataset and then on self-created FingerVideo-BP and FaceVideo-BP datasets. On the FingerVideo-BP dataset the model gave a MAE of 10.3 mmHg and 8.2 mmHg, and on the FaceVideo-BP dataset a MAE of 7.9 mmHg and 5.8 mmHg for Systolic Blood Pressure (SBP) and Diastolic Blood Pressure (DBP) respectively.