A unique processing engine is at the heart of IntelliProve

We detect the blood flow in the face using the camera on your mobile device and measure health parameters through innovative signal processing and machine learning techniques.

From a video of the face to health insights

Just point your smartphone camera at the face and we’ll extract health parameters, such as respiratory rate and mental health index, directly from the information captured by the lens.

Face detection / quality check

Correcting for face and camera motion

01

A video camera records a short video feed of the face. Using different algorithms, this video is corrected for face and camera motion. The recording conditions (e.g. light) are checked to ensure a qualitative measurement.

ROI detection

Detecting the region-of-interest in the face

02

Small facial regions of interest (ROI) with a good skin blood perfusion are selected and cropped from the full image.

ROI detection

Biomedical signal selection

Extracting the necessary information from the ROI

03

Ambient light penetrates the skin and reflects off blood vessels. The pixel values in the facial ROI are used to construct a raw signal. This signal harbours tiny changes in colour, originating from the pulsation of the heart, which are key to measuring the health parameters.

Signal processing / machine learning

Processing the raw signal into a qualitative PPG signal

04

The raw signal is converted into a qualitative photoplethysmogram (PPG) waveform and facial microexpression patterns using innovative signal processing and machine learning techniques.

Signal processing / machine learning

Health parameters

Calculating the vital parameters from the waveform

05

Health parameters are accurately calculated from this waveform using unique IntelliProve algorithms.

Our engine in numbers

0
Heart beats detected
0
Years of scientific research
0
Patients included
0 %
Heart rythm accuracy

Our engine in numbers

9

Our technology is backed by more than 9 years of academic and clinical research.

1,000,000+

More than 1 million datapoints are actively used for testing and validating our models.

94

Percentage overall algorithm accuracy

40,000+

More than 40,000 photoplethysmogram waveforms are being processed to train our engine.

Science behind IntelliProve

From face to biomedical signal

IntelliProve makes use of photoplethysmography (PPG), an optical method to measure cardiac-synchronous blood volume change in body extremities such as the face, finger and earlobe. As the heart pumps blood, the volume of blood in the arteries and capillaries changes by a small amount in sync with the cardiac cycle. This change in blood volume in the arteries and capillaries underneath the skin leads to small changes in the skin color (visible in the Red/Green/Blue spectrum), from which a PPG waveform is estimated. 

The PPG signal is typically collected using a device (e.g. pulse oximeter) that emits light and measures the amount of light that is absorbed or reflected by the tissue. In our case, we use ambient light from the environment (the light source) and the camera from a smart device (the sensor) to collect the ‘remote’-PPG. The region of interest is the particularly the face because of the ‘thin’ tissue and optimum blood perfusion. 

References:

  • Remote plethysmographic imaging using ambient light. Verkruysse W. et al. 2008. Opt. Express 16, 21434-21445.
  • Use of ambient light in remote photoplethysmographic systems: comparison between a high-performance camera and a low-cost webcam. Sun Y. et al. 2012 Mar 23. J Biomed. Opt. 17(3):037005.
  • DistancePPG: Robust non-contact vital signs monitoring using a camera. Kumar M. et al. 2015 Apr 6. Biomed Opt Express. 6(5):1565-88.
  • Effectiveness of Remote PPG Construction Methods: A Preliminary Analysis. Haugg F. et al. 2022 Sep 20. Bioengineering 9(10):485.

Relying on modern computer vision technology, it is possible to detect and extract facial dynamics and microexpression patterns, reflecting the hemifacial asymmetry in emotion expressions. This makes it possible to objectively detect and predict mental disorders. Typical patterns are raised eyebrows, leaden gazes, swollen faces and hang-dog mouth expressions.

Reference:

  • The Face of Affective Disorders. Pilz, C. S. et al. 2022 Aug 2; arXiv preprint arXiv:2208.01369.

It is crucial to extract qualitative biomedical signals from the face in different measurement conditions. Our technology effectively corrects for different variables such as face motion and camera movement (through region-based motion tracking), lighting variations (by outlier detection and detrending filters) and smartphone variations (by resampling techniques).

Additionally, prior to every measurement, a quality check is performed to assess the video measurement conditions. This includes, for example, checking if there is sufficient lighting, checking the face-camera distance or analyzing the user’s motion. 

Finally, at every step of the algorithm, the intermediate signal quality is assessed by in-house quality indices based on science-backed quality metrics. If quality criteria are not met, no results will be returned.

References:

  • The Effects of Motion Artifact and Low Perfusion on the Performance of a New Generation of Pulse Oximeters in Volunteers Undergoing Hypoxemia. Gehring, H. et al. 2002. Respir Care vol. 47.
  • Does the pleth variability index indicate the respiratory-induced variation in the plethysmogram and arterial pressure waveforms? Cannesson M. et al. 2008. Anesth. Analg. 106, 1189–1194.
  • Ability of the Masimo pulse CO-Oximeter to detect changes in hemoglobin. Colquhoun D. et al. 2012. J. Clin. Monit. Comput. 26, 69–73.
  • Analysis: An optimal filter for short photoplethysmogram signals. Liang, Y. et al. 2018. Sci. Data 5.
  • Optimal signal quality index for photoplethysmogram signals. Elgendi M. 2016. Bioengineering 3.
  • Two-stage approach for detection and reduction of motion artifacts in photoplethysmographic data. Krishnan R. et al. 2010. IEEE Trans. Biomed. Eng. 57, 1867–1876.

From biomedical signal to vital parameter

The PPG signal reflects changes in blood volume, which are caused by changes in the amount of blood that is flowing through the blood vessels. Different approaches are developed in order to analyze the PPG signal and determine indices which are used in evaluating and assessing vital physiological parameters. Different approaches can be used to measure the vital parameters from a PPG waveform:

  • Heart rate can be estimated by counting the systolic peaks per minute in the PPG waveform.
  • RR can be estimated from a PPG waveform by different techniques in association with amplitude modulation (AM), frequency modulation (FM), and baseline wandering (BW). Additionally, computer visioning can be used to monitor the chest movements and movements in the face (e.g. mouth, nose) of the patient related to the breathing.
  • Machine learning models can use different key parameters or features of the PPG waveform (e.g. PPG height, systolic/diastolic area, dicrotic notch height, peak-to-peak interval,…) to predict blood pressure.
  • Oxygenated hemoglobin (HbO2) differs in color from deoxygenated hemoglobin (Hb). According to the Beer-Lambert law and by using quantitative light-absorption measurements, the percentage of hemoglobin that is oxygenated in blood (i.e. SpO2) can be determined.

References:

  • Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam. Poh M. et al. 2010 Oct 14. IEEE Transactions on Biomedical Engineering 58(1):7-11, 
  • Non-contact detection of oxygen saturation based on visible light imaging device using ambient light. Kong L. et al. 2013 Jul 29. Opt Express. 21(15):17464-71.
  • Camera-based pulse-oximetry – validated risks and opportunities from theoretical analysis. Van Gastel M. et al. 2017 Dec 5. Biomed. Opt. Express. 9(1):102-119.
  • Blood Pressure Estimation from Photoplethysmogram Using a Spectro-Temporal Deep Neural Network. Slapničar G. et al. 2019 Aug 4. Sensors 19(15):3420.
  • Non-Contact Respiratory Monitoring Using an RGB Camera for Real-World Applications. Romano C. et al. 2021 Jul 29. Sensors 21(15):5126.
  • Assessment of Non-Invasive Blood Pressure Prediction from PPG and rPPG Signals Using Deep Learning. Schrumpf F. et al. 2021 Sep 8. Sensors. 21(18):6022.

The evaluation of the performance of the vital parameter algorithms is performed by step-by-step comparative analyses with the current state-of-the-art medical devices (pulse oximeter, sphygmomanometer,…). The accuracy and precision are checked by calculating for example the Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE) and Correlation Coefficient (R).

Currently, more than 1 million datapoints (resulting from various collaborations with clinical organisations) are actively used for testing and validating our models.

From biomedical signal to mental biomarker

Heart Rate Variability, or HRV for short, is a non-invasive measure of your autonomic nervous system, which is the body’s main control center. It is widely considered as one of the best objective metrics for physical fitness and determining your body’s readiness to perform. HRV is literally the variance in time between the beats (NN interval) of your heart.

Heart rate variability (HRV) can be derived from a ECG or PPG signal. A commonly used statistical metric for representing short-time HRV (over a time duration of the order of 10s to 1 min) is the Standard Deviation of NN intervals (SDNN).

References:

  • Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. 1996 Mar 1. Circulation 93(5):1043-65.
  • Dependence of Heart Rate Variability on Stress Factors of Stress Response Inventory. Salahuddin L. 2007 Jun 22. 9th International Conference on e-Health Networking, Application and Services:236-239
  • The relationship of autonomic imbalance, heart rate variability and cardiovascular disease risk factors. Thayer J. 2010 May 28. Int. J. Cardiol. 141(2):122-31.
  • A meta-analysis of heart rate variability and neuroimaging studies: implications for heart rate variability as a marker of stress and health. Thayer J. et al. 2011 Nov 30. Neurosci Biobehav Rev. 36(2):747-56.
    Heart rate variability analysis: physiological foundations and main methods. Baevsky R. et al. 2017. Cardiometry. 66-76. 10.12710

The intimate connection between the heart and the brain is well described. HRV is known to be regulated by the prefrontal cortex (i.e. brain regions involved in the regulation of ANS activity), is considered to indirectly reflect complex patterns of brain activation and provides information on the central nervous system (CNS) functional organization and the bidirectional interaction between the CNS and the ANS.

There is substantial evidence that HRV is not only a risk marker for cardiovascular disease, but that also decreases in HRV have close associations with depression, schizophrenia, and post-traumatic stress disorder. On the other hand, high HRV is generally associated with a relaxed and resilient mental state.

References:

  • Autonomic balance revisited: panic anxiety and heart rate variability. Friedman B. et al. 1998 Feb 13. J. Psychosom. Res. 44(1):133-51.
  • Relationship between major depression and heart rate variability. Clinical consequences and implications for antidepressive treatment. Agelink M. et al. 2002 Dec 15. J. Psychiatry Res. 113(1-2):139-49.
  • The effect of mental stress on heart rate variability and blood pressure during computer work. Hjortskov N. et al. 2004 Feb 27. Eur J Appl Physiol 92:84–89.
  • Impact of depression and antidepressant treatment on heart rate variability: a review and meta-analysis. Kemp A. et al. 2010 Jun 1. Biol. Psychiatry. 67(11):1067-74.
  • Low heart rate variability in patients with clinical burnout. Lennartsson A et al. 2016 Nov 25. Int. J. Psychophysiol. 110:171-178.

Although stress has a psychological origin, it affects several physiological processes in the human body. When a person is exposed to a stressor, the autonomic nervous (ANS) system is triggered: the parasympathetic nervous system is suppressed and the sympathetic nervous system is activated. This results in the secretion of the hormones epinephrine and norepinephrine into the blood stream which leads to, for example, vasoconstriction of blood vessels, increased blood pressure, increased muscle tension and a change in heart rate (HR) and heart rate variability (HRV).

References:

  • Influence of Mental Stress on Heart Rate and Heart Rate Variability. Taelman J. et al. 2009. In: Vander Sloten, J., Verdonck, P., Nyssen, M., Haueisen, J. (eds) 4th European Conference of the International Federation for Medical and Biological Engineering. IFMBE Proceedings, vol 22. Springer, Berlin, Heidelberg.
  • Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature. Kim H. 2018. Psychiatry Investig. 2018 Mar;15(3):235-245.

The evaluation of the performance of the mental biomarker algorithms is performed by step-by-step comparative analyses with the current state-of-the-art hardware devices (PPG and HRV sensors) or questionnaires (e.g. PHQ-9, GAD-7). The accuracy and precision are checked by calculating for example the Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE) and Correlation Coefficient (R).

Currently, more than 1 million datapoints (resulting from various collaborations with clinical organisations) are actively used for testing and validating our models.

Privacy and security

Our top priority

At IntelliProve, we take privacy and security seriously. Our policies have been structured to ensure the highest level of confidentiality and integrity when processing your data. IntelliProve undergoes regular penetration testing and security reviews, designed to be GDPR compliant, and encrypts data at rest and in transit.

Secure and Reliable Infrastructure

IntelliProve uses Amazon Web Services (AWS) for the hosting of production environments. AWS data centers are monitored by 24×7 security, biometric scanning, video surveillance and are SOC 1, SOC 2, and SOC 3 certified.

World Class Application Security

Our data privacy and protection policy is based on five pillars: