Institutional Research Information Service
UCL Logo
Please report any queries concerning the funding data grouped in the sections named "Externally Awarded" or "Internally Disbursed" (shown on the profile page) to your Research Finance Administrator. Your can find your Research Finance Administrator at http://www.ucl.ac.uk/finance/research/post_award/post_award_contacts.php by entering your department
Please report any queries concerning the student data shown on the profile page to:

Email: portico-services@ucl.ac.uk

Help Desk: http://www.ucl.ac.uk/ras/portico/helpdesk
Publication Detail
Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging
  • Publication Type:
    Journal article
  • Publication Sub Type:
  • Authors:
    Cho Y, Julier S, Berthouze N
  • Publication date:
  • Journal:
    JMIR Mental Health
  • Volume:
  • Issue:
  • Article number:
  • Status:
  • Keywords:
    Automatic stress recognition, mobile devices, PPG, low-cost thermal imaging, blood volume pulse, variability, nose temperature, physiological computing, affective computing, machine learning
Background: A smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. A smartphone camera-based PhotoPlethysmoGraphy (PPG) and a low-cost thermal camera can be used to create cheap, convenient and mobile monitoring systems. However, to ensure reliable monitoring results, a person has to remain still for several minutes while a measurement is being taken. This is very cumbersome and makes its use in real-life mobile situations quite impractical. Objective: We propose a system which combines PPG and thermography with the aim of improving cardiovascular signal quality and capturing stress responses quickly. Methods: Using a smartphone camera with a low cost thermal camera added on, we built a novel system which continuously and reliably measures two different types of cardiovascular events: i) blood volume pulse and ii) vasoconstriction/dilation-induced temperature changes of the nose tip. 17 healthy participants, involved in a series of stress-inducing mental workload tasks, measured their physiological responses to stressors over a short window of time (20 seconds) immediately after each task. Participants reported their level of perceived mental stress using a 10-cm Visual Analogue Scale (VAS). We used normalized K-means clustering to reduce interpersonal differences in the self-reported ratings. For the instant stress inference task, we built novel low-level feature sets representing variability of cardiovascular patterns. We then used the automatic feature learning capability of artificial Neural Networks (NN) to improve the mapping between the extracted set of features and the self-reported ratings. We compared our proposed method with existing hand-engineered features-based machine learning methods. Results: First, we found that the measured PPG signals presented high quality cardiac cyclic information (relative power Signal Quality Index, pSQI: M=0.755, SD=0.068). We also found that the measured thermal changes of the nose tip presented high quality breathing cyclic information and filtering helped extract vasoconstriction/dilation-induced patterns with fewer respiratory effects (respiratory pSQI: from M=0.714 to M=0.157). Second, we found low correlations between the self-reported stress scores and the existing metrics of the two cardiovascular signals (i.e. heart rate variability and thermal directionality metrics) from short measurements, suggesting they were not very dependent upon one another. Third, we tested the performance of the instant perceived stress inference method. The proposed method achieved significantly higher accuracies than existing pre-crafted features based-methods. In addition, the 17-fold Leave-One-Subject-Out (LOSO) cross-validation results showed that combination of both modalities produced higher accuracy in comparison with the use of PPG or thermal imaging only (PPG+Thermal: 78.33%; PPG: 68.53%; Thermal: 58.82%). The multimodal results are comparable to the state-of-the-art automatic stress recognition methods that require long term measurements (usually, at least a period of 2 minutes is required for an accuracy of around 80% from LOSO). Lastly, we explored effects of different widely-used data labeling strategies on the sensitivity of our inference methods. Our results showed the need for separation of and normalization between individual data. Conclusions: Results demonstrate the feasibility of using smartphone-based imaging for instant mental stress recognition. Given that this approach does not need long-term measurements requiring attention and reduced mobility, we believe it is more suitable for mobile mental healthcare solutions in the wild.
Publication data is maintained in RPS. Visit https://rps.ucl.ac.uk
 More search options
UCL Researchers
UCL Interaction Centre
Dept of Computer Science
Dept of Computer Science
University College London - Gower Street - London - WC1E 6BT Tel:+44 (0)20 7679 2000

© UCL 1999–2011

Search by