2. after surgery, patients are transferred on this area for closed monitoring.

Introduction

Clinical complications following surgery remain a major concern worldwide1, therefore admission to the intensive care unit [ICU] following major surgery is standard practice in many healthcare systems. Patients discharged from an ICU to the ward following a post-operative ICU admission are at high risk of adverse events2.

In-hospital adverse events may be prevented with earlier detection of physiological deterioration followed by prompt intervention3,4,5. Early detection of deterioration in post-ICU ward patients has been shown to reduce the rate of ICU readmission6, which is associated with worse outcomes7. The incidence of adverse events, such as cardiac arrest or unexpected admission to the ICU, could also be reduced3,8, which in turn is associated with improved outcomes9,10.

Patient care within hospital settings relies on a system of escalation based on a weighted assessment of patients’ vital signs. The monitoring of vital signs, including heart rate [HR], respiratory rate [RR], peripheral blood oxygen saturation [SpO2], blood pressure and core temperature provides valuable insights into the overall condition of the patient11,12. Patterns of critical illness are often preceded by periods of physiological abnormality13,14, detected in the recorded vital signs well before patients show physical signs of distress15,16. Effective patient monitoring strategies, therefore, aim to detect abnormal patterns in the vital signs as often and as soon as they occur so that acute care for deteriorating patients can be escalated2,17.

Clinical staff evaluates vital signs with a measurement frequency ranging from hourly to 12 h, depending on the acuity level of the ward, and individual patient condition18,19. High-acuity patients such as those in the ICU are monitored continuously using automated bedside monitors so that acute changes can be detected and acted upon in a prompt manner. Heart rate and cardiac function are monitored through electrocardiography [ECG]. A pulse oximeter attached to a well-perfused part of the body, such as a finger or an earlobe, is often used to provide non-invasive estimates of SpO2 via two-wavelength photoplethysmography [PPG]. Respiratory rate is measured using Impedance Pneumography [IP]. IP is a convenient method if patients are already monitored by ECG as it relies on the existing set of electrodes. High-acuity patients may also require invasive monitoring, including arterial cannulation to continuously monitor the blood pressure.

Post-operative ward patients are typically less acute. This is reflected in the level of monitoring they receive, with most patients’ vital signs measured manually every 1–4 h for the immediate post-operative period. Ward level measurements of vital signs are usually performed using a portable monitor, such as DINAMAP®, which incorporates a pulse oximeter and a blood pressure cuff20. Respiratory rate is counted manually. The frequency of manual observations can be increased to multiple measurements per hour for post-operative patients who require intensive monitoring. Continuous monitoring equipment may also be used. Conventional contact monitors are, however, poorly tolerated by patients21. They limit patient mobility, rehabilitation and are uncomfortable to wear over prolonged periods of time22. One previous study that aimed to monitor patients in acute medical and surgical wards for 72 h found that a significant proportion of patients declined to be monitored for the entirety of this period21. Numerous monitoring wires are cumbersome and make patient transfers more difficult23. Nursing staff have reported many staff-hours spent troubleshooting current monitoring systems, with common causes of monitoring failures including sensor disconnection, poor vital-sign readings, and equipment malfunctio24. Poor probe contact has been identified as the leading cause of false respiratory alarms in patients monitors25.

For continuous vital-sign monitoring to be feasible in hospitalised patients it must maintain measurement accuracy while being widely acceptable to patients and staff. A way of achieving this is by using less intrusive monitors instead of the conventional contact-based sensors26. There is growing support that video camera-based vital-sign monitoring may fulfil this role27,28,29,30, a view reinforced by recent findings that such monitoring is feasible in several hospital environments31,32,33,34,35.

Photoplethysmographic imaging [PPGi] is based on the principle that a cardiorespiratory waveform may be extracted from the ambient light reflected from an area of exposed skin using conventional camera devices36,37. Numerous studies in controlled laboratory environments have demonstrated that heart rate and respiratory rate can be derived from the PPGi waveform with a high degree of accuracy38,39. Work published in recent years has shown that changes in SpO2 may also be derived40,41,42. Methods continue to be developed which improve the quality of the PPGi waveform extracted from video feeds43,44. Recent efforts to improve the robustness of non-contact monitoring to motion artefacts have also been reported45,46,47.

While there are clear advantages to conducting experiments in controlled environments, this practice cannot emulate the challenging conditions of an ICU. There are substantial challenges that PPGi must overcome in this setting, including unforeseeable signal artefacts due to patient movement, clinical interventions or changes in the illumination from ambient light sources31. PPGi-based monitoring of heart rate and respiratory rate has so far been demonstrated in non-critical in-hospital areas such as the outpatient clinic31,32, the neonatal high dependency unit33, and the intra-operative environment34,35. While one pilot study collected early data in a post-operative ICU following elective cardiac surgery for 28.6 ± 2.8 min [per patient]48, the feasibility of non-contact monitoring for extended periods of time remains untested.

We designed a clinical study to assess the feasibility of using a video camera-based system to estimate heart rate and respiratory rate over clinically relevant time periods from post-surgical patients with a planned post-operative admission to the ICU in the Churchill Hospital in Oxford, UK. We aimed to establish the proportion of time that camera signals of sufficient quality for reliable vital-sign estimation could be obtained in this patient group, as well as the accuracy of the vital-sign estimates obtained from the video camera system in comparison to wired monitoring collected concurrently as per usual patient clinical care in the ICU.

Results

Prior to their operation, 25 patients agreed to take part over the course of one year. Recording data was available for a cohort of 15 patients as 10 patients were not admitted to the ICU for one of the following reasons: patient admitted to an alternative Recovery Unit [4 patients], surgery cancelled or rescheduled [3 patients], or unavailable research staff or equipment [3 patients].

A summary of demographic data for the study cohort is included in Table 1. The average age of monitored patients was 62.2 years. Nine patients were male [60%] and six patients were female [40%]. Ten patients were admitted to the ICU following maxillofacial surgery [60%], four [26.6%] following gastrointestinal [GI] tract surgery and one patient was admitted following gynae-oncological surgery. The average APACHE II score was 15.7 ± 4.5 at the time of admission to the ICU. The majority [n = 9] commenced study recording with a Richmond Agitation-Sedation Scale RASS  20 breaths/min] is an established criterion for sepsis-evoked Systemic Inflammatory Response Syndrome [SIRS] in hospitalised patients53.

The non-contact camera system produced valid vital-sign estimates between routine manual observations in a busy ICU environment [see e.g. Fig. 3b]. Once patients are discharged to a lower-acuity ward, where nurse observations are separated by several minutes or hours, the ability to track vital-sign changes between consecutive manual observations would be important in the context of early warning scores [EWSs]. Considering that an elevated EWS score is an institutionally-defined criterion for escalating care, it is important that all times of increased EWS are noted54. The ability to provide continuous heart rate and respiratory rate estimates in addition to manual observations of the other vital signs may, therefore, be a useful addition to routine monitoring in the ward.

Although vital signs can be indicative of impending clinical deterioration, routine night-time monitoring adds to the already fragmented sleep of inpatients. Sleep disruption is prevalent among ICU patients55 and is associated with several negative outcomes, including elevated blood pressure and deliriu56,57,58. Therefore, it is important to consider strategies to reduce the burden of overnight monitoring. To our knowledge, ours is the first study to evaluate a non-contact video camera system in the night-time ICU environment. Primary outcomes were evaluated for the day-time period [102.0 h] and the night-time [131.5 h] period. Using an identical protocol for the recording and analysis of day-time and night-time data, comparable performance among the two periods has been demonstrated [Table 3]. Lower error rates and higher correlation coefficient were observed for estimates of both vital signs during the night-time period. While further investigation is warranted to determine whether this is attributable to environmental factors, it is clear that the low-light setting did not prevent night-time video monitoring.

Video camera systems may provide diagnostic information in addition to vital-sign estimates. We revisited the case study of a patient who suffered a pneumothorax and ruptured diaphragm [Fig. 5], where we observed that alterations in the respiratory rhythm could be detected in the camera signal prior to diagnosis. Additionally, the processing of the video camera data revealed asymmetries between regional breathing movements over the chest and thorax. These findings highlight the potential role of video camera monitors not only as a viable method of monitoring the respiratory rate but also of providing a functional assessment through a contextual understanding of morphology. This is made possible through the access to visual cues not available to conventional monitors, or even wearable wireless monitoring systems. Further research will aim to show that the features in the video signal can similarly capture key haemodynamic events in this patient group, such as the onset of clinically significant hypovolaemia or the occurrence of mottled skin [see ISRCTN study reference59].

Some study limitations need to be addressed. Data were collected at a single ICU in a teaching hospital located in Oxford, UK. Thus, our results may neither be generalisable to other ICUs in the UK, nor to other healthcare systems, where clinical protocols for ICU admission, surveillance, and discharge of surgical patients may differ. A demonstration of feasibility on ICU does not necessarily translate to lower-acuity units, where patients are usually not tethered to a patient monitor. A wide-angle lens system is needed for day-time monitoring as patients can ambulate freely, as has been achieved in mental health settings by a system developed in parallel with the ICU system60. The mean age of the patient cohort was 62.2 years [Table 1], which may limit our findings to older populations. Similarly, the study sample did not include participants with skin type ⩾ IV on the Fitzpatrick scale. The performance of heart rate estimation algorithms is known to be negatively affected by darker skin tones and thus further validation is required on a more diverse cohort. The study was not adequately powered to assess the impact of patient position on the accuracy and coverage of video camera-based monitoring. Finally, the use of the privacy blind to obscure the view of the patient from the camera did overlap with some noteworthy events. Several periods of tachycardia were associated with moments where the privacy blind was drawn, precluding non-contact monitoring throughout the associated interventions. The accuracy of camera algorithms over a range of heart rate values wider than that observed in the study cohort has been shown in previous studies in the adult33 and the neonatal32 population. All data analysis was carried out offline and did not affect care in real-time.

These limitations notwithstanding, our results demonstrate the potential of non-contact continuous vital-sign monitoring over clinically relevant time periods to allow real-time detection of physiological deterioration.

Methods

This was an observational study designed to evaluate the feasibility of introducing a non-contact monitoring system into the adult ICU of the Churchill Hospital, in Oxford, United Kingdom. The study was part of a research initiative of the Oxford University Hospitals National Health Service [NHS] Foundation Trust and the Oxford Biomedical Research Centre. The research was compliant with the relevant government regulations. Ethical approval was granted by the Wales Research Ethics Committee 5 [Bangor] under reference number 16/WA/0024. The study was registered in the NIHR Clinical Research Network [CRN] Portfolio under CPMS ID 30402 [IRAS ID 182738].

Study design and protocol

Elective post-surgical patients with a planned post-operative admission to the Oxford Churchill Hospital ICU were considered for inclusion. We aimed to recruit 25 patients. Eligible patients were recruited from the pre-operative assessment clinics servicing the following specialities: maxillofacial surgery, gastrointestinal surgery, hepatobiliary surgery, renal transplant, pancreatic transplant, urology, and gynaecology. Candidate patients were screened by members of the clinical team against inclusion/exclusion criteria, and eligible patients were approached for informed consent. The exclusion criteria in Table 4 were considered.

Table 4 Participant exclusion criteria.

Full size table

Recording was initiated as soon as practically possible after admission to the ICU following surgery. Each patient was monitored continuously for up to 48 h, or until discharge from the ICU. For the duration of the study, concurrent video and physiological data collected by the patient monitor system used as part of routine care were recorded. Equipment was chosen such that it would not affect the experience of the patient or clinical staff. Vital sign estimates from the non-contact monitoring software were computed retrospectively and were not available to clinical staff during the study.

A privacy blind [an opaque sheet of plastic attached to a hinged fitting of the video camera] was included. This blind could be used to obscure the view of the patient from the video camera and thus prevent the recording during dignity-compromising situations [such as routine hygiene]. Patients could ask for the blind to be drawn at any time. Patients, their visitors, or staff members could also ask for the equipment to be turned off for any reason throughout the study.

Select clinical information was documented by clinical research staff on paper-based case report forms: patient demographics, admission category, recording start and end times, APACHE II score61, Richmond Agitation-Sedation Scale [RASS], Fitzpatrick score, and free-text clinical notes. The Fitzpatrick Scale is a numerical skin classification system, from Type I [pale white, always burns, never tans] to Type VI [darkest brown, never burns, never tans]49.

Instrumentation

The monitoring equipment is shown in Fig. 6a. As per the standard of care, vital-sign monitoring was provided using a IntelliVue® MP70 patient monitor [Philips, Eindhoven, Netherlands]. Using proprietary software, reference values of HR and RR were derived at 1-s increments. HR was derived from the single-lead ECG [sampled at 250 Hz], while RR was derived from the bipolar IP signal collected [at 62.5 Hz] using the same set of electrodes. Pulse rate was obtained from the photoplethysmogram [PPG] signal acquired by a pulse oximetry Philips ® M1941A finger probe [Philips, Eindhoven, Netherlands] and sampled at 125 Hz.

Fig. 6: Monitoring apparatus in the Churchill Hospital ICU in Oxford, United Kingdom.

a Patient monitor at the bedside and the non-contact vital-sign monitoring equipment at the end a patient’s bed. b Set-up of the trolley containing the non-contact vital-sign monitoring equipment. The components are shown in their respective positions—[A] optical camera; [B] infrared LED source; [C] thermal camera; [D] interface equipment; [E] workstation [in cabinet]; [F] privacy blind [folded up].

Full size image

Patients were monitored using a 3.2 megapixel Point Grey Research Grasshopper3® GS3-U3-32S4M-C monochrome camera [FLIR Instruments, Oregon, USA]. A non-polarised lens with a 12.5 mm fixed focal length and an aperture range of f/1.4-16 was attached [Goyo Optical® GMHR412514MCN, Saitama, Japan], as well as an infrared-pass filter [Schneider Kreuznach® 098, Germany]. The imaging device was set to acquire raw uncompressed 8-bit image data at a resolution of 1024 × 768 pixels and a nominal sampling rate of 100 Hz. Illumination was provided using a Metaphase® FL201-IRN-24 850 nm infrared illuminator [Metaphase, Bristol, Pennsilvania, USA]. Since 850 nm is outside the visible range, the illuminator did not emit any noticeable glow light. The infrared illuminator powered by a Metaphase® ULC-2 LED controller [Metaphase, Bristol, Pennsylvania, USA] connected to a Gardasoft® CC320 sequence controller [Gardasoft, Cambridge, UK] was strobed at 50 Hz to illuminate alternate video frames.

The equipment was designed to be non-disruptive to standard care. A medical-grade hospital trolley was used as the basis of the non-contact monitoring system. Patients in the ICU are accommodated on any available bed, and so equipment needed to be mobile rather than installed at one particular bed space. The video camera devices and the infrared illuminator were mounted on a hospital trolley positioned at the end of the patient’s bed, as shown in Fig. 6a. The trolley was easily movable so it would not impede access to the patient. As is usual practice, patients had access to a chair beside the bed, on which they would occasionally sit. Data were captured for both locations.

All recorded video and physiological data were relayed to an 8 TB Hewlett-Packard [HP] workstation. The workstation was replaced every 12 h with an identical one, due to the large storage requirements [approximately 6.7 GB per minute for the digital video data]. All video data were encrypted using industry standard methods.

Data analysis

The video data were reviewed by research nurses and any privacy-compromising sections of data were deleted. Furthermore, video data containing individuals who had not been consented [e.g. family members, nurses, etc.] were also removed by research nurses using a bounding box around the patient. In instances where the patient and the non-consenting individual could not be separated in the video frame [e.g. because they obscured part of the view of the patient], the entire frame was deleted.

Prior to vital-sign estimation, the collected video data were analysed so that time periods for which the privacy blind was drawn, or nurse-reviewed frames were deleted, were discarded from the valid time available for vital-sign estimation. The methods used to obtain non-contact estimates of vital signs during valid time were subsequently applied.

Cardiac-synchronous and respiratory waveforms were extracted from the video using image processing algorithms that analyse several regions of interest across the available frame areas, such as the patient’s face or chest. Our pipeline for video processing has been reviewed in previous studies31,32,33 and so it is described here in brief.

To assess the feasibility of non-contact monitoring, the video camera-derived estimates of both vital signals were assessed against the gold-standard values provided by a clinical-grade patient monitor from analysis of the physiological signals acquired at the bedside. We assessed performance in terms of error measures, Pearson correlation, and estimated time, expressed as a percentage of total and valid recording times.

Reference physiological values

The measurement of the physiological signals of hospital patients with contact devices is subject to periods of artefactual noise due to factors such as patient movement, or poor probe placement. For this reason, some processing of the heart rate and respiratory rate values obtained from the patient monitor is needed prior to their use as references. A gold-standard reference of heart rate was derived based on the redundancy of vital-sign measurements provided by the multiple sources available at the bedside. The results of this procedure are provided as a supplemental material to this submission.

The performance of the proposed vital-sign estimation system was assessed over those periods for which this gold-standard was available, referred to as valid time.

Heart rate estimation

A signal processing pipeline for the estimation of heart rate validated in clinical studies31,32,33 was adopted. A PPGi signal was extracted from a grid of ROIs defined over the available frame area. We then used tested algorithms to assess the quality of the cardiac pulses found in each PPGi signal. For each time window, a candidate HR value was estimated from the PPGi signal extracted from each sub-region using both time-domain and frequency-domain algorithms. A definitive heart rate estimate for that time window was computed by combining the individual estimates from each of the previous methods using a data fusion technique. Finally, the time window was shifted by a number of PPGi samples corresponding to a time shift of 1 s, and these steps were repeated for the new window. Therefore, heart rate estimates were reported every second. A window length of 15 s was used.

Extraction of PPGi signal

A region of interest ROIS over the available frame area was obtained for every image frame. A total of N = 81 sub-regions of ROIS organised in a 9 × 9 grid over this ROI were defined. In the few instances where the bounding box selected by the research nurses was limited to small pixel area, a 3 × 3 grid was used instead. The average pixel intensity over each sub-region was subsequently computed for every image frame. Several step changes can occur during video recording. A Bayesian change point detection algorithm was applied to 30-s running windows with a 5-s overlap62. The resulting time series {ROIS,1, . . . , ROIS,N} is known to contain signal components correlated with the cardiac frequency as well as artefactual components due to noise sources, such as motion and changes in ambient illumination. To attenuate spectral components outside the range of physiological heart values, from 36 to 210 beats/min63, the PPGi signals were linearly detrended and a 50th order finite-impulse response [FIR] band-pass filter with cut-off frequencies of 0.6 Hz and 3.5 Hz was applied.

The filtered \[{\{{\rm{RO}}{{\rm{I}}}_{S,i}\}}_{i = 1,...,N}\] waveforms were then subdivided into 15-s running windows with a time shift between consecutive windows of 1 s. For a given time window, a set N filtered waveforms, i.e. one per each of the N sub-regions, were available.

Estimation of heart rate from PPGi

The PPGi signals extracted for each window were used as inputs to a heart rate estimation process reported in earlier studies32,33. In summary, a peak and onset detection algorithm based on Zong et al.64 and extended by Villarroel et al.32 was applied to identify salient points in the PPGi signal. Accurate peak and onset detection allowed beat-by-beat assessment of the pulsatile signal quality using a signal quality index [SQI] by comparing each beat with a running template using several metrics that measure: amplitude differences, the occurrence of step changes, beat pulses outside the expected physiological range and a similarity metric based on a multi-scale Dynamic Time Warping [DTW] algorithm65.

Heart rate was computed using an autoregressive spectral analysis31. Once candidate heart rate estimates had been computed for every 15-s window [one from each sub-region], the heart rate estimate for a given window was computed as the median of all the heart rate estimates from each colour channel from the ROIS that had an associated SQI greater than 0.8. As in Villarroel et al., a Kalman filter was applied to fuse these estimates over time based on their signal quality65, thus reducing the effects of transient changes in pixel intensity. Fused heart rate estimates were reported every second.

Respiratory rate estimation

During normal respiration, the contraction and relaxation of the abdominal and intercostal muscles causes the volume of the chest cavity to increase or decrease. The volumetric changes with airflow cause movement of the body, principally from regions around the thorax and diaphragm, which can be recorded by a video camera from areas of exposed skin but also from those covered by tight-fitting or highly patterned clothing.

This phenomenon has previously been exploited for the non-contact continuous monitoring of respiration. Several methods have been proposed to extract respiratory signal from the breathing-related movements of the thorax66,67,68, thoracoabdominal area66,69,70, or face area71. Previous studies have shown that respiratory signals can be detected as modulations to the PPGi signal extracted from the subject’s skin areas32,39,72.

In this work, respiratory signals were acquired by measuring the respiratory-induced intensity changes in multiple ROIs across the available frame area. Respiratory rate was then computed by using data fusion algorithms from the analysis of the signals extracted from the ROIs. This methodology is described below. The pipeline adopted for both the extraction of the respiratory signal and the estimation of respiratory rate have been validated in previous clinical trials32,33.

Extraction of respiratory signal

A second region of interest [ROIT] over the available frame area was obtained for every image frame. A total of M = 9 sub-regions organised in a 9 × 9 grid over ROIT were defined.

For every frame, {ROIT,1, . . . , ROIT,M} signals were extracted by averaging the pixel intensity signal over each sub-region. The M signals extracted are analogous to the {ROIS,1, . . . , ROIS,N} time series extracted in the previous sections for heart rate estimation. The extracted signals contain both cardiac and respiratory information. It has been shown that the relative weight of the respiratory-to-cardiac related pulsatile components in the extracted signal depends to a great extent on the size of the region of interest31. In addition, the signals extracted are often contaminated by baseline drifts and high-frequency noise70. To remove these artefacts, each M signal was detrended and filtered using a cascade of a 8th order high-pass IIR filter with a cut-off frequency at 0.1 Hz and an 2nd order low-pass IIR filter with a cut-off frequency at 0.8 Hz. These filters encompass respiratory rates values in the range of 6–48 breaths/min. Once the {ROIT,1, . . . , ROIT,M} were processed, the resulting waveform was subdivided into 30-s running windows with a step of 5 s between consecutive windows. Similar to the methodology adopted for the extraction of PPGi signals, the {Respii} respiratory waveforms obtained were used as inputs to a respiration rate estimation block.

Estimation of respiratory rate

Our general approach to deriving respiratory rate for each time window is as follows. The peaks and onsets were detected using an adaptive algorithm previously described33. For every window, a candidate respiratory rate value was computed for each of the Respi using the autorregressive methods described for heart rate estimation.

Respiration has been shown to be more prominent in quiet research participants with minimal body motion. Thus, for each time window and for each of the respiratory signal extracted, signal quality indices were computed as detailed in the previous work33. Once both the candidate respiratory rate and signal quality index had been estimated for all respiratory signals, a data fusion technique based on multiple Kalman filters [as for heart rate estimation] was applied to combine the multiple respiratory estimates from the same time window and produce a final respiratory rate for each time window. Fused respiratory rate estimates were reported every 5 s.

Localised respiration mapping

Spatial maps of respiratory signal amplitude were reported for a patient with abnormal respiratory physiology using methods developed in33. Each super-pixel in the maps was coloured according to the amplitude of the respiratory signal computed from the 30 × 30 ROI centred on the super-pixel. The resulting respiratory maps were smoothed using a 2D Gaussian filter with a filter size of 3 × 3 [super-pixels] and a kernel width of 0.5 such that the filter only operated on neighbouring super-pixels.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

To protect the privacy of the patients involved in the study, the datasets analysed in this study are not available under the granted research ethics application.

Code availability

The codes used in the study are proprietary, and available only with the permission of the licensors.

References

1. Ghaffar, S., Pearse, R. M. & Gillies, M. A. ICU admission after surgery: who benefits? Curr. Opin. Crit. Care 23, 424–429 [2017].

   Article  PubMed  Google Scholar 

2. Beaumont, K., Luettel, D. & Thomson, R. Deterioration in hospital patients: early signs and appropriate actions. Nurs. Stand. 23, 43 [2008].

   Article  PubMed  Google Scholar 

3. Adam, S. & Odell, M. An acute problem? a report of the national confidential enquiry into patient outcome and death. Nursing Crit. Care 10, 225–227 [2005].

   Article  Google Scholar 

4. Berwick, D. M., Calkins, D. R., McCannon, C. J. & Hackbarth, A. D. The 100 000 lives campaign: setting a goal and a deadline for improving health care quality. J. Am. Med. Assoc. 295, 324–327 [2006].

   Article  CAS  Google Scholar 

5. Hillman, K. et al. Antecedents to hospital deaths. Inter. Med. J. 31, 343–348 [2001].

   Article  CAS  Google Scholar 

6. McQuillan, P. et al. Confidential inquiry into quality of care before admission to intensive care. BMJ 316, 1853–1858 [1998].

   Article  CAS  PubMed  PubMed Central  Google Scholar 

7. Kause, J. et al. A comparison of antecedents to cardiac arrests, deaths and emergency intensive care admissions in Australia and New Zealand, and the United Kingdom-the ACADEMIA study. Resuscitation 62, 275–282 [2004].

   Article  PubMed  Google Scholar 

8. Goldhill, D. Preventing surgical deaths: Critical care and intensive care outreach services in the postoperative period. Br. J. Anaesth. 95, 88–94 [2004].

   Article  PubMed  Google Scholar 

9. Buist, M., Bernard, S., Nguyen, T. V., Moore, G. & Anderson, J. Association between clinically abnormal observations and subsequent in-hospital mortality: a prospective study. Resuscitation 62, 137–141 [2004].

   Article  PubMed  Google Scholar 

10. Jacques, T., Harrison, G. A., McLaws, M.-L. & Kilborn, G. Signs of critical conditions and emergency responses [SOCCER]: a model for predicting adverse events in the inpatient setting. Resuscitation 69, 175–183 [2006].

    Article  PubMed  Google Scholar 

11. Ahrens, T. The most important vital signs are not being measured. Australian Crit. Care 21, 3–5 [2008].

    Article  Google Scholar 

12. Elliott, M. & Coventry, A. Critical care: the eight vital signs of patient monitoring. Br. J. Nurs. 21, 621–625 [2012].

    Article  PubMed  Google Scholar 

13. Schein, R. M., Hazday, N., Pena, M., Ruben, B. H. & Sprung, C. L. Clinical antecedents to in-hospital cardiopulmonary arrest. Chest 98, 1388–1392 [1990].

    Article  CAS  PubMed  Google Scholar 

14. Buist, M. D., Burton, P. R., Bernard, S. A., Waxman, B. P. & Anderson, J. Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care: a pilot study in a tertiary-care hospital. Med. J. Australia 171, 22–25 [1999].

    Article  CAS  PubMed  Google Scholar 

15. Franklin, C. & Mathew, J. Developing strategies to prevent inhospital cardiac arrest: analyzing responses of physicians and nurses in the hours before the event. Crit. Care Med. 22, 244–247 [1994].

    Article  CAS  PubMed  Google Scholar 

16. Hillman, K. M. et al. Duration of life-threatening antecedents prior to intensive care admission. Intensive Care Med. 28, 1629–1634 [2002].

    Article  PubMed  Google Scholar 

17. Klepstad, P. K., Nordseth, T., Sikora, N. & Klepstad, P. Use of National early Warning score for observation for increased risk for clinical deterioration during post-ICU care at a surgical ward. Therapeutics Clin. Risk Manag. 15, 315 [2019].

    Article  Google Scholar 

18. Evans, D., Hodgkinson, B. & Berry, J. Vital signs in hospital patients: a systematic review. Int. J. Nurs. Stud. 38, 643–650 [2001].

    Article  CAS  PubMed  Google Scholar 

19. Mitchell, I. & Van Leuvan, C. et al. Missed opportunities? An observational study of vital sign measurements. Crit. Care Resuscitation 10, 111 [2008].

    Google Scholar 

20. Ward, M. & Langton, J. A. Blood pressure measurement. Continuing Educ. Anaesth. Crit. Care Pain 7, 122–126 [2007].

    Article  Google Scholar 

21. Watkinson, P. et al. A randomised controlled trial of the effect of continuous electronic physiological monitoring on the adverse event rate in high risk medical and surgical patients. Anaesthesia 61, 1031–1039 [2006].

    Article  CAS  PubMed  Google Scholar 

22. Hodgson, C. L., Capell, E. & Tipping, C. J. Early mobilization of patients in intensive care: organization, communication and safety factors that influence translation into clinical practice. Crit. Care 22, 1–7 [2018].

    Article  Google Scholar 

23. Waydhas, C. Equipment review: Intrahospital transport of critically ill patients. Crit. Care 3, R83 [1999].

    Article  CAS  PubMed  PubMed Central  Google Scholar 

24. Ede, J. et al. Non-contact vital sign monitoring of patients in an intensive care unit: A human factors analysis of staff expectations. Appl. Ergonom. 90, 103149 [2021].

    Article  Google Scholar 

25. Jorge, J. et al. Assessment of signal processing methods for measuring the respiratory rate in the neonatal intensive care unit. IEEE J. Biomed. Health Inform. 23, 2335–2346 [2019].

    Article  PubMed  Google Scholar 

26. Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25, 44–56 [2019].

    Article  CAS  PubMed  Google Scholar 

27. Zhao, F., Li, M. & Tsien, J. Z. Technology platforms for remote monitoring of vital signs in the new era of telemedicine. Expert Rev. Med. Devices 12, 411–429 [2015].

    Article  CAS  PubMed  Google Scholar 

28. Kevat, A. C., Bullen, D. V., Davis, P. G. & Kamlin, C. O. F. A systematic review of novel technology for monitoring infant and newborn heart rate. Acta Paediatrica 106, 710–720 [2017].

    Article  PubMed  Google Scholar 

29. Chen, W. Neonatal Monitoring Technologies: Design for iNtegrated Solutions: Design for Integrated Solutions [IGI Global, 2012].

30. Harford, M., Catherall, J., Gerry, S., Young, J. D. & Watkinson, P. J. Availability and performance of image-based, non-contact methods of monitoring heart rate, blood pressure, respiratory rate, and oxygen saturation: a systematic review. Physiol. Meas. 40, 06TR01 [2019].

31. Tarassenko, L. et al. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiol. Meas. 35, 807 [2014].

    Article  CAS  PubMed  Google Scholar 

32. Villarroel, M. et al. Non-contact vital-sign monitoring of patients undergoing haemodialysis treatment. Sci. Rep. 10, 1–21 [2020].

    Article  Google Scholar 

33. Villarroel, M. et al. Non-contact physiological monitoring of preterm infants in the neonatal intensive care unit. npj Digital Med. 2, 1–18 [2019].

    Article  Google Scholar 

34. Rubins, U., Miscuks, A. & Lange, M. Simple and convenient remote photoplethysmography system for monitoring regional anesthesia effectiveness. In: EMBEC & NBC 2017 [378–381] [Springer, 2017].

35. Trumpp, A. et al. Camera-based photoplethysmography in an intraoperative setting. Biomed. Eng. Online 17, 33 [2018].

    Article  PubMed  PubMed Central  Google Scholar 

36. Verkruysse, W., Svaasand, L. O. & Nelson, J. S. Remote plethysmographic imaging using ambient light. Opt. Express 16, 21434–21445 [2008].

    Article  CAS  PubMed  Google Scholar 

37. Wieringa, F. P., Mastik, F. & van der Steen, A. F. Contactless multiple wavelength photoplethysmographic imaging: a first step toward “spo 2 camera” technology. Ann. Biomed. Eng. 33, 1034–1041 [2005].

    Article  CAS  PubMed  Google Scholar 

38. Monkaresi, H., Calvo, R. A. & Yan, H. A machine learning approach to improve contactless heart rate monitoring using a webcam. IEEE J. Biomed. Health Inform. 18, 1153–1160 [2013].

    Article  Google Scholar 

39. Poh, M.-Z., McDuff, D. J. & Picard, R. W. Advancements in noncontact, multiparameter physiological measurements using a webcam. Trans. Biomed. Eng. 58, 7–11 [2010].

    Article  Google Scholar 

40. Guazzi, A. R. et al. Non-contact measurement of oxygen saturation with an rgb camera. Biomed. Opt. Express 6, 3320–3338 [2015].

    Article  CAS  PubMed  PubMed Central  Google Scholar 

41. Shao, D. et al. Noncontact monitoring of blood oxygen saturation using camera and dual-wavelength imaging system. IEEE Trans. Biomed. Eng. 63, 1091–1098 [2015].

    Article  PubMed  Google Scholar 

42. Verkruysse, W. et al. Calibration of contactless pulse oximetry. Anesth. Analgesia 124, 136 [2017].

    Article  Google Scholar 

43. Poh, M.-Z., McDuff, D. J. & Picard, R. W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18, 10762–10774 [2010].

    Article  CAS  PubMed  Google Scholar 

44. McDuff, D., Gontarek, S. & Picard, R. W. Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera. IEEE Trans. Biomed. Eng. 61, 2948–2954 [2014].

    Article  PubMed  Google Scholar 

45. De Haan, G. & Jeanne, V. Robust pulse rate from chrominance-based rppg. IEEE Trans. Biomed. Eng. 60, 2878–2886 [2013].

    Article  PubMed  Google Scholar 

46. Feng, L., Po, L.-M., Xu, X. & Li, Y. Motion artifacts suppression for remote imaging photoplethysmography. In: 2014 19th International Conference on Digital Signal Processing 18–23 [IEEE, 2014].

47. Takano, C. & Ohta, Y. Heart rate measurement based on a time-lapse image. Medical Engineering & Physics 29, 853–857 [2007].

    Article  Google Scholar 

48. Rasche, S. et al. Camera-based photoplethysmography in critical care patients. Clinical Hemorheology and Microcirculation 64, 77–90 [2016].

    Article  CAS  PubMed  Google Scholar 

49. Fitzpatrick, T. Soleil et peau. J. Med. Esthetique 2, 33–34 [1975].

50. Elgendi, M. Optimal signal quality index for photoplethysmogram signals. Bioengineering 3, 21 [2016].

    Article  PubMed Central  Google Scholar 

51. Association for the Advancement of Medical Instrumentation, AAMI. Cardiac Monitors, Heart Rate Meters, and Alarms [American National Standard [ANSI/AAMI EC13: 2002] 1–87 [AAMI, 2002].

52. Hochhausen, N., Barbosa Pereira, C., Leonhardt, S., Rossaint, R. & Czaplik, M. Estimating respiratory rate in post-anesthesia care unit patients using infrared thermography: an observational study. Sensors 18, 1618 [2018].

    Article  PubMed Central  Google Scholar 

53. Dellinger, R. P. Surviving sepsis campaign guideline committee. surviving sepsis campaign: International guidelines management of severe sepsis and septic shock: 2008. Crit. Care Med. 36, 296–327 [2008].

    Article  PubMed  Google Scholar 

54. Tomlinson, H. R. et al. Smoothing effect in vital sign recordings: Fact or fiction? a retrospective cohort analysis of manual and continuous vital sign measurements to assess data smoothing in postoperative care. Anesth. Analgesia 127, 960 [2018].

    Article  Google Scholar 

55. Darbyshire, J. L. et al. Measuring sleep in the intensive care unit: Electroencephalogram, actigraphy, or questionnaire? J. Intensive Care Soc. 21, 22–27 [2020].

    Article  PubMed  Google Scholar 

56. Figueroa-Ramos, M. I., Arroyo-Novoa, C. M., Lee, K. A., Padilla, G. & Puntillo, K. A. Sleep and delirium in icu patients: a review of mechanisms and manifestations. Intensive Care Med. 35, 781–795 [2009].

    Article  PubMed  Google Scholar 

57. Gangwisch, J. E. et al. Short sleep duration as a risk factor for hypertension: analyses of the first national health and nutrition examination survey. Hypertension 47, 833–839 [2006].

    Article  CAS  PubMed  Google Scholar 

58. Davidson, S. et al. Vital-sign circadian rhythms in patients prior to discharge from an icu: a retrospective observational analysis of routinely recorded physiological data. Crit. Care 24, 1–13 [2020].

    Article  Google Scholar 

59. Biomed Central ISRCTN10417167: Optical monitoring of changes in perfusion map of lower limbs compared to traditional haemodynamic monitoring methods in response to pharmacological challenges in healthy volunteers or Mapping Of Lower Limb skIn pErfusion [MOLLIE]. Registered on: July 17, 2019.

60. Lloyd-Jukes, H., Gibson, O. J., Wrench, T., Odunlade, A. & Tarassenko, L. Vision-based patient monitoring and management in mental health settings. J. Clin. Eng. 46, 36–43 [2021].

    Article  Google Scholar 

61. Knaus, W. A., Draper, E. A., Wagner, D. P. & Zimmerman, J. E. Apache ii: a severity of disease classification system. Crit. Care Med. 13, 818–829 [1985].

    Article  CAS  PubMed  Google Scholar 

62. Adams, R. P. & MacKay, D. J. Bayesian online changepoint detection. Preprint at //arxiv.org/abs/0710.3742 [2007].

63. Khonsary, S. A. Guyton and Hall: Textbook of Medical Physiology, vol. 8 [Scientific Scholar, 2017].

64. Zong, W., Heldt, T., Moody, G. & Mark, R. An open-source algorithm to detect onset of arterial blood pressure pulses. In Computers in Cardiology, 2003 [259–262] [IEEE, 2003].

65. Li, Q., Mark, R. G. & Clifford, G. D. Robust heart rate estimation from multiple asynchronous noisy sources using signal quality indices and a Kalman filter. Physiol. Meas. 29, 15 [2007].

    Article  PubMed  PubMed Central  Google Scholar 

66. Tan, K. S., Saatchi, R., Elphick, H. & Burke, D. Real-time vision based respiration monitoring system. In: International Symposium on Communication Systems, Networks & Digital Signal Processing [770–774] [IEEE, 2010].

67. Janssen, R., Wang, W., Moço, A. & de Haan, G. Video-based respiration monitoring with automatic region of interest detection. Physiol. Meas. 37, 100 [2015].

    Article  PubMed  Google Scholar 

68. Lin, K.-Y., Chen, D.-Y. & Tsai, W.-J. Image-based motion-tolerant remote respiratory rate evaluation. IEEE Sensors J. 16, 3263–3271 [2016].

    Article  Google Scholar 

69. Bai, Y.-W., Li, W.-T. & Chen, Y.-W. Design and implementation of an embedded monitor system for detection of a patient’s breath by double webcams. In: International Workshop on Medical Measurements and Applications [171–176] [IEEE, 2010].

70. Jorge, J. et al. Non-contact monitoring of respiration in the neonatal intensive care unit. In 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition [286–293] [IEEE, 2017].

71. Iozza, L. et al. Monitoring breathing rate by fusing the physiological impact of respiration on video-photoplethysmogram with head movements. Physiol. Meas. 40, 094002 [2019].

    Article  PubMed  Google Scholar 

72. van Gastel, M., Stuijk, S. & de Haan, G. Robust respiration detection from remote photoplethysmography. Biomed. Opt. Express 7, 4941–4957 [2016].

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors acknowledge the support of Innovate UK [File Ref 102110] and Oxehealth Ltd. J.J. acknowledges the RCUK Digital Economy Programme, grant number EP/G036861/1 [Oxford Centre for Doctoral Training in Healthcare Innovation] as well as funding from the Fundação para a Ciência e Tecnologia, Portugal grant SFRH/BD/85158/2012. M.V. was supported by the Oxford Centre of Excellence in Medical Engineering funded by the Wellcome Trust and EPSRC under grant number WT88877/Z/09/Z. H.T. was funded by the Rhodes Trust. M.H. was funded by an NIHR Academic Clinical Fellowship between 2014 and 2018. M.H., J.E., L.T. and P.W. were supported by the NIHR Biomedical Research Centre Programme, Oxford. We would like to thank Soubera Rymell and David Vallance who screened and recruited patients for this study. We are indebted to staff from the intensive care unit and the pre-op assessment clinic for accommodating the study and research team in their work spaces. We would also like to thank our patient advisor, Imelda Cook, for her help and assistance.

Author information

Author notes

1. These authors contributed equally: João Jorge, Mauricio Villarroel.

Authors and Affiliations

1. Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK

   João Jorge, Mauricio Villarroel, Hamish Tomlinson & Lionel Tarassenko

2. NIHR Biomedical Research Centre, Oxford, UK

   João Jorge, Mauricio Villarroel, Jody Ede, Mirae Harford, Lionel Tarassenko & Peter Watkinson

3. Oxehealth Ltd., The Oxford Science Park, Magdalen Centre North, Oxford, UK

   Oliver Gibson

4. Kadoorie Centre for Critical Care Research and Education, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK

   Julie L. Darbyshire, Jody Ede, Mirae Harford, John Duncan Young & Peter Watkinson

5. Oxford University Hospitals NHS Trust, Oxford, UK

   Peter Watkinson

Authors

1. João Jorge

   View author publications

   You can also search for this author in PubMed Google Scholar

2. Mauricio Villarroel

   View author publications

   You can also search for this author in PubMed Google Scholar

3. Hamish Tomlinson

   View author publications

   You can also search for this author in PubMed Google Scholar

4. Oliver Gibson

   View author publications

   You can also search for this author in PubMed Google Scholar

5. Julie L. Darbyshire

   View author publications

   You can also search for this author in PubMed Google Scholar

6. Jody Ede

   View author publications

   You can also search for this author in PubMed Google Scholar

7. Mirae Harford

   View author publications

   You can also search for this author in PubMed Google Scholar

8. John Duncan Young

   View author publications

   You can also search for this author in PubMed Google Scholar

9. Lionel Tarassenko

   View author publications

   You can also search for this author in PubMed Google Scholar

10. Peter Watkinson

    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.V., J.J. and H.T. performed the image/video analysis and signal processing methods and drafted the manuscript. M.V. and O.G. wrote the real-time software for the recording of the data. O.G. contributed to the image/video analysis. J.E. recruited the study patients and collected the dataset. P.W. and L.T. conceived the study. J.D contributed to protocol design and local delivery of the study. M.H., J.D.Y. and P.W. provided clinical guidance. L.T. provided overall guidance. All authors reviewed the manuscript.

Corresponding author

Correspondence to João Jorge.

Ethics declarations

Competing interests

P.W. declares research grants from Wellcome Trust, the National Institute for Health Research, and Sensyne Health. He was Chief Medical Officer for Sensyne Health and holds shares in the company. L.T. is a non-executive Director of Oxehealth and holds a small number of shares in the company. O.G. is VP at Oxehealth. The remaining authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Reporting Summary

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author[s] and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit //creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Cite this article

Jorge, J., Villarroel, M., Tomlinson, H. et al. Non-contact physiological monitoring of post-operative patients in the intensive care unit. npj Digit. Med. 5, 4 [2022]. //doi.org/10.1038/s41746-021-00543-z

What is the best way to monitor a patient after surgery?

Where does a patient go after surgery but before the recovery room the or the ICU the PACU the ER?

How will you monitor a patient in recovery room?

What happens in the recovery room after surgery?