Google Research just published a Nature paper describing a system that turns your phone's front-facing camera into a passive heart monitor. Called PHRM (Passive Heart Rate Monitoring), it captures short clips of your face after routine face unlock events, runs a small neural network on-device, and aggregates the results into a daily resting heart rate estimate that matches the accuracy of a Fitbit, no wearable required.

The release includes a GitHub repository with a pre-trained PHRM-mini model and what Google describes as the largest, most diverse smartphone-video rPPG dataset publicly available for research. Access is restricted to qualified researchers with IRB approval and non-commercial intent.

The unlock-and-measure loop

PHRM leverages the front-facing camera to capture video of the user's face in the seconds after face unlock events. The signal it extracts is photoplethysmography, the same technique pulse oximeters use, which detects tiny color fluctuations in skin caused by each pulse of blood. The difference is that the camera does it remotely, with no physical contact.

The on-device pipeline processes 8-second facial video clips and uses computationally-efficient temporal shift convolutional neural networks to predict HR along with a confidence score. Temporal shift networks are a trick borrowed from efficient video understanding: instead of running expensive 3D convolutions across time, they shift a portion of feature channels along the time axis inside an otherwise standard 2D CNN, which gives you temporal reasoning at roughly the cost of a single-frame model.