Google Unveils LSM-2 with AIM to Revolutionize Learning from Incomplete Wearable Data

Challenges of Incomplete Wearable Data

Wearable devices capture continuous physiological and behavioral signals such as heart rate, activity, temperature, and skin conductance. However, real-world data from these devices is often incomplete due to sensor failures, device removal, charging, motion artifacts, battery-saving modes, and other interruptions. This missingness is structured and pervasive, posing a significant challenge for self-supervised learning (SSL) models, which generally require complete, regular data streams. Traditional approaches relying on data imputation or discarding incomplete samples risk bias and loss of valuable information.

Introducing LSM-2 and Adaptive and Inherited Masking (AIM)

Google DeepMind researchers developed the LSM-2 framework with a novel Adaptive and Inherited Masking (AIM) strategy to address these issues. AIM integrates two masking types:

Inherited Mask: Identifies tokens corresponding to actual missing data in sensors.
Artificial Mask: Randomly masks observed tokens to create reconstruction targets during self-supervised pretraining.

These masks are combined and processed through a transformer-based encoder-decoder architecture, enabling the model to learn directly from incomplete, non-imputed data and dynamically adapt to real-world missingness during inference. This results in robust representations that handle both partial and systematic data gaps effectively.

Pretraining Strategies and Dataset

The pretraining involved various masking strategies simulating real-world missingness:

Randomly dropping 80% of tokens to mimic sensor noise.
Dropping 50% of temporal windows to simulate periods where all sensors are missing.
Dropping 50% of sensor channels for selective sensor off periods.

LSM-2 was trained on a massive dataset comprising 40 million hours of multimodal sensor data collected from 60,440 participants aged 18 to 96, covering diverse demographics. Sensors included photoplethysmography (PPG), accelerometer, electrodermal activity (EDA), skin temperature, and altimeter.

Evaluation and Performance

LSM-2 with AIM was evaluated on several downstream tasks:

Classification: hypertension, anxiety prediction, and 20-class activity recognition.
Regression: age and BMI estimation.
Generative: recovery of missing sensor data.

Compared to its predecessor LSM-1, LSM-2 showed notable improvements:

| Task | Metric | LSM-1 | LSM-2 w/ AIM | Improvement | |--------------------|-------------|-------|--------------|-------------| | Hypertension | F1 | 0.640 | 0.651 | +1.7% | | Activity Recognition| F1 | 0.470 | 0.474 | +0.8% | | BMI (regression) | Correlation | 0.667 | 0.673 | +1.0% | | Random Imputation | MSE (↓) | 0.30 | 0.20 | 33% lower | | 2-signal Recovery | MSE (↓) | 0.73 | 0.17 | 77% lower |

LSM-2 demonstrated 73% smaller performance drops under targeted missingness scenarios compared to LSM-1, highlighting its robustness. For instance, removing accelerometry data reduced activity recognition F1 by 57% in LSM-2 versus 71% in LSM-1.

Key Technical Insights

Direct Learning from Incomplete Data: LSM-2 is the first wearable foundation model trained and evaluated directly on incomplete data without imputation.
Hybrid Masking: AIM combines dropout masking for computational efficiency and attention masking for flexibility.
Generalizable Embeddings: Even with a frozen backbone and simple probes, LSM-2 achieves state-of-the-art results across clinical and event-level tasks.
Dual Generative and Discriminative Capability: LSM-2 can reconstruct missing signals and generate embeddings for varied downstream applications, making it valuable for medical and behavioral monitoring.