Skip to content
Back to projects
ML / SensorsCase Study

Subject-independent HAR on WISDM

v1.0.0 is the June phone XGBoost notebook (0.8559 accuracy, leaky split). v2.0.0 is session-safe 20 Hz repair, GroupKFold, and a CPU API. Same 5 s phone flatten: 0.8925 leaky macro-F1 vs 0.2924 GroupKFold. Watch statistical XGBoost is 0.7031.

PythonWISDMXGBoostscikit-learnFastAPIONNXMLflow
GitHubHugging Facev2.0.0v1.0.0 notebooks

0.7031

Watch macro-F1, Protocol B

0.89 → 0.29

Phone flatten, leaky vs GroupKFold

18

Activity classes (A–S, no N)

51

Subjects (UCI 507)

Overview

Public WISDM (UCI 507) smartphone and smartwatch IMU data, 18 activities. v1 shuffled overlapping phone windows and reported 0.8559 accuracy. v2 is a package that refuses that split for any number I would cite. Primary metric is macro-F1. Accuracy is secondary. Every cell names a protocol. The rebuild story is in the blog post. This page is the frozen table, the protocol contract, and the serving surface.

Dataset

51 subjects (1600–1650), phone in a pocket and watch on the dominant hand, accelerometer and gyroscope. This extract matches Weiss at 15,630,426 rows. The archived notebook concatenated 15,649,253. Claimed 20 Hz; sessions also sit at 25, 50, and 100 Hz. 35 empty subject×activity×stream cells. No demographics, so no fairness slice. Codes A–S skipping N.

Method

Repair

Interpolate each session onto a shared 20 Hz grid. Align accel and gyro by coverage intersection, not an exact-timestamp join. Reorient and 15 s trim exist as ablations and stay off by default.

Windows

5.0 s length, 1.0 s hop, inside one (subject, activity, device) session. Statistical features are 104-d. Flattened raw windows exist only to compare against v1.

Models

Stratified dummy, logistic regression, random forest, XGBoost. Trees beat the classical phone ladder, so no 1D CNN or TCN ships. Hierarchical group-then-expert is an ablation.

Protocols

A1/A2 leaky clones of the notebook split. B is 5-fold GroupKFold on subject_id. C is 46/5 × 3 grouped holdout, not 51-fold LOSO. D (phone↔watch transfer) is not run.
A1 / A2 leakyshuffle windowsSame 51 idsboth sidesB GroupKFoldgroups = subject_idMain tablecite thisC 46/5 x 3grouped holdoutNot LOSOtracks B, not AD phone vs watchhardware transferNot runspecified only
Every public number names a protocol. A2 vs B is the leakage pair. C tracks B, not A. D is specified and not run.
Mermaid source
mermaid
flowchart TB
  A["A1/A2 leaky shuffle"] --> A2["same 51 people both sides"]
  B["B GroupKFold on subject_id"] --> B2["main table, no shared people"]
  C["C 46/5 x 3 grouped holdout"] --> C2["phone check, not 51-fold LOSO"]
  D["D phone vs watch"] --> D2["specified, not run"]

Frozen results

Full 51-subject UCI 507, repaired to 20 Hz. Cite Protocol B watch 0.7031 from docs/reports/protocol_b_watch_stat_xgb.json, not from the served ONNX file. A2 vs B is the leakage pair on the same 5 s flatten. Do not treat A2 vs the notebook 0.8559 as leakage-only: A1/A2 already sit on repaired parquet.

Student notebook (leaky)
0.8559 accmacro-F1 not reported
A1 leaky 80-sample flatten
0.8490macro-F1 · 0.8475 acc
A2 leaky 5 s flatten
0.8925same 51 people both sides
B same flatten, GroupKFold
0.2924leakage pair with A2
B phone dummy
0.0151chance floor
B phone logreg
0.2767statistical 104-d
B phone random forest
0.3131statistical 104-d
B phone XGBoost
0.3272200 trees · 5 s
B watch XGBoost
0.7031number to cite
B concat (stacked 6-ch rows)
0.5236not 12-channel fusion
C phone 46/5 × 3
0.2985tracks B, not A

Failure cases

Pocket phone can tell locomotion as a group and still cannot name eating or sitting. Watch on the wrist flips that. Sandwich is the hard served class.

Phone locomotion group F1
0.8873Protocol B stat XGB
Phone eating group F1
0.4945pasta 0.0749
Phone sitting F1
0.1943worse than stairs
Watch eating group F1
0.8450sandwich 0.2816
Watch locomotion group F1
0.9292stairs 0.7028

Ablations (phone statistical XGBoost, Protocol B)

Control 5 s XYZ
0.3272eating group 0.4945
Window 10 s
0.3422only clear 18-way win
Window 2 s
0.2951worse
Trim 15 s
0.3247eating group drops
Reorient on
0.3230default stays off
Magnitude only
0.314232 features
Hierarchical
0.3271eating group 0.5855

Serving

CPU FastAPI for one 5 s, 20 Hz window (T=100, C=6). Default bundle is watch statistical XGBoost. Trees are ONNX; statistical features stay in Python. Wrong T, C, device, or Hz is 422. Bodies over 1 MiB are 413. Abstain is max(proba) < threshold; default threshold 0.0 never abstains. p95 2.7 ms is FastAPI TestClient on this CPU, not Docker. The watch trees also live on Hugging Face as axlesubash/wisdm-watch-stat-xgb. Cite Protocol B from docs/reports/protocol_b_watch_stat_xgb.json, not an export-fit score from the Hub file. Phone windows are a different bundle.

Watch windows5 s, 20 Hz, C=6104-d statsPythonXGBoost treesONNX/predictCPUExport refitnot a GroupKFold foldCite JSON0.7031 Protocol B
Cite GroupKFold from the metrics JSON. The served ONNX is a refit, not a fold. Wrong device or window length is 422.
Mermaid source
mermaid
flowchart LR
  cfg["protocol_b_watch_stat_xgb"] --> fit["Refit on all windows"]
  fit --> onnx["ONNX trees"]
  py["Python 104-d stats"] --> api["FastAPI POST /predict"]
  onnx --> api
  api --> ok["18-way label"]
  api --> bad["422 on T/C/device/Hz"]
bash
python -m har.models.export --config configs/protocol_b_watch_stat_xgb.yaml --out models/watch_stat_xgb.onnx
export HAR_MODEL_PATH=models/watch_stat_xgb.onnx
make serve

Key findings

  • Same 5 s phone flatten: 0.8925 leaky macro-F1 vs 0.2924 GroupKFold. That drop is the leakage finding.
  • Watch statistical XGBoost is 0.7031 under Protocol B. Phone statistical XGBoost is 0.3272. Concat stacked rows is 0.5236, not 12-channel fusion.
  • Eighteen classes, not nineteen. Sitting is a weak phone class under GroupKFold (F1 0.1943), not a standout.
  • Trees beat dummy, logreg, RF, and flatten on the phone B ladder, so no TCN shipped.
  • 10 s windows are the only ablation that clearly beats 5 s (0.3422). Reorient and a 15 s trim do not.

What I no longer claim

The June writeup said sitting and writing were easy, fusion would lift the rest, and a CNN-LSTM could pass 90%. Under subject-grouped splits, sitting is a weak phone class, concat without alignment sits between phone and watch, and I did not train a network because trees already beat flatten, logreg, and RF on this ladder. I would rather run aligned fusion and LOSO than chase the leaky 0.86.

Limits

  • No demographics, so no fairness slice.
  • Protocol C is 46/5 × 3, not 51-fold LOSO. Protocol D is not run.
  • Served ONNX is a refit. Abstain is uncalibrated. Features still run in Python.
  • v1.0.0 notebooks are archived and are not the training path.