System evaluation
Compares single models (image, sound, video) with the combined multimodal model.
File sample.json
FAKE SAMPLE DATA used only to build and test the dashboard. Not measured. Replace with real metrics files.
Generated 2026-01-01T00:00:00Z
Single models vs combined
| Model | Accuracy | Sensitivity | Specificity | F1 | Latency (ms) | Model size |
|---|---|---|---|---|---|---|
| Skin image | 71.7% | 71.7% | 85.8% | 71.5% | 180 | 9.4 MB |
| Audio | 70.7% | 70.7% | 85.3% | 70.5% | 95 | 4.1 MB |
| Respiration video | 68.3% | 68.3% | 84.2% | 68.1% | 40 | 15.0 KB |
| Multimodal fusion | 81.7% | 81.7% | 90.8% | 81.6% | 320 | 14.6 MB |
Comparison chart
Sensitivity / specificity per class
| Class | Sensitivity | Specificity |
|---|---|---|
| green | 78.0% | 89.5% |
| amber | 60.0% | 84.5% |
| red | 77.0% | 83.5% |
| Class | Sensitivity | Specificity |
|---|---|---|
| green | 80.0% | 86.5% |
| amber | 62.0% | 83.5% |
| red | 70.0% | 86.0% |
| Class | Sensitivity | Specificity |
|---|---|---|
| green | 82.0% | 85.0% |
| amber | 58.0% | 81.5% |
| red | 65.0% | 86.0% |
| Class | Sensitivity | Specificity |
|---|---|---|
| green | 88.0% | 93.5% |
| amber | 72.0% | 90.0% |
| red | 85.0% | 89.0% |
Confusion matrix
Rows = actual, columns = predicted
| Actual \ Predicted | green | amber | red |
|---|---|---|---|
| green | 78 | 14 | 8 |
| amber | 15 | 60 | 25 |
| red | 6 | 17 | 77 |
| Actual \ Predicted | green | amber | red |
|---|---|---|---|
| green | 80 | 12 | 8 |
| amber | 18 | 62 | 20 |
| red | 9 | 21 | 70 |
| Actual \ Predicted | green | amber | red |
|---|---|---|---|
| green | 82 | 12 | 6 |
| amber | 20 | 58 | 22 |
| red | 10 | 25 | 65 |
| Actual \ Predicted | green | amber | red |
|---|---|---|---|
| green | 88 | 8 | 4 |
| amber | 10 | 72 | 18 |
| red | 3 | 12 | 85 |
Test set
- Skin image: FAKE SAMPLE: 300 invented cases (100 per tier). Not measured on any real data.
- Audio: FAKE SAMPLE: 300 invented cases (100 per tier). Not measured on any real data.
- Respiration video: FAKE SAMPLE: 300 invented cases (100 per tier). Not measured on any real data.
- Multimodal fusion: FAKE SAMPLE: 300 invented cases (100 per tier). Not measured on any real data.
File skin.json
MEASURED on a held-out, group-level (lesion/patient/case) test split of HAM10000 + PAD-UFES-20 + SCIN + healthy-skin patches cut from PAD-UFES-20 / SCIN photos. skin-v1.1.0 int8 ONNX (20 classes incl. healthy-skin), centre-square crop, sensitivity-first tier thresholds tuned on validation. 'before' = skin-v1.0.0 as shipped, same test images. Baseline for a school project: not a diagnostic device. Per-source before/after, 20-class results and Fitzpatrick breakdown are in the extra keys and in docs/model-cards/skin-v1.md.
Generated 2026-10-30T09:35:17+00:00
Single models vs combined
| Model | Accuracy | Sensitivity | Specificity | F1 | Latency (ms) | Model size |
|---|---|---|---|---|---|---|
| Skin image | 70.2% | 71.0% | 87.0% | 57.9% | 6.1 | 3.8 MB |
Comparison chart
Sensitivity / specificity per class
| Class | Sensitivity | Specificity |
|---|---|---|
| green | 67.8% | 92.7% |
| amber | 52.8% | 84.6% |
| red | 92.3% | 83.8% |
Confusion matrix
Rows = actual, columns = predicted
| Actual \ Predicted | green | amber | red |
|---|---|---|---|
| green | 1461 | 371 | 323 |
| amber | 35 | 105 | 59 |
| red | 8 | 22 | 362 |
Test set
- Skin image: Held-out TEST split: 2746 images from 1676 lesions/patients/cases (HAM10000 1423, PAD-UFES-20 517, SCIN 806; 574 are healthy-skin patches). Tier = sensitivity-first thresholds on class-probability sums; 'uncertain' answers are left out of the confusion matrix. Latency = desktop CPU onnxruntime, not a phone.