CopCo TYP
Dyslexia Detection (CopCo)
Test
| Model | Unseen Reader Balanced Accuracy | Unseen Text Balanced Accuracy | Unseen Text and Reader Balanced Accuracy | Average Balanced Accuracy | Unseen Reader AUROC | Unseen Text AUROC | Unseen Text and Reader AUROC | Average AUROC |
|---|
| Majority Class / Chance | 50.3 ± 0.3 | 49.6 ± 0.4 | 50.1 ± 0.1 | 49.9 ± 0.0 | 50.3 ± 0.3 | 49.6 ± 0.4 | 50.1 ± 0.1 | 49.9 ± 0.0 |
| Reading Speed | 57.7 ± 2.2 | 54.9 ± 1.8 | 50.6 ± 2.7 | 55.1 ± 2.1 | 60.7 ± 2.0 | 56.2 ± 2.0 | 50.9 ± 4.8 | 56.6 ± 2.1 |
| Text-Only Roberta | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.0 ± 0.0 | 47.0 ± 4.4 | 50.0 ± 0.2 | 50.4 ± 1.1 | 50.1 ± 0.4 |
| Logistic Regression [meziere2023using] | 75.5 ± 3.1 | 76.6 ± 1.6 | 63.5 ± 5.1 | 73.8 ± 2.1 | 83.1 ± 3.1 | 83.3 ± 1.6 | 68.9 ± 6.6 | 80.6 ± 2.3 |
| SVM [hollenstein2023zuco] | 70.7 ± 2.4 | 77.4 ± 1.7 | 64.7 ± 3.1 | 72.5 ± 1.6 | 70.7 ± 2.4 | 77.4 ± 1.7 | 64.7 ± 3.1 | 72.5 ± 1.6 |
| Random Forest [makowski2024detection] | 69.8 ± 4.2 | 81.5 ± 2.2 | 59.7 ± 4.9 | 72.7 ± 1.9 | 80.1 ± 3.6 | 91.5 ± 1.5 | 65.9 ± 6.6 | 82.9 ± 1.5 |
| AhnRNN [ahn2020towards] | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.1 ± 0.1 | 50.0 ± 0.0 | 50.0 ± 0.1 | 50.0 ± 0.1 |
| AhnCNN [ahn2020towards] | 77.7 ± 1.8 | 77.5 ± 2.7 | 65.6 ± 2.4 | 75.0 ± 0.8 | 85.3 ± 1.6 | 85.7 ± 2.3 | 74.9 ± 2.8 | 83.4 ± 1.1 |
| BEyeLSTM [reich_inferring_2022] | 71.9 ± 2.1 | 76.8 ± 1.7 | 64.7 ± 5.0 | 72.9 ± 1.3 | 79.4 ± 2.8 | 85.0 ± 1.3 | 69.2 ± 6.2 | 80.2 ± 1.5 |
| PLM-AS [Yang2023PLMASPL] | 55.2 ± 4.3 | 57.3 ± 3.4 | 55.9 ± 2.2 | 56.0 ± 3.2 | 57.6 ± 5.9 | 58.5 ± 4.9 | 59.4 ± 0.9 | 57.9 ± 4.6 |
| PLM-AS-RM [haller2022eye] | 60.9 ± 2.8 | 71.6 ± 2.4 | 54.6 ± 1.1 | 63.7 ± 0.9 | 63.9 ± 4.3 | 80.1 ± 1.9 | 55.0 ± 1.9 | 69.2 ± 1.0 |
| RoBERTEye-W [Shubi2024Finegrained] | 70.0 ± 4.0 | 68.5 ± 3.0 | 61.9 ± 4.6 | 67.4 ± 2.9 | 78.3 ± 3.2 | 76.7 ± 2.9 | 68.2 ± 5.3 | 75.6 ± 2.3 |
| RoBERTEye-F [Shubi2024Finegrained] | 60.6 ± 2.1 | 60.3 ± 2.4 | 54.0 ± 2.1 | 58.9 ± 1.4 | 71.9 ± 4.2 | 74.7 ± 1.3 | 63.3 ± 2.3 | 70.8 ± 1.9 |
| MAG-Eye [Shubi2024Finegrained] | 47.2 ± 2.4 | 49.7 ± 0.2 | 51.4 ± 1.2 | 50.3 ± 0.3 | 45.9 ± 7.5 | 54.7 ± 4.1 | 56.1 ± 1.0 | 53.3 ± 3.7 |
| PostFusion-Eye [Shubi2024Finegrained] | 64.7 ± 4.3 | 68.9 ± 2.3 | 57.0 ± 3.7 | 64.4 ± 2.4 | 73.1 ± 4.0 | 78.1 ± 1.6 | 65.5 ± 3.4 | 73.2 ± 1.5 |
Validation
| Model | Unseen Reader Balanced Accuracy | Unseen Text Balanced Accuracy | Unseen Text and Reader Balanced Accuracy | Average Balanced Accuracy | Unseen Reader AUROC | Unseen Text AUROC | Unseen Text and Reader AUROC | Average AUROC |
|---|
| Majority Class / Chance | 50.9 ± 0.8 | 50.9 ± 0.8 | 50.0 ± 0.0 | 50.5 ± 0.4 | 50.9 ± 0.8 | 50.9 ± 0.8 | 50.0 ± 0.0 | 50.5 ± 0.4 |
| Reading Speed | 57.9 ± 4.1 | 55.1 ± 2.2 | 52.1 ± 2.1 | 55.1 ± 2.5 | 60.4 ± 4.5 | 57.7 ± 4.6 | 50.9 ± 4.8 | 56.3 ± 2.6 |
| Text-Only Roberta | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.0 ± 0.0 | 53.5 ± 2.5 | 51.6 ± 1.1 | 49.4 ± 0.5 | 52.4 ± 1.7 |
| Logistic Regression [meziere2023using] | 72.1 ± 2.9 | 79.2 ± 3.3 | 64.1 ± 3.3 | 72.4 ± 1.5 | 78.3 ± 2.4 | 85.3 ± 3.5 | 69.7 ± 5.6 | 78.7 ± 2.0 |
| SVM [hollenstein2023zuco] | 72.8 ± 2.2 | 83.2 ± 2.6 | 67.0 ± 3.1 | 74.8 ± 1.0 | 72.8 ± 2.2 | 83.2 ± 2.6 | 67.0 ± 3.1 | 74.8 ± 1.0 |
| Random Forest [makowski2024detection] | 72.7 ± 4.0 | 86.9 ± 2.2 | 66.0 ± 2.7 | 75.2 ± 1.8 | 81.5 ± 3.8 | 95.2 ± 1.2 | 73.1 ± 3.6 | 83.9 ± 2.3 |
| AhnRNN [ahn2020towards] | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.0 ± 0.0 | 50.0 ± 0.0 | 49.9 ± 0.1 | 49.9 ± 0.1 | 49.9 ± 0.0 | 49.9 ± 0.0 |
| AhnCNN [ahn2020towards] | 73.9 ± 3.0 | 77.5 ± 2.6 | 68.3 ± 1.1 | 73.5 ± 1.6 | 82.0 ± 1.8 | 86.5 ± 3.1 | 74.9 ± 1.7 | 81.3 ± 1.5 |
| BEyeLSTM [reich_inferring_2022] | 72.6 ± 2.2 | 76.7 ± 3.2 | 68.3 ± 3.7 | 73.1 ± 2.1 | 80.3 ± 2.5 | 85.0 ± 3.7 | 74.0 ± 4.0 | 80.4 ± 2.6 |
| PLM-AS [Yang2023PLMASPL] | 52.5 ± 2.0 | 61.6 ± 3.3 | 53.3 ± 1.2 | 55.8 ± 2.2 | 54.2 ± 3.6 | 66.4 ± 4.5 | 55.4 ± 2.1 | 58.8 ± 2.3 |
| PLM-AS-RM [haller2022eye] | 63.9 ± 4.6 | 68.9 ± 5.0 | 57.1 ± 3.8 | 62.8 ± 2.1 | 65.9 ± 5.7 | 74.2 ± 6.4 | 62.1 ± 4.2 | 67.2 ± 3.0 |
| RoBERTEye-W [Shubi2024Finegrained] | 62.4 ± 3.3 | 69.8 ± 3.8 | 59.6 ± 2.4 | 63.5 ± 1.7 | 71.2 ± 3.2 | 77.6 ± 4.2 | 67.5 ± 3.2 | 71.6 ± 1.8 |
| RoBERTEye-F [Shubi2024Finegrained] | 57.0 ± 1.5 | 61.2 ± 3.2 | 55.2 ± 1.8 | 57.6 ± 1.7 | 70.5 ± 3.1 | 75.5 ± 3.4 | 65.7 ± 2.2 | 69.9 ± 1.8 |
| MAG-Eye [Shubi2024Finegrained] | 51.2 ± 1.0 | 52.8 ± 2.5 | 49.8 ± 0.2 | 51.7 ± 1.5 | 60.5 ± 1.9 | 62.0 ± 5.9 | 59.4 ± 4.0 | 60.6 ± 4.0 |
| PostFusion-Eye [Shubi2024Finegrained] | 65.6 ± 2.3 | 72.0 ± 4.2 | 60.6 ± 2.8 | 65.9 ± 2.2 | 75.7 ± 2.0 | 79.3 ± 4.0 | 70.7 ± 2.7 | 74.1 ± 1.8 |