IITBHGC CV

Claim Verification (IITBHGC)

Test

Model	Unseen Reader Balanced Accuracy	Unseen Text Balanced Accuracy	Unseen Text and Reader Balanced Accuracy	Average Balanced Accuracy	Unseen Reader AUROC	Unseen Text AUROC	Unseen Text and Reader AUROC	Average AUROC
Majority Class / Chance	49.5 ± 0.4	51.0 ± 0.9	50.6 ± 0.6	50.4 ± 0.3	49.5 ± 0.4	51.0 ± 0.9	50.6 ± 0.6	50.4 ± 0.3
Reading Speed	55.6 ± 0.4	55.5 ± 1.0	55.8 ± 1.4	55.6 ± 0.6	56.5 ± 1.1	56.9 ± 1.4	58.0 ± 1.3	57.3 ± 0.7
Text-Only Roberta	55.9 ± 2.6	49.8 ± 0.3	49.7 ± 0.7	52.5 ± 1.4	62.5 ± 2.7	55.1 ± 1.2	56.1 ± 5.4	58.8 ± 1.5
Logistic Regression [meziere2023using]	53.3 ± 1.1	54.2 ± 1.5	55.7 ± 0.8	53.9 ± 1.0	53.2 ± 0.9	55.5 ± 1.6	55.6 ± 0.7	54.6 ± 1.1
SVM [hollenstein2023zuco]	52.9 ± 0.8	55.0 ± 1.4	57.8 ± 1.4	54.5 ± 0.6	52.9 ± 0.8	55.0 ± 1.4	57.8 ± 1.4	54.5 ± 0.6
Random Forest [makowski2024detection]	56.0 ± 1.6	52.2 ± 0.5	51.6 ± 2.1	53.4 ± 0.6	59.6 ± 0.8	54.0 ± 0.4	55.7 ± 1.4	56.4 ± 0.3
AhnRNN [ahn2020towards]	50.0 ± 0.0	50.0 ± 0.0	50.0 ± 0.0	50.0 ± 0.0	51.2 ± 1.0	50.7 ± 0.5	50.6 ± 0.5	50.9 ± 0.7
AhnCNN [ahn2020towards]	50.9 ± 1.0	51.4 ± 1.1	55.4 ± 2.3	51.8 ± 0.9	51.4 ± 1.2	52.9 ± 1.8	55.5 ± 2.2	52.9 ± 1.0
BEyeLSTM [reich_inferring_2022]	51.6 ± 1.8	49.0 ± 1.4	51.7 ± 0.9	50.2 ± 1.1	53.3 ± 2.3	48.9 ± 1.4	53.1 ± 1.2	51.3 ± 1.2
PLM-AS [Yang2023PLMASPL]	53.3 ± 2.3	48.8 ± 0.9	47.5 ± 2.0	50.6 ± 0.9	53.0 ± 2.4	50.4 ± 0.7	50.3 ± 4.2	51.4 ± 0.6
PLM-AS-RM [haller2022eye]	52.6 ± 1.2	50.6 ± 0.8	50.6 ± 1.8	51.3 ± 0.8	55.6 ± 2.6	51.7 ± 1.1	51.3 ± 1.7	53.4 ± 1.5
RoBERTEye-W [Shubi2024Finegrained]	55.9 ± 3.0	50.5 ± 1.1	53.1 ± 1.7	53.4 ± 2.1	64.3 ± 1.5	53.0 ± 2.0	54.5 ± 3.2	58.0 ± 2.2
RoBERTEye-F [Shubi2024Finegrained]	53.4 ± 2.9	49.4 ± 0.6	49.8 ± 0.2	50.9 ± 0.7	62.2 ± 2.3	54.9 ± 0.6	58.8 ± 2.5	58.4 ± 1.1
MAG-Eye [Shubi2024Finegrained]	57.1 ± 2.9	49.7 ± 1.2	50.8 ± 2.3	52.8 ± 1.5	65.3 ± 1.9	51.7 ± 2.4	55.2 ± 4.4	58.0 ± 2.0
PostFusion-Eye [Shubi2024Finegrained]	51.9 ± 0.4	50.9 ± 0.1	52.6 ± 1.4	51.5 ± 0.3	60.2 ± 1.9	56.3 ± 2.6	59.0 ± 0.9	57.5 ± 1.4

Validation

Model	Unseen Reader Balanced Accuracy	Unseen Text Balanced Accuracy	Unseen Text and Reader Balanced Accuracy	Average Balanced Accuracy	Unseen Reader AUROC	Unseen Text AUROC	Unseen Text and Reader AUROC	Average AUROC
Majority Class / Chance	50.6 ± 0.5	51.4 ± 1.2	51.3 ± 1.1	51.0 ± 0.8	50.6 ± 0.5	51.4 ± 1.2	51.3 ± 1.1	51.0 ± 0.8
Reading Speed	56.1 ± 0.8	56.1 ± 0.9	56.5 ± 1.5	56.2 ± 0.4	56.9 ± 1.5	56.0 ± 1.0	58.0 ± 1.3	57.0 ± 0.7
Text-Only Roberta	60.0 ± 4.0	51.1 ± 1.2	48.9 ± 1.2	53.9 ± 1.8	66.1 ± 3.5	54.7 ± 1.3	55.2 ± 2.0	59.2 ± 1.9
Logistic Regression [meziere2023using]	54.1 ± 1.5	54.4 ± 1.2	55.9 ± 2.2	54.6 ± 1.2	54.8 ± 1.8	56.2 ± 0.5	55.2 ± 2.2	55.2 ± 1.4
SVM [hollenstein2023zuco]	57.6 ± 1.3	55.6 ± 1.2	54.6 ± 1.3	56.1 ± 0.6	57.6 ± 1.3	55.6 ± 1.2	54.6 ± 1.3	56.1 ± 0.6
Random Forest [makowski2024detection]	60.0 ± 1.2	56.5 ± 0.8	55.3 ± 1.8	57.5 ± 0.8	63.7 ± 1.6	60.6 ± 2.0	57.1 ± 2.1	61.2 ± 1.4
AhnRNN [ahn2020towards]	50.0 ± 0.0	50.0 ± 0.0	50.0 ± 0.0	50.0 ± 0.0	50.9 ± 0.7	50.9 ± 0.6	50.1 ± 0.1	50.8 ± 0.6
AhnCNN [ahn2020towards]	53.7 ± 1.5	52.2 ± 0.3	53.7 ± 1.0	53.2 ± 0.6	57.5 ± 0.4	55.4 ± 0.7	55.6 ± 1.4	56.1 ± 0.3
BEyeLSTM [reich_inferring_2022]	56.6 ± 0.9	52.9 ± 1.5	52.2 ± 0.4	54.0 ± 0.9	61.8 ± 0.7	55.3 ± 1.1	54.9 ± 0.9	57.5 ± 0.7
PLM-AS [Yang2023PLMASPL]	52.6 ± 1.1	53.6 ± 1.8	51.7 ± 2.4	53.1 ± 1.0	54.0 ± 1.8	56.7 ± 3.4	51.8 ± 2.2	54.6 ± 2.2
PLM-AS-RM [haller2022eye]	57.1 ± 2.9	50.7 ± 0.8	50.0 ± 1.0	53.2 ± 1.5	64.1 ± 3.3	50.6 ± 0.6	51.0 ± 2.8	56.1 ± 1.2
RoBERTEye-W [Shubi2024Finegrained]	58.7 ± 3.6	54.0 ± 2.2	51.3 ± 1.3	55.5 ± 2.6	71.4 ± 2.3	59.4 ± 1.8	56.2 ± 0.9	63.9 ± 1.9
RoBERTEye-F [Shubi2024Finegrained]	55.2 ± 4.2	49.7 ± 0.3	50.6 ± 0.6	51.6 ± 1.3	65.8 ± 3.1	55.8 ± 2.4	56.0 ± 2.9	59.5 ± 0.4
MAG-Eye [Shubi2024Finegrained]	62.3 ± 4.2	52.0 ± 1.2	50.7 ± 1.6	55.7 ± 1.9	72.2 ± 1.4	56.0 ± 0.9	55.3 ± 2.0	62.9 ± 1.1
PostFusion-Eye [Shubi2024Finegrained]	51.9 ± 0.7	50.9 ± 0.8	53.4 ± 0.7	51.6 ± 0.5	59.3 ± 1.1	56.7 ± 2.0	60.0 ± 1.2	57.7 ± 0.6