Publications
JOURNAL (INTERNATIONAL) Deciding to Stop Early or Continue the Experiment After Checking p-Values at Interim Points: Introducing Group Sequential Designs to UI-Based Comparative Studies
International Journal of Human–Computer Interaction (IJHCI)
October 09, 2024
Null hypothesis significance testing (NHST) is widely used in the field of Human-Computer Interaction (HCI), and caution has been advised against deciding whether to stop or continue an experiment based on checking p-values. However, in clinical trials, NHST with interim analysis is commonly employed by adjusting the alpha levels to control Type I error rates. This approach, known as a group sequential design, can also be beneficial in the HCI field. We analyzed existing datasets of target-pointing tasks using group sequential designs and experimentally demonstrated that, depending on the $\alpha$-level correction method, we could reduce the number of participants by 40.4-56.5%.
Paper :
Deciding to Stop Early or Continue the Experiment After Checking p-Values at Interim Points: Introducing Group Sequential Designs to UI-Based Comparative Studies
(external link)