Publications
カンファレンス (国際) Improving Data Quality via Pre-Task Participant Screening in Crowdsourced GUI Experiments
Takaya Miyama (Meiji University), Satoshi Nakamura (Meiji University), Shota Yamanaka
The ACM CHI conference on Human Factors in Computing Systems (CHI 2026)
2026.4.13
In crowdsourced user experiments that collect performance data from graphical user interface (GUI) interactions, some participants ignore instructions or act carelessly, threatening the validity of performance models. We investigate a pre-task screening method that requires simple GUI operations analogous to the main task and uses the resulting error as a continuous quality signal. Our pre-task is a brief image-resizing task in which workers match an on-screen card to a physical card; workers whose resizing error exceeds a threshold are excluded from the main experiment. The main task is a standardized pointing experiment with well-established models of movement time and error rate. Across mouse- and smartphone-based crowdsourced experiments, we show that reducing the proportion of workers exhibiting unexpected behavior and tightening the pre-task threshold systematically improve the goodness of fit and predictive accuracy of GUI performance models, demonstrating that brief pre-task screening can enhance data quality.
Paper :
Improving Data Quality via Pre-Task Participant Screening in Crowdsourced GUI Experiments
（外部サイト）