Publications
CONFERENCE (INTERNATIONAL) ReflecTrace: Hover Interface using Corneal Reflection Images Captured by Smartphone Front Camera
Yudai Nakamura (Keio University), Kaori Ikematsu, Naoto Takayanagi (Keio University), Kunihiro Kato (Tokyo University of Technology), Yuta Sugiura (Keio University)
The ACM Symposium on User Interface Software and Technology (UIST2025)
September 27, 2025
We propose a method to detect finger hover input on a smartphone screen using corneal reflection images captured by its built-in front-facing camera. This method requires no external sensors or additional hardware and enables the detection of finger positions in the near-screen space that cannot be directly captured by the front camera. We use a convolutional neural network (CNN) to estimate the 2D position of a hovering finger and identify which cell in a predefined grid is being hovered over. Our model achieved high accuracy in 2D position estimation across different grid settings, exceeding 90 percent with coarse grids such as 2 × 2 and maintaining over 80 percent even with finer divisions up to 5 × 5, demonstrating its robustness and adaptability.
Paper :
ReflecTrace: Hover Interface using Corneal Reflection Images Captured by Smartphone Front Camera
(external link)