"Is Not the Truth the Truth?": Analyzing the Impact of User Validations for Bus In/Out Detection in Smartphone-Based Surveys

Knowledge of passenger flow underpins any optimal public transport application, such as new facilities design and operations. The interactions between passengers and sensors may provide the foundation to measure this flow and dismiss users and staff from the measures' validation loop. Removing humans and their errors may impact significantly and improve measures' cost and quality. The literature considered smartphones the leading enabler due to market penetration and embodied sensors. Smartphones allow users' localization, identification, authentication, and billing. Via Bluetooth, smartphones detect short-range implicit interactions, device-to-device. We model passenger states on buses, either be-in or be-out (BIBO). The BIBO use case identifies a fundamental building block of continuously-valued passenger flow, which this paper describes through a Human-Computer interaction experimental setting involving two autonomous buses and a proprietary smartphone-Bluetooth sensing platform. The resulting dataset of 14,000 observations/sensors contains two ground-truth levels: the first is the passengers' validation; the second is validation by video cameras surveilling buses and tracks. This study verifies separately classification based on Bluetooth and GPS signals, as well as an inertial navigation system, evaluating signals and related machine learning (ML) classifiers against measurement and ground-truth noise. The paper contributes a Monte Carlo simulation of labels-flip to emulate human errors in the labeling process, as in smartphone surveys, and a novel unsupervised variational auto-encoder classifier. Experimental results indicate error-free human validation is unlikely. The impact of mistakes on model performance bias can be significant. This use case supports the potential substitution of human validation with independent Bluetooth validation.