Hung Luu: Can we test machine learning models without ground truth?
The verification of machine learning (ML) models is challenging because traditional approaches rely on annotated data, external observations, or trusted reference models as ground truth, which are often unavailable, rare or expensive to obtain. In this talk, we introduce a testing paradigm that does not require such ground truth, but instead uses a model’s own properties to evaluate the correctness of its predictions. The approach is based on metamorphic testing, which uses metamorphic relations (MRs) between multiple inputs and outputs of the model or system under test to reveal failures. We identify a set of MRs and demonstrate that they can detect bugs in regression programs, reveal failure modes in perception systems in autonomous driving, and help understand internal consistencies in NeuralGCM, a state-of-the-art ML weather forecasting model. This study will be of interest to ML researchers, particularly those are wworking on the model development, verification and explainability.