Please report any queries concerning the funding data grouped in the
sections named
"Externally Awarded"
or
"Internally Disbursed"
(shown on the profile page) to your Research Finance Administrator.
Your can find your Research Finance Administrator at http://www.ucl.ac.uk/finance/research/post_award/post_award_contacts.php
by entering your department
Please report any queries concerning the student data shown on the
profile page to:
Email: portico-services@ucl.ac.uk
Help Desk: http://www.ucl.ac.uk/ras/portico/helpdesk
Email: portico-services@ucl.ac.uk
Help Desk: http://www.ucl.ac.uk/ras/portico/helpdesk
Publication Detail
Perturbation Validation: A New Heuristic to Validate Machine Learning Models
-
Publication Type:Working discussion paper
-
Authors:Zhang JM, Harman M, Guedj B, Barr ET, Shawe-Taylor J
-
Publisher:ArXiv
-
Publication date:24/05/2019
-
Place of publication:Ithaca, NY, USA
-
Keyword:cs.LG, cs.LG, stat.ML
-
Publisher URL:
Abstract
This paper introduces Perturbation Validation (PV), a new heuristic to validate machine learning models. PV does not rely on test data. Instead, it perturbs training data labels, re-trains the model against the perturbed data, then uses the consequent training accuracy decrease rate to assess model fit. PV also differs from traditional statistical approaches, which make judgements without considering label distribution. We evaluate PV on 10 real-world datasets and 6 synthetic datasets. Our results demonstrate that PV is more discriminating about model fit than existing validation approaches and it accords well with widely-held intuitions concerning the properties of a good model fit measurement. We also show that PV complements existing validation approaches, allowing us to give explanations for some of the issues present in the recently-debated "apparent paradox" that high capacity (potentially "overfitted") models may, nevertheless, exhibit good generalisation ability.
› More
search options
UCL Researchers
Show More