Black Box Models

DSA ADS Course - 2022

Discuss the proper use and misuse of modeling. Discuss formulating appropriate assumptions and danger of simplifying assumptions. Discuss methods to evaluate and judge reasonable, dubious or untestable assumptions.

Discuss appropriate setting of model specifications and parameters.

Discuss subjecting models to rigorous empirical tests to avoid creating an illusion of reality.

Review:

DSA ADS Course, 2022

Data Quality, Black Box Models, Bad Models, Origins of SARS-CoV-2, Epidemiology, Non-pharmaceutical Interventions, Mitigation Strategies, COVID19, Health Policy, Public Policy

Review data quality evaluations, bad model design, policy decision-making based on data science, black box models, unwarranted assumptions in models, and evidence based policy making with near real-time data.

July, 2021

Abstract

This short note is a reminder that all models only say what they are told to say. There can therefore be no discovery by models—discoveries go into models—and thus models should only be used in their predictive form, and only trusted when they have demonstrated independent success.

See also: Proper Use and Misuse of Modeling - DSA ADS Course 2022

DSA ADS Course - 2022

November, 2018

Abstract

There are black box models now being used for high stakes decision-making throughout society. The practice of trying to explain black box models, rather than creating models that are interpretable in the first place, is likely to perpetuate bad practices and can potentially cause catastrophic harm to society. There is a way forward – it is to design models that are inherently interpretable.