Quantum and Quality of Evidence

Professional data scientists rank quantum and quality of evidence. Evidence can be strong, weak (or somewhere in between) or uncertain and incomplete.

Pursuant to Rule 8 of the Data Science Code of Professional Conduct, a data scientist shall rate and disclose evidence to enable informed decisions. The data scientist understands high quality data provides evidence that may be weak, strong or uncertain and shall take reasonable measures to protect the client/employer from relying and making decisions based on weak or uncertain evidence. 

It is critical for the ability to make optimal decisions to rank quantum and quality of evidence in a reasonable and understandable manner for the client/employer. Claiming weak or uncertain evidence is strong evidence may create an illusion of reality (or understanding complexity) and may lead to suboptimal decision making or potential disaster.

Note a data scientist shall NOT present weak, uncertain or incomplete evidence as strong or solid data science evidence to base decisions. In this case a data scientist may present both upside and downside risks from uncertain evidence - or present a theory constituting incomplete evidence but shall label and clearly communicate uncertain or incomplete evidence. This happens frequently and helps the client/employer weigh both upside and downside scenarios and potentially avoid massive downside risks that may cause significant damage.

Ranking quantum and quality of evidence is critical for optimal decision making and to avoid catastrophic downside risks in complex environments.