Professional data scientists rank quality and veracity of data. Recently we have seen a significant rise in the amount of untruthful data and false data creation. During COVID19, we often see both untruthful and truthful data taken out of context and thus creating a misleading interpretation. Data scientists sometimes call this torturing the data to fit a narrative or theory.
Making predictions is currently in vogue yet most real world futures are unpredictable. Human behavior, social life, biology, government and business occur in high causal density environments. Complex pathogens like COVID19 are unpredictable yet both trained epidemiologists and amateurs attempt to predict and advocate policies to achieve vague goals. They mostly fail and look foolish.
Today's world is overloaded with data and information yet lacking in knowledge and wisdom. It is increasingly difficult to distinguish between reality and narrative. Remember wisdom of Freeman Dyson:
While dealing with COVID19 we must also consider a more serious and pernicious type of pathogen. Viruses of the mind appear to be a more serious long-term problem in the world today. We must honor the distinction between theoretical and practical reason.
Noise is the Signal
Professional data scientists are trained to distinguish between noise and signal. Detecting signal in large, diverse and messy data sets can add massive competitive advantage. Yet as successful professional financial traders can tell you, sometimes noise is the signal. There is a huge amount of weak signal within the noise - extremely difficult to detect - that correctly interpreted can add great value. Learning algorithms and high performance compute can help a well trained data scientist acquire high value from weak signals within noise.
Surveillance and Early Warning Systems > Models Using Bad Data and Speculative Assumptions
Low Cost Insurance > High Cost in Human Lives / Economic Damage
Manageable Healthcare Logistics > Healthcare Delivery Disaster
Making Policy Decisions with Uncertain & Unreliable Data
We are at war with a mysterious, unseen, global and deadly pathogen with exponential human-to-human transmission rates. We have very little reliable data about the pathogen. This may, or may not be a serious epidemic that infects and kills a significant number of people. We are flying blind and need a prudent plan of action to kill or disrupt the pathogen as soon as possible.
The Graphical Kernel System (GKS) is a document produced by the International Standards Organization (ISO) which defines a common interface to interactive computer graphics for application programs. GKS has been designed by a group of experts representing the national standards institutions of most major industrialized countries. The full standard provides functional specifications for some 200 subroutines which perform graphics input and output in a device independent way. Application programs can thus move freely between different graphics devices and different host computers.