Michael.Walker's blog

You are a seasoned professional data scientist. You love your work and made a cool US $2.4 million dollars in income last year.

Society anoints you the new Master of the Universe. You help elect US presidents and make your clients/employers huge profits. Others want to be like you and some want to love you.

Yet you must at all costs STAY HUMBLE. Successful people act with humility and treat others with respect.

Professional data scientists rank quality and veracity of data. Recently we have seen a significant rise in the amount of untruthful data and false data creation. During COVID19, we often see both untruthful and truthful data taken out of context and thus creating a misleading interpretation. Data scientists sometimes call this torturing the data to fit a narrative or theory.

Making predictions is currently in vogue yet most real world futures are unpredictable. Human behavior, social life, biology, government and business occur in high causal density environments. Complex pathogens like COVID19 are unpredictable yet both trained epidemiologists and amateurs attempt to predict and advocate policies to achieve vague goals. They mostly fail and look foolish.

Today's world is overloaded with data and information yet lacking in knowledge and wisdom. It is increasingly difficult to distinguish between reality and narrative. Remember wisdom of Freeman Dyson:

While dealing with COVID19 we must also consider a more serious and pernicious type of pathogen. Viruses of the mind appear to be a more serious long-term problem in the world today. We must honor the distinction between theoretical and practical reason.

Noise is the Signal

Professional data scientists are trained to distinguish between noise and signal. Detecting signal in large, diverse and messy data sets can add massive competitive advantage. Yet as successful professional financial traders can tell you, sometimes noise is the signal. There is a huge amount of weak signal within the noise - extremely difficult to detect - that correctly interpreted can add great value. Learning algorithms and high performance compute can help a well trained data scientist acquire high value from weak signals within noise.

Surveillance and Early Warning Systems > Models Using Bad Data and Speculative Assumptions

Low Cost Insurance > High Cost in Human Lives / Economic Damage

Manageable Healthcare Logistics > Healthcare Delivery Disaster

Making Policy Decisions with Uncertain & Unreliable Data

We are at war with a mysterious, unseen, global and deadly pathogen with exponential human-to-human transmission rates. We have very little reliable data about the pathogen. This may, or may not be a serious epidemic that infects and kills a significant number of people. We are flying blind and need a prudent plan of action to kill or disrupt the pathogen as soon as possible.