Michael.Walker's blog

Seasoned data scientists attempt to simplify complexity. Yet the process of simplification is itself complex.

Elite professional athletes, surgeons and traders will tell you that "simple yet hard" is the norm. One must master complex techniques and conceptual frameworks yet simplify and embed them deep into the subconscious for elite performance. This takes many years of learning, hard work and experience.

The world is creating a massive amount of data every day and organizations need a technology, information and data science strategy to collect, store, analyze and distribute data from multiple sources.

The "Data Lake" strategy is to collect and store all of this data.

The "Smart Data" strategy is to ONLY collect and store high value data relevant to achieve specific goals.

5G and the Internet of Things is now just starting to be developed at scale. We are about to enter a new age of pervasive connectivity and ubiquitous computing where technology recedes into the background of our lives. The new architecture will be invisible, quiet and a calm extension of human intelligence that informs but doesn't demand our focus or attention. 

Recent advances appear to make the High Energy Electromagnetic Field Generator (HEEMFG) a reality. HEEMFG has the potential to create massive amounts of cheap energy and more powerful propulsion engines.

HEEMFG has both commercial and military capabilities and has the potential to be a game changing technology.

Data science, high performance computing (HPC) and machine learning algorithms allow organizations to make better decisions and create game-changing strategies. The integration of high quality smart data, computing resources and data scientists using learning algorithms is the secret sauce to achieving the fundamental goal of creating durable competitive advantage.

Organizations with the best data scientists and learning algorithms (AI) have a significant competitive advantage. Professional data scientists use artificial narrow intelligence for massive advantage. Machine learning algorithms can add great value in certain circumstances within appropriate contexts and within data science processes. 

Seasoned data scientists learn to love uncertainty. Making high quality decisions under uncertainty creates massive competitive advantage. The more limited information and uncertainty the greater the advantage for the professional data scientist.

Acquiring awesome data and applying probability theory is optimal strategy when making decisions under uncertainty. Over time, high quality decisions will provide consistent probability advantages and mitigating risks (reducing bad decisions) will lower probability of blowing up.

The nature of war has changed. Traditional kinetic warfare is still operational yet evolving from massive air carriers and big tanks with large ground forces to smaller special forces relying on near real-time intelligence to achieve specific goals within macro strategy.

The easiest person in the world to fool is yourself. Data scientists sometimes fool themselves - in matters trivial and important. Thus, I suggest we acknowledge real or subconscious biases in ourselves, the data, the analysis and group think. It is prudent for data science teams to have both internal and external checks and balances to expose potential biases and better understand objective reality. Here are a few ways data scientists sometimes fool themselves:

Confirmation bias: tendency to favor data that confirms beliefs or hypotheses.

The Nuremberg Code was drafted to protect humans from medical experiments based on exploitation of prisoners by German scientists and physicians during World War II. Ten (10) elements of Code: