Michael.Walker's blog

Simple is Complex

Seasoned data scientists attempt to simplify complexity. Yet the process of simplification is itself complex.

Elite professional athletes, surgeons and traders will tell you that "simple yet hard" is the norm. One must master complex techniques and conceptual frameworks yet simplify and embed them deep into the subconscious for elite performance. This takes many years of learning, hard work and experience.

Smart Data vs. Data Lake

The world is creating a massive amount of data every day and organizations need a technology, information and data science strategy to collect, store, analyze and distribute data from multiple sources.

The "Data Lake" strategy is to collect and store all of this data.

The "Smart Data" strategy is to ONLY collect and store high value data relevant to achieve specific goals.

5G and Pervasive Connectivity

5G and the Internet of Things is now just starting to be developed at scale. We are about to enter a new age of pervasive connectivity and ubiquitous computing where technology recedes into the background of our lives. The new architecture will be invisible, quiet and a calm extension of human intelligence that informs but doesn't demand our focus or attention. 

Data Science + Learning Algorithms + High Performance Compute = Competitive Advantage

Data science, high performance computing (HPC) and machine learning algorithms allow organizations to make better decisions and create game-changing strategies. The integration of high quality smart data, computing resources and data scientists using learning algorithms is the secret sauce to achieving the fundamental goal of creating durable competitive advantage.

Artificial Narrow Intelligence vs. Artificial General Intelligence

Organizations with the best data scientists and learning algorithms (AI) have a significant competitive advantage. Professional data scientists use artificial narrow intelligence for massive advantage. Machine learning algorithms can add great value in certain circumstances within appropriate contexts and within data science processes. 

In Land of Blind - One Eyed Man is King

Seasoned data scientists learn to love uncertainty. Making high quality decisions under uncertainty creates massive competitive advantage. The more limited information and uncertainty the greater the advantage for the professional data scientist.

Acquiring awesome data and applying probability theory is optimal strategy when making decisions under uncertainty. Over time, high quality decisions will provide consistent probability advantages and mitigating risks (reducing bad decisions) will lower probability of blowing up.

Kinetic vs. Non-kinetic War

The nature of war has changed. Traditional kinetic warfare is still operational yet evolving from massive air carriers and big tanks with large ground forces to smaller special forces relying on near real-time intelligence to achieve specific goals within macro strategy.

Data Scientists Sometimes Fool Themselves

The easiest person in the world to fool is yourself. Data scientists sometimes fool themselves - in matters trivial and important. Thus, I suggest we acknowledge real or subconscious biases in ourselves, the data, the analysis and group think. It is prudent for data science teams to have both internal and external checks and balances to expose potential biases and better understand objective reality. Here are a few ways data scientists sometimes fool themselves:

Confirmation bias: tendency to favor data that confirms beliefs or hypotheses.