Data driven decision making. Everyone talks about it, knows its critical and says they're doing it. But when push comes to shove, and decisions need to be made, as many would have experienced, it's far from straight forward.
What do you do when competing ideas are each backed by their own facts? What if the subject matter expert disagrees with what customers have told us (survey)? Without a way to compare and weigh data against each other, productive discussions can quickly become deadlocked and counter productive very quickly.
There is a general framework regarding decision making and data that I have found useful, which I call my 'Decision Data hierarchy'.
One point of clarification : I use the term "data" to describe all information set forth in the discussion to support a point of view. For example, if person A makes the following statement "I support idea X because of Y", the data to be used to weigh idea X is both 'CTO' and 'Y'.
General principle : Not all data (A & Y) is created equal
2 principles when weigh competing opinions backed by their own Y....
(1) the 'harder' data wins.
(2) the market always wins
Harder data wins
Data can be categorized into 3 layers described by "hardness". And the harder data wins.
Hard data : data that can be described in specific accurate detail and would be considered fact.
(this experiment show) doing C increased response rate by 50% more then D.
Soft data : data that is based on experience. This too is based on specific accurate details but it may include biases.
(in my x number years of experience) D will increase response rate more then C.
Fuzzy data : data that is based on logic/common sense. Common sense is a powerful tool, but so many things in startup world are counter intuitive and unknown. You can't logically think through the unknown. (eg. Aristotle, whose writings had remained unquestioned for over a 1,000 years up until Galileo's time, not only did heavier objects fall faster than lighter ones, but an object that weighed twice as much as another would fall twice as fast. - PBS.org)
(logically) C is the better choice then D since [insert seemingly found logic]
Invisible data : This is unexplainable intuition. Though not zero in value and weight, its marginally better then zero. Option C feels like a better option D, but I can't explain it
Application of Data hardness
3 people (were) are split between option C and option D with the goal of increasing our marketing response rate. Our rock star summer intern is in favor of C because she ran an online experiment and found that C had a 50% higher response rate compared to D. Our CMO supports D because it has worked better the C during his 10 year experience. I (CEO) also support D because, logically speaking D seems to be the more logical explanation.
winner : option C - Rock stare summer intern
the Market always wins
Market - SME (Subject Matter Expert) - everyone else.
When there are competing ideas, the person closer to the problem wins. But the market ALWAYS wins. For example, if I'm in favor of doing a deep into technical framework E but our CTO opposes doing so, our CTO wins.
I (CEO) think we should deep dive into framework E because I heard a lot of chatter about it
I (CTO) think its a waste of time because what we have works perfectly fine.
winner : no deep dive - CTO
but if I change the data point to include the market such as
I (CEO) think we should deep dive into framework E because it is being adopted by over 50% of our comparable startups.
winner : deep dive - Market
Like all principles, it is never absolute and/or clear cut . What happens of a SME brings invisible data (intuition) and someone else brings fuzzy data (logic) or any other combination for that matter? That will be a topic for another post.