Dunning-Kruger Effect: A Blind Spot Every Product Manager Should Watch Out For ...
In 2009, My team and I were working on a prototype assuming that one day TVs will be connected to the internet and when that happens, advertising will become programmatic similar to what was online display advertising on the internet in those days. but more than that, people will also have second screen devices and the content they’ll be watching on the TV will be influenced with their second screen devices activity. AWS cloud was catching up steam and getting popular and we were working on how to get STB software on the cloud and also, build a developer platform for TV apps just like android play store or apple app store.
During the project discussions, one of my team members started a pitch to management with an underlying assumption that it’ll not be TVs that’ll be connected but Set Top boxes will be connected to internet. While his experience if STB domain was limited, he had a rich experience of working with one of the largest TV OEMs in the world. He was quite confident that TV manufacturers will always depend on STB or similar devices for any content consumption. Also, this was due to the fact that the company we worked for was one of the largest STB software / Pay TV company in the world. He was very confident about his idea as it would have resonated well with the management too given that everyone wanted STB market to grow leaps and bounds. at that time, Connected TVs was an idea that could have killed the STB industry in itself. Netflix was a small player and was contained to desktops/mobile devices.
To build the pitch, he gathered all data points from his previous experience and research and along with STB industry reports, he started working on the case. During a few discussions over a month, couple of us asked him to think about STB market and the risks associated with it. Technically, enabling an STB with internet seemed a lot more challenging that enabling a TV with the internet. And applying the Porter’s five forces, if TV manufacturers were ready to invest in HD and 4K televisions, adding internet capability will be a peanut cost to a television compared to an STB and a threat of substitute was imminent too. As a manager, I still wanted to make sure that we had all the options open on the table. but my team member was adamant on his proposal.
A few weeks back, during recent discussion online with a group of product managers about blind spots that a product manager can face, I got reminded of this past episode. and that made me talk about Dunning-Kruger Effect to those product managers.
 |
|---|
Dunning-Kruger Effect
For those who do not know what Dunning Kruger effect is :
The Dunning–Kruger effect is a cognitive bias whereby people with low ability, expertise, or experience regarding a certain type of a task or area of knowledge tend to overestimate their ability or knowledge.
In their research paper published in 1999, David Dunning and Justin Kruger implied that people who have limited experience are more confident about their knowledge compared to the people who are vastly experienced. This high confidence with limited knowledge stems from the ignorance they have about their own knowledge levels or capabilities resulting in them thinking that they know a lot more than others and hence, falsely makes them believe that they know it all. In reality, people with real experience and knowledge tend to have a room for self-doubt on their knowledge levels thinking that others in the same domain will also have similar knowledge levels and hence, there may be things to learn and possibly, there are more unknowns to discover.
In context of product management, this is highly relevant. Product managers deal will high level of ambiguity on a regular basis and are responsible for influencing decisions that can affect the future of organizations. Being a high visible and high stakes role of product manager, it is all the more required to be careful to evaluate all aspects of the discussion, being open to adverse feedback, listen to completely tangential viewpoints and then, try to present the probabilities of the various options rather than presenting one’s own line of thought as a fact.
 |
|---|
It is well know saying “A players hire A players, B players hire C Players” . What it means is people who really understand their own capabilities want to work with similar high calibre people but people with mediocre capabilities want to hire less capable folks so that they can be confident and manage superiority over others. In most of the cases, B players are always have high confidence levels with quite less intellectual humility and it is visible even during interviews when they are asking questions to prove that they knew more than the candidate. This is nothing but a representation of Dunning-Kruger effect where low calibre people overestimate their ability and end up rejecting high calibre people because such smart people had different opinions or ideas that did not match the expectations.
Intellectual Humility
 |
|---|
To achieve this, intellectual humility is highly important for a product manager. But people who face Dunning-Kruger Effect also lack in intellectual humility. Intellectual humility can be defined as the importance of knowing that one can be wrong. As a product manager, we learn management methodologies like SWOT ( Strength, Weakness, Opportunities, Threats), Porter’s five forces and many other theories either in business schools or by experience. but to actually apply these theories or methodologies, intellectual humility is a lot more important. Being humble helps us listen to the opportunities and challenges that others might know about. but if a person is facing Dunning-Kruger effect, it also makes him arrogant at the same time. it makes the person thinks that he knows everything but then the world is not listening to him and makes him go into a downward spiral where his or her team player abilities get affected without him realising about it.
Many of you might have heard of Amazon’s 14 leadership principles and how they apply these principles in their interviewing process. One of the amazon’s 14 leadership principle is “Are right .. a lot”. It is defined as “Leaders are right a lot. They have strong judgement and good instincts. They seek diverse perspectives and work to disconfirm their beliefs."
This is a great principle for a product manager. To execute any ambigious open ended project, product managers need to be right, a lot. But that come with a caveat of intellectual humility. That’s what it means in the second part of that definition when it asks to seek diverse perspectives and work to disconfirm their beliefs. But for a person with Dunning-Kruger effect, that’s where confirmation bias steps in.
People with Dunning-Kruger Effect seek out diverse opinions or perspectives to depict themselves as a team player but in reality, it is to prove their point so that they can claim they are right, a lot. Confirmation bias drives people to identify only those opportunities or data points that help them to confirm their beliefs. To persevere with one’s beliefs, the person can even start interpreting the data that goes against the belief into a favourable condition or completely ignore that point. Confirmation bias makes people selective in their choice of data points. Sometimes, it makes them selective to an extent that they choose to work with people (or invite only those people in meetings) who will tend to agree with them or have similar opinion and ignore the rest who may have a different opinion.
Such situations are a hotbed for conflicts. This is a perfect situation for a PM behavioural interview question “tell me an example when you didn’t agree with one of your team mates and how you arrived at a decision”. The answer to this question will reveal certain traits of a person on dealing with conflicting situations but more than that, it may also reveal about how a person thinks about himself, whether he is highly confident about his own views but still considers others opinions seriously or not. Even if people practice to answer this question several times, the way they’ll answer will still give away a lot.
How to avoid such conflicts?
The idea of any discussion is to focus on the problem and not on the people. There are always exceptions to this rule where some discussions are done only to escalate a conflict (pun-intended towards the politicians across the world)
For a product manager, admitting mistakes or acknowledging the knowledge gaps help in reducing the potential errors and sets us up for success. Also, it helps in building trust about our capabilities in the teams we work with. As a PM, we should consider all the estimations and data published as probabilities and not facts. When some data or statement is considered a probability, our brain makes us think about its chances of failure, in turn, making us anxious. Anxiety forces us to think a lot more on edge case scenarios or “pit-falls” that can be avoided. This anxiety also makes us consult not only people who are subject matter experts but also others who can guide us out of this ambiguous problem statement helping us opening up our mind to broader set of possibilities. Thus, we become more upfront in outlining challenges and threats in decision making. That makes us a calculated risk taker while being proactive and optimist at the same time.
Intellectual humility is the path that every product manager should follow to be aware of one’s own knowledge gaps. Once we are aware of our own knowledge gaps, it makes us honest in our conversations and helps saying “I don’t know” a lot more than ever. Every time a product manager says “I don’t know”, it helps to go back, study, learn a lot more about the topic to gain knowledge and to become a real expert. Any expert we meet today will always add an element of doubt in their statements even if they are almost 100% sure about them being right. That element of doubt stems from their awareness and knowledge of probability of almost everything in the world can never be 100%. There is always a chance of failure and that chance makes them intellectually honest and humble.
Epilogue
The incident I mentioned in the beginning happened in 2009. As with all the jobs, people move on. I moved on in 2012 to become an entrepreneur. My team member moved on to join another company. That incident just became one of those vague work memories that don’t make any sense in hindsight. In 2013, Netflix launched the series “House of Cards” changing the world of content production and TV viewing. We are in 2022 now. While Set Top Boxes still exist, Connected TVs have taken over the world. Even the biggest of satellite TV providers moved to online streaming. The advertising models have changed due to CTV ( acronym for connected TV) being a dominant force. Not only the TVs became connected, the data became the new oil that feeds into advertising. What we called second screen devices then is just a context with respect to TV but essentially, personal devices are the ones that drove the change.