It goes without saying that being data-centric is one of the most highly valued skills for Product Managers today. You already know why making data-driven decisions is important, but why is it such a difficult skill to master?
Firstly, it’s because the majority of Product Managers today do not have any formal training in data science/analysis and yet are expected to know the data behind their products inside out. In fact, studies show that up to 62% of Product Managers feel that they do not have the adequate data training to perform their roles1. They are mostly self-taught, which leads to low confidence in gathering & interpreting data.
An added complication is the sheer volume and availability of data today. In the digital world, data is literally everywhere and it can become super overwhelming to try and decipher which data is important and which metrics are simply ‘vanity metrics’. On the flip side, in fast paced tech startups, not enough time is usually dedicated to ensuring that tracking mechanisms are built into every new feature, so there are often a horde of features that exist, whose usage is either not being tracked at all or is being tracked inaccurately. This creates the contradiction of at once having too much data at one’s fingertips, but also a lack of the right type of data.
To further compound the problem, a lot of tech companies don’t have Data Analysts or Data teams, especially startups. It then invariably falls on the Product Manager’s shoulders (since, if it’s nobody else’s job, it’s your job!) to not only make sense of the mess that’s in place, but then to also somehow glean actionable insights from that data. At my first Product role, our company was doubling in size every year, but it still took us 4 years to hire our first Data Analyst and till today, I shudder at the thought of the chaos that was compounding everyday that we didn’t get our data in order.
However, overcoming such challenges is part and parcel of the Product Manager role. At the end of the day, when employers ask for Product Managers to be ‘data driven’, what they really want to do is to minimize risk. Since nobody can predict the future, data is the only way for us to form educated guesses about what future impact our work can deliver, thereby improving our chances for success. With that in mind, how can Product Managers thrive amongst the chaos and harness data to build long term Product success?
Below are 5 strategies to become the ‘data-driven’ Product Manager that your company needs:
1. Familiarize yourself with your company’s data infrastructure:
The first step is to get an understanding of what type of data is available to you and where it is stored. Below are the different kinds of data your company is likely to be collecting:
| Data Type | Example(s) | Source(s) |
| User Behavior | Views, visits | Google Analytics, Adjust |
| Performance | Revenue, retention, churn rate | Power BI, Tableau |
| AB Testing & Experimentation | Results from experiments | Sitecore, Optimizely |
| Customer Feedback | Reviews, surveys, support tickets, social media comments | Trustpilot, Zendesk |
| Market Research | Industry reports, market trends, user demographics, audience segmentation | Forrester, Gartner |
| Technical Performance | Crash rate, uptime, error rates | New Relic, Sentry |
| Team | Velocity, sprint burndown | Jira,Basecamp |
Once you understand what type of data your company is collecting and where to find it, your next step is to
2. Have a clear idea of what information you are looking for:
The worst idea is to go into each and every data source and start looking through everything that’s available. That is the best way to get lost and overwhelmed. Instead, be intentional with what question you are trying to answer and only then, go into your data source.
There are two types of data analysis or monitoring you will likely perform in your Product Management role. The first is to keep abreast of top line numbers, that indicate the general health of your business. What these are depends on your type of business, but in general they are metrics like revenue, conversion rate, average order value etc. These are numbers that you expect to be improving steadily over time, but are not particularly volatile. You need to know these numbers like the back of your hand, and the best way to do this is to establish a routine around monitoring them, whether that’s weekly, monthly or quarterly. Ideally, these numbers should be in a dashboard that is updated automatically, with notifications set up to alert you if a number falls below a specific threshold, which would indicate that something big has gone wrong. This sounds simple enough, however a lot of Product Managers get caught up in the day-to-day delivery of features or updates and can forget to keep an eye on the bigger picture.
The other type of data analysis you will do relates to trying to answer specific questions or solving specific problems. Be specific with your intention before entering into the data source, ie “I want to understand how our checkout improvements have impacted our shopping cart abandonment rate” is a better question than “I want to know how our checkout funnel is performing”.
3. Up-skill strategically:
Once you understand the types of data available to you, and the general ‘data culture’ in your company (ie what data related skill sets are present in your team? Is there a dedicated data analyst, or does all of this fall to you? etc), the next step is to understand how you can level-up your data skills in a way that brings the most value to your company.
For example, in the role I mentioned above where I worked as a Product Manager at a startup without a Data Analyst, our data visualization tools were disorganized and often inaccurate. I therefore had to rely on asking our developers to run SQL queries directly onto our database in order to answer the questions I had. However, after taking a basic SQL course, I was able to learn how to run simple and complex queries myself, therefore saving lots of time for our Development team. These skills later helped me in another role, where we were setting up a new product from scratch and the database schema had to be designed. Given what I had learned about database schemas in the SQL course I had taken, I was very well prepared to hold a technical conversation with our Development team on how our database should be structured and how the various data points should relate to each other.
In another role, where the company had a strong A/B testing and experimentation culture, I chose to brush up on my knowledge of statistics. Understanding key concepts like statistical significance, confidence levels, margins for error etc helped me to run AB tests independently and confidently.
Data Science is a field of its own and as a Product Manager, you should not be aiming to become an expert, however, you can choose to learn the data skills that are complementary to those of your teammates and therefore make the most sense for you and your company.
4. Blend qualitative data with quantitative
The best Product Managers I know have a knack for blending qualitative data with quantitative in order to drive impact. As Product Managers, it’s easy to get caught up in the numbers trap, believing that only quantitative data is valuable, however, do not underestimate how much you can learn by simply speaking to a customer in person.
I’ve often found that numbers are a great way to spot patterns and trends, however qualitative data (that which you can get from surveys, or speaking to customers) can help you understand the ‘why’ behind those patterns. At my last role, I set aside a week each quarter to speak to customers directly, and that was often the week in which I was able to derive the most fascinating and impactful insights, some that corroborated and explained trends I had seen in our quantitative data, and some that made me aware of problems and/or opportunities that I had never thought of. Equally, open-ended questions in surveys and feedback forms can sometimes provide much deeper insights than your typical Analytics dashboard. Make sure to take advantage of the countless AI tools on the market like Monkeylearn that can help you sort through large sets of qualitative data.
Being data-driven is not just about being a whiz with the numbers, it’s about using every data point available to you, whether those are conversion metrics or conversations with customers, in order to drive product success.
5. Play your part in Data Governance
Data Governance is the act of managing the availability, usability, integrity and security of data. While you as a Product Manager are not expected to be an expert on matters like data security, you can play a part in making sure that data is ‘available’ and ‘usable’ to yourself and your team members.
You can do this by ensuring that every new feature is launched with tracking capabilities built-in. That is, you must establish what the success metric of your new feature or product is and make sure that it is measurable once you launch. Launching a new feature but not being able to measure its usage is a rookie Product Management mistake. That said, you should also ensure that the tracking mechanism you are building is tested thoroughly by your QA team to make sure that it is accurate. Collaborate with your Data Analytics team and/or other stakeholders to ensure that the data is ‘usable’ and visualized in a way that makes sense to them.
Conclusion:
Being data-driven is a core requirement of all Product Management positions, however it’s also one of the hardest parts of the job. However, by familiarizing themselves with the company’s data infrastructure, being clear about what information they’re searching for, up-skilling strategically, blending qualitative data with quantitative and playing an active role in Data Governance, Product Manages can harness data in a way that helps them to build long-term product success.
What do you think are other practical steps that Product Managers can take to become more ‘data-driven’? Let me know in the comments below!
Leave a Reply