We've all found ourselves lost in the sea of different internal process and planning tools. I seem to find myself there more often than most, mainly because I enjoy optimizing the Sprint planning meetings that we use at my company. I've generally seen that finding the right mix of innovation and rigidity can be a bit difficult. That's why I have spent the last 6 months attempting to find a methodology for my team and me to use when planning out and grooming the extensive backlog of tasks that we find ourselves with.
With such a massive backlog, as well as varying opinions of what to do next, we've struggled to find the right mix of features for our last few Sprints. I've tried a variety of tools and plans that I've found online, but many have felt like trying to roll a boulder up a hill. They are entirely too much work for the amount of progress that we make. We either get too far into the weeds of the 'how' rather than the 'why', or we just dismiss items subjectively based on what we think a customer might want.
What I knew that I needed to find was a tool that would force us to prioritize features on merit and workload alone. It needed to also foster discussions at a very high, business-oriented level. With those requirements in mind, I recently settled on utilizing the Product Priority Matrix. It provides a great means for determining, at a basic level, what we might seek to achieve over the next 3 months at our company. While it hasn't been perfect, and our team has still fallen into old traps, it has been a pivotal tool for prioritizing Sprint candidates. In this post, I'll outline what I've learned so far, as well as why you might want to utilize the Product Priority Matrix during your next Sprint Planning meeting.
What is a Product Priority Matrix?
For those unfamiliar with the Product Priority Matrix, it is a simple Punnett square that is used to prioritize items based on two metrics, effort and impact. What this allows you to do is to boil down any feature discussion to the most basic of business building blocks, time and money. Will the feature make a substantial impact on our bottom line? Great, now lets talk about how long it will take to recoup the cost of building it. Every decision for this matrix is something that points directly to how much a feature will affect your revenue, and how long that feature will take to pay for itself. While not perfect, the Product Priority Matrix allows you to focus discussions about what really matters in a feature release.
If a feature is a quick win that has a measurable impact on customers and their engagement/spend, then it goes in the top left quadrant of the matrix. These are the items that are prime candidates for the next Sprint. Anything that can take minimal effort and produce great results is a major success in any business person's book. However, the trap that is easy to fall into within this quadrant is over-estimating the impact of your features. If your team has a track record of incorrectly predicting the impact of features, then I'd highly recommend basing your decisions on collected data. Figure out a metric that really matters to your business (monthly recurring revenue, spend per client, daily active users, etc) and track that metric in relation to every feature release. As you start to see the trends of your decisions, it will allow your team to more accurately reflect on the supposed impact of your high priority features. Sometimes you have to face up to your poor rationalization for a decision, but it will help you make better choices in the long run.
As for the top right quadrant of the matrix, these are the sorts of things that will take longer to pay for themselves. That's not to say that they are not important, but they do typically take a significantly higher level of precision and effort. If you take on too many of these sorts of projects at once, timelines can slip and scope can expand exponentially. You also want to be careful with these features to correctly outline exactly what customers want/need. I am a big proponent of the Build, Measure, Learn feedback loop of Lean Startup, and this is the perfect area for this methodology. Whether you are looking at a 3 or 30 month project, it's important to understand your customer requirements and build iteratively. Using a Minimum Viable Product mindset for these sorts of decisions can make a huge impact, so take the time to plan out a strategy for features in this area of the matrix.
As for the bottom half of the matrix, this is where things tend to get a bit more difficult to justify when it comes to having a place in a Sprint. If you find a pet feature in the bottom left of your chart, then you need to re-evaluate exactly why it needs to be built. That can be either through expanding the scope of the feature to have more impact, or bundling it together with other customer requirements or use cases in order to create a better feature. Most items in this area are presented as a priority in Sprint Planning meetings because of the overreaction of one or more team members. Don't fall into the trap of exaggerating customer feedback when looking at these features. Be realistic about their importance and freely drop anything that doesn't past muster.
Finally, we have the bottom right quadrant of our matrix. This is where features go to die. Anything that is a massive undertaking and has little to no impact on customer happiness or ROI is a giant timesuck. Many companies waste their time on these features once or twice a year, and they quickly lead to bloated software and unmet expectations. Spend time on every other quadrant and simply ignore anything that finds itself accurately placed in this area of the Product Priority Matrix.
How I've used the Product Priority Matrix
We have used this matrix several times over the last few months in Sprint Planning meetings. It serves as a perfect focal point for our meetings and helps to center discussions about mutually agreed priorities as a company. While we have definitely found success with this tool, I've still felt like there are things that I could do better the next time I create one of these.
First, the overall successes
Using this matrix allowed us to take a large list of highly requested features, which we had mulled over for 6+ months, and prioritize them efficiently. We ended up with a good idea of what we wanted to tackle in upcoming Sprints, and we effectively pulled out a small number of tasks for our Q4 release. As you can see in the example above, we broke our features down into each section of the matrix after having a 3-5 minute discussion about effort and impact. At the end of the process, we really only had one item that fell extremely far below the fold for consideration (which we knew it would).
What the Product Priority Matrix also allowed me to do with the team is to refocus our success/failure metric for each Sprint onto something that related to ROI. This means that we now have a single common goal for all of our product considerations, and it gave me the ability to frame all future product discussions about impact on these metrics. We also moved another step closer to using a customer-first focus for all of our product decisions. When we came to a major unknown or disagreement on one of these items, it was simple enough for me to say "Well, let's just ask some customers about that and find out". This allowed us to rely on our primary stakeholders to direct us between two equally important features.
Now for the failures
One thing that I figured out pretty quickly is that we had entirely too many items that were far to the left and high up in our matrix. Could we really be sitting on that many simple, high impact projects? Probably not. So, over the last few months, we have continued discussions about these features in order to determine just how much impact and effort each one might entail. Our track record has been steadily improving over the last year when it comes to accurately predicting impact and ROI, so this is still a journey. However, I'm confident that we will continue to re-evaluate each of these items and more accurately place each on our matrix.
Another key learning from this process has been just how inaccurate effort estimates can be with certain larger projects. We initially pulled out a single Post-It note off the wall relating to a high effort, high impact candidate for release in late 2018. What progressed over the following few months was an arduous journey that I will need to fully document later in a more detailed post. Suffice to say, what was initially projected as the scope of the project was not what we ended up with. We quickly ballooned from a 2-3 month project to a 5-6 month project, which directly impacted all of our other candidates for that Sprint and release. Had we stuck to our guns, we would most definitely have lowered the impact of our feature by a bit, but we would have also lowered the effort significantly. This is the quintessential version of 'scope creep', and it just goes to show that the Product Priority Matrix is not immune to these sorts of issues.
The final learning from this process was that this matrix needs to be a living document that is never truly finalized. Whether it is re-evaluating impact/effort, or adding new feature requests, you shouldn't stop analyzing the Product Priority Matrix. There will always be way too many items in your product backlog, and each will be vying for a spot in an upcoming Sprint. While this matrix does a great job of representing the feature development needs of the moment, without constant grooming, you'll end up with something that is extremely outdated and misleading. I recommend revisiting your Priority Matrix after every Sprint to discuss missing features and each task on the board in order to ensure that you have correctly captured your most important items. You don't necessarily have to restart your entire Priority Matrix after each Sprint if you have items left, but don't take for granted the decisions and thoughts of a prior, less informed you.
In summary, we still have quite a bit of progress left to make when it comes to using the Product Priority Matrix for our Sprint Planning meetings. We continue to have setbacks that make these sorts of decisions extremely difficult. Whether it is neglecting to accurately represent the scope of a project, or naïvely assuming that we will make millions on a single feature release, we still have room to grow. However, I do think that the Product Priority Matrix offers a huge benefit to our team when it comes to analyzing and representing our product backlog. We will definitely continue to utilize this tool to determine ROI and effort, and with outside data analysis between our Development leads and myself, I believe we can continue to improve this process.