OKRs: The Keys to Effective Development
As a technical team, you have to ensure your work is valuable to your customers. All of the work you do must provide value for your end-users. Some of the projects you will undertake have long-term benefits for the users; development process improvements, spikes to increase confidence in the system, technical debt maintenance, and these are just a few of the projects that are up for debate during every sprint.
How can you measure the effectiveness of these projects? Given you are continually evaluating ways to improve, how should your metrics evolve over time?
Throughout this post, I’d like to elaborate on these questions, how my team has evolved over time, and how we have continually improved not only our team’s process but also the metrics to evaluate continual improvement.
As mentioned by Cristos Goodrow, Vice President of Engineering at Youtube, “Engineers struggle with goal-setting in two big ways. They hate crossing off anything they think is a good idea, and they habitually underestimate how long it takes to get things done.” This rings true to almost every developer I’ve ever worked with, including myself. One of the techniques for measuring and setting goals where I’ve seen success is in what’s typically abbreviated to “OKR”s, or Objective and Key Results. This methodology was originally introduced by Intel in the 1970s, and this metering strategy links very well with continuous improvement through Agile development. In my experience, it is important to enforce alignment in OKRs from various perspectives. Therefore, many of the types of Objectives that will be measured on both team and individual levels will need to correlate with Objectives from upper management.
To briefly cover OKRs, this is a framework for setting constant Objectives and matching those Objectives with key results that will be achieved over a period of time. These objectives may or may not differ from team-to-team, but the key results will vary from team-to-team. To see some examples, check out this fantastic list!
The reason I absolutely love OKRs is that it does one thing extremely well: constant assessment of your current situation. Continuous improvement is one of the underlying pieces that drives me in different directions. If you don’t know how to improve, how can you ever become the best? How do you get better, more reliable, and more resilient? You continuously ask yourself: “How can I be better?”. Your life, your country, your world is constantly changing. How do you keep up with your goals if you only assess yourself based on prior contexts? Think about this...
Looking back to one year ago, we were in a world that was riddled with deciding whether or not you are still a fan of Game of Thrones after they released what could have been the worst ending to such a culturally significant series in history. During this same time period, I set myself an objective: Get better. Learn differently. From this, I decided to take a leap from a company that was absolutely incredible from a support system perspective, into a team where I was the fourth person on a team with hyper scaling implications. Fast forward one year forward to the beginning of a new decade, where massive culturally significant pieces have altered not only the way we live on a day-to-day basis but also the way we work on a day-to-day basis. How do you navigate the professional context of mandated Stay-at-Home orders and continue to get better?
Your objectives and key results will change from year to year, month to month, and even day today. You have to constantly ask yourself how you can accomplish where you want to be, and whether or not your short-term objectives still align with your long-term objectives. Throughout this post, I want to dive into different pros and cons to consider when outlining objectives to set for a period of time.
Let’s bring ourselves back to a less politically driven topic, and discuss how we can look at setting our professional objectives. Consider the time period after upper management provides your team with a set of company-wide objectives, you need to focus on their higher-level guidelines to suit both your individual team and yourself. How do you come up with these? What are the costs and benefits you should weigh when considering different objectives for your team to conquer? I’m going to use some of the benefits I’ve witnessed outlined in proposed projects, which are common characteristics throughout the four separate teams I’ve taken part in.
Confidence in the System
Technical Debt Maintenance
As a development team, it’s very typical to have stories or tasks that fall into any of the five categories above. The question you should be thinking about as you look at these, and this will need to remain in alignment with the product team: “How do these buckets align with the higher-level objectives, and which ones should you follow?” On my teams, we typically have a general allocation for distinct categories. For instance, if we are preparing for a normal quarter, which is the typical amount of time we will perform our OKRs, we will expect to have roughly 60% of our time go toward new features, 20% of our time to technical debt, and 20% of our time toward bug-fixes. This is obviously a condensed version of the five categories above, but I am sure you can see the relation is pretty large between a few of those categories. BUT! Before we decide, we have to understand the costs of setting our goals as well. The typical costs I’ve seen fall into five distinct categories.
One of the hardest pieces for management is deciding the best way to meter your team. There are so many variables while developing, and it’s hard to pinpoint which ones lead to the successful completion of your objectives. Metering your work is important for everyone. It’s the best way for anyone to support their argument. What are the costs to hit our objectives? With the old adage, “you are what you measure”, how can I ensure we are measuring the right pieces? Should we alter our key results or our objectives? These are the questions you should ask yourself constantly. In any given situation, there will always be costs of performing a task.
To give you an idea around the scope of the Objectives I’ve had with OKRs, and how they relate to the benefits and costs listed above, I am going to get personal and share one of the Objectives I set in tandem with my boss. First, we had an issue with Quality going out of the door given that we were in the start-up phase, corners had to be cut, and that will ultimately result in bugs and/or performance issues. So, the very first Objective that I was given:
Objective: Increase Customer Reliability
This seems like a pretty standard objective from a high level, but what did it mean to me? How do you even begin to measure something like that? There were two main things we measured for our OKRs at this time. The very first step when breaking an Objective down is to understand how this metric relates to my team. As you can imagine, there are a ton of different pieces to this Objective, which is why I want to outline how important context is in defining key results. For us, we knew the underlying issue of Customer Reliability was two-fold: we were increasing our high and critical bugs and we didn’t have many tests in-place (again, corner-cutting was taken). First, let’s dive into our bugs. We have objective guidelines on differentiating bugs based on severity, and there are four levels: Low, Medium, High, and Critical. We discussed the significance of how many high and critical bugs were introduced, and we decided this would be the first metric we will focus on. So, without further ado:
Key Result One: Decrease the number of high and critical bugs presented to the system by 75%.
This was a pretty reasonable goal given we introduced 2-3 significant refactors (including setting up our entire system to migrate to angular) where each spanned roughly 20-30% of our codebase. Needless to say, we had accomplished most of the refactoring we would do in the future, so we believed we could plausibly reduce the critical bugs by 75%.
For the second part of this Objective, every single individual on the team wanted to increase the total number of tests in our system. We believe that introducing integration tests into our system had the most bang for the buck. So, our second key result revolves solely around introducing integration systems into our three separate applications.
Key Result Two: Increase our integration tests from 0 to 100
As you can see, we attempted to increase the total number of tests around our system to enhance both our confidence in the system and Customer Reliability. To this point, we have increased the total number of tests in our API from 0 to … 1500! Our integration tests also served as a way of ensuring our API security was enhanced based on our desire to create an Open API. As you can see, we have already hit our goal, and we should have shot a bit higher, but that’s okay! These key results are supposed to be obtainable, and some are going to be easier than others. Keep in mind that this was only 1 of many key results we needed to hit throughout a quarter. Also, our other two systems, still do not have as many tests as our API, and we are working on building these out.
There are a few things that are important to look at as I have outlined one objective and two key results here. First, the objective is very broad on purpose. The objective can span across weeks, months, quarters, individuals, teams, and just left up to the company. This objective should provide both alignment and direction for everyone inside of the company. Second, there may or may not be multiple key results for each objective. It is very likely that any objective set will involve many different areas of the workload from a company, team, or individual perspective. My last takeaway for this example is that all of our key results have a finite number, and they are obtainable. The goal is to set yourself up to achieve your objectives. At the end of the time period you associate with your OKRs, you should have made considerable progress towards obtaining your key results. For us, we continually analyze how we are doing, and we can adjust ourselves both during the time period and after the time period based on where we are toward hitting our key results or if any major global pandemics, I mean events, happen that would have an effect on hitting these goals. Remember: STAY AGILE!