As companies expand into new markets, they have to scale their engineering orgs to support increased demand. However, the more developers you hire, the more managers you need to help teams communicate.
This rapid hiring pace is unsustainable, which is why companies like Google focus on a more cost-effective solution: making each developer more productive.
Ciera Jaspan and Collin Green lead the Engineering Productivity Research Team at Google. The team has pioneered a data-driven approach to understanding and unlocking developer productivity, which Jaspan and Green recently discussed on Abi Noda's Engineering Enablement Podcast.
Based on the team's research, Google has established a method for measuring developer productivity that:
Balances short and long-term productivity
Sets goals, signals, and metrics to gather meaningful data
Keeps data collection as simple as possible
Let's explore how Google defines and measures developer productivity.
5 Components of Developer Productivity
According to Google's Engineering Productivity Research Team, developer productivity can't be reduced to a single number. The team takes a holistic view of productivity, incorporating product-focused outcomes as well as facets of the developer experience.
The resulting definition, QUANTS, breaks developer productivity into five components:
Quality of the Code: Do test cases prevent regressions? To what extent can an architecture mitigate risk and changes?
Attention from Engineers: How often do developers reach a flow state? Are they distracted by notifications or encouraged to context switch?
Intellectual Complexity: How much cognitive effort does this task require? Are developers dealing with unnecessary complexity?
Tempo and Velocity: How quickly can developers complete tasks? How many tasks can they complete within a set timeframe?
Satisfaction: How satisfied are developers with their tools, work, and end product? Are developers experiencing burnout?
QUANTS recognizes that developer productivity is influenced by a complex interplay of factors that extend beyond simple code output. For example, rapid efficiency gains might accelerate developer productivity in the short term — but they can also lead to technical debt and developer burnout over time.
By considering elements such as well-being and work satisfaction, Google aims to create a more nuanced and comprehensive picture of what truly drives productivity. As a result, engineering leaders can work towards building a more effective and sustainable development process.
Setting Goals
Google resists defining a blanket set of metrics for their engineering org. Instead, they emphasize capturing meaningful data that aligns with team and project-specific goals.
To ensure data is useful for its context, Google guides engineering leaders in setting high-level goals that cover as many QUANTS components as possible. By defining goals with multiple components in mind, developer teams can avoid inadvertently improving one component while diminishing another.
In her chapter from the book Software Engineering at Google, Ciera Jaspan provides sample goals from Google's readability team:
QUANTS | Goal |
Quality of the code | Engineers write higher-quality code as a result of the readability process; they write more consistent code as a result of the readability process; and they contribute to a culture of code health as a result of the readability process. |
Attention from engineers | We did not have any attention goal for readability. This is OK! Not all questions about engineering productivity involve trade-offs in all five areas. |
Intellectual complexity | Engineers learn about the Google codebase and best coding practices as a result of the readability process, and they receive mentoring during the readability process. |
Tempo and velocity | Engineers complete work tasks faster and more efficiently as a result of the readability process. |
Satisfaction | Engineers see the benefit of the readability process and have positive feelings about participating in it. |
Without clear goals, teams risk prioritizing metrics that seem impressive or important, but don't align with the project's true objectives. The ripple effect of misaligned metrics can be profound:
Decreased motivation among developers who feel their work is being reduced to mere numbers without context or purpose.
Haphazard resource allocation that can increase technical debt and undermine project quality.
Mismatched expectations with stakeholders, who might lose trust in a team that prioritizes the "wrong" indicators of success.
Defining Signals and Metrics
Once developer teams set high-level goals, the next step is to define corresponding signals and metrics.
Signals: How we know we've achieved our goal. These indicators may or may not be measurable.
Metrics: Concrete, quantifiable measurements that capture signals.
Source: "Measuring Productivity" by Ciera Jaspan
Signals and metrics help teams move from abstract goals to actionable steps. However, not every goal needs a "formal" quantitative or qualitative metric to enable progress. In fact, overcomplicating data collection can slow teams down.
Google encourages leaders to employ the simplest metric that will provide valuable insight:
A team leader will come to us and say, “We’re really trying to buy down the technical debt in our code base. Can we run your technical debt survey to understand whether we’ve done that?” I’m like, “How big is your team?” They’re like, “It’s 12 engineers.” It’s like, “Why don’t you guys just have a meeting and you can talk about where you still see hindrances? Your engineers are experts at this. They’ve actually waded around in your technical data and your code base. They don’t need to take a survey to understand that.” —
Takeaways for Your Dev Team
As an industry leader in developer productivity research, Google has nearly 2,000 engineers dedicated to optimizing developer tools and processes. The Engineering Productivity Research Team also works directly with developer teams to define goals, signals, and metrics that generate actionable data.
Few companies have the resources to implement such robust methods for measuring and enhancing developer productivity. However, your team can adapt these key insights from Google's approach to create a method that suits your context.
Establish a nuanced definition of developer productivity that captures output (e.g., code quality and velocity) and developer experience (e.g., cognitive load, job satisfaction).
Set high-level goals that cover multiple components of developer productivity.
Define corresponding signals and metrics to help you measure progress towards goals.
Avoid overcomplicating data collection. Sometimes, talking with your team can give you as much insight into productivity blockers as surveys or logs data.
For a deeper dive into Google's research on developer productivity, check out Ciera Jaspan's keynote presentation at the 2022 International Conference on Program Comprehension.
Free Resources
