DevOps Dojo: Lean Product – Part 2

In Part 1 of our “DevOps Dojo: Lean Product” blog, we discussed the need to build foundational concepts of the product-centric model. This enables us to explore more advanced concepts in the lean product-centric model.

I: Intro of Lean Product in Orange Belt

Companies have been building software products for decades in various industries, but that doesn’t mean they have been building digital products using a lean product-centric approach. In this blog, we will review the following topics in detail.

We will discuss the following topics:

1. Demonstrate waterfall product-centric model through a real example.

2. Describe the key components in the lean product model’s taxonomy.

3. Deep dive into Product Discovery of lean product mode and key roles.

4. Explain how we use product discovery to build lean products in Dojo.

5. Discuss how to scale lean product process with anti-patten examples.

II: Why Lean Product

Moving from a project-centric model to product-centric model doesn’t mean you are using a lean product-centric model. There are many software product companies that have been building products using a waterfall product-centric model as shown below.

This is an example of a waterfall product-centric model used by an industry software company. This approach is not unusual for many industry software providers.

Opportunity

• Business examines market data and other info to prioritize features.
• Engineering identifies technical and architectural opportunities.
• Engineering wants to increase velocity and deliver more value.
• Enterprise wants zero downtime to deploy and high availability.

Ideas

• Business ideas are mostly decided by business tower leaders.
• IT wants to break up a monolithic app to support speed to market and availability.
• The feasibility of the solution is determined by the architecture board.
• Architecture board is involved in the decisions on ideas/features.
• Hackathon once a year to generate ideas at enterprise level.

Roadmap

• Three-year strategy defined at the executive level.
• Business has Objective Key Results (OKRs), but these are not shared with IT.
• Government compliance functions always get prioritized first.
• Technical debts are deprioritized in favor of features and functions.
• Features come from the business side of product management.
• Prioritization for technology comes from the architecture review board.
• Architecture and business partner up to decide the features in roadmap.

Business case

• How much will it cost?
• How long does it take?
• Who will be in charge?
• What is the return on investment?
• What is the sense of urgency?
• Iteration from a market perspective and provide a high-level estimation.
• Engineering and architecture review board provide high-level estimates.
• Architecture review board is involved for business case review.

Requirements

• Business product team creates functional requirements in requirement tools.
• The global operation team creates non-functional requirements in spreadsheet.
• Both requirements require the architecture review board to review for approval.

Design/Build

• Application functions are built by a business product team.
• The application platform functions are built by the App Platform team.
• The environment is built by the infra team with a manual hand-over.
• There is no linkage between application/ops code with requirements.
• The functional testing is done by various teams depending on types of testing.
• Security testing is done by various teams depending on types of security testing.
• The infra testing is done by a global operational team in the QA stage of delivery.
• There is no CI and CD in the current process due to monolith applications. Deploy/Release
• Twice a year production deployment for major releases with hot fixes as needed.
• Large size of release into production creates huge pains for everyone involved.
• The release window can be more than 24 hours without access by users. User
• Feature usage is not measured so no way to determine how to disable unused features.
• No DevOps KPIs: lead time, deployment frequency, MTTRs, deployment failure rate.
• Any releases that impact users will use maintenance windows, which can take over 24 hours.
• Measurement is on output, not outcome. IT is not aware of business OKRs.

Consequences

• The company loses many talents and has a hard time keeping talents.
• The company’s customer renewal rate is decreasing year after year.
• The company’s revenue is decreasing year after year.
• The company is disrupted and loses opportunities in the market.

Problem Analysis

• Ideas are primarily coming internally so it can be based on a lot of false assumptions.
• Ideas are not involved with the engineering team which is normally the source of innovation.
• The assumptions are not validated, the dependency is built based on unvalidated assumptions
• There is no way to know what will change in three years, but a plan has been made.
• No one knows how much it will cost, no one knows how long it will take, no one knows the return on investment, many ideas are false, and the estimate is totally unknown.
• Even though teams use Agile ceremony, it is largely a waterfall process.
• Multiple teams are disconnected, and there is no traceability between requirements and code.
• The engineering team takes requirements to build and test without customer feedback.
• Organizational misalignment during the software delivery process creates huge waste in human capital.
• In the end, users perceived a long lead time, users perceived a lack of innovation, users perceived a lack of value, users are dissatisfied, and users move to competitors.

Conclusion

As you can see from this example, moving from project-centric model to product-centric doesn’t guarantee that you deliver great value if your model only handles product-based funding and persistent staffing. But do we have a better way? In the next section, we will introduce a lean product-centric model which removes more waste and creates higher value.

III: What is Lean Product

The lean product-centric model is the management of the complete life cycle of a product applying the principles and practices of Lean Startup, Lean, Customer-centricity, Business Modelling, Financial Viability, Outcome-driven Innovation, Behavioral Economics, and Strategy. Lean product is about establishing a culture that puts the user first and builds the organization and teams around that customer to ensure that you are building the best product possible.

In a product-centric model, product teams perform product vision and strategy, product delivery, and product adoption. However, Lean product-centric model goes beyond those steps. Let’s look at how lean product-centric model works.

Like the product-centric model, the opportunities can occur when a company starts a transformation journey, or builds a new line of business, or sees growing competition in the market, or senses the market technology disruption, or receives customer demands.

Ideas can come from customers, partners, product management team, UX team, engineering team, or internal hackathon events. In fact, the engineering team is normally a source of innovation.

There are two important concepts used in the lean product-centric model: Objectives and Key Results (OKRs) and dual track Agile approach with Product Discovery and Product delivery.

What is OKR

Objectives and Key Results (OKR) is a popular and proven technique for setting and communicating goals and results. OKR is designed to connect strategic goals set by leadership with the activities software development teams perform on a day-to-day basis. The OKR system helps the team identify the optimal result and creates clarity around what real success looks like.

Objectives are the goals

• Tell you where to go
• Set a clear direction and provide motivation
• Easy to understand
• Three to four ambitious goals the team is pursuing for up to the next year

Each Objective has a few Key Results

• Should have a number to score progress, i.e., be measurable
• Three to five measures of progress per objective that will measure success for this quarter or period
• In the end, you can look and without any arguments ask: Did I or did I not do that? Yes? No? Simple. No judgement in it. Three key elements in OKR
• A framework for defining clear objectives: It provides clarity on the intent and direction at all levels in the organization.
• Re-enforced with measurable key results: Key results are outcomes by which success is measured.
• Drives an outcome mindset culture: Clear shift from an output mindset to an outcome mindset.

Business Outcome vs. Product Outcome

As mentioned in Part 1, a business outcome measures how well the business is progressing. A product outcome measures how well the product is moving the business forward. Product team will make progress on a product outcome rather than a business outcome. A product outcome is within the product team’s span of control, but a business outcome often requires coordination across many business functions such as marketing and customer support aside from the product team. Product outcome can be measured by lagging metrics or leading metrics. The lagging metrics measure something after it has happened. The leading metrics predict the direction of the lagging metrics. Defining leading metrics is critical but not easy in any product organizations.

We only briefly introduce OKRs in this blog, but we will have a dedicated blog to talk about OKRs and KPIs in upcoming blogs. In this blog, we will only focus on product discovery in Lean Product model.

IV: Product Discovery

Why do we need product discovery? We all tend to fall in love with our own ideas. As humans, we are not capable of assessing how good our ideas are. This tendency is called confirmation bias, which is the tendency to search for, interpret, favor, and recall information in a way that affirms one’s prior beliefs or hypotheses.

We need a scientific method to test and measure how much value a product improvement would bring. We need value testing on our product ideas. If we want to discover great products, it really is essential that we get our ideas in front of real users and customers early and often.

Let’s understand what the principles of Product Discovery are. Fundamentally, we want to test if the features or the Minimal Viable Products (MVP) are valuable, usable, feasible and viable before we harden our product features to minimize waste of products. Sometimes, we call it sustainability in digital products.

Four questions we need to ask ourselves are:

Will the customer buy or use our product?

• Customers don’t know what’s possible
• Customers don’t know what they really want till they see it
• Most customers aren’t well-versed in technology
• Nor do they have time to dream up what’s possible
• It is product people’s job to read things and figure out what is valuable

Can the user figure out how to use our product?

• A good user experience is critical for product adoption
• The average human attention span is now shorter than a goldfish’s
• Functionality of the products can and will impact design
• Technology can impact design and functionality as well

Can we build the product with the available technology?

• Expect many ideas won’t work out due to technical limitations
• Should we wait or develop if the technology is not available
• Need to validate ideas as early and cheaply as possible

Does this solution work for our business?

• The opportunity might not support our strategic objectives
• The opportunity might be controversial, so you need more political capital
• Many of the ideas won’t work for the business, for example
• Due to financial, marketing, sales, legal, and other considerations

In lean product, the concern is split across four roles, each of which owns a different part of VALUABLE, USABLE, FEASIBLE, VIABLE and SELLABLE. 1–highest priority, 4–lowest priority.

V: How do we walk the talk in Dojo?

In Microsoft, we always do dogfooding, or in other words, we use what we develop. Why would people buy or use your products and services if you didn’t even use your own products and services? Given this, how do we create lean products in Dojo?

Two years ago, we decided to pivot from Azure DevOps to the GitHub platform. We had a lean product called “Make GitHub Real” in Dojo, and here are our four dimensions of this lean product.

VALUE PRINICPLES

• Deliver E2E GitHub solution for fields
• Seen as valuable by field organizations
• Leveraged by Dojo master class labs
• Seen as valuable by Dojo Lab Coaches
• Become a Reference Implementation

USABILITY PRINCIPLES

• InnerSource the content for everyone
• Self-services for provision instances
• Reference branch by Dojo capability
• Modular by continuous capability

FEASIBILITY PRINCIPLES

• Integrate GitHub Codespaces into CI
• Integrate GitHub Advanced Security in CICD
• Test ADO and GitHub bi-directional Integration
• Test DORA KPIs on Grafana & GitHub

VIABILITY PRINCIPLES

• Scale Internal users in GHEC with self-service
• Readiness for customers—public/private repo
• Readiness for customers—customer’s repo
• Managed services vs. dedicated services

IS VALUABLE?

The first step was to test if it is valuable and desirable for our users. Our goal was to have a holistic view of DevOps in GitHub while integrated with Azure DevOps (ADO) to leverage ADO’s for portfolio/Agile management, so we get the best of both sides to be useable for most users in the field.

IS FESIBLE?

The feasibility in technology also influences functionality to be valuable and viable. For example, at the time of development, the Codespaces was still in preview, the Advanced security required special license, chat bot integration with GitHub was not available, and DORA KPIs had no future release date. Therefore, we must consider all those aspects to decide if we should code it or wait until it is available. If we wait, then the use cases we designed won’t be available. If we decide to develop it, additional effort must be considered, which also impacts the functional view.

IS USABLE?

One of the key elements of usability is self-service. For broader adoption, we must enable self-services to automate the process. Self-service requires user inputs/validation, backend automation services, notification capability, exception handling, secrets handling, purging processes, and report capability.

IS VIABLE?

Finally, viability is equally important as other aspects. Even if it is valuable, usable, and feasible, if it is not viable for business, everything we have done is wasted. In our case, we had to work with security, one engineering team, and the GitHub team to understand the license aspect, security aspect, and legal aspect to scale the solution.

With value testing on all four aspects of the Make GitHub Real (MGR), we had great success before scaling. If you wish to learn more, please see our YouTube Video series here: GitHub Bootcamp – YouTube

WHAT DID WE LEARN?

What we have learned so far in our MGR lean product development are:

1. Our users and customers don’t know what’s possible, many customers don’t know what they really want until they see it. So, in this case, creating an MVP is very important to help our users see what they can’t see or understand.

2. A good user experience such as self-service in our case is extremely critical for the adoption of the product.

3. Product functionality, design, and technology are inherently intertwined. In this case, Codespaces, Chatbot and DORA KPIs demonstrated this point clearly.

4. We expect that many of our ideas won’t work out. We did try to have an error budgeting use case in our design. For example, when the errors in production exceed SLA, we stop feature shipping and only accept bug fixes. It requires a much more complicated design initially due to the resource limitation, so we couldn’t do it.

5. We must validate our ideas on real users and customers. We opened it for all internal users, where one of the main issues was the documentation of use cases. For example, new GitHub users or new Azure users often become lost in the details, so a flow diagram would be much more helpful. New Azure users sometimes forget to stop the resources, so their monthly cost exceeded their limits.

6. Our goal in discovery is to validate our ideas in the fastest and cheapest way possible. Since everything in MGR is innersourced, it is open to anyone to run and test to get feedback as soon as possible.

7. We need to validate the feasibility of our ideas during discovery, not after. We did this very well when we started PoC for all unknown and new capabilities before implementing other capabilities. We also actively reached out to GitHub to understand their roadmap and availability of the new features so that we could make intelligent decisions on what to build and what to wait for.

8. We need to validate the business viability of our ideas during discovery, not after. This includes financial considerations, marketing, security, licenses, and legal. We learned a lot during this process because we are one of the very few frontrunners in the field to adopt GitHub in such an extensive way.

It is worth mentioning that the MGR lean product was developed through innerSourcing by the Dojo community. We will have a dedicated blog about how we do innerSourcing.

VI: Scale Process or Scale Leaders in Lean Product

Please note, our Orange Belt is about scaling DevOps. In the context of lean products, fundamentally we have two ways to scale lean products: One is to scale the process and one is to scale leaders. Before we discuss our recommendation in scaling lean products, let’s look at some anti-patterns in scaling lean product processes.

Anti-pattern #1: DevOps/Lean Product leadership is at a lower organization level.

Our colleague Dr. Krikor Maroukian shared this survey result in his research paper, The Link Between Transformational and Servant Leadership in DevOps-Oriented Organizations. Most participants believed that the DevOps/Lean Product leadership role should report to the C-level to be effective. This is a transformational role, and the primary responsibility of leadership is the creation of an environment that enables a DevOps culture. When it is too low at the organization, cultural resistance and low levels of discipline will create significant failure rates for DevOps/Lean Product adoption.

Anti-pattern #2: DevOps/Lean Product leaders had no previous transformational experience.

According to the Mckinsey research article, “Managing large technology programs in the digital era”, given the complexity of large program implementations, it is crucial to have people who have already done them, or something comparable. There is just no substitute for that kind of experience and “pattern recognition.” Without it, failure is far more likely. As might be expected, these people are hard to find, especially since these sorts of large programs happen infrequently for most organizations.

In our experience, we have seen that when leadership assigned managers who have never done DevOps or built any digital products to lead DevOps/Lean Product program, those managers either make it a religion or make it a marketing buzz word.

When the managers make the process a religion, they force everyone to use it regardless of the situation. People struggle with understanding what is expected from managers, so it creates a lot of confusion and resentment.

When the managers make the process a marketing buzz word, the rest of the people don’t understand or believe in it, so there is an illusion of action, but very little impact. It is common to see a lot of activity but mostly in the old mindset of doing “something”, so it looks impressive but there is very little substance and change.

Anti-pattern #3: Missing key capabilities and attributes in selecting product managers

The product manager is the CEO of the product, so they drive the vision, strategy, design, and execution of the product. Product managers need three types of knowledge: organizational knowledge, product knowledge, and industry knowledge. As might be expected, good and mature product managers are hard to find.

As product-centric model becomes more accepted and more organizations move to the product delivery model, the role of product manager is becoming increasingly important—but also harder to fill. Faced with the challenges of staffing the product manager role, many organizations default to using business analysts, scrum masters, or project managers. Traditional project managers, business analysts or scrum masters often struggle to be effective product managers due to significant skill and knowledge gaps.

Anti-pattern #4: Scaling readiness through top-down military style.

We have seen that managers scale readiness by using the military style top-down execution of process. Their goal is to track the quantity instead of quality. Even though experiential learning would drive enthusiasm, understanding, and mastery, and help culture transformation, those managers would dismiss it as they consider it to be not scalable. Those managers don’t understand the truth about what motivates us: Autonomy, Mastery and Purpose. Instead of forcing people to take the training, we should focus our efforts on creating environments so our innate psychological desire can flourish.

Our Recommendations

Our strong opinion is to scale leaders in Dojo. Process itself is not inherently bad, but when process people make it like a religion, it can lead to lower innovation and higher resistance. Process is needed only when process helps a product team to create business value and customer satisfaction.

We have adopted the attributes of a powerful Dojo Coach as suggested by Gartner. With additional industry knowledge, people with attributes of a powerful Dojo coach can be good candidates for product managers.

Attributes of Powerful Dojo coach:

1. Lean Thinker

2. Agile Mindset

3. Strong Listener

4. Development Experience

5. Hands-on DevOps Experience

6. Easily Builds Trusting Relationships

7. Focused on Continual Improvement

8. Charismatic, String Leadership Traits

9. High Level of Emotional Intelligence (EQ)

10. Experience with Organizational Change Management (OCM)

Our colleague Dr. Krikor Maroukian researched on which leadership styles enable DevOps adoption in his research paper. The research shows that the top leadership skills identified are:

1. Communication and collaboration

2. Active listening

3. Customer centric mindset

4. Technical background

5. Problem solver

6. Multi-cultural mindset

7. Influential

8. Agile management

9. Strategic thinking

10. Project management

Even with strong leaders, there are essentially two main ways that you can lead Lean Product organizations. You can lead by what’s known as “command and control,” which means explicitly telling the team what you need them to do usually by assigning them a roadmap of features to build. In this model the leaders and stakeholders are making most of the meaningful decisions, and the feature teams are there to carry out those decisions. Sometimes we call it Agile without brain.

The alternative is that you can lead by empowering the teams, instead assigning teams customer problems to solve, and then letting the teams determine the best way to solve those problems. However, if you choose to push key decisions down to the teams, then you will need to provide those teams with the broader context necessary for them to make good decisions. This is called leading by context, not control.

VII: What is Next

Moving to the Lean Product Model is about embracing high velocity in a secure and reliable manner in the software product development lifecycle. Thus, it requires executives’ understanding, support, sponsorship, and investment in the Lean Product model.

In the Lean Product model, the expectation is that people will work together, including how they will fund and manage the products. The product-centric model brings significant benefits, such as getting value much quicker, serving customers better, adapting change quickly, creating leaner organizations, etc. However, this model will upset traditional control and governance of detailed annual budgets. It will also upset traditional expectations for detailed budgets/plans before funding and is difficult for many stakeholders, executives, and even IT people to deal with. Changing the model requires a change in expectations, objectives, metrics, and culture.

In Part 3, we will discuss various operating models in digital product organizations, the relationship between operating models and cloud, and the leadership role in the DevOps transformation.

Special Dedication

This blog is written and reviewed by the following authors in Dojo Community.

• April Edwards – April.Edwards@microsoft.com
• Krikor Maroukian – krmaro@microsoft.com
• Paul Fijnvandraat – Paul.Fijnvandraat@microsoft.com
• Kan Tang – kan.tang@microsoft.com