Learning is a very old topic. While developing Dojo, we learned that DevOps learning was very different from other types of skill learning because of its complexity, breadth, cultural context, team orientation, and difficulty to measure. As we promised in our first blog, we will share real-life lessons we learned in our journey, and it is not all smooth and rosy. The blogs are NOT about showing how great we are; they are about sharing what we have learned and where we have stumbled.

To embrace the DevOps culture of experimentation, we as a community conducted various experiments in DevOps learning. This blog is about experiential learning, where we had the greatest culture/mindset clash in our Dojo ecosystem.

Here are eight types of experiential learning

• I: Learning from Experts
• II: Learning from Peers
• III: Learning through MOOC
• IV: Learning through Teaching
• V: Learning through Writing
• VI: Learning through Playing
• VII: Learning through Pairing
• VIII: Learn to Unlearn and Relearn

---

I: Learning from Experts

From grade school all the way on up through college, we always learned from subject-matter experts—our teachers and professors. Then at work, we are mostly trained by a couple of professional trainers by subject areas.

When it came to DevOps learning, we were all stumped. As mentioned in the first blog, it took us at least six months of heated debates to agree on a definition of DevOps and DevOps at Services. Here are multiple dimensions of complexity in learning DevOps at Microsoft Consulting Services.

Dimension 1: Twelve subject areas covered in DevOps Dojo

• Pillar 1: Culture
• Pillar 2: Lean Product
• Pillar 3: Architecture
• Pillar 4: Technology
• Capability 1: Continuous Planning
• Capability 2: Continuous Integration
• Capability 3: Continuous Delivery
• Capability 4: Continuous Operations
• Capability 5: Continuous Quality
• Capability 6: Continuous Security
• Capability 7: Continuous Collaboration
• Capability 8: Continuous Improvement

Dimension 2: Three phases in each capability covered in DevOps Dojo

• Phase 1: Theory and science behind the concepts
• Phase 2: Discussion and application of the concepts
• Phase 3: Hands-on technical labs on tools

Dimension 3: We are internally organized by technical domains

Microsoft is a technology company with various products. Like most hi-tech companies, our services are organized by product lines and technical domains. For example, we have domain expertise for App Dev, Infrastructure, Security, Data/AI, Quality, UX, Operations, Adoption Change Management, etc. This is highly effective for domain specific problems, but for DevOps, which requires almost every domain to work together, it becomes challenging. Why?

Conway’s law is an adage named after computer programmer Melvin Conway, who introduced the idea in 1967. The law is based on the reasoning that in order for a software module to function, multiple authors must communicate frequently with each other. Therefore, the software interface structure of a system will reflect the social boundaries of the organization(s) that produced it, across which communication is more difficult.

“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.”

— Melvyn Conway

Due to Conway’s law, when we communicate across domains, the coordination cost becomes high. On the other hand, DevOps is about the economies of flow, and flow is the focus of the DevOps organization.

Dilemma

Now with four pillars and eight capabilities to cover all spectrums of DevOps, three phases of each capability to cover theory/science, discussion/practice, and hands-on/labs, and an organizational structure by technical domain, it is impossible to have one to two experts cover the entire class’s agenda. It was evident that we couldn’t individually cover all DevOps topics, although we could collectively cover every aspect. But how could we operationalize this model internally?

Solution

Like anything in life, your biggest strength might be your biggest weakness. While receivers benefit significantly from this format in terms of learning and mindset change, givers have more challenges to manage. This is especially true in face-to-face learning because we simply can’t send ten to twelve coaches on-site for each master class.

Before the pandemic, we had to send four coaches to regions for each master class: two coaches to cover most topics in presentations, two coaches to cover technical aspects including labs, and all four coaches to handle discussion, breakout sessions, and facilitation.

However, the pandemic changed everything. The bigger the problems, the bigger the opportunities, which couldn’t be more true for the Dojo delivery. Now all Dojo live deliveries are remote, and aside from two full-time Dojo captains, all other coaches are using a time-sharing model to cover the areas they are experts in.

This model enables us to offer the best experience and expertise to receivers, while also removing the internal staffing challenges we faced in the pre-pandemic era. The time-sharing model can not only bring the best regional coaches for the expertise areas, but also it allows us to tap into worldwide talents in this increasingly connected world.

By using this model for delivery, we brought together over 200 years of collective experience from coaches, a level of learning not possible from the traditional one to two trainer courses. This is disruptive in every way from staffing to execution to management and learning. Customers see tangible values immediately, and coaches feel a great sense of community and support in their delivery.

Back to eight types of experiential learning

II: Learning from Peers

The trainings in the past were normally arranged by technical capability. However, DevOps is totally different. DevOps is a people problem, so its performance is directly influenced by organizational culture and mindset. How do we change people’s mindsets and beliefs through training?

So, by embracing hypothesis-driven development practice, our initial hypothesis to maximize culture/mindset change was to mix and match people from various technical domains and various roles so that they would have a chance to practice, express, and align the new beliefs together.

For example, we would put a mix of people from various competencies in the same team:

• App Dev
• Quality and SRE
• Infrastructure
• Security
• UX
• Operations
• Data/AI
• Adoption Culture Management

We would also consider a mix of people from various roles in the same team:

• Scrum Master
• Project Manager
• Architect
• Consultant
• Account Manager
• Customer Engineer
• Delivery Manager
• Architect Manager

After each master class, we release a survey to all participants to understand what they liked and what they wished to do more of or wanted to be improved. Some of the best feedback has always been about the across domain/across role learning from peers. Here are some feedbacks from participants:

“The Dojo master class was an amazing opportunity to not only learn from experts, but also learn from others who were attending the class and learning along with me. Questions would be asked about a particular practice or use case to dive deeper into what is possible and how my peers in the class could utilize practice within their day-to-day interactions. When we learn from those around us, we can grow our own capabilities and even return the favor to our peers when we share our own ideas. We all have such diverse perspectives that we learn a great deal more when initial concept is reinforced by seeing how peers are thinking about the application of a given capability. “

“Having a smaller team provided a forum for a more personalized set of agenda and discussion. The speed and depth of content of the master class was very dynamic based upon the needs of the team being trained leading to productive conversations and debate.”

“In my opinion this type of training brings teams closer together because you are all focused on learning the same thing and you not only learn the technical pieces, but also get a better sense of those around you: how they think, how they learn, and what is meaningful to them. The Dojo master class was one of the most effective classes I have been to and a big reason for that was my peers who were learning right along with me.”

Our hypothesis was positively validated. Participants learned so much from their peers, they gained a much better understanding and appreciation of what their peers were doing, and most importantly, the live Dojo class instantly increased information flow. Why is information flow so important in DevOps?

“The organizational culture predicts the way information flows through an organization. Good informational flow is critical to the safe and effective operation of high-tempo and high-consequence environments, including technology organizations.”

— from Accelerate

Back to eight types of experiential learning

III: Learning through MOOC

After our initial success in Dojo, more and more people wanted this style of Dojo master class in various business units. The live Dojo delivery was immersive and impactful, but the cost involved was higher than that of traditional trainings and we still had only trained a small number of people in the organization. Because of this, our training organization suggested a new way to scale Dojo called MOOC.

What is MOOC? A Massive Open Online Course (MOOC) is an online course aimed at unlimited participation and accessible via the web. In addition to traditional course materials, such as filmed lectures, readings, and problem sets, many MOOCs provide interactive courses with user forums or social media discussions to support community interactions among students, professors, and teaching assistants, as well as immediate feedback to quick quizzes and assignments. MOOCs are a recent and widely researched development in distance education that were first introduced in 2008 and emerged as a popular mode of learning in 2012.

After the initial Dojo White Belt MOOC was released, the decision was made to discontinue Live Dojo Delivery for the reasons of scaling and cost consideration. This was when we had the biggest debate within the Dojo ecosystems. To hear the voice of the community, we decided to send a survey to all key stakeholders in various Business Units. Here is the short survey sent out.

Is White Belt MOOC a replacement of Live Dojo White Belt Delivery?

1) Yes

2) No

3) I don’t know

Interestingly, we didn’t receive a single Yes. Here are a few interesting and provocative responses from Dojo community:

Respondent #1:

“I don’t believe that the MOOC and the full Dojo can ever deliver the same level of learning, for two main reasons:

1. The MOOC is like having a sandwich at your desk while you work, and the Dojo is like sitting down for dinner with a dining companion. Yes, you are consuming calories in both, but one is gone almost straightaway, and you haven’t really engaged in the process. When you learn like that you may take in some facts which build your knowledge – how much you can learn will very much depend on how much you know already – and what you do learn may well fly through your brain without touching the sides. The exception would be if you know that you are going to have to use this information soon, so that you focus on being ready for a job or conversation – so the MOOC as a Readiness for something specific might be more useful.

2. You miss the conversation that might change your point of view. Like Dinner with a companion, you get to talk about what you are doing and to have an exchange of views that help you to reassess your assumptions and opinions, to let the new information rattle around your brain and sink in. If the MOOC is thinking ‘fast’ then the Dojo is thinking ‘slow’ (which is good 😊). When you think fast you have knee-jerk reactions – panic, saying No out of fear, shouting at your kids (just me?). When you think slow you contemplate different ‘scenarios’ – the way things may play out – so you find new ways of thinking about things and thinking about the results and implications of things. Engaging with ideas at this level is how learning happens, and how the learning sticks.

Additionally – and I think this is crucial. The MOOC is a learning event – it’s a course that any of us can sit and do by ourselves with a cup of tea in a few spare minutes. The Dojo is a team master class. We want a team to come and not only learn but work out how to work together. You can’t teach someone how to collaborate in a DevOps team by themselves. That’s like asking someone to learn to play Tennis Doubles by themselves: as soon as their doubles partner turns up on court, they will be falling over each other. The team must work out a new way of working for the team.

So, I think there is a place for the MOOC. But I don’t think it can ever replace the Dojo master classes. For individuals in the company who will not work in or with a DevOps team or project – then the MOOC (with the videos as suggested maybe) might be a great place to start. For anyone who needs to take to the field then the Dojo is a whole different level of learning. “

Respondent #2:

“After the 4-customer dojo’s and 5 master classes I have delivered, if I need to choose 1 aspect that makes Dojo so different and so immersive is the group discussion element embedded in its structure. The most effective learning and the greatest growth I have seen happening are achieved during the valuable discussions in the group.

No online training can give this effect no matter how interactive it is. So, the answer is of course a big, bold NO.”

Respondent #3:

“I (obviously) agree with what everyone said.

2-No. Not even close!

The MOOC is a good replacement for the 2-hour intro video we needed to do before the internal training workshop.

In my opinion, the most valuable part of the dojo is the open discussion, critique, and exchange of ideas, between students and between students and instructors.

The MOOC cannot really replace that. There are some ‘discussions’ as part of some modules, but it is the open-ended impromptu ones, that I find most valuable any time I deliver a process-oriented workshop.

When I went through White Belt training, I had questions, concerns, and arguments. If not for the amazing trainers that I had the fortune of learning from, these questions would go unanswered.

In all truth, they’d likely not even be asked; if I had a strong counter-opinion, I’d just write off the entire online training as wrongful and maybe useless and move on.

In a people, process, and practice training, like DevOps, agility, lean management, and Scrum, people with strong opinions on the matter – both those who think they know how it is done and people who think doing it is wrong – will ignore training material that disagrees with them, if they cannot voice their opinions in a meaningful way.

And no, a ‘feedback’ button is not considered an opportunity to voice one’s opinions. Not by most people.

Bottom line – you can teach ‘how’ to practice DevOps with a MOOC, but only after someone qualified explained and convinced you AND YOUR ORGANIZATION of ‘what’ needs to be done.

To my knowledge, no organization has ever changed after going through a MOOC. “

Respondent #4:

“No!

Curiosity is heavily associated with all aspects of human development, which derives the process of learning and desire to acquire knowledge and skill.

So, during the instructor led master classes we get into real time discussions between people coming from diverse backgrounds. We exchange so many ideas and learn from each other experiences. This is not possible in case of MOOC as the interactive sessions are minimum.

Also, in instructor led sessions we can talk in detail about topics where we feel the audience needs more understanding so the things can be tweaked real time based on needs. MOOC has scripted content, so it generally does not take care of different learning capabilities of the audience.

MOOC is good to create the general awareness but if the idea is to have cultural and mindset shift instructor led trainings have no replacement.”

Conclusion

MOOC is a good approach for awareness and self-study. It is even better if MOOC is bite sized so busy professionals can learn at their own pace. However, MOOC can’t replace real human interaction, nor can it be effective for mindset change. All the success we have made in Live Delivery was primarily due to the human interactions.

As a side note, one architect manager made an interesting comment, “As much as the debate has been so heated, but it shows psychological safety in our organizations. This kind of debate and direct challenge of status quo probably is impossible for some companies where psychological safety is very low or doesn’t exist. So, we all should be grateful this could even happen in our organizations.”

Back to eight types of experiential learning

IV: Learning through Teaching

In two studies published in the journals Science and Intelligence, researchers found that first-borns exhibit a higher IQ than their siblings. However, they aren’t simply born brighter; they’ve worked at it—their higher IQ is strongly linked to the time they’ve spent teaching their younger brothers and sisters.

One of the amazing discoveries we learned in our coaching was how much more we learned through teaching—the key to a deeper learning experience. Teaching really makes us better learners because it makes us work harder to understand, remember, and apply what we are studying.

When we are studying, we can fool ourselves into thinking that we have learned all we need to know. But do we really understand what we are learning? Teaching others removes the possibility of self-deception and shows us exactly how much we know and how well we understand it. Many of us learned so much more through teaching. Here are the testimonials from our new Dojo coaches:

Learning through teaching story, by Margarita Sanz del Rio

“The first time I attended a White Belt Dojo master class, I attended as a bystander. I had the opportunity to see the content and listen to the discussions and debates that were generated. At that moment I thought that everything was so clear and obvious that it fell under its own weight.

The second time I attended, I assisted in the labs. I thought there was little to learn, I had already done it once, and besides, now I would only have to deal with technical problems that might arise. However, I encountered students who went beyond that, who looked at the practice from another point of view that I had not considered. So, I kept learning.

The third time I helped both in the breakout rooms and in the labs. By then I had learned everything, I had participated in everything, so nothing new for me. But once again, I was wrong. New teams come along with new discussions, new challenges, new points of views and new experiences. I have now participated in more than 5 Dojo White Belt master classes, I keep learning and growing from every single one and that’s what I value the most.”

Learning through teaching story, by Giulia Cupani

“When I joined the DevOps Dojo team, I was amazed by the resilience and the innovation of the team. I remember that my first thought was: ‘WOW, this community is resourceful! Experienced people are working on unique content with an efficient approach based on continuously learning and improving. How can I contribute, how can I help this team impacting even more?

I started by testing and providing feedback on the hands-on labs and in a few months, I found myself as a coach of one of the White Belt master classes in Seattle. It has been one of the most exciting things I have ever done. I had done the labs multiple times and I felt confident in providing guidance on how to execute them as well as troubleshoot errors, if ever needed. Nevertheless, I have been challenged with interesting conversations and discussions that lead to different questions to which I did not always had the answer to. Every master class taught me something new and brought me out of my comfort zone.

One of the experiences that taught me the most has been to provide guidance to attendees that had no technical background without losing them on the technical steps but helping them to understand the big picture. I had to adapt to a new audience that made me revisit the way I used to deliver the labs and even in that case I have been challenged by interesting questions that led me to learn something new and step out of my comfort zone to deliver the content in the best way possible. Every master class gave me the opportunity to improve and learn something new, no matter how many times I delivered the content.

Stepping out of our own comfort zone can be frightening but for me it has been enriching. The DevOps Dojo team is a community that backs you up in all circumstances. In every challenge I faced I never felt alone, I have always had the opportunity to turn to the team and ask for suggestions until the time to provide suggestions to new team members came for me too.”

Learning through teaching story, by Harleen Kaur

“First time when I attended a master class, it was a remote delivery, and I was one of the lab coaches. I did practice the labs in demo environment and was able to help the students complete the labs but in happy path scenario. I knew little about what I would do if some configuration went wrong, but I had a psychological safety that my peers from Dojo community would help. Next time, I pushed myself to go one step further and analyze where things could go wrong in labs, think about the fixes, and validate my knowledge with the team. Also, when I was teaching the labs, I could find improvement areas in our own documentation. So, it did help me on self-improvement journey. I learned how I had to teach the technical content to a person who needs 30,000 feet view and to a person who needs a 5,000 feet view.

While I was part of the master classes as a lab coach, I would listen in to the PoV presentations by different coaches and as they shared their stories about their journey it helped me learn as well as excited me to deliver the Point of View(PoV) presentations. One of the best things about Dojo community is that people here encourage you to try new things, so I was able to deliver PoV presentations in internal master classes as well as customer deliveries. As this was received well, it motivated me and the learnings I had personally while preparing encouraged me to deliver more sessions.

It feels so good to challenge myself every single day to achieve things I would have never done if I had not been part of this community. Another important aspect is about being able to share your learnings with the team so each one of us can build on each other’s experience. Being able to coach is a big responsibility, I feel empowered to do it with all the experiences and support the Dojo team provides.”

Learning through teaching story, by Aakanksha Aakanksha

“When I first joined Dojo as a .Net Engineer, I was revamping the existing Smart Hotel 360 solution from a software engineering project mindset. I had no clue what we were going to achieve as an outcome of Dojo. Then I flew to Seattle for our first train the trainer event and I was amazed on how my contributions caused a major impact. I met all the senior DevOps practitioners who introduced me to the real DevOps which was not about technology but rather a huge cultural mind shift. For about one year, I travelled around the globe covering Munich, Shanghai, Seattle multiple times to delivery DevOps Dojo master classes. The most important thing I learnt in Dojo master class delivery was adaptation of content and continuous improvement. Each class was different as same set of agenda might not work from one class to other class.

APJ 1st remote master Class was very dear and near to my heart as I was very closely involved in planning the class from scratch. But we encountered a huge setback during the first two days where we saw participants dropping and negative retrospectives. So, I felt that almost everything went wrong for that class, we even had a discussion to pause the class if situation didn’t improve. But one thing I learnt during those times was to keep calm and being positive. To increase interactivity, we introduced an early bird coffee chat for participants who wished to ask questions prior to master class, which was very popular and successful, and Dojo DJ etc. We closed so strongly, and it led to series of successful master Classes in APJ region.

Dojo is very different from all initiatives I have worked on because it focuses on a maintaining a balance between Autonomy, mastery and purpose. Through Dojo I can say I evolved as a person with more thoughtfulness, resilience, and empathy. Our Dojo team is an innovation powerhouse.”

Conclusion

The real people with real experiences shown above support what the Learning Pyramid demonstrates: teaching others is the best method of retention in learning.

Back to eight types of experiential learning

V: Learning through Writing

Dojo Innersource as a publishing method within Services was originally designed to solve a major problem of having too many copies of Dojo content in a close-sourced ecosystem.

What is Innersource? Innersource is an open-source community behind your firewall. When done right, Innersource is one of the most effective ways to tap into trapped value in the enterprise. Please note that Innersource is not just for source code, as it can be used for any content including sales, presales, marketing, education, or delivery IPs in your organizations. In fact, this blog you are reading is co-created by the Dojo community through Innersource using Markdown.

Innersource publishing created another wonderful learning channel for the Dojo community. Why is that? Well, writing is thinking on paper, so anyone who thinks clearly can write clearly about anything at all. Clear thinking becomes clear writing; one can’t exist without the other. Thinking clearly is a conscious act that writers must force on themselves. Writing makes you think, thinking makes you reflect, reflection makes you deliberate, deliberation makes you learn more deeply.

Rewriting is the essence of writing, by Kitty Chiu

“Bill Gates once said, ‘Measuring programming progress by lines of code is like measuring aircraft building progress by weight.’ This is the same as writing. It is not a matter of how much is written but what concept is being conveyed.

Our experience of writing has been about synthesizing the vast amount of industry practices and our own field experience with customers and condensing them into a body of knowledge that is easy to navigate and consume. This has not been easy job because like coding it is harder to write less, particularly when various SMEs want to emphasize their viewpoints. In our writing, we constantly challenge each other, learn from one another, and iterate collectively.

An example is our write up for Continuous Operations capability in Dojo Innersource. Recently we have done a major revision of all the content. In the initial version, we have many angles for the practices, in some sense they are quite fragmented. After feedback from the field and more Subject-Matter experts (SMEs) contributing, we saw the need to add a particular storyline for this capability to bring the community on the journey. The team had rebuilt the whole Continuous Operations structure with the Site Reliability Engineering (SRE) focused and consolidated the eight practices into five. In White and Orange Belts of DevOps Dojo, we have designed to provide the breath of the Continuous Operations in SRE, and we introduced Green Belt for SRE to provide the depth. Today the body of knowledge in this capability is ever more holistic and richer than our competitors and Microsoft partners.”

All writing is ultimately a question of solving a problem, by Harleen Kaur

“If you want to learn a thing, read that; If you want to know a thing, write that; if you want to master a thing teach that.

This is so true when we create the content for Dojo be in presentations, labs, or case studies for discussions. Creating lab guides is essential for immersive learning so users can understand how they can apply the learnings, however the journey of creating the lab guides is topsy-turvey. The emphasis is to ensure that the objectives for the labs are clear and the users at any point do not feel that it is just a sequence of steps written in text. It should be more like a conversation they are in where they understand what to do, why it is needed and how do they do it.

When I work with the team to write the lab guides, most of the times the use case seems simple at first glance, but when I start writing it, it triggers my curiosity levels not only from technical but also from process standpoint. I had to think about writing the content from end user standpoint and ensure it made sense without creating confusion. Writing the content for the lab guides helped me to learn so much about Azure that I was able to clear my certifications with minimal reference to study materials.

Creating content is still easy but maintaining it requires far more effort. This made me think of ways in which we can minimize the redundancy so while maintaining we do not make same changes at multiple places. Earlier while writing the labs the team would capture snapshot after each step to show how the screen looked after performing the step but in the long run this is not a viable way as UI changes very frequently, so team decided to create GIF for series of steps to capture things from logical standpoint.

We also ensured the users have a clear view on what is the next topic they can refer to for further learning. I have seen myself and my colleagues in the Dojo community getting so mature at writing content for easy consumption.”

Writing is at the core of our learning, by Alvaro Guadamillas Herranz

“How is Innersource connected to writing – and to learning?

As a learner, my first preference is always to learn by reading. Tutorials, articles, blogs, technical documentation, API documentation, READMEs. This approach gives me time to focus and absorb concepts at my own pace and let me arrange my own personalized learning path. As a consultant for a technology company who lives in a world in constant change, learning is the ability that I practice more often, and reading is the most accessible approach that I find to learning. So, other’s writing is crucial for my work – and even for my personal life.

As a software developer, I read code from others, write my own code, contribute to other’s code, and write documentation for future developers or for users that need to learn how to use the software or tools that I am building. So, writing is essential for the success of my work and for the success of others.

With Open Source, I collaborate at scale, asynchronously, with people that I will probably never get to know and making everything immediately transparently available to the entire world. For the success of Open Source, writing is important in all aspects – code conventions, comments, technical documentation, API documentation, Pull request documentation – as well as in discussions. Open Source is only effective when addressed with the right culture, with the appropriate maintenance and with the right writing culture.

Reading and writing is at the core of my work in a bi-directional way: I learn from other’s writings; I share my own experiences and knowledge to others through my writings. And Open Source has shown that we can efficiently and transparently learn and share with anyone in real time, which makes the process incredibly effective to keep us at the pace of the innovations.

For these reasons, and after more than one year piloting first and working with an Innersource-first mindset later, we in Dojo actively Innersource all our work and collaborate with other teams that Innersource their work, achieving the great benefits: enable our talented workforce; create Collaboration as a Cultural Norm; facilitate Component Reuse; and build a better community.

Certainly, my learning process has achieved its most efficient stage thanks to Open Source and Inner Source, and writing is at the core of this process.”

Back to eight types of experiential learning

VI: Learning through Playing

Learning through play is a term used in education and psychology to describe how a child can learn to make sense of the world around them. Through play, children can develop social and cognitive skills, mature emotionally, and gain the self-confidence required to engage in new experiences and environments. However, in adulthood, we don’t really apply the learning though playing strategy often. So, we decided to try some learning through playing techniques in Dojo, here is what we have learned.

The very first test was about an accessibility debate by two Dojos from two different BUs. Both were “argumentative,” passionate, and opinionated. This debate remains the most memorable and impactful debate in our Dojo history. Coincidentally, this was right before the presidential debate in the US, so this made our play a lot more fun.

Dojo Accessibility Debate by Harry Chen & Assaf Stone

“The main point of the discussion was centered around the observation that the importance of building accessibility in application/software has not been widely recognized and is often treated as nice-to-have features of an application. For consulting services, time, cost, and schedule are always the main constraints. If we don’t build accessibility into initial discussions with customers during contracting, accessibility is likely to be pushed to low priority or not be addressed at all during the engagement.

To adequately address the accessibility in applications, we first need to educate ourselves on the importance of the accessibility. Within Microsoft, we have had organizational and leadership support for accessibility awareness and many of us have gone through the training, so we have a good start on awareness. However, to effectively implement accessibility, we need to treat it as an integral part of software quality and create practical guidelines and processes to help services estimate and implement accessible applications. Dojo Accessibility is an important tool developed to help that effort. We believe accessibility is important not only for end users but also for business outcomes and success. “

Learning through playing story, by Yue Sheng

“In our face-to-face master classes, we have introduced several interactive games, including icebreakers to help people know each other and competitions to intensify learning, which can actively engage participants physically, intellectually, emotionally, and socially. For instance, in the GUESS WHAT game, one person must describe the phrase related to DevOps without spelling out the actual terms. The other person must guess out what it is in a limited time. With physical performance and language expression, keywords must be described clearly and vividly. And guesser also needs to quickly associate and search in the knowledge base learned in class. As the countdown ticked, all audience were involved and held their breath to watch the game. With the excitement in the air, it’s also a creative outlet to test proficiency and mastery of their knowledge in DevOps.

What’s more, in the final challenge session presentation, participants were encouraged to be creative and original. One team worked until late night every day to develop a real play showing dramatic conflict between various team members due to very different mindset. The play was so real while so hilarious, it captured our attention for the entire time! Suddenly we realized that Dojo was producing actors and actresses in such an organic way! As much as the team produced a humor production, everything they did behind the scenes was serious. They strived for excellence and hoping to add humor along the way, ultimately, two are intertwined to reach the best learning experience.”

Make DevOps Real story, by Charlie Gu

“Here is another example of how our team made DevOps real in a Dojo master class. Our team designed a live demo, which started from a customer’s change request in the beginning due to a production outage. During the presentation, we demonstrated all practices learned along the way in the class. We pushed the code change and tests, and the code changes were moved from stage to stage automatically. The magic happened at the end of the fifteen-minute presentation: the fix was released to production within the demo time! Through a playful process, we were able to integrate learning and quickly adopt it to real scenarios. You could feel our pride and unity after our successful play!

When people exhibit playfulness, learning is fun and enlightening. We in play mode construct our own knowledge and apply our learning to new situations. We take risks and strive for innovative solutions. But the fun doesn’t just stop after the class, it goes beyond. We shared our positive learning experience with our co-workers and managers after we finished the Dojo class, so now more and more people are keen to join the next master class. The enthusiasm of each class brings everyone together, and we started to see culture and mindset shift.

In short, we lived the immersive experience of Dojo Shu, Ha and Ri through playing. “

GitHub Bootcamp with Octocats, by Josh Sanderlin

“During our recent GitHub Bootcamp event in Dojo community, we had a lot of fun creating our own Octocat profiles. We all walked out of the GitHub Bootcamp feeling smarter, happier, and even healthier with all the good laughs!

This GitHub bootcamp for Dojo community included solution architects from the office of the regional CTO organization who were brought in to be educated on the capabilities of GitHub. These architects came from very diverse backgrounds and have very different, but complimentary skillsets. I was one of those architects, and I can say from personal experience, that the things we learned in GitHub and the subsequent creation of octocats brought us together as a team and allowed us to have some fun sharing what we had learned. As we learned, GitHub has a great set of technical capabilities, but GitHub is also a very dynamic organization. The team providing the class was well connected, aligned, and motivated. Many sessions that I have attended in the past have not had the type of team alignment and energy that the GitHub team had, and they formed a cohesive unit where everyone had their part to play. This has proven to be very effective.

The creation of the octocats provided a way for us to come closer together as a team and rally around something we had learned. It was such a small thing, but the team now has something they did together, that was fun, that also commemorated the learning that took place. Often, we complete training and just check a box, in this instance that was not the case, and we had a bit of fun (I envision custom t-shirts in our near future). Working and playing together as a team binds the team together, the octocats are a symbol, and the act of creating that symbol was a way to begin to establish team unity. Creating our own octocats allowed us to express our interests through our individual cats and learn more about our teammates in the process. “

Back to eight types of experiential learning

VII: Learning through Pairing

Learning through pairing story, by Dojo Captains

We compare running a Dojo master class as flying a Boeing 777 for a long-distance international flight. For any long-distance international flight, in normal circumstances, crew does everything that seems to be effortless, passengers just enjoy their flights; in special circumstances, for example facing mechanical problems or severe weather conditions, the captains and crew are responsible for your safety with your corporation. This is exactly how we work in Dojo coaching.

In all remote deliveries, we always have two captains per class with 10-12 or more coaches with time-sharing model. Just like flight captain and first officer, two Dojo captains have tremendous responsibilities before, during and after each master class. We are very careful in selecting captains, as they can make or break the classes. Dojo captain must be accountable, collaborative, communicative, knowledgeable, flexible, and adoptable. Ideally captains should have known each other and worked well together in the past, so the chemistry is right between two captains that they make the class comfortable and smooth.

In each class, we always had surprises, or unexpected events, so the captains had to think on their feet, be flexible and resourceful to resolve issues as soon as possible. So, it is very important that captains can rely on each other, can trust each other, and can accept each other’s differences to achieve the end goals.

Many Dojos haven’t appreciated the captains enough till they became one. It takes 2-4 weeks intensive effort to prepare for a class by both captains. The real test is to deliver the master class. Just to give you a few examples what happened during our deliveries:

• sudden loss of power during the presentation by coaches or captains;

• personal and family emergency from the crew before the class;

• personal and family emergency from the participants due to COVID-19;

• extremely outspoken and active class that led the class behind on agenda;

• extremely quiet class that led coaches’ uncertainty of their performance;

• the flaw in labs that prevented one team from continuing to next step;

• teams were very competitive on final challenge so hard to pick the winner;

• keep the participants engaging and interested in topics in remote setting;

• deal with culture difference, class maturity difference, and style difference.

Just like flight pilots facing turbulence, both Dojo captains must handle all the difficulties and dynamics behind the scenes while keeping the class under the control. Trust is everything, our exceptional captains have handled all sorts of unexpected events, now they can call each other “Comrades in arms!”

Within a week after delivery, both captains had to work tirelessly to conduct the final survey results with improvement plan, and newsletter to celebrate the completion of each class. It can be emotional during this period. A coach made an emotional conclusion on last day of the class: “at the end somehow it made us sad, it seemed like a real bond was created among us in those 2 weeks. With the debates, competitions, laughs, jokes, and all the fun we had, it was just difficult to say goodbye at end of Friday.”

Work doesn’t have to be cut-throat and zero-sum game; work can be done in a different way—a respectful, trustworthy, productive, and enlightening way. Through Dojo pairing, our captains built authentic and long-lasting friendship during the Dojo delivery even most of them have never met.

Learning through pairing using GitHub Codespaces, by Margarita Sanz del Rio & Deep Mehta

“DevOps Dojo is a community that crosses borders. DevOps Dojo is an opportunity to do things you love, with people who are passionate about what they do, that you might not otherwise have been able to meet. About a year ago, we started a great adventure with GitHub, defining a new architecture, experimenting with new technologies, but most of all learning from each other. From India to Spain (passing through many other countries with the rest of the team of this initiative), we have worked side by side planning, developing, and testing. We have failed many times along the way, but we have been able to support each other to find new ways, new solutions, to face each challenge.

We have had the great opportunity to access new features and use them to make this project a success. One example is GitHub Codespaces – a tool that has allowed us to collaborate in a much closer way, for example we can watch each other typing code and comments synchronously in the same IDE. During past nine months, we have not been able to sit at the same table, we have not been able to have a coffee and discuss in person what more we can add, we have not even coincided in the same time frame. However, we have collaborated more closely than we sometimes do with our teams at the customer. It has been a wonderful experience, a great example of what can be achieved when you have a space to do what you love, which in our case is to innovate. We are excited for what is yet to come, for the next steps we are going to take and the new projects in which we will be able to continue collaborating and supporting each other.”

Back to eight types of experiential learning

VIII: Learn to Unlearn and Relearn

If you have been in IT for more than twenty years, you would have heard about, seen, or used a maturity model as it was popular for many standards, frameworks, and management approaches. If you have been in DevOps since early 2010 in IT Services companies, you would have heard about, seen, or even used a DevOps maturity model or DevOps Assessment. In fact, just a few years ago in our services organizations, we had three DevOps assessment tools. At one point, we all agreed to consolidate all three assessments into a single DevOps maturity model assessment.

This was not an easy task. We wanted a lightweight assessment, so we decided to have less than eighty questions. However, since there was no industry standard for what maturity levels to use and what each level meant, we could barely agree on the names of each level, let alone define what each level really meant. And what about the logic and formulas we would use behind the scenes, given that questions and answers might be interdependent? How could we really provide an accurate maturity level when the organizational complexity might have been much bigger than what we expected? Moreover, what about if customers disagreed with our assessment results on their maturity level?

Finally, we put together a single DevOps assessment and then started to test it internally for multiple projects. This was where we learned something new. First, the assessment output from those eighty questions was not consistent with reality had we have done live discussion with the project teams. Second, people didn’t like to be judged when receiving a maturity score, they didn’t always agree with the output, and most importantly, there was no prioritized and actionable backlog after assessment. It was something nice to have, but it didn’t have substantial value.

While we were struggling with our strategy, the research in book Accelerate shed some light. We were indeed on a wrong path. We had to unlearn to NOT use the maturity model for DevOps and re-learn to use the capability model. In the first chapter of Accelerate, Dr. Forsgren described four reasons why a maturity model is not the appropriate tool to use for DevOps.

First, maturity models focus on helping an organization “arrive” at a mature state and then declare themselves done with their journey, whereas technology transformations should follow a continuous improvement paradigm. Alternatively, capability models focus on helping an organization continually improve and progress by realizing that the technological and business landscape is ever-changing.

Second, maturity models are quite often a “lock-step” or linear formula, prescribing a similar set of technologies, tooling, or capabilities for every set of teams and organizations to progress through. Maturity models assume that “Level 1” and “Level 2” look the same across all teams and organizations, but those of us who work in technology know that this is not the case. In contrast, capability models are multidimensional and dynamic, allowing different parts of the organization to take a customized approach to improvement, and focus on capabilities that will give them the most benefit based on their current context and their short and long-term goals.

Third, capability models focus on key outcomes and how the capabilities drive improvement in those outcomes—that is, they are outcome based. Most maturity models simply measure the technical proficiency or tooling install base in an organization without tying it to outcomes. These end up being vanity metrics: while they can be relatively easy to measure, they don’t tell us anything about the impact they have on the business.

Fourth, maturity models define a static level of technological, process, and organizational abilities to achieve. They do not take into account the ever-changing nature of the technology and business landscape. What is good enough and even “high-performing” today is no longer good enough in the next year. In contrast, capability models allow for dynamically changing environments and allow teams and organizations to focus on developing the skills and capabilities needed to remain competitive.

With this realization, we created DevOps Lensed Sprint 0 (LS0). LS0 supercharges regular Sprint 0 by using a structured and persona-based requirements collection framework and is harvested from years of Microsoft transformation experience and reviewed by subject matter experts to drive comprehensive DevOps quality requirements. In LS0, we use a capability model and build a requirement bootstrap that leverages over ten years of MSFT DevOps tested practices so that it can serve as a baseline for reuse without re-inventing the wheel.

Today, we use LS0 to understand customers’ real needs before we start Dojo so that we can tailor the content for customers’ preferences and needs. The learning that took us from a maturity model to a capability model might be one of the biggest unlearn-and-relearn exercises we have lived through in the Dojo community. We would like to end with Brené Brown’s quote on learning.

Conclusion

Before the crisis, CEOs rated skills gaps as a top business challenge according to Gartner. Enterprises couldn’t hire all the talent they needed for digital transformation through such changes as agile ways of working, digital product management, and DevOps. The COVID-19 crisis won’t change this because few workers with these skills are being laid off.

How do we boost skilling in this fast-paced world? Our stories show that learning is continuous and needs to be experiential. The best way of learning is to find the best options for individuals based on their needs and preferences. We don’t believe that learning has one size that fits all. For those organizations that treat learning as a checkbox and measure the number of people who have been trained by topics, think twice. We would like to use Adam Grant’s quote here on measurement:

“Many people chase wealth, status, and achievement because progress is easy to measure, failing to realize that the gains that count the most are the hardest to count. Real growth is building character—striving to improve in generosity, integrity, humility, fairness, or courage.”

-Adam Grant

In DevOps, real growth is to enable people to work together to solve the hardest problems and create extraordinary products and services. The real examples and real people in this blog demonstrated how our community enables people to work together to continuously improve our process, mindsets, capabilities, and services.

So far in our blogs, we have talked mostly about how we have empowered ourselves to enable DevOps by walking the talk, but the end goal is to empower worldwide customers to innovate at the speed of business. In the next blog, we will discuss DevOps Dojo – Customers and Trust in our DevOps journey.

Special Dedications

This blog is written and reviewed by the following authors in Dojo Community.

• Steven Murawski Steven.Murawski@microsoft.com
• Kitty Chiu Kitty.Chiu@microsoft.com
• Margarita Sanz del Rio Margarita.Sanz@microsoft.com
• Giulia Cupani Giulia.Cupani@microsoft.com
• Yue Sheng Sheng.Yue@microsoft.com
• Harleen Kaur Harleen.Kaur@microsoft.com
• Aakanksha Aakanksha aak@microsoft.com
• Charlie Gu chagu@microsoft.com
• Harry Chen harrchen@microsoft.com
• Josh Sanderlin Josh.Sanderlin@microsoft.com
• Alvaro Guadamillas Herranz Alvaro.Guadamillas@microsoft.com
• Beste Altinay Beste.Altinay@microsoft.com
• Assaf Stone asstone@microsoft.com
• Niki Mossman nimoss@microsoft.com
• Deep Mehta Deep.Mehta@microsoft.com
• Dave McKinstry dmckinstry@github.com
• Dave Burnison daveburnisonms@github.com
• Kan Tang Kan.Tang@microsoft.com