{"id":12996,"date":"2017-05-24T11:11:44","date_gmt":"2017-05-24T16:11:44","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/bharry\/?p=12996"},"modified":"2019-02-27T23:17:29","modified_gmt":"2019-02-27T23:17:29","slug":"the-largest-git-repo-on-the-planet","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/bharry\/the-largest-git-repo-on-the-planet\/","title":{"rendered":"The largest Git repo on the planet"},"content":{"rendered":"

It’s been 3 months since I first wrote about our efforts to scale Git to extremely large projects and teams <\/a>with an effort we called “Git Virtual File System”.\u00a0 As a reminder, GVFS, together with a set of enhancements to Git, enables Git to scale to VERY large repos by virtualizing both the .git folder and the working directory.\u00a0 Rather than download the entire repo and checkout all the files, it dynamically downloads only the portions you need based on what you use.<\/p>\n

A lot has happened and I wanted to give you an update.\u00a0\u00a0Three months ago, GVFS was still a dream.\u00a0 I don’t mean it didn’t exist – we had a concrete implementation, but rather, it was unproven.\u00a0 We had validated on some big repos but we hadn’t rolled it out to any meaningful number of engineers so we had only conviction that it was going to work.\u00a0 Now we have proof.<\/p>\n

Today,\u00a0I want\u00a0\u00a0to share our results.\u00a0 In addition, we\u2019re announcing the next steps in our GVFS journey for customers, including expanded open sourcing to start taking contributions and improving how it works for us at Microsoft, as well as for partners and customers.<\/p>\n

Windows is live on Git<\/strong><\/p>\n

Over the past 3 months, we have largely completed the rollout of Git\/GVFS to the Windows team at Microsoft.<\/p>\n

As a refresher, the Windows code base is approximately 3.5M files and, when checked in to a Git repo, results in a repo of about 300GB.\u00a0 Further, the Windows team is about 4,000 engineers and the engineering system produces 1,760 daily “lab builds” across 440 branches\u00a0in addition to thousands of pull request validation builds.\u00a0 All 3 of the dimensions (file count, repo size and activity), independently, provide daunting scaling challenges and taken together they make it unbelievably challenging to create a great experience.\u00a0 Before the move to Git, in Source Depot, it was spread across 40+ depots and we had a tool to manage operations that spanned them.<\/p>\n

As of my writing 3 months ago, we had all the code in one Git repo, a few hundred engineers using it and a small fraction (<10%) of the daily build load.\u00a0 Since then, we have rolled out in waves across the engineering team.<\/p>\n

The first, and largest, jump happened on March 22nd when we rolled out to the Windows OneCore team of about 2,000 engineers.\u00a0 Those 2,000 engineers worked in Source Depot on Friday, went home for the weekend and came back Monday morning to a new experience based on Git.\u00a0 People on my team\u00a0were holding their breath that whole weekend,\u00a0praying we weren’t going be\u00a0pummeled by a mob of angry engineers who showed up Monday unable to get any work done.\u00a0 In truth, the Windows team had done a great job preparing backup plans in case of\u00a0mishap and, thankfully, we didn’t have to use any of them.<\/p>\n

Much to\u00a0my surprise, quite honestly, it went\u00a0very smoothly and engineers were productive from day one.\u00a0 We had some issues, no doubt.\u00a0 For instance, Windows, because of the size of the team and the nature of the work, often has VERY large merges across branches (10,000’s of changes with 1,000’s of conflicts).\u00a0 We discovered that first week that our UI for pull requests and merge conflict resolution simply didn’t scale to changes that large.\u00a0 We had to scramble to virtualize lists and incrementally fetch data so the UI didn’t just hang.\u00a0 We had it resolved within a couple of days and overall, sentiment that week was much better than we expected.<\/p>\n

One of the ways we measured our success was by doing surveys of the engineering team.\u00a0 The main question we asked was “How satisfied are you?” but, of course, we also mined a lot more detail.\u00a0\u00a0Two weeks into the rollout, our first survey resulted in:\n\"gitsurvey\"<\/a><\/p>\n

I’m not going to jump up and down and celebrate those numbers, but for a team that had just had their whole life changed, had to learn a new way of working and were living through a transition that was very much a work in progress, I felt reasonably good about it.\u00a0 Yes, it’s only 251 survey responses out of 2,000 people but welcome to the world of trying to get people to respond to surveys. \ud83d\ude42<\/p>\n

Another way we measured success was to look at “engineering activity” to see if people were still getting their work done.\u00a0 For instance, we measured number of “checkins” to official branches.\u00a0 Of course, half the team was still on Source Depot and half had moved to Git so we looked at combined activity over time.\u00a0 In the chart below you can see the big drop in Source Depot checkins and\u00a0the big jump in Git pull requests but overall the sum of the two stayed reasonable consistent.\u00a0 We felt that the data showed that the system was working and there were no major blockers.<\/p>\n

\"activity\"<\/a><\/p>\n

On April 22nd, we onboarded the next wave of about 1,000 engineers.\u00a0 And then on May 12th we onboarded another 300-400.\u00a0 Each successive wave followed roughly the same pattern and we now have about 3,500 of the roughly 4,000 Windows engineers on Git.\u00a0 The remaining teams are currently working to deadlines and trying to figure out when is the best time to schedule their move, but I expect, in the next few months we’ll complete the full engineering team.<\/span><\/span>\nThe scale the system is operating at is really amazing.\u00a0 Let’s look at some numbers…<\/p>\n