Holy Toledo! The growth in data on our dogfood server just isn’t slowing down. The file count (in particular) is really rocketing. We’re now over 33 million (yes, million) files and I don’t see any evidence that it’s going to slow down soon. A while ago, I predicted that we’d hit 100 million this year – I’m now starting to wonder if that was conservative.\nFollowing up from last time, we’ve spec’d out and ordered a new SAN to better be able to handle the I\/O load we’re puting on the system. Here’s the stats:<\/p>\n
It’s arranged as 15 paired 300GB data drives (and 1 spare), 4 paired 146GB TempDB drives and 2 paired 146GB Log drives. As a conservative estimate, with each disk providing 125 IOPS (I\/O’s per second), that should give us a capacity of 3,750 IOPS on the data drive. Currently we are peaking around 500 IOPS in production. This should give us ample room to grow, especially when added to the fact that we are constantly making improvements that reduce the number of I\/Os we do per operation.\nOne thing to note about this configuration is that our TempDB drive set is much bigger than Log (most importantly in terms of spindle count, not GB). We are using 8 drives for TempDB and 4 for Log. This is because TFS is a pretty heavy user of TempDB. On our production system, we are seeing TempDB peek at about 200 IOPS and Log at about 100 IOPS.\nAs I said, we continue to work on performance. In the past few weeks, we’ve made the following improvements:<\/p>\n
<\/span><\/font> <\/li>\n
<\/span><\/font> <\/li>\n
<\/span><\/font> <\/li>\n
<\/span><\/font> <\/li>\n<\/ul>\n
And on to the actual stats… As I mentioned, they are growing, growing growing. But they would be even bigger except that we did a bunch of house keeping this month. We deleted some old projects that are no longer in use and hundreds of workspaces and thousands of shelvesets that had not been used in over 6 months. This effort reduced some tables by thousands of rows and others by many millions.\nUsers Brian<\/p>\n","protected":false},"excerpt":{"rendered":" Holy Toledo! The growth in data on our dogfood server just isn’t slowing down. The file count (in particular) is really rocketing. We’re now over 33 million (yes, million) files and I don’t see any evidence that it’s going to slow down soon. A while ago, I predicted that we’d hit 100 million this year […]<\/p>\n","protected":false},"author":244,"featured_media":14617,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[3],"class_list":["post-10561","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-tfs-dogfood-statistics"],"acf":[],"blog_post_summary":" Holy Toledo! The growth in data on our dogfood server just isn’t slowing down. The file count (in particular) is really rocketing. We’re now over 33 million (yes, million) files and I don’t see any evidence that it’s going to slow down soon. A while ago, I predicted that we’d hit 100 million this year […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts\/10561","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/users\/244"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/comments?post=10561"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts\/10561\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/media\/14617"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/media?parent=10561"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/categories?post=10561"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/tags?post=10561"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
<\/u><\/strong>Recent users: 747 (up 68)
Users with assigned work items: 1,958 (up 300)
Version control users: 1,404 (up 90)<\/font><\/font>\nWork items
<\/u><\/strong>Work items: 103,832 (down 2,000)
Areas & Iterations: 6,188 (up 100)
Work item versions: 817,086 (up 45,000)
Attached files: 32,839 (up 4,300)
Queries: 10,299 (up 300)<\/font><\/font>\nVersion control
<\/u><\/strong>Files\/Folders: 33,580,679\/5,646,201 (up 14M\/3M)
LocalVersion: 128.8M (up 18M)
Total compressed file sizes: 243.8G (up 50G)
Workspaces: 2,174 (down 530)
Shelvesets: 2,952 (down 1,100)
Checkins: 100,300 (up 19,000)
Pending changes: 876,649 (up 480,000)<\/font><\/font>\nCommands (last 7 days)
<\/u><\/strong>Work Item queries: 335,821 (up 85,000)
Work Item updates: 40,212 (up 3,500)
Work Item opens: 193,782 (up 36,000)
Gets: 25,066 (up 11,000)
Downloads: 10.9M (up 6.5M)
Checkins: 2,130 (down 1,600)
Uploads: 212,493 (up 201,000)
Shelves: 248 (down 200)<\/font><\/font>\nCheers,<\/p>\n