{"id":4501,"date":"2010-10-28T10:53:00","date_gmt":"2010-10-28T10:53:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/bharry\/2010\/10\/28\/tfs-on-windows-azure-at-the-pdc\/"},"modified":"2024-06-21T12:11:13","modified_gmt":"2024-06-21T19:11:13","slug":"tfs-on-windows-azure-at-the-pdc","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/bharry\/tfs-on-windows-azure-at-the-pdc\/","title":{"rendered":"TFS on Windows Azure at the PDC"},"content":{"rendered":"<p>**UPDATED May 18th 2001** &#8211; See <a href=\"http:\/\/blogs.msdn.com\/b\/bharry\/archive\/2011\/05\/18\/update-on-tfs-on-azure.aspx\">http:\/\/blogs.msdn.com\/b\/bharry\/archive\/2011\/05\/18\/update-on-tfs-on-azure.aspx<\/a><\/p>\n<p>Hosting of ALM in the cloud as software as a service is gradually becoming more and more popular.\u00a0 The vision, of course, is ALM as a seamless service \u2013 making it really easy to get started, easy to scale, easy to operate, easy to access, \u2026\u00a0 You\u2019ve seen me write from time to time about our work with <a href=\"http:\/\/www.microsoft.com\/visualstudio\/en-us\/products\/2010-editions\/team-foundation-server\/hosting\">3rd party hosting and consulting companies offering TFS services<\/a>.\u00a0 We did a bunch of work in TFS 2010 on both the technical and licensing front to enable a new generation of cloud based TFS services.<\/p>\n<p>Several months ago, I wrote a <a href=\"http:\/\/blogs.msdn.com\/b\/bharry\/archive\/2010\/04\/18\/tfs-and-azure.aspx\">post<\/a> about our initial investigation into porting TFS to the Windows Azure platform.\u00a0 Since then, we\u2019ve continued to pursue it and today, at this year\u2019s PDC, I demoed Team Foundation Server running on the Windows Azure platform.\u00a0 We announced that we\u2019ll deliver a CTP (basically an early preview) next year.\u00a0 We aren\u2019t, by any means, done with the technical work, but, for now,\u00a0 it\u2019s a great case study to see what is involved in porting a large scale multi-tier enterprise application to Azure.<\/p>\n<p>The demo I did today represents an important step forward in getting TFS running on the Azure cloud platform.\u00a0 When I wrote my post a few months ago, I talked about a few of the challenges porting the TFS data tier from SQL Server to SQL Azure.\u00a0 What I demoed today included not only that but also the remaining components of TFS running in the cloud \u2013 the ASP.NET application tier (running as a Web Role), the TFS Job Service (formerly a Windows service for periodic background tasks, now running as a worker role) and the TFS Build controller\/agent (running in an Azure VM role).\u00a0 I demoed connecting from a web browser, Visual Studio and from Microsoft Test Manager.<\/p>\n<p>One of the cool things (but makes for a mundane demo :)) is that, for the end user, TFS in the cloud looks pretty much like TFS on-premises.\u00a0 Other than the fact that you can log in with an internet identity rather than a windows identity, you\u2019ll find that the Visual Studio experience, for example, looks pretty much identical to a local TFS scenario.\u00a0 Here\u2019s a screenshot of the \u201cnew experience\u201d of logging in with a internet id:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/8400.TFSintheCloudNewProject2_thumb_48FCF252.png\"><img decoding=\"async\" class=\"alignnone size-full wp-image-16314\" src=\"https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/8400.TFSintheCloudNewProject2_thumb_48FCF252.png\" alt=\"Image 8400 TFSintheCloudNewProject2 thumb 48FCF252\" width=\"804\" height=\"454\" srcset=\"https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/8400.TFSintheCloudNewProject2_thumb_48FCF252.png 804w, https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/8400.TFSintheCloudNewProject2_thumb_48FCF252-300x169.png 300w, https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/8400.TFSintheCloudNewProject2_thumb_48FCF252-768x434.png 768w\" sizes=\"(max-width: 804px) 100vw, 804px\" \/><\/a><\/p>\n<p>And here\u2019s a screenshot of VS connected to TFS on Azure (as you can see, there\u2019s not much difference):<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/3632.TFSintheCloudVS_thumb_140FED15.png\"><img decoding=\"async\" class=\"alignnone size-full wp-image-16315\" src=\"https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/3632.TFSintheCloudVS_thumb_140FED15.png\" alt=\"Image 3632 TFSintheCloudVS thumb 140FED15\" width=\"804\" height=\"454\" srcset=\"https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/3632.TFSintheCloudVS_thumb_140FED15.png 804w, https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/3632.TFSintheCloudVS_thumb_140FED15-300x169.png 300w, https:\/\/devblogs.microsoft.com\/bharry\/wp-content\/uploads\/sites\/8\/2010\/10\/3632.TFSintheCloudVS_thumb_140FED15-768x434.png 768w\" sizes=\"(max-width: 804px) 100vw, 804px\" \/><\/a><\/p>\n<p>The good news is that a lot of the work we did in TFS 2010 to improve the TFS architecture to scale from simple client installs (TFS Basic) to very complex, centrally hosted enterprise installs really helped prepare us for the port to Azure.\u00a0 The result is that it has been a pretty reasonable project.\u00a0 I\u2019ll describe some aspects of the effort.<\/p>\n<h3>Porting TFS to SQL Azure<\/h3>\n<p>As I mentioned, step 1 was getting the SQL ported to SQL Azure.\u00a0 TFS is a very large SQL app and therefore, we feared this would not be a simple task.\u00a0 TFS has over 900 stored procedures and over 250 tables.\u00a0 Despite how involved the app is, the system was up and running with an investment of 2 people for about 1 month.\u00a0 The biggest issues we had to deal with were:<\/p>\n<ul>\n<li>OpenXML \u2013 SQL Azure does not support the OpenXML mechanism that SQL 2008 does for passing lots of data to a stored procedure.\u00a0 Our initial port was to move from OpenXML to the XML data type.\u00a0 However, we\u2019ve found that to not be the best solution (some performance issues and XML escaping issues) and ultimately ported to Table Valued Parameters (TVPs).<\/li>\n<li>Select INTO \u2013 SQL Azure does not support \u201cselect into\u201d because it requires that all tables have a clustered index and select into doesn\u2019t allow for that.\u00a0 We ported all of our select into occurrences to use explicit temp tables.<\/li>\n<li>Sysmessages \u2013 SQL Azure doesn\u2019t allow you to add messages to sysmessages.\u00a0 This means you can\u2019t properly raise custom errors (something we make heavy use of).\u00a0 The truth is, even if you could, it wouldn\u2019t be a great solution because sysmessages doesn\u2019t really have a good localization store in a multi-tenant environment with different tenants with different languages.\u00a0 We have created a new mechanism for raising errors from SQL and ensuring that the end user ultimately gets a localized, intelligible message.<\/li>\n<li>Clustered indexes \u2013 As I mentioned above, SQL Azure requires clustered indexes on all tables.\u00a0 Some of ours didn\u2019t have them.\u00a0 It&#8217; was pretty straight forward to go back and add them though.<\/li>\n<li>No full text index \u2013 SQL Azure does not yet support full text indexing.\u00a0 For now we\u2019ve disabled it in TFS (the only place we used it was in work item tracking) and are figuring out what our long term plan for that will be.<\/li>\n<li>Database directives \u2013 There are a lot of database directives that we use that SQL doesn\u2019t support \u2013 partitioning, replication, filegroups, etc.\u00a0 Removing them was not a particularly big issue.<\/li>\n<\/ul>\n<p>While this is not an exhaustive list of the issues, it\u2019s a pretty good list of the larger ones we had to deal with.<\/p>\n<h3>Porting to the Web Role<\/h3>\n<p>While the TFS app tier is also pretty big, a straight port of it was surprisingly simple \u2013 about 1 person, 1 month.\u00a0 Azure\u2019s implementation of ASP.NET is pretty faithful to the on-premises solution so most stuff ports pretty well.\u00a0 The harder parts of this (that have taken a few people several months) have been adapting to some of the cloud platform differences \u2013 with identity being the biggest one.\u00a0 Here\u2019s some detail\u2026<\/p>\n<ul>\n<li>Identity \u2013 Windows Azure doesn\u2019t have Windows identities for use in the same way you would use them in an on-premises app.\u00a0 You need to use an internet identity system.\u00a0 Our first cut at this was to attempt direct use of LiveID \u2013 um, it turned out to be much more complicated than we originally expected.\u00a0 After getting our head out of the sand, we realized the right thing was to use App Fabric Access Control Services (otherwise known as ACS).\u00a0 ACS gives us an authentication system that is OpenID compatible and supports many providers, such as LiveID, Yahoo, Google, Facebook, and Active Directory Federation.\u00a0 This enables people to connect to our Azure based TFS service with whatever identity provider they choose.\u00a0 TFS was pretty baked to understand Windows identities so it was a fair amount of work to port and had some pretty non-obvious ramifications.\u00a0 For instance, we can\u2019t put up a username\/password dialog any more.\u00a0 These services (LiveID, etc) require that you use their web page to authenticate.\u00a0 That means that everywhere we used to get Windows credentials, we now have to pop up a web page to enable the user to enter their username\/password with their preferred provider.\u00a0 From a technical stand point, identity has been the most disruptive change.<\/li>\n<li>Service identities \u2013 Not only do we have to deal with user identities (as described above) but TFS also has a number of cooperative services and they need to authenticate with each other.\u00a0 The end user oriented authentication mechanisms don\u2019t really support this.\u00a0 Fortunately, ACS has introduced something called \u201cService identities\u201d for this exact purpose.\u00a0 They enable headless services to authenticate\/authorize each other using ACS as the broker.<\/li>\n<li>Calls from the cloud \u2013 The relationship between a TFS server and a TFS build controller (in TFS 2010 and before) is essentially peer to peer.\u00a0 Connections can be initiated from either side.\u00a0 This is potentially a big issue if you want your TFS server in the cloud and a build machine potentially on the other side of a Network Address Translation (NAT) device.\u00a0 One option is to use the Azure Service Bus to broker the bi-directional communication.\u00a0 We looked at our alternatives and ultimately decided to change the communication protocol so that the build controller acts like a pure client \u2013 it contacts the TFS server but the TFS server never contacts it.\u00a0 This is going to allow us, long term, to have simpler trust relationships between them and simplify management.<\/li>\n<li>Deployment \u2013 Another sizable chunk of work has been changing TFS to deploy properly to Azure.\u00a0 TFS used an MSI, wrote to the Registry, the GAC, etc.\u00a0 None of this is allowed on Azure \u2013 Azure apps are deployed as a zip file with a manifest that basically controls where files go.<\/li>\n<li>Tracing \u2013 We had work to do to hook our tracing (System.Diagnostics) and event logging into the Azure diagnostics store \u2013 neither of these is automatic.<\/li>\n<li>Admin console \u2013 The admin console had cases where it accessed the TFS database directly but that\u2019s not very desirable in Azure.\u00a0 The SQL Azure authentication system is different.\u00a0 We didn\u2019t want to complicate the world by exposing two authentication systems to administrators so we\u2019ve eliminated the direct database access from the admin console and replaced it with web service calls.<\/li>\n<\/ul>\n<p>Again, this isn\u2019t a comprehensive list but captures many of the largest things we\u2019ve had to do to get TFS on Azure.<\/p>\n<h3>Tuning for Azure<\/h3>\n<p>There\u2019s a class of work we are doing that I call \u201cTuning for Azure\u201d and by this, I mean, it\u2019s not strictly necessary to provide a TFS service on Azure but it makes the service more practical.\u00a0 My best examples off the top of my head are:<\/p>\n<ul>\n<li>Blob storage \u2013 In our on-premises solution, all TFS data is stored in SQL Server \u2013 work item attachments, source code files, Intellitrace logs, \u2026\u00a0 This turns out not to be a good choice on Azure \u2013 really for 2 reasons: 1) SQL Azure databases are capped at 50GB (might change in the future, but that\u2019s the limit today) and 2) SQL Azure storage is more than 50X more expensive per GB than Windows Azure storage.\u00a0 As a result we\u2019ve done work to move all of our blob data out of SQL and into a blob storage service that can be hosted on Windows Azure blobs.\u00a0 We believe this will both significantly reduce the pressure on the database size limit and reduce our operational cost.<\/li>\n<li>Latency and bandwidth \u2013 The fact that your TFS server will always be across the WAN means you need to think even harder about how you use it.\u00a0 As much as can be local needs to be local.\u00a0 Calls to the server really need to be async and cancelable.\u00a0 You have to really minimize round trips and payload size, etc.<\/li>\n<\/ul>\n<h3>Running as a Service<\/h3>\n<p>The biggest category of work we have left is stuff I call \u201cRunning as a Service\u201d.\u00a0 Most all of this applies to running a traditional on-premises 3 tier app as a service but it becomes more important the larger the scale of the service is and a worldwide service like Azure makes it really important.\u00a0 An example of something in this category is upgrades.\u00a0 In TFS today, when you upgrade your TFS version, you take the server down, do the upgrade and then bring the server back up.\u00a0 For a very large on-premises installation, that might entail a few hours of down time up to about a day (for crazy big databases).\u00a0 For a service the size of a global service, you can\u2019t take the service down for days (or even hours) while you upgrade it.\u00a0 It\u2019s got to be 24&#215;7 with minimal interruptions for small segments of the population.\u00a0 That means that all upgrades of the TFS service have to be \u201conline\u201d without ever taking a single customer down longer than it takes to update their own data.\u00a0 And that means running multiple versions of the TFS simultaneously during the upgrade window.\u00a0 It\u2019s by no means the only big \u201crunning as a service\u201d investment but it\u2019s a good example of how expectations of this kind of service are just different.<\/p>\n<p>We also haven\u2019t yet tackled what billing would look like for this kind of online service.\u00a0 That can also be way more complicated than you might, at first, think.<\/p>\n<p>That\u2019s a lot of detail but it\u2019s a pretty cool milestone to be able to demo the great progress we\u2019ve made and share our experiences porting a large existing on-premises app to a cloud architecture.\u00a0 As always, it\u2019s pretty fun to be messing around with the latest and greatest technology.\u00a0 As I said at the beginning, we\u2019re still primarily looking at this as technological exercise and aren\u2019t ready to talk about any product plans.\u00a0 For now, TFS provides a great on-premises solution and we have a growing set of partners providing hosted services for those that want to go that route.\u00a0 The move to the cloud is certainly gaining momentum and we\u2019re making sure that your investment in TFS today has a clear path to get you there in the future.<\/p>\n<p>Thanks,<\/p>\n<p>Brian<\/p>\n","protected":false},"excerpt":{"rendered":"<p>**UPDATED May 18th 2001** &#8211; See http:\/\/blogs.msdn.com\/b\/bharry\/archive\/2011\/05\/18\/update-on-tfs-on-azure.aspx Hosting of ALM in the cloud as software as a service is gradually becoming more and more popular.\u00a0 The vision, of course, is ALM as a seamless service \u2013 making it really easy to get started, easy to scale, easy to operate, easy to access, \u2026\u00a0 You\u2019ve seen [&hellip;]<\/p>\n","protected":false},"author":244,"featured_media":14617,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[5],"class_list":["post-4501","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-tfs"],"acf":[],"blog_post_summary":"<p>**UPDATED May 18th 2001** &#8211; See http:\/\/blogs.msdn.com\/b\/bharry\/archive\/2011\/05\/18\/update-on-tfs-on-azure.aspx Hosting of ALM in the cloud as software as a service is gradually becoming more and more popular.\u00a0 The vision, of course, is ALM as a seamless service \u2013 making it really easy to get started, easy to scale, easy to operate, easy to access, \u2026\u00a0 You\u2019ve seen [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts\/4501","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/users\/244"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/comments?post=4501"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts\/4501\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/media\/14617"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/media?parent=4501"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/categories?post=4501"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/tags?post=4501"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}