{"id":935,"date":"2021-06-07T08:00:54","date_gmt":"2021-06-07T07:00:54","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/sustainable-software\/?p=935"},"modified":"2021-06-17T15:39:32","modified_gmt":"2021-06-17T14:39:32","slug":"the-current-state-of-affairs-and-a-roadmap-for-effective-carbon-accounting-tooling-in-ai","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/sustainable-software\/the-current-state-of-affairs-and-a-roadmap-for-effective-carbon-accounting-tooling-in-ai\/","title":{"rendered":"The current state of affairs and a roadmap for effective carbon-accounting tooling in AI"},"content":{"rendered":"

The societal case for carbon accounting of AI systems<\/span><\/h3>\n

 <\/p>\n

Digital services consume a lot of energy<\/span><\/a> and it goes without saying that in a world with accelerating climate change, we must be conscious in all parts of life with our carbon footprints. In the case of the software that we write, specifically, the AI systems we build, these considerations become even more important because of the <\/span>large upfront computational resources that training some large AI models consume<\/span><\/a>, and the subsequent carbon emissions resulting from it. Thus, effective carbon accounting for artificial intelligence systems is critical!<\/span><\/p>\n

 <\/p>\n

\"Image<\/a>
Carbon accounting for artificial intelligence systems will be essential to build a more sustainable software engineering practice<\/figcaption><\/figure><\/p>\n

 <\/p>\n

An equally important consideration is to think about the social inequalities perpetuated by the emphasis on large-scale models. It encourages a winner-take-all dynamic in the AI research ecosystem where only those who have access to large-scale compute and data infrastructure, typically well-funded industry and academic labs are able to embark on such research. Consequently, by being able to do so, they are able to publish results and secure more funding feeding the cycle further.\u00a0<\/span><\/p>\n

Ultimately, for any system that we build, there is an investment of resources: human, computational, and financial. We need to be able to assess more accurately the costs and benefits of utilizing different approaches. For example, <\/span>neural architecture search<\/span><\/a> (NAS) can help find more efficient architectures that we would not find through human-guided and targeted search and tuning. While NAS can be computationally expensive, if it yields a solution that is computationally- and carbon-efficient, and it is a system that is run many times over, the potential long-run savings can outweigh the short-run costs.\u00a0<\/span><\/p>\n

To be able to assess all of this, we need to have a more accurate cost estimation. While we are able to do so quite well for the <\/span>financial side of things<\/span><\/a>, getting a better handle on the carbon side of things will further clarify our trade-off analyses.\u00a0<\/span><\/p>\n

 <\/p>\n

The current state of affairs of carbon-accounting tooling<\/span><\/h3>\n

 <\/p>\n

Most of the tooling is split into two factions at the moment: web-based and code-based tools.\u00a0<\/span><\/p>\n

Web-based tools like <\/span>MLCO2 calculator<\/span><\/a> offer practitioners an option where after the training phase is complete, they can enter information about the cloud provider, the region where resources are hosted, training time, and type of hardware to estimate the CO2eq that were potentially emitted as a part of the training process of your AI system.\u00a0<\/span><\/p>\n

Code-based tools like <\/span>energyusage<\/span><\/a> and <\/span>CodeCarbon<\/span><\/a> provide developers the option of integrating snippets of code, akin to how you would track artifacts in your machine learning workflow with tools like MLFlow, to capture the energy consumption of different functions within your code. These are later converted to CO2eq by the tools to provide a comparable metric.\u00a0<\/span><\/p>\n

Unfortunately, due to the recency and general lack of awareness, the uptake of these tools has been quite limited. This is evidenced by both the number of downloads and stars for the code repositories on GitHub, but also from anecdotal evidence. You\u2019d be hard-pressed to find fellow practitioners engaging with these tools in their day-to-day work. It continues to remain an abstract discussion and a distant concern, though the tide is shifting.<\/span><\/p>\n

 <\/p>\n

Where we\u2019re headed and what\u2019s happening\u00a0\u00a0\u00a0<\/span><\/h3>\n

 <\/p>\n

Introducing friction into workflows is a proven way to kill the adoption of a new process. The initial crop of tooling in this space, which was primarily web-based, was plagued by this problem. Just as documentation creation and maintenance is challenging, similarly having to enter information in a separate portal to calculate the carbon footprint breaks the natural workflow of a developer. The trend inching towards code-based tools is a positive step in that direction as it integrates more naturally into existing workflows. The latest integration that CodeCarbon offers with <\/span>CometML<\/span><\/a> is an example of this.\u00a0<\/span><\/p>\n

In addition, there is a <\/span>rising awareness<\/span><\/a> that our work in AI can have significant environmental impacts, which is driving this to become a topic of discussion and research at various top-tier machine learning conferences like NeurIPS, ICML, ICLR, ACL, etc.\u00a0<\/span><\/p>\n

But, not all is great!\u00a0<\/span><\/p>\n

We are currently at an inflection point: we need to arrive at a consensus that will help us agree on metrics and build tooling that aids in comparability. This is going to be critical if this movement is to be successful in the near future.\u00a0<\/span><\/p>\n

 <\/p>\n

\"The<\/a>
The current state of carbon accounting for AI systems<\/figcaption><\/figure><\/p>\n

 <\/p>\n

To start with, there is a lot of debate currently in terms of what might be the most effective way to capture the environmental impact of AI systems. There are some approaches that ask us to look at the entire lifecycle of all the products (hardware and software) that go into making an AI system. While comprehensive, these are hard to manifest accurately in practice. There is also debate on whether floating point operations, power consumption values from GPUs and CPUs, or other measures should be used to arrive at the CO2eq value that can be used to compare different systems. Most approaches today focus just on the training aspect of AI systems. But, if a model is going to be used for inference millions of times, that impact can quickly add up to be significant and rival the financial and environmental cost of training the system.\u00a0<\/span><\/p>\n

Finally, at the moment, there is a lack of actionable insight from the tools, both web- and code-based, that can help to trigger behavior change in the developers. This is the place we want to get to without which we would end up in a fruitless cycle of creating additional documentation that doesn\u2019t lead to any action.\u00a0<\/span><\/p>\n

 <\/p>\n

Let\u2019s try and cross the chasm\u00a0<\/span><\/h3>\n

 <\/p>\n

For those familiar with <\/span>Geoffrey Moore\u2019s Crossing the Chasm<\/span><\/a>, in the product adoption and diffusion lifecycle, there is a chasm that exists between the \u201cearly adopters\u201d and the \u201cearly majority\u201d. We are currently just past the \u201cinnovators” mark with the idea of carbon-accounting for AI systems even though the idea for the carbon impact of computing has been around for many years. To achieve a meaningful impact from embarking on this journey in the first place, we need to get to the stage of the late majority at which point the practice of carbon accounting for AI systems will become normalized.\u00a0<\/span><\/p>\n

But, most certainly we will have to cross the chasm on that journey and it will require us to borrow lessons from good product and tool design to be able to make that leap. In a <\/span>saturated tooling world, especially with the onslaught of MLOps<\/span><\/a>, adding more to the plates of developers will only aggravate the problem of low traction.\u00a0<\/span><\/p>\n

My prognosis for us to get there is that we will need to focus on the following items:\u00a0<\/span><\/p>\n

    \n
  1. Workflow-native tooling\u00a0<\/span><\/li>\n
  2. Comparability of metrics\u00a0<\/span><\/li>\n
  3. Verifiability of claims made about the carbon figures\u00a0<\/span><\/li>\n
  4. Standardization of metrics and reporting<\/span><\/li>\n
  5. Certification<\/span><\/li>\n<\/ol>\n

    We are making progress towards #4 through the recently launched <\/span>Green Software Foundation<\/span><\/a>. In addition, I\u2019ve done some work on #5 as a part of the <\/span>Montreal AI Ethics Institute<\/span><\/a> titled <\/span>SECure: A Social and Environmental Certificate for AI Systems<\/span><\/a>. But, there is a lot more to be done. The best way forward is to start implementing some of these ideas in practice, borrowing ideas liberally from the broader carbon accounting community, and testing and iterating on those ideas to tease out pieces that work well and those that don\u2019t.\u00a0\u00a0<\/span><\/p>\n

    I\u2019d encourage you to <\/span>reach out to me<\/span><\/a> or the SSE team at Microsoft if you\u2019re working on something that can address any of the above points or if you have ideas on what can be done to improve the state of carbon accounting for AI systems.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

    The societal case for carbon accounting of AI systems   Digital services consume a lot of energy and it goes without saying that in a world with accelerating climate change, we must be conscious in all parts of life with our carbon footprints. In the case of the software that we write, specifically, the AI […]<\/p>\n","protected":false},"author":57074,"featured_media":946,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[22],"tags":[83,146,74,73,80,120,68,127,94,86,121,84,70,24,62,82],"class_list":["post-935","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sustainable-software-engineering","tag-ai","tag-carbon-accounting","tag-carbon-intensity","tag-carbon-aware","tag-cloud","tag-deep-learning","tag-electricity","tag-energy","tag-gpu","tag-greenai","tag-machine-learning","tag-ml","tag-power-measurement","tag-sse","tag-sustainability","tag-sustainable-software-engineering"],"acf":[],"blog_post_summary":"

    The societal case for carbon accounting of AI systems   Digital services consume a lot of energy and it goes without saying that in a world with accelerating climate change, we must be conscious in all parts of life with our carbon footprints. In the case of the software that we write, specifically, the AI […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/posts\/935","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/users\/57074"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/comments?post=935"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/posts\/935\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/media\/946"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/media?parent=935"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/categories?post=935"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/sustainable-software\/wp-json\/wp\/v2\/tags?post=935"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}