{"id":21273,"date":"2008-08-12T10:00:00","date_gmt":"2008-08-12T10:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2008\/08\/12\/the-implementation-of-iterators-in-c-and-its-consequences-part-1\/"},"modified":"2008-08-12T10:00:00","modified_gmt":"2008-08-12T10:00:00","slug":"the-implementation-of-iterators-in-c-and-its-consequences-part-1","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20080812-00\/?p=21273","title":{"rendered":"The implementation of iterators in C# and its consequences (part 1)"},"content":{"rendered":"<p><P>\nLike\n<A HREF=\"http:\/\/blogs.msdn.com\/oldnewthing\/archive\/2006\/08\/02\/686456.aspx\">\nanonymous methods<\/A>,\niterators in C# are very complex syntactic sugar.\nYou could do it all yourself (after all, you <I>did<\/I> have to do\nit all yourself in earlier versions of C#),\nbut the compiler transformation makes for much greater convenience.\n<\/P>\n<P>\nThe idea behind iterators is that they take a function with\n<A HREF=\"http:\/\/blogs.msdn.com\/ericgu\/archive\/2006\/03\/08\/546296.aspx\">\n<CODE>yield return<\/CODE><\/A>\nstatements\n(and possible some <CODE>yield break<\/CODE> statements)\nand convert it into a state machine.\nWhen you <CODE>yield return<\/CODE>, the state of the function is\nrecorded, and execution resumes from that state the next time the\niterator is called upon to produce another object.\n<\/P>\n<P>\nHere&#8217;s the basic idea:\nAll the local variables of the iterator (treating iterator parameters\nas pre-initialized local variables, including the hidden <CODE>this<\/CODE>\nparameter)\nbecome member variables of a helper class.\nThe helper class also has an internal <I>state<\/I> member that keeps\ntrack of where execution left off and an internal <I>current<\/I>\nmember that holds the object most recently enumerated.\n<\/P>\n<PRE>\nclass MyClass {\n int limit = 0;\n public MyClass(int limit) { this.limit = limit; }<\/p>\n<p> public IEnumerable&lt;int&gt; CountFrom(int start)\n {\n  for (int i = start; i &lt;= limit; i++) {\n   yield return i;\n  }\n }\n}\n<\/PRE>\n<P>\nThe <CODE>CountFrom<\/CODE> method produces an integer\nenumerator that spits out the integers starting at <CODE>start<\/CODE>\nand continuing up to and including <CODE>limit<\/CODE>.\nThe compiler internally converts this enumerator into\nsomething like this:\n<\/P>\n<PRE>\n <FONT COLOR=\"blue\">class MyClass_Enumerator : IEnumerable&lt;int&gt; {\n  int state$0 = 0;\/\/ internal member\n  int current$0;  \/\/ internal member\n  MyClass this$0; \/\/ implicit parameter to CountFrom\n  int start;      \/\/ explicit parameter to CountFrom\n  int i;          \/\/ local variable of CountFrom<\/p>\n<p>  public int Current {\n   get { return current$0; }\n  }<\/p>\n<p>  public bool MoveNext()\n  {\n   switch (state$0) {\n   case 0: goto resume$0;\n   case 1: goto resume$1;\n   case 2: return false;\n   }<\/p>\n<p> resume$0:;<\/FONT>\n   for (i = start; i &lt;= <FONT COLOR=\"blue\">this$0.<\/FONT>limit; i++) {\n    <FONT COLOR=\"blue\">current$0 =<\/FONT> i;\n    <FONT COLOR=\"blue\">state$0 = 1;\n    return true;\n resume$1:;<\/FONT>\n   }<\/p>\n<p>   <FONT COLOR=\"blue\">state$0 = 2;\n   return false;\n  }\n  &#8230; other bookkeeping, not important here &#8230;\n }<\/FONT><\/p>\n<p> public IEnumerable&lt;int&gt; CountFrom(int start)\n {\n  <FONT COLOR=\"blue\">MyClass_Enumerator e = new MyClass_Enumerator();\n  e.this$0 = this;\n  e.start = start;\n  return e;<\/FONT>\n }\n<\/PRE>\n<P>\nThe enumerator class is auto-generated by the compiler\nand, as promised, it contains two internal members for the\nstate and current object,\nplus a member for each parameter\n(including the hidden <CODE>this<\/CODE> parameter),\nplus a member for each local variable.\nThe <CODE>Current<\/CODE> property merely returns the current object.\nAll the real work happens in <CODE>MoveNext<\/CODE>.\n<\/P>\n<P>\nTo generate the <CODE>MoveNext<\/CODE> method, the compiler\ntakes the code you write and performs a few transformations.\nFirst, all the references to variables and parameters need to\nbe adjusted since the code moved to a helper class.\n<\/P>\n<UL>\n<LI><CODE>this<\/CODE> becomes <CODE>this$0<\/CODE>,\n    because inside the rewritten function, <CODE>this<\/CODE>\n    refers to the auto-generated class, not the original class.\n<LI><CODE>m<\/CODE> becomes <CODE>this$0.m<\/CODE> when\n    <CODE>m<\/CODE> is a member of the original class\n    (a member variable, member property, or member function).\n    This rule is actually redundant with the previous rule,\n    because writing the name of a\n    class member <CODE>m<\/CODE> without a prefix is just\n    shorthand for <CODE>this.m<\/CODE>.\n<LI><CODE>v<\/CODE> becomes <CODE>this.v<\/CODE> when\n    <CODE>v<\/CODE> is a parameter or local variable.\n    This rule is actually redundant, since writing <CODE>v<\/CODE>\n    is the same as <CODE>this.v<\/CODE>, but I call it out\n    explicitly so you&#8217;ll notice that the storage for the variable\n    has changed.\n<\/UL>\n<P>\nThe compiler also has to deal with all those <CODE>yield return<\/CODE>\nstatements.\n<\/P>\n<UL>\n<LI>Each <CODE>yield return x<\/CODE> becomes\n<PRE>\n current$0 = x;\n state$0 = n;\n return true;\nresume$n:;\n<\/PRE>\n<P>\nwhere <CODE>n<\/CODE> is an increasing number starting at 1.\n<\/UL>\n<P>\nAnd then there are the\n<CODE>yield break<\/CODE> statements.\n<\/P>\n<UL>\n<LI>Each <CODE>yield break<\/CODE> becomes\n<PRE>\n state$0 = n2;\n return false;\n<\/PRE>\nwhere <CODE>n2<\/CODE> is one greater than the highest state\nnumber used by all the <CODE>yield return<\/CODE> statements.\nDon&#8217;t forget that there is also an implied <CODE>yield break<\/CODE>\nat the end of the function.\n<\/UL>\n<P>\nFinally, the compiler puts the big state dispatcher at the top of the\nfunction.\n<\/P>\n<UL>\n<LI>At the start of the function, insert\n<PRE>\nswitch (state$0) {\ncase 0: goto resume$0;\ncase 1: goto resume$1;\ncase 2: goto resume$2;\n&#8230;\ncase n: goto resume$n;\ncase n2: return false;\n}\n<\/PRE>\n<P>\nwith one <CODE>case<\/CODE> statement for each state,\nplus the initial zero state and the final <CODE>n2<\/CODE> state.\n<\/P>\n<\/UL>\n<P>\nNotice that this transformation is quite different from\n<A HREF=\"http:\/\/blogs.msdn.com\/oldnewthing\/archive\/2004\/12\/31\/344799.aspx\">\nthe enumeration model we built based on coroutines and fibers<\/A>.\nThe C# method is far more efficient in terms of memory usage\nsince it doesn&#8217;t consume an entire stack (typically a megabyte in size)\nlike the fiber approach does.\nInstead it just borrows the stack of the caller,\nand anything that it needs to save across calls to <CODE>MoveNext<\/CODE>\nare stored in a helper object (which goes on the heap rather than the stack).\nThis fake-out is normally quite effective&mdash;most\npeople don&#8217;t even realize that it&#8217;s happening&mdash;but there are places\nwhere the difference is significant, and we&#8217;ll see that shortly.\n<\/P>\n<P>\n<B>Exercise<\/B>:\nWhy do we need to write\n<CODE>state$0 = n2;<\/CODE> and add the\n<CODE>case n2: return false;<\/CODE>?\nWhy can&#8217;t we just transform each <CODE>yield break<\/CODE>\ninto <CODE>return false;<\/CODE> and stop there?\n<\/P><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Like anonymous methods, iterators in C# are very complex syntactic sugar. You could do it all yourself (after all, you did have to do it all yourself in earlier versions of C#), but the compiler transformation makes for much greater convenience. The idea behind iterators is that they take a function with yield return statements [&hellip;]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-21273","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>Like anonymous methods, iterators in C# are very complex syntactic sugar. You could do it all yourself (after all, you did have to do it all yourself in earlier versions of C#), but the compiler transformation makes for much greater convenience. The idea behind iterators is that they take a function with yield return statements [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/21273","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=21273"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/21273\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=21273"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=21273"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=21273"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}