{"id":18543,"date":"2020-07-31T09:46:05","date_gmt":"2020-07-31T17:46:05","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/powershell\/?p=18543"},"modified":"2020-07-31T09:46:05","modified_gmt":"2020-07-31T17:46:05","slug":"semantic-highlighting-in-the-powershell-preview-extension-for-visual-studio-code","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/powershell\/semantic-highlighting-in-the-powershell-preview-extension-for-visual-studio-code\/","title":{"rendered":"Semantic Highlighting in the PowerShell Preview extension for Visual Studio Code"},"content":{"rendered":"<h2>Semantic Highlighting in the PowerShell Preview extension for Visual Studio Code<\/h2>\n<p>Hi everyone!\nI&#8217;m Justin and I am currently an intern on the PowerShell team.\nOne of my projects was to add PowerShell semantic highlighting support in VS Code allowing for more accurate highlighting in the editor.\nI&#8217;m excited to share that the first iteration has been released.<\/p>\n<h2>Getting started<\/h2>\n<p>Great news!\nYou don&#8217;t have to do anything to get this feature except for making sure you have at least the\n<code>v2020.7.0<\/code> version of the\n<a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=ms-vscode.PowerShell-Preview\">PowerShell Preview extension for Visual Studio Code<\/a>.<\/p>\n<blockquote>\n<h3>IMPORTANT<\/h3>\n<p>You have to use a theme that supports Semantic Highlighting.\nAll the inbox themes support it and the PowerShell ISE theme supports it but it&#8217;s not guaranteed that every theme will.\nIf you don&#8217;t see any difference in highlighting,\nthe theme you&#8217;re using probably doesn&#8217;t support it.\nOpen an issue on the theme you&#8217;re using to support Semantic Highlighting.<\/p><\/blockquote>\n<h3>For theme authors: Supporting Semantic Highlighting<\/h3>\n<p>If you are a theme author, make sure to add <code>{semanticHighlighting: true}<\/code> to the\ntheme.json file of your VS Code theme.<\/p>\n<p>For a more complete guide into supporting Semantic Highlighting in your theme,\nplease look at:<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/microsoft\/vscode\/wiki\/Semantic-Highlighting-Overview\">The Semantic Highlighting Overview<\/a><\/li>\n<li><a href=\"https:\/\/code.visualstudio.com\/api\/language-extensions\/semantic-highlight-guide\">The Semantic Highlighting Guide<\/a><\/li>\n<\/ul>\n<p>The rest of this blog post will discuss the shortcomings of the old syntax\nhighlighting mechanism and how semantic highlighting addresses those issues.<\/p>\n<h2>Syntax Highlighting<\/h2>\n<p>Currently, the syntax highlighting support for PowerShell scripts in VS Code leverages\n<a href=\"https:\/\/macromates.com\/manual\/en\/language_grammars\">TextMate grammars<\/a>, which are mappings\nof regular expressions to tokens. For instance, to identify control keywords, something like\nthe following would be used<\/p>\n<pre><code class=\"text\">{\r\n    name = 'keyword.control.untitled';\r\n    match = 'b(if|while|for|return)b';\r\n}\r\n<\/code><\/pre>\n<p>However, there are some limitations with regular expressions and their ability to recognize different syntax patterns.\nSince TextMate grammars rely on these expressions,\nthere are many complex and context-dependent tokens these grammars are unable to parse,\nleading to inconsistent or incorrect highlighting.\nJust skim through the issues in the\n<a href=\"https:\/\/github.com\/PowerShell\/EditorSyntax\/issues\">EditorSyntax repo<\/a>,\nour TextMate grammar.<\/p>\n<p>Here are a few examples where syntax highlighting fails in\ntokenizing a PowerShell script.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/powershell\/wp-content\/uploads\/sites\/30\/2020\/07\/syntax-highlighting.png\" alt=\"Syntax Highlighting Bugs\" \/><\/p>\n<h2>Semantic Highlighting<\/h2>\n<p>To solve those cases\n(and many other ones)\nwe use the PowerShell tokenizer\nwhich describes the tokens more accurately than regular expressions can,\nwhile also always being up-to-date with the language grammar.\nThe only problem is that the tokens generated by the PowerShell tokenizer do not align perfectly to the semantic token types predefined by VS Code.\nThe\n<a href=\"https:\/\/code.visualstudio.com\/api\/language-extensions\/semantic-highlight-guide\">semantic token types<\/a> provided by VS Code are:<\/p>\n<ul>\n<li>namespace<\/li>\n<li>type, class, enum, interface, struct, typeParameter<\/li>\n<li>parameter, variable, property, enumMember, event<\/li>\n<li>function, member, macro<\/li>\n<li>label<\/li>\n<li>comment, string, keyword, number, regexp, operator<\/li>\n<\/ul>\n<p>On the other hand, there are over 100\n<a href=\"https:\/\/docs.microsoft.com\/dotnet\/api\/system.management.automation.language.tokenkind\">PowerShell token kinds<\/a>\nand also many\n<a href=\"https:\/\/docs.microsoft.com\/dotnet\/api\/system.management.automation.language.tokenflags\">token flags<\/a>\nthat can modify those types.<\/p>\n<p>The main task (aside from setting up a semantic tokenization\nhandler) was to create a mapping from PowerShell tokens to VS Code semantic token types. The\nresult of enabling semantic highlighting can be seen below.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/powershell\/wp-content\/uploads\/sites\/30\/2020\/07\/semantic-highlighting.png\" alt=\"Semantic Highlighting Examples\" \/><\/p>\n<p>If we compare the semantic highlighting to the highlighting in PowerShell ISE, we can see they are quite similar (in tokenization, not color).<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/powershell\/wp-content\/uploads\/sites\/30\/2020\/07\/powershellISE.png\" alt=\"PowerShell ISE Screenshot\" \/><\/p>\n<h2>Next Steps<\/h2>\n<p>Although semantic highlighting does a better job than syntax highlighting in identifying tokens,\nthere remain some cases that can still be improved at the PowerShell layer.<\/p>\n<p>In Example 5, for instance, while the enum does have better highlighting, the name and members\nof the enums are highlighted identically. This occurs because PowerShell tokenizes them\nall of them the same way (as identifiers with a token flags denoting that they are member names meaning that the semantic highlighting has no way to differentiate them.<\/p>\n<h2>How to Provide Feedback<\/h2>\n<p>If you experience any issues or have comments on improvement, please raise an issue in\n<a href=\"https:\/\/github.com\/PowerShell\/vscode-powershell\">PowerShell\/vscode-powershell<\/a>. Since this was\njust released, any feedback will be greatly appreciated.<\/p>\n<p>Justin Chen\nPowerShell Team<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Semantic Highlighting now supported in VS Code for PowerShell scripts<\/p>\n","protected":false},"author":35848,"featured_media":13641,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[248],"class_list":["post-18543","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-powershell","tag-powershell"],"acf":[],"blog_post_summary":"<p>Semantic Highlighting now supported in VS Code for PowerShell scripts<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts\/18543","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/users\/35848"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/comments?post=18543"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts\/18543\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/media\/13641"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/media?parent=18543"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/categories?post=18543"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/tags?post=18543"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}