<\/p>\n
<\/p>\n
<\/p>\n
I\u2019m sure most of you are familiar with the powerful tools for text parsing available in PowerShell.\u00a0A presentation<\/a> at the PowerShell Summit a couple of weeks ago provides a good overview of these and mentions a new Powershell cmdlet, ConvertFrom-String, that was introduced in Windows Management Framework 5.0 Preview September 2014<\/span><\/a>.\u00a0ConvertFrom-String lets you parse a file by providing a template that contains examples of the desired output data rather than by writing a (potentially complex) script.<\/span><\/p>\n <\/p>\n <\/p>\n The namesAndCities.input.txt attached to this post contains simple names together with cities, and namesAndCities.namesOnly.template.txt copies the first two records and wraps their names in template markup:<\/span><\/p>\n <\/p>\n {Name*:Craig Trudeau} <\/span><\/p>\n <\/p>\n Buffalo, NY<\/span><\/p>\n <\/p>\n \u00a0<\/span><\/p>\n <\/p>\n {Name*:Merle Baldridge} <\/span><\/p>\n <\/p>\n Baltimore, MD<\/span><\/p>\n <\/p>\n \u00a0<\/span><\/p>\n <\/p>\n This defines the extraction of all names from the file. (In this case a single example would have worked due to the distinct formatting of the lines, but in general it is better to supply two examples to give FlashExtract a better idea of the context.) Now let\u2019s run it:<\/span><\/p>\n \n <\/p>\n gc .\\namesAndCities.input.txt | ConvertFrom-String -templateFile .\\namesAndCities.namesOnly.template.txt <\/span><\/p>\n <\/p>\n \u00a0<\/span><\/p>\n <\/p>\n ExtentText\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Name<\/span><\/span><\/span><\/p>\n <\/p>\n ———-\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 —-<\/span><\/span><\/span><\/p>\n <\/p>\n Craig Trudeau …\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Craig Trudeau<\/span><\/span><\/span><\/p>\n <\/p>\n Merle Baldridge …\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Merle Baldridge<\/span><\/span><\/span><\/p>\n <\/p>\n Vicente Saul …\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Vicente Saul<\/span><\/span><\/span><\/p>\n <\/p>\n Lydia Parsons …\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Lydia Parsons<\/span><\/span><\/span><\/p>\n <\/p>\n Cheryl Booth …\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Cheryl Booth<\/span><\/span><\/span><\/p>\n <\/p>\n Shannon Holland …\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Shannon Holland<\/span><\/span><\/span><\/p>\n <\/p>\n Libby Stevens …\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Libby Stevens<\/span><\/span><\/span><\/p>\n <\/p>\n Thomas Donnelly …\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Thomas Donnelly<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/span><\/span><\/p>\n <\/p>\n \u00a0<\/span><\/p>\n <\/p>\n The rest of this post describes how ConvertFrom-String works and develops a more full-featured address file and templates for these addresses. I\u2019ll also describe some ways to figure out what to change when you don\u2019t get the results you want.<\/span><\/p>\n <\/p>\n <\/p>\n ConvertFrom-String is built on top of FlashExtract<\/a>, a program-synthesis technology developed by Microsoft Research. FlashExtract uses an improved version of the substring-extraction techniques that were developed in Flash Fill<\/a>, which ships in Excel 2013<\/a>.\u00a0 In Flash Fill, those substrings are extracted from one or more source strings and combined into a target string. FlashExtract learns substrings to perform a top-down partitioning of the file into regions that are either nested or completely non-overlapping, and then to extract the contents of some subset of those regions as the desired output strings. In ConvertFrom-String, the regions are defined by examples in the template markup, and the substrings that are extracted by FlashExtract become the values of properties on a sequence of output objects.<\/span><\/p>\n <\/p>\n The program synthesis in FlashExtract is based upon analyzing the substrings surrounding the beginning and ending of each example region and generating programs that are combinations of various primitive string operations such as regular expressions. For each region, it finds the set of these programs that are consistent with all examples for that region and ranks them. The combination of the best-ranking sub-program for each region becomes the final FlashExtract program.<\/span><\/p>\n <\/p>\n <\/p>\n Now let\u2019s look at a more realistic address file and template. Here are the first two examples in addresses.PersonInfo.template.txt:<\/span><\/p>\n <\/p>\n {PersonInfo*:{Name:Craig Trudeau} <\/span><\/p>\n <\/p>\n {Address:{Street:4567 Main St NE}<\/span><\/p>\n <\/p>\n {[string]City:Buffalo}, {State:NY} {Zip:98052}}<\/span><\/p>\n <\/p>\n {Phone:(425) 555-0100}}<\/span><\/p>\n <\/p>\n \u00a0<\/span><\/p>\n <\/p>\n {PersonInfo*:{Name:Merle Baldridge} <\/span><\/p>\n <\/p>\n {Address:{Street:1234 First Ave}<\/span><\/p>\n <\/p>\n {City:Baltimore}, {State:MD} {Zip:98101}}<\/span><\/p>\n <\/p>\n {Phone:(425) 555-0101}}<\/span><\/p>\n <\/p>\n \u00a0<\/span><\/p>\n <\/p>\n In this template we define a hierarchy with the PersonInfo structure defined at the highest level and within this the individual fields of the structure, including another structure for Address.\u00a0 In a bit more detail we are defining:<\/span><\/p>\n <\/p>\nA Simple Example<\/span><\/h2>\n
How ConvertFrom-String Works<\/span><\/h2>\n
Defining a Structure and Fields<\/span><\/h2>\n
\n