{"id":77751,"date":"2016-04-12T00:01:15","date_gmt":"2016-04-12T07:01:15","guid":{"rendered":"https:\/\/blogs.technet.microsoft.com\/heyscriptingguy\/?p=77751"},"modified":"2019-02-18T09:10:50","modified_gmt":"2019-02-18T16:10:50","slug":"read-a-text-tile-and-do-frequency-analysis-using-powershell","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/scripting\/read-a-text-tile-and-do-frequency-analysis-using-powershell\/","title":{"rendered":"Read a text file and do frequency analysis by using PowerShell"},"content":{"rendered":"

Summary<\/strong>: Learn how to read a text file and do a letter-frequency analysis using Windows PowerShell in this article written by the Microsoft Scripting Guy, Ed Wilson.<\/p>\n

This is the third post in a multi-part series of blog posts that deal with how to determine letter frequency in text files. To fully understand this post, you should read the entire series in order.<\/p>\n

Here are the posts in the series:<\/p>\n

    \n
  1. Letter frequency analysis of text by using PowerShell<\/a><\/li>\n
  2. How to skip the beginning and ending of a file with PowerShell<\/a><\/li>\n
  3. Read a text file and do frequency analysis by using PowerShell<\/li>\n
  4. Compare the letter frequency of two text files by using PowerShell<\/a><\/li>\n
  5. Calculate percentage character frequencies from a text file by using PowerShell<\/a><\/li>\n
  6. Additional resources for text analysis by using PowerShell<\/a><\/li>\n<\/ol>\n

    Today I am going to put the script I wrote yesterday together with the script that I wrote on Friday. After I do that, I will be able to get a more accurate letter-frequency analysis of a text file. The code that I wrote the other day reads a text file by using the Get-Content<\/strong> cmdlet. Then I join the strings together so that I can have a single string to parse. I then convert the script to all uppercase, get the enumerator, group my results, and sort my results.<\/p>\n

    So, first of all, here is the basic letter-frequency analysis code that I wrote the other day:<\/p>\n

    $a = Get-Content C:\\fso\\ATaleOfTwoCities.txt\n$a.Count\n$ajoined = $a -join \"`r\"\n$ajoinedUC = $ajoined.ToUpper()\n$ajoinedUC.GetEnumerator() | group -NoElement | sort count -Descending<\/code><\/p>\n

    Put the script together<\/h2>\n

    The first thing I do is copy the code to a blank page in my Windows PowerShell integrated scripting environment (ISE). This is shown here:<\/p>\n

    \"Screenshot<\/a><\/p>\n

    Now I need to take the code that I wrote yesterday. This code removes the beginning and ending portions of the text file.<\/p>\n

    $a= Get-Content 'C:\\fso\\MobyDick.txt'<\/code><\/p>\n

    $array = @()\nfor ($i = 0; $i -lt $a.Count; $i++)\n{\nIf ($a[$i] -cmatch ‘START’)\n{$array +=$i }\nIf ($a[$i] -like “End of *Project*”)\n{$array += $i }\n}<\/p>\n

    $start = $array[0] +7\n$end = $array[1] -1\n$a[$start .. $end]<\/p>\n

    This script also reads the text file. It then creates an empty array, loops through the text, and looks for start and end strings. It then saves the line numbers that it finds so that I can use array notation to return a range of text from the file.<\/p>\n

    I paste this code at the beginning of my new script page because I need to grab the\u00a0correct text BEFORE I convert it all to a single line of text, convert it to uppercase, and count the letters. So, at this point, my script appears as shown here:<\/p>\n

    \"Screenshot<\/a><\/p>\n

    Clean up the code<\/h2>\n

    Well, there are some redundancies. The code as it stands is shown here:<\/p>\n

    $a= Get-Content 'C:\\fso\\MobyDick.txt'<\/code><\/p>\n

    $array = @()\nfor ($i = 0; $i -lt $a.Count; $i++)\n{\nIf ($a[$i] -cmatch ‘START’)\n{$array +=$i }\nIf ($a[$i] -like “End of *Project*”)\n{$array += $i }\n}<\/p>\n

    $start = $array[0] +7\n$end = $array[1] -1\n$a[$start .. $end]<\/p>\n

    $a = Get-Content C:\\fso\\ATaleOfTwoCities.txt\n$a.Count\n$ajoined = $a -join “`r”\n$ajoinedUC = $ajoined.ToUpper()\n$ajoinedUC.GetEnumerator() | group -NoElement | sort count -Descending<\/p>\n

    So, the obvious duplication is the second Get-Content<\/strong> line. I delete it, and my script is shown here:<\/p>\n

    $a= Get-Content 'C:\\fso\\MobyDick.txt'<\/code><\/p>\n

    $array = @()\nfor ($i = 0; $i -lt $a.Count; $i++)\n{\nIf ($a[$i] -cmatch ‘START’)\n{$array +=$i }\nIf ($a[$i] -like “End of *Project*”)\n{$array += $i }\n}<\/p>\n

    $start = $array[0] +7\n$end = $array[1] -1\n$a[$start .. $end]<\/p>\n

    $a.Count\n$ajoined = $a -join “`r”\n$ajoinedUC = $ajoined.ToUpper()\n$ajoinedUC.GetEnumerator() | group -NoElement | sort count -Descending<\/p>\n

    The next thing I need to do is to delete the $a.count<\/code> line because I do not need it either. The script now is shown here:<\/p>\n

    $a= Get-Content 'C:\\fso\\MobyDick.txt'<\/code><\/p>\n

    $array = @()\nfor ($i = 0; $i -lt $a.Count; $i++)\n{\nIf ($a[$i] -cmatch ‘START’)\n{$array +=$i }\nIf ($a[$i] -like “End of *Project*”)\n{$array += $i }\n}<\/p>\n

    $start = $array[0] +7\n$end = $array[1] -1\n$a[$start .. $end]<\/p>\n

    $ajoined = $a -join “`r”\n$ajoinedUC = $ajoined.ToUpper()\n$ajoinedUC.GetEnumerator() | group -NoElement | sort count -Descending<\/p>\n

    The last thing I need to do is to store the result of grabbing my text from array notation. So that I do not need to modify my copied frequency code, I simply store the $a[$start ... $end]<\/code> code back into the $a<\/code> variable. This revised line is shown here:<\/p>\n

    $a = $a[$start .. $end]<\/code><\/p>\n

    The entire script is shown here:<\/p>\n

    $a= Get-Content 'C:\\fso\\MobyDick.txt'<\/code><\/p>\n

    $array = @()\nfor ($i = 0; $i -lt $a.Count; $i++)\n{\nIf ($a[$i] -cmatch ‘START’)\n{$array +=$i }\nIf ($a[$i] -like “End of *Project*”)\n{$array += $i }\n}<\/p>\n

    $start = $array[0] +7\n$end = $array[1] -1\n$a = $a[$start .. $end]<\/p>\n

    $ajoined = $a -join “`r”\n$ajoinedUC = $ajoined.ToUpper()\n$ajoinedUC.GetEnumerator() | group -NoElement | sort count -Descending<\/p>\n

    The script is shown here in the ISE:<\/p>\n

    \"Screenshot<\/a><\/p>\n

    The output from this script is shown here:<\/p>\n

    \"Screenshot<\/a><\/p>\n

    I invite you to follow me on Twitter<\/a> and Facebook<\/a>. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum<\/a>. Also check out my Microsoft Operations Management Suite Blog<\/a>. See you tomorrow. Until then, peace.<\/p>\n

    Ed Wilson<\/strong>\nMicrosoft Scripting Guy<\/p>\n

     <\/p>\n

     <\/p>\n

     <\/p>\n","protected":false},"excerpt":{"rendered":"

    Summary: Learn how to read a text file and do a letter-frequency analysis using Windows PowerShell in this article written by the Microsoft Scripting Guy, Ed Wilson. This is the third post in a multi-part series of blog posts that deal with how to determine letter frequency in text files. To fully understand this post, […]<\/p>\n","protected":false},"author":596,"featured_media":87096,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[568,683,641],"tags":[3,4,617,45],"class_list":["post-77751","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hey-scripting-guy","category-text-files","category-windows-powershell","tag-scripting-guy","tag-scripting-techniques","tag-text","tag-windows-powershell"],"acf":[],"blog_post_summary":"

    Summary: Learn how to read a text file and do a letter-frequency analysis using Windows PowerShell in this article written by the Microsoft Scripting Guy, Ed Wilson. This is the third post in a multi-part series of blog posts that deal with how to determine letter frequency in text files. To fully understand this post, […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts\/77751","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/users\/596"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/comments?post=77751"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts\/77751\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/media\/87096"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/media?parent=77751"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/categories?post=77751"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/tags?post=77751"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}