{"id":65423,"date":"2007-02-28T01:04:00","date_gmt":"2007-02-28T01:04:00","guid":{"rendered":"https:\/\/blogs.technet.microsoft.com\/heyscriptingguy\/2007\/02\/28\/how-can-i-tally-up-all-the-words-found-in-a-text-file\/"},"modified":"2007-02-28T01:04:00","modified_gmt":"2007-02-28T01:04:00","slug":"how-can-i-tally-up-all-the-words-found-in-a-text-file","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/scripting\/how-can-i-tally-up-all-the-words-found-in-a-text-file\/","title":{"rendered":"How Can I Tally Up All the Words Found in a Text File?"},"content":{"rendered":"

\"Hey, <\/H2>\n

Hey, Scripting Guy! While browsing the Internet I found a script that showed me how to get a list of all the unique words in a text file. That\u2019s useful, but I\u2019d like to go one step further: how can I determine the number of times each of those words occurs??

— TZ<\/P>\"Spacer\"\"Hey,\"Script<\/A> \n

Hey, TZ. As it turns out, that\u2019s a lot easier than you might think. In fact, all you have to do is \u2013 hang on a second, we just got an email. And not just any email: based on the subject line \u2013 underpaid and not appreciated?<\/B> \u2013 this must be a legitimate email that\u2019s truly intended for the Scripting Guy who writes this column. Let\u2019s see what it says:<\/P>

I wanted to write and tlel you about a great new De!gree program, i tried it out and it worked!<\/p>\n
I got my M!asters in 2 weeks ;]. Call the folowing number, this program really works great i was very surprised!\n<\/PRE>\n
Now that\u2019s<\/I> a good deal. Although you might find this hard to believe, the Scripting Guy who writes this column already has<\/I> a Masters degree, and from the University of Washington to boot. (Further proof that the value of a college education is highly overrated.) But the Scripting Guy who writes this column didn\u2019t get his degree in just two weeks; instead, it took him almost two years. Furthermore, in his<\/I> degree program he had to know spelling and grammar along with everything else. This new program sounds way<\/I> better! And while we aren\u2019t positive that a M!asters degree is the same thing as a Masters degree, as long as it enables the Scripting Guy who writes this column to get the money and appreciation he deserves, well \u2026.<\/P>\n
Anyway, it looks like the Scripting Guy who writes this column will be going back to college, at least for two weeks anyway. That means he has a lot to do: buy some cinder blocks to build a bookcase; stock his cupboards with Top Ramen and boxed macaroni-and-cheese; and write home to his parents asking if they can send him some money. Oh: and show you how to tally up the words found in a text file. You know, by using a script like this one:<\/P>
Const ForReading = 1<\/p>\n
Set objDictionary = CreateObject(“Scripting.Dictionary”)<\/p>\n
Set objFSO = CreateObject(“Scripting.FileSystemObject”)\nSet objFile = objFSO.OpenTextFile(“c:\\scripts\\test.txt”, ForReading)<\/p>\n
strText = objFile.ReadAll\nobjFile.Close<\/p>\n
strText = Replace(strText, “,”, ” “)\nstrText = Replace(strText, “.”, ” “)\nstrText = Replace(strText, “!”, ” “)\nstrText = Replace(strText, “?”, ” “)\nstrText = Replace(strText, “>”, ” “)\nstrText = Replace(strText, “<“, ” “)\nstrText = Replace(strText, “&”, ” “)\nstrText = Replace(strText, “*”, ” “)\nstrText = Replace(strText, “=”, ” “)\nstrText = Replace(strText, vbCrLf, ” “)<\/p>\n
arrWords = Split(strText, ” “)<\/p>\n
For Each strWord in arrWords\n    If Len(strWord) > 0 Then\n        If objDictionary.Exists(strWord) Then\n            objDictionary.Item(strWord) = objDictionary.Item(strWord) + 1\n        Else\n            objDictionary.Add strWord, 1\n        End If<\/p>\n
    End If\nNext<\/p>\n
colKeys = objDictionary.Keys<\/p>\n
For Each strKey in colKeys\n    Wscript.Echo strKey & ” — ” & objDictionary.Item(strKey)\nNext\n<\/PRE>\n
In case any of you are thinking that the Scripting Guy who writes this column is too old and too out-of-touch to go back to college, we can set your mind at ease by pointing out that he cheated in order to finish today\u2019s assignment: in particular, he copied an existing script<\/B><\/A> from the Internet and then just modified it slightly to meet his needs. And even though he cheated, he still waited until the last possible minute to complete the assignment. If that doesn\u2019t sound like a college student, well, we don\u2019t know what does.<\/P>\n
Let\u2019s take a few minutes to discuss how this script works; that will be good practice when it comes time for the Scripting Guy who writes this column to defend his master\u2019s thesis. (And yes, we are a little<\/I> concerned about having just two weeks to complete 45 credits of coursework and<\/I> write a master\u2019s thesis. But no doubt this school knows what it\u2019s doing.)<\/P>\n
The script starts out by defining a constant named ForReading and setting the value to 1; we\u2019ll need this constant when we open our text file. Next we create an instance of the Scripting.Dictionary<\/B> object. Why? We\u2019ll get to that in just a second. For now, let\u2019s forget about the Dictionary object and focus on the next two lines of code, which create an instance of the Scripting.FileSystemObject<\/B> and use the OpenTextFile<\/B> method to open the file C:\\Scripts\\Test.txt:<\/P>
Set objFSO = CreateObject(“Scripting.FileSystemObject”)\nSet objFile = objFSO.OpenTextFile(“c:\\scripts\\test.txt”, ForReading)\n<\/PRE>\n
Once the file is open we use the ReadAll<\/B> method to read the entire file into memory and store it in a variable named strText:<\/P>
strText = objFile.ReadAll\n<\/PRE>\n
Now that the contents of Test.txt are stored in memory we call the Close<\/B> method and close the file.<\/P>\n
Our next task is to identify the individual words in the file. In general that\u2019s fairly easy; as you\u2019ll see, we simply use the Split<\/B> method to create an array from all the words in strText. And how will we identify the individual words? By splitting on the blank space (\u201c \u201c), acting under the assumption that all the words in the file are separated by blank spaces.<\/P>\n
For the most part, that will work pretty well. However, there is a potential problem here. For example, consider this sample text file:<\/P>
I saw the cat. The cat was black.\n<\/PRE>\n
How many times does the word cat<\/I> appear in this file? We\u2019d agree: it appears twice. However, our script won\u2019t agree; instead, the script sees two different words that happen to include the letters c-a-t:<\/P>\n\n\n\n\n
\u2022<\/TD>\n\n
cat.<\/P><\/TD><\/TR>\n
\u2022<\/TD>\n\n
cat<\/P><\/TD><\/TR><\/TBODY><\/TABLE>\n
See the problem? It\u2019s the period immediately following the first instance of cat<\/I>. Because our script doesn\u2019t know anything about punctuation (which definitely makes it a candidate for a M!asters degree) it doesn\u2019t know to ignore the period at the end of the sentence. Punctuation \u2013 and carriage return-linefeeds \u2013 can create problems in this script. Therefore, we use a series of Replace<\/B> commands to find these characters and replace them with blank spaces:<\/P>
strText = Replace(strText, “,”, ” “)\nstrText = Replace(strText, “.”, ” “)\nstrText = Replace(strText, “!”, ” “)\nstrText = Replace(strText, “?”, ” “)\nstrText = Replace(strText, “>”, ” “)\nstrText = Replace(strText, “<“, ” “)\nstrText = Replace(strText, “&”, ” “)\nstrText = Replace(strText, “*”, ” “)\nstrText = Replace(strText, “=”, ” “)\nstrText = Replace(strText, vbCrLf, ” “)\n<\/PRE>\n
That turns our practice file into something that looks like this:<\/P>
I saw the cat  The cat was black\n<\/PRE>\n
In turn, our script now tells us that the word cat<\/I> appears twice.<\/P>\n\n<\/THEAD>\n\n\n
\n
Note<\/B>. For a somewhat more detailed discussion of this issue, see this Hey, Scripting Guy! column<\/B><\/A>.<\/P><\/TD><\/TR><\/TBODY><\/TABLE>\n
<\/DIV>\n
After we\u2019ve cleaned up our text file, we then use the Split function to create an array consisting of the individual words found in the text file. For our simple example, that means the array arrWords will contain these items:<\/P>
I \nsaw \nthe \ncat  \nThe \ncat \nwas \nblack\n<\/PRE>\n
And now it\u2019s time to start tallying the number of times each individual word occurs. That\u2019s what this block of code, and the Dictionary object, is for:<\/P>
For Each strWord in arrWords\n    If Len(strWord) > 0 Then\n        If objDictionary.Exists(strWord) Then\n            objDictionary.Item(strWord) = objDictionary.Item(strWord) + 1\n        Else\n            objDictionary.Add strWord, 1\n        End If<\/p>\n
    End If\nNext\n<\/PRE>\n
What are we doing here? Good question. The first thing we\u2019re doing is setting up a For Each loop that will loop through all the items in the array; in other words, through all the words in the text file. For each word we first verify that the length (Len<\/B>) is greater than 0 characters. (Why? See our previous column<\/B><\/A> on this topic for details.) Assuming that the length is<\/I> greater than 0 we then use the following line of code the see if the word in question already exists in our Dictionary:<\/P>
If objDictionary.Exists(strWord) Then\n<\/PRE>\n
Let\u2019s assume that the word can\u2019t<\/I> be found in the Dictionary. In that case, we use the Add<\/B> method to add the word as a new Dictionary Key<\/B>. At the same time, we set the value of the corresponding Dictionary Item<\/B> to 1:<\/P>
objDictionary.Add strWord, 1\n<\/PRE>\n
Why 1? Because, so far, we\u2019ve found 1 occurrence of that particular word.<\/P>\n
If the word already exists in the Dictionary we don\u2019t try adding it a second time; that would cause an error. Instead, we simply increment the value of the Item property by 1:<\/P>
objDictionary.Item(strWord) = objDictionary.Item(strWord) + 1\n<\/PRE>\n
You probably don\u2019t need us to tell you this, but if the Item was equal to 1 then, after we execute this line of code, the Item will be equal to 2. You probably also don\u2019t need is to tell you why we chose to use the Dictionary object; unlike an array, it\u2019s easy to 1) locate a specified key; and 2) determine whether a key already exists. (See this Sesame Script<\/B><\/A> article for more on how the Dictionary object works.<\/P>\n
From there all we do is loop around and repeat the process with the next word in the array.<\/P>\n
After we\u2019ve finished our For Each loop we use this block of code to report back all the Keys and Item values in the Dictionary:
<\/P>
colKeys = objDictionary.Keys<\/p>\n
For Each strKey in colKeys\n    Wscript.Echo strKey & ” — ” & objDictionary.Item(strKey)\nNext\n<\/PRE>\n
That\u2019s going to give us a report similar to this:<\/P>
I — 1\nsaw — 1\nthe — 1\ncat — 2\nThe — 1\nwas — 1\nblack \u2013 1\n<\/PRE>\n
And yes, it would<\/I> be nice if those words were sorted alphabetically, wouldn\u2019t it? But that\u2019s a task for another day.<\/P>\n
As for going back to college, the Scripting Guy who writes this column is actually having second thoughts. Granted, the idea that you could get a Masters degree in two weeks is a bit suspicious; it\u2019s even more suspicious that the telephone number provided in the email is an unlisted number. But the big problem is that, as near as he can tell, the alleged school has neither a football team nor a basketball team. No football team or basketball team? Then why even have<\/I> a college?<\/P>\n
Besides, there\u2019s no need for him to waste two weeks of his life getting a Masters degree. After all, according to another email he just received the Scripting Guy who writes this column can make $50,000 a month while working home; that works out to $600,000 a year. Sure, that would be a bit of a pay cut, but it might be worth giving up a little money for the chance to work from home.<\/P><\/p>\n","protected":false},"excerpt":{"rendered":"
Hey, Scripting Guy! While browsing the Internet I found a script that showed me how to get a list of all the unique words in a text file. That\u2019s useful, but I\u2019d like to go one step further: how can I determine the number of times each of those words occurs??— TZ Hey, TZ. As […]<\/p>\n","protected":false},"author":595,"featured_media":87096,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[3,4,14,5],"class_list":["post-65423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-scripting","tag-scripting-guy","tag-scripting-techniques","tag-text-files","tag-vbscript"],"acf":[],"blog_post_summary":"
Hey, Scripting Guy! While browsing the Internet I found a script that showed me how to get a list of all the unique words in a text file. That\u2019s useful, but I\u2019d like to go one step further: how can I determine the number of times each of those words occurs??— TZ Hey, TZ. As […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts\/65423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/users\/595"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/comments?post=65423"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts\/65423\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/media\/87096"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/media?parent=65423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/categories?post=65423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/tags?post=65423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}