{"id":13411,"date":"2011-07-06T00:01:00","date_gmt":"2011-07-06T00:01:00","guid":{"rendered":"https:\/\/blogs.technet.microsoft.com\/heyscriptingguy\/2011\/07\/06\/use-a-powershell-hash-table-to-simplify-file-backups\/"},"modified":"2011-07-06T00:01:00","modified_gmt":"2011-07-06T00:01:00","slug":"use-a-powershell-hash-table-to-simplify-file-backups","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/scripting\/use-a-powershell-hash-table-to-simplify-file-backups\/","title":{"rendered":"Use a PowerShell Hash Table to Simplify File Backups"},"content":{"rendered":"

Summary<\/b>: Learn how to use a Windows PowerShell hash table to simplify backing up unique files.<\/p>\n

 <\/p>\n

\"Hey,Hey, Scripting Guy! Your series on hash tables has been pretty interesting. I am wondering if you have some practical uses for hash tables. Can you provide a few examples of how using a hash table would be useful?<\/p>\n

—RE<\/p>\n

 <\/p>\n

\"Hey,Hello RE,<\/p>\n

Microsoft Scripting Guy Ed Wilson here. This morning the Scripting Wife received an email message from one of the hotels where we had stayed during our last trip to Australia. They were advertising their winter specials. The temperature around here has hovered near 100 degrees Fahrenheit (37.7 degrees Celsius according to my Windows PowerShell conversion module<\/a>). One of the things I love about Australia (besides Tim Tams, Lamingtons, great scuba diving, awesome scenery, and especially wonderful people) is the fact that when the weather is oppressively hot in the Deep South, it is winter down under. A quick flight to Brisbane and one has escaped the hot sticky weather of Charlotte, North Carolina, in July. <\/p>\n

RE, to answer directly your question let me begin by detailing a scenario. Suppose there is a directory structure that contains a number of folders and files. Inside the different folders, each file name is, of course, unique. However, inside the subfolders, there are duplicate files. A sample file structure containing duplicate files in nested folders is shown in the following figure.<\/p>\n

\"Image<\/a><\/p>\n

If I need to flatten this directory structure (move it from several nested folders, some of which have duplicate files, to a single folder with no duplicates), there is a variety of approaches I can use. One is to use the graphical user interface of the Windows Explorer tool. The problem with this approach is it is interactive, and involves a lot of clicking, “Yes I know there is an existing file and I want to write over that file.” This makes things proceed rather slowly. In addition, I might want to have a job that I can schedule on a nightly basis for backup purposes. Writing a bit of Windows PowerShell code will allow me the flexibility to solve the problem in multiple ways. In addition, I can avoid multiple mouse clicks. A hash table will simplify the coding that is required.<\/p>\n

If I use the name property (of the System.IO.FileInfo<\/b> object) as the key for my hash table, and the fullname<\/b> property (of the same object) as the value that is associated with the key, I can filter out all of the duplicate files. This filtering occurs because the key<\/b> property of a hash table must be unique.<\/p>\n

In the command that follows, I first create an empty hash table and store it in the $hash<\/b> variable. I use a semicolon to allow me to continue another command on the same line. Next, I use dir<\/b> to get a directory listing of my current directory (the C:\\hsgtest folder is the working directory as indicated by the Windows PowerShell prompt). I use the recurse<\/b> switch to cause the command to work through all of the nested directories. The dir<\/b> command is an alias for the Get-ChildItem <\/b>cmdlet. I pipe the results to the Where-Object<\/b> cmdlet (?<\/b> Is an alias for the Where-Object<\/b> cmdlet). Inside the script block (the braces), I use the !<\/b> operator (means not) to cause the Where-Object <\/b>to return only items that are not containers (in other words, files). I pipe the files to the ForEach-Object<\/b> cmdlet (%<\/b> is an alias for the ForEach-Object<\/b> cmdlet). Inside the script block (braces) associated with the ForEach-Object<\/b> cmdlet I add the name<\/b> property and the fullname<\/b> property from the fileinfo<\/b> objects to the hash table by using the add<\/b> method. The name<\/b> of each file becomes the key in the hash table, and the fullname<\/b> (which is the full path to the file) becomes the value associated with each item. When the command runs, errors appear for each duplicate file name that are attempted to be added to the hash table. This is expected, and is confirmation the command works properly. The complete command is shown here:<\/p>\n

PS C:\\hsgTest> $hash = @{}; dir -recurse | ? { !$_.psiscontainer} | % { $hash.add($_.name,$_.fullname) }<\/p>\n

To see the consolidated list of files, I inspect the contents of the $hash<\/b> variable as shown here:<\/p>\n

PS C:\\hsgTest> $hash<\/p>\n

 <\/p>\n

Name                           Value<\/p>\n

—-                           —–<\/p>\n

testfile30.txt                 C:\\hsgTest\\testfile30.txt<\/p>\n

testfile27.txt                 C:\\hsgTest\\hsgtest2\\testfile27.txt<\/p>\n

testfile21.txt                 C:\\hsgTest\\hsgtest2\\testfile21.txt<\/p>\n

testfile20.txt                 C:\\hsgTest\\testfile20.txt<\/p>\n

testfile25.txt                 C:\\hsgTest\\hsgtest2\\testfile25.txt<\/p>\n

testfile10.txt                 C:\\hsgTest\\testfile10.txt<\/p>\n

testfile6.txt                  C:\\hsgTest\\testfile6.txt<\/p>\n

testfile35.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile35.txt<\/p>\n

testfile34.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile34.txt<\/p>\n

testfile28.txt                 C:\\hsgTest\\testfile28.txt<\/p>\n

testfile1.txt                  C:\\hsgTest\\testfile1.txt<\/p>\n

testfile23.txt                 C:\\hsgTest\\hsgtest2\\testfile23.txt<\/p>\n

testfile2.txt                  C:\\hsgTest\\testfile2.txt<\/p>\n

testfile29.txt                 C:\\hsgTest\\hsgtest2\\testfile29.txt<\/p>\n

testfile24.txt                 C:\\hsgTest\\testfile24.txt<\/p>\n

testfile9.txt                  C:\\hsgTest\\testfile9.txt<\/p>\n

testfile38.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile38.txt<\/p>\n

testfile40.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile40.txt<\/p>\n

testfile8.txt                  C:\\hsgTest\\testfile8.txt<\/p>\n

testfile36.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile36.txt<\/p>\n

testfile33.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile33.txt<\/p>\n

testfile32.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile32.txt<\/p>\n

testfile26.txt                 C:\\hsgTest\\testfile26.txt<\/p>\n

testfile22.txt                 C:\\hsgTest\\testfile22.txt<\/p>\n

testfile31.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile31.txt<\/p>\n

testfile39.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile39.txt<\/p>\n

testfile37.txt                 C:\\hsgTest\\hsgtest2\\hsgTest3\\testfile37.txt<\/p>\n

testfile5.txt                  C:\\hsgTest\\testfile5.txt<\/p>\n

testfile4.txt                  C:\\hsgTest\\testfile4.txt<\/p>\n

testfile7.txt                  C:\\hsgTest\\testfile7.txt<\/p>\n

testfile3.txt                  C:\\hsgTest\\testfile3.txt<\/p>\n

Of course, I could simply use the Copy-Item<\/b> cmdlet to flatten the hierarchy, but unfortunately, the nested folders still get copied. The container<\/b> switched parameter causes the Copy-Item<\/b> cmdlet to duplicate the file structure, including any nested folders. By default, the container<\/b> switched parameter has a value of TRUE and will duplicate the existing hierarchy as seen in the following figure.<\/p>\n

\"Image<\/a><\/p>\n

To supply a value of FALSE to the switched parameter requires the trick of using a colon and then the $false<\/b> value. The resulting command is shown here:<\/p>\n

PS C:\\hsgTest> Copy-Item -Path . -Destination C:\\hsgBackup -Recurse -Container:$false<\/p>\n

All of the files are copied into the root of the destination, but the two nested folders still appear. The two nested folders are empty, but still present. This is shown in the following figure.<\/p>\n

\"Image<\/a><\/p>\n

One of the cool things about the Copy-Item <\/b>cmdlet is that it will accept an array for the path<\/i> parameter. Using the values<\/b> property from the hash table stored in the $hash<\/i><\/b> variable, I have the full path to each unique file in the directory structure. I can copy the unique files to the hsgbackup<\/b> directory by using the Copy-Item <\/b>cmdlet. The sub expression $()<\/b> is required to force evaluation of the $hash.values<\/b> property before executing the Copy-Item <\/b>command.<\/p>\n

PS C:\\hsgTest> Copy-Item -Path $($hash.values) -Destination C:\\hsgBackup<\/p>\n

A quick look at the hsgbackup<\/b> <\/i>directory reveals that the copy proceeded as expected—no nested folders appear. The backup directory is shown in the following figure.<\/p>\n

\"Image<\/a><\/p>\n

The complete command used to backup unique files from nested directories follows this paragraph. Refer to earlier portions of this article for a complete explanation of the commands and aliases. (To simplify the command syntax, I set my working directory to the directory I wanted to backup.)<\/p>\n

$hash = @{}; dir -recurse | ? { !$_.psiscontainer} | % { $hash.add($_.name,$_.fullname) }<\/p>\n

Copy-Item -Path $($hash.values) -Destination C:\\hsgBackup<\/p>\n

RE, that is all there is to using hash tables in scripts. Hash Table Week will continue tomorrow when I will talk about using hash tables in conjunction with other Windows PowerShell commands.<\/p>\n

 <\/p>\n

I invite you to follow me on Twitter<\/a> and Facebook<\/a>. If you have any questions, send email to me at scripter@microsoft.com<\/a>, or post your questions on the Official Scripting Guys Forum<\/a>. See you tomorrow. Until then, peace.<\/p>\n

Ed Wilson, Microsoft Scripting Guy<\/b><\/p>\n

 <\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Summary: Learn how to use a Windows PowerShell hash table to simplify backing up unique files.   Hey, Scripting Guy! Your series on hash tables has been pretty interesting. I am wondering if you have some practical uses for hash tables. Can you provide a few examples of how using a hash table would be […]<\/p>\n","protected":false},"author":596,"featured_media":87096,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[18,51,3,4,45],"class_list":["post-13411","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-scripting","tag-arrays-hash-tables-and-dictionary-objects","tag-getting-started","tag-scripting-guy","tag-scripting-techniques","tag-windows-powershell"],"acf":[],"blog_post_summary":"

Summary: Learn how to use a Windows PowerShell hash table to simplify backing up unique files.   Hey, Scripting Guy! Your series on hash tables has been pretty interesting. I am wondering if you have some practical uses for hash tables. Can you provide a few examples of how using a hash table would be […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts\/13411","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/users\/596"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/comments?post=13411"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts\/13411\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/media\/87096"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/media?parent=13411"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/categories?post=13411"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/tags?post=13411"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}