{"id":7151,"date":"2015-03-21T00:01:00","date_gmt":"2015-03-21T00:01:00","guid":{"rendered":"https:\/\/blogs.technet.microsoft.com\/heyscriptingguy\/2015\/03\/21\/psimaging-part-3-group-imagefile\/"},"modified":"2019-02-18T10:30:12","modified_gmt":"2019-02-18T17:30:12","slug":"psimaging-part-3-group-imagefile","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/scripting\/psimaging-part-3-group-imagefile\/","title":{"rendered":"PSImaging Part 3: Group-ImageFile"},"content":{"rendered":"<p><b style=\"font-size:12px\">Summary<\/b><span style=\"font-size:12px\">: Guest blogger, Ben Vierck, talks about grouping similar images with Windows PowerShell.<\/span><\/p>\n<p>Microsoft Scripting Guy, Ed Wilson, is here. Today we have Ben Vierck back for Part 3 in his series about images. Before you begin, you might like to read:<\/p>\n<ul>\n<li><a href=\"https:\/\/devblogs.microsoft.com\/scripting\/psimaging-part-1-test-image\/\" target=\"_blank\">PSImaging Part 1: Test-Image<\/a><\/li>\n<li><a href=\"\/b\/heyscriptingguy\/archive\/2015\/03\/20\/psimaging-part-2-export-text-from-images.aspx\" target=\"_blank\">PSImaging Part 2: Test-Image<\/a><\/li>\n<\/ul>\n<p>In first two blog posts of this series, we wrote the Windows PowerShell functions <b>Test-Image<\/b> and <b>Export-Text<\/b> into a new Windows PowerShell module named <b>PSImaging<\/b>. The purpose of this exercise is to give us a set of common atomic tools that we can use to automate some of day-to-day document management tasks that require manual human intervention.<\/p>\n<p>Today we&#039;re going one step further by building and using tools that give Windows PowerShell a rudimentary level of vision so that our scripts can <i>see<\/i> whether two images are similar to one another.<\/p>\n<p>What do we mean by sorting images by similarity? By the end of the exercise, we want to be able to write a script that takes a folder that looks like this:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-1.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-1.png\" alt=\"Image of folder\" title=\"Image of folder\" \/><\/a><\/p>\n<p>&#8230;and turn it into a folder that looks like this:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-2.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-2.png\" alt=\"Image of folder\" title=\"Image of folder\" \/><\/a><\/p>\n<p>As humans, we have no problem sorting these images based on image similarity. It&#039;s a trivial task. Our computers, though, don&#039;t come with native vision. We&#039;ll have to give it these tools, one-at-a-time. Let&#039;s start with the ability to identify image similarity.<\/p>\n<p>When choosing an algorithm, we want to optimize for two things:<\/p>\n<ul>\n<li>Minimal compute time of runtime comparison<\/li>\n<li>Resilience in the face of minor image transformations such as skew, resizing, and cropping<\/li>\n<\/ul>\n<p>To satisfy the first goal, I&#039;ve chosen to break the problem into two pieces:<\/p>\n<ul>\n<li>Compute a signature which can be stored for later retrieval<\/li>\n<li>Compare signatures<\/li>\n<\/ul>\n<p>In the final production-ready module, we should store signatures on a disk for quick retrieval later. By doing so, we&#039;ve moved the compute time from runtime to some other time of our choosing&mdash;for example, a nightly indexing of files.&nbsp;<\/p>\n<p>I&#039;ve chosen a signature schema developed by H. Chi Wong, Marshall Bern, and David Goldberg of Xerox, and published in 2002: <i>An image signature for any kind of image<\/i>. The algorithm itself is brilliant, comparing relative brightness levels of regions within the image. This method means it satisfies our second requirement: resilience to some resizing, cropping, and compression.<\/p>\n<p>As before, we&#039;ve wrapped up an open source implementation with a Windows PowerShell layer. Our PSImaging module is stored on GitHub: <a href=\"https:\/\/github.com\/Positronic-IO\/PSImaging\" target=\"_blank\">Positronic-IO\/PSImaging<\/a>. If you haven&#039;t already, you can install the module with this one-liner:&nbsp;<\/p>\n<p style=\"margin-left:30px\">&amp; ([scriptblock]::Create((iwr -uri http:\/\/tinyurl.com\/Install-GitHubHostedModule).Content)) <br \/>-GitHubUserName Positronic-IO -ModuleName PSImaging -Branch &#039;master&#039; -Scope CurrentUser<\/p>\n<p>Let&#039;s start with the <b>Get-ImageHash<\/b> cmdlet. It takes two parameters, <b>Path<\/b> and <b>Level<\/b>. Let&#039;s try it:<\/p>\n<p style=\"margin-left:30px\">Get-ImageHash .\\1.tiff<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-3.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-3.png\" alt=\"Image of command output\" title=\"Image of command output\" \/><\/a><\/p>\n<p>The default parameter of the new cmdlet is <b>Path<\/b>. The cmdlet returns a string that contains a hash. This is the signature described in the Wong, Bern, Goldberg paper. Now let&#039;s put it to work. Let&#039;s get hashes for two images that we know are<b> <\/b>similar:<\/p>\n<p style=\"margin-left:30px\">$hash1 = Get-ImageHash .\\1.tiff -Level 5<\/p>\n<p style=\"margin-left:30px\">$hash10 = Get-ImageHash .\\10.tiff -Level 5<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-4.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-4.png\" alt=\"Image of folder\" title=\"Image of folder\" \/><\/a><\/p>\n<p>Let&#039;s compare those two hashes by using the <b>Compare-ImageHash<\/b> cmdlet:<\/p>\n<p style=\"margin-left:30px\">Compare-ImageHash $hash1 $hash10<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-5.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-5.png\" alt=\"Image of command output\" title=\"Image of command output\" \/><\/a><\/p>\n<p>The result is 0.8125, or 81.25%, similarity. Let&#039;s get a hash for an image that we know is <b>not<\/b> similar:<\/p>\n<p style=\"margin-left:30px\">$hash2 = Get-ImageHash .\\2.tiff -Level 5<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-6.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-6.png\" alt=\"Image of folder\" title=\"Image of folder\" \/><\/a><\/p>\n<p>Let&#039;s compare the two hashes of the images we know are <b>not<\/b> similar:<\/p>\n<p style=\"margin-left:30px\">Compare-ImageHash $hash1 $hash2<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-7.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-7.png\" alt=\"Image of command output\" title=\"Image of command output\" \/><\/a><\/p>\n<p>The result is 0.5241699, or 52.42%, similarity. This confirms what we can see visually. Images 1 and 10 are significantly more similar than images 1 and 2.<\/p>\n<p>That&#039;s useful on a case-by-case basis. Let&#039;s put it to work on a whole collection of images by using the <b>Group-ImageFile<\/b> cmdlet. Under the covers, the <b>Group-ImageFile<\/b> cmdlet uses <b>Get-ImageHash<\/b> and <b>Compare-ImageHash<\/b> to sort a collection of files into groups. Let&#039;s see how it works:<\/p>\n<p style=\"margin-left:30px\">$groups = dir | Group-ImageFile<\/p>\n<p style=\"margin-left:30px\">$groups<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-8.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-8.png\" alt=\"Image of command output\" title=\"Image of command output\" \/><\/a><\/p>\n<p>Now let&#039;s examine the files that were grouped with a <b>High<\/b> degree of confidence:&nbsp;<\/p>\n<p style=\"margin-left:30px\">$groups | ? Confidence -eq High | select -ExpandProperty Files<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-9.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-9.png\" alt=\"Image of command output\" title=\"Image of command output\" \/><\/a><\/p>\n<p>Let&#039;s check the output visually:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-10.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/29\/2019\/02\/wes-3-21-15-10.png\" alt=\"Image of folder\" title=\"Image of folder\" \/><\/a><\/p>\n<p>That&#039;s perfect.<\/p>\n<p>Now you&#039;re armed with the right tools to start managing scanned document images. As we&#039;ve shown in this short series, Windows PowerShell has a limitless capacity to be extended with very little effort. The project on display here was written in under an hour. Imagine how powerful it would be if we&#039;d put in 150 hours.<\/p>\n<p>With legacy imaging systems, most of the effort goes into getting the images processed and put into the system. It&#039;s a fragile process. Imagine instead, that you could send your processes to the images where they live. The characteristic of such a system would follow the philosophy of Windows PowerShell: repeatable, transparent, and completely scriptable.<\/p>\n<p>Follow me at @xcud on Twitter to keep abreast of the latest in Windows PowerShell, document imaging, and computer vision.<\/p>\n<p>~Ben<\/p>\n<p>Thank-you, Ben, for an insightful series.<\/p>\n<p>I invite you to follow me on <a href=\"http:\/\/bit.ly\/scriptingguystwitter\" target=\"_blank\">Twitter<\/a> and <a href=\"http:\/\/bit.ly\/scriptingguysfacebook\" target=\"_blank\">Facebook<\/a>. If you have any questions, send email to me at <a href=\"mailto:scripter@microsoft.com\" target=\"_blank\">scripter@microsoft.com<\/a>, or post your questions on the <a href=\"http:\/\/bit.ly\/scriptingforum\" target=\"_blank\">Official Scripting Guys Forum<\/a>. See you tomorrow. Until then, peace.<\/p>\n<p><b>Ed Wilson, Microsoft Scripting Guy<\/b><span style=\"font-size:12px\">&nbsp;<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Summary: Guest blogger, Ben Vierck, talks about grouping similar images with Windows PowerShell. Microsoft Scripting Guy, Ed Wilson, is here. Today we have Ben Vierck back for Part 3 in his series about images. Before you begin, you might like to read: PSImaging Part 1: Test-Image PSImaging Part 2: Test-Image In first two blog posts [&hellip;]<\/p>\n","protected":false},"author":596,"featured_media":87096,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[571,122,56,123,3,61,45],"class_list":["post-7151","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-scripting","tag-ben-vierck","tag-graphics","tag-guest-blogger","tag-multimedia","tag-scripting-guy","tag-weekend-scripter","tag-windows-powershell"],"acf":[],"blog_post_summary":"<p>Summary: Guest blogger, Ben Vierck, talks about grouping similar images with Windows PowerShell. Microsoft Scripting Guy, Ed Wilson, is here. Today we have Ben Vierck back for Part 3 in his series about images. Before you begin, you might like to read: PSImaging Part 1: Test-Image PSImaging Part 2: Test-Image In first two blog posts [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts\/7151","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/users\/596"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/comments?post=7151"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/posts\/7151\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/media\/87096"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/media?parent=7151"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/categories?post=7151"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/scripting\/wp-json\/wp\/v2\/tags?post=7151"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}