{"id":17812,"date":"2008-12-24T07:25:30","date_gmt":"2008-12-24T15:25:30","guid":{"rendered":"http:\/\/devblogs.microsoft.com\/powershell\/?p=17812"},"modified":"2019-06-07T07:27:03","modified_gmt":"2019-06-07T15:27:03","slug":"get-markuptag","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/powershell\/get-markuptag\/","title":{"rendered":"Get-MarkupTag"},"content":{"rendered":"

On my personal blog (Media And Microcode<\/a>), I’ve been posting a series called "Scripting the Web", which introduced a function called Get-MarkupTag. Get-MarkupTag is a very handy little function that coerces individual tag elements of a web page into HTML, so you can scrape data from a webpage.<\/p>\n

I’ve updated Get-MarkupTag a tiny bit for CTP3, marking the tag name as value from pipeline so I can get multiple tag types from a single document. I’m posting it again here so that a wider audience can make use of it, and so that I can use it in some later blog posts. Since I’ve already got inline help for the function, I simply used Write-CommandBlogPost<\/a> to output its documentation and post it again here.<\/p>\n

Enjoy!<\/p>\n

<\/p>\n

Synopsis:<\/b> <\/p>\n

Extracts out a markup language (HTML or XML) tag from within a document <\/p><\/blockquote>\n

<\/p>\n

Syntax:<\/b> <\/p>\n

Get-MarkupTag [[-tag] [<Object>]] [[-html] [<String>]] [-Verbose] [-Debug] [-ErrorAction [<ActionPreference>]] [-WarningAction [<ActionPreference>]] [-ErrorVariable [<String>]] [-WarningVariable [<String>]] [-OutVariable [<String>]] [-OutBuffer [<Int32>]] [<CommonParameters>] <\/p><\/blockquote>\n

<\/p>\n

Detailed Description:<\/b> <\/p>\n

Extracts out a markup language (HTML or XML) tag from within a document.
Returns the tag, the text within the tag, and, if possible, the tag converted
to XML <\/p><\/blockquote>\n

<\/p>\n

Examples: <\/p>\n

\n
    -------------------------- EXAMPLE 1 --------------------------\r\n\r\n\r\n\r\n\r\n\r\n# Download the Microsoft front page and extract out links and div tags\r\n$microsoft = (New-Object Net.Webclient).DownloadString("http:\/\/www.microsoft.com\/")\r\n"a", "div" | Get-MarkupTag -html $microsoft\r\n    <\/pre>\n<\/blockquote>\n
\n
    -------------------------- EXAMPLE 2 --------------------------\r\n\r\n\r\n\r\n\r\n\r\n# Extract the rows from ConvertTo-HTML\r\n$text = Get-ChildItem | Select Name, LastWriteTime | ConvertTo-HTML | Out-String \r\nGet-MarkupTag "tr" $text\r\n    <\/pre>\n<\/blockquote>\n
<\/p>\n
Command Parameters: <\/p>\n
\n\n\n\n<\/colgroup>\n\n\n\n\n
Name<\/th>\nDescription<\/th>\n<\/tr>\n
tag<\/td>\nThe tag to extract, e.g. "a", "div"<\/td>\n<\/tr>\n
html<\/td>\nThe text to extract the tag from<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/blockquote>\n
<\/p>\n
Here’s Get-MarkupTag: <\/i><\/p>\n
\n
function Get-MarkupTag {\r\n           \r\n    #.Synopsis\r\n    #   Extracts out a markup language (HTML or XML) tag from within a document\r\n    #.Description\r\n    #   Extracts out a markup language (HTML or XML) tag from within a document.\r\n    #   Returns the tag, the text within the tag, and, if possible, the tag converted\r\n    #   to XML\r\n    #.Parameter tag \r\n    #   The tag to extract, e.g. "a", "div"\r\n    #.Parameter html\r\n    #   The text to extract the tag from\r\n    #.Example\r\n    #   # Download the Microsoft front page and extract out links and div tags\r\n    #   $microsoft = (New-Object Net.Webclient).DownloadString("http:\/\/www.microsoft.com\/")\r\n    #   "a", "div" | Get-MarkupTag -html $microsoft\r\n    #.Example\r\n    #   # Extract the rows from ConvertTo-HTML\r\n    #   $text = Get-ChildItem | Select Name, LastWriteTime | ConvertTo-HTML | Out-String \r\n    #   Get-MarkupTag "tr" $text\r\n    param(\r\n        [Parameter(ValueFromPipeline=$true,Position=0)]$tag,\r\n        [Parameter(Position=1)[string]$html)\r\nbegin {\r\n    \r\n        $replacements = @{\r\n            "<BR>" = "<BR \/>"\r\n            "<HR>" = "<HR \/>"\r\n            "&nbsp;" = " "\r\n            '&macr;'='¯'\r\n            '&ETH;'='Ð'\r\n            '&para;'='¶'\r\n            '&yen;'='¥'\r\n            '&ordm;'='º'\r\n            '&sup1;'='¹'\r\n            '&ordf;'='ª'\r\n            '&shy;'='­'\r\n            '&sup2;'='²'\r\n            '&Ccedil;'='Ç'\r\n            '&Icirc;'='Î'\r\n            '&curren;'='¤'\r\n            '&frac12;'='½'\r\n            '&sect;'='§'\r\n            '&Acirc;'='â'\r\n            '&Ucirc;'='Û'\r\n            '&plusmn;'='±'\r\n            '&reg;'='®'\r\n            '&acute;'='´'\r\n            '&Otilde;'='Õ'\r\n            '&brvbar;'='¦'\r\n            '&pound;'='£'\r\n            '&Iacute;'='Í'\r\n            '&middot;'='·'\r\n            '&Ocirc;'='Ô'\r\n            '&frac14;'='¼'\r\n            '&uml;'='¨'\r\n            '&Oacute;'='Ó'\r\n            '&deg;'='°'\r\n            '&Yacute;'='Ý'\r\n            '&Agrave;'='À'\r\n            '&Ouml;'='Ö'\r\n            '&quot;'='"'\r\n            '&Atilde;'='Ã'\r\n            '&THORN;'='Þ'\r\n            '&frac34;'='¾'\r\n            '&iquest;'='¿'\r\n            '&times;'='×'\r\n            '&Oslash;'='Ø'\r\n            '&divide;'='÷'\r\n            '&iexcl;'='¡'\r\n            '&sup3;'='³'\r\n            '&Iuml;'='Ï'\r\n            '&cent;'='¢'\r\n            '&copy;'='©'\r\n            '&Auml;'='Ä'\r\n            '&Ograve;'='Ò'\r\n            '&Aring;'='Å'\r\n            '&Egrave;'='È'\r\n            '&Uuml;'='Ü'\r\n            '&Aacute;'='Á'\r\n            '&Igrave;'='Ì'\r\n            '&Ntilde;'='Ñ'\r\n            '&Ecirc;'='Ê'\r\n            '&cedil;'='¸'\r\n            '&Ugrave;'='Ù'\r\n            '&szlig;'='ß'\r\n            '&raquo;'='»'\r\n            '&euml;'='ë'\r\n            '&Eacute;'='É'\r\n            '&micro;'='µ'\r\n            '&not;'='¬'\r\n            '&Uacute;'='Ú'\r\n            '&AElig;'='Æ'\r\n            '&euro;'= "€"        \r\n        }\r\n    \r\n}\r\nprocess {\r\n\r\n        foreach ($r in $replacements.GetEnumerator()) {\r\n            $l = 0 \r\n            do {\r\n                $l = $html.IndexOf($r.Key, $l, [StringComparison]"CurrentCultureIgnoreCase")\r\n                if ($l -ne -1) {\r\n                    $html = $html.Remove($l, $r.Key.Length)\r\n                    $html = $html.Insert($l, $r.Value)\r\n                }\r\n            } while ($l -ne -1)         \r\n        }\r\n     \r\n        $r = New-Object Text.RegularExpressions.Regex ('<\/' + $tag + '>'), ("Singleline", "IgnoreCase")\r\n        $endTags = @($r.Matches($html))\r\n        $r = New-Object Text.RegularExpressions.Regex ('<' + $tag + '[^>]*>'), ("Singleline", "IgnoreCase")\r\n        $startTags = @($r.Matches($html))\r\n        $tagText = @()\r\n        if ($startTags.Count -eq $endTags.Count) {\r\n            $allTags = $startTags + $endTags | Sort-Object Index   \r\n            $startTags = New-Object Collections.Stack\r\n            foreach($t in $allTags) {\r\n                if (-not $t) { continue } \r\n                if ($t.Value -like "<$tag*") {\r\n                    $startTags.Push($t)\r\n                } else {\r\n                    $start = $startTags.Pop()\r\n                    $tagText+=($html.Substring($start.Index, $t.Index + $t.Length - $start.Index))\r\n                }\r\n            }\r\n        } else {\r\n            # Unbalanced document, use start tags only and make sure that the tag is self-enclosed\r\n            $startTags | Foreach-Object {\r\n                $t = "$($_.Value)"\r\n                if ($t -notlike "*\/>") {\r\n                    $t = $t.Insert($t.Length - 1, "\/")\r\n                }\r\n                $tagText+=$t\r\n            } \r\n        }\r\n        foreach ($t in $tagText) {\r\n            if (-not $t) {continue }\r\n            # Correct HTML which doesn't quote the attributes so it can be coerced into XML\r\n            $inTag = $false\r\n            for ($i = 0; $i -lt $t.Length; $i++) {\r\n                if ($t[$i] -eq "<") {\r\n                    $inTag = $true\r\n                } else {\r\n                    if ($t[$i] -eq ">") {\r\n                        $inTag = $false\r\n                    }\r\n                }\r\n                if ($inTag -and ($t[$i] -eq "=")) {\r\n                    if ($t[$i + 1] -notmatch '[''|"]') {\r\n                        $endQuoteSpot = $t.IndexOfAny(" >", $i + 1)\r\n                        # Find the end of the attribute, then quote\r\n                        $t = $t.Insert($i + 1, "'")\r\n                        $t = $t.Insert($endQuoteSpot + 1, "'")                    \r\n                        $i = $endQuoteSpot\r\n                    } else {\r\n                        # Make sure the quotes are correctly formatted, otherwise,\r\n                        # end the quotes manually\r\n                        $whichQuote = "$($Matches.Values)"\r\n                        $endQuoteSpot = $t.IndexOf($whichQuote, $i + 2)\r\n                        $i = $endQuoteSpot\r\n                    }\r\n                }\r\n            }        \r\n            $t | Select-Object @{\r\n                Name='Tag'\r\n                Expression={$_}\r\n            }, @{\r\n                Name='Xml'\r\n                Expression= {\r\n                    ([xml]$t).$tag      \r\n                    trap {\r\n                        Write-Verbose ($_ | Out-String) \r\n                        continue\r\n                    }\r\n                }\r\n            }    \r\n        }\r\n    \r\n}\r\n\r\n}\r\n    <\/pre>\n<\/blockquote>\n<\/p>\n
<\/p>\n
Hope this Helps,<\/p>\n
James Brundage [MSFT]<\/p>\n
Automatically generated with Write-CommandBlogPost<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
On my personal blog (Media And Microcode), I’ve been posting a series called "Scripting the Web", which introduced a function called Get-MarkupTag. Get-MarkupTag is a very handy little function that coerces individual tag elements of a web page into HTML, so you can scrape data from a webpage. I’ve updated Get-MarkupTag a tiny bit for […]<\/p>\n","protected":false},"author":600,"featured_media":13641,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[97,137,175],"class_list":["post-17812","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-powershell","tag-advanced-functions","tag-ctp3","tag-get-markuptag"],"acf":[],"blog_post_summary":"
On my personal blog (Media And Microcode), I’ve been posting a series called "Scripting the Web", which introduced a function called Get-MarkupTag. Get-MarkupTag is a very handy little function that coerces individual tag elements of a web page into HTML, so you can scrape data from a webpage. I’ve updated Get-MarkupTag a tiny bit for […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts\/17812","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/users\/600"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/comments?post=17812"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts\/17812\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/media\/13641"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/media?parent=17812"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/categories?post=17812"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/tags?post=17812"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}