OCR<\/strong><\/a>. The data is then used to train a predictive model using a multiclass neural network (with default settings), and finally published as a web service.<\/p>\nDataset<\/h2>\n
We are basing our example on a private dataset of ~1200 images of receipts of different expense types, such as snacks, groceries, dining, clothes, fuel and entertainment. The figure below shows the distribution of these six classes.\n![\"jpg:](\"https:\/\/devblogs.microsoft.com\/cse\/wp-content\/uploads\/sites\/55\/2017\/01\/data-distribution-6-classes.jpg\")
<\/p>\n
Below is an example of how you can call Microsoft\u2019s Cognitive Services from within Azure ML Studio using the Execute Python Script<\/code><\/strong> module. The Python code below will extract texts out from those images via Microsoft\u2019s OCR. This code should reside within the Execute Python Script<\/code><\/strong> module.<\/p>\nThe snippet below shows the required packages and sets the URL for OCR in the Vision API from Microsoft Cognitive Services.<\/p>\n
\n
# The script MUST contain a function named azureml_main<\/span>\r\n # which is the entry point for this module.<\/span>\r\n\r\n # imports up here can be used to <\/span>\r\n import<\/span> pandas<\/span> as<\/span> pd<\/span>\r\n import<\/span> json<\/span>\r\n import<\/span> time<\/span>\r\n import<\/span> requests<\/span>\r\n from<\/span> io<\/span> import<\/span> StringIO<\/span>\r\n \r\n # url for Microsoft's Cognitive Services - Vision API - OCR<\/span>\r\n #_url = 'https:\/\/api.projectoxford.ai\/vision\/v1.0\/ocr' # previous url, still work<\/span>\r\n _url<\/span> =<\/span> 'https:\/\/westus.api.cognitive.microsoft.com\/vision\/v1.0\/ocr'<\/span> # latest url<\/span>\r\n \r\n # maximum number of retries when posting a request<\/span>\r\n _maxNumRetries<\/span> =<\/span> 10<\/span>\r\n<\/code><\/pre>\n<\/div>\nBelow is the entry point function for Execute Python Script<\/code><\/strong> module within Azure ML Studio experiment. It sets up parameters for the OCR API, processes requests, and returns a new data frame which contains text extracted from a receipt, and its associated label (that is, its expensing category).<\/p>\n\n
# The entry point function can contain up to two input arguments:<\/span>\r\n # Param<dataframe1>: a pandas.DataFrame<\/span>\r\n # Param<dataframe2>: a pandas.DataFrame<\/span>\r\n def<\/span> azureml_main<\/span>(<\/span>dataframe1<\/span> =<\/span> None<\/span>,<\/span> dataframe2<\/span> =<\/span> None<\/span>):<\/span>\r\n\r\n # Get the OCR key<\/span>\r\n VISION_API_KEY<\/span> =<\/span> str<\/span>(<\/span>dataframe2<\/span>[<\/span>'Col1'<\/span>][<\/span>0<\/span>])<\/span>\r\n \r\n # Load the file containing image url and label<\/span>\r\n df_url_label<\/span> =<\/span> dataframe1<\/span>\r\n \r\n # create an empty pandas data frame<\/span>\r\n df<\/span> =<\/span> pd<\/span>.<\/span>DataFrame<\/span>({<\/span>'Text'<\/span> :<\/span> [],<\/span> 'Category'<\/span> :<\/span> [],<\/span> 'ReceiptID'<\/span> :<\/span> []})<\/span>\r\n \r\n # extract image url, setting OCR API parameters, process request<\/span>\r\n for<\/span> index<\/span>,<\/span> row<\/span> in<\/span> df_url_label<\/span>.<\/span>iterrows<\/span>():<\/span>\r\n imageurl<\/span> =<\/span> row<\/span>[<\/span>'Url'<\/span>]<\/span>\r\n \r\n # setting OCR parameters<\/span>\r\n params<\/span> =<\/span> {<\/span> 'language'<\/span>:<\/span> 'en'<\/span>,<\/span> 'detectOrientation '<\/span>:<\/span> 'true'<\/span>}<\/span> \r\n headers<\/span> =<\/span> dict<\/span>()<\/span>\r\n headers<\/span>[<\/span>'Ocp-Apim-Subscription-Key'<\/span>]<\/span> =<\/span> VISION_API_KEY<\/span>\r\n headers<\/span>[<\/span>'Content-Type'<\/span>]<\/span> =<\/span> 'application\/json'<\/span> \r\n \r\n image_url<\/span> =<\/span> {<\/span> 'url'<\/span>:<\/span> imageurl<\/span> }<\/span> ;<\/span> \r\n image_file<\/span> =<\/span> None<\/span>\r\n result<\/span> =<\/span> processRequest<\/span>(<\/span> image_url<\/span>,<\/span> image_file<\/span>,<\/span> headers<\/span>,<\/span> params<\/span> )<\/span>\r\n \r\n if<\/span> result<\/span> is<\/span> not<\/span> None<\/span>:<\/span>\r\n # extract text<\/span>\r\n text<\/span> =<\/span> extractText<\/span>(<\/span>result<\/span>);<\/span> \r\n \r\n # populate dataframe<\/span>\r\n df<\/span>.<\/span>loc<\/span>[<\/span>index<\/span>,<\/span>'Text'<\/span>]<\/span> =<\/span> text<\/span>\r\n else<\/span>:<\/span>\r\n # populate dataframe<\/span>\r\n df<\/span>.<\/span>loc<\/span>[<\/span>index<\/span>,<\/span>'Text'<\/span>]<\/span> =<\/span> None<\/span>\r\n \r\n # 'Category' is the label<\/span>\r\n df<\/span>.<\/span>loc<\/span>[<\/span>index<\/span>,<\/span>'Category'<\/span>]<\/span> =<\/span> row<\/span>[<\/span>'Category'<\/span>]<\/span>\r\n df<\/span>.<\/span>loc<\/span>[<\/span>index<\/span>,<\/span>'ReceiptID'<\/span>]<\/span> =<\/span> imageurl<\/span>[<\/span>-<\/span>17<\/span>:<\/span>-<\/span>4<\/span>]<\/span>\r\n \r\n # Return value must be a sequence of pandas.DataFrame<\/span>\r\n return<\/span> df<\/span>\r\n<\/code><\/pre>\n<\/div>\nextractText<\/code> seeks and extracts only texts recognized by OCR, and ignores other information such as regions<\/code>, lines<\/code> and words<\/code>. While that information is not utilized in this example, it could be useful if the location of the text is of interest.<\/p>\n\n
# Extract text only from OCR's response<\/span>\r\n def<\/span> extractText<\/span>(<\/span>result<\/span>):<\/span>\r\n text<\/span> =<\/span> \"\"<\/span>\r\n for<\/span> region<\/span> in<\/span> result<\/span>[<\/span>
![\"Jpg:](\"https:\/\/devblogs.microsoft.com\/cse\/wp-content\/uploads\/sites\/55\/2017\/01\/mlstudio-main.jpg\")
<\/p>\n