Computer Vision Made Easy with Cognitive Services

Premier Developer

Premier

This post is provide by App Dev Manager, Andrew Kanieski, who takes K9 obedience training to a new level with computer vision and Cognitive Services.


Computer vision is truly amazing technology! It can be used to distill rich information from images that can help you breathe life into your applications. Microsoft’s Cognitive Services – Computer Vision API gives developers the power to develop their software with the ability to truly see and comprehend the world around it.

When I come upon a new technology I try to make application in my life. I ask myself how could I see myself using this? There are many ways in which we can use Computer Vision to make our day to day lives easier!

Context: Enter the family dog

One problem in my home we always struggle with is keeping our family dog
out of our bedrooms. As much as we train him, he always sneaks in to nap on our beds. How can we write software to help us train our family friend?

Problem: I need to be alerted when our dog enters a room

This is where I applied my new found knowledge of the Computer Vision API. What if every time my dog entered our bedroom I was alerted via SMS message?

Prerequisites
  • IP Camera (or other means of collecting images) – I am using an
    Amcrest IP2M-841B
  • .NET Core Installed – Install Instructions
    Here
  • VSCode (or other editor) – Install Instructions
    Here
  • Twilio Account (Trial or Subscriber) – More Info on Twilio + Azure
    Here

Assuming you have .NET Core installed, let’s start by creating a new .NET Core Console Application:

From console:

This will create our initial application with a Program.cs and a dog-watcher.csproj

In Program.cs I added the needed variables that will be passed into our application via Environment. Other means of storing and working with configuration are also fine. I just happen to prefer environment variables.

In this case I am using an IP based camera that provides you with a REST API for getting a snapshot from the camera via HTTP.

Now that we have our needed configuration collected, let’s begin by creating logic to fetch our snapshots. Our IP camera uses HTTP GET with basic auth.

First, lets add System.Net.Http;

Now you can go ahead and use the System.Net.Http namespace.

Program.cs

Now that we have logic to collect images, let’s create some classes that match with the response from the Computer Vision API. This is only mapping Tags since that is the data element we are working with. In this case tags will provide us with a list of descriptors that computer vision has identified and with what level of confidence it feels those descriptors are identified.

Now let’s post those images to Cognitive Services for analysis. Notice the use of the variable AZURE_VISION_URL and AZURE_VISION_KEY these are coming directly from the Azure Portal after we’ve gone and added our Coginitive Services subscription.

Now let’s wire up the loop that watches our room.

Note: that trial Twilio accounts can only send SMS messages to verified devices.

Program.cs

Now we can simply adjust our camera for ideal viewing angle and run our application.

Conclusion

Working with Microsoft’s Cognitive Services is an easy way to build intelligent applications. Additionally, Azure provides a seamless hosting environment for applications that interact with Cognitive Services.


Premier Support for Developers provides strategic technology guidance, critical support coverage, and a range of essential services to help teams optimize development lifecycles and improve software quality.  Contact your Application Development Manager (ADM) or email us to learn more about what we can do for you.

Premier Developer
Premier Developer

Premier Support for Developers

Follow Premier   

0 comments

    Leave a comment