{"id":2775,"date":"2017-04-10T18:22:40","date_gmt":"2017-04-11T01:22:40","guid":{"rendered":"https:\/\/www.microsoft.com\/reallifecode\/?p=2775"},"modified":"2020-03-15T05:44:35","modified_gmt":"2020-03-15T12:44:35","slug":"object-detection-using-cntk","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/object-detection-using-cntk\/","title":{"rendered":"Object Detection Using Microsoft CNTK"},"content":{"rendered":"

We recently collaborated with InSoundz<\/a>, an audio-tracking startup, to build an object detection system using Microsoft\u2019s open source deep learning framework, Computational Network Toolkit (CNTK)<\/a>.<\/p>\n

The Problem<\/h2>\n

InSoundz captures and models 3D audio of live sports events to enhance live video feeds of these events for fans.\u00a0In order to enable automatic discovery of interesting scenarios that would be relevant to their solution, InSoundz wanted to integrate\u00a0object detection capabilities into their system.<\/p>\n

Any solution needed to be as flexible as possible, and also had to support adding new object types and creating detectors for various data types with ease. Since the object detection component evaluates images from a live camera feed, the detection also had to be fast, with near real-time performance.<\/p>\n

Object Detection vs. Object Recognition<\/h3>\n

Often when people talk about \u201cobject detection,\u201d they actually mean a combination of object detection<\/strong> (e.g. where is the cat\/dog in this image?<\/em>) and object recognition<\/strong> (e.g. is this a cat or a dog?<\/em>). That is, they mean that the algorithm should solve a combination of two problems: detecting both where<\/em> there is an object in a given image, then recognizing what<\/em> that object is. In this post, we will use this more common definition.<\/p>\n

In practice, the task of finding where an object is translates to finding a small bounding box that surrounds the object. While the tasks of recognition and object detection are both well-studied in the domain of computer vision, up until recently they were mainly solved using \u201cclassic\u201d approaches. These methods utilized local image features like Scale Invariant Feature Transform (SIFT)<\/a> and Histogram of Oriented Gradients (HOG)<\/a>.<\/p>\n

Classic algorithms for pedestrian detection<\/a>, for example, scan different regions in an image using a grid-like approach. They use the HOG features of the region and a pre-trained linear classifier, like a support vector machine (SVM)<\/a>, and decide which region contains an image of a pedestrian.<\/p>\n

In recent years, however, deep learning methods like convolutional neural networks (CNNs)<\/a>\nhave become the prominent tool for object recognition tasks. Due to their outstanding performance in comparison to the classical methods, deep learning methods also became a popular tool for object detection based tasks.<\/p>\n

The Solution<\/h2>\n

One such deep learning-based method for object detection is the Fast-RCNN algorithm<\/a>. Ross Girschik<\/a>\u00a0developed the method during his time in MSR<\/a> and it is considered to be one of the top methods in the field of object detection and recognition. In our solution, we utilized the CNTK implementation of the Fast-RCNN algorithm<\/a>.<\/p>\n

The Fast-RCNN method for object detection provides the following possible capabilities:<\/p>\n