{"id":2553,"date":"2017-04-10T18:17:26","date_gmt":"2017-04-10T18:17:26","guid":{"rendered":"https:\/\/www.microsoft.com\/reallifecode\/?p=2553"},"modified":"2020-03-25T17:23:15","modified_gmt":"2020-03-26T00:23:15","slug":"end-end-object-detection-box","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/end-end-object-detection-box\/","title":{"rendered":"End to End Object Detection in a Box"},"content":{"rendered":"

This code story outlines a new end to end video tagging tool, built on top of the Microsoft Cognitive Toolkit (CNTK)<\/a>, that enables developers to more easily create, review and iterate their own object detection models.<\/p>\n

\n

Background<\/h2>\n

We recently worked with\u00a0Insoundz<\/a>, an Israeli startup that captures sound at live sports events using audio-tracking technology. In order to map this captured audio, they needed a way to identify areas of interest in live video feeds. For this purpose, we worked with them\u00a0to build an object detection model with CNTK<\/a>.<\/p>\n

Object detection models require a large quantity of tagged image data to work in production. The training data for an object detection model consists of a set of images, where each image is associated with a group of bounding boxes surrounding the objects in the image, and each bounding box is assigned a label that describes the object.<\/p>\n

\"Image
Tagged Image Data<\/figcaption><\/figure><\/p>\n
\n

Having quality training data is one of the most important aspects of the model building process. However, in most existing object detection training pipelines, image datasets are compiled and expanded upon independently of training. As a result, models can only be optimized by fine tuning hyperparameters and by hoping that requests for additional training data encapsulate the edge cases of an existing model.<\/p>\n<\/div>\n

\n

Since Insoundz\u2019s capture solution supports a wide set of event scenarios (each with unique areas of interest) they needed a scalable way to aggregate data as they train, validate, and iterate new object detection models.<\/p>\n

The Solution<\/h2>\n

To enable Insoundz\u00a0to generate large quantities of high quality, tagged image data quickly, we developed a semi-automated, cross-platform Electron<\/a>\u00a0application. This video tagging application exports data directly to CNTK format and allows users to run and validate a trained model on new videos to generate stronger models.\u00a0The application supports Windows, OSX, and Linux.<\/p>\n

\"tag
Object Detection in a Box Architecture<\/strong><\/figcaption><\/figure><\/p>\n

One major advantage of this object detection training architecture over other publicly available architectures, is the validation feedback loop<\/em>. The feedback loop increases iterative model performance while decreasing the number of training samples needed for training.<\/p>\n

How the tool works<\/h3>\n

The Visual Object Tagging Tool supports the following features<\/strong>:<\/p>\n