Deploying AI with Azure IoT Hub
Tiliter creates computer vision software and hardware that automate and enhance everyday processes. Early adoption of our technology was in retail, on identification processes such as item recognition and in-store detection. Today, we are growing out our AI for a myriad of detection applications, and we have customers on four continents. And we couldn’t have arrived at this point without the right infrastructure to support our deployment strategy.
The challenges of managing global deployment
Anyone who knows anything about deep learning computer vision technology knows there has been a step-change in how the world approaches computer vision problems within the last decade.
Convolutional Neural Networks (CNNs) have proved to be a workhorse for computer vision applications in numerous use cases, from identifying cancer to helping self-driving cars see. However, we are still facing unsolved challenges of deploying these technologies at scale.
From the early days at Tiliter, we were shipping our products all over the globe. This was great for us as a business. However, as a small start-up with a handful of engineers, managing these deployments in multiple regions was not without its barriers. We needed to offer our customers a reliable product, consistency and speed.
Software versioning and deployment is often challenging regardless of how sophisticated your deployment environment and processes. Managing versioning and deployment with edge devices, often in diverse customer labs, is even more so.
How Tiliter stays agile with a small team
As a scaling business, our goal is to be proactive as much as possible — it is vital for us to iterate, but we also need to respond to customer requirements quickly. This often means creating bespoke, customer-specific software modifications that need to be deployed to remote devices or device groups fast.
Working responsively and moving quickly means that it is essential to be able to roll back too. Our customers expect that flexibility from us.
To help our small team keep our customers’ devices up to date with the latest iterations or changes, we use the following stack:
- Azure IoT Hub
- Containerized software using Docker
- Azure Container Registry
Figure 1: Docker container layers. Layers are stacked to make it easier to update components, such as the User interface or Neural Network Model, without updating the core software layers.
Azure IoT Hub is a PaaS that enables organizations to manage IoT and Edge devices management. The IoT Hub is a great out of the box tool that gives us a cloud-hosted backend to connect our devices virtually. It allows developers to easily deploy containerized applications on remote edge devices. So, we can remotely manage our devices and scale our deployments with ease. Containers are created in a special way, using several layers. As each layer is cached independently on the client devices, if a new version of a container only changes the latest layer, the client will only download this layer. Using the Azure IoT Hub also means we have the added benefit of knowing our devices are securely connected to the internet without building or maintaining this service in-house.
Containerizing our software offers a wide range of advantages, not least the ability to update individual components without pushing an entire software update.
For example, several of the updates we do involve just updating the user interface. In this case, we don’t need to push the whole container every time, just the user interface component.
- What makes this simple is having our devices connected to Azure IoT Hub. To trigger an update, we simply change the target container in the IoT Hub mirror, and the rest is taken care of.
Figure 2: IoT hub managing two deployments with multiple devices. The container registry holds multiple software. Deployment A and Deployment B are allocated Container Version 156.
Figure 3: Container Version 156 vs Container Version 157. The difference is shown only in the User Interface layer.
Figure 4: Following figure 2. IoT hub managing two deployments with multiple devices. Deployment B is updated to Container Version 157, only the difference is pushed to update i.e. the user interface layer (see Figure 3).
The IoT Edge Agent on the devices – a fully managed service built on the IoT Hub — pulls the additional layer and transitions the software to the new container once complete. This approach means the new container involves a single additional layer being added to the existing container. And typically, this update is small enough to retain the previous component of the old container.
This turns out to be very handy when rolling changes back since no additional data needs to be pulled. A rollback means changing the target container in the device mirror back to the previous version, and the IoT Edge Agent takes care of the orchestration.
Tiliter looks to the future
Azure IoT has allowed us to deploy our software to any of our devices in any region with ease. A great advantage for any business in the start-up and scale-up space. With the added benefits of a trusted name, stringent security protocols and a proven solution underpinning our deployment, we look forward to bringing the next generation of Tiliter’s products to market.
Light
Dark
0 comments