Azure IoT Edge went generally available in June, with official support for AMD64 and ARM32 platforms. Recently, more and more developers from the community requested to run Azure IoT Edge on ARM64 (a.k.a. AArch64) devices such NVIDIA TX2. While the ARM64 support of Azure IoT Edge is still in progress, this post will introduce a workaround.

When we talk about “running Azure IoT Edge”, what we actually mean is running both the Azure IoT Edge security daemon (including its dependencies such as hsmlib) and the Azure IoT Edge runtime images (i.e., edgeAgent and edgeHub). As a result, the workaround requires changes to the official tutorial for these two parts.

IoT Edge Security Daemon

To run IoT Edge security daemon on ARM64 devices, you need to either:

1. Install ARM32 (a.k.a., armhf) architecture Azure IoT Edge security daemon, hsmlib, and all their dependencies libs. @marktayl1 has provided installation steps in this gist
2. Compile both Azure IoT Edge security daemon and hsmlib for ARM64 from source code, or use the pre-built binaries from the community (for example, @vjrantal has provided pre-built 1.0.2 version)

Cross-compile IoT Edge Security Daemon and hsmlib for ARM64 on Linux AMD64 Machines

You can skip this section if you would like to use pre-built binaries from the community.

1. Install Docker-CE, and follow this tutorial to configure managing Docker as a non-root user
2. Install rustup: curl https://sh.rustup.rs -sSf | sh, and add Rust toolchain to $PATH: source ~/.cargo/env
3. Clone the source code of IoT Edge OSS project: $ git clone https://github.com/Azure/iotedge.git
4. $ cd iotedge/edgelet/build/linux/docker/cross-compile
5. Build the container image with cross-compile toolchain: $ ubuntu16.04/build_arm64_toolchain_container.sh aarch64-unknown-linux-gnu

The above command targets ARM64 and Ubuntu 16.04. Read the README file to learn about the commands targeting other OS such as Debian and Centos, as well as what tools are inside the build container.

6. $ cd ../../ubuntu16.04/aarch64/
7. $ chmod +x *.sh
8. Open package-iotedged.sh and package-libiothsm.sh in text editors, and comment the line with docker pull "$IMAGE"

By default, the images built in Step 5 are tagged with a private registry which requires credentials to pull. By bypassing the docker pull "$IMAGE" step we will always use the locally built ones.

9. Cross-compile hsmlib: ./package-libiothsm.sh
10. Cross-compile IoT Edge Security Daemon: ./package-iotedged.sh

IoT Edge Runtime Images

For IoT Edge runtime images, edgeAgent (mcr.microsoft.com/azureiotedge-agent:1.0) and edgeHub (mcr.microsoft.com/azureiotedge-hub:1.0), the Docker tags are multi-arch. This means when you pull them on Linux AMD64 machines you get the Linux AMD64 variant, and when you pull them on ARM32 machines you get the ARM32 variant. However, these multi-arch tags don’t support ARM64 yet. When you pull them on ARM64 machines, Docker daemon will try the find the ARM64 variant and finally get a 404 error. To workaround this, you need to explicitly specify the ARM32 variants, for example, mcr.microsoft.com/azureiotedge-agent:1.0.4-linux-arm32v7 and mcr.microsoft.com/azureiotedge-hub:1.0.4-linux-arm32v7.

Full Steps to Run the Simulated Temperature Sensor

Here are the full steps I used to set up IoT Edge runtime on NVIDIA TX2 running Ubuntu 16.04 and deploy the simulated temperature sensor module (a.k.a., the Hello World for Azure IoT Edge):

1. Install Docker CE

   You may need to check the Tegra-Docker project to learn how to enable Docker support on NVIDIA Tegra devices. Others from the community reported Moby also worked for them.

   1. $ sudo apt-get update
   2. $ sudo apt-get install apt-transport-https ca-certificates curl software-properties-common
   3. $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
   4. $ sudo add-apt-repository "deb [arch=arm64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
   5. $ sudo apt-get update
   6. $ sudo apt-get install docker-ce
   7. Make sure Docker runs successfully: $ sudo docker run hello-world
2. Install the ARM64 IoT Edge daemon and hsmlib(here we take the 1.0.2 version pre-built by @vjrantal for example)
   1. $ wget https://github.com/vjrantal/iot-edge-darknet-module/files/2423742/iotedge-libiothsm-std-aarch64.zip
   2. Unzip iotedge-libiothsm-std-aarch64.zip
   3. $ sudo apt-get install ./libiothsm-std-1.0.2-aarch64.deb
   4. $ sudo apt-get install ./iotedge_1.0.2-1_aarch64.deb
3. Create an IoT Hub
4. Create an IoT Edge device and retrieve the device connection string
5. Update IoT Edge Runtime configurations: sudo vi /etc/iotedge/config.yaml
   1. Set provisioning -> device_connection_string with the device connection string retrieved in the last step
   2. Set agent -> config -> image with mcr.microsoft.com/azureiotedge-agent:1.0.4-linux-arm32v7
6. Restart IoT Edge runtime: $ sudo systemctl restart iotedge
7. Check IoT Edge runtime status: $ sudo systemctl status iotedge 
8. Deploy simulated temperature sensor module
   1. In the 5th step of the above linked tutorial, use mcr.microsoft.com/azureiotedge-simulated-temperature-sensor:1.0.4-linux-arm32v7 instead of mcr.microsoft.com/azureiotedge-simulated-temperature-sensor:1.0.
   2. Before the 9th step of the above linked tutorial, click the “Configure advanced Edge runtime settings” button 
      1. Set Edge Hub’s Image with mcr.microsoft.com/azureiotedge-hub:1.0.4-linux-arm32v7
      2. Set Edge Agent’s Image with mcr.microsoft.com/azureiotedge-agent:1.0.4-linux-arm32v7
9. Wait for about a few seconds, check the status of edgeAgent, edgeHub and the simulated temperature sensor module
   1. $ sudo iotedge list 
   2. $ sudo docker ps 
   3. $ sudo docker logs -f <module_name> to check module logs，for example: