{"id":44971,"date":"2023-03-21T12:25:54","date_gmt":"2023-03-21T19:25:54","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=44971"},"modified":"2024-12-13T14:16:50","modified_gmt":"2024-12-13T22:16:50","slug":"securing-containers-with-rootless","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/securing-containers-with-rootless\/","title":{"rendered":"Secure your .NET cloud apps with rootless Linux Containers"},"content":{"rendered":"
\n

This post was updated on April 12, 2024<\/strong> to reflect the latest releases.<\/p>\n<\/blockquote>\n

Starting with .NET 8<\/a>, all of our Linux container images will include a non-root user<\/a>. You’ll be able to host your .NET containers as a non-root user with one line of code. This platform-level change will make your apps more secure and .NET one of the most secure developer ecosystems. It is a small change with a big impact for defense in depth<\/a>.<\/p>\n

This change was inspired by our earlier project enabling .NET in Ubuntu Chiseled containers<\/a>. Chiseled (AKA “distroless”) images are intended to be appliance-like so non-root was an easy design choice for those images. We realized that we could apply the non-root feature of Chiseled containers to all the container images we publish. By doing that, we’ve raised the security bar for .NET container images.<\/p>\n

This post is about the benefit of non-root containers, workflows for creating them, and how they work. A follow-on post will discuss how to best use these images with Kubernetes. Also, if you want a simpler option, you should check out built-in container support for the .NET SDK<\/a>.<\/p>\n

\n

Note: .NET Chiseled images<\/a> are now GA.<\/p>\n<\/blockquote>\n

Least privilege<\/h2>\n

Hosting containers as non-root aligns with principle of least privilege<\/a>. It’s free security provided by the operating system. If you run your app as root<\/code>, your app process can do anything in the container, like modify files, install packages, or run arbitrary executables. That’s a concern if your app is ever attacked. If you run your app as non-root, your app process cannot do much, greatly<\/em> limiting what a bad actor could accomplish<\/a>.<\/p>\n

Non-root containers can also be thought of as contributing to secure supply chain<\/a>. Most of the time, people talk about secure supply chain in terms of blocking bad dependency updates or auditing component pedigree<\/a>. Non-root containers come after those two topics. If a bad dependency slips through your process (and there is a probability that one will), then a non-root container may be your best last defense. Kubernetes hardening best practices<\/a> require running containers with a non-root user for this same reason.<\/p>\n

Meet app<\/code><\/h2>\n

All of our Linux images — starting with .NET 8 — will contain an app<\/code> user. The app<\/code> user will be able to run your app, but won’t be able to delete or change any of files that come with the container image (unless you explicitly allow that). The naming is appropriate since the user can do little more than run your app.<\/p>\n

The user app<\/code> isn’t actually new. It is the same one we’re using for our Ubuntu Chiseled images<\/a>. That’s a key design point. Starting with .NET 8, all of our Linux container images will contain the app<\/code> user. That means that you can switch between the images we offer, and the user and uid will be the same.<\/p>\n

I’ll describe the new experience in terms of the docker<\/code> CLI.<\/p>\n

Meet app<\/code>.<\/p>\n

$ docker run --rm mcr.microsoft.com\/dotnet\/aspnet:8.0 cat \/etc\/passwd | tail -n 1\r\napp:x:1654:1654::\/home\/app:\/bin\/sh<\/code><\/pre>\n
That’s the last line of the \/etc\/passwd<\/code> file<\/a> in our images. That’s the file that Linux uses for managing users.<\/p>\n
We selected a uid just above 1000<\/a> to avoid reserved ranges<\/a>. We also decided that this user should have a home directory<\/a>.<\/p>\n
$ docker run --rm -u app mcr.microsoft.com\/dotnet\/aspnet:8.0 bash -c \"cd && pwd\"\r\n\/home\/app<\/code><\/pre>\n
We looked around a bit and discovered that Node.js, Ubuntu 23.04+, and Chainguard are all on this same plan. Nice!<\/p>\n
$ docker run --rm node cat \/etc\/passwd | tail -n 1\r\nnode:x:1000:1000::\/home\/node:\/bin\/bash\r\n$ docker run ubuntu:noble cat \/etc\/passwd | tail -n 1\r\nubuntu:x:1000:1000:Ubuntu:\/home\/ubuntu:\/bin\/bash\r\n$ docker cp $(docker create --name cg cgr.dev\/chainguard\/dotnet-runtime):\/etc\/passwd .\/passwd && docker rm cg && cat .\/passwd | tail -n 1 && rm .\/passwd\r\nnonroot:x:65532:65532:Account created by apko:\/home\/nonroot:\/bin\/sh<\/code><\/pre>\n
The last one is Chainguard<\/a>. Those images are structured differently (for good reason), so a different pattern was used. It is fine for everyone to create their own users, however, it is best to avoid matching UIDs.<\/p>\n
There are lots of users in container images<\/a>, but none of them are considered appropriate for this use case<\/a>. It would be nice to reduce the number of users<\/a>, however, that’s unlikely to happen and one of the benefits of using distroless\/Chiseled images.<\/p>\n
Windows Containers already have a non-admin capability<\/a>, with the ContainerUser<\/code> user. We opted against adding app<\/code> to our Windows Container images. You should follow Windows Team guidance on how to best secure Windows Container images.<\/p>\n
Using app<\/code><\/h2>\n
\n
“Non-root-capable”: Configure your container as non-root with a one-line USER<\/code> instruction.<\/p>\n<\/blockquote>\n
Docker<\/a> and Kubernetes<\/a> make it easy to specify the user you want to use for your container. It’s a one-liner. Per our definition, “non-root-capable” means you can switch to non-root as a one-liner. That’s very powerful, since the ease of a one-liner removes any reason to not run more securely.<\/p>\n
Note: aspnetapp<\/code> is used throughout as a substitute for your app.<\/p>\n
You can set the user via the CLI with -u<\/code>.<\/p>\n
$ docker run --rm -u app mcr.microsoft.com\/dotnet\/runtime-deps:8.0 whoami\r\napp<\/code><\/pre>\n
Specifying the user via the CLI is fine, but more for testing or diagnostic scenarios. It is best for production apps to define the USER<\/code> in a Dockerfile<\/a>, with the username or uid.<\/p>\n
As the user:<\/p>\n
USER app<\/code><\/pre>\n
As the UID:<\/p>\n
USER 1654<\/code><\/pre>\n
Via an environment variable for the UID<\/a>.<\/p>\n
USER $APP_UID<\/code><\/pre>\n
We consider this pattern a best practice because it makes it obvious which user you are using, avoids duplicated magic numbers, and uses a UID, all of which work well if you are using Kubernetes. We’ll have a post on non-root hosting with Kubernetes shortly.<\/p>\n
The following command demonstrates the value of the environment variable.<\/p>\n
docker run --rm -u app mcr.microsoft.com\/dotnet\/runtime-deps:8.0 bash -c \"echo \\$APP_UID\"\r\n1654<\/code><\/pre>\n
If you don’t do anything, everything will be the same as before and your image will continue to run as root<\/code>. We hope you take the extra (small) step and run your container as the app<\/code> user. You might be wondering why we didn’t switch to the non-root user by default. That will be covered in a later section.<\/p>\n
Switching to port 8080<\/code><\/h2>\n
The biggest sticking point of the project was the ports that we expose. In fact, it is so much of a sticking point that we had to make a breaking change<\/a>.<\/p>\n
We decided to standardize on port 8080<\/code> for all container images going forward. This decision was based on our earlier experience with Chiseled images, which already listen on port 8080<\/code><\/a>. All the images now match.<\/p>\n
However, ASP.NET Core apps (using our .NET 7 and earlier container images) listen on port 80<\/code><\/a>. The problem is that port 80 is a privileged port<\/a> that requires root<\/code> permission (at least in some places). That’s inherently incompatible with non-root containers.<\/p>\n
You can see how the ports are configured in our images.<\/p>\n
For .NET 8:<\/p>\n
$ docker run --rm mcr.microsoft.com\/dotnet\/aspnet:8.0 bash -c \"echo \\$ASPNETCORE_HTTP_PORTS\"\r\n8080<\/code><\/pre>\n
For .NET 7 (and earlier):<\/p>\n
$ docker run --rm mcr.microsoft.com\/dotnet\/aspnet:7.0 bash -c \"echo \\$ASPNETCORE_URLS\"\r\nhttp:\/\/+:80<\/code><\/pre>\n
Note: We also changed the environment variable we use to set the port. More on that shortly.<\/p>\n
Going forward, your port mapping will need to change. You can do this via the CLI. You’ll need 8080<\/code> on the right-hand of the mapping. The left-hand side can match or be another value.<\/p>\n
docker run --rm -it -p 8000:8080 aspnetapp<\/code><\/pre>\n
Some users will want to continue to use port 80 (and root<\/code>). You can still do that.<\/p>\n
You can re-define ASPNETCORE_HTTP_PORTS<\/code> in your Dockerfile or via the CLI.<\/p>\n
For Dockerfile:<\/p>\n
ENV ASPNETCORE_HTTP_PORTS=80<\/code><\/pre>\n
For Docker CLI:<\/p>\n
docker run --rm -e ASPNETCORE_HTTP_PORTS=80 -p 8000:80 aspnetapp<\/code><\/pre>\n
.NET 8 Windows Container images use port 8080<\/code> as well.<\/p>\n
$ docker run --rm mcr.microsoft.com\/dotnet\/aspnet:8.0-nanoserver-ltsc2022 cmd \/c \"set | findstr ASPNETCORE\"\r\nASPNETCORE_HTTP_PORTS=8080<\/code><\/pre>\n
ASPNETCORE_HTTP_PORTS<\/code><\/a> is a new environment variable for specifying the port (or ports) for ASP.NET Core (actually, Kestrel<\/a>) to listen on. It takes a semi-colon delimited list of port values. .NET 8 images use this new environment variable, instead of ASPNETCORE_URLS<\/code> (which is used in .NET 6 and 7 images). ASPNETCORE_URLS<\/code> remains a useful advanced feature. It enables specifying both raw HTTP and TLS ports in one configuration and overrides both ASPNETCORE_HTTP_PORTS<\/code> and ASPNETCORE_HTTPS_PORTS<\/code>.<\/p>\n
Non-root in action<\/h2>\n
Let’s take a look at what non-root looks like from a few different angles so that you can better understand what’s actually going on. I’m using Ubuntu 24.04.<\/p>\n
You can use our aspnetapp<\/a> sample to try this scenario yourself. It configures the container to always run as app<\/code>.<\/p>\n
$ pwd\r\n\/home\/rich\/git\/dotnet-docker\/samples\/aspnetapp\r\n$ cat Dockerfile | tail -n 2\r\nUSER $APP_UID\r\nENTRYPOINT [\".\/aspnetapp\"]\r\n$ docker build --pull -t aspnetapp -f Dockerfile .<\/code><\/pre>\n
Let’s see if we can observe the user in action, which has been set in the Dockerfile<\/a> we’re using.<\/p>\n
$ docker run --rm --name aspnetapp -d -p 8000:8080 aspnetapp\r\n5bde77feebdf76ff370815f41a8989a880d51a4037c91e2ac8c6f2c269b759ad\r\n$ curl http:\/\/localhost:8000\/Environment\r\n{\"runtimeVersion\":\".NET 8.0.4\",\"osVersion\":\"Debian GNU\/Linux 12 (bookworm)\",\"osArchitecture\":\"Arm64\",\"user\":\"app\",\"processorCount\":8,\"totalAvailableMemoryBytes\":4113694720,\"memoryLimit\":0,\"memoryUsage\":34250752,\"hostName\":\"ead528d37b0e\"}\r\n$ docker exec aspnetapp ls -l\r\ntotal 200\r\n-rw-r--r-- 1 root root   154 Feb 22 05:46 appsettings.Development.json\r\n-rw-r--r-- 1 root root   151 Jul 11  2023 appsettings.json\r\n-rwxr-xr-x 1 root root 72544 Apr 12 17:11 aspnetapp\r\n-rw-r--r-- 1 root root   457 Apr 12 17:11 aspnetapp.deps.json\r\n-rw-r--r-- 1 root root 61952 Apr 12 17:11 aspnetapp.dll\r\n-rw-r--r-- 1 root root 44696 Apr 12 17:11 aspnetapp.pdb\r\n-rw-r--r-- 1 root root   469 Apr 12 17:11 aspnetapp.runtimeconfig.json\r\ndrwxr-xr-x 5 root root  4096 Apr 12 17:11 wwwroot\r\n$ docker top aspnetapp  \r\nUID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD\r\n1654                7833                7815                0                   17:13               ?                   00:00:00            .\/aspnetapp<\/code><\/pre>\n
Notice the user<\/code> property in the JSON content returned from Environment<\/code> endpoint, above.<\/p>\n
You can see that application is running as app<\/code> and that the files are owned by root<\/code>. That means that the application files are protected from being altered by this user.<\/p>\n
From Dockerfile reference<\/a>:<\/p>\n
\n
All new files and directories are created with a UID and GID of 0, unless the optional --chown<\/code> flag specifies a given username, groupname, or UID\/GID combination to request specific ownership of the copied content<\/p>\n<\/blockquote>\n
Let’s try some rootful actions on this container, using docker exec<\/code> on the same container.<\/p>\n
$ docker exec aspnetapp rm aspnetapp.pdb\r\nrm: can't remove 'aspnetapp.pdb': Permission denied\r\n$ docker exec aspnetapp touch \/file\r\ntouch: \/file: Permission denied\r\n$ docker exec aspnetapp which dotnet\r\n\/usr\/bin\/dotnet\r\n$ docker exec aspnetapp rm \/usr\/bin\/dotnet\r\nrm: can't remove '\/usr\/bin\/dotnet': Permission denied\r\n$ docker exec aspnetapp apt-get update && apt-get install -y curl\r\nReading package lists...\r\nE: List directory \/var\/lib\/apt\/lists\/partial is missing. - Acquire (13: Permission denied)\r\n$ docker exec aspnetapp sudo apt-get update && sudo apt-get install -y curl\r\nOCI runtime exec failed: exec failed: unable to start container process: exec: \"sudo\": executable file not found in $PATH: unknown<\/code><\/pre>\n
The result: Permission denied<\/code> and sudo<\/code> isn’t present to work around that. That’s what we want. Let’s try again, but elevate to root<\/code>.<\/p>\n
$ docker exec -u root aspnetapp bash -c \"rm aspnetapp.pdb && ls aspnetapp.pdb\"\r\nls: aspnetapp.pdb: No such file or directory\r\n$ docker exec -u root aspnetapp bash -c \"touch \/file && ls \/file\"\r\n\/file\r\n$ docker exec -u root aspnetapp bash -c \"rm \/usr\/bin\/dotnet &&  ls \/usr\/bin\/dotnet\"\r\nls: \/usr\/bin\/dotnet: No such file or directory\r\n$ docker exec -u root aspnetapp bash -c \"apt-get update && apt-get install curl\"   \r\nGet:1 http:\/\/deb.debian.org\/debian bookworm InRelease [151 kB]\r\nGet:2 http:\/\/deb.debian.org\/debian bookworm-updates InRelease [55.4 kB]\r\nGet:3 http:\/\/deb.debian.org\/debian-security bookworm-security InRelease [48.0 kB]\r\nGet:4 http:\/\/deb.debian.org\/debian bookworm\/main arm64 Packages [8685 kB]\r\nGet:5 http:\/\/deb.debian.org\/debian bookworm-updates\/main arm64 Packages [12.5 kB]\r\nGet:6 http:\/\/deb.debian.org\/debian-security bookworm-security\/main arm64 Packages [147 kB]\r\nFetched 9099 kB in 2s (4099 kB\/s)\r\nReading package lists...\r\nReading package lists...\r\nBuilding dependency tree...\r\nReading state information...\r\nThe following additional packages will be installed:\r\n  krb5-locales libbrotli1 libcurl4 libgssapi-krb5-2 libk5crypto3 libkeyutils1\r\n  libkrb5-3 libkrb5support0 libldap-2.5-0 libldap-common libnghttp2-14 libpsl5\r\n  librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1\r\n  publicsuffix\r\n...\r\ncurl 7.88.1 (aarch64-unknown-linux-gnu) libcurl\/7.88.1 OpenSSL\/3.0.11 zlib\/1.2.13 brotli\/1.0.9 zstd\/1.5.4 libidn2\/2.3.3 libpsl\/0.21.2 (+libidn2\/2.3.3) libssh2\/1.10.0 nghttp2\/1.52.0 librtmp\/2.3 OpenLDAP\/2.5.13\r\nRelease-Date: 2023-02-20, security patched: 7.88.1-10+deb12u5\r\nProtocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp\r\nFeatures: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd<\/code><\/pre>\n
We can now kill the container.<\/p>\n
$ docker kill aspnetapp\r\naspnetapp\r\n$ docker ps\r\nCONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES<\/code><\/pre>\n
You can see that root<\/code> is able to do a lot more, in fact, anything it wants. After curl<\/code> is installed into the Debian image, an attacker could start executing scripts from any webserver they choose. In the case of Alpine, it ships with wget<\/code>, which removes a step in that chain.<\/p>\n
Surely, the answer is to remove the root<\/code> user to avoid these risks. No. In fact, removing the root<\/code> user has undefined behavior. The best option is to run as a non-root user. It removes a whole class of attacks via well-defined mechanisms.<\/p>\n
The use of docker exec -u root<\/code> might seem scary. If an attacker can run docker exec -u root<\/code> on your running container, then they already have access to the host, and you’re already in far more trouble than anything that is addressed by this post.<\/p>\n
What about sudo<\/code>? sudo<\/code> isn’t included in our images and never will be.<\/p>\n
Chiseled<\/h2>\n
We have a similar sample<\/a> that is hosted at mcr.microsoft.com<\/code>. It is more limited in what it can do, by design, by virtue of being based on an Ubuntu Chiseled image<\/a>.<\/p>\n
$ docker run --rm -d --name aspnetapp -p 8000:8080 mcr.microsoft.com\/dotnet\/samples:aspnetapp-chiseled \r\n1d2120f991562e51b65dd09fba693a68f6230e914c75e21a93811219bb52c16c\r\n$ curl http:\/\/localhost:8000\/Environment \r\n{\"runtimeVersion\":\".NET 8.0.4\",\"osVersion\":\"Ubuntu 22.04.4 LTS\",\"osArchitecture\":\"Arm64\",\"user\":\"app\",\"processorCount\":8,\"totalAvailableMemoryBytes\":4113694720,\"memoryLimit\":0,\"memoryUsage\":37683200,\"hostName\":\"1d2120f99156\"}\r\n$ docker kill aspnetapp\r\naspnetapp<\/code><\/pre>\n
This image also uses the app<\/code> user.<\/p>\n
Hosting in Azure container services<\/h2>\n
It is straightforward to adopt this pattern in Azure container services. There are two aspects to consider, the port and the user.<\/p>\n
Some container services offer a higher-level experience than Kubernetes and require a different configuration option.<\/p>\n