How to run rootless containers

Using rootless containers, or unprivileged containers, is one of the most effective container security best practices.

The thing is, you don’t need to run processes as root the vast majority of the time. On a bare-metal server, most services don’t run as root. So why is it that 80% of containers run as root?

The root of all evil (pun intended) comes from Docker’s defaults. If you don’t specify otherwise, Docker will run a container’s processes as UID 0, the root user. Developers became used to this: not providing unprivileged versions of their images, and other container runtimes maintained this behaviour to maintain compatibility.

Luckily, running unprivileged has gotten easier over time.

In this article, we will:

• Cover the risks of running as root.
• Discover how to adapt an image to run as an unprivileged user.
• Leverage User Namespaces to isolate privileged containers.

The dangers of running as root

In summary, we are making things easier for attackers if a container gets compromised.

When that happens, attackers will have root access — meaning full control — inside the container. This allows them to:

• Run processes and modify binaries.
• Find credentials in files you thought were secured, enabling lateral movement.

And overall, it’s easier for them to exploit vulnerabilities and break out into the host, as you can see in these blog posts:

• New runc vulnerabilities allow container escape.
• Detecting ‘Leaky Vessels’ Exploitation in Docker and Kubernetes.

That’s why running containers as rootless is such a good practice. It allows us to further isolate our workloads, minimizing damage caused by a compromised container.

Tradeoffs of running rootless

Although it’s getting easier over time, running containers as unprivileged is not straightforward.

You need to adapt your images so processes can run unprivileged, which involves changing ports and reviewing file permissions. We’ll cover this process in the next section.

You will also lose low-level access to some resources. Security probes, such as the Sysdig Agent, need this access to fetch detailed context about what’s running on your host. It’s OK to run these workloads as root, as they are a tiny fraction of your containers, and it’s easy to implement extra controls for them.

You can also use User Namespaces and Capabilities to run containers with limited privileges; we’ll cover how below. The tradeoff for these features is that processes are still somewhat privileged, especially if you are overly permissive with the CAP_SYS_ADMIN and CAP_NET_ADMIN capabilities. We covered this topic in our article “How to detect the containers’ escape capabilities with Falco.”

Prepare your image to be rootless

In theory, preparing your image to run unprivileged is simple:

1. Only use ports above 1024.
2. Review file permissions so unprivileged users can access what they need.

In practice, this is not always as easy to implement, so let’s learn with a real-world
example. We’ll use the nginx-unprivileged image, as its code is available and has
ample documentation.

The nginx-unprivileged image achieves rootlessness by making several changes on
the /etc/nginx/conf.d/default.conf config file:

# implement changes required to run NGINX as an unprivileged user
RUN sed -i 's,listen       80;,listen       8080;,' /etc/nginx/conf.d/default.conf \
    && sed -i '/user  nginx;/d' /etc/nginx/nginx.conf \
    && sed -i 's,\(/var\)\{0\,1\}/run/nginx.pid,/tmp/nginx.pid,' /etc/nginx/nginx.conf \
    && sed -i "/^http {/a \    proxy_temp_path /tmp/proxy_temp;\n    client_body_temp_path /tmp/client_temp;\n    fastcgi_temp_path /tmp/fastcgi_temp;\n    uwsgi_temp_path /tmp/uwsgi_temp;\n    scgi_temp_path /tmp/scgi_temp;\n" /etc/nginx/nginx.conf \
    && sed -i 's,PIDFILE=${PIDFILE:-/run/nginx.pid},PIDFILE=${PIDFILE:-/tmp/nginx.pid},' /etc/init.d/nginx \

Let’s review them.

First, the 8080 port is set as the default since any user can listen on ports above 1024, but only a privileged user can listen on port 80.

sed -i 's,listen       80;,listen       8080;,' /etc/nginx/conf.d/default.conf

Then, you can properly map your ports in your container runtime:

docker run -p 8080:80 nginx-unprivileged

Next, ensure there are no user changes by removing the user directive:

&& sed -i '/user  nginx;/d' /etc/nginx/nginx.conf \

It continues by changing several folders to “tmp”, accessible to unprivileged users:

&& sed -i 's,\(/var\)\{0\,1\}/run/nginx.pid,/tmp/nginx.pid,' /etc/nginx/nginx.conf \
    && sed -i "/^http {/a \    proxy_temp_path /tmp/proxy_temp;\n    client_body_temp_path /tmp/client_temp;\n    fastcgi_temp_path /tmp/fastcgi_temp;\n    uwsgi_temp_path /tmp/uwsgi_temp;\n    scgi_temp_path /tmp/scgi_temp;\n" /etc/nginx/nginx.conf \
    && sed -i 's,PIDFILE=${PIDFILE:-/run/nginx.pid},PIDFILE=${PIDFILE:-/tmp/nginx.pid},' /etc/init.d/nginx \

The changed files are the PID files and temporary files.

Finally, it makes the configuration and cache folders writable by the user:

# nginx user must own the cache and etc directory to write cache and tweak the nginx config
    && chown -R $UID:0 /var/cache/nginx \
    && chmod -R g+w /var/cache/nginx \
    && chown -R $UID:0 /etc/nginx \
    && chmod -R g+w /etc/nginx

You may have already guessed that modifying an image to run unprivileged is easy, but only as long as the services inside the container are prepared for that effect. The NGINX server is highly parameterized and can change its behaviour from configuration files without modifying the source code.

Luckily, most software follows these principles. The trick is knowing the list of files that need to be moved to the /tmp directory.

Extra security tips related to root privileges

Preparing your image to run unprivileged is only the first step. You can further strengthen the security of your container with a few steps.

First, ensure your binaries are root-owned so an attacker can’t modify them if the container gets compromised. If your Dockerfile looks like this:

...
WORKDIR $APP_HOME
COPY --chown=app:app app-files/ /app
USER app
ENTRYPOINT /app/my-app-entrypoint.sh
Code language: JavaScript (javascript)

Drop the --chown=app:app flag for your binaries, or avoid RUN chown commands. Your user doesn’t need to own the binaries; they only need execute permissions.

Another tip: If possible, run your containers in read-only mode with an allowlist for the files that need read-write. Use the --readonly flag to mount the container’s volume as read-only, then use -v to list the files that can be modified:

$ docker run --read-only -v /tmp

For nginx-unprivileged, you would do something like this:

$ docker run -d -p 8080:80 --read-only -v $(pwd)/nginx-cache:/var/cache/nginx -v $(pwd)/nginx-pid:/var/run nginx-unprivileged

Lastly, if you need to run commands like apt during the build phase, you can use multi-stage builds to build as root, then run the main process as a regular user:

# This is the builder stage
FROM gcr.io/distroless/static-debian10 as builder
[…]
RUN apt-get [your apt and build commands here]
[…]
# Final stage, copy required artifacts from builder
FROM gcr.io/distroless/static-debian10 as builder
[…]
COPY --from=builder /file/to/copy
USER myuser
[…]

Check out our Top 20 Dockerfile best practices article for more tips on securing containers.

Capabilities and user namespaces

As mentioned earlier, some containers need to perform privileged operations, such as accessing the network stack directly or using ptrace.

Capabilities

The most secure way to allow this is to use Linux capabilities. You can use capabilities to grant specific privileges to a regular user.

You can configure this on Docker with the --cap-add flag:

$ docker run --cap-add=SYS_PTRACE […]

You can define capabilities for your Kubernetes Pods inside securityContext:

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo-4
spec:
  containers:
  - name: sec-ctx-4
    image: gcr.io/google-samples/hello-app:2.0
    securityContext:
      capabilities:
        add: ["NET_ADMIN", "SYS_TIME"]

User namespaces

However, sometimes you don’t have the time to adapt a container image to run unprivileged or to use capabilities.

In these cases, you can use user namespaces to abstract the user running inside a container from the user executing the workload on the host. That way, you can run a process as root inside the container while it runs as a regular user on the host.

With user namespaces, if an attacker escapes the container, they won’t have special privileges on the host, limiting the damage they can cause.

To enable user namespaces in Docker, you need to configure the mappings in the /etc/subuid and /etc/subgid files. These steps are similar for other container runtimes, you will find links to their documentations right after this example.

For a given user with the following ids:

$ id testuser
uid=1001(testuser) gid=1001(testuser) groups=1001(testuser)

We could create a user namespace by adding an entry like this one to both /etc/subuid and /etc/subgid:

testuser:231072:65536

This namespace reserves 65536 user ids, starting with 231072. Within this range, all user ids will be mapped to testuser, with 231072 mapped to UID 0 inside the namespace.

Then you would start dockerd with the --userns-remap flag to specify the default host user for containers to run as.

$ dockerd --userns-remap="testuser:testuser"

Most container runtimes support user namespaces in Linux, including Podman, CRI-O, runc, and containerd.

Kubernetes supports user namespaces for Pods. You can enable this feature by setting hostUsers: false in a Pod’s deployment yaml:

apiVersion: v1
kind: Pod
metadata:
  name: userns
spec:
  hostUsers: false
[…]

Support for user namespaces in Kubernetes continues to improve. For example, an alpha enhancement in Kubernetes 1.35 decouples user namespaces from access to the host network stack.

Conclusion

By using rootless containers, we can create a line of defense in case one of our containers is compromised.

Running unprivileged has become easier over time, and we can use tools like capabilities or user namespaces when rootless is not an option.

There’s never been a better time to try this out.