Speed up Building with Docker BuildX and Graviton2 EC2 – Docker Blog

As the expansion in arm usage continues, building your images on arm is crucial to making images available and performant across all architectures which is why we’ve invested in making it super easy to build arm and multi-arch images. In a previous blog we outlined how to build multi-arch images locally using the QEMU emulator that comes pre-packaged with Docker Desktop. In this blog we outline how to get started using a remote builder to accomplish the same goal, for our purposes we will be using an Amazon EC2 instance. In December of 2019, Amazon announced the new Amazon EC2 instances powered by AWS Graviton2 Processors that significantly improve performance over the first-generation AWS Graviton processors. Using a Graviton2 instance to build your arm images remotely will speed up the process, making it even easier to develop containers on, and for, arm servers and devices. 

We will walk through the following:

1. Using Graviton2 EC2 instance as a remote host
2. Registering the Graviton2 EC2 instance as remote builder
3. Building  the docker image on Graviton2 EC2 instance
4. Adding additional instances as required

To learn more about buildX and remote builders, you can visit our documentation.

Getting Started With a Remote Builder

Prerequisites

First we’ll have to ensure that you are using a remote host instead of your local machine. Common ways to access remote Docker instances are via mTLS or SSH. Here we will use SSH for simplicity. In order to start with using a remote builder with Buildx and BuildKit on Graviton2 the host needs to be accessible through the Docker CLI, using ssh://<USERNAME>@<HOST> URL as follows:

$ docker -H ssh://me@graviton2-instance info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-65-gad9dddc3)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.3)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 20.10.9
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-1045-aws
 Operating System: Ubuntu 20.04.2 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 1.837GiB

Create the remote builder with Buildx

Now you can register this remote Graviton2 instance to Docker Buildx using the create command:

$ docker buildx create --name graviton2 
  --driver docker-container 
  --platform linux/arm64 
  ssh://me@graviton2-instance
graviton2

--platform is specified so that this node will be preferred for arm64 builds when we add other nodes to this build cluster.

Bootstrap the builder to check and create the BuildKit container on the remote host:

$ docker buildx inspect --bootstrap --builder graviton2
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 2.2s done
#1 creating container buildx_buildkit_node1
#1 creating container buildx_buildkit_node1 1.4s done
#1 DONE 3.7s
Name:   graviton2
Driver: docker-container

Nodes:
Name:      node1
Endpoint:  ssh://me@graviton2-instance
Status:    running
Platforms: linux/arm64*, linux/arm/v7, linux/arm/v6

Build your image

Now we will create a simple Dockerfile:

FROM busybox as build
ARG TARGETPLATFORM
ARG BUILDPLATFORM
RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log
FROM busybox
COPY --from=build /log /log
RUN cat /log
RUN uname -a

And let’s build the image against our builder:

docker buildx build --builder graviton2 
  --push -t example.com/hello:latest 
  --platform linux/arm64 .
#2 [internal] load .dockerignore
#2 transferring context:
#2 transferring context: 2B 0.2s done
#2 DONE 0.2s

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 246B 0.3s done
#1 DONE 0.4s

#3 [internal] load metadata for docker.io/library/busybox:latest
#3 ...

#4 [auth] library/busybox:pull token for registry-1.docker.io
#4 DONE 0.0s

#3 [internal] load metadata for docker.io/library/busybox:latest
#3 DONE 1.5s

#5 [build 1/2] FROM docker.io/library/busybox@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57
#5 resolve docker.io/library/busybox@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57 0.0s done
#5 DONE 0.0s

#5 [build 1/2] FROM docker.io/library/busybox@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57
#5 sha256:7560ee4921c3fab4f1d34c83600f6f65841ec863e072374f4e8044ff01df156f 821.72kB / 821.72kB 0.1s done
#5 extracting sha256:7560ee4921c3fab4f1d34c83600f6f65841ec863e072374f4e8044ff01df156f 0.0s done
#5 DONE 0.2s

#6 [build 2/2] RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log
#6 DONE 0.1s

#7 [stage-1 2/4] COPY --from=build /log /log
#7 DONE 0.0s

#8 [stage-1 3/4] RUN cat /log
#8 0.078 I am running on $BUILDPLATFORM, building for $TARGETPLATFORM
#8 DONE 0.1s

#9 [stage-1 4/4] RUN uname -a
#9 0.081 Linux buildkitsandbox 5.4.0-1045-aws #47-Ubuntu SMP Tue Apr 13 07:04:23 UTC 2021 aarch64 GNU/Linux
#9 DONE 0.1s

#10 exporting to image
#10 exporting layers
#10 exporting layers 0.3s done
#10 exporting manifest sha256:7097ec1c09675617e2c44b5924b76f7863c4ff685c640b32dfaa1b1e8f2bc641 0.0s done
#10 exporting config sha256:d18ca92a45b563373606f0a06d0a1d2280d0c11976b1ca64dba0e567d540a3e2 0.0s done
#10 pushing layers
#10 pushing layers 0.7s done

Add other instances

You can also append other Graviton2 instances (node) to the same builder with:

docker buildx create --name graviton2 
  --node node2 
  --driver docker-container 
  --platform linux/arm64 
  --append ssh://me@graviton2-instance2

Note the --append flag to add a node to the existing graviton2 builder.

Now, you’ve built your image remotely using a Graviton2 instance! These are just some of the things you can do with buildx. We’d love your feedback on how it went and what you’d like to see us do next, you can submit it feedback to our public roadmap.