Creating Kubernetes Extensions in Docker Desktop – Docker

This guest post is courtesy of one of our Docker Captains! James Spurin, a DevOps Consultant and Course/Content Creator at DiveInto, recalls his experience creating the Kubernetes Extension for Docker Desktop. Of course, every journey had its challenges. But being able to leverage the powerful open source benefits of the loft.sh vcluster Extension was well worth the effort!

Ever wondered what it would take to create your own Kubernetes Extensions in Docker Desktop? In this blog, we’ll walk through the steps and lessons I learned while creating the k9s Docker Extension and how it leverages the incredible open source efforts of loft.sh vcluster Extension as crucial infrastructure components.

Why build a Kubernetes Docker Extension?

When I initially encountered Docker Extensions, I wondered:

“Can we use Docker Extensions to communicate with the inbuilt Docker-managed Kubernetes server provided in Docker Desktop?”

Docker Extensions open many opportunities with the convenient full-stack interface within the Extensions pane.

Traditionally when using Docker, we’d run a container through the UI or CLI. We’d then expose the container’s service port (for example, 8080) to our host system. Next, we’d access the user interface via our web browser with a URL such as http://localhost:8080.

While the UI/CLI makes this relatively simple, this would still involve multiple steps between different components, namely Docker Desktop and a web browser. We may also need to repeat these steps each time we restart the service or close our browser.

Docker Extensions solve this problem by helping us visualize our backend services through the Docker Dashboard.

Combining Docker Desktop, Docker Extensions, and Kubernetes opens up even more opportunities. This toolset lets us productively leverage Docker Desktop from the beginning stages of development to container creation, execution, and testing, leading up to container orchestration with Kubernetes.

Challenges creating the k9s Extension

Wanting to see this in action, I experimented with different ways to leverage Docker Desktop with the inbuilt Kubernetes server. Eventually, I was able to bridge the gap and provide Kubernetes access to a Docker Extension.

At the time, this required a privileged container — a security risk. As a result, this approach was less than ideal and wasn’t something I was comfortable sharing…

Let’s dive deeper into this.

Docker Desktop uses a hidden virtual machine to run Docker. We also have the Docker-managed Kubernetes instance within this instance, deployed via kubeadm:

Docker Desktop conveniently provides the user with a local preconfigured kubeconfig file and kubectl command within the user’s home area. This makes accessing Kubernetes less of a hassle. It works and is a fantastic way to fast-tracking access for those looking to leverage Kubernetes from the convenience of Docker.

However, this simplicity poses some challenges from an extension’s viewpoint. Specifically, we’d need to find a way to provide our Docker Extension with an appropriate kubeconfig file for accessing the in-built Kubernetes service.

Finding a solution with loft.sh and vcluster

Fortunately, the team at loft.sh and vcluster were able to address this challenge! Their efforts provide a solid foundation for those looking to create their Kubernetes-based Extensions in Docker Desktop.

When launching the vcluster Docker Extension, you’ll see that it uses a control loop that verifies Docker Desktop is running Kubernetes.

From an open source viewpoint, this has tremendous reusability for those creating their own Docker Extensions with Kubernetes. The progress indicator shows vcluster checking for a running Kubernetes service, as we can see in the following:

If the service is running, the UI loads accordingly:

If not, an error is displayed as follows:

While internally verifying that the Kubernetes server is running, loft.sh’s vcluster Extension cleverly captures the Docker Desktop Kubernetes kubeconfig. The vcluster Extension does this using a javascript hostcli call out with kubectl binaries included in the extension (to provide compatibility across Windows, Mac, and Linux).

Then, it posts the captured output to a service running within the extension. The service in turn writes a local kubeconfig file for use by the vcluster Extension. 🚀

// Gets docker-desktop kubeconfig file from local and save it in container's /root/.kube/config file-system.
// We have to use the vm.service to call the post api to store the kubeconfig retrieved. Without post api in vm.service
// all the combinations of commands fail
export const updateDockerDesktopK8sKubeConfig = async (ddClient: v1.DockerDesktopClient) => {
    // kubectl config view --raw
    let kubeConfig = await hostCli(ddClient, "kubectl", ["config", "view", "--raw", "--minify", "--context", DockerDesktop]);
    if (kubeConfig?.stderr) {
        console.log("error", kubeConfig?.stderr);
        return false;
    }

    // call backend to store the kubeconfig retrieved
    try {
        await ddClient.extension.vm?.service?.post("/store-kube-config", {data: kubeConfig?.stdout})
    } catch (err) {
        console.log("error", JSON.stringify(err));
    }

How the k9 Extension for Docker Desktop works

With loft.sh’s ‘Docker Desktop Kubernetes Service is Running’ control loop and the kubeconfig capture logic, we have the key ingredients to create our Kubernetes-based Docker Extensions.

The k9s Extension that I released for Docker Desktop is essentially these components, with a splash of k9s and ttyd (for the web terminal). It’s the loft.sh vcluster codebase, reduced to a minimum set of components with k9s added.

The source code is available at https://github.com/spurin/k9s-dd-extension

While loft.sh’s vcluster stores the kubeconfig file in a particular directory, the k9s Extension expands this further by combining this service with a Docker Volume. When the service receives the post request with the kubeconfig, it’s saved as expected.

The kubeconfig file is now in a shared volume that other containers can access, such as the k9s as shown in the following example:

When the k9s container starts, it reads the environment variable KUBECONFIG (defined in the container image). Then, it exposes a terminal web-based service on port 35781 with k9s running.

If Kubernetes is running as expected in Docker Desktop, we’ll reuse loft.sh’s Kubernetes control loop to render an iframe, to the service on port 35781.

if (isDDK8sEnabled) {
            const myHTML = '<style>:root { --dd-spacing-unit: 0px; }</style><iframe src="http://localhost:35781" frameborder="0" style="overflow:hidden;height:99vh;width:100%" height="100%" width="100%"></iframe>';
            component = <React.Fragment>
            <div dangerouslySetInnerHTML={{ __html: myHTML }} />
            </React.Fragment>
        } else {
            component = <Box>
                <Alert iconMapping={{
                    error: <ErrorIcon fontSize="inherit"/>,
                }} severity="error" color="error">
                    Seems like Kubernetes is not enabled in your Docker Desktop. Please take a look at the <a
                    href="https://docs.docker.com/desktop/kubernetes/">docker
                    documentation</a> on how to enable the Kubernetes server.
                </Alert>
            </Box>
        }

This renders k9s within the Extension pane when accessing the k9s Docker Extension.

Conclusion

With that, I hope that sharing my experiences creating the k9s Docker Extension inspires you. By leveraging the source code for the Kubernetes k9s Docker Extension (standing on the shoulders of loft.sh), we open the gate to countless opportunities.

You’ll be able to fast-track the creation of a Kubernetes Extension in Docker Desktop, through changes to just two files: the docker-compose.yaml (for your own container services) and the UI rendering in the control loop.

Of course, all of this wouldn’t be possible without the minds behind vcluster. I’d like to give special thanks to loft.sh’s Lian Li, who I met at Kubecon and introduced me to loft.sh/vcluster. And I’d also like to thank the development team who are referenced both in the vcluster Extension source code and the forked version of k9s!

Thanks for reading – James Spurin

—

Not sure how to get started or want to learn more about Docker Extensions like this one? Check out the following additional resources:

You can also learn more about James, his top tips for working with Docker, and more in his feature on our Docker Captain Take 5 series.