Docker Move Brings Universal Container Operations A Step Closer

Getting to a standard, shared runtime environment in which containers from different suppliers run predictably took a step closer to reality this week as Docker opened up a key feature of its Docker Engine, containerd.

Containerd (pronounced ContainerDee) represents a key slice of code, previously distributed in different parts of the Docker Engine. The code has been pried loose and combined into a stand-alone component, available for use by other companies, including Docker competitors.

Docker had pledged to place it in the public sphere and lived up to that pledge on Dec. 14. Containerd is what's known in coding as a daemon or active agent that can activate a variety of processes on behalf of a running container. The daemon has processes for transferring an assembled container or container image, starting its execution, and supervising its operation.

It’s a key to the container runtime environment. The Docker Engine assembles an application and its dependencies as a set of files, according to a prescribed format. Once assembled, the container is initiated by its host operating system, but still must have a number of runtime requirements met to do its work. The runtime is something like the turntable and stylus of a record player. The container can't display its wares without them. As the activating agent, containerd might be considered equivalent to the needle on the stylus.

What IT managers and container users in general want to see is a universal runtime environment where containers from different suppliers can do their work in a predictable fashion. That goal eluded an earlier phase of the virtualization market, when suppliers such as VMware, Citrix and Microsoft maintained competing formats and runtimes, which continue as separate worlds today.

Want to learn more about a key Docker competitor? See Mesosphere Links To Key Supporting Systems For Container Use.

Docker appeared heading down the same path as it built up a head of steam in the marketplace and launched multiple initiatives through its open source project. Then two years ago cracks in the facade of its Docker community began to appear as member CoreOS and others objected that Docker might be pushing container development in directions they didn't necessarily want to go.

The result in 2015 was the Open Container Initiative in which Docker, CoreOS and other members of the ecosystem agreed they would all implement container technologies that met specifications set by OCI.

Containerd itself runs atop runC, an OCI specified part of the container runtme. Containerd leverages runC to manage containers in a specific system. Solomon Hykes, CTO of Docker, said public cloud providers will be among the first to make use of containerd to set container runtime environments inside their public cloud services. "Large public cloud vendors are a great audience for this," he said in an interview.

Docker's announcement of its move enlisted supporting statements from Amazon Web Services, Google, IBM and Microsoft. In addition, the giant Chinese cloud provider Alibaba has "committed to providing maintainers and contributors to the project," Docker said in its announcement, alongside the other cloud suppliers.

Hykes said the release of containerd is reassuring to enterprises that want to expand their use of containers. They can get started with Docker but know they are unlikely to have only Docker wares to choose from in the future when it comes to orchestration and management. 

Containerd's release "will unlock a whole new phase of innovation and growth across the entire ecosystem," Hykes predicted. Docker is already bidding to expand its software into container cluster creation, container distribution across a cluster and container management through Docker Swarm. But it faces competition from the Kubernetes project and Mesosphere's DC/OS, which offer their own orchestration and deployment systems.

The release of containerd eases but doesn't remove tensions among those jockeying for position in container development.

It was CoreOS CEO Alex Polvi's pointed criticism of Docker and launch of the competing Rocket container runtime two years ago that started a more organized dissent from some Docker initiatives. In response to Docker's containerd move, CoreOS CTO Brandon Philip took some credit in an emailed statement Dec. 15. CoreOS "started the conversation on compatible, portable and secure container runtimes with rkt (Rocket)…. We are working closely with the community to get OCI image and runtime specifications ready for 1.0, which is paramount for the widespread adoption of containers."

The 1.0 version of the specification is expected in early 2017, and is needed to guide competing implementations of container runtimes. But the availability of containerd as a free standing components makes it more likely the vendor-neutral, universal runtime will actually be achieved.

Philip's statement also said: "Giving users choice and flexibility with runtimes will help make it easy to take the next step to orchestrate containers with Kubernetes." CoreOS produces an orchestration product, Tectonic, based on Kubernetes. Kubernetes is a container cluster-building project that was initiated by Google, based on its experience in launching two billion containers a week in its search and internal operations.

The statement nowhere mentioned Docker or containerd as the object of the "discussion on the container execution architecture" that Philip described as "great to see," a sign that old tensions persist within the container development community and could still split apart the fragile unity.

Docker has moved from a position of strength through its early adoption by developers into producing a full platform covering the lifecycle of how containers are built, deployed and managed. Those ambitions haven't always sat well with Docker's own allies, which early on included Red Hat and Google. The pair set off on a distinct path away from the platform and started a second project to ensure that their own technologies play a role in future container deployments.

Google and Red Hat teamed up a year ago to start the Open Container Initiative daemon project, or OCI-d, which would allow containers to be called out of registries, Docker's or otherwise, and run in a Kubernetes runtime environment. The runtime environment to a container is something like the round table and stylus to a record. They're essential if the record, whether a 33.3 or a 45, is going to display what it can do. The OCI-d project in September changed its name to Container Runtime Interface – Open or CRI-O, a change appears to reflect the desire of the Open Container Initiative, a broad, cross-industry group, to remain un-entangled in any project with a specific vendor interests vested in it.

Red Hat made an early commitment to Docker containers at the same time it was investing heavily in its own open source development platform, OpenShift. Many future container deployments in production will occur under IT's favorite Linux, Red Hat Enterprise Linux, and Red Hat wants its own pieces of software in the infrastructure stack.

Google likewise adopted the Docker container formatting engine as a competent technology without any intention of giving up its own deployment technology. Google's own container expertise gives it cache among many IT managers looking to manage their own infrastructure via containers. Kubernetes is one of the few open source projects that's gained legs capable of running with Docker Swarm, Docker's deployment software. In June Docker announced that it had embedded its deployment software into the popular Docker Engine.

Red Hat did not comment in Docker's announcement of the release of containerd. Google's VP of Product Management Sam Ramji, former CEO of the Cloud Foundry Foundation, was quoted in the announcement: "As a company that has supported running workloads in container for more than 10 years, we are excited to see Docker offering a stable, reliable and principally layered container runtime that is designed to be consumed by higher layer systems. This aligns well with Google Cloud Platform's philosophy to build an open cloud for all businesses."

Despite tensions, Docker is trying to broaden its appeal and respond to community members who continue to pressure it to keep a level playing field. It may be enlightened self-interest or community pressure. But regardless, Docker's opening up of containerd is a big step toward a universal runtime. That environment is now in view, along with a broader, richer selection of container management products.