Containers Are The Future But The Future Isn’t Finished

Containers are a big deal, and they're only going to get bigger. That's my view after attending the latest KubeCon (and CloudNativeCon) in Seattle last week.

A year ago, I was confused about what containers mean for IT, because the name ‘container' had me thinking it was about the little box that code was stored in: the container image. I'm here to tell you that the container image format itself (Docker, rkt, whatever you like) is not the point.

The most important thing about containers is the process of using them, not the things themselves. The process is heavily automated. No more installing software by sitting in front of a console and clicking ‘Next' every five minutes. Unix people everywhere rejoice that Windows folk have discovered scripting is a good thing.

The process is also heavily integrated. Deployment of software is coupled to the software development process itself. This is why cloud native is being driven by application developers: they write the applications to deploy the software into software abstractions of infrastructure that they also wrote. When they check in code to Github, other code (that they wrote) notices and starts the process of testing and building and packaging and deployment. If you want to, the entire process can be completely automated so that new code is pushed to production as a direct result of checking in the new code.

The developers are at the centre of this heavily automated IT factory, much of which they designed themselves.

There are still two very large gaps.

The first is that the cloud native/DevOps folk, at least those at this conference, assume a layer of robust infrastructure exists. Containers need compute and RAM and networks and storage. This is fine if containers are being used inside a pre-existing cloud like AWS, Google Cloud Engine (GCE), or Azure, but if you're thinking about using containers on some other infrastructure, there is a lot of work to be done before you start using containers themselves.

I refer to this as the “Create the universe” problem, after Carl Sagan's famous statement that “If you wish to make an apple pie from scratch, you must first invent the universe.”

And, because this is a gap, there are various solutions for it, like Supergiant.io or OpenShift or Rancher. They all deal with slightly different aspects of getting the infrastructure environment stood up in the first place. It's a very similar, but not the same, problem that OpenStack attempts to solve. It was very noticeable that a lot of the KubeCon attendees I spoke to were OpenStack refugees. Make of that what you will.

The second gap is related: cloud native/DevOps folk are only just starting to grapple with the issue of stateful data management. The assumption for container-based workloads is that they are stateless. If a container dies, it is replaced with another, identical container to take its place. If an application is upgraded, existing containers are killed off and replaced wholesale with new ones, often gradually through a process called canarying.

This method of maintaining software is extremely powerful, particularly the automated testing involved in canarying. Canarying is where a small subset of containers are replaced first to act as a canary in the coalmine and if they work okay, more containers are rolled out and tested. This process assumes, in fact requires that the containers be essentially stateless.

While that's a useful simplifying assumption, it doesn't help for all the real world applications that have actual value. I care much less for the way the internet banking app works than I do about how much money is in my account. Similarly, the list of customer contacts in your CRM are where a lot of value is. Most of value of email is in the actual emails, not the email app you happen to use.

And this is where containers really struggle. Kubernetes and Docker both have the concept of Persistent Volumes, but they've been added later. The fact that statefulness wasn't added from the beginning is really obvious to me, an infrastructure guy who spent a lot of time in storage land.

As every problem is also an opportunity, there are plenty of companies either starting up from scratch (StorageOS) or adding container interfaces to their existing offerings. I spent a bunch of time with NetApp people at the show learning about how they can expose NFS based FlexVols to Kubernetes and have them created, attached, and even destroyed in line with the Kubernetes storage lifecycle. It's a glorious hack.

It's gold-rush time in container land, and while we've seen this before (many times) I'm going to go out on a limb and say that containers are the future and you'd best get on board.

We need your help to make it good.

I travelled to KubeCon as a guest of the Linux Foundation. Read my full disclosure statement on my blog here.

This article first appeared in Forbes.com here.