AI/ML is the Killer App for WebAssembly

I think data analysis pipelines, such as those used for AI/ML, could be the killer app for WebAssembly (WASM) that ushers in the Age of WASM.

I think this because of several different moving parts around the IT industry that I believe could all combine to great a new and useful thing. A kind of WASM Voltron.

In fact, there are quite a few parts, not a mere five human piloted robotic lions, so it's more of a coordinated swarm. A SwASM. SwarASM? Naming is hard.

Anyhow, here are the bits that I think could create our WASM Voltron made of bees:

• The WASM component model.
• Dynamic linking of WASM components versus the static linking metaphor of containers.
• The focus on stateless/low-state compute in data analysis pipelines.
• The significantly smaller size of WASM components and binaries compared to containers.
• Faster start and shutdown of WASM compared to containers, VMs, and bare-metal.
• The promise, and lessons from the (qualified) failure of unikernels.
• 20+ years of lessons from Java and JIT compilation.
• Historical lessons from CORBA and COM.
• The continued attraction of write once, run anywhere, and that WASM might finally allow it.
• Automated introspection of WASM bytecode.
• An urgent need for robust software supply chain security.

All these features combine into something that is better enough than what came before to be pretty compelling for a range of workloads, eventually. And they are sufficiently compelling to gain near-immediate adoption for AI/ML, provided some still-needed pieces arrive.

Why use a WASM Voltron?

AI/ML uses a lot of resources. VMs were more efficient than physical servers, and containers were more efficient yet again. WASM continues this trend.

VMs were great because you didn't have a bunch of stranded resources sitting on servers you couldn't access. You didn't have wasted overhead of copies of all the network cables, rack space, power and cooling, etc. of physical servers. You could share compute, memory, and storage with different programs, and still have logical isolation to help with operational management (and privacy, security, etc.). Not perfect, but better than what came before. Better enough.

Containers were great because you didn't have a bunch of stranded resources allocated to VMs you couldn't access. You didn't have wasted overhead of full copies of operating systems, and you still had logical isolation to help with operational management. Again, not perfect, but better than what came before. Better enough.

With WASM binaries, we repeat the pattern. Less waste in unnecessarily duplicated overhead. The ability to share common components and resources.

With each step, bin-packing is easier because the size of the components are smaller. But we keep the operational benefits of managing discrete, logical chunks rather than trying to operate a sea of continuous, non-laminar ones and zeros.

Dynamic versus static linking

Dynamic linking has a lot of benefits compared to static linking. It's why we moved away from self-contained static binaries to dynamically-linked libraries. A loosely-coupled modular design means you can update the cryptography library to fix a bug without having to recompile every binary on the entire system.

And yet containers took us back to statically compiled chunks in many ways. We couldn't actually rely on different versions of libraries, static or dynamic, being compatible with our software. Upgrading the cryptography library might have fixed a bug in the library, but our apps might no longer work. And we'd find out after the upgrade to production. Operationally, it was a pain, especially at scale.

Containers helped, because we kept the dynamic linking of shared components in the host operating system, and the app specific parts were kept in the container. Development and operations became loosely-coupled modular components with all the benefits that brings.

WASM keeps the loose-coupling of development and operations with a similar abstraction to the host environment. This was the promise of Java, but it stumbled because we couldn't rely on the host environments being compatible enough. We couldn't write once and run anywhere.

With WASM, that changes because the host environments are reliably compatible and enough people see the value in keeping that compatibility. Advances in JIT techniques in the past 10-15 years have also opened up new possibilities that weren't available in the early days of Java. The structured nature of WASM bytecode makes it more amenable to automated scanning than source languages, and modern CI/CD approaches make it easier to actually do. WASM builds on the shoulders of giants.

Similarly, advances in cryptography have both made it easier to use and better understood. There is substantial demand now for better security due to the constant data breaches everyone is having. Cryptographically secured software supply chains have a lot of appeal, and the WASM component model is including ways to ensure that components can be decoupled but the end-to-end system remains trusted and secure. If this can be proven robust and effective, the enhanced security will be enough on its own to attract many workloads to WASM.

The WASM folks are also alert to the allure, and the burnt fingers, of CORBA and COM and the similar component approaches to software. They are keeping the good bits and avoiding the frustrating complexity that made earlier approaches too hard for mere humans to use. This is very encouraging, though not without risks. The whole thing could come unstuck if the WASM component model becomes too complex, or lacks features, or otherwise fails to be sufficiently elegant.

WASM for AI/ML

The greater efficiency of WASM binaries, both for bin-packing efficiency reasons but also startup/shutdown speed, provides an advantage over containers. This gets magnified by scale, and AI/ML workloads operate at very large scale. Even a 10% saving in time or resources is big money here, which is why I think it will be the first place WASM adoption takes off, assuming it does.

The scale of AI/ML also magnifies the operational challenge. If 10,000 WASM bundles fit where only 200-ish containers do on a Kubernetes node today, that's one or two orders of magnitude more complexity to manage. Just as containers added thousands of components to manage on top of hundreds of VMs running on dozens of servers.

Today the operations challenge is addressed with Kubernetes. It's used a lot by AI/ML teams and others with similar data processing workloads because it provides many benefits. But it's still very complex to configure and operate, and it suffers from what I feel could be a fatal flaw in the Age of WASM: Kubernetes is tightly coupled to containers as its operational metaphor.

This creates negative pressure pushing people away from Kubernetes. If the WASM ecosystem can provide sufficiently less-complex workload scheduling and operations platform, I can see a lot of AI/Ml workloads jumping ship. WASM doesn't need to be perfect, it just needs to be better than what people have now.

Killing Kubernetes

Kubernetes forces developers to deal with the infrastructure of containers because they are not sufficiently abstracted away. If WASM can solve the “why do I have to think about infrastructure?” question sufficiently better than Kubernetes, it will have an edge.

If WASM can also provide, say, 20% overhead savings compared to Kubernetes for just the “running workloads on this” part of deployment, that will make it compelling enough to try for large AI/ML workloads. And any similar compute-heavy, low-statefulness workloads, of which there are many (HPC, media streaming, CGI rendering, high-frequency trading, etc.)

And if the complexity of metaphor is sufficiently well-hidden by making the platform layer simpler than Kubernetes, adoption will be easier. That translates into even greater savings from getting things done faster and easier. Less wrestling with the tools and more just using them. With expensive developer salaries, the savings add up pretty quickly on that front.

All these features are why I think WASM could be very significant indeed, and there is every indication that the people building it out are well aware of what needs to be done. They have learned the lessons from the past and are trying to avoid making the same mistakes. They're also abandoning ideas that are no longer necessary as advances in the state-of-the-art allow us to do things that we couldn't back in the mid-1990s or 2000s. We don't have to do things the way we've always done them, but we don't have to do new things just because they're new, either.

It's a very pragmatic approach. I'm sensing less ideology about there being a One True Way and more of a well-informed pragmatism that is trying to strike a balance between elegance and usefulness. That is the mark of what great engineering is about.

I also have a nagging feeling, an intuition if you will, that WASM will be important. It's a kind of mental shape, or perhaps flavour or smell, that I can't properly describe yet. I've caught glimpses of it that hint at something larger. With age, I've learned to pay attention to certain intuitions or feelings, and this is one of them.

I may well be wrong, but I'll enjoy finding out.