Tech of VMworld 2017: Monitoring

VMworld 2017 was a reinvigorated ecosystem, as I wrote about previously, and there was plenty of interesting tech about datacenter infrastructure was what you wanted to know about.

One of the main categories I spent time to look at was monitoring and telemetry systems. These are software tools used for managing infrastructure in the VMware ecosystem, which is rapidly extending into cloud options.

The most impressive find was the newly rebranded Densify. Previously called Cirba, the company has performed one of the most successful and comprehensive re-positioning efforts I've seen in a long time. It chose to focus on its densification

CTO and co-Founder Andrew Hillier showed me how the tool can do a cost comparison between multiple hosting options, including the just-announced VMware-on-AWS offering, so help customers figure out how to optimise their spend on infrastructure.

Anyone who has dealt with an AWS bill, or attempted to plan out a non-trivial environment in any cloud system and compare it to on-site options, knows how daunting it can be. Densify does this pretty much automatically (on-site pricing is a little trickier, on which more in a moment) and it presents the information is a dead-simple-to-understand dollar cost per month. I understood what it was showing me immediately, and could see the value just as quickly.

There are other costing tools that do similar things, but not in the way Densify does it. Densify uses the real, running workload you have in VMware and can tell you what it would cost in a given cloud. It also helps you optimize your current AWS or other cloud configuration, and helps you fix mis-allocations, even if you've inadvertently purchased Reserved Instances that you really shouldn't have. Densify makes it really simple to see what's mis-allocated, and what to do to fix it. It's just simple and practical and useful.

Densify was the Best of VMworld Gold winner for Workload Management and Migration, and justifiably so.

I don't mean to oversell this, so always check things for yourself, but it's just so rare for me to see a company absolutely nail their positioning and messaging around a product that has clear value.

I also spent time with

Turbonomic CEO Ben Nye, who explained to me why the company's focus on performance is so important.

I've been following Turbonomic for a while—since it was called VMTurbo—and the company has always had an obsession with the free market analogy behind its software model of infrastructure. It's designed around software actors buying and selling in a simulated market where you define budgets for different components based on your organizational priorities.

Ignore all that.

The important thing about Turbonomic is that focusing on ensuring application performance (using this internal model that you don't need to care about) means you also fix system issues that cause outages. Turbonomic isn't about efficient allocation of resources, it's about making sure applications perform well as they provide end-user functions.

Turbonomic helps to manage the end-to-end experience, and proactively avoids problems by dynamically re-allocating resources to where they are needed in a system that becomes impossible for a human to manage at scale. Turbonomic helps remove bottlenecks that would otherwise cause problems and helps you to find the real bottlenecks that you can then focus on managing. Read Goldratt's The Goal for a more detailed understanding of the operations management going on here, but know that it's good science and engineering used in manufacturing for decades.

Turbonomic makes these decisions, thousands of them, which are often counter-intuitive. “That's why most humans shouldn't be allowed to make these decisions,” says Nye. We just aren't any good at it.

I also spoke to Blue Medora who make a series of plugins for VMware's vRealize Operations (VROPS) and New Relic's application monitoring platform.

Blue Medora is all about filling in the gaps that the native products don't do, adding plugins for things like AWS' RDS and ECS with management packs. The general idea is that by filling in those gaps you can get more use out of the tools you already have, and do more with them.

We talked about the challenge of visibility as the world moves to distributed micro-services architectures where monitoring the CPU and RAM utilization on a couple of servers tells you pretty much nothing about what's actually going on. While the nirvana of a single, unified dashboard that suits everyone—operator and developer alike—is probably an illusion, providing information to people so they can understand what's going on in an increasingly complex system is very important.

The challenge of configuring such a system to be powerful and comprehensive, but also simple to use, is a daunting one. However, without getting the information in the first place, an organization is essentially flying blind, and reliant on low-level diagnostic tools that are time-consuming and complex to use. They should be reserved for deep-dives by specialists when things have gone terribly awry. The rest of the time, we should be able to get high-level information about the health of the system using something like Blue Medora, while something like Turbonomic manages the details of standard operations, and Densify helps us keep costs optimized and alerts us to possible alternatives that are better or cheaper.

A good way to manage the complexity of a modern IT environment—a hairy mix of on-site, cloud, and mobile weirdness—is to have a suite of tools, all working in concert.

That's the story I get from the VMworld ecosystem, and it sounds pretty good to me.

This article first appeared in Forbes.com here.