Archive

Costly Enterprise Neglect

denis-lorain-hBftkCeuKmQ-unsplash

Things to note

Octopus Deploy acquired Codefresh. Codefresh is the main sponsor of the Argo project that's used to manage Kubernetes clusters with a GitOps style workflow. There's been such a proliferation of developer tools that some consolidation is inevitable, plus the shift in market conditions makes it even more likely. I expect to see more of these sorts of buyouts.

Antivirus software company Avast settled a dispute with the US FTC over charges Avast spied on customers. An Avast spokesdroid said the company has stopped doing this now, but how can you ever trust them again?

Reddit will apparently get about $60 million a year from Google so Google can train AI models on what people post to Reddit for free. I'm not sure how much autogenerated /r/pol the world really needs, but apparently we're all about to find out. Stack Overflow is also doing it, again with Google.

Tom's Hardware has an Intel presentation on its chip roadmap that talks up a lot of automation couched as AI. There's also some detail on how moving to being an external foundry for others and itself will help keep its production lines open longer and get better margins. There's still a fair bit of lead time for these changes to happen.

Samsung has a new 36GB DRAM module called the HBM3E 12H that it's pitching at AI workloads. Samsung claims the chips can handle 1.225TB/s (yes, that's terabytes per second) per stack. Which is… a lot. The hyperscalers doing lots of LLMs are clearly the main audience for this stuff, so I doubt we'll see this in PCs any time soon.

India has approved three new chip plants valued at around $15.2 billion. They're supposed to start construction within 100 days and will be built by firms including India's Tata Group, CG Power, and Japan's Renesas.

The Financial Times has a good piece explaining some of the details of modern microchip manufacturing. Not likely to be new to readers of The Crux but I learned things, such as the changes to separate power and signalling pathways.

Snowflake announced that CEO Frank Slootman is retiring. This made The Line sad and the stock went down over 20% despite Q4 revenue going up 32% YoY to $774.7 million.

It looks like online document management company Egnyte is going to go public soon. It seems the jitters about IPOs have abated and tech companies are starting to go public again.

Regulators in the EU are taking a look at Microsoft to see if it's preventing customers from buying alternatives to its own Entra ID service. Entra ID is the new name for Azure Active Directory.

Cheap IoT doorbells cameras are a security nightmare. The products in this story didn't even have the required US FTC's external labelling, but cyber labelling scheme will definitely help. Why not print your own?

Meanwhile JFrog found a bunch of malware backdoors in models on Hugging Face. At least we can deploy malware onto GPUs at scale, eh?

GitHub has enabled push protection by default for all public repos. Hopefully this will reduce the number of people exposing their private API keys after checking them in by accident. I bet we'll still see a lot from people disabling this feature, though.

Dell released its full year results. Total revenue of USD $88.4 billion, down 14% YoY. Net income up 32% YoY from $2.42 billion to $3.195 billion. The Line was happy about it, and apparently believes Dell's claims that people will buy AI-optimised servers, possibly because it'll be hard not to since no one is going to sell anything that doesn't have AI wedged into it somewhere.

HPE also made a slight profit in Q1, USD $387 million, but this is less than the previous two quarters ($642 million and $501 million respectively). Revenues are also down, USD $6.755 billion this quarter ($7.351 billion and $7.81 the previous two quarters). The company is talking up its growth in subscriptions, now $1.4 billion, perhaps forgetting that it no longer runs a printer ink ransom business. That's HP Inc, not HPE.

Facebook won't renew deals with Australian media companies now that the initial deals are expiring. Who could have predicted this would happen? It'll be fun watching MPs discover what the text of the bargaining code law actually says.

Longer reads

A well-written piece about the experience of going to the Austin F1 race from cycling journalist Kate Wagner.

“If you wanted to turn someone into a socialist you could do it in about an hour by taking them for a spin around the paddock of a Formula 1 race. The kind of money I saw will haunt me forever.”

The piece has been pulled from Road and Track magazine, hence the archive link. I wonder why?

And a bonus piece about Salesforce CEO Marc Benioff buying land in Hawaii.

Maintain or Replace: Costly Enterprise Neglect

Australia's Bureau of Meteorology is doing a big project called Robust that will cost some vague number of billions of dollars to do. My friend Mike mused that it might be a big rip-and-replace project after years of neglect, which is something I've seen a few times.

I replied that these big replacement projects are a hard sell because of the large cost, but that can be an advantage. There's a weirdly counter-intuitive approval bias for big-number projects. Smaller, more concrete projects tend to attract more detailed scrutiny because they seem more understandable.

IT projects are complex and involve a lot of fairly abstract components like software. The senior executives or board members charged with reviewing and approving these large projects don't tend to have an IT background. Other projects, such as buying a factory or new machinery or something similarly tangible, are easier to scrutinise. Even if machinery operations isn't within an executive's background, it seems easier to understand because it's more concrete. Anyone can grasp that a machine exists and that its makes stuff, so you can ask questions about the size of it or how many more widgets it'll make and feel like you've made a contribution.

With something more abstract like IT software, it's harder to find useful-seeming questions to ask if you don't have the background. Executives rely more on specialists, like the CTO or CIO who are making the proposal, to provide the deeper scrutiny that might be required. It's a bit of a conflict of interest when the CIO is keen to make a name for themselves with the project, but if you don't trust your CIO already, then you have bigger problems.

Which is how a $300 million IT project can get relatively less scrutiny than buying $300 million worth of warehouse space. It's a form of bike-shedding.

This overzealous scrutiny of relatively trivial things is also why smaller maintenance or upgrade projects tend not to happen. A series of upgrades that would have avoided the need for a big rip-and-replace project didn't get approved because they didn't quite stack up. It was just a bit too expensive, or didn't do quite enough, or just didn't seem important enough compared to all the other things that people wanted to do. The detailed scrutiny of smaller scope found plenty of reasons to say no. And then there's no time for detailed scrutiny of the big Must Happen project that is now Urgent because things are starting to break.

IT managers also contribute to the problem by working around the lack of budget. Instead of not doing the work that was in the project that wasn't approved, they "do their best" with existing resources, finding workarounds and shortcuts to keep things running. They mean well, but they inadvertently train executives to disbelieve the requested budget figures. If you got the work done for 30% less than what you asked for last year, surely you can do it again this year?

Which means the big project now has to contend with an accumulated morass of hacks, workarounds, and bodges that aren't well documented or understood. In a complex enterprise system, no one person remembers how everything fits together. Since a lot of it wasn't written down, the people who left took the knowledge with them, and now you only discover how something works when it breaks and you have to figure out how to fix it.

Which is why it's impossible to collect a full set of requirements for a big rip-and-replace system, no matter how hard you try. Lots of the requirements will only be discovered when you've replaced the old system. Then, a few months later, you get to discover that the old VAX/VMS system in a dusty corner of the datacentre was running a script every quarter that happened to be vital for reconciling the accounts.

If you're lucky, all the really critical functions of the system will be known, well-implemented and tested before golive. If you're really lucky, the project will have a sufficient Operational Weirdness Budget to discover and fix all the exciting corner cases the new system uncovers.

Most people are not this lucky.

All these reasons (and more besides) contribute to making these big projects run over-time and over-budget. I will not be surprised if the BoM's big $1 billion project ends up costing $2.5 billion or more and takes a lot longer than planned.

Disappointed, but not surprised.

Subscriber Only Content

This content is only available to PivotNine subscribers.

Contact sales today to learn more about our subscription services.

Apply Now

Already a subscriber? Log in to view the rest of this content.