Rise of the Hyperplatforms
When I was at Microsoft, the joke was that nobody takes your product seriously unless it’s a platform. I bet the engineers working on MS Paint had visions — thwarted by perfidious PM — of turning it into a platform for all image manipulation needs.
Today nobody talks about platforms. Instead, we’re talking about tech stacks filled with frameworks, components and microservices: LAMP, MERN, MEAN, and other four-letter words. Startups dream — not of creating a new platform — but of creating a new logging system, better than any other logging system, able to corner the logging system market.[1] As developers, we’re spoiled for choice — we can pick and choose from thousands of components and choose the best ones for our app, which is awesome because…
Actually, it’s not that awesome. It’s great in a capitalist-efficiency-labor-specialization way. And it’s great if you’re in the top 1% of software engineers who have the talent and drive to master this complex web of components. But the rest of us either end up with cargo cult architecture or struggle needlessly with complexity unrelated to the actual business problem we’re trying to solve.
For instance, suppose I wanted to create something like Google Sheets. To the user, Google Sheets appears as a single program, but under the covers, it consists of thousands of individual programs operating in a vast distributed system. I’m just a simple programmer. All I want is to write some code, click Run, and have a working program. But that’s impossible with modern web apps — a single program running on a single machine could never scale.
But here’s the thing: anything is possible with just another layer of abstraction.
Imagine if we created a layer of abstraction, not over the hardware, as a traditional operating system does, but over the entire data center. Programs written to this layer would believe they’re running on a machine with thousands of cores and nearly infinite storage. This layer of abstraction would take care of scheduling computations appropriately, storing data reliably, and sharing state across however many users are needed. Ideally it would even take care of rendering UI to a browser or mobile client.
I call this layer of abstraction a “hyperplatform.”
While virtualization platforms and container systems carve up a single machine into smaller virtual computers, a hyperplatform creates a unified, colossal virtual computer from multiple physical nodes. Unlike distributed compute clusters that handle specific tasks, like parallelized data processing or machine learning, a hyperplatform is an integrated, general-purpose platform.
The purpose of a hyperplatform is to handle the common complexities across all applications, leaving only the domain-specific solution. I should be able to create a map function over a large array and use as many cores as needed to process it. I shouldn’t have to care about where my users are storing data — S3 or Google Drive or Dropbox — the hyperplatform should have a driverized storage system to handle it for me. And I should be able to pop up a dialog box and not care whether the user is on a browser or a mobile device.
This sounds both incredibly obvious and like an impossible dream. And yet the evolution towards these hyperplatforms is already visible.
AWS Lambda pioneered serverless computing and remains a remarkable service today. Hyperplatforms will extend serverless computing by integrating storage, state synchronization, and UI. For example, Vercel+Next.js integrate serverless functions with server-side UI. Google Firebase integrates compute, storage, and data synchronization (for collaboration). As these services become more capable and integrated, they get closer to the hyperplatform ideal.
Another example is Bubble.io, a nocode platform. Bubble.io integrates UI, business logic, and storage in a single platform. I can create a program with Bubble.io’s IDE, click Run to test it out, and easily deploy it for the entire world to use. That’s the promise of a hyperplatform. The only limitation right now is that I can’t write arbitrary code — I’m limited to the nocode configurations that they expose.
Which brings me to GridWhale. I started working on GridWhale because I wanted a simpler way to build modern web applications. I ended up with a hyperplatform design: integrated UI, storage, and computes, all running in the cloud. Initially, I feared my approach would be too idiosyncratic to gain traction. But now I think that the concept of a hyperplatform is in the air. Replit, Heroku, and even domain-specific platforms like Shopify, are all expanding their capabilities, and the popularity of nocode platforms demonstrates the pent-up need for simplicity.
At the dawn of the personal computer revolution, apps like Lotus 1–2–3 and WordPerfect each had their own UI layer, storage abstractions, and even printer drivers. They had to, as DOS was too primitive. But then Windows (and MacOS and others) came along and gave apps access to an integrated platform. With the center of the computing world now shifting to the cloud and the datacenter, I believe the time is ripe for the rise of the hyperplatforms.
[1] No hate on Loggly, Splunk, Sumo Logic, Datadog, Graylog, Logz.io, Scalyr, or the other logging systems I found in my cursory search. I’m sure they’re all awesome, and I’ll probably end up using one of them.
About: I’m George Moromisato, a software engineer building GridWhale, a new cloud-computing platform. Every once in a while, I’ll post an update about its development. Interested in helping out? Write to us: contact@gridwhale.com.
