GridWhale Demo

I love programming. Sometimes, all the pieces of a program click in your head, and you pour out the code through the click-clack rhythms of the keyboard. You’re like an athlete gliding through the field, or an artisan wielding a tool with precision and grace. You feel at one with your task.

But other times, it’s a slog. In my last post I talked about the complexity of modern software systems. The solutions and services that we’re building today are inherently distributed. We’re writing a menagerie of cooperating programs: a front-end user interface program talking to a back-end app program, in turn talking to a constellation of microservice programs and database programs. The complexity is exhausting.

We have to do it this way because of scalability. A single piece of hardware could never support all the users of, for example, Gmail. We distribute the load across thousands of machines. Each machine talks to other machines in a complex grid of computation. And programmers make it all work with an ecosystem of frameworks, paradigms, and abstractions.

But what if we had a single, super-powerful machine? What if the UI, data, and files for millions of users were all on the same machine? Programs would be a lot simpler because they could access everything directly, without worrying about network access, caching, or asynchronous events. No such hardware exists, of course, but what if we built a virtual super-powerful machine? What if we created an environment that behaves as a single, super-powerful machine, but under the covers implements everything on top of a massive grid of discrete computers?

For the last year I’ve been working on a project called GridWhale to explore this question. GridWhale is a virtual computer, running on multiple machines, but acting as one. Programs running on GridWhale get the best of both worlds: they’re as easy to program as on a single PC, but able to access the resources of the entire data center. There are no limits to scaling across users or datasets.

In this video I show off a working version of GridWhale. I hope the simplicity of the programming model will bring back the joys of programming, without giving up the power of modern distributed applications.

Let me know what you think:

GridWhale Prototype Demo 02

GridWhale Listings

Are you interested in trying out GridWhale for yourself? All of the programs shown on the video are listed below. You can go to https://gridwhale.com and enter the programs yourself.

Listing 1: Hello, World!

# GridLang Program
print("Hello, World!")

Listing 2: Chat

# Chat
# Copyright (c) 2022 Kronosaur Productions, LLC.
# All Rights Reserved.
using Data
# Chat Log ---------------------------------------------------------
# Create a chat log and subscribe to changes
var chatLog = Data.open("chatLog", "dataQueue", { })
chatLog.onchange = function (op, value) begin
   print(value)
end
# Intro ------------------------------------------------------------
print("Chat")
print("Release 1.0")
print("")
# Print current log
var oldLines = chatLog.getValue()
for i = 0 to oldLines.length -1 begin
   print(oldLines[i])
end
# Main loop -------------------------------------------------------
var done = null
while !done begin
var line = input({
      prompt = "> "
      noEcho = true
      })
if (line == "!quit") then
      done = true
   elseif line == "" then
   else
      chatLog.push(System.username + ": " + line)
   end
end

Listing 3: Database

# Table Dataset Test
# Copyright (c) 2022 Kronosaur Productions, LLC.
# All Rights Reserved.
using Data
using UI
schema AsteroidRecord begin
   var name:String
   var mass:Float64         # Mass in quadrillion tons (1e18 kg)
   var meanRadius:Float64   # Mean radius in km
   var minDist:Float64      # Min distance from the sun in AUs
   var maxDist:Float64      # Max dist in AUs
end
var AsteroidTable = Data.open("AsteroidTable", "table", { schema=AsteroidRecord })
if AsteroidTable.getValue().length == 0 then
   var Init:AsteroidRecord = [
      { name="Ceres",
        mass=938.35,
        meanRadius=469.73,
        minDist=2.55,
        maxDist=2.98 },
      { name="Pallas",
        mass=204.0,
        meanRadius=256.0,
        minDist=2.14,
        maxDist=3.41 },
      { name="Juno",
        mass=28.6,
        meanRadius=127.0,
        minDist=1.99,
        maxDist=3.35 },
      { name="Vesta",
        mass=259.0,
        meanRadius=262.7,
        minDist=2.15,
        maxDist=2.57
        }
   ]
   AsteroidTable.setValue(Init)
end
UI.addCommand("Add Asteroid", function () begin
var NewAsteroid = UI.dialog(AsteroidRecord)
   if NewAsteroid then
      AsteroidTable.insert(NewAsteroid)
   end
end)
UI.addCommand("Close", function () begin
   UI.stop()
end)
var TableControl = UI.create("table")
TableControl.data = AsteroidTable
UI.show(TableControl)
UI.run()

Listing 4: Mandelbrot

# Mandelbrot
# Copyright (c) 2022 Kronosaur Productions, LLC.
# All Rights Reserved.
using Image
using UI
schema MandelbrotDesc begin
   var x:Float64
   var y:Float64
   var zoom:Float64
   var cores:Int32
end
var mostRecentDesc = {
   x=-0.75
   y=0
   zoom=3
   cores=4
}
var canvas = UI.create("canvas")
function MakeMandelbrot () begin
   # Ask the user for coordinates
   var desc = UI.dialog(MandelbrotDesc, mostRecentDesc)
   if (!desc) then return 0 end
   # Sanitize input a little bit.
   var x = desc.x
   var y = desc.y
   var scale
   if desc.zoom <= 0 then
     scale = 0.01
   else
     scale = 0.01 / desc.zoom
   end
   canvas.clearRect()
   # Generate Mandelbrot
   var coresUsed
   var nodesUsed
   var startTime = DateTime()
   var computeTime
   var mandelbrot = Image.mandelbrot(x, y, scale,
      canvas.width, canvas.height,
      { cores=desc.cores }, 
      fn (progress) begin
         if progress.progress == 100 begin
            coresUsed = progress.cores
            nodesUsed = progress.nodes
            computeTime = progress.time
         end
      end
      )
   print("--------------------------------------------------------")
   print("Time: ", DateTime() - startTime)
   print("Nodes Used: ", nodesUsed)
   print("Cores Used: ", coresUsed)
   print("Compute Time: ", TimeSpan(1000 * computeTime))
   # Show it
   canvas.drawImage(mandelbrot, 0, 0)
   # Remember settings
   mostRecentDesc = desc
   end
# Main Program ----------------------------------------------------
print("Mandelbrot v1.0")
UI.addCommand("Mandelbrot", MakeMandelbrot)
UI.addCommand("Close", fn () begin UI.stop() end)
UI.show(canvas)
UI.run()

About: I’m George Moromisato, a software engineer building GridWhale, a new operating environment. Every couple of weeks I’ll post an update about its development. Interested in helping out? Write to me: contact@kronosaur.com.