Prologue

Pyman, Capes are Cool

Right around the time of the impending apocalypse of 2012, I had envisioned an action platformer where the player would be able to interface with the world through programming. What resulted after a couple months of toiling was a very rough C++ SDL game with moving squares and a platform that took values interpretted from a Python script. In my head I had imagined various modules for platform manipulation and weapon mods, but fought myself with how to restrict the player appropriately so that something like god mode wasn’t easily accessible. I was too stuck in the weeds mulling over the programming mechanics that I felt paralyzed about how to further implement the embedding system. Thus, the Python embedding remained as a simple run command that ingested a file. At the time I had been playing Bastion, which received a port to Chrome through the NaCl interface taht allowed Chrome to run native sandoxed code. After a lot of interal screaming, I managed to wrap the game loop so that it worked with Chrome. Why did I do this? Today, I could say that back then I had the incite to realize that the web was the future of crossplatform development, but in reality I just sort of wanted to torture myself.

A couple years went by, and I had some free time while not studying for finals, so I decided to dive back into the code a bit. Turns out, in classic Google fashion, NaCl had been discontinued and there was no way to compile my project for deployment to Chrome. Enter asm.js. Not sure what Google-foo led me to this solution, but I think this turned out to be just as annoying as setting up NaCl compilation. The upside was that web assembly is still a thing in 2020 and has a very bright future that isn’t dependent on the whim of Google. Thanks Mozilla. The downside was that Python->C->C++->Javascript didn’t work at the time. At that point I felt satisfied with what I had learned to move on to other projects. I’m still proud of what I made though. My programs are my children, no matter how ugly.

Subsidiary, EVERYTHING IS ON FIRE!

For legal reasons, I won’t go into the inspiration for this project. I’m not quite sure what the timeline was, but a friend invited me to brainstorm for a game jam. The idea for an over-the-top depiciton of a dysfunctional startup company based in California immediately garnered the group’s attention when I pitched it. This came on the heels of a particularly bad year for California fires and… an average year for earthquakes. There were lots of crazy ideas for gags and level design, of which no notes were taken; don’t worry everything is in my head. One feature in particular that I was adamant on was integration of a full developer experience equipped with server communication, Docker container deployments, and CICD goodness. I wanted this to true to life software engineering game that informs the player about the failures and successes of the tech industry. I don’t think the others were particularly keen on that. My group and I lost interest and never even started a prototype for the idea, though my vision for the project would have taken much longer than the allocated time for the game jam. I had some time over that Christmas break to think about and research the different tech I wanted to use for the project.

Maintenance is a bitch and I’m a pretty lazy person, so I reasoned that rather than write a text editor from scratch, which would inevitably become a buggy timesink, I should embed someone else’s battle tested project. I get bug testing for free, with the exception of the embedding gotchas. There was a lot of contemplation about what language to use, but I eventually settled on NodeJS to leverage web portability and the tools already provided by many open sources packages. I don’t quite remember how long the ordeal took, but eventually I was able to embed an iframe into a rendering library inside an electron app. There were some issues getting the security warnings to go away, but once I had the iframe stable, I was able to hook it up to a locally launched linux web console https://github.com/butlerx/wetty. Why did I not use one of the many nifty engines built in NodeJS? Well the only rendering library that allowed for rendering iframes as objects in the scene at the time of writing this was three.js. The code that I integrated into my project comes from https://github.com/jeromeetienne/threex.htmlmixer/blob/master/examples/basic.html.

When break ended, it was back to work for me, and I lost a lot of steam getting back into the grind but did ocassionally chip away at some TODOs. There were some efforts to apply some of what I was learning for Subsidiary to work in order to gain some momentum, but I started to fall out of love with Javascript… actually I never loved Javascript, I tolerate it. I decided that I would shelve the project for a while, maybe convert it to Golang someday. I still love the story idea.

The project can be found here: https://github.com/Knights-of-the-Functional-Calculus/subsidiary

Angstrom, Streaming for the Memes

My contract with Insomniac Games had been terminated, and I had a lot of free time to do whatever. My first thought was to start working on Subsidiary again, and I tried to start up again, but I just didn’t feel like it was the right time. I decided to just look into random things to study while waiting for my trip to Japan to get closer.

I don’t quite know how I came up with the idea, but my brain thought that allowing streamers switch genders like in Snapchat would be amazing… and lucrative. Enter Project Deep Thot. Rapid Googling ensued. I needed to figure out first how get some machine vision model to graft a wireframe onto a human body in order to render the model on top of the streamer. Apparently this was a hard problem in a branch of machine learning called pose estimation. I hacked up a prototype in Python which can be found here: https://github.com/j4qfrost/pose-estimation-stream. Most pose estimation models were trained using 2D information or aren’t built for accurate realtime detection, so I didn’t have particularly great results. In addition to not having great models, my Python solution was slow and put my laptop on the edge of burst to flames. I needed to find a way to grab the streamed frames as soon as possible and as fast as possible, which meant building my own streaming application in a performant language. I also wanted a language that had reasonable machine learning support.

I found a great multiplatform library called Flutter by Google that I wanted to use, which had an embedder written for C/C++. C++ was a prime development candidate, but I personally felt that I was going to hit a lot of segfaults, and I didn’t think I wanted to deal with tooling issues while working with unfamiliar libraries. Rust and Golang were the only other languages supported. While both have excellent tooling, Rust has a smaller team of developers and exhibited performance issues compared to Golang. The advantage Rust has over Golang though is that its ffmpeg binding seem to be stable and don’t cause memory leaks. After a lot of pain, I had decided to go with Golang, since it had a bigger community and was more battle tested.

In the process of researching for this project, I found a treasure trove of tools in Rust for engineering a programming game the way I wanted.

Alloy, A Love Story

Tech

I was looking into foreign function interface (ffi) support for different languages in my fit of frustrations with the current state of embedded Flutter, and remembered that I had wanted to build a game that essentially leveraged this functionality. After I reached an appropriate stopping point for Angstrom, I returned to the Rust curation websites to evaluate game engines and looked at different programming languages that were hot on the market and had good ffi support. Python or Javascript? No, I want to avoid languages I know too well so that I could sort of have the perspective of a student going in. Lua? This is a popularity contest as much as it is a functionality contest. Perl or PHP? PTSD… Racket? The language is close to my heart and has great ffi support, but unless you have a few screws loose already, Lisp-inspired languages tend to drive you crazy. Ruby? High programmer productivity, something new for me, pretty straightforward ffi bindings https://github.com/danielpclark/rutie, and they’re both oxidized (this is of vital importance).

Initially, I shopped around for an engine that had sufficient tools for building a text editor out of the box, not expecting to be able to pull off a solution similar to iframe embedding that I was able to find with NodeJS. Amethyst seemed provide an out of box solution but didn’t handle multi-line editing very well. I considered building another editor, but that would’ve presented me with the same maintenance issues that I wanted to avoid with Subsidiary.

Running on stubborness and 4 hours of sleep, I realized I needed to step back a bit. If I wanted to integrate a terminal into this game engine, then it would stand to reason that path of least resistance would involve finding a terminal that was developed using the same windowing framework. Eureka! https://github.com/alacritty/alacritty I dove into the codebase trying to find some way to avoid writing more code, and it turns out that Alacritty has window embedding support. The downside is that the code was configured to only work on Linux. Solution? Throw it into a Docker container and download an X11 bridge for my MacBook. I just grossly simplified what took half a week to configure correctly. So now the state of the project is a Docker container that runs a game in Amethyst that starts up a window hosted by X11 which displays an Alacritty terminal. It doesn’t look impressive, but a lot of unknowns were knowned…

Creative

Since the days of ancient Greeks and likely before that, humans have been anthropomorphizing animals and forces of nature, perhaps as a result of the loneliness inherent in sentience. Nowadays, degenerates have started anthropomorphized things that really have no business being anthropomorphized. I would like to partake in that degeneracy by anthropomorphizing programming languages and shipping them. Both Ruby and Rust have so much in common: both are 4 letter words that share 2, both are metal oxides, both are shades of red. And they have differences that I can use to craft characters that can serve as interesting meta-commentary about the different levels of programming in addition to shaping their personalities.

All these whimsical decisions about what tools to use essentially serve the goal of making programming more fun, making them far less whimsical. Choosing a tool with a cool name is important, because what’s the point if it isn’t fun? I think I would be low in inspiration if instead of Rust and Ruby I chose to write the project in Cobol and Javascript. Cobol is the boomer language that is seriously lacking in meme potential and Javascript feels bad. My muses must be elegant, practical, and have endearing quirks and tradeoffs.

Closing

This post was pretty content heavy. I think I will do a biweekly post on my progress and keep them focused and brief. This week I will be doing more planning and thinking for Alloy and my other projects and start up programming again next week. I’ve come a long way since the end of the Mayan calendar. Back in those days I had a very rough conception of modular design and testing; I’d like to think I am at a professional level now. I have also made insights about how to avoid writing code I don’t need to. Best code is no code :D

See you in space Cowboy…