HumphBot

Head Dino of the Dinoware Collective Galen came up with the idea of converting their Humph Day generator into a discord bot. Yours truly then volunteered to make it happen on the weekend for reasons still not clear to them. So begins an adventure into NodeJS and the Discord Bot Program!

Before getting stuck in with the horrors of server-side javascript lets take a brief tour of Humphrey’s Chomp and Stomp. The original game was developed for the Pico-8 virtual console which provides its own Lua looking language. Galen had since been working on expanding their original design by creating their own client-side javascript version which among other things powers the original Humph Day. It’s this browser version that forms the base of HumphBot.

For those of you blissfully unaware of NodeJS and the history of server-side javascript a brief and incomplete history: In the begining there was a web browser and it was pretty ok I guess. Someone had the bright idea of making a scripting language to give the browser some interactivity and javascript was born. Jump forward a decade or so and other people wanted to write web applications in just one language and with javascript the only choice on the front it was moved to the back. The end result was NodeJS and the birth of the current javascript ecosystem.

I had so far avoided ever using NodeJS to do server-side development and was happily writing regular web applications using it. That’s code that gets processed a bit by Node before being packaged up and run in your trust web browser of choice. In doing so I’ve gathered enough experience to know my way around javascript fairly well and to develop some rather strong opinions on the language. None of this experience was enough to prepare me for the journey undertaken creating HumphBot.

A Tale Of Two Documentation

With enough understanding of NPM and javascript in my brainbox getting a new project set up was a well trodden path for me. Wanting to know more I dived head first into the NodeJS documentation and things began to unfurl. I have long sung the praises of Microsoft’s documentation which clearly explains what every object and function does along with providing examples for almost all of them. In contrast the NodeJS documentation seems to provide the bare minimum of what properties are accepted and what they might do.

In contrast the documentation provided by Discord was a pleasant surprise for how well laid out and easy to understand it was. Helping this was almost certainly Discord’s straightforward REST API design which underpins most of the information needed from documentation. Example projects and tutorials covered almost all of the code I needed to get HumphBot up and running.

It’s no secret I was not a fan of NodeJS before this experience and it has done little to change my view on using it for any serious work. Almost every issue I faced with NodeJS required further searching after reading the documentation compared to Discord and my experience with Microsoft. Compounding this issue was the unclear difference between CJS and EMS providing two separate ways to implement code.

GIFs are Inefficient

Before you might think that Discord can do no wrong let me present a new roadblock. The discord client has support for Animated PNGs which are used to create animated stickers. The APNG format aims to replace the rather old GIF format as a way to, you won’t believe it, display animated images! The goal of this bot is to create animations from the game based on user input so the output format is a big part of the process!

Thanks to Discord’s choice of monetization the support for APNG is limited to only stickers, if a regular APNG file is uploaded or linked to then the inline preview will only display the first frame. This reduces the usefulness of APNG when making animated images to a nice round zero. Thanks to this choice the choice of output format is reduced to GIF or give up.

Why does this even matter you ask, due to needing to encode the output data in javascript the process takes twice as long when using a GIF encoder than it does with APNG. As I’m running this bot on a free Azure instance (thanks Microsoft!) this results in a 4 second APNG encode and 8 seconds for GIF. Not supporting them is a serious drain on the number of Humphrey’s that can be generated a day within the free tier limits.

Converting the Game

Finally the fun and interesting part of the project! Converting a browser based game to run on a headless server and output the frames so they can be turned into a GIF. Step one is to remove all references to the browser global window object; this could be tricky if we had to replicate functionality but as the game will run automated things like input and audio just need deleted. Thanks to the good development practices used by Galen these references only exist in a few files and, save for a handful of temporary test mentions, were a breeze to remove.

Next step was to replace the asset loading code which used the browser fetch process to download them from the same server the game is hosted on. Thankfully while lots of different asset types are fetched only a handful of places actually call fetch so replacing them with file open was straightforwards. Only a minor change to remove the URL property being used and replace with filename was needed to finish this process.

The biggest change required was to extract the main game loop and allow it to tell our bot process when it was done generating the animation. In the browser the end of the animation kicks off a GIF encode and continues to run the game loop. As a server we’d need to run the game loop as fast as possible instead of when the browser repaints and find out when the animation is done to stop! Thanks to the multi-step accumulator style game loop used in the original game it was easy to replace the check for elapsed time with a constant ensuring a single tick per call. Then instead of requesting the next animation frame the loop could exit.

To find out when the scene was done I decided to take the result on a journey back up the stack. Only the scene object in use knew when it was finished and this was not directly accessible from outside the game. Since the game made no use of return objects it was straightforwards to return the finished state from the scene update and in turn from the game update function. Combined with the adjusted tick this gave the server all the info required to run the game in a loop until all the animation frames were generated.

Finally instead of letting the user input the text and options with an in-game modal box they could be passed in as function parameters. The command used to generate GIFs in Discord allows separate blocks of text which are fed to the existing scene for each line. Originally the user could select what colour and accessories to use but exposing this complexity via a Discord command would be unwieldy. Randomly selecting which ones to use also gives the creator a fun surprise making the resulting animation more fun.

Discord is Impatient

Using a command in the client sends a POST to the server and waits for the response to declare the success. Unfortunately for us this times out in about 2 seconds which is well below the fastest encode possible. Instead of attempting to squeeze the encoding time down the sensible option is to hand the work off to a separate process and respond when done. Another restriction of NodeJS is not having easily accessible multi-threading, at least from what I understood reading the documentation but as already covered…

The solution is to use Worker processes which let you run a separate task and pass messages back and forth. With a full name of Web Workers you might get the hint that this tech comes from the client world of Web Browsers and is not something created on the server. Leading to the first issue with using Workers: how to load code into them to let them do anything. For browsers this is a simple process of supplying a JS file URL and letting the Worker download and execute it unfortunately for our server side process this is a little trickier.

Trying to pass the file name of our worker code results in generic errors and the server crashing without giving me much to work with. Looking through the documentation provided no hints to the issue but searching around hinted that the file argument needs the current file path first. Trying to follow the provided solution and use __pathname creates another error as this global variable is not supplied in the type of module in use. Luckily this error was easier to solve and with some more imports the right path could be passed into the Worker and now we’re ready to go.

One final hurdle to overcome was the restrictions on what code is allowed to be loaded into the worker. No specific errors are given and so through many rounds of try and see the result was no way to run the game itself. Passing data is also restricted so copying a built instance of the game was forbidden too. Many more iterations and the end result is passing an array filled with each frame of the animation.

In the end running the game to generate the frame data only took 300ms which is well within our response limit. Handing off the data to a worker to encode keeps the bot responsive even on the low power server. While the individual endpoint returns are fast enough the start up time of the server is substantial. Frequently the first command or two will fail to respond in time as the server starts up.

Weekend Wasted?

Absolutely not!

It was a great way to learn more about using NodeJS on the server and resoundingly confirmed my belief that it was terrible. The framework and language is bearable but the documentation makes working through problems a nightmare. I’ll try making other bots for Discord which thanks to their great API and documentation was a breeze but I’ll be sticking to my trusty C#/.net combo.

Galen enjoyed having their baby poked and prodded by another developer and was quite surprised by how well their code took to running headless. Dinofans have been enjoying creating their own animations which is all the feedback I needed!

Originally published on 2023/03/29