Reborn dev-ops (Patreon)

This post is brought to you by Enumag, as mentioned before! It will begin here, but because much of this is information that will be prudent to the greater fangame development community, this will also be cross-posted to the dev blog at some point in the future. You all get the early access~
Over to the guy~

###########

Hi, everyone! Enu here. I’m the dev-ops guy. I don’t work on Reborn exclusively. These days I’m more of a shared resource between Reborn, Rejuvenation and Desolation, supporting all three teams as needed. My main aim was to make the development and release process easier for the teams and later to improve some aspects of player experience. This post could be especially useful for those of you who work on your own mods or even your own games.

Maybe a brief introduction first. I first found Reborn around episode 14 and it quickly became my favorite Poke game. Occasionally I tried to look into some bugs that were really annoying (Ame mentioned them in her previous post). Also I know Git quite well (it’s a version control system, a great thing for software development) so when Ame’s team started to use it I gave her an introduction about what it can do. Later when she started talking about episode 19 alpha on Patreon I half jokingly asked where to apply. I didn’t really expect to get in but thanks to those past involvements Ame actually did invite me into the alpha testing group. And after a while I got Git access as well. Little did she know what I’d do once granted such power. I had some ideas but didn’t tell her at the time because I didn’t want to make promises. I wasn’t sure what was possible and if the team would like it. But I have wanted to make improvements for Reborn for years and now I finally could. It was time to automate things!

Eevee

One notable thing about version control systems like Git is that they work well with text files but not with binary files. (Such files are basically computer gibberish which can mean anything and therefore unreadable for Git.) The issue with RPG Maker XP is that it stores maps and some other things in a binary format. You know, the .rxdata files in the Data directory of the game? Yeah those. I didn’t like that so after some research I found a super old tool capable of decoding them. So I forked that, made it more useful for actual development and called it the Easy Essentials VErsioning Engine, aka eevee.

Now what can this little fella do for you? Well it can transform .rxdata files to .yaml which is a text format. And vise-versa. This means Git can understand it better and to some extent merge changes done by two people on the same map. Plus the yaml file names also contain the map name, making it much easier to see what locations were affected by which commit. Also yamls can be searched. Wanna know all the places where you can obtain a particular item? Or bulk replace some outdated function call when refactoring? Find all events where a pokemon is removed because all of those were bugged? (And yes, they are bugged.) We can do that now. It’s not an easy tool to use since even the yamls contain a lot of useless gibberish. This could be improved by implementing a custom format but that would be a lot of extra work so I didn’t. Still, even with the yamls capable modders can potentially use this to merge some of their changes with the changes we’ve made for the next version of the base game as long as neither side did anything too crazy. (Please make sure that the IDs of your variables, switches, tilesets, common events, etc. don't collide with those added to the base game before attempting this!) Internally we did successfully update Cad’s Gen 8 mod this way. Reborn’s dev branch already had a few hundred extra bug fixes from myself so we wanted to synchronize it.

Game developers can use this for development. The way it works is that eevee is running in the background while you’re working in RPG Maker and automatically transforms the .rxdata files into yamls whenever you save the changes. Then instead of the .rxdata files you commit the yamls into Git and when your team-mate pulls the changes, their eevee will detect that their local .rxdata files are outdated and will recreate them from the yamls. I migrated Reborn to use this during e19 alpha. Both Rejuvenation and Desolation are also using it these days.

Patches

Next thing on my hit list was the release process. I wanted to automate the process of detecting all files that were changed, zip it and upload the zip somewhere for the players to download. The first part was this:

git diff --name-only --diff-filter=ACMRTUX <commit>..HEAD

This command gives you a list of all the files that were changed since the commit you specified. You can feed the result to another command that would zip all those files. Now with eevee involved it’s not as easy because git would give you the yaml files instead of the rxdata files that the players need. So I wrote the logic to generate a patch into eevee directly. It uses this command internally but then detects which files are the yamls that should be replaced by rxdata files in the patch.

GitHub Actions

Time to shift gears. Even when you have the patch zip you still need to upload it somewhere. And while we’re at it why would you even make the zip yourself even when it’s faster using the command? Software developers automate all of that these days. GitHub Actions can do exactly that so I wrote a simple workflow that would trigger any time Ame tagged a new release in Git, use eevee to generate a patch and immediately upload it to her server where the players can get it. With that making a new patch was just a few clicks, Ame could focus more on development and easily release several patches per hour if she felt like it. I think we first used this for Reborn’s e19 public release. If you’re curious, the workflow looked something like this back then. Since then it has evolved a lot. Desolation 6 and Rejuvenation 13.5 were also using some newer versions of this and I made even more changes for the upcoming version of Reborn.

mkxp-z

With eevee and my workflows being adopted by Reju and Deso I suddenly had Git access to not just Reborn but all three games. So… time for more fun, right?

The workflows I mentioned above could only handle the small incremental patches. But the main releases of Reborn e19 and Desolation 6 were still handled manually. Changing that would be a much bigger fish to catch. More like a whale in fact. The main complication is that the games were not windows-only but also had macOS and Linux releases. It wasn’t enough to just remove the dev-specific files and zip the rest because the repository didn’t contain the runtimes for the other platforms. The engine the games are using for runtime is called mkxp-z. It can be built for all three platforms and compiling it is not an easy process. Particularly because to compile it for a particular platform you need to compile it on that platform. And I don’t have a mac. There are pre-compiled versions of it available but to complicate things further the games were using a slightly modified version of mkxp-z so that they could have a custom game icon and increased frame rate cap. If only there was a way to compile this automatically with our modifications, right? Well, guess what, there is. The maintainers of mkxp-z have their own GitHub Actions workflows to compile that thing for all platforms. So I didn’t need my own mac. I just forked it, applied the tweaks we needed and soon I was able to get a compiled version for Reju. It took several tries and each compilation takes over an hour because mkxp-z is a beast. But it worked so afterwards I could set up another GitHub Actions workflow to combine the compiled mkxp-z with the game files. Couple test versions later and we could fully automate Rejuvenation’s main release 13.5.0 for all platforms, no need for Jan, Cass or anyone else to painstakingly prepare it on their own. And of course it will be reused for Reborn and Desolation as well when the time comes.

I won’t share the workflows here as I still need to occasionally tweak them so the public version could soon become outdated. Yet there is no harm in sharing these so in case you want to set any of this up for your own games, hit me on Discord. Eevee is fully open-source so you can use that freely.

Things up to this point were all already used in Rejuvenation 13.5 but it was all changes for the devs, not for players. Everything below here on the other hand is completely new and focused on player experience. These improvements will be first used in Reborn 19.5.0. (In case you’re confused about the version number since the current Reborn version is 19.16, it’s because it should actually be 19.0.16. The convention just wasn’t correct back then. Also there never was a 19.5 release, it was 19.05 which should have been 19.0.5.)

Gamepad controls

The nice thing about this new mkxp-z setup is that it’s now much easier for us to update it to a newer version and to make occasional custom tweaks when needed. For instance I wanted to tweak the default controls because it always bugged me that the Previous Page and Next Page (for pokedex, bag etc.) were only bound to Q and W but not to Page Up and Page Down by default. These defaults are baked in mkxp-z’s source code. Yes, players can add them through the F1 menu but I wanted to improve the defaults. Also not all things can be changed through the F1 menu because for instance the F12 soft reset is an internal function of mkxp-z which you can’t rebind. This meant there was no way to soft reset while using a gamepad except for some clunky external programs where you would make a gamepad button trigger F12. I tweaked our mkxp-z fork so that a gamepad guide button triggers soft-reset. Then I went through the default controls for both keyboard and gamepad, discussed it with the team a little and changed a few default keybinds to make more sense. Keyboard only got small changes while the gamepad controls were tweaked quite a bit.

Also after some research I found out that I can actually detect the strength of the press on a gamepad L2 and R2 buttons. This is a bit of an experiment but I made turbo speed dynamic depending how strong you hold L2. This might be a subject to change depending on player feedback but we can adjust it quite easily. There is another button to make turbo permanent without holding anything if you prefer that.

During the game intro there is this screen with the default controls, right? Except it was a bit outdated because it was an image which was difficult to edit. And there was no such screen for gamepad controls. So I scrapped that and made a new screen which is up to date and actually generated by code so it’s easy to edit later. Then a second one for gamepads. And you can access them any time, not just during the game intro. There are a few conditions in there to account for game differences because Reborn has rotating tile puzzles and will eventually have follower mod integrated unlike the other games.

Old screen (quite a few keys are incorrect here):

New screens:

JoiPlay

One thing we noticed when Rejuvenation 13.5 came out was that quite a few people tried to play on Android using JoiPlay. This was on my radar for a while but I didn’t know there were so many people who wanted to play on their phone so it didn’t seem important. Well I clearly underestimated the demand. Several times a day someone asked on either Reju’s or JoiPlay’s Discord about how to make it work. I reached out to the person who prepared the fixing patch who is quite knowledgeable regarding JoiPlay and got a lot of useful info about how I can make the games run better on JoiPlay and how to set up default controls. So the next major releases of all three games will come out with an official JoiPlay version, set up with good default controls and a few other tweaks.

For instance JoiPlay has a feature where it can optimize the tilesets to be smaller to fix some problems. I took the script they’re using, added it into eevee (fixing two bugs I found in the script) and made our workflows automatically prepare these optimized tilesets when building a release. This involved some more technicalities such as building a Docker image for eevee where I had to compile a library which has been unmaintained for 11 years but let’s skip that.

I also tried to set up good default controls while using JoiPlay with a gamepad. Here I’m not actually sure if and how well it works because JoiPlay can only recognize around half of the buttons of my Dual Sense. With a different gamepad it could work better but I can’t test it myself. Feedback will be appreciated when this comes out.

Built-in updater

Reborn episode 19 came with an updater (shout out to Aie who wrote this) which you could use to get patches. It did its job well but had some issues such as mac support. Later on when Desolation 6 was about to come out I whipped up my own updater called gogoat. (It’s written in Golang so why not continue the pokemon naming convention right?) After some tweaks this worked on mac as well by the time Rejuvenation 13.5 entered testing. However the problem was that even this would be clunky on Android even if I could compile it to run there. If we were to make a release for JoiPlay I wanted the patches to be easily available there as well. So back to the drawing board.

The JoiPlay community member I mentioned earlier came up with an idea to download and apply a patch from a preload script. This was a cool proof of concept but if we can make the game update itself, it should work that way on all platforms. It sounds very cool and user-friendly so I wanted to see if it was feasible.

To achieve this I basically needed two things: To download a patch and to unzip it. Meaning a http client and a zip library. A http client is like a web browser for programs. It makes a request to a server and gets a response back - in our case the patch zip. Sounds pretty easy but is actually around 100 times more complicated than it has any right to be. Why? Because the Ruby instance which mkxp-z and JoiPlay are running internally is severely restricted. Neither a http client nor an unzipping library is available in mkxp-z. Well technically mkxp-z has HTTPLite but you can’t really use that for file downloads. So I reached out to the mkxp-z community and after some trials and errors we managed to get Ruby’s standard library working in mkxp-z on all three platforms. This contains a http client. The rubyzip gem is written purely in ruby so we could just copy its source code and get it working as well. However this didn’t support https. (Basically a security layer for web browsing, it’s very important.) After a while and some more help from mkxp-z maintainer I managed to patch it to make openssl available and https was working correctly. It didn’t have root certificates but I can get those elsewhere. (These are needed for openssl to decide if the server can be trusted.) So in the next release you might see directories called stdlib and gems, two new dlls for openssl and crypto and a cacert file. All of that is purely for the built-in updater to work. But even if it’s quite a few extra files, it’s only around 15 MBs total so not a big deal.

Alright, now that mkzp-z is done, let’s get back to JoiPlay. While there is no way to get Ruby’s entire standard library running on JoiPlay, it actually comes with an additional method called HTTPLite.download built-in. So it should be easier to download the patch there, right? Wrong. For one while it does make https requests I suspect it doesn’t actually check the certificate is valid. But I can neither verify that nor do anything about it. At least it works. Except for one important detail. In some cases (patreon-only builds) we also need to send an additional authorization header to the server with an access password. The download method does have a headers argument but it’s broken and the headers are not actually sent. So I had to implement some server-side band-aid to accept the authorization in the url instead of a header. As for unzipping it while the rubyzip gem doesn’t fully work on JoiPlay because some dependencies are missing and impossible to add, the important part - unzipping - does actually work after a few tweaks in the code to suppress unrelated errors.

Finally after all of this the new built-in updater seems to work correctly on all four platforms.

I only got this thing fully working very recently and it wasn’t tested by players yet. I really really hope the idea won’t fall apart due to some unforeseen and unfixable problem. We’ll see during Reborn 19.5 testing. Fingers crossed!

That’s all for today. If you have some questions feel free to ask in the comments. Hopefully you’ll enjoy the improvements!