Quick Look: 4x Machine Learning Upscaler in Gaffer (Patreon)

I've posted a few times about using the ONNX Super Resolution image upscaler in both Gaffer and Houdini. It provides nice results, but the model is a bit old (2016) and has some quirks and limitations. I was studying the topic in more detail and found some interesting alternate upscaling models.

I settled on one in particular to focus on, 4xNomos2_hq_dat2_fp32, and updated my Gaffer scene to work with it. The nice thing is that it accepts any image resolution and dimensions and it also doesn't need conversion to and from YCbCr. That's all great, but what's the catch?

Well... it takes about one hour to upscale a single 1920x1080 image! The results are spectacular, so you could argue it's time well spent, but for regular use it would be impractical. John, head Gaffer, mentioned that adding GPU support to Gaffer ML might alleviate slow processing times. I had thought of that too, as I assume most ML/AI models are optimized for GPU processing. Regardless, it's on the Gaffer team's radar.

To play around with it for yourself, grab upscale_v3.gfr and tile_v3.ps1 from my GitHub here: https://github.com/BrianHanke/GafferML/

Instructions for downloading the model and setting up the scene are in the readme: https://github.com/BrianHanke/GafferML/blob/main/README.md

Finally, attached is a massive 8K butterfly to showcase the model's results. The level of detail is incredible!