Texture Conversion Script Upgrade (Patreon)

I’ve recently been using RenderMan in Gaffer and the need to pre-convert textures to RenderMan’s .tex format has come up. A while ago I shared a script I wrote in PowerShell and Python that automates the conversion process for RenderMan, Arnold and Karma. See the original article here: https://www.patreon.com/posts/99745842

As I was slowly grinding through a big folder the other day I happened to think: wouldn’t it be nice if this was multithreaded?

Indeed, and after doing some reading I was pleasantly surprised to learn that PowerShell 7+ supports multithreading!

I rewrote the script to take advantage of this very powerful feature. It’s actually easy to implement, simply add a -Parallel flag and a -ThrottleLimit if you want to control how many threads are used.

Well… there’s one more detail to consider. You can’t access variables from outside the Parallel block. This makes sense: you shouldn’t have multiple simultaneous processes messing with the same variables. Bad things will happen.

Trying to figure out how to make all the variables in my script somehow contained within the Parallel block would have been a pain, but thankfully there’s an easy solution. You can read any variable by making a copy with the $using: syntax. This way every variable in the script can be made available to all threads. It’s pretty great!

I took the opportunity to update a few other parts of the script as well. I made use of switch instead of if in a couple places. I think it’s a bit easier to read.

I also updated the texture processing commands. In particular, RenderMan’s txmake defaults to a TIFF/LZW combo that produces files that are three times larger than they need to be. I switched to OpenEXR/ZIP and all is well now. Arnold’s and Karma’s utilities have good defaults and their file sizes are fine.

Lastly, I did some fancy PowerShell tricks and processed the entire folder in advance, storing all the needed data in a PowerShell object.

The threads can then access this object and process or skip files as needed without having to scan the folder again.

As always, there are a few things that could be improved. The main one that comes to mind is that the entire folder is stored in the object, including files that won’t be processed. Most of the time this won’t be a problem since usually the target folder will contain only (or mostly) the textures in question. But for maximum efficiency that behavior should be changed. Anyway, have to leave something for version 1.1!

The script is available for download below. In my tests it speeds up conversion by at least 4x. Just update the variables at the beginning to reflect your system paths and OCIO settings. Interestingly, I think this should work on Linux and Mac since PowerShell is cross-platform. I’d be curious if anybody has success with that. Let me know!