Witcher Rig Progress (in detail) (Patreon)
Content
*Wipes sweat*
Well, I warned you guys(and myself) that this would be a massive undertaking, and it is pretty much as expected. I've been working on this for the whole month, and it is coming along although perhaps a bit slower than I expected, still better than I expected!
I'm sad that I couldn't give you any animations this month since this is the only thing I've worked on all month. That's not to say I was lazy, I put in more hours this month than possibly ever before, maybe only coming in second to that one hardcore October in 2017.
I hope you'll be able to appreciate the long term benefits of working on a massive project like this, since of course, with great rigs come great animations from fellow artists, for everyone! ^^
By coincidence, working on this rig also had me work toward my $2000 goal which maybe everyone else forgot about, but I did not! I said I wouldn't get started on that goal (which is to create free scripts and tutorials to help upcoming artists) until Blender 2.8 comes out, and since I decided to make this rig for Blender 2.8, it seemed like a good time to get started on this too.
So let's jump RIGHT into it.
Figuring out Export/Import
Before finally resorting to extracting from the game, I checked the available SFM and XNA models, just in case they would fit my goals, which was unlikely. Unfortunately it looked like all of them had gone through one destructive process or another, meaning some information from the original game files was lost, usually involving textures and materials. The game uses detailmapping to have an appearance of ridiculously high quality textures, and most people don't bother with said detailmaps, but beside that, the specularity maps from the game were nowhere to be found, since for the phong-based renderers of SFM and XNA, these maps had to be merged into the diffuse texture.
This meant I had no choice but to extract from the game , which turned out to be nothing less than a massive pain in the ass.
I had Google-and some incredibly helpful guys on the SmutBase discord on my side. Shoutouts to Ganonmaster and Irastris. Irastris put me on the right track for using the best available extraction tools(which are still shit but better than what I could find), and Ganonmaster found a way to fix blender's .fbx importer so that it could import the files that were pooped out by the extractor correctly.
At this point I could import individual meshes with some basic ass materials, weights and bones, but no bone hierarchy.
The bone hierarchy was a fairly easy fix, just need to manually write a dictionary that contains the parent for each bone, then run a script to do said parenting.
The game's materials extracted reasonably well into an .xml file containing each material's parameters' name, type and value, but all that got imported by the fbx importer is diffuse and normal textures. This meant I will have to process the .xml file myself and create a material based on the information within.
Among many issues, there was one major one: The faces(Eyes, Mouth, Eyelashes) of the characters did not get exported by the regular method. More on this later.
Furthermore, each character, although neatly organized into their own folders, consisted of around a couple dozen separate .fbx files. Each of these would import with their own skeleton, which would need to be merged into one.
So, I decided to write an addon which would import an entire folder while taking care of all these issues. But I need to take care of some things first before I can get started on that, such as...
Figuring out the Witcher 3 Shaders
So at this point I said okay, let's pretend I already have a script which imports all the information from the XML. What the hell do I do with it once I have it? I loaded some meshes and textures manually to see what I'm working with, and attempt to recreate a material that looks like in-game.
Trying to make sense of what kind of shading model The Witcher 3 uses, I was terrified to realize that they use Specular/Glossy PBR maps which not only was unfamiliar to me, but not directly supported by Blender Cycles, yet. This could've meant that I have no hopes of consistently and faithfully recreating the looks of the game in Blender.
Thankfully, it is supported by Eevee, the new renderer in Blender 2.8, but even better, some smart people found a way to make it work in Cycles anyways.
Unfortunately before I found this I burned about a day and a half trying to convert Specular/Glossy maps to Metallic/Roughness. Doing it the other way around is apparently not an issue, but the way I needed it is simply not possible. It's like asking someone "I thought of two numbers and their sum is 36. What were the two numbers?". The best you can do is guess, and sometimes you get lucky, but for the most part you're screwed.
But still, while converting the textures from one shading model to another is not possible, mimicing the shading itself with Cycles nodes apparently is, and that is all I need.
Picture from a few days later, when the script first imported detail maps. Notice how the metal squares on the waist and the bronze buckles are indeed, metallic.
Is this... Vibrance?
While testing my material setups, I've also imported Geraldo's armor just as a test case. I just went with the one I happened to have on me in my save. This ugly motherfucker, Mastercrafted Legendary Griffin Armor. And when looking at the textures for this armor, things just didn't make sense. It looked like they're applying Adobe's Vibrance filter in their shader. So I went to Photoshop and tested it myself.
Albedo texture. (The big dark green blob on the left is the main armor bit. Notice how metals are black in this texture, because that's the way it is in Specular/Glossy PBR.)
Specular texture. Surprisingly blue/turqoise, tbh, but that's the advantage of this shading model, you can get creative by reaching just a bit outside of realism. In this case, the green armor part tints its reflections slightly turqoise.
Color Dodge them on top of each other (which is not accurate, just a rough estimation of the final colors). Now, I would expect this to look fairly similar to the final result in the game, but it is not even remotely close! That green still looks more like gray or brown, it's incredibly desaturated. But obviously I can't use saturation since that would destroy the reds and really everything(leather bits and more).
And finally, Vibrance to 100. Suddenly, it just looks like it's pretty much what I was looking for.
So how the hell am I gonna replicate Vibrance using nodes? Note that unlike things like Saturation or Luminance, "Vibrance" is not a standard filter or procedure. The Vibrance you can find in Adobe products is their very own proprietary implementation, they invented it, they named it, and they are not gonna share it.
Here's where the Smutbase discord comes in with the save once again, this time from Zeddb who linked me to the exact lines in this code by Frederic van der Essen, who seems to have done a great job of replicating the Vibrance effect. Now it was just a matter of turning that code into Cycles nodes, which is always a joy. This one was especially so, since the code is written in C, which is a big boy language and therefore completely alien to my puny pythonic brain cells.
And the result:
Before.
After.
This turned out to be the best implementation for Vibrance that I've seen so far(because this was not the first time I tried to reproduce the effect). Don't ask me how or why it works though because I have no idea. I just copied the other guy's homework. Big props to him.
Now of course if you scroll back up to the in-game screenshot there is still a world of difference, but if you can isolate the color component of that material in your mind, I think you'd say that the after picture is a better match than the before picture. So that's mission accomplished.
Unfortunately so far I couldn't find any values in the XML files that seemed to correlate to the vibrance values, so for now I will be resorting to entering the values manually when needed, by comparing to in-game screenshots. It does not seem to be used that frequently though.
Making the Witcher 3 Shader
Alright, we've got everything we want, now it's just a matter of making a nodegroup that combines all the things and expects inputs that are named according to the parameter naming convention found in the XML files. Here's an example:
The XML contains a "material" element for each material that will be found in the FBX(named accordingly, which unfortunately is just "Material1", "Material2" etc.). Within each element there are "param" tags which have a "name", "type" and "value" as I mentioned before. This will be useful later when scripting the material importer. For now, we just care about the parameter names, and guessing what they are responsible for.
Thankfully, the naming convention of the material names is fucking abysmal, so at this point I knew that when I'll have to script the importer, it will be a complete mess. But that's not what I need to worry about right now, just plugging in things to where they need to be.
So here's our Witcher3_Main nodegroup(aka shader) which is what most materials will be using. There's also Witcher3_Skin, Witcher3_Hair and Witcher3_Eyes which are similar but different. The script will use the "base" tag in the XML to decide which one should be used by each material.
The input node on the left has a bunch of pins, named after the more common parameter names in the XML files.
Notice that there's quite a few pins that don't go anywhere from the input node. That's because I'm not a 100% sure what they do and they probably aren't very important, but I'll keep them around just in case. If I ever do find out what they do, I can just plug them in to where I want them to, and it will affect every material in the scene that uses this shader, so no extra work will be involved.
In any case, this shader is done. In case you're confused how this will be used, a material will look something like this when the script is done:
I just load all the values from the XML into nodes(the type of node is decided by the "type" tag of each "param" in the XML) and plug them into the shader simply by matching their name to the input pin's name. Ie., "AOPower" node will be plugged into the "AOPower" input socket. Simple as a brick!
Writing the importer
So, while I was figuring out the above material related stuff, I was realizing that I will need an importer that can batch import a folder of fbx's and do the following:
Have an actual user interface so I can release it into the wild.
Import an entire folder of Witcher 3 FBX's (ie. Import a character).
Delete meshes that end in 'lod1' or 'lod2' since we don't need those meshes(low poly made for game optimization).
Clean up the rest of the meshes with some Blender operators(Seams from Islands, Tris to Quads, Remove Doubles, Weight Normals) and renaming stuff. In case any of this ends up breaking certain meshes, they needed to be options in the addon's UI.
Combine all the skeletons into one.
Create the bone hierarchy.
Fix bone rotations.
Load the .xml file with the same name as the .fbx
Create Blender materials using the parameters in the XML and the material stuff described above.
Detect duplicate materials, to avoid creating multiple materials that have the exact same parameters(which is very common)
I won't be going into much detail with the code, but I will say that I was an idiot for starting to write this on Blender 2.79, and then switching to 2.8 halfway through. That was a solid 5 hours of completely avoidable headaches.
0. Batch Import & Mesh Cleanup
The easy bits. I go through a folder and import every fbx, delete the LODs, then run some blender operators on the imported meshes. Also rename the objects and materials to the fbx filename, because otherwise the imported names are just "Mesh" and "Material" for everything, which is not very useful.
1. Armature Cleanup
The imported skeletons have no hierarchy. Nothing is parented to anything. Thankfully though, they do have consistent bone names(even if the bone names are full of typos and grammatical errors, they are at least consistent)
Ganonmaster made some ports of Witcher 3 girls to SFM almost 4 years ago, and thankfully he still had some of his scripts around from that time, so he hooked me up with a script to parent bones, including a list(dictionary to be precise) of child:parent relationships, and another script to combine armatures, which I wrote my own version of instead because I don't like messing with Blender's duplicate naming .001 bs.
From there it was just a matter of improving and expanding the dictionary. I imported some characters and every time they had some unparented bones, I figured out what bone they should be parented to(using my eyeballs and brain cells) and added the child:parent relationships to the dictionary. If you're wondering, a dictionary is a basic data structure that looks something like this, and it's just a bunch of key:value pairs.
And yes, I hate myself so I'm doing all of this in Notepad++ instead of a real IDE. (I plan to change that soon)
With this done, now I can finally move my characters' limbs without them coming apart, yay!
The bones also import without any rotations, which, to be fair, are not that important, but I wanted to fix them anyways. I did that using a bunch of rules based on how many parents/children/siblings a bone has, and some special rules for the children of the head and jaw bones, plus an extra dictionary for some bones that I wanted connected to specific other bones.
Before mesh & armature clean-up.
After.
2. Loading the materials
Probably the trickiest part of the process, because of how inconsistent the XML files were, the code for this logic is such a fucking mess, but it's really as clean as it can be.
Loading the XML into a nicely accecssible format (ElementTree) is easy with Python's XML API, thank god that exists and thank god the material info extracted into such a standard format.
But from there it's all downhill. Now, unlike most of the info I've written in this long ass post so far, I feel like the stuff I would put here is the least useful because it is only applicable to this specific problem, which I already solved so no one else has to, so I won't go into much detail, but let me just say: CDPR's material parameter naming convention is not good.
But, all is well that ends well.
3. Turning it into an actual add-on.
If you've been tuning into my streams you know I do a shitton of scripting, so it might surprise you that this is more or less my first or second time of trying to make an "actual addon", meaning adding a user interface, parameters, and making it installable so that an actual human bean can use it.
To contrast with the previous point, this info could be super useful for a lot of people, but there is already loads of documentation on how to do this, so once again, I won't go into more detail than what I can follow up with pretty pictures, since you've scrolled enough already, and there's still a lot to talk about.
First off, the importer requires the path to the folder of the uncooked Witcher 3 game files, which is always the same, so I added a path for that in the Add-Ons UI.
Then I added a new button under File->Import called "Witcher 3 FBX" which I might rename in the future to something more descriptive.
And then added some import options.
And as you can see, it's all done! You just pick a folder, let it run(can take up to 2 minutes for a character) and the result(after hiding some extra meshes):
Is pretty good! Except...
The eyes and the mouth
Yep, this gets its own header, that's how painful it is. Truth be told, all I had to do to get the eyes, eyelashes and mouth, was follow a simple guide.
"After this step, for the export itself, check the informations given on the tuto for TW2, it's exctacly the same things. The workflow described is not super user friendly, but I havn't find better yet."
Well, actually, all that guide tells you is to go and find the Witcher 2 extraction guide. But still, the guide is accurate.
"To get the characters with their skeletons, follow the process described previously. The only difference is that to export the skeleton, you should select the B3D format (Blitz3D), because this is the only exporter that support skeleton. This format not a very common format, but if you search a plugin or a conversion tool for your 3D software, you can probably find a way to use it. For example to load my files in Blender, I use the software Fragmotion to convert the B3D file to another format (if you use it, be patient, the models take approximately 2/3 minutes to load). This software is presented has a commercial software, but you can use it for free by typing a little text each week."
HO BOI. Thankfully, I didn't have to do this for entire characters, just the face parts. Although this guide was written MANY years ago, to this day the only tool on the entirety of the internet that can import .b3d files is this beautiful piece of software called fragmotion. (Click that link at own risk). In order to activate the free license of this magnificence, you must type out Matthew 6:9-13. Nope, not kidding, you type out the prayer word by word to activate the software for 7 days.
So once you're done praying to your lord and saviour, you can then export those .w2mesh files from The Witcher 3 Models Converter into .b3d format which could not be extracted with CDPR's own modding tools(wcc_lite.exe) for some reason. Then you can import that to Fragmotion, and export it from there into Blender. Well, actually... you need to export it to .ms3d, which is the format for another software, Milkshape 3D, but alas, Blender has an importer for it. Blender 2.7 does, anyways. So I import the .ms3d into Blender 2.79, clean it up, save it, and append it into 2.8, combine the armatures of the eyes to the face with some parts of the importer script, and bam, done! Truly, what could be simpler?
Also I have no way to find out what eye texture belongs to what character beside going in-game and checking, so that will be fun when I get there.
I will be writing a tutorial about the entire extraction and import process, which will include a lot of what's written here, and more, but that is gonna be a while from now.
The nude body
What, you think CDPR modeled pussies on them? Well, I can't complain because at least the rest of the body didn't need to be made from scratch, but sadly no, it's just an abyss of dark compression artifacts.
The first step when making a body hack like this is to choose the right parts. Frankly I had a plethora of vagina models to choose from, but pretty much all of them were quite stylized. Ellowas's, Noodlebox's, other models', what have you. The only reasonably realistic candidate was the DAZ vag, and I hate the DAZ vag. It feels like it was modeled by a gay gynecologist. An impeccable attention to anatomy without a shred of... sex appeal?
But alas, it is after all customizable, and the other options didn't seem viable, so I went and moved some sliders around until I was happy:
Default
Close, Sesame.
However, I actually took the butthole mesh from Ellowas's body because the DAZ one was unnecessarily high poly(so is the vag, but that was easier to handle)
I also removed the polys on the inside which were completely pointless. By doing these two things, I probably saved as many verts as there are in the rest of the body, which will be noticable in the viewport performance.
Inside of the DAZ genitals.
My result.
And then of course it was just a matter of UV unwrapping and texturing it, which I'm not going to go into at all since I don't really see the point, I brought the mesh into Substance Painter and c̶l̶o̶n̶e̶ ̶s̶t̶a̶m̶p̶e̶d̶ ̶t̶h̶e̶ ̶s̶h̶i̶t̶ ̶o̶u̶t̶ ̶o̶f̶ ̶e̶v̶e̶r̶y̶t̶h̶i̶n̶g did some texture art.
The shape could still use some improvement since it looks a bit off.
Weight painting the vagina bones was a bit of a pain in the ass because the weight paint tools, and the viewport during weight painting especially, is quite work in progress in 2.8, no point going into detail there, I just rigged it with 4 bones(top, bottom, left, right) and 2 shape keys which came with the DAZ body.
Rigging the anus on the other hand, I did the same way as I did on Elizabeth, and this is the main reason I didn't want to use the DAZ anus mesh. With this setup, each vertex in the edge loop gets its own bone, then I open the butt with those bones and very carefully make a shape key to clean up the shape. The bones can then be used with the Shrinkwrap constraints to be automatically animated. I will probably make a separate tutorial on this in the future as well, since it's actually a somewhat involved process, with a lot of important details that I don't want to get into here.
The Rig
I have meshes, materials, skeletons and weights, it's time for a control rig. I've loaded up MetsRig, the rig I've been using for all my Ultimate rigs, and not much will change in it for now in terms of controls, however the cloth switching system is getting rebuilt from scratch, and that is no easy task.
The New UI
First let me show you the new dynamic UI:
This UI is now a tab on the N panel (in previous versions of blender, this would've been on the T panel on the other side). It may look a bit intimidating here, but the great thing is that there are separate tabs which can be expanded and collapsed(Most of the time the IK/FK and Extras panels would be collapsed) In essence this does the same thing as the Properties bone's Custom Properties did in the past, but it's always available when the rig is the active object(don't need to select a specific bone, although the screenshot is deceiving), can have buttons, menus, sliders and dropdowns and all of those can dynamically change!
So why did the cloth swapping system need to be rebuilt from scratch? Well, for one, for this UI to work, but also for other reasons. And the following explanation for those reasons will be pretty dry, have no pretty pictures and lots of blender jargon. (Haha, as if anyone is still reading this)
The problems with Drivers
In my previous Ultimate rigs, I've used drivers for changing skins and clothes and other stuff. There are several things I don't like about drivers however:
1. They're not user friendly. If you put a driver on a value, the user will no longer be able to control it directly, not even temporarily. They will have to either delete the driver or find what controls the driver.
2. Although their impact on viewport playback performance is not that big, it is noticable(maybe a few fps)
3. They're a pain in the ass to set up
4. They're impossible to update en masse. If you want to add new functionality, you have to go through every single driver and add it manually.
The solution: More Python!
In Blender, you can include a .py file in your .blend file, which is a python script that can run when the file is loaded, and add almost any kind of functionality to a scene, or in this case, to a rig. And the potential benefits of these are:
1. I can develop a custom UI for the rig which will make it easier to use and expand in the future. Doesn't prevent temporarily editing values that are controlled by the script, since the script will only update them when you interact with the rig's UI.
2. The impact on viewport performance is lower, since I have the option to only run certain code when the relevant values are updated. In exchange, updating said values can be a bit slower than drivers, but since you'd never change outfits mid-animation, this trade-off is worth it.
3. They're 10x as much a pain in the ass to set up the first time around, but adding characters/skins/outfits is easier.
4. Since it's a single script handling all the cloth switching, only the script needs to be updated if I want to add new functionality in the future.
How do I achieve this?
So how do I replace a bunch of drivers, which were on specific objects? Drivers that used specific variables and checked them against specific values? Well, I did say adding new characters/skins/outfits is easier, but it's still not trivial. The trick is to make that information the drivers relied on available in the object. So what is this information?
Each object needs to be aware of:
what Character and/or Outfit they belong to.
what settings(properties) within that Character and/or Outfit they are controlled by.
what the value of those properties needs to be in order for this object to be visible.
Such information can be stored in objects using Custom Properties. For example, in Ciri's sword, I would store the following properties:
"Outfit" = "Ciri_Default"
"Sword" = 1
Which means that this object should be visible when the selected Outfit is "Ciri_Default" and the "Sword" value on that outfit is exactly 1.
I've also given myself the option to enter a list of values, such as [1, 2, 3] or even an expression like "# > 0" where # will be replaced by the value of the property, so the sword would be visible whenever the "Sword" property is greater than 0. And of course an object can have more than one requirement.
The properties for the outfits and characters are stored in bones, exactly like before. The only difference is that these values now have an extra layer of interface so instead of you having to interact with them directly, you can do so from the rig's UI.
A similar treatment is necessary for Mask Modifiers and Shapekeys. They also need to be aware of the same three things that were listed above, but they don't have custom properties.(They kinda do but this solution is better) So instead this information is simply stored in their names. For example:
- the "Ciri_Default+M-Corset1" mask should be enabled when the outfit is "Ciri_Default", and its "Corset" value is exactly 1.
- the "M-Hood23-Coat1" mask should be enabled when the "Hood" value is either 2 or 3 AND the "Coat" value is exactly 1. The outfit is unspecified, so it could be anything, for this object it's not important. (Since if the object belongs to an outfit, it will be hidden anyways if the relevant outfit is not selected)
TODO
And, well, that's pretty much all that is done so far. What's still missing then?
Hiding character/outfit-specific bones by moving them to a designated armature layer
Streamlining the process for combining new characters' skeletons into the rig. (Currently I have to run a bunch of separate code snippets. I need to put them all into one.)
Automatically adjusting the face bones' rest positions when switching characters. (Every other bone is a perfect match between all characters, but the face proportions and therefore face bone locations are slightly different)
Making some modified version of some clothes for lewdness(exposed tops, ripped trousers, etc)
Making masks and shape keys to allow for clothes to be removed without clipping
Some simple UI buttons are not yet implemented (IK/FK switches, hair override, modifier toggles)
And then of course, adding more characters!
Sticking to the original plan, I will first just have Ciri/Yen/Triss working, and release an Alpha for testing. When that happens I will probably make an animation or two, not only to test that everything is working but because I'm gonna need a "break" from all this. Then I'll get back to adding more characters, and when I'm satisfied, I will release it to the general public on SmutBase.
What the fuck was that giant wall of text
Obviously this post was stupidly long, and I don't expect anyone to read all of it, so if you just scrolled down to the bottom wondering why I even bothered to write everything out like this, I think there are multiple reasons. First, you guys can see that your support this month wasn't for nothing. Second, it was a useful excercise in organizing my thoughts on what I've done and what I need to do. Third, if anyone ever asks me in the future how I made some part of the rig, I can just link them to this post.
So yeah, enjoy scrolling and thanks for your support! Have an awesome day.
Update 29/01/19: minor text fixes