facial control/capture dev (log) (Patreon)

Honestly, I've always been terrible at facial animation. Daz's Genesis 9 deserves most of the blame, but I also never used its expression morphs, and the ARKit controls Faceit generates aren't that easy to work with either.

That changed once I tried MetaHuman (through a third-party addon called MHFRT). The workflow is genuinely practical, even if a few parts of it are clunky, so I'm doing some secondary development on it.

It really works. This is the best facial control and capture setup I've ever seen.

Good results do come with more complex controls and a steeper learning curve. Still, all the basic Genesis 9 facial bones are there, ready to pull up whenever you need them. The MetaHuman control rig will be baked into shape-key controllers (725+ of them) so it can be used without the addon. I'll write up the specific workflow in a separate document later.

To get capture that good, you need to record with Unreal Engine. An Apple device with a TrueDepth camera works even better, but a regular webcam will do.

For importing, I used my own FBX addon on this control rig, and I'll put together documentation for it too. Faceit's ARKit will be phased out going forward. If you have any thoughts, feel free to reach out anytime.

I used high-definition output for the last preview GIF, so it might load a bit slowly.

Actually, I did this research two weeks ago and have been looking into a lot of new things since. I just checked my drafts and there's another post in there — I'll publish it alongside this one later.

Once the current and next models are updated, I'll probably add this workflow to both. If you're interested and want an early look, feel free to DM me on Discord.