"And now for something completely different..." (Patreon)
Videos
-
Source.mp4
Downloads
Content
I recently posted (https://www.deviantart.com/sedartonfokcaj/art/1297389985) a video generated by AI.
For this post, I'm doing something a bit different. But also to say that I've been working on a 3D version of both the mask and the pads. The kneepad is meshed out, remains untextured and untested for shape change, and it's something I'm worrying about but scared to test and see.
For right now, I'm going to run through how the Black Omen introduction video featuring Erica Deville was created.
It involves some existing assets (i.e., my EWF logo), a few key pieces of software. It also required a great deal of trial and error, which means a lot of time and dead ends.
To start, I render the AI using a separate computer running Ubuntu. It's not really required for most people, as there is a self-contained Windows version and I have run it on my PC for a while as the card in that machine is 24GB. The problem is that's the machine I render DAZ with, and I can't run both at the same time. Which explains why I have a dedicated machine.
Inside that server are two nVidia GPUs: A 3060TI and a 5060TI. Given the limitations of ComfyUI, only my 5060TI is used for image, audio and video generation. The 3060TI is running ollama and so I can run local chatbots using it, which can be integrated into ComfyUI workflows.
As for ComfyUI, it's basically a set of python scripts that also runs a webserver. This is why I'm using my PC and a web browser to interface with it, though I'm running an SSH session to start the ComfyUI system itself; I still haven't gotten around to having it autostart as a service.
I created a local script inside my /opt/ComfyUI directory that makes it a single command to start. In it, I force the webserver to listen on a local IP address and also to force the use of the 5060TI, as ComfyUI out of the box only allows once GPU per instance. That means the 5060TI is going to be used. It's a 16GB card as opposed to the other which is only 8GB.
Now, I haven't done it yet, but a lot of people have said that disabling GPU-based VAE Decoding in favour of CPU-based decoding will lead to more stability, yet takes more time. I'm hoping to avoid having to do that.
For those new, I do recommend installing ComfyUI-Manager as the first set of custom nodes to install.
The following is some serious nerdness:
I run my /opt off my nVME drive, but I also have an 8TB SATA HDD. On that SATA drive I store all the various models, checkpoints and so on. This allows me to delete my ComfyUI directory, and I've had to a number of times. I essentially link the models directory off my sda device into the /opt/ComfyUI folder. Even on gigabit fibre, losing 15GB-sized or larger files is painful. The nice thing about this approach is that I can use wget to access and download models easily via an SSH command line, even if some require authentication keys. Nothing copy and paste to notepad then into a terminal can't get around.
I used to have my models stored on the NAS so that it was also accessible on other machines, but for speed purposes, all of them are now on an internal SATA drive.
Next up is the output directory. That's where ComfyUI drops all generated items. In this case, I have this linked to a point that connects to my NAS over CIFS, which automounts during boot using /etc/fstab. That makes all the output easily accessible to anything on my network, so I can use my PC or my Mac without having to copy stuff over or fiddle with USB drives.
Finally, I've learned to link up the /users/default/workflows to my NAS as well for the same reason I link the models directory. If I ever have to delete my ComfyUI setup, I don't lose the workflows. This setup means that whatever I generated is instantly saved and accessible over the NAS. I had simply decided to mount the entire users folder, but the sql subsystem ComfyUI really doesn't like that, hence why I only do the workflows directory, which is the only folder in that entire tree that I even care for.
The entire project began around Qwen Image 2512, which generated a series of images from a text prompt. I did a whole bunch of images with Erica Deville in front of various wrestlers in a managerial role. The workflow itself was used to help me visualized a proposed project involving an Appalachian wrestling federation. And so I created this locker room and a pair of wrestlers behind the current federation owner.
This is probably where my current aspect ratio is coming from as I like to maximize it at 928x1664, which is the maximum image size that it can work with.
Here's what I managed to get from my text prompts:
As you can see, I just had to put Deville in. I did a bunch of different kinds of teams, all male teams, all female teams, but a lot of intergender teams. Then, because it's AI, I moved the federations to Mexico and Japan. My choices ended up limited to "American" people, given the limitations of the voice generation system (it can't do accents). That ruled out Mexico and Japan, leaving a pair of masked female wrestlers and masked male wrestlers.
Next thing I did was use tts-audio-suite and the Qwen3 voice system to create the voices for Deville, Eclipse and Stygian. There's a lot of trial and error there. I also set up the ability to use emotion control for all three voices (note: convert to WAV using something like Audacity to avoid crashes in certain workflows). I made individual voice samples for Eclipse and Stygian, then used Davinci to line them up into the timeline. When using voice actors, it's a good idea to create a text file with the same file name so that it will appear in the Index-TT2 workflow. I made each person say the same things so it's just a matter of copying the file and renaming it.
This allows me to somewhat control the tone and speed of the speech, and is a lot of trial and error. Once I generated the voices, I could then
I ran the ladies through the WAN 2.2 14B image-to-video workflow first. That workflow generates silent 5-second video clips. It came out AWFUL. Mostly because the female hair just looked like it was plastic. The nice thing about the guys is that there's no hair to speak off and, quite frankly, it came out way better than it should have been. I use this to generate my "b-roll" involving the guys.
So the first clip I did was the one where I put in the audio in by hand (it's attached as Source.mp4 below). This is when I knew I had to try making a longer video.
I also cropped the image so that each wrestler had their own face so it doesn't confuse the lip sync. If you have multiple faces, it often makes everyone sing a chorus.
For the lip synch workflow, I used the WAN 2.2 S2V Audio-driven Video workflow. That was also a fair bit of trial an error. If you have good voice tracks.
At this point, when I had enough generated clips, I used Davinci Resolve on my Mac to assemble the video, generate the few effects, throw on the EWF logo and then export it out.
Now, all three of the workflows (Qwen and WAN) are straight from the templates provided after ComfyUI is installed.
What I've learned:
When using WAN workflows, if the output image is this pink output, simply restarting ComfyUI will not fix it. I've had to reboot completely. One of these days I'm going to put back in the custom nodes that allows for ComfyUI to clear VRAM.
I've had a lot of freezes where I cannot even ssh back into the Ubuntu server, forcing complete power cycle. Not seeing any reason at all in the various logs.
For Davinci, don't delete files off your HDD until you're project is complete and exported. I suppose there's an option somewhere where i might save clips into the project file and not just be links to a file somewhere else.
I managed to do this without having to resort to first frame/last frame workflow.