Setting the Scene (Patreon)
Content
Lately I've been experimenting with using more reference images in a bid to improve the continuity of the background scenes, overall visual style and tone, and character detail between the various clips that make up a longer scene.
In addition, I've started approaching scene planning from a bit more of a traditional storyboarding standpoint, with a goal of getting the clips that I need with less randomness, less wasted time and money.
So I've been taking my initial Stable Diffusion renders, then using them to generate a set of images in Nano Banana that show the setting from various angles (images like the ones at the top of this post). I cherry-pick the best of them, then use Nano Banana or Seedream to generate specific starting images for key video beats, using my Stable Diffusion character shots as reference images.
Then I pick the best of those, tweak them if needed to make them even more visually consistent, and use them as the initial frames of shorter clips that I can later edit together to make a visually consistent longer video.
Lost Cat was the first test drive for this approach, although I really only made a small number of initial images and relied like always on generating a bunch of longer clips and just working with the bits that worked best and that fit together cleanly. I love the result but I spent way more time and money on it than was necessary - the scene I'm re-working right now uses the approach a lot more aggressively. It's more work on the front end but should result in a higher percentage of usable footage and less work on the back end.
The real challenge, I think, is to make the Nano Banana/Seedream clips look and feel as much like my own work in Stable Diffusion as possible, so that the resulting videos to still reflect my personal style.