AI ART🎄 Using Stable Diffusion AI to 'photoshop' details (Patreon)

Most people from what I've seen use Stable Diffusion to generate random images. The true power of this Stable Diffusion I think is in selective editing and having it "photoshop" certain specific details while keeping the rest of the image intact.

For very custom shots, instead of going through random results, you can have the AI make variations of a specific part of the image, pick one you like and then have the AI move to another part of the image.

To do that you can use the Draw and Inpaint features. Here's an example of how those can be used to get more details from a VAM screenshot. This is the image I started with:

STEP 1: Add initial image and click Inpaint

I started by describing the image in the prompt. I set an image size and then clicked Inpaint, this shows a preview of the ouput image size based on what size you picked. If the size doesn't match (image ratio looks weird), you can click cancel and update the Image Size to better match the ratio of the input image, and then hit Inpaint again.

Inpaint allows you to select an area where the AI will work. It will keep the rest as it is. The opacity thing doesn't matter, it's just a visual thing for you. I don't think the sharpness does anything at the moment, nor colorpicker.

There are two actions I use:
- draw - I set opacity to the most transparent option (last option) and draw on the image
- erase - I set opacity to full (first option) and erase. you have to erase an area completely for it to actually be left out

You can also uncheck Inpaint or hit Clear if you don't want the AI to take it in consideration.

STEP 2: Select with Inpaint an area to change
Here's what I drew:

I picked just the pants area and told the AI to generate a few images.

STEP 3: Pick the one you like

the AI generated a few examples, editing just the jeans in the area I selected

I wanted to make things a bit more interesting so I tried forcing the AI in a different direction. I changed the prompt and told it that the person is wearing "black spandex pants" instead of blue jeans and did another series of images. The AI came up with these:

It went towards spandex pants but didn't believe me they were black. To help it I clicked on Draw, picked black and set opacity to be a a bit transparent and drew over the pants like this to make them darker:

it doesn't have to be perfect at all, the AI figures it out. After I added this in Draw, inpaint resets so I had to go in Inpaint again and select the pants area. Inpaint resets when you change the image size or the input image.

and then I generated images again:

I could have added more black paint over the pants for darker variations but these were good enough. I picked a version of the pants that I liked and selected Use as Input on it.

STEP 4: Repeat the process for each individual thing you want to change

After changing the pants I proceeded with the tshirt. I went in Inpaint, hit Clear and selected the tshirt area.

I told the AI in the prompt that the person is wearing a 'white hoodie' so the AI will go more towards that direction

the results felt kinda bland so I pumped up the Guidance Scale from 6.9 to 20 to make the AI go a bit crazier with it:

It went more creative with it but I went and added to the prompt "wet" to the hoodie and also added "sexy,erotic" to force the AI to go in a sexier direction with it:

Picked one that I liked by hitting Use as Input (to have the AI keep that and continue working on that version of the image). Then I went to Inpaint and selected the hair. I drew over the head and then selected erase and cleared the face:

Then generated a few images and picked one (selected Use as Input). I didn't add anything in the prompt about hair so the AI went with what it thought about it seeing the vam image. I could have added "braids" for example or "buns" and force the AI to go in that direction.

You can keep the VAM face if you want continuity across multiple images. I wanted to change the face too so I went to inpaint again and selected that. I also selected the arms to let the AI work a bit on them too and add more details to them as a final touch:

The first faces were meh so I added a few more things to the prompt to help the AI: "gorgeous, closeup, makeup, beautiful face, model" and also enabled Fix incorrect faces and eyes. (on zoomed out images, when the face is a small part of the image, that's needed for better details)

And these were the results:

You can change the sliders to get more variation. I picked one result as a final image and hit Upscale to get more details from it. Final result:

Nothing crazy but very specific in the direction that I wanted and with details that I picked. Doing random images you might end up with parts you like and parts you don't. Using Inpaint & Draw you can steer the AI in the direction that you want and have more control over the final result.

TIP

If you have problems with fingers in some images as it usually happens with Stable Diffusion, it's best to just keep the original hands from VAM. You can have the AI modify other things with Inpaint and avoid drawing in inpaint over the hands like I did at step2. When I selected the pants I also selected the hands and the AI ended up changing the fingers too in my case

BONUS

For the barbie banner image I used:

and used Inpaint to do something like this: