Home Creators Posts Import Register Favorites Logout

Downloads

Content

Tutorial : https://youtu.be/O8QdvZbRDp4

Attached Workflow is a lightweight AnimateDiff workflow, and demonstrate how to use SAM2 and Florence 2 for segmentation in an AI animation scenario.

Not gonna explain too much about AnimateDiff in this tutorial , because I did too many in previous videos and posts. Scroll back to old post if you missed it.

Related To AnimateDiff :

https://www.patreon.com/collection/393434?view=expanded

https://www.patreon.com/aifuturetech/posts?filters%5Btag%5D=AnimateDiff+Workflows

Segment-anything-2

https://ai.meta.com/blog/segment-anything-2/

https://github.com/facebookresearch/segment-anything-2

Custome Node https://github.com/kijai/ComfyUI-segment-anything-2

Model To Download : https://huggingface.co/Kijai/sam2-safetensors/tree/main

Save to ComfyUI/models/sam2

Files

Comments

Brian G

Hi Benji! Thanks for providing this workflow. I have been diving into it, but I am having trouble getting the SAM2 model to mask the shirt in my footage correctly. It's a two tone light pink floral shirt with a heavy pink collar and button fabric. I enter the keywords, and Florence does the correct bounding box and identifies the entire shirt, but when it goes through SAM2 and creates the mask, it will just mask the collar and button fabric, ignoring the light pink floral pattern of the rest of the blouse. Any suggestions on what I might be doing wrong? Thanks in advance!

Brian G

Thank you!