BFL, the team behind the Flux model, has released a few tools to accompany their model. These tools are available via API on BFL’s website, FAL & Replicate, and are also available for ComfyUI, on which we did our tests.
We set out to find out how good these tools are for gaming use cases, and of course, I’ll write here exactly what you need to know if you would like to test it yourself on your own local ComfyUI.
Fill (Outpaint)
Oh, the joy of adding margins to images, plus a second joy—not needing to write a prompt for it.
How it works
Easy. Upload an image, decide which of the four possible directions you would like to add (can be all four directions altogether), smash the enter button, and voilà.
Does a tremendous job on real images and illustrations alike. A few examples:
How to use it on ComfyUI
The file you need: flux1-fill-dev.safetensors (23.8GB)
Get it from: https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev
Put the file here: ComfyUI/models/diffusion_models/
Don't miss the link to the workflows at the end of this post!
Fill (Inpaint)
We didn’t want to admit it, but inpainting on Flux was lame. And boy, we tried. Needless to say, we were thrilled to get the latest inpaint model and yes, finally, it does a good job.
How it works
Upload an image, mask the area you want to change, remove from or add to, write a simple prompt (few words describing the item), and here it is. We were glad to find that the inpainted part kept the style and adhered to the lighting in the scene.
A few examples because we care:
How to use it on ComfyUI
It's actually the same model that does the outpainting, so scroll up and get it.Don't miss the link to the workflows at the end of this post!
ControlNet - Canny
About a week after Flux's inception, a first ControlNet popped up by a company (a group? A team? A bunch of renegades?) called Xlabs. While sending virtual hugs for trying, we weren’t swept off our feet by the results. So, again, I personally was psyched to hear that BFL have released their own ControlNet models, and that they weigh tons! (24GB, which, of course, is a seal of excellence when it comes to AI models.)
In reality, I have to admit that my expectations were too high. While staying true to the sketch, it seems like ControlNet degrades the quality of the overall image.
See for yourself:
How to use it on ComfyUI
The file you need: flux1-canny-dev.safetensors (23.8GB)
Get it from: https://huggingface.co/black-forest-labs/FLUX.1-Canny-dev
Put it in: ComfyUI/models/diffusion_models/
There's also a smaller LoRA version of this ControlNet model, which allows us to adjust the ControlNet's strength.
ControlNet - Depth
Same story. Huge file (wow!), and results that do the job but we have become hard to satisfy. Really. We are awful people.
Here are some examples:
How to use it on ComfyUI
The file you need: flux1-depth-dev.safetensors (23.8GB)
Get it here: https://huggingface.co/black-forest-labs/FLUX.1-Depth-dev
Put it there: ComfyUI/models/diffusion_models/
And yes, there's also a smaller LoRA version.
Flux Redux
That's a different name for IPAdapter. In other words: style reference or character reference. A very needed adapter indeed; we were blind but now we see. And... it's okay.
The first thing we tried is to get an image of a character without training a LoRA on that character. As you can see, the results would not pass a gaming studio.
However, we did find out that if the strength of the model is reduced to minimum (0.05–0.2), the model is able to get the overall gist of the original image, like materials and colors, and do a quick style transfer. Like in this example, where I uploaded King Robert's image but asked for a treasure chest. Not bad!!
One could also merge 2 or more images together, and let AI decide on the output. Frankly, either than fun I couldn't identify a real use case with this feature, but hey, it's a lot of fun. We need fun these days.
In conclusion
Flux's new generative fill model is a straight A. Perfect, couldn’t ask for more. ControlNets: They do the job, but still not SDXL quality. Redux/IPAdapter: Shows potential. We need to investigate more.
Here are the workflows! No Strings Attached! Yay!!!
Commenti