BlackForestLabs has just released some new controlnet-style models for FLUX.1, addressing a notable gap in the model’s toolkit.
Advertisement
Flux.1 has basically become the go-to free local image generation model for most of us due to it’s incredible quality that matches and even outperforms the paid options in many cases. With the addition of these models, we should be seeing some even better output than we’ve been getting (hopefully).
Here’s a breakdown of the new FLUX.1 Tools suite and what they bring to the table.
What’s New in FLUX.1 Tools?
The release includes four major features, each available in both open-access [dev] versions and through their professional API:
FLUX.1 Fill
This looks to be a pretty impressive inpainting and outpainting implementation, with benchmarks showing it performing well against the current best options.
Inpainting and outpainting is used for the usual editing tasks – removing objects, extending images, and making controlled modifications to existing images. A welcome addition that I am excited to test out.
You can download the Flux Fill model from Huggingface here.
FLUX.1 Depth & Canny Lora Models
These are the ControlNet-style models we’ve been waiting for. Both are 12B parameter models that have been distilled down to LoRA weights for the dev version:
- Depth: Uses depth maps for structural guidance. Available as both a full model and a LoRA that can be applied to FLUX.1 [dev]. Download the Lora model from Huggingface here
- Canny: Same concept but with canny edge detection. Like Depth, it comes in both full model and LoRA formats. Download the Lora model from Huggingface here
Depth and canny are very commonly used controlnet models and so it is nice to see these two being released.
Both models do what you’d expect if you’re familiar with ControlNet – they help maintain image structure during transformations. If you want a certain image composition for your generation, these are basically essential.
FLUX.1 Redux
This model is an adapter for image variations and restyling. Works with all FLUX.1 base models and can be used for both simple variations and text-guided restyling. When used with FLUX1.1 [pro] Ultra, it supports 4MP outputs with flexible aspect ratios.
Think of this as the “remix” option that you can find in other image generators like Ideogram. If you are a local image generation kind of person, this sounds pretty similar to an implementation of Ipadapter.
You can download the Flux Redux model from Huggingface here.
Sample Results From The New Models
I’ll be testing each of these models out today and share my results in our next blog post, but for now here are some of the sample results from Blackforestlabs themselves.
Flux Fill Model
Flux Canny Model
Flux Depth Model
Flux Redux Model
Accessibility and Availability
All tools are available for free as FLUX.1 [dev] models with full weights and inference code on Hugging Face and GitHub under the Flux Dev License.
Professional API: Enhanced [pro] versions are accessible through the BFL API.
Additionally, these models will be available through their cloud-based generation partners including fal.ai, Replicate, Freepik, krea.ai, etc. making them widely accessible to different user groups.
You ca also use a service like Runpod.io if you want to rent a more powerful GPU to run this model yourself.
Why This Matters
Let’s face it, the current controlnet models for Flux just aren’t cutting it. Many of us who use Flux for image generation frequently have been eagerly waiting for some decent models to be released for a while now, and from early impressions, it looks like these models are quite good.
For more details about FLUX.1 Tools and their technical specifications, you can read the full announcement from BlackForestLabs.
So what do you think of these new models? Let me know in the comments below!
Hi, I’m Carmine, the guy behind Helpful Tiger. Tired of crappy search results? Me too. That’s why I started this website. It’s a one-stop shop for simple tutorials and guides on a little bit of everything. Have questions? Reach out, and I’ll do my best to help!