Flux.1-dev ControlNets
Collection
A collection of ControlNet models for Flux.1-dev by Jasper Research
•
4 items
•
Updated
•
15
This is Flux.1-dev ControlNet for Surface Normals map developed by Jasper research team.
This model can be used directly with the diffusers
library
import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetModel
from diffusers.pipelines import FluxControlNetPipeline
# Load pipeline
controlnet = FluxControlNetModel.from_pretrained(
"jasperai/Flux.1-dev-Controlnet-Surface-Normals",
torch_dtype=torch.bfloat16
)
pipe = FluxControlNetPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev",
controlnet=controlnet,
torch_dtype=torch.bfloat16
)
pipe.to("cuda")
# Load a control image
control_image = load_image(
"https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Surface-Normals/resolve/main/examples/surface.jpg"
)
prompt = "a man showing stop sign in front of window"
image = pipe(
prompt,
control_image=control_image,
controlnet_conditioning_scale=0.6,
num_inference_steps=28,
guidance_scale=3.5,
height=control_image.size[1],
width=control_image.size[0]
).images[0]
image
💡 Note: You can compute the conditioning map using the NormalBaeDetector
from the controlnet_aux
library
from controlnet_aux import NormalBaeDetector
from diffusers.utils import load_image
normal_bae = NormalBaeDetector.from_pretrained("lllyasviel/Annotators")
normal_bae.to("cuda")
# Load an image
im = load_image(
"https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Surface-Normals/resolve/main/examples/output.jpg"
)
surface = normal_bae(im)
This model was trained with surface normals maps computed with Clipdrop's surface normals estimator model as well as an open-souce surface normals estimation model such as Boundary Aware Encoder (BAE).
This model falls under the Flux.1-dev model licence.
Base model
black-forest-labs/FLUX.1-dev