Edit model card

âš¡ Flux.1-dev: Depth ControlNet âš¡

This is Flux.1-dev ControlNet for Depth map developed by Jasper research team.

How to use

This model can be used directly with the diffusers library

import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetModel
from diffusers.pipelines import FluxControlNetPipeline

# Load pipeline
controlnet = FluxControlNetModel.from_pretrained(
  "jasperai/Flux.1-dev-Controlnet-Depth",
  torch_dtype=torch.bfloat16
)
pipe = FluxControlNetPipeline.from_pretrained(
  "black-forest-labs/FLUX.1-dev",
  controlnet=controlnet,
  torch_dtype=torch.bfloat16
)
pipe.to("cuda")

# Load a control image
control_image = load_image(
  "https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Depth/resolve/main/examples/depth.jpg"
)

prompt = "a statue of a gnome in a field of purple tulips"

image = pipe(
    prompt, 
    control_image=control_image,
    controlnet_conditioning_scale=0.6,
    num_inference_steps=28, 
    guidance_scale=3.5,
    height=control_image.size[1],
    width=control_image.size[0]
).images[0]
image

💡 Note: You can compute the conditioning map using for instance the MidasDetector from the controlnet_aux library

from controlnet_aux import MidasDetector
from diffusers.utils import load_image 

midas = MidasDetector.from_pretrained("lllyasviel/Annotators")

midas.to("cuda")

# Load an image
im = load_image(
  "https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Depth/resolve/main/examples/output.jpg"
)

depth = midas(im)

Training

This model was trained with depth maps computed with Clipdrop's depth estimator model as well as open-souce depth estimation models such as Midas or Leres.

Licence

This model falls under the Flux.1-dev licence.

Downloads last month
3,462
Inference Examples
Inference API (serverless) has been turned off for this model.

Model tree for jasperai/Flux.1-dev-Controlnet-Depth

Finetuned
(263)
this model

Space using jasperai/Flux.1-dev-Controlnet-Depth 1

Collection including jasperai/Flux.1-dev-Controlnet-Depth