jakedahn/flux-latentpop

Flux latentpop

flux-latentpop features vibrant backgrounds with grungy limited screenprinting color goodness.

It does great with t-shirt designs, general illustrations, and character portraits.

Prompt

screenprint tshirt design, a happy cat holding a sign that says "I LOVE VE REPLICATE", LNTP illustration style

Prompt

a t-shirt, LNTP illustration style

Prompt

a young girl playing piano, yellow background, LNTP illustration style

Prompt

a book with the words 'Don't Panic!' written on cover, an homage to the hitchhikers guide to the galaxy, LNTP cartoon style

Prompt

a robot, blue background, LNTP illustration style

Prompt

girl, orange background, LNTP illustration style

It was trained on Replicate, here: https://replicate.com/ostris/flux-dev-lora-trainer/train

The training set is comprised of 23 images generated on MidJourney using the --sref 3102110963 and --personalize 3xdy3qw flags. You can find the entire training set here in this repo: ./2024-08-24-latentpop.zip

Below are the training parameters I used, which seem to work fairly well for illustration/cartoony Flux LoRAs:

{
  "steps": 1300,
  "lora_rank": 24,
  "optimizer": "adamw8bit",
  "batch_size": 4,
  "resolution": "512,768,1024",
  "autocaption": true,
  "input_images": "https://replicate.delivery/pbxt/Lg3C1KUPfrRZZvJFaaSTmQ9qtAyXSonLvLSuTuj4Nop9vcSu/2024-08-24-latentpop.zip",
  "trigger_word": "LNTP",
  "learning_rate": 0.0002,
  "autocaption_suffix": "LNTP style",
  "caption_dropout_rate": 0.05,
}

Shoutout to @ciguleva on x who originally shared this sref on x: https://x.com/ciguleva/status/1827398343779098720

Usage

You should use LNTP to trigger the image generation. The output images look more stylistically interesting with a guidance_scale of ~`2.5`.

Use it with the 🧨 diffusers library

from diffusers import AutoPipelineForText2Image
import torch

pipeline = AutoPipelineForText2Image.from_pretrained('black-forest-labs/FLUX.1-dev', torch_dtype=torch.float16).to('cuda')
pipeline.load_lora_weights('jakedahn/flux-latentpop', weight_name='lora.safetensors')
image = pipeline('your prompt').images[0]

For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers