Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,55 @@
|
|
1 |
-
---
|
2 |
-
license: cc-by-4.0
|
3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: cc-by-4.0
|
3 |
+
---
|
4 |
+
|
5 |
+
![Intro Image](cosmicman_samples.png)
|
6 |
+
|
7 |
+
CosmicMan is a text-to-image foundation model specialized for generating high-fidelity human images. For more information, please refer to our research paper: [CosmicMan: A Text-to-Image Foundation Model for Humans](https://arxiv.org/abs/2404.01294). Our model is based on [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5). This repository provide UNet checkpoints for CosmicMan-SD.
|
8 |
+
|
9 |
+
|
10 |
+
## Requirements
|
11 |
+
|
12 |
+
```python
|
13 |
+
conda create -n cosmicman python=3.10
|
14 |
+
source activate cosmicman
|
15 |
+
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
|
16 |
+
pip install accelerate diffusers datasets transformers botocore invisible-watermark bitsandbytes gradio==3.48.0
|
17 |
+
```
|
18 |
+
|
19 |
+
|
20 |
+
## Inference
|
21 |
+
|
22 |
+
```python
|
23 |
+
import torch
|
24 |
+
from diffusers import StableDiffusionPipeline, UNet2DConditionModel, EulerDiscreteScheduler
|
25 |
+
from huggingface_hub import hf_hub_download
|
26 |
+
from safetensors.torch import load_file
|
27 |
+
|
28 |
+
base_path = "runwayml/stable-diffusion-v1-5"
|
29 |
+
unet_path = "cosmicman/CosmicMan-SD"
|
30 |
+
|
31 |
+
# Load model.
|
32 |
+
unet = UNet2DConditionModel.from_pretrained(unet_path, torch_dtype=torch.float16)
|
33 |
+
pipe = StableDiffusionPipeline.from_pretrained(base_path, unet=unet, torch_dtype=torch.float16, variant="fp16").to("cuda")
|
34 |
+
pipe.scheduler = EulerDiscreteScheduler.from_pretrained(base_path, subfolder="scheduler", torch_dtype=torch.float16)
|
35 |
+
|
36 |
+
|
37 |
+
# Generate image.
|
38 |
+
positive_prompt = "A closeup portrait shot against a white wall, a fit Caucasian adult female with wavy blonde hair falling above her chest wears a short sleeve silk floral dress and a floral silk normal short sleeve white blouse"
|
39 |
+
negative_prompt = ""
|
40 |
+
image = pipe(positive_prompt, num_inference_steps=30,
|
41 |
+
guidance_scale=7.5, height=1024,
|
42 |
+
width=1024, negative_prompt=negative_prompt, output_type="pil").images[0].save("output.png")
|
43 |
+
```
|
44 |
+
|
45 |
+
|
46 |
+
## Citation Information
|
47 |
+
```
|
48 |
+
@article{li2024cosmicman,
|
49 |
+
title={CosmicMan: A Text-to-Image Foundation Model for Humans},
|
50 |
+
author={Li, Shikai and Fu, Jianglin and Liu, Kaiyuan and Wang, Wentao and Lin, Kwan-Yee and Wu, Wayne},
|
51 |
+
journal={arXiv preprint arXiv:2404.01294},
|
52 |
+
year={2024}
|
53 |
+
}
|
54 |
+
|
55 |
+
```
|