|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
This repository stores the pre-trained weights for [Readout Guidance: Learning Control |
|
from Diffusion Features](https://readout-guidance.github.io/). The code implementation of our method can be found at https://github.com/google-research/readout_guidance. |
|
|
|
# Readout Head Weights |
|
The `weights/` folder contains the pre-trained weights of the readout heads, named according to the following convention: |
|
``` |
|
readout_<base-model>_<task-type>_<head-type> |
|
``` |
|
|
|
**Spatially Aligned Control** |
|
- `readout_sdxl_spatial_pose.pt` |
|
- `readout_sdv15_spatial_pose.pt` |
|
- Readout head trained with [OpenPose](https://arxiv.org/abs/1812.08008) pose skeletons as supervision on [PascalVOC](http://host.robots.ox.ac.uk/pascal/VOC) images, filtered only to those containing people. |
|
- `readout_sdxl_spatial_depth.pt` |
|
- `readout_sdv15_spatial_depth.pt` |
|
- Readout head trained with [MiDaS](https://arxiv.org/abs/1907.01341) depth maps as supervision on [PascalVOC](http://host.robots.ox.ac.uk/pascal/VOC) images. |
|
- `readout_sdxl_spatial_edge.pt` |
|
- `readout_sdv15_spatial_edge.pt` |
|
- Readout head trained with [HED](https://arxiv.org/abs/1504.06375) edge detections as supervision on [PascalVOC](http://host.robots.ox.ac.uk/pascal/VOC) images. |
|
|
|
|
|
**Drag-Based Manipulation** |
|
- `readout_sdxl_drag_correspondence.pt` |
|
- `readout_sdv15_drag_correspondence.pt` |
|
- Readout head trained with a contrastive loss with [CoTracker](https://co-tracker.github.io) point tracks across pairs of [DAVIS](https://davischallenge.org) video frames. |
|
- `readout_sdxl_drag_appearance.pt` |
|
- `readout_sdv15_drag_appearance.pt` |
|
- Readout head trained with a triplet loss with real frames as positives and [SDEdit-ed](https://sde-image-editing.github.io) frames as negatives derived from [DAVIS](https://davischallenge.org) videos. |
