README.md

---
license: apache-2.0
---

This repository stores the pre-trained weights for [Readout Guidance: Learning Control
from Diffusion Features](https://readout-guidance.github.io/). The code implementation of our method can be found at https://github.com/g-luo/readout-guidance.

# Readout Head Weights
The `weights/` folder contains the pre-trained weights of the readout heads, named according to the following convention:
```
readout_<base-model>_<task-type>_<head-type>
```

**Spatially Aligned Control**
- `readout_sdxl_spatial_pose.pt` 
- `readout_sdv15_spatial_pose.pt`
    - Readout head trained with [OpenPose](https://arxiv.org/abs/1812.08008) pose skeletons as supervision on [PascalVOC](http://host.robots.ox.ac.uk/pascal/VOC) images, filtered only to those containing people.
- `readout_sdxl_spatial_depth.pt`
- `readout_sdv15_spatial_depth.pt`
    - Readout head trained with [MiDaS](https://arxiv.org/abs/1907.01341) depth maps as supervision on [PascalVOC](http://host.robots.ox.ac.uk/pascal/VOC) images.
- `readout_sdxl_spatial_edge.pt`
- `readout_sdv15_spatial_edge.pt`
    - Readout head trained with [HED](https://arxiv.org/abs/1504.06375) edge detections as supervision on [PascalVOC](http://host.robots.ox.ac.uk/pascal/VOC) images.


**Drag-Based Manipulation**
- `readout_sdxl_drag_correspondence.pt`
- `readout_sdv15_drag_correspondence.pt`
    - Readout head trained with a contrastive loss with [CoTracker](https://co-tracker.github.io) point tracks across pairs of [DAVIS](https://davischallenge.org) video frames.
- `readout_sdxl_drag_appearance.pt`
- `readout_sdv15_drag_appearance.pt`
    - Readout head trained with a triplet loss with real frames as positives and [SDEdit-ed](https://sde-image-editing.github.io) frames as negatives derived from [DAVIS](https://davischallenge.org) videos.