Model Card for MineDreamer 🔥
MineDreamer is an instructable embodied agent for simulated control and it is developed on top of recent advances in Multimodal Large Language Models (MLLMs) and diffusion models!
MineDreamer can follow instructions steadily by employing a Chain-of-Imagination (CoI) mechanism to envision the step-by-step process of executing instructions and translating imaginations into more precise visual prompts tailored to the current state; subsequently, it generates keyboard-and-mouse actions to efficiently achieve these imaginations,
This repo is used for hosting MineDreamer's InstructPix2Pix checkpoints, which are not only the baseline checkpoints but the training stage 2 checkpoints for Imaginator as well.
For more details or tutorials see https://github.com/Zhoues/MineDreamer.