Model Card for MineDreamer 🔥

MineDreamer is an instructable embodied agent for simulated control and it is developed on top of recent advances in Multimodal Large Language Models (MLLMs) and diffusion models!

MineDreamer can follow instructions steadily by employing a Chain-of-Imagination (CoI) mechanism to envision the step-by-step process of executing instructions and translating imaginations into more precise visual prompts tailored to the current state; subsequently, it generates keyboard-and-mouse actions to efficiently achieve these imaginations,

This repo is used for hosting MineDreamer's Q-Former checkpoints, which are the training stage 1 checkpoints for Imaginator.

For more details or tutorials see https://github.com/Zhoues/MineDreamer.