Spaces:
Configuration error
Configuration error
| <html> | |
| <head> | |
| <title>WebSD | Home</title> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> | |
| <link rel="stylesheet" | |
| href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta.2/css/bootstrap.min.css" | |
| integrity="sha384-PsH8R72JQ3SOdhVi3uxftmaW6Vc51MKb0q5P2rRUpPvrszuE4W1povHYgTpBfshb" | |
| crossorigin="anonymous"> | |
| <link rel="stylesheet" | |
| href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css"> | |
| <link rel="stylesheet" href="/assets/css/main.css"> | |
| <link rel="stylesheet" href="/assets/css/group.css"> | |
| <!-- <link rel="stylesheet" href="/css/table.css"> --> | |
| <link rel="shortcut icon" href="/assets/img/logo/mlc-favicon.png"> | |
| <meta http-equiv="origin-trial" content="Agx76XA0ITxMPF0Z8rbbcMllwuxsyp9qdtQaXlLqu1JUrdHB6FPonuyIKJ3CsBREUkeioJck4nn3KO0c0kkwqAMAAABJeyJvcmlnaW4iOiJodHRwOi8vbG9jYWxob3N0Ojg4ODgiLCJmZWF0dXJlIjoiV2ViR1BVIiwiZXhwaXJ5IjoxNjkxNzExOTk5fQ=="> | |
| <meta http-equiv="origin-trial" content="AnmwqQ1dtYDQTYkZ5iMtHdINCaxjE94uWQBKp2yOz1wPTcjSRtOHUGQG+r2BxsEuM0qhxTVnuTjyh31HgTeA8gsAAABZeyJvcmlnaW4iOiJodHRwczovL21sYy5haTo0NDMiLCJmZWF0dXJlIjoiV2ViR1BVIiwiZXhwaXJ5IjoxNjkxNzExOTk5LCJpc1N1YmRvbWFpbiI6dHJ1ZX0="> | |
| <script src="dist/tvmjs_runtime.wasi.js"></script> | |
| <script src="dist/tvmjs.bundle.js"></script> | |
| </head> | |
| <body> | |
| <div class="container"> | |
| <!-- This is a bit nasty, but it basically says be a column first, and on larger screens be a spaced out row --> | |
| <div class="header d-flex | |
| flex-column | |
| flex-md-row justify-content-md-between"> | |
| <a href="/" id="navtitle"> | |
| <img src="/assets/img/logo/mlc-logo-with-text-landscape.svg" height="70px" | |
| alt="MLC" id="logo"> | |
| </a> | |
| <ul id="topbar" class="nav nav-pills justify-content-center"> | |
| <li class="nav-item"> | |
| <a class="nav-link active" | |
| href="/"> | |
| Home | |
| </a> | |
| </li> | |
| <li class="nav-item"> | |
| <a class="nav-link " | |
| href="https://github.com/mlc-ai/web-stable-diffusion"> | |
| Github | |
| </a> | |
| </li> | |
| </ul> | |
| </div> | |
| <!-- Schedule --> | |
| <h1 id="web-stable-diffusion">Web Stable Diffusion</h1> | |
| <p>This project brings stable diffusion models to web browsers. <strong>Everything runs inside the browser with no need of server support.</strong> To our knowledge, this is the the world’s first stable diffusion completely running on the browser. Please check out our <a href="https://github.com/mlc-ai/web-stable-diffusion">GitHub repo</a> to see how we did it. There is also a <a href="#text-to-image-generation-demo">demo</a> which you can try out.</p> | |
| <p><img src="img/fig/browser-screenshot.png" alt="Browser screenshot" width="100%" /></p> | |
| <p>We have been seeing amazing progress through AI models recently. Thanks to the open-source effort, developers can now easily compose open-source models together to produce amazing tasks. Stable diffusion enables the automatic creation of photorealistic images as well as images in various styles based on text input. These models are usually big and compute-heavy, which means we have to pipe through all computation requests to (GPU) servers when developing web applications based on these models. Additionally, most of the workloads have to run on a specific type of GPUs where popular deep-learning frameworks are readily available.</p> | |
| <p>This project takes a step to change that status quo and bring more diversity to the ecosystem. There are a lot of reasons to get some (or all) of the computation to the client side. There are many possible benefits, such as cost reduction on the service provider side, as well as an enhancement for personalization and privacy protection. The development of personal computers (even mobile devices) is going in the direction that enables such possibilities. The client side is getting pretty powerful. For example, the latest MacBook Pro can have up to 96GB of unified RAM that can be used to store the model weights and a reasonably powerful GPU to run many of the workloads.</p> | |
| <p>Wouldn’t it be fun to directly bring the ML models to the client, have the user open a browser tab, and instantly run the stable diffusion models on the browser? This project provides the first affirmative answer to this question.</p> | |
| <h2 id="text-to-image-generation-demo">Text to Image Generation Demo</h2> | |
| <p>Because WebGPU is not yet fully stable, nor have there ever been such large-scale AI models running on top of WebGPU, so we are testing the limit here. It may not work in your environment. So far, we have only tested it on Mac with M1/M2 GPUs in Chrome Canary (a nightly build of Chrome) because WebGPU is quite new. We have tested on Windows and it does not work at this moment due to possible driver issues. We anticipate the support broadens as WebGPU matures. Please check out the <a href="#instructions">use instructions</a> and <a href="#notes">notes</a> below.</p> | |
| <h3 id="instructions">Instructions</h3> | |
| <p>If you have a Mac computer with Apple silicon, here are the instructions for you to run stable diffusion on your browser locally:</p> | |
| <ul> | |
| <li>Install <a href="https://www.google.com/chrome/canary/">Chrome Canary</a>, a developer version of Chrome that enables the use of WebGPU.</li> | |
| <li>Launch Chrome Canary. <strong>You are recommended to launch from terminal with the following command:</strong> | |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/Applications/Google<span class="se">\ </span>Chrome<span class="se">\ </span>Canary.app/Contents/MacOS/Google<span class="se">\ </span>Chrome<span class="se">\ </span>Canary <span class="nt">--enable-dawn-features</span><span class="o">=</span>disable_robustness | |
| </code></pre></div> </div> | |
| <p>This command turns off the robustness check from Chrome Canary that slows down image generation to times. It is not necessary, but we strongly recommend you to start Chrome with this command.</p> | |
| </li> | |
| <li>Enter your prompt, click “Generate” – we are ready to go! The image generation will start after downloading and fetching the model parameters to local cache. The download may take a few minutes, only for the first run. The subsequent refreshes and runs will be faster.</li> | |
| <li>Feel free to enter different prompts as well as negative prompts to generate the image you want.</li> | |
| <li>We provide an option to render images for the intermediate steps of UNet stage. Select “Run VAE every two UNet steps after step 10” for “Render intermediate steps” and click “Generate” again, and you will see how an image gets generated along the process.</li> | |
| </ul> | |
| <h3 id="demo">Demo</h3> | |
| <script> | |
| var tvmjsGlobalEnv = tvmjsGlobalEnv || {}; | |
| </script> | |
| <script type="module"> | |
| import init, { TokenizerWasm } from "./dist/tokenizers-wasm/tokenizers_wasm.js"; | |
| var initialized = false; | |
| async function getTokenizer(name) { | |
| if (!initialized) { | |
| await init(); | |
| } | |
| const jsonText = await (await fetch("https://huggingface.co/" + name + "/raw/main/tokenizer.json")).text(); | |
| return new TokenizerWasm(jsonText); | |
| } | |
| tvmjsGlobalEnv.getTokenizer = getTokenizer; | |
| </script> | |
| <script src="dist/stable_diffusion.js"></script> | |
| <div> | |
| Input prompt: <input name="inputPrompt" id="inputPrompt" type="text" value="A photo of an astronaut riding a horse on mars" size="77" /> <br /> | |
| Negative prompt (optional): <input name="negativePrompt" id="negativePrompt" type="text" value="" size="77" /> | |
| </div> | |
| <div> | |
| Select scheduler - | |
| <select name="scheduler" id="schedulerId"> | |
| <option value="0">Multi-step DPM Solver (20 steps)</option> | |
| <option value="1">PNDM (50 steps)</option> | |
| </select> | |
| <br /> | |
| Render intermediate steps (may slow down execution) - | |
| <select name="vae-cycle" id="vaeCycle"> | |
| <option value="-1">No</option> | |
| <option value="2">Run VAE every two UNet steps after step 10</option> | |
| </select> | |
| <div id="progress"> | |
| <label id="gpu-tracker-label"></label><br /> | |
| <label id="progress-tracker-label"></label><br /> | |
| <progress id="progress-tracker-progress" max="100" value="100"> </progress> | |
| </div> | |
| <button onclick="tvmjsGlobalEnv.asyncOnGenerate()">Generate</button> | |
| </div> | |
| <div> | |
| <canvas id="canvas" width="512" height="512"></canvas> | |
| </div> | |
| <div id="log"></div> | |
| <h3 id="notes">Notes</h3> | |
| <ul> | |
| <li>WebGPU spec does comes with FP16 support already, but the implementation does not yet support this feature at this moment. As a result, the memory consumption of running the demo is about 7GB. For Apple silicon Mac with only 8GB of unified memory, it may take longer (a few minutes) to generate an image. This demo may also work for Mac with AMD GPU.</li> | |
| <li>Please check out our <a href="https://github.com/mlc-ai/web-stable-diffusion">GitHub repo</a> for running the same shader flow locally on your GPU device through the native driver. Right now, there are still gaps (e.g., without launching Chrome from command line, Chrome’s WebGPU implementation inserts bound clips for all array index access, such that <code class="language-plaintext highlighter-rouge">a[i]</code> becomes <code class="language-plaintext highlighter-rouge">a[min(i, a.size)]</code>, which are not optimized out by the downstream shader compilers), but we believe it is feasible to close such gaps as WebGPU dispatches to these native drivers.</li> | |
| </ul> | |
| <h2 id="disclaimer">Disclaimer</h2> | |
| <p>This demo site is for research purposes only. Please conform to the <a href="https://huggingface.co/runwayml/stable-diffusion-v1-5#uses">uses of stable diffusion models</a>.</p> | |
| </div> <!-- /container --> | |
| <!-- Support retina images. --> | |
| <script type="text/javascript" | |
| src="/assets/js/srcset-polyfill.js"></script> | |
| </body> | |
| </html> | |