Dominik Klotz's picture

1

Dominik Klotz

programmnix-askui

·

programminx-askui

AI & ML interests

Working on our vision of a better digital world. Prompt-to-Automation. VGQ, Object Detection, Text Detection, Icon Classification, ... Vision ❤️

Recent Activity

Reacted to maxiw's post with 🚀 2 days ago

You can now try out computer use models from the hub to automate your local machine with https://github.com/askui/vision-agent. 💻 ``` import time from askui import VisionAgent with VisionAgent() as agent: agent.tools.webbrowser.open_new("http://www.google.com") time.sleep(0.5) agent.click("search field in the center of the screen", model_name="Qwen/Qwen2-VL-7B-Instruct") agent.type("cats") agent.keyboard("enter") time.sleep(0.5) agent.click("text 'Images'", model_name="AskUI/PTA-1") time.sleep(0.5) agent.click("second cat image", model_name="OS-Copilot/OS-Atlas-Base-7B") ``` Currently these models are integrated with Gradio Spaces API. Also planning to add local inference soon! Currently supported: - https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct - https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct - https://huggingface.co/AskUI/PTA-1 - https://huggingface.co/OS-Copilot/OS-Atlas-Base-7B

Reacted to maxiw's post with 🚀 8 days ago

🤖 Controlling Computers with Small Models 🤖 We just released PTA-1, a fine-tuned Florence-2 for localization of GUI text and elements. It runs with ~150ms inference time on a RTX 4080. This means you can now start building fast on-device computer use agents! Model: https://huggingface.co/AskUI/PTA-1 Demo: https://huggingface.co/spaces/AskUI/PTA-1

Reacted to maxiw's post with 👍 14 days ago

Exciting to see open-source models thriving in the computer agent space! 🔥 I just built a demo for OS-ATLAS: A Foundation Action Model For Generalist GUI Agents — check it out here: https://huggingface.co/spaces/maxiw/OS-ATLAS This demo predicts bounding boxes based on screenshot + instructions as input.

View all activity

Organizations

models

None public yet

datasets

None public yet