AI & Machine Learning
★ 42.6k
Python
exo-explore/exo
42.6k
Stars
2.9k
Forks
108
Issues
Python
Language
exo lets you run frontier AI models locally by connecting multiple devices into a distributed AI cluster. Automatically discovers devices on your network and distributes large language models across them using tensor parallelism. Supports RDMA over Thunderbolt for reduced latency, provides API compatibility with OpenAI, Claude, and Ollama formats, and includes a built-in web dashboard. Performance scales significantly as more devices join the cluster, making massive AI models accessible without expensive GPU servers.
View on GitHub
git clone https://github.com/exo-explore/exo.git
Quick Start Example
bash
# Install exo
pip install exo
# Start a cluster node
exo run
# On other devices, they auto-discover
# and join the cluster
# Use OpenAI-compatible API
curl http://localhost:52415/v1/chat/completions \
-d '{"model":"llama-3.1-70b","messages":[{"role":"user","content":"Hello"}]}'