airunner

Sun, 18 May 2025 15:27:03 +0800

Capsize-Games/airunner

AI Runner: Offline AI Inference Engine for Hackers, Makers, and Builders.

Here are some of the things you can do with AI Runner:

✅ Voice-based chatbots for real-time conversations (three text-to-speech engines each with multiple voices to choose from)
✅ Customizable agents with names, moods and personalities for more engaging conversations
✅ Retrieval-Augmented Generation (RAG) for documents and websites
✅ Text-to-Image generation with Stable Diffusion and ControlNet
✅ Image manipulation with inpainting, outpainting, ControlNet and Image filters

For extra security, performance, and compatibility, AI Runner is built with Wayland support, Python 3.13, and the latest stable torch libraries.

System Requirements

Specification	Minimum	Recommended
OS	Ubuntu 22.04, Windows 10	Ubuntu 22.04 (Wayland)
CPU	Ryzen 2700K or Intel Core i7-8700K	Ryzen 5800X or Intel Core i7-11700K
Memory	16 GB RAM	32 GB RAM
GPU	NVIDIA RTX 3060 or better	NVIDIA RTX 4090 or better
Network	Broadband (used to download models)	Broadband (used to download models)
Storage	22 GB	50 GB

💾 Installation Quick Start

🔧 Installation Steps

Install system requirements

sudo apt update && sudo apt upgrade -y
sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python3-openssl git nvidia-cuda-toolkit pipewire libportaudio2 libxcb-cursor0 gnupg gpg-agent pinentry-curses espeak xclip cmake qt6-qpa-plugins qt6-wayland qt6-gtk-platformtheme mecab libmecab-dev mecab-ipadic-utf8
sudo apt install espeak
sudo apt install espeak-ng-espeak

Create airunner directory

1
2

sudo mkdir ~/.local/share/airunner
sudo chown $USER:$USER ~/.local/share/airunner

Install AI Runner - Python 3.13+ required pyenv and venv are recommended (see wiki for more info)

pip install "typing-extensions==4.13.2"
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
pip install airunner[all_dev]
pip install -U timm

Run AI Runner
1

airunner

For more options, including Docker, see the Installation Wiki.

Note: AI Runner does not distribute AI art models. You are responsible for obtaining and your own.

🛠️ Usage

Basic Usage

Run AI Runner: airunner
Build templates: airunner-build-ui

⭐ Features

Below is a high-level list of capabilities in AI Runner:

Feature	Description
LLMs & Communication
Voice-based chatbot conversations	Have real-time voice-chat sessions with an LLM (speech-to-text + text-to-speech)
Text-to-speech (TTS)	Convert text to spoken audio using OpenVoice, SpeechT5, and Espeak
Speech-to-text (STT)	Convert spoken audio to text with Whisper
Customizable chatbots	Create AI personalities and moods for more engaging conversations
Retrieval-Augmented Generation	Use local doc or website data to enrich chat responses
Image Generation
Stable Diffusion (1.5, SDXL, Turbo)	Generate images from textual prompts, sketches, or existing images
Drawing tools & ControlNet	Fine-tune image outputs with extra input or guides
LoRA & Embeddings	Load LoRA models or textual embeddings for specialized image generation
Image Manipulation
Inpaint & Outpaint	Modify portions of generated images while keeping context
Image filters	Blur, film grain, pixel art, etc.
Utility
Offline	Everything runs locally, no external API required
Fast generation	E.g., ~2 seconds on an RTX 2080s for stable diffusion
Docker-based approach	Simplifies setup & ensures GPU acceleration works out of the box
Dark mode	Built-in theming (Light / Dark / System)
NSFW toggles	Enable or disable NSFW detection for images
Ethical guardrails	Basic guardrails for safe LLM usage (optional)
Extensions	Build your own feature add-ons via the extension API
Python Library	`pip install airunner` and embed it in your own projects
API Support	Optionally use OpenRouter or other external LLMs

⚙️ System Requirements

System Requirements

Specification	Minimum	Recommended
OS	Ubuntu 22.04, Windows 10	Ubuntu 22.04 (Wayland)
CPU	Ryzen 2700K or Intel Core i7-8700K	Ryzen 5800X or Intel Core i7-11700K
Memory	16 GB RAM	32 GB RAM
GPU	NVIDIA RTX 3060 or better	NVIDIA RTX 4090 or better
Network	Broadband (used to download models)	Broadband (used to download models)
Storage	22 GB (with models), 6 GB (without models)	100 GB or higher

Models

These are the sizes of the various models that power AI Runner.

Model	Size
Controlnet (SD 1.5)	10.6 GB
Controlnet (SDXL)	320.2 MB
Safety Checker + Feature Extractor	3.2 GB
SD 1.5	1.6 MB
SDXL 1.0	6.45 MB
LLM	5.8 GB
e5 large (embedding model)	1.3 GB
Whisper Tiny	155.4 MB
Speech T5 (Voice)	654.4 MB
OpenVoice (Voice)	4.0 GB

AI Models

By default, AI Runner installs essential TTS/STT and minimal LLM components.
You must supply additional Stable Diffusion models (e.g., from Hugging Face or Civitai).

Organize them under your local AI Runner data directory:

~/.local/share/airunner
├── art
│   └── models
│       ├── SD 1.5
│       │   ├── lora
│       │   └── embeddings
│       ├── Flux
│       ├── SDXL 1.0
│       │   ├── lora
│       │   └── embeddings
│       └── SDXL Turbo
│           ├── lora
│           └── embeddings

Unit Tests

To run all tests:

`1`	`python -m unittest discover -s src/airunner/tests`

Or a single test:

`1`	`python -m unittest src/airunner/tests/test_prompt_weight_convert.py`

Database

AI Runner supports a simple database system. See the Wiki for how to:

Switch engines (SQLite, etc.)
Make schema changes
Run migrations

Advanced Features

Memory Optimization: TF32 Mode, VAE/Attention Slicing, Torch 2.0, sequential CPU offload, ToMe token merging.
Experimental Integrations: Weather-based chatbot prompts, advanced command-line arguments (--perform-llm-analysis, --disable-setup-wizard, etc.).
Safety & Guardrails: Optional NSFW content detection and adjustable guardrails for LLMs.

Contributing

We welcome pull requests for new features, bug fixes, or documentation improvements. You can also build and share extensions to expand AI Runner’s functionality. For details, see the Extensions Wiki.

Take a look at the Contributing document and the Development wiki page for detailed instructions.

Thank You!

Thanks for checking out AI Runner.
Get started with local AI inference in minutes—no more endless environment setup.
Questions or ideas? Join our Discord or open a GitHub Issue.

Happy building!

AI Runner on Producthunt daily

airunner

Capsize-Games/airunner

AI Runner: Offline AI Inference Engine for Hackers, Makers, and Builders.

System Requirements

💾 Installation Quick Start

🔧 Installation Steps

🛠️ Usage

Basic Usage

⭐ Features

⚙️ System Requirements

System Requirements

Models

AI Models

Unit Tests

Database

Advanced Features

Contributing

Thank You!