OnlyAgents — A Runtime for AI Agents
Agents are treated as long-running, stateful systems — with their own memory, capabilities, and coordination logic — and provides the runtime needed to operate them reliably.
Think of it less like a library and more like an operating system for AI agents.
The Core Idea
At the center of OnlyAgents is a simple model:
- One executive agent that understands intent, plans, and orchestrates
- A set of specialized sub-agents with constrained capabilities
- A kernel that routes all communication through an event-driven system
Instead of one model doing everything, the system behaves like a small, coordinated team.
Architecture (High-Level)
-
Executive Agent Plans tasks, decomposes requests, delegates work, and synthesizes results.
-
Sub-Agents Focused, capability-bounded agents (e.g. productivity, research, devops). Each can use different models depending on cost and complexity.
-
Kernel (Runtime) Handles agent lifecycle, routing, concurrency, and the event bus. No direct agent-to-agent calls — everything flows through the kernel.
-
Skills Define what agents can do (calendar, tasks, CLI tools, web search).
-
Connectors Define where data comes from (local storage, APIs like Gmail, Notion, etc).
This separation keeps the system modular and predictable.
Memory System
One of the more interesting parts of the project is how memory is structured.
Instead of just storing chat history, OnlyAgents builds layered memory:
- Episodes — semantic summaries of past sessions (vector search)
- Nexus — a lightweight knowledge graph of entities and relationships
- Praxis — behavioral patterns extracted over time
This allows agents to recall (and remember):
- what happened
- what matters
- and how the user tends to work
—not just what was said.
Why Go + Single Binary
Most agent systems come with heavy runtimes and complex setup.
OnlyAgents compiles to a single statically linked binary with:
- embedded web UI
- SQLite storage
- in-process scheduler
No external services required.
You can:
scp onlyagents <server>
./onlyagents start…and have a full multi-agent system running.
What Makes It Different
- Event-driven architecture instead of synchronous tool chains
- Multi-agent by design, not simulated within a single prompt
- Local-first — runs entirely on your machine or VPS
- Composable system — agents, skills, connectors can evolve independently
- No dependency hell — one binary, minimal setup
Status
Still actively evolving.
Current focus areas:
- refining the execution model and workflow engine
- improving memory extraction and retrieval
- stabilizing agent-to-agent communication (A2A protocol)
Why I Built It
I’ve wanted to build something like this for a while.
A couple of years ago, I experimented with similar ideas in Pragyantra but that was more of a research project, exploring how software systems might coordinate for something like humanoid robotics. It never really became a usable system.
More recently, with how quickly LLMs have evolved and the rise of agent-based systems (like OpenClaw), the idea started to feel a lot more practical.
Instead of treating agents as prompt chains or one-off tools, I wanted to explore what it looks like to build them as long-running systems with the an holistic architecture that I’ve discussed above.