The AI Fleet Architect: Why We Killed Our Own RAG Server and Moved Memory Into SQLite

1. Bug One: Unbounded Document Growth

Our old RAG server’s document count had grown into the tens of thousands, almost entirely duplicates. Root cause: our own state-writing helpers — the functions every agent calls to persist a heartbeat, a health file, a routine status update — were also pushing that same write into the RAG server as a brand-new document, every single time, with no replace-on-save. A heartbeat that fires every few minutes for months turns into tens of thousands of near-identical documents, all competing for the same search relevance the real, useful memory (incident postmortems, architectural decisions) needed.

The fix wasn’t a smarter prune job — it was stopping the bleeding at the write path with a real dedupe key and a rolling window:

_STATE_KEEP = 5

def remember_state(scope: str, event: str, detail: str = "",
                    status: str = "info", **meta) -> bool:
    """Standardized operational-state write into the local RAG store.
    Same (scope, event) pair always dedupes to the same rolling
    window -- a routine heartbeat no longer creates a new document
    every time it fires."""
    return remember(
        scope=scope,
        title=f"{scope} {event}",
        content=f"Agent: {scope}\nEvent: {event}\nStatus: {status}\nDetail: {detail}",
        dedupe_key=f"state:{scope}:{event}",
        keep=_STATE_KEEP,
        **meta,
    )

Every routine write now keeps only the most recent 5 entries per (scope, event) pair instead of growing forever. Real incidents and architectural decisions — the content that’s actually worth searching — still get their own dedicated, durable entries.

2. Bug Two: A Health Check That Couldn't Tell 'Alive' From 'Hung'

The scarier find: the RAG server’s underlying local database engine could hang completely — stop responding to any real request — while its network port stayed open and kept accepting TCP connections. Our existing health check only verified the port was open, so a fully hung service read as healthy on every check. The only way to catch it was a real authenticated request against an actual API endpoint, checking that a genuine response came back, not just that something picked up the socket.

def check_service_actually_alive(base_url: str, timeout: float = 5.0) -> bool:
    """A port being open only proves something is listening -- not that
    it's answering. This hits a real authenticated endpoint and checks
    for a real response instead of trusting the TCP handshake."""
    try:
        resp = requests.get(f"{base_url}/api/v1/auth", timeout=timeout,
                             headers={"Authorization": f"Bearer {get_local_token()}"})
        return resp.status_code == 200
    except Exception:
        return False

On a confirmed hang, our watchdog now kills and relaunches the process automatically instead of quietly reporting green.

3. The Bigger Decision: Delete the Server, Not Just Patch It

Both bugs traced back to the same root cause: a separate, always-on HTTP service with its own database engine, its own process-hang failure class, and its own health-check surface to get wrong. We already run a local SQLite database (WAL mode) for the rest of the fleet’s state. SQLite’s FTS5 extension gives full-text search natively, in-process, with zero network hop and zero separate process to hang.

# bytesize_core/rag_bus.py -- local-first memory, no external server
from bytesize_core.db import ingest_rag_document, query_rag_documents

def remember(scope: str, title: str, content: str,
             dedupe_key: str = None, keep: int = 1, **meta) -> bool:
    """Every agent's memory write goes straight into the shared
    SQLite FTS5 store -- no HTTP call, no separate process that
    can hang independently of the fleet itself."""
    return ingest_rag_document(
        title=title, content=content,
        dedupe_key=dedupe_key, keep=keep, metadata=meta,
    )

We fully decommissioned the external RAG server on 2026-08-12. Every agent’s `remember()`/`recall()` call now reads and writes the same local `bytesize.db` file every other stage of the pipeline already depends on. One fewer network hop, one fewer process that can silently hang, one fewer health check that can lie to us.

Summary

The instinct when a service misbehaves is to make the health check smarter. Sometimes the actual fix is removing the service. We had already built the dedupe/rolling-window fix and the real-auth health probe before we made that call — both were the right fixes for the system as it existed. But once we’d fixed both, the honest question was whether we needed a second database engine and a second process at all, and for us the answer was no.

Key Takeaways for Builders

• A health check that only verifies a port is open cannot tell you a process has hung — verify with a real authenticated request against a real endpoint, not just a TCP handshake.
• Give every ‘routine write’ a real dedupe key and a rolling-window cap (keep=N) at the write path, not a cleanup job after the fact — unbounded growth from heartbeats and routine state writes is the most common way a memory/RAG store silently bloats.
• When a bug traces back to ‘a separate always-on service with its own failure class,’ ask whether you need the separate service at all before you patch its health check — sometimes the fix is one fewer moving part.