AI-Spirit — behind the scenes

How this actually works

AI-Spirit is a persona chat platform — Osho, Nelson Mandela, an unhinged therapist, a yandere romance character, and others, each holding a conversation and a memory of you. This is the engineering walkthrough: the request pipeline, what's actually enforced versus just asked for, and what's tested rather than assumed.

← Back to AI-Spirit

The shape of it

Every chat message runs through the same handler, pages/api/chat.js, whether it streams to the UI token by token or returns in one shot. The message is checked before anything is spent on it, then persona context, memory, and conversation history are gathered in parallel, then a model is called, then — only after the reply has already reached the user — the turn is written back into memory.

flowchart LR
    user["User message"] --> mod["moderateContent()"]
    mod -->|blocked| reject["Rejected, nothing spent"]
    mod -->|clean| gather["Gather in parallel:\ncontext · memories · summaries"]
    gather --> budget["checkCostThreshold($15/day)"]
    budget -->|over budget| refuse["Refuse the call"]
    budget -->|ok| model["Call a model"]
    model --> reply["Reply streamed/returned to user"]
    reply --> persist["extractAndSaveMemories()\nupdateConversationSummary()"]
The write-back step runs after the response is already on its way to the browser — it's awaited, not fired-and-forgotten, because a serverless function is frozen the instant the handler returns. An earlier version of this skipped that and personas simply never remembered anything.

Guardrails

moderateContent() in lib/moderation.js runs on every incoming message before it reaches a model. It's a set of deterministic checks, not a call to another API:

flowchart TB
    msg["Incoming message"] --> len["Length: 1-2000 chars"]
    len --> pii["PII patterns:\nSSN · card · phone · email · Aadhaar"]
    pii --> inj["Prompt-injection phrasing\n(\"ignore previous instructions\" style)"]
    inj --> spam["Spam: repeated chars,\nexcessive URLs, ALL CAPS"]
    spam --> ok["Passed to the model"]
Profanity filtering is deliberately off — personas need to speak in-character, and the models' own safety layers are the backstop for that, not this filter. On the way back out, sanitizeForDisplay() HTML-escapes a reply before it's rendered, closing the obvious XSS path.

What this list does not cover: the model's own reply. Nothing in production inspects what a persona actually says back — that's caught, if at all, by the eval suite's safety cases (below), not by a live check on every message.

Model reliability

All models here are free-tier. The chain exists because any one of them can be rate-limited, empty, or — a specific failure mode worth calling out — leak its internal reasoning as plain, unmarked prose instead of a clean in-character reply.

flowchart LR
    call["OpenRouter call"] --> m1["qwen3-next-80b"]
    m1 -->|leaked reasoning| m2["nemotron-3-nano-30b"]
    m2 -->|leaked reasoning| m3["llama-3.3-70b"]
    m3 -->|leaked reasoning| m4["gemma-4-26b"]
    m1 -->|clean| out["Reply"]
    m2 -->|clean| out
    m3 -->|clean| out
    m4 -->|clean| out
    out -.->|OpenRouter chain empty| groq["Groq fallback"]
looksLikeLeakedReasoning() is what decides "leaked" — the same function runs in production and in every eval case, so a model that leaks scratchpad text fails the eval before it ever reaches a user. Streaming and non-streaming requests pick a different primary provider (self-hosted Ollama, if configured, is tried first in both), but Groq is the universal safety net when the primary path returns nothing.

Memory

"Remembering you" is two separate systems, not one, and both are scoped to a (user, persona) pair — a fact told to Osho does not surface when you talk to Mandela.

flowchart TB
    turn["A finished turn"] --> facts["Layer 1 — facts\nextractMemoriesLLM() pulls durable\nfacts, sanitized, saved per persona"]
    turn --> summary["Layer 2 — rolling summary\nsummarizeTranscriptLLM() refreshes a\nper-persona conversation summary"]
    facts --> ctx["formatMemoriesForContext()"]
    summary --> ctx2["formatSummariesForContext()"]
    ctx --> prompt["Injected into next system prompt"]
    ctx2 --> prompt
Both layers write independently after every turn and both feed the next reply's system prompt. This is the layer with the worst track record in this codebase — it has broken twice before (memories not surviving a new conversation) — which is why it's the most eval-covered system on this page.

Evals

npm run eval runs 21 deterministic cases against a real, live /api/chat — no judge model, no labelled dataset, just regex assertions against what a persona actually said.

flowchart LR
    case["Case: persona + turns"] --> live["Real call to /api/chat"]
    live --> reply["Reply to the last turn"]
    reply --> assert["required / requireAny / forbidden\nregex + length checks"]
    reply --> leak["looksLikeLeakedReasoning()\n(same guard as production)"]
    assert --> result["Pass / fail, model attributed"]
    leak --> result
Constrained by the OpenRouter free tier: 50 model calls/day, and the current suite spends 30 of them. The runner prints the cost before spending anything and refuses to start if a run would exceed the day's budget — which is why this is a pre-release check, not a per-commit one.
21

Deterministic cases

Voice/character consistency, era & canon-fact anchors, register and length, in- and cross-conversation memory recall, memory non-confabulation, date/context injection timing, and cross-persona isolation.

6

Safety cases

Self-harm handling under an obsessive or unhinged persona, age-boundary breaks in romance personas, medical misinformation and emergency-symptom refusal, and dangerous fitness prescriptions — run against the personas whose prompts pull hardest the wrong way.

0

Voice/tone judged

Nothing here rates whether a reply actually sounds like the persona. That needs a judge model or human rating — the thumbs up/down now written to message_feedback is the intended source, pending a migration.

Data & access

Auth
Supabase Auth, Google OAuth
Row-level security
Profiles, conversations, and messages scoped to auth.uid(); personas are publicly readable by design
Spend guard
checkCostThreshold() refuses a model call once a $15/day budget is hit
Rate limiting
Built (lib/rate-limit.js, 10 req/min) but not wired into the chat route yet — disabled on purpose until userbase grows

Known gaps

Listed here on purpose, not swept into the stack section:

  • Reply content isn't screened live. Guardrails run on what the user sends, not on what a persona sends back — the eval safety cases are the only thing standing between a persona's register and a harmful reply today.
  • Voice and tone are unjudged. A reply can be factually correct, in era, and still sound nothing like the persona, and no automated check catches that yet.
  • Rate limiting is dormant. The module exists and is tested in isolation, but the chat API doesn't call it — a deliberate, temporary tradeoff, not an oversight.
  • The in-repo safety audit is stale. SAFETY-RELIABILITY-AUDIT.md dates to 2025-11-15 and describes moderation as far weaker than the current lib/moderation.js — it hasn't been refreshed since.

Stack

Framework
Next.js 14 (pages router), React 18
Database
Supabase Postgres, row-level security
Models
OpenRouter (4 free models) → Groq → self-hosted Ollama, provider order depends on path
Observability
Sentry (client, server, edge)
Hosting
Vercel
Evals
21 deterministic cases, npm run eval
This page documents AI-Spirit's own engineering systems, generated from the current codebase rather than a fixed spec — it will drift out of date the day a system changes and nobody updates it. Back to AI-Spirit.