A support desk for Tunely, a fictional music-streaming company. The central problem is not generating a reply: it is deciding when AI should answer on its own, and when a person needs to take over.
- Status
- Completed · live demo
- Stack
- Next.js · Express · Python · PostgreSQL
From question to reply
The system finds relevant help documents, writes a draft with its sources, then checks whether a public article clearly covers the question. Safe, covered questions get an automatic reply; risky topics, uncertainty, and reliance on internal notes go to an agent.
The agent receives an editable draft and a saved decision trail. Customers can ask for a person after an automatic reply, and only the customer can mark a conversation solved.
Where the model stops
Rules in the Express API have the final say. They can override the model for money problems, account-security incidents, weak sources, and staff-only notes. An AI suggestion is not permission to send a reply.
Documents are treated as untrusted input. A copy guard checks for phrases of eight or more words taken from internal notes, both in Python and again in the API before sending. Separate checks look for unsupported words and numbers in a draft.
Three services, clear responsibilities
Next.js handles the interface and talks to the API from the server. Express owns sessions, saved history, usage limits, local document embeddings, and the final routing rules. A Python standard-library service handles retrieval, model calls, and draft checks. PostgreSQL stores the shared state; the model key stays out of the browser.
The public demo runs as on-demand Vercel functions. Pull requests also deploy the containerised services to a temporary Kubernetes cluster for browser tests. Database-enforced limits allow five live drafts per visitor and 200 per day overall; exhausted limits send questions to a person instead of breaking the conversation.
Checks and known limits
Saved evaluations route 8 of 8 labelled cases correctly and block 3 of 3 internal-note leak attempts. Document search finds the right article in the top three for 10 of 11 questions, compared with 7 of 11 for keyword matching.
Answer checks pass 17 of 19 cases. The two failures remain visible: a wrong priority suggestion and a contradiction missed by the word-based check. The copy guard also cannot catch every paraphrase or short quote.
These small, fictional datasets use a fake model to make CI free and repeatable. They demonstrate the checks, not general AI accuracy. Replies stay inside the demo conversation; no email is sent.