Case study
SupportPilot
A multi-tenant support copilot that finds relevant company documents, drafts grounded replies, and escalates when confidence is low.
Context
Overview
SupportPilot helps teams answer support questions using retrieval-augmented generation (RAG): it searches company documents first, then drafts a reply from that retrieved context instead of inventing facts. Each workspace owns its knowledge base. Documents are chunked and embedded with OpenAI text-embedding-3-small (1,536 dimensions) into pgvector with HNSW cosine retrieval. Uncertain cases can be escalated for human review. The product is in development and is not presented as a commercial customer-support platform with live client teams.
At a glance
Product
A multi-tenant support copilot that finds relevant company documents, drafts grounded replies, and escalates when confidence is low.
Audience / need
Support teams need faster answers without inventing facts — especially when knowledge is scattered across documents and multiple companie…
Ownership
Improved Precision@5 by approximately 28% during retrieval evaluation
Result
+~28% — Precision@5
Context
The problem
- Generic chatbots invent answers when company knowledge is incomplete or poorly retrieved.
- Multi-tenant support products must isolate each company’s documents and tickets.
- Teams need a clear path to escalate when the model is unsure.
Approach
The solution
- Tenant workspaces with isolated document processing and authorization.
- Chunking and embedding pipelines that prepare knowledge for semantic retrieval.
- A planner and drafting workflow where backend rules enforce escalation and access control.
- Review tools so humans can inspect and improve AI-assisted replies.
Role
What I owned
- Improved Precision@5 by approximately 28% during retrieval evaluation
- Used OpenAI text-embedding-3-small with 1,536-dimensional embeddings and pgvector HNSW cosine search
- Built tenant-aware authorization using Supabase JWTs and backend validation
- Used a planner and drafting workflow while enforcing important rules in backend code
- Designed the system around a clear principle: the LLM suggests; the backend enforces
Outcomes
Key results
+~28%
Precision@5
Retrieval quality improved by approximately 28% during development evaluation
1,536-d
Embeddings
text-embedding-3-small with pgvector HNSW cosine search
Backend rules
Safety principle
The LLM suggests; the backend enforces
Product
Core product flow
- Multi-tenant knowledge bases
- Document chunking and embeddings
- Semantic retrieval with pgvector
- RAG response generation
- Confidence and escalation workflows
- Feedback and review tools
- Tenant-aware authorization
- Auditable AI outputs
System
Architecture
High-level system pieces involved in the product.
- 01
Next.js multi-tenant UI for documents, tickets, playground, and settings
- 02
FastAPI services for chunking, embedding, retrieval, and drafting
- 03
PostgreSQL with pgvector for tenant-scoped semantic search
- 04
OpenAI text-embedding-3-small embeddings stored as 1,536-dimensional vectors
- 05
Supabase JWTs validated on the backend for authorization
- 06
Audit-friendly storage of retrieved context and generated drafts
Engineering
Important decisions
Constraints
Challenges and trade-offs
Tools
Technology stack
- Frontend
- Next.js · TypeScript
- Backend
- FastAPI · Python · Docker
- Data
- Supabase · PostgreSQL
- AI
- OpenAI
- Other
- pgvector · Retrieval-Augmented Generation · text-embedding-3-small
Visuals
Additional screenshots
