I Somehow Built a RAG Chatbot with Fine-Grained Authorization—But Don’t Know How
Vibecoding an authorized RAG chatbot with minimal coding experience, facing a critical security incident, and learning how to debug something I don’t quite understand.
TABLE OF CONTENTS
App URL: https://ai-interview.up.railway.app
Just imagine- you are applying for a job; but there is virtually no way of getting “realistic” practice in, as in talking to an actual person informed about your resume as well as the job you are applying for.
Now, you can simulate a conversation using the ChatGPT UI- but talking face-to-face with an interviewer is a completely different beast. For that reason, I wanted to “vibecode” an app where you can practice face-to-face with an AI avatar.
Building the App
I first started discussing the architecture of the app with ChatGPT o4-mini-high so that I was prepared to tell the Cursor agent running on Claude Sonnet 4 how to exactly structure the app and which APIs I wanted to use. I asked ChatGPT to structure my initial prompt for Claude- then it was time to build it step-by-step and debug along the way.
🛠️ Tech Stack
Frontend
- Next.js 14 - React framework with App Router
- TypeScript - Type safety and better developer experience
- Tailwind CSS - Utility-first CSS framework for rapid styling
- Framer Motion - Animation library for smooth UI transitions
- Heroicons - Beautiful SVG icon library
- React Hot Toast - Elegant toast notifications
Backend/API Integration
- Next.js API Routes - Serverless API endpoints
- OpenAI GPT-4 - AI conversation and response analysis
- D-ID API - Realistic avatar video generation
- ElevenLabs API - Text-to-speech for audio fallback
- Web Speech API - Browser-native voice recording and transcription
Issues
I ran into a bunch of issues while building this. For instance, Claude used the incorrect payload for the D-ID API endpoint, resulting in an error. The other issue which still persists is how long the D-ID avatar video generation takes. This is likely a limitation of today’s technology available to us, which will probably improve very rapidly over the next months, even. With that said, although the video generation ruins the realism of the conversation, the face-to-face aspect is still there; and my vibecoded app works!
Next Steps
I’d love to see the latency between the request and the AI avatar video response to be improved. If it does, the app I created is only the beginning. It can be applied to many other areas such as getting clients ready for court hearings & more.
Related Posts
How Should Startups Approach Conversion Rate Optimization & A/B Testing
Thoughts on how to improve conversion rate by betting on more comprehensive tests & tradeoffs that come with this approach.