Creating a Customer Support RAG Assistant on a Cloud Platform
When you ask a language model something specific about your product, it responds quickly and fluently, but sometimes the information is incorrect. This issue was the starting po...
When you ask a language model something specific about your product, it responds quickly and fluently, but sometimes the information is incorrect. This issue was the starting point for a workshop focused on improving the accuracy of such responses. On a particular day, a live session was held to construct a customer support assistant, named HelpBot, using a cloud-based AI platform. Initially, HelpBot was as uninformed as any standard model, but step-by-step modifications were implemented to enhance its capabilities.
Key Workshop Topics:
- Identifying the Problem: Demonstrations showed HelpBot providing incorrect answers about a product, highlighting the need for improvement.
- RAG and Knowledge Bases: By integrating our documentation, HelpBot began to provide answers based on real content rather than guesswork, utilizing specific document sections.
- Inference Router: A router was used to select the best model for each request, based on criteria like cost and speed. This setup allowed for automatic failover without needing code changes.
- Guardrails: Safety measures were added to prevent leaks of personal information, block unauthorized access attempts, and manage inappropriate content.
- Evaluations: An additional AI model evaluated HelpBot's responses for accuracy and safety, enabling automated quality checks.
- Final Integration: The entire system was tested with the original query to demonstrate improvements.
By the end of the workshop, attendees witnessed the transformation of the assistant from a basic command to a deployable solution. Prior experience with calling a language model API is sufficient to follow along, with no advanced AI background needed. Detailed instructions are available through both video and written guides.
Preparations Before Starting:
- Account Setup: Ensure you have access to a cloud account with inference capabilities and a model access key.
- Documents for Knowledge Base: Have a small set of documents ready for the assistant to reference.
- Optional API Token: For automation, an API token with specific permissions may be needed.
Step-by-Step Guide:
Step 1: Orientation and Initial Inference Call
Objective: Understand the platform and make an authenticated inference call.
- Confirm your model access key in the Console.
- Execute your first call using a standard payload format.
curl https://inference.do-ai.run/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $GRADIENT_MODEL_ACCESS_KEY" \
-d '{
"model": "llama3.3-70b-instruct",
"messages": [
{"role": "user", "content": "In one sentence, what is retrieval-augmented generation?"}
]
}'
Step 2: Establishing RAG and Knowledge Bases
Objective: Create a knowledge base, understand document processing, and validate retrieval methods.
- Navigate to the Knowledge Base section to create a new base.
- Choose a data source, ensuring the content is relevant.
- Configure chunking and embedding options before creating the knowledge base.
- Use the RAG Playground to test retrieval and ensure accurate answers.
Step 3: Utilizing the Inference Router
Objective: Implement a router to enhance resilience, cost-effectiveness, and latency management.
- Set up a router with tasks, selection policies, and a model pool.
- Use the router as a replacement for direct model calls.
Step 4: Implementing Guardrails
Objective: Add security features to the assistant to handle sensitive data and content moderation.
- Attach guardrails to the agent for monitoring both prompts and responses.
- Customize and test these guardrails in the agent playground.
Step 5: Evaluations for Quality Assurance
Objective: Use an AI model to evaluate the assistant’s responses against predefined metrics.
- Prepare an evaluation dataset and upload it.
- Run evaluations and analyze results to ensure quality standards are met.
Step 6: Finalizing and Deploying the Assistant
Objective: Integrate all components and deploy the assistant with a robust endpoint.
- Set up the agent’s API for external access.
- Test the full functionality, including guardrails and evaluations, through the final endpoint.
Building Your Own AI Assistant
The cloud platform provides all necessary tools for creating a robust AI assistant, including serverless inference, model routing, and configurable safety measures, all within an integrated system.