RAG & Document Retrieval

Retrieval Augmented Generation (RAG) allows your assistants to search through uploaded documents and include relevant context in their responses.

How It Works

1. Upload - You upload documents (PDFs, Word docs, etc.) in the dashboard
2. Process - Documents are automatically chunked and embedded
3. Retrieve - When users ask questions, relevant chunks are retrieved
4. Generate - The assistant uses the context to generate accurate responses

Setting Up RAG

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.assistantrouter.com/v1',
  apiKey: process.env.ASSISTANTROUTER_API_KEY,
});

const response = await client.chat.completions.create({
  // @ts-ignore
  assistant_id: 'your-assistant-uuid',
  messages: [
    { role: 'user', content: 'How do I reset my password?' }
  ],
});

console.log(response.choices[0].message.content);

Managing Documents

Supported File Types

Document Status

Organizing Documents

Documents can be associated with specific assistants:

1. Go to the assistant's Settings
2. Under File Search, select which documents to include
3. You can add all workspace documents or select specific ones

Best Practices

1. Chunk wisely - Keep related content together in documents
2. Use descriptive names - Helps identify documents in the dashboard
3. Update regularly - Keep your knowledge base current
4. Test retrieval - Verify the right content is being found
5. Split large documents - Better retrieval with smaller, focused documents

Understanding Retrieval

When you send a message to an assistant with RAG enabled:

1. Your query is converted to an embedding
2. Similar document chunks are retrieved (typically top 5-10)
3. Retrieved context is injected into the prompt
4. The model generates a response using the context

Response Metadata

The API response includes information about tools used:

{
  "choices": [...],
  "x_assistantrouter": {
    "cost_cents": 0.15,
    "tools_used": ["file_search"],
    "was_fallback": false
  }
}

Troubleshooting

Documents Not Being Retrieved

• Ensure the document status is ready in the dashboard
• Check that file search is enabled on the assistant
• Verify the query is relevant to the document content
• Try rephrasing the question

Poor Quality Responses

• Improve document formatting (clear headings, sections)
• Add metadata/descriptions when uploading
• Consider splitting large documents into smaller pieces
• Ensure documents contain the relevant information

Processing Failures

• Check file size is within limits
• Verify file is not corrupted
• Try re-uploading the document
• Check for unsupported file formats