Help Your Chatbot Learn: The Train Blaze Page

Introduction to AI learning

Blaze's AI assistant is designed to understand your database structure automatically by analyzing foreign keys, existing comments, and data types. It possesses broad industry knowledge and strong reasoning capabilities, often inferring the correct approach without explicit instruction.

However, many companies have unique business logic, definitions, and tribal knowledge. This information is often to know without being told, just as a new human analyst—no matter how intelligent—would need to consult documentation or ask colleagues to understand company-specific context.

Consider these examples:

• Ambiguous Definitions: If a user asks, "Who are the 10 most active users?", what defines "active" in your company? Is it based on logins, messages sent, or specific actions?
• Business Logic: When someone asks for "revenue this financial year," Blaze needs to know when your company's specific financial year begins (e.g., January 1st or July 1st).
• Database Peculiarities: Blaze needs to know if certain tables or columns are unreliable, or if specific data should be ignored (e.g., excluding old product plans or statuses that are deprecated/no longer active from reports, even though they still exist in the database).

Teaching Blaze this information ensures consistency and accuracy across your entire organization.

Understanding How Blaze Learns

To use the training tools effectively, it helps to first understand how Blaze retains information. The core of the learning system is the Knowledge Base.

Think of the Knowledge Base as Blaze's long-term memory. It stores instructions, definitions, business logic, and any other context Blaze needs to understand your company and database reliably. This information is stored as "Knowledge Notes."

Populating the Knowledge Base

There are two ways to add knowledge:

1. Automatically (Recommended Starting Point): Blaze is designed to learn from feedback. Using the training tools (like Training Questions) helps identify knowledge gaps and automatically generates the necessary notes for you.
2. Manually: You can add notes directly if you know obvious information Blaze will need (low-hanging fruit).

The Most Important Principle: Be Selective

It is crucial not to dump all your documentation into the Knowledge Base. Blaze can figure out a lot on its own—you might be surprised how much. The Knowledge Base is primarily for information it cannot reasonably infer.

The AI has a limited attention span. If you overload it with information it doesn't need, it becomes harder for it to focus on the truly essential instructions. You should only add knowledge if it genuinely improves Blaze's accuracy.

The Path to Success: Onboarding Strategy

Training Blaze is straightforward and fast. You don't need to make it perfect before inviting users. The typical process involves only a few hours of active work spread over 1-2 weeks per department.

Here is the recommended approach (start with one department, e.g., Marketing):

1. Initial Training: Add and review about 10 Training Questions representative of the department's needs.
2. Invite & Iterate: Invite 1-5 initial test users from that department. Monitor their chats for about a week using the Chat Review tab, and feed their real-world questions back into the Training Questions workflow.
3. Full Rollout: Once performance is reliable for the test group, invite the rest of the department and repeat the process for the next team.

The Train Blaze Page

The Train Blaze page is the centralized location for teaching the AI assistant, monitoring its performance, and managing its knowledge. It is accessible to Admins and Technical users and consists of three tabs:

1. Training questions: The best place to start. Add example requests, review Blaze's answers, and allow it to automatically insert knowledge into the Knowledge Base.
2. Knowledge base: Directly view, edit, and add "Knowledge Notes" about your database and company.
3. Chat review: Monitor and quality-check chats from your teammates to identify knowledge gaps.

1. Training Questions (Recommended Starting Point)

The Training Questions tab allows you to proactively test Blaze's understanding and teach it automatically. You provide a question, review the AI's answer, and give feedback.

Why Start with Training Questions?

This approach is highly recommended because:

• It's Easy: You don't need to understand how BlazeSQL works internally or how to write prompts. You only need a question a business user might ask and the ability to verify the answer.
• Empirical Testing: Rather than guessing what knowledge you need to add, you are empirically testing what Blaze can infer on its own. This prevents adding redundant information and keeps the Knowledge Base lean. (Note: For very clean databases, you may find little or no training is needed).
• Automatic Learning: Blaze handles the hard part—generating the necessary knowledge notes and testing them for you.
• Skill Building: By going through 10-20 training questions, you will develop a better intuition for Blaze's genuine knowledge gaps and learn how effective notes are structured by example. This allows you to confidently bypass training questions and add knowledge directly to the Knowledge Base if you prefer. Both are viable approaches.

The Recommended Workflow

The goal is to add requests that are representative of what your team will ask.

1. Start Focused: Pick a specific team (e.g., Marketing) or topic to begin onboarding.
2. Add Questions: Add questions exactly as that team would ask them. Start with one.
3. Iterate: If Blaze answers correctly, great. Keep adding questions until Blaze makes a mistake, then guide it to the correct answer.
4. Reach Milestones: Aim for about 10 validated questions before inviting initial test users (see the "Path to success" sidebar in the app).

Note: You do not need to cover every possible question a user might ask. The goal is to cover the core concepts relevant to a specific domain.

Collecting Questions and Onboarding Users

It's crucial to get realistic questions from business users early in the process, rather than waiting until the setup is perfect.

1. Share Link: Use the "Share link to collect questions" button. Send this link to business stakeholders so they can submit requests in their own words. These submissions automatically appear in your Training Questions list for review.
2. Initial Onboarding: After validating around 10 questions, invite 1-5 test users from that team.
3. Review and Refine: Use the Chat Review tab (Section 3) to monitor their actual chats. This provides the most realistic test cases, which you can then feed back into the Training Questions.

What Makes a Good Training Question?

• Representative: It reflects real questions your team asks, using their terminology.
• Specific: It's a question that can be answered relatively directly, ideally with a single query, rather than a request for deep, multi-step analysis.
• Verifiable: You know the correct answer or can easily verify it.

How the Process Works

1. Add a Question: Enter a representative question or select one submitted by your team.
2. Review the Simulation: Blaze simulates a chat session, attempting to answer the question. You will be prompted to review the answer (Status: "Please review").
3. Verify the Answer: Review the generated SQL and the results. Blaze will ask you if the answer is correct.

Automatic Learning and Knowledge Generation

• If the answer is Correct: Mark it as correct and move to the next question. Continue this process until you find a knowledge gap.
• If the answer is Incorrect: Click "No" and provide guidance on what Blaze did wrong.
• • Be Specific: The more guidance you provide, the better Blaze learns. Try to explain the knowledge gap that led to the mistake (i.e. help it actually deeply understand). For example, explain the definition it missed rather than just giving the correct number.
  • If you aren't sure why it failed, you can provide directional feedback (e.g., "The revenue number is too low," or "You forgot to filter by status 'active'").
• Knowledge Generation: Blaze will retry the question using your feedback. Once you confirm the correct answer, Blaze analyzes the conversation and distills its learnings into Knowledge Notes. It even tests these notes internally to ensure they would have led to the correct answer initially.
• Final Review and Save: These auto-generated notes are presented to you. You can edit them if necessary before confirming and saving them directly into the Knowledge Base.

2. Knowledge Base

The Knowledge Base tab is where all the information Blaze has learned is stored. This includes notes generated automatically via Training Questions and notes you add manually.

While the Training Questions workflow is the easiest way to add knowledge, you should also add knowledge directly for "low-hanging fruit"—information you know for certain the AI assistant will need (though you might be surprised at what it can infer).

Knowledge Notes

Blaze stores information as individual Knowledge Notes. These are flexible prompts where you can provide literally anything you want the AI to know or use to change its behavior.

This can include, but is not limited to:

• Definitions: Defining company-specific terminology (e.g., "What is a 'power user'?").
• Metrics: How specific metrics should be calculated.
• Business Rules: Core operational logic (e.g., "Our financial year starts on July 1st").
• Instructions: Specific rules on how to handle or interpret data (e.g., "Always exclude test accounts when calculating revenue").
• Clarification Prompts: Importantly, you can instruct Blaze when it should ask the user for clarification rather than making an assumption (e.g., "If the user asks for 'active users' without specifying a timeframe, always ask for clarification").
• Anything else: Any other instructions, context, or behavioral adjustments you want Blaze to remember and apply.

You can add as many notes as you like at any level: Database (General notes), Schema, Table, or Column.

Adding Knowledge Directly

You can add knowledge directly by clicking the "Add knowledge" button and selecting the appropriate scope (Schema, Table, or Column).

Auto-Learning from Main Chats

Blaze also learns during everyday use. If you are in the main Chat interface and Blaze makes a mistake, you can correct it in the chat. Blaze will often ask you to confirm if the new answer is correct. If you confirm, Blaze will automatically generate suggested Knowledge Notes based on that interaction.

These suggestions will appear in the Knowledge Base tab for your review and approval.

Principles for Writing Knowledge Notes

When adding notes manually or reviewing auto-generated ones, follow these core principles to ensure optimal performance:

1. Less is More

• Only add descriptions for non-obvious elements. Blaze can infer obvious relationships.
• Focus on actual knowledge gaps identified during Training Questions.
• Keep descriptions concise and clear. Avoid redundancy (e.g., do not explain that user_id is the user ID).

2. Focus on General Understanding1

• Teach broader concepts rather than fixing specific queries.2
• Add descriptions that help with multiple similar cases.3
• Explain core business rules and data structure principles4.

3. Be Explicit When Necessary

• If you notice Blaze is not using a knowledge note in the way you intended, or if it struggles to infer a complex relationship, provide a more explicit instruction. Sometimes direct instructions are clearer than descriptive context.

4. Provide Notes at the Lowest Relevant Level (Locality)

Add the note to the most specific element it applies to.

• If a note is only relevant to one specific column, add it to the Column level.
• If it applies to multiple columns within a table, add it at the Table level.
• If it applies across the entire database, add it at the Database level (General notes).
• Why? This prevents the AI from applying specific instructions globally. If you put a table-specific instruction at the database level, the AI might incorrectly assume it applies to all tables.

5. Separate Unrelated Information (Granularity)

Do not group unrelated instructions or information into a single note. Create separate notes for separate concepts.

• Why? The AI filters notes based on the user's request. If you bundle relevant information with irrelevant information in one large note, the AI cannot filter out the irrelevant parts, potentially confusing the query generation process.

3. Chat Review

The Chat Review tab allows Admins and Technical users to monitor how the rest of the team is using Blaze and perform quality control.

This is essential during the initial onboarding phase. Reviewing real-world interactions helps ensure Blaze is performing reliably and identifies areas for improvement.

Reviewing Chats

You can open any chat session to see the conversation, the data structure used, and the SQL queries generated.

Quality Control and Feedback Loop

1. Mark as Reviewed: Once you have checked a chat, you can mark it as "Reviewed" to track which conversations have been audited.
2. Add to Training Questions: If you notice a mistake or something you’d like Blaze to handle differently, you can click "+ Add to training questions". This immediately feeds the request into the Training Questions workflow, allowing you to correct the mistake and enable Blaze to automatically learn from it.

Summary: The Best Approach to Training

To summarize the recommended strategy:

1. Start Focused: Pick a specific team (e.g., Marketing).
2. Add Obvious Knowledge (Optional): Directly add notes to the Knowledge Base for low-hanging fruit (e.g., financial year definition).
3. Collect Questions: Use the "Share link" feature to gather representative questions from the team, or add your own.
4. Use Training Questions: Process about 10 questions. When Blaze makes mistakes, provide clear guidance so it can automatically generate and save Knowledge Notes.
5. Initial Onboarding: Invite 1-5 test users from the team.
6. Monitor and Iterate: Use the Chat Review tab to monitor their chats. Feed any issues directly back into Training Questions.
7. Expand: Once performance is reliable for the test group, invite the rest of the team and move on to the next domain.

Updated on: 08/09/2025