Building Commence AI: Developing a Contextually Intelligent Chatbot for an Enhanced User Experience

In this article, we explore the technical foundations of Commence AI, highlighting the design choices and core components that enable its efficiency, precision, and responsiveness. We’ll examine its advanced features -adaptive chunking, dynamic classification, and a memory system-that work together to create a seamless, intuitive experience, offering users an assistant that truly understands their needs.

Commence AI is a web-based chatbot built to leverage Commencis’ internal knowledge base, enabling it to assist website visitors with comprehensive, contextually relevant information. By combining efficient information retrieval with a robust memory component, Commence AI provides users with a smooth, personalized experience that enhances engagement and usability.

Building a responsive and precise retrieval system was essential for Commence AI, given the extensive scope of our knowledge base. After laying a foundation of answers related to chatbot functionality, we developed a structured approach to align relevant data with user queries effectively. Our primary goal was to ensure that Commence AI could reliably answer queries within its scope and avoid producing irrelevant responses.

We structured the knowledge base by dividing it into smaller, semantically meaningful documents that correspond to specific content areas. This segmentation enabled the chatbot to navigate large datasets with accuracy, improving response time and precision. The class system we implemented maps queries to these document segments, ensuring that user inquiries are directed to the most relevant content. For instance, queries about company policies are routed exclusively to policy-specific documents, while technical questions are sent to troubleshooting resources.

To further enhance retrieval accuracy, we developed a custom chunking system. Instead of applying a one-size-fits-all approach, our chunking strategy is adaptive. Each document has unique characteristics—some are concise and require larger chunking for broader context, while others contain dense, nuanced information that benefits from finer chunking for pinpoint relevance. An intelligent agent within the system dynamically selects the best chunking approach for each document, optimizing segmentation to match content type and usage needs. This structured, agent-driven chunking method not only keeps response times low but also ensures that retrieval pulls the most contextually appropriate information.

To manage user queries effectively, we implemented a classification module that assigns each query to the appropriate document class. This classification directs the query to a relevant data flow, allowing for quick access to pertinent information.

Once classified, the system retrieves related chunks and sends them to the answer-generation module. Our design uses independent language models tailored for each class, allowing prompt customization for each specific query type. This modularity improves flexibility and enhances debugging, as each generation module can be refined independently. For example, a product inquiry uses a prompt fine-tuned for product details, while a troubleshooting question engages a separate, context-specific model and prompt. This setup has proven crucial for maintaining answer consistency across diverse query types.

To further enhance conversational flow, Commence AI incorporates a memory system. The system retains details from recent interactions within the ongoing session, ensuring that the chatbot provides contextually relevant follow-up answers without redundancy. By focusing on recent interactions, the memory system enabled chatbot to avoid confusion that could be caused by outdated details. This enhances the conversational experience by maintaining a natural flow, even in complex, multi-step exchanges.

This approach makes conversations more personalized, as Commence AI can recall previous exchanges, respond to follow-up questions, and create a sense of continuity.

With all the processes explained above, a simplified flow looks like below:

To ensure that Commence AI delivers consistent, high-quality interactions, we undertook a rigorous testing process with a focus on key areas:

• Class Assignment Accuracy: We scrutinized the classification module’s performance, checking that user queries were consistently mapped to the correct class. Misclassifications often led to irrelevant or “empty” responses, where the chatbot fails to answer questions, it should recognize. To improve accuracy, we fine-tuned our classification prompts, minimizing such errors.
• Chunk Retrieval Accuracy: By reviewing retrieved chunks, we ensured that responses consistently pulled the most relevant data from our knowledge base. This included examining our document structure to confirm alignment between query classifications and corresponding content, thereby reducing the risk of retrieving unrelated chunks.
• Answer Consistency and Style: We monitored the generation module for tone, clarity, and coherence in responses. Each language model was adjusted to maintain a consistent style across different query types. This prompt refinement was essential for delivering a uniform, professional tone that aligns with Commencis’ brand voice, enhancing the user experience.

Commence AI is the result of an integrated approach to knowledge management, adaptive chunking, modular classification, and memory system. Through this development journey, we’ve built a chatbot that is both precise and flexible, providing context-aware, efficient answers that meet user needs. We look forward to seeing how Commence AI transforms website interactions, making them more engaging, intuitive, and valuable for our users.

If you’d like to learn more about how Commence AI can enhance your business, please contact our team. We’d be delighted to discuss how our chatbot can bring intelligent, seamless support to your customers.