Exploring the Technical Foundations of AI in Justice
On February 19, 2025, the National Center for State Courts (NCSC) and the TRI AI Policy Consortium hosted the second installment of their webinar series on AI-powered legal chatbots, titled « Tech for All: A Technical Deep Dive into AI-Powered Justice Chatbots. »
The first session, held on January 29, 2025, introduced Beagle Plus (British Columbia) and AVA (Alaska) as AI-driven tools designed to improve access to justice. That discussion focused on how these chatbots assist self-represented litigants in navigating legal challenges. To read the first session’s summary, click here.
In this second webinar, the focus shifted from what these chatbots do to how they work. The discussion moved beyond their general functionalities to explore the technical complexities behind their development, covering AI memory management, training methodologies, and ethical considerations.
Bringing together legal experts, AI developers, and court technology specialists, the webinar took a deep dive into how these AI-powered systems are designed, tested, and refined. Speakers explained how cutting-edge technologies like retrieval-augmented generation (RAG), vector databases, and ChatGPT-4o are shaping the accuracy and reliability of legal chatbots. More importantly, they addressed critical challenges such as reducing AI hallucinations, ensuring legal compliance, and balancing automation with human oversight.
By taking a closer look at the mechanics behind Beagle Plus and AVA, this session offered valuable insights into how AI is not just transforming legal assistance but also redefining how people access justice in a digital world.
Building Beagle Plus: Enhancing Legal Assistance with AI
The People’s Law School in British Columbia, in partnership with TangoWork, developed Beagle Plus, an AI-powered chatbot designed to provide British Columbians with clear and reliable legal information. Integrated into the organization’s website and Facebook page, Beagle Plus offers users an accessible and structured way to find legal answers without requiring legal representation.
Since its launch in early 2024, Beagle Plus has interacted with over 65,500 users, providing legal guidance and connecting people with trusted resources. But how does Beagle Plus ensure that its responses are accurate, relevant, and jurisdiction-specific? The answer lies in its technical architecture and continuous refinement.
How Beagle Plus Works: A Look Under the Hood
Unlike traditional chatbots that rely on pre-programmed scripts, Beagle Plus uses retrieval-augmented generation (RAG) to dynamically fetch and process up-to-date legal information.
Instead of relying solely on pre-trained knowledge, the chatbot integrates real-time legal content retrieval, ensuring that its responses remain relevant to British Columbia’s legal landscape.
To address key challenges such as context retention and legal accuracy, Beagle Plus operates on three fundamental pillars:
- Maintaining conversation context dynamically: Since AI models do not retain memory between interactions, Beagle Plus reconstructs the conversation by including previous chat history in each new query. This allows the AI to provide responses that make sense in context, rather than treating every message as an isolated request.
- Retrieving legal information from a structured database: Beagle Plus does not rely on general AI knowledge alone. Instead, it retrieves relevant legal snippets from a vector database powered by Pinecone, ensuring its responses are jurisdiction-specific and factually accurate.
- Updating its knowledge base nightly: A web crawler scans legal websites daily to detect any changes in laws, regulations, or case precedents. These updates are seamlessly integrated into the chatbot’s system, ensuring that responses remain current and legally sound.
By combining retrieval-based search, dynamic memory handling, and automated knowledge updates, Beagle Plus delivers AI-powered legal assistance that is both reliable and responsive.
Training Beagle Plus: From GPT-3.5 to GPT-4o
During its early development, Beagle Plus was tested using ChatGPT-3.5, but the model struggled with legal accuracy and context retention. It frequently misinterpreted legal terms, leading to inconsistent or misleading responses. Additionally, it suffered from hallucinations, where the AI generated incorrect yet convincing legal information.
To improve performance, the development team implemented a multi-step refinement process:
- Upgraded to GPT-4o, increasing accuracy from 70% to 90%.
- Enhanced the retrieval system, ensuring AI responses were based strictly on verified legal sources.
- Refined manual oversight, with legal experts reviewing over 5,300 chatbot interactions to catch and correct inaccuracies.
- Implemented continuous monitoring, using a dashboard that tracks chatbot responses and flags potential errors for further evaluation.
Through this combination of AI advancements, structured retrieval, and human oversight, Beagle Plus has evolved into a highly reliable self-help tool, making legal information more accessible than ever for British Columbians.
AVA: A Knowledge-Based AI for the Alaska Court System
Unlike Beagle Plus, which dynamically updates its legal content, AVA (Alaska Virtual Assistant) uses a structured and curated knowledge repository from day one. Developed by the Alaska Court System in collaboration with LawDroid, AVA’s creators extracted frequently asked legal questions from official resources and converted them into structured Q&A pairs. This approach ensures that AVA’s responses are built on verified legal information specific to Alaska, maintaining accuracy and reliability.
During its initial testing, AVA’s developers evaluated 91 sample legal queries, identifying instances where the chatbot inferred incorrect legal principles. To eliminate misinformation, every reference and response underwent strict legal vetting before integration. This pre-approved knowledge base dramatically reduced AI hallucinations and enhanced AVA’s legal accuracy.
One of AVA’s standout features is its interactive guidance style. Instead of providing generic answers, AVA engages users in a brief dialogue, asking clarifying questions to better understand their legal situation. This step-by-step approach ensures responses are tailored to each user’s specific needs, making AVA an effective and personalized tool for self-represented litigants.
The Future of AI-Powered Legal Chatbots
The evolution of Beagle Plus and AVA represents a major step toward making legal assistance more accessible. While these AI-powered chatbots cannot replace legal professionals, they serve as valuable first points of contact, helping users navigate legal questions and connect with reliable resources.
Balancing automation with human oversight remains a priority. Beagle Plus benefits from real-time content updates, ensuring efficiency and cost-effectiveness, while AVA maintains a structured knowledge base with manual vetting to guarantee accuracy. Both teams continue to refine their approaches to maximize reliability without compromising efficiency.
With ongoing advancements in AI-driven monitoring, multilingual support, and interactive legal guidance, chatbots like Beagle Plus and AVA are poised to bridge the justice gap and provide meaningful legal support to those who need it most.
For those interested in exploring these tools further, the full webinar recording and additional resources are available here:
📌 Box Link 1
📌 Box Link 2
📌 webinar recording
📌 YouTube Session 1
📌 YouTube Session 2
For more details on AI applications in legal assistance, visit:
🌍 NCSC AI Initiative
Ce contenu a été mis à jour le 9 janvier 2026 à 9 h 53 min.