AI for a law firm: how to implement an assistant that doesn't make up case numbers

A guide for law firm partners, in-house counsel, and people responsible for IT at legal businesses. We explain how AI for a law firm based on RAG and Polish law works, how it differs from a general-purpose chatbot, and what to ask a vendor before implementation.

Why a general-purpose chatbot isn't suitable for legal research

A general-purpose AI assistant answers from the model’s memory. It sounds confident and gives a case number in the correct format, but the case-law database has no such ruling, or it concerns something else entirely. The lawyer then has to check everything from scratch, so the time savings disappear.

The second problem is currency. The model remembers a provision as worded on the date of training, and Polish law changes constantly. Without access to the consolidated text as of a given date and the act's status, an answer may be correct for the state of the law a year ago.

The third problem is organizational. A law firm rarely asks one-off questions. It works on a case with dozens of files, scans, and correspondence it keeps coming back to for weeks. A tool built for one-off questions can't handle that.

RAG on Polish law: how it works and what it requires

RAG (retrieval-augmented generation) is a pattern in which the model searches for and reads sources before answering, and only then writes the answer based on them. In law, this means the quality of the system depends primarily on the database and the search engine, and only secondarily on the model itself.

In the implementation described above, the database covers 3.9 million documents in 14 collections, including 744,000 court rulings. These include rulings of the common courts, the Supreme Court, the Constitutional Tribunal, the National Appeals Chamber (KIO), and the administrative courts (NSA and WSA), articles of statutes and codes with their wording history, tax rulings, decisions of the UODO, KNF, and UOKiK regulators, legal acts with ELI metadata, and the work of the Sejm. The index contains more than 20 million text fragments.

Collecting the data alone isn't enough. Legal language requires hybrid search: semantic matching captures the meaning of the question, while full-text search catches exact case numbers and institution names. Finally, a reranker trained on Polish legal language orders the results by relevance. The database also has to be refreshed regularly from official sources, in this case daily.

The practical takeaway: when asking a vendor about RAG on Polish law, ask first about the scope and freshness of the database and how search works. The name of the language model matters less, because the model can be swapped, while building a well-indexed database of case law and legislation is the most labor-intensive part of the project.

AI hallucinations in law: verifying citations at the source

RAG reduces the risk of hallucinations but doesn't eliminate it. The model can still add a case number it hasn't read. That's why Prawbot has an additional layer: citation guards compare every case number and every article in the answer against the list of documents actually read in that conversation. A citation that can't be anchored in a source is flagged as requiring verification.

This mechanism came out of testing. The team built its own set of legal benchmarks with case problems from several areas of law, including traps involving special provisions and fabricated case-law holdings. The conclusion was clear: errors occurred almost exclusively where the model answered without consulting the database.

A practical tip when choosing a tool: ask to see what happens when the system can't find a source. Good AI for a law firm should clearly flag that, not answer in a confident tone.

Attorney-client privilege and a language model on your own servers

A contract, an indictment, or litigation documents pasted into a public chatbot end up on infrastructure the law firm has no control over. For a profession bound by confidentiality, that's a question of liability, not a technical detail.

The solution is an open-weight model running on servers controlled by the service provider or the law firm itself. In Prawbot, the model runs on its own GPU servers, and the platform and data sit on a dedicated server in an OVHcloud data center in Warsaw. Queries and documents are not sent to an external model provider. For law firms with stricter requirements, a dedicated instance isolated from the shared cloud is available.

On top of that comes the formal layer. During implementation, it's worth checking whether the vendor has a data processing agreement compliant with Art. 28 GDPR, an AI disclosure compliant with Art. 50 of the AI Act, and a procedure for handling data subject rights. In the project described, a full set of such documents was part of the product, and the code and client isolation were verified by two security audits.

What AI for a law firm should be able to do beyond answering questions

Chat with citations is just the beginning. In day-to-day work, what counts are features that support the entire case:

• a provision as of a given date, with the status of the act (in force, repealed, amended) and a list of amendments,
• case memory, which retains findings between conversations,
• reading scans and photos of documents uploaded to the case,
• deep research: a plan of research questions, parallel specialist agents for individual areas of law, and synthesis into a memo with footnotes,
• legal document generator, in Prawbot over 400 document types,
• custom agent workflows, in which the law firm assembles a repeatable process across dozens of files.

Technically, all of these features use a single set of more than 40 tools exposed via the Model Context Protocol (MCP): search judgments, read an article as of a given date, check the status of an act, show related rulings. As a result, the chat, research agents, and external MCP clients all work on the same verified sources.

How to choose an AI implementation partner for a law firm

Before signing a contract, it’s worth asking the vendor a few specific questions:

• Does the system read the source before answering, and how does it flag citations it hasn't verified?
• What data was it tested on, and can you test it on our documents?
• Where do the model and database physically run, and who is the data processor?
• How often is the legal database refreshed, and where does the data come from?
• What does maintenance look like after implementation, not just the project itself?

The answers to these questions tell you more than a demo. A demo shows well-chosen examples, but in practice, a tool's usefulness depends on how it behaves on a difficult case with incomplete data.

We describe the ready-made platform on the page Prawbot: an AI platform for lawyers, and we bring the same RAG pattern with source verification to other companies' documentation and procedures as part of AI implementations, and the easiest way to start a conversation about your specific law firm is through the contact.