A Word add-in that helps civil servants rewrite letters to residents at B1 level, with a local language model and the civil servant in charge.
Problem
Letters from municipalities are often too complex for a large part of the population. Writing guides exist, but under pressure staff don’t always manage to write at B1 level. Generative AI could help, but legal risks, privacy and reliability make it a sensitive tool for a municipality.
Approach
We worked in four iterations of increasing fidelity: a separate text editor to test feasibility, a paper prototype about mental models and explainability, a Word add-in that fits the existing workflow, and a high-fidelity version with error handling and recovery. We tested every version against legal, ethical and functional requirements.
The system
A local Llama 3 model runs through Ollama, so letters never leave the municipality. The pipeline first anonymises the letter, finds difficult words with a word list and semantic retrieval, rewrites the text and then restores the personal details.
We scored every iteration on readability and meaning with LINT, BERTScore, SARI and ROUGE.
Design
The interface is built on trust. Suggestions come with a confidence level, words that may carry legal meaning ask for confirmation, a banner reminds staff that AI can make mistakes, and every letter can be rated. Nothing changes in the letter until the civil servant accepts it.
My research
My own research question was how the structure of a prompt affects how often a model hallucinates. For the symposium I built a tutorial that checks whether a simplified text stays faithful to the original, using semantic search, an entailment model (DeBERTa) and an LLM as judge.
Design & poster
Empathy map, design patterns, flow, screens and the final poster (in Dutch). Tap a page to see it large.