Hedi Amine Chaabani

Summary

What I did

• Used LLM-as-a-judge and manual spot checks to verify groundedness of claims.
• Utilized user feedback to guide platform improvements.
• Built eight specialized agents within an Azure-based multi-agent system.
• Modeled a contract compliance workflow in LangGraph as a stateful graph with a planner node for step decomposition and tool routing.
• Implemented a hybrid search tool to locate contracts among 10,000+ documents.
• Developed an extraction tool to pull relevant clauses into a validated schema.
• Created a legal retrieval tool to search applicable laws.
• Built a reasoning agent to compare clauses against laws and return answers with citations.
• Defined extraction targets like contract parties, dates, clauses, and prices as Pydantic models.
• Used Pydantic models with Azure OpenAI native structured output and function calling to constrain generation to a schema.
• Implemented Pydantic validation to catch business rules, date consistency, and range errors for feedback into retry loops.
• Applied typed models for all tool inputs and outputs within LangGraph.

Results

• Built self-correction loops where the agent retries extraction with error feedback if schema validation fails.
• Implemented a grounding check that triggers re-retrieval and rewriting if an agent's claim is not supported by a cited source.
• Maintained fast chat performance across 10,000+ documents.
• Measured success through retrieval quality using recall and ranking metrics on a test set of real user questions.
• Tracked extraction accuracy by comparing fields like parties, dates, prices, and clauses against a manually validated sample.
• Monitored schema validation pass rates on first-try outputs.
• Evaluated answer quality through groundedness, ensuring every claim was backed by a cited source.
• Monitored end-to-end latency and cost per query.