Mohib Kazmi

Summary

What I did

• Architected enterprise RAG system using LlamaIndex, FastAPI, and Gradio, serving 20+ concurrent users with 2-4 second response times for document Q&A, summarization, and semantic search across a 10GB+ corpus.
• Migrated on-premises LLM infrastructure from Ollama to AWS Bedrock, cutting ingestion time 8.4x (251s → 30s) and operational costs by 40%; converted the synchronous architecture to async (asyncio) for multi-user concurrency.
• Migrated vector storage from Qdrant to AWS OpenSearch Serverless for 1M+ embeddings, weighing HNSW tuning against managed-uptime guarantees; selected models via blind A/B testing across 15+ LLMs and 5+ embedding models.
• Led a dependency migration from 2-year-old library versions to LlamaIndex 0.14.4 with zero downtime.
• Designed and deployed an enterprise Text-to-SQL system letting 50+ non-technical users self-serve reports from 300+ table schemas - analyst workload down 60%, 200+ queries daily at 75% first-attempt accuracy.
• Evolved the system from fixed RAG workflows to a fully agentic LangGraph architecture - custom tools with structured outputs (Pydantic + function calling) and multi-agent pipelines replacing a monolithic LLM call.
• Added a guardrails layer blocking destructive queries (DROP, DELETE, TRUNCATE) before execution, RAGAS-based result-accuracy evaluation, and a feedback loop with bi-weekly audits improving accuracy 2-3% monthly.
• Added Plotly visualization generation via LLM code synthesis and a custom Power BI visual built in React - 10+ chart types from natural language at 75% first-attempt success, plus Whisper voice input.
• Extended file support from 3 to 8+ formats with a PandasQueryEngine, an LLM-driven multi-sheet Excel handler, and automated statistical summarization, cutting manual analysis by 80%; tested 5 chunking strategies for a 30% gain in retrieval relevance, tracked with RAGAS.
• Used AWS Bedrock on-demand resources for cost-efficiency.
• Utilized OpenSearch as a backup database.
• Highly optimized a local Qdrant instance for the specific project use case.
• Developed a multi-agent system using an orchestrator agent that routes queries to specialized sub-agents for manufacturing and sales domains.
• Implemented specialized agents for manufacturing subdomains including scrap, repair, and operational insights.
• Implemented sub-agents that utilized custom Python tools defined as LangGraph tools with Pydantic schemas for arguments.
• Developed a schema/metadata tool to fetch relevant tables, columns, and descriptions for specific domains to avoid context overflow.
• Built a sample-values lookup tool for agents to verify real column values like status codes and plant names.
• Created an execute-SQL tool called by the orchestrator after validation.
• Implemented a guardrail layer to block non-SELECT SQL statements and destructive commands.
• Designed a retry mechanism that feeds database errors and table schemas back to agents for query correction with a retry cap.

Results

• Improved SQL query writing accuracy by shifting from a RAG and single-agent architecture to a multi-agent architecture.
• Promoted from Data Scientist to Senior Data Scientist in Feb 2026.
• Cut ingestion time 8.4x (251s to 30s).
• Reduced operational costs by 40%.
• Reduced analyst workload by 60%.
• Improved accuracy 2-3% monthly through feedback loops.
• Achieved 75% first-attempt accuracy for Text-to-SQL and visualization generation.