Haseeb Javaid

Summary

What I did

• Designed and implemented an AI-powered text analysis pipeline using an in-house LLM service (sglang).
• Developed preprocessing logic to clean and normalize raw text (Unicode decoding, HTML unescaping, whitespace handling).
• Engineered custom prompt for extracting sentiment, emotion, category, sub-category, and named entities.
• Built post-processing modules with regex-based validation for phone numbers and emails, ensuring high-quality outputs.
• Implemented strict NER filtering and enrichment for allowed entity types (Person, Organization, Location, Phone, Email).
• Enhanced system reliability with fallback mechanisms to handle invalid or incomplete JSON outputs.
• Deployed the inference pipeline into production with FastAPI services and Kafka integration for asynchronous request handling.
• Developed an end-to-end pipeline for suspect and target video/audio analysis.
• Implemented audio extraction, conversion, and segmentation for efficient processing.
• Applied speaker embedding extraction and cosine similarity to detect suspect's voice in target videos.
• Automated chunk management, FTP uploads, and AI-based transcription via external APIs.
• Generated word cloud visualizations from transcriptions to highlight frequent terms.
• Produced structured AI responses including match metadata, time segments, transcripts, and summaries.
• Built an end-to-end pipeline for audio and video transcription using Whisper (faster-whisper).
• Implemented audio preprocessing by splitting large files into smaller chunks for efficient transcription.
• Integrated Whisper large-v3 model with GPU acceleration, supporting on-demand model loading and unloading to optimize memory usage.
• Automated handling of YouTube links and direct audio inputs with audio extraction and conversion.
• Designed post-processing logic to generate original transcription, English translation, and detected spoken languages.
• Ensured reliability with temporary file cleanup, structured response generation, error handling, and logging.
• Designed and implemented a Retrieval-Augmented Generation (RAG) pipeline to enhance LLM accuracy using vector-based semantic search.
• Integrated custom embedding models with Milvus for efficient context retrieval from structured and unstructured data sources.
• Built an LangGraph enabling autonomous reasoning, tool use, and API interaction through function-calling mechanisms.
• Orchestrated multi-agent workflows for task delegation and coordination between retrieval, reasoning, and response-generation agents.
• Deployed the RAG and agentic inference services using FastAPI integrated with Kafka for scalable, asynchronous request handling.
• Optimized inference latency and throughput by leveraging GPU-based deployment and caching mechanisms for retrieved context.
• Built and deployed an image captioning system using FastAPI and Hugging Face vision-language models.
• Implemented on-demand model loading/unloading with 4-bit quantization to optimize GPU utilization.
• Designed a robust pipeline to download images, generate descriptive captions, clean outputs, and handle errors.
• Integrated Kafka for asynchronous request handling and real-time response streaming.
• Ensured thread-safe execution with locks to manage concurrent GPU inference requests.
• Built a FastAPI-based microservice for multilingual text translation using Meta's SeamlessM4T-v2 model.
• Integrated FastText for automatic language detection and mapping to supported translation codes.
• Implemented sentence segmentation and preprocessing pipeline for improved translation accuracy.
• Developed robust error handling, logging, and response validation mechanisms.
• Supported dataset acquisition and annotation using CVAT and Roboflow.
• Performed image preprocessing and fine-tuned YOLOv5 for custom object detection.
• Converted models to TensorRT for high-performance inference.
• Built and deployed end-to-end NVIDIA DeepStream pipelines for real-time video analytics.
• Managed AI service deployment across servers (Dev to Production).
• Developed Recorder and Deepstreamer services for processing CCTV/drone feeds in real time.
• Enabled surveillance capabilities like military object detection, crowd analysis, and threat monitoring.
• Assisted in collecting high-resolution satellite imagery and annotating data using CVAT.
• Preprocessed images using SAHI for slice-based object detection.
• Fine-tuned a YOLOv5 (1280) model for vehicle detection.
• Built and managed an end-to-end FastAPI inference pipeline.
• Deployed AI services on servers (Dev to Production).
• Implemented a reconnaissance model to detect and classify vehicles (civilian, military, tanks) using SAHI.
• Implemented and deployed a facial recognition system to track individuals and detect anomalies from live camera feeds.
• Utilized a combination of RetinaFace for face detection and ArcFace for generating facial embeddings.
• Stored facial vectors in Milvus Vector Database to enable fast and scalable similarity search.
• Packaged the service as a Dockerized microservice and deployed it on GPU infrastructure for real-time performance.
• Worked on live feed for number plate detection and text extraction using PaddleOCR and EasyOCR.
• Applied different image pre-processing for text extraction.
• Worked on pedestrian detection and count on live feed.
• Created a custom drone detection model by training YOLOv8 on an in-house collected drone dataset.
• Built a FastAPI-based inference microservice, containerized the application using Docker, and deployed it on GPU servers.
• Enabled real-time drone detection from aerial footage to enhance surveillance capabilities.
• Used SAM segmentation models to detect and delineate water bodies, urban areas, and grid stations.
• Converted segmentation masks to GeoJSON for geospatial visualization and analysis.
• Fine-tuned YOLOv8 for bridge detection on custom datasets.
• Transformed bridge detections into geo-referenced bounding boxes using raster metadata for GIS integration.
• Developed RAG and agentic systems for a social-listening and intelligence platform to automate the analysis of high-volume social media and news data.
• Implemented a LangGraph agent for query understanding and routing to interpret analyst questions and determine retrieval paths based on platform, subtopic, and time range.
• Integrated RAG over crawled posts using Milvus and multilingual-e5 embeddings to generate grounded answers from vector stores.
• Defined expected LLM outputs as Pydantic models to serve as contracts for downstream services.
• Implemented schema-constrained generation using tool-calling modes like LangChain's with_structured_output.
• Developed validation and retry logic in LangGraph nodes to handle ValidationErrors by feeding error messages back to the LLM.
• Applied field-level constraints including Enums, Literal types, and custom validators to prevent hallucinated references.
• Utilized typed graph states in LangGraph to ensure explicit node inputs and outputs.
• Implemented nested Pydantic models for multi-step workflows to ensure every pipeline stage's output is typed and malformed results fail immediately.
• Developed cross-field validation using Pydantic v2 @model_validator for complex rules like date range logic and source post verification.
• Applied discriminated unions in Pydantic to handle different result types, such as cache hits versus full RAG results, within the same workflow.
• Used mode='before' validators to normalize messy inputs like platform name casing and whitespace trimming before strict validation.
• Handled batching, throughput, and memory tuning for local inference using vLLM and SGLang.

Results

• Optimized an audio processing pipeline to reduce the processing time for a 16-minute audio file to 4 seconds.
• Developed a semantic query cache that compares incoming queries against past queries to return existing results, reducing latency and LLM costs.
• Built a target summarization system that provides immediate summaries of latest posts while background processes cluster and merge all posts into a consolidated summary.
• Centralized business rules within Pydantic models to keep the social listening workflow easier to reason about and test.