Washakh Ali

Summary

What I did

• 300,000+ production image/video generations sustained across concurrent GPU models, engineered as diffusion-based generation APIs in FastAPI with a custom GPU memory-management system that held under high-concurrency load.
• Reduced inference latency on concurrent production workloads by architecting parallel multi-GPU inference pipelines and optimizing model layer caching, without degrading output quality.
• Integrated the latest open-source models across image, video, audio, LLM and 3D modalities into a unified FastAPI inference layer, expanding platform capabilities well beyond 2D generation.
• Cut compute cost per generation by optimizing these models for faster inference with CUDA parallel computing, multi-GPU execution and layer caching, so the company served more workloads on the same GPU fleet.
• Enabled reproducible, portable inference across environments by containerizing model services with Docker.
• Handled structured outputs by passing Pydantic schemas to Gemini with JSON mode enabled and validating via model_validate_json.
• Quantized diffusion and LLM weights to reduce GPU memory and latency.

Results

• Sustained 300,000+ production image/video generations.
• Reduced inference latency and cut compute costs per generation.