Generative AI · Agentic AI · LLMOps

Akash
Chaudhari

LangGraph MCP Server Multi-Agent A2A Protocol RAG · GraphRAG LlamaIndex Multimodal · TTS DeepEval · RAGAS Langfuse · LangSmith LLM-as-Judge Azure · Databricks Kubernetes · FastAPI PydanticAI LLM Evaluation MS AI & Cybersecurity · IIT Patna ’27

// agent_orchestration.live

About

Who I am

Around 5 years building multi-agent LLM systems and RAG pipelines across public sector and enterprise, along with the LLMOps that keeps them dependable: tracing, evaluation gates, guardrails, and cost-tiered routing.

I began by deploying sovereign RAG systems for the Government of Maharashtra, where auditability and grounded answers mattered more than novelty. I'm now an Assistant Manager at Deloitte, building enterprise GenAI.

At Deloitte I own the OKR and Knowledge agents within a 12-agent enterprise platform that cleared 93% client UAT across a 117-prompt adversarial suite. The orchestration was the straightforward part; the real engineering went into LLM-to-SQL reliability — schema grounding, sqlglot validation, and a semantic check that catches the silent-failure case where a query executes cleanly but returns the wrong rows.

I also led AI design for Studio, an enterprise AI avatar, voice, and script-to-video platform in staging. The pattern is LLM-proposes, code-disposes: the model plans scenes and verifies spoken consent, while deterministic code owns safety, cost-tiered provider routing, reserve-before-spend credit flow, retries, and final 9:16 assembly. I defined the production-readiness controls — identity consistency, voice similarity, credit reconciliation, provider-failure handling, idempotency, and observability.

Throughout, I treat evaluation as non-negotiable: RAGAS, DeepEval, and LLM-as-judge wired into CI gates so a regression is caught at the merge, before it reaches users. My stack centers on Azure OpenAI, Databricks, FastAPI, and Azure AI Foundry, with MLflow-based evaluation and observability. Alongside full-time work, I'm pursuing an MS in AI & Cybersecurity at IIT Patna, completing in 2027.

// a quick intro

YouTube ↗

0

Years Building LLM Systems

0

Enterprise Deployments

0

GitHub Projects

0

Certifications

Education

M.S. — AI & Cybersecurity

IIT Patna

2027

B.E. — Automobile Engineering

University of Mumbai

2018

Certifications

Microsoft AI · Coursera

AI & ML Engineering

Mar 2025 · PTX8XBG3KH6K

IBM · Coursera

Generative AI for Data Scientists Specialization

Mar 2025 · M89N894KLOV2

Vanderbilt University · Coursera

AI Agents & Agentic AI with Python

Jun 2025 · 2MUKV1I2TQAP

IBM · Coursera

Fundamentals of AI Agents Using RAG & LangChain

Jun 2025 · KQCY9L3490ZP

Google Cloud · Coursera

Build & Deploy an Agent with Vertex AI Reasoning Engine

Jun 2025 · 6C7QLACT4SQJ

Microsoft AI · Coursera

Microsoft Azure for AI & Machine Learning

Mar 2025 · 9JF7X7ZM8W2T

Akash Chaudhari · Applied AI Engineer · GenAI | Agentic AI · 2026

Skills

Technical Stack

The stack behind my enterprise work and 7 portfolio projects — production-grade where it counts, project-proven everywhere else.

Agentic AI & Orchestration

LangGraphLangChainMulti-Agent SystemsTool CallingFunction CallingHuman-in-the-LoopReActMCP Server BuildA2A Protocol

Generative AI & RAG

RAG PipelinesHybrid SearchEmbeddingsSemantic SearchAzure OpenAIHugging FacePrompt EngineeringGuardrailsSelf-RAGCorrective RAGGraphRAGLlamaIndexCross-Encoder Re-rankingAzure AI SearchCitation Validation

Multimodal & Speech

Speech-to-Text (STT)Text-to-Speech (TTS)Azure SpeechMultimodal (text/image/video/audio)Speech SynthesisText-to-ImageText-to-VideoAzure Vision

Vector Databases

FAISSChromaPineconeWeaviate

LLM Engineering & Evaluation

Structured OutputsAI-assisted Code GenLangSmithDeepEvalRAGASLLM-as-JudgePydanticAILangfuseCI/CD Eval GatesPrompt Versioning (MLflow)Regression BenchmarksPrompt-Injection Testing

Cloud, Infra & Deployment

Azure FunctionsAzure App ServiceAzure AI ServicesDatabricksDelta LakeADLSMLflowDockerAzure DevOps CI/CDFastAPIsqlglotAzure AI FoundryAzure Key VaultSemantic SQL Validation

Engineering Practices

PythonSQLREST APIsPytestLocust Load TestingStructured LoggingGitBM25 + Dense HybridPandas

The tools I build with — enterprise work and portfolio projects

Experience

Work History

Production systems across public sector and enterprise, built for real users and real workloads.

Aug 2025 — Present

Deloitte

Generative AI Engineer and Data Scientist (Assistant Manager)

Mumbai, India

Procurement GPT | OKR + Knowledge Agents

Designed and shipped production-grade LLM backend services in Python with Pydantic schema validation, structured outputs, retry logic, and circuit-breaker error handling — improving reliability across a multi-agent procurement intelligence platform.
Built a LLM-to-SQL reliability stack with dynamic schema grounding, vocabulary normalisation, embedded policy rules, and sqlglot AST-level validation with bounded self-heal loops — solving deterministic SQL generation against a 25-year time-series dataset, eliminating the silent-failure class where SQL executes but returns semantically wrong rows.
Engineered LangGraph workflows with conditional routing, multi-turn conversational analytics, and follow-up query rewriting — supporting enterprise procurement leadership across complex, multi-hop analytical sessions at production load.
Implemented MLflow as the observability spine — capturing prompts, generated SQL, execution results, retry events, and per-span latency; LangSmith tracing for agent-level audit; Pytest regression suites as CI quality gates.
Built a calibrated evaluation framework combining RAGAS metrics, LLM-as-judge scoring, and mandatory human review — cleared client UAT at 93% on a 117-prompt adversarial test set.
Built the Knowledge Agent for enterprise RAG over internal procurement policies, SOPs, playbooks, supplier-onboarding and compliance documents — owning ingestion, chunking, metadata, embeddings, and Azure AI Search hybrid retrieval.
Added semantic reranking, citation validation, and a no-answer fallback so answers stay grounded in retrieved sources, traced and evaluated with the same MLflow discipline as the OKR Agent.

Studio — AI Avatar, Voice and Script-to-Video Platform

Led AI design for an enterprise AI avatar, voice, and script-to-video platform in staging and active production-readiness testing — a bounded pipeline where the LLM plans scenes and verifies consent while deterministic code controls safety, cost, provider routing, credit reservation, retries, and media assembly.
Designed the core components — LLM scene planner, spoken-consent verification for personal avatars, locked identity/voice/wardrobe/background references, engine registry, credit calculator with reserve-before-spend, tiered provider routing, async worker pipeline with SSE progress, and final 9:16 MP4 assembly.
Defined production-readiness controls — planner schema validity, identity consistency, voice similarity, credit reconciliation, provider-failure handling, idempotency, and observability.

LangGraphMCPLLM-to-SQLLLMOpsAzureDatabrickssqlglotRAGASAzure AI SearchStudioCost-Tiered RoutingAzure Speech

Dec 2021 — Jun 2025

Government of Maharashtra

PC GenAI Analyst

Mumbai, India

Developed end-to-end NLP and Generative AI pipelines covering text preprocessing, chunking strategies, prompt engineering, and retrieval evaluation for citizen-facing public-sector applications.
Built embedding-based knowledge retrieval systems using FAISS, Chroma, and hybrid retrieval strategies — grounding LLM responses in authoritative government documentation and reducing hallucination risk in high-stakes contexts.
Designed automated document ingestion and indexing pipelines enabling continuous knowledge base updates at scale without service disruption.
Applied RAGAS evaluation frameworks to measure answer faithfulness, context relevance, and groundedness — driving iterative improvements with traceable, auditable outputs.

RAGFAISSRAGASPrompt EngineeringNLPHybrid Search

Around 5 years · 2 enterprise deployments

Projects

Featured Work

Seven projects — two enterprise production deployments and five portfolio repos, every one backed by real, runnable code.

PROJECT_01 · FLAGSHIP

Multi-Agent OKR Analytics System

Azure · LangGraph · GPT-4o · Databricks · ADLS · sqlglot · LLMOps · HITL

Multi-agent LangGraph procurement platform for a global FMCG client — cleared client UAT at 93% on a 117-prompt adversarial test set
OKR Agent owns 5-stage validation (schema, freshness, supplier-ID, numeric sanity, provenance) with HITL at validation queue
LLM-to-SQL reliability stack with sqlglot AST-level self-heal against a 25-year time-series dataset
Audit-grade ADLS pipeline with immutable artifact versioning; LangSmith for enterprise compliance

LangGraphLLMOpsAzure

PROJECT_02

Advanced RAG Chatbot — Public-Sector Knowledge Retrieval

RAG · Hybrid Search · FAISS · Chroma · RAGAS Evaluation · Govt of Maharashtra

Dense embeddings + hybrid search (semantic + keyword) over FAISS and Chroma — evidence-grounded, low-hallucination responses
Iterative prompt tuning, re-ranking strategies, and RAGAS-driven evaluation cycles
Structured fallback handling and confidence thresholds routing ambiguous inputs to human review

RAGRAGASHybrid Search

PROJECT_03

MCP Tool Server

Model Context Protocol · LangGraph · Dynamic Tool Discovery

Custom MCP server exposing domain tools with dynamic tool discovery and manifest-based calling
Standardised tool protocol enabling agent-agnostic, reusable tool servers across multiple LangGraph agents
Runtime tool registration — a rare and in-demand capability in 2026 JDs

MCPLangGraph

PROJECT_04

Hybrid RAG Engine

BM25 · Dense Embeddings · Cross-Encoder · Self-RAG · GraphRAG · Pinecone

BM25 + dense hybrid search with Reciprocal Rank Fusion — built from scratch
Two-stage retrieval: fast hybrid → accurate cross-encoder re-ranking
Self-RAG loop + GraphRAG via Microsoft open source for structured knowledge retrieval

GraphRAGSelf-RAGLlamaIndex

PROJECT_05

RAG Eval Harness with CI/CD Gate

DeepEval · RAGAS · LLM-as-Judge · GitHub Actions · MLflow

Custom LLM-as-Judge evaluator — scores faithfulness 1–5 with structured reasoning
Full RAGAS suite automated with Pytest; GitHub Actions CI/CD gate blocks deployment if faithfulness < 0.75
MLflow experiment tracking — prompt A/B comparison with metric visualisation

DeepEvalLLM-as-JudgeCI/CD Gate

PROJECT_06

Production Agent on AKS

FastAPI · Docker · Kubernetes AKS · PydanticAI · Locust · GitHub Actions

FastAPI backend wrapping LangGraph agent with PydanticAI output validation on all LLM responses
Docker containerised → Azure Kubernetes Service (AKS) via GitHub Actions CI/CD pipeline
CI pipeline runs full RAGAS eval on every commit — Locust load tested at 100 concurrent users

KubernetesFastAPIPydanticAI

PROJECT_07 ★ OPEN SOURCE

forgeSDLC — MCP SDLC Orchestration Platform

MCP Protocol · LangGraph · 14-Agent Pipeline · Electron · npm · pip

MCP server adding structured SDLC phases to Cursor, Claude Code, and Copilot — one line of config
14-agent LangGraph pipeline, HITL "100% GO" gate, PostgreSQL app DB, 5-layer cross-session memory; 392 tests green
Ships as npm/pip package AND Electron desktop app (.exe/.dmg/.deb) — v1.0.0

MCPLangGraphElectron14-Agent

GitHub

7 projects · github.com/Akash-1512

Contact