Purvesh Patidar
I build low-latency, production-grade LLM systems with real-time streaming and deterministic orchestration.
85 tokens/sec streaming • <50ms TTFT • 95% test coverage
Architecting for
Scale & Reliability.
State & Infra
MongoDB, Redis, Docker, WebSocket tuning handling 1k+ daily peak CCUs.
Architecture
Node.js/FastAPI hybrid services for optimized ML inference & reliable data delivery.
Product-focused
Generative AI.
- Predictive Forecasting94% Accuracy
- Course Recommendation+22% Engagement
- Sentiment ClassifierStreamlit UI
Production Experience
Engineering Case Studies
Production systems with measurable performance, real constraints, and engineering tradeoffs.
Impact
Performance you can feel
Measured on GPT-4-class models with ~20 concurrent SSE clients (representative load).
Measured from request accepted to first SSE token (warm path).
Integration + evaluation harness coverage across agent flows and regressions.
Retrieval + generation + response assembly for document Q&A scenarios.
Retrieval step latency for typical query vectors (embedding precomputed).
Engineering Decisions
The choices behind the outcomes
Why LangGraph over traditional agents
I prefer explicit graphs over opaque agent loops. Nodes and state transitions make orchestration deterministic, debuggable, and testable under real failure modes.
Tradeoff: It requires up-front workflow modeling and ongoing state-schema maintenance as capabilities evolve.
Why SSE for streaming
For real-time token streaming, SSE keeps the path simple and HTTP-native. It works cleanly with typical proxies/LBs and keeps the client implementation lightweight for server->client updates.
Tradeoff: SSE is not full duplex; if you need true bidirectional messaging, WebSockets can be a better fit.
Why ChromaDB vs alternatives
For many RAG workloads, I optimize for iteration speed and predictable retrieval behavior. ChromaDB is a pragmatic vector store for development and smaller deployments without heavy operational overhead.
Tradeoff: At higher scale or strict HA needs, you may need to migrate to a managed/distributed vector database.
Why deterministic outputs (and validation) matter
I avoid systems that cannot be reliably parsed, tested, or monitored. Structured outputs plus validation make failures explicit and keep UX stable as models drift.
Tradeoff: Stricter constraints add prompt complexity and may require retries when the model deviates from the schema.
How I Think
Opinionated engineering, shipped.
I build production LLM systems that are fast, inspectable, and predictable under load. If it cannot be measured or tested, it does not ship.
What I optimize for
- Low latency and fast time-to-first-token
- Deterministic orchestration over "magic"
- Observability: traces, metrics, and replayable evals
What I avoid
- Black-box agents with unclear control flow
- Untestable prompts and "it works on my machine" demos
- Latency surprises hidden behind async complexity
How I approach tradeoffs
- Pick the simplest architecture that meets the SLO
- Make the failure modes explicit (timeouts, fallbacks, budgets)
- Measure first, then optimize with guardrails
Active R&D
Currently Building
Exploring agent orchestration patterns that balance flexibility with deterministic control.
- ▸ Multi-step agent workflows with observable state transitions
- ▸ Cost-aware RAG pipelines with dynamic retrieval strategies
- ▸ Evaluation systems for measuring LLM output quality at scale
Focus is on making these systems production-ready — not just experimental.