Harshul Jain

Harshul Jain

Senior Software Engineer - ML/AI · Audible

Twitter / XLinkedInWebsiteBlog
Share

Bio

Harshul Jain is a Senior Software Engineer at Audible (Amazon) who builds ML and LLM infrastructure at scale — AI Search serving 10M users, a feature store processing 100K transactions per second, and LLM serving and evaluation systems powering GenAI in production. He is writing LLM Inference at Scale, a benchmark-driven handbook on GPU memory engineering, attention optimization, and production LLM serving backed by a companion repository that gained 100+ clones in its first week with zero promotion.

Sessions (2)

Day 112:10pm-1:10pmTrack 3 · Room 2003
2 hr deep dive on LLM Inference at Scale — Part 1 of 2
Day 11:15pm-2:15pmTrack 3 · Room 2003
2 hr deep dive on LLM Inference at Scale — Part 2 of 2

Additional Speaker Details

Using publicly available information we constructed an analysis to help you get a feel for this speaker before deciding to attend their session.

Harshul Jain is a Senior Software Engineer at Audible (Amazon) working on ML/AI infrastructure and production agent systems. He is the creator of AgentShip, an open-source, runtime-agnostic framework for writing AI agents that run on ADK, LangGraph, or other engines without rewrites, and he writes a practitioner handbook on serving LLMs at scale. Expect a hands-on, practitioner view of avoiding framework lock-in and production LLM serving, drawn from his open-source work.

Speaking style

Battle-testedDemo-drivenClear teacher

GitHub

@harshuljain13

Recent writing (6)