Productionizing LLM Gateways: Architecture, Tradeoffs, and Hard Lessons from the Trenches