idx.002 · system
5s → 148ms
Rebuilding a forecasting engine's hot path so the answer was already waiting before the request arrived.
The engine forecasts across more than a million orders a day in thirty-five markets. The model was fine. The problem was where the work happened: every request assembled its features and ran the model from scratch, and under load the tail crept toward five seconds — too slow to be worth waiting for.
Most of that time wasn't the model. It was the feature work around it — the same expensive aggregations recomputed on nearly every call, for inputs that barely moved between requests.
So I moved the work off the request and ahead of it: precompute the features on a schedule, keep them in a compact store, serve them from memory. The request path shrank to a lookup and one model evaluation. Median fell from about five seconds to 148ms, and the tail came down with it — the worst case went from unbounded computation to a bounded read.