Shipping AI Features That Feel Fast and Stay Cheap

Adding an AI feature to a web app looks like an afternoon of work: call the API, render the response, ship. The afternoon version also feels broken. It hangs behind a spinner, dumps a wall of text all at once, has no idea what to do when the request fails, and quietly runs up a bill nobody put in the budget. Good AI features are mostly restraint and craft wrapped around a model call that takes ten lines.

Stream, or do not bother

Ten seconds behind a spinner reads as a bug even when nothing is wrong. The same ten seconds with words arriving as they are generated feels alive. Streaming is the highest-return change available for perceived performance: it converts dead time into visible progress and tells the user immediately that the system heard them. If the feature is not streaming, fix that before optimizing anything else.

Perceived speed beats measured speed

Users never experience your p95. They experience waiting. Time to first token, optimistic UI, and progressive rendering shape that far more than shaving a few hundred milliseconds off total generation.

Design the failures, all of them

Models time out. Rate limits hit at the worst possible moment. Content gets filtered. The connection drops mid-stream. Each of those needs a real state in the interface, not a generic crash. Say what happened, offer a retry, and never lose what the user typed. A feature that fails clearly reads as trustworthy. One that throws a stack trace reads as a beta nobody asked to join.

• Stream responses; a spinner is not a loading state for AI
• Design explicit loading, empty, error, and rate-limited states
• Preserve the user’s input across every failure path
• Set timeouts and degrade to something useful when the model is slow

Control the cost curve early

A static feature costs the same whether ten people or ten thousand use it. An AI feature does not. Without controls, success is the expensive outcome. Cache repeated queries, debounce as the user types, use a small model wherever a small model is enough, and set per-account limits before you need them. The aim is cost that scales with value rather than with enthusiasm.

Put a limit on the input, too

Context length is a cost multiplier hiding inside a text field. Cap what can be pasted in, trim what gets forwarded, and summarize long histories instead of resending them every turn. Most runaway bills start as a conversation history nobody thought to truncate.

“An AI feature that feels instant and costs pennies beats a smarter one that hangs and bleeds money. The difference is craft, not the model.”

Make it feel native

The AI features people keep using do not announce themselves with a glowing badge and a sparkle icon. They appear where the user already was, do one thing well, and get out of the way. Restraint is the whole difference between a feature that becomes a habit and one that gets tried once.