Improving AI UX through *product* focused evals

When you are building an agent or AI workflow, how the experience feels is a combination of LLM, tool calling, integrations, sandboxes, and more. It’s as much about what the model is doing as what is happening before and after it responds.

I’ll be diving into what I built to track this end to end. And what I have learned along the way. From latency to learnings ;)