How do we stop using one AI setup for every request?

[Picasso]

Our customer support agent handles everything from simple order questions to refunds and account problems, but right now every request goes through the same model, prompt setup, and tools. That feels wasteful because easy questions burn the same resources as complicated ones, while changing the setup globally can make the harder cases worse. I’d like the agent to learn which configuration works best for different types of requests while still respecting rules like tool access and cost limits. Is there a good way to optimize that automatically?

[Cabanas]

One agent doing everything from where’s my order to account recovery is bound to get weird sooner or later. I’d split the traffic into a few request types first and test different model, prompt, and tool combinations against each bucket. LangSmith can help with that because you can build eval sets, compare runs, and see which setup performs better before changing anything in production. That gives you a much clearer picture of where the expensive configuration is actually earning its keep.

[Bermudo]

Give it a month and you’ll have three spreadsheets, five prompt versions, and nobody remembering why one model is attached to refunds while another handles shipping questions. That’s usually the point where tuning the agent itself needs to become part of the process. You can do agent optimization here: https://eignex.com/ . It can compare different configurations against the goals and limits you set, then surface which ones perform better for different request types. That keeps the experimentation moving without turning every change into another round of hand-testing.