Multimodal simulation: test agents across channels with real-world conditions

AI agents have moved from pilot programs to the front line. They now answer the phone, take orders and handle claims around the clock. As they take on more work, the cost of a bad interaction rises too. Poor call quality, policy changes or an updated workflow can quietly change how effectively an agent handles thousands of conversations each day.

Yet most AI agent testing, especially for voice agents handling customer service, still happens in the cleanest conditions possible: pre-crafted messages, stable testing personalities, quiet rooms, and clean simulation data. Meanwhile, real customers call from moving cars, talk over background noise and can arrive frustrated. An agent that passed in a chat test may still break on a bad phone line in the field.

Today, we’re bringing new AI agent testing capabilities to OASYS with voice simulation, testing personas, and A/B testing across channels.

Voice agent simulation: test your agent in poor conditions

Simulate voice calls handled by AI, not just a chat. Set the audio quality to clean or poor, add real background noises (like a caller driving through traffic), and recreate scenarios from real customer calls with live agents.

Every test case is reviewed turn by turn. Goal-based testing checks each step against the outcome you expected: whether the agent achieved the goal for that step, and whether the right agents and functions actually ran. OASYS also flags where conversations stalled, evaluates the audio itself, including whether speech came through clearly and was transcribed correctly, and captures detailed telemetry and observability data for every turn, such as latency and token usage. When a test does fail, you know whether the problem was what the agent decided, what it heard, or what it did. All before your AI voice agent talks to a real customer.

Personas that behave like real customers

In OASYS, you can now generate detailed test personas to use across simulations, each with its own background, emotional state and tone of voice. See how your agent handles an angry customer who wants a refund now, a confused first-time caller, or someone trying to talk their way around a policy. Choose from the generated personas, customize them, or create your own.

A/B testing across channels and environments

Run the same test up to three ways at once and compare the results side by side, across every turn. Compare chat resolution against voice, for example, so you can perfect and maximize each channel’s success. Compare what happens with a clean line versus a noisy one, or test your current live agent against an optimized version in draft. Keep one run as the baseline, and you’ll know whether a change actually made your agent better before it goes live.

Test cases from real conversations

Build tests that reflect reality faster than ever. Point OASYS at your existing environment to draft test cases for its live agents, or upload a batch of past conversations and let OASYS turn them into tests. Your test suite starts from what your customers actually say, not what your team (or a generic LLM) guesses they will say.

Transform your service with confidence

The teams that scale AI agents fastest are the ones that can change them with confidence. When every update can be tested against real voices, real moods and real conditions, there are fewer surprises in production and faster releases.

‍