The playground lets you safely test your AI agent before deploying it with real customers. You can experiment with responses, test guidance, and ensure everything works correctly without affecting actual customer interactions.
What is the playground?
The playground is a simulation environment where you can chat with your AI agent using real customer data from your Shopify store. Test different scenarios, validate responses, and refine your AI agent's behavior before enabling it on live conversations.
You can find the playground by navigating to AI agent → Playground or simply clicking the Test button on other AI agent pages, which opens the playground on the side of the page.
Good to know: Order lookups only work when the Playground contact is a real customer with orders in Shopify. On the Playground page you select a contact before you write the first message. In the side panel that opens from the Test button, a test contact without orders is used unless you select one. To test order questions, select a customer who has orders, or click Test under a customer message in a real conversation.
Testing your AI agent
Chat directly
Type questions and messages to see how your AI agent responds. Test various customer scenarios and language patterns to ensure appropriate responses.
Replay existing conversations
Import previous customer interactions using the Test button found below any customer message in your conversation history. This lets you test how your AI agent would handle real-world scenarios with real order data from your actual support tickets.
Select channel
Choose which channel you want to simulate: Chat, DMs, Email or Social-media comments. The AI agent adapts its tone to the selected channel and only uses guidance that lists that channel under Apply to channels.
Simulate customers
Pick specific customers to simulate by searching their name or email address. This is required for testing order lookups and Shopify actions, since the AI agent verifies the customer's identity against Shopify before sharing order details. Select a customer with recent orders to use their real order data in your test.
Which guidance runs in the Playground
- Guidance with the status Testing or Enabled runs. Disabled guidance never runs, in the Playground or live.
- A guidance only runs when its Apply to channels list includes the channel you selected. The Playground starts on Email.
- If an enabled guidance has unpublished changes, the Playground uses your draft. Customers get the published version until you publish.
- Every test message uses your guidance and knowledge as they are at that moment. Replies in an older test conversation stay as they were, also when you disabled the guidance later.
What is simulated and what is real
Nothing in the Playground reaches a customer. Playground conversations don't count towards your usage, and you can test while the AI agent is switched off.
Simulated. The step appears as a line that starts with "[Simulated]" and nothing changes:
- Handovers. You see a line that starts with "[Simulated] Escalation to agent" and the AI agent stays in the test conversation, so you can keep testing.
- Assigning to a teammate or team, labels and status changes.
- Order changes in Shopify: cancellations, refunds, reshipments, address changes and item changes.
- Subscription changes in Loop, Recharge, Skio, Stay AI, Appstle and Kaching.
Real:
- Order and customer lookups read the selected contact's Shopify data.
- Knowledge and product searches use your current knowledge.
- Custom actions call your real endpoint. Test those with care.
- Private notes are written, but only into the Playground conversation.
Two things differ from live conversations. The check whether an order has shipped always answers "not shipped", so a cancellation can go through in a test and be refused live. And the customer's earlier conversations are not loaded.
Common problems
The AI agent says it handed over, but no handover shows
A real handover never happens in the Playground. Look for the line that starts with "[Simulated] Escalation to agent". If it is missing, the AI agent did not decide to hand over. Check that a guidance matched and that it lists the channel you are testing. Write the handover as its own step with a clear condition, for example "If no order is found, hand over."
A private note, label or assignment from my guidance is missing
Labels and assignments are simulated, so look for a line that starts with "[Simulated]". Private notes are written into the test conversation. If you see neither, the AI agent did not take that step. Write each one as a separate step in your guidance, in the order you want them, and name the person, team or label.
Managing AI knowledge
If you notice issues like repetitive canned responses or inappropriate knowledge sources, visit the Knowledge page. You can exclude or edit problematic knowledge sources to help your AI agent respond more naturally and effectively.
Regular testing in the playground helps identify knowledge gaps and ensures your AI agent provides consistent, accurate responses across all customer interactions.