Browser checks for your agent
Your LLM provider can be online while your in-app agent is still broken. OnlineOrNot opens your product in a real browser, prompts the agent, verifies the customer-visible response, and alerts you when the experience fails.
Check script
Open app, send prompt, assert answer
await page.goto('/app/assistant')
await page.getByRole('textbox').fill('Can I export invoices?')
await page.getByRole('button', { name: 'Send' }).click()
await expect(page.getByText('billing settings')).toBeVisible()Last run passed
8.4s
Answer contained expected guidance
Retry policy
2x
Avoid pages for transient provider blips
The new failure mode
The server is up. The agent is not useful.
The chat widget renders but never answers
The model gateway returns an empty response
The agent can log in, but cannot complete the workflow
The app streams tokens for 40 seconds, then times out
How it works
Monitor the full prompt-to-response path
Treat the agent as a product surface, not an invisible backend dependency. Browser checks verify what users actually see after the model, retrieval, tools, streaming, and frontend all do their jobs.
Open the real app
Run a browser check against the same page your customers use, not a mocked health endpoint.
Ask the agent a known prompt
Click into your chat or agent UI, send a prompt, and wait for the actual in-app response.
Assert on the outcome
Check that expected text appears, the error state stays hidden, and the page does not stall.
Alert the team that owns it
Send failures to Slack, PagerDuty, Discord, email, SMS, or webhooks after retries reduce noise.
Built for LLM products
Good checks look like real user questions
Use deterministic prompts, expected answer fragments, and normal browser interactions to catch the failures that health endpoints miss.
AI support copilots
Verify your customer-facing assistant still loads context, responds, and avoids generic fallback errors.
Internal workflow agents
Monitor browser-based agents that summarize tickets, draft responses, update records, or trigger actions.
LLM-powered onboarding
Catch broken personalization flows where the app loads but the generated next step never appears.
Why browser checks
Health checks stop too early
AI features fail across several layers: auth, UI, prompts, retrieval, provider latency, tool calls, output parsing, and rendering. A browser check waits until the customer-facing answer exists.
Know when your agent stops answering
Add a browser check that asks your agent a real question, validates the response, and alerts your team before the support tickets arrive.