How Checkout Doctor's test browser works

What it does

On a schedule, a real browser opens your storefront as a desktop window and as a phone. It loads the home page, opens a product, adds it to the cart, opens checkout, fills the contact and shipping fields with a fixed test contact, clicks continue, and reads the shipping rates and totals the page shows. The test contact is: email qa@agntwrk.com, name QA Check, address 350 5th Ave, New York, NY 10118, phone 2125550100. Shipping tests use other sample addresses.

Paid plans also apply your discount codes, compare the price on the page with the price in your admin, check the shipping rate for sample destinations, and try a product that your admin says is in stock. Each step is timed and screenshotted, and a video and the browser's console are kept.

What it never does

It never enters card or wallet details and never places an order. The test browser cannot submit a payment or complete a checkout, whatever the test does.

It only visits the store that installed the app, and it refuses private and internal addresses.

What your store will see

Carts with one item, checkout page visits, and sessions from a browser whose user agent ends in the suffix below. Because the checkout test types the test contact above, Shopify can record some of these sessions as abandoned checkouts with the email qa@agntwrk.com and the name QA Check; filter on either. Nothing is ever paid or ordered.

How it identifies itself

The user agent of every request ends with the text below, on desktop and phone. The first page of each test also carries the query parameters utm_source=checkoutdoctor and utm_medium=monitor, unless the URL already has a utm_source.

CheckoutDoctor/1.0 (+https://checkoutdoctor.agntwrk.com/bot)

Filter it out of analytics

Google Analytics 4 does not receive the user agent, so use the campaign: create a data filter or an audience exclusion where Session source / medium equals checkoutdoctor / monitor.

In Shopify analytics, filter reports by UTM source checkoutdoctor. In Shopify's bot protection or a firewall, you can allow the CheckoutDoctor/1.0 user agent.

Store journeys

On paid plans the app also learns journeys through your own store: it reads a few public pages (home, a collection, a product, search, the cart, a few menu pages) and proposes paths such as a search or a variant picker.

Journeys run in the same test browser, with the same protection against paying or ordering. They never type an email, name, phone, address, message, card or password, so they cannot create signups, contacts or orders, and they stop at checkout. The number kept per store is 5, 10 or 20 depending on the plan.

Passwords and bot protection

A password-protected storefront is signed in once per test using the storefront password you save in the app's settings. The password is never logged or put in screenshots. When a store answers with a password page, a captcha, or a firewall challenge, the tests are recorded as blocked, not failed, and a finding explains why.

Crawler access: in Shopify admin (Online Store, Preferences, Crawler access) you can create a signature, three request headers for one domain of your store that let a crawler past Shopify's bot protection. Paste them into the app's settings and the test browser attaches them to requests to that domain (and its www twin) only, never to other sites. A signature lasts at most 3 months and cannot be renewed, so the app warns you 14 days before it ends. The values are stored for your store only and are never logged or shown again.

Shopify does not let any crawler into checkout, signed or not. The storefront tests therefore cover the storefront and the cart, up to the checkout page, and checkout pricing, discounts, and shipping are verified separately, without an order.

Questions

Write to hello@agntwrk.com.