Verification Targets: Overall Conversion Rate Test
Overall Test Targets
These are the known-good values computed from the experiment dataset. Use them to verify your AI-generated statistical test results.
| Metric | Value |
|---|---|
| Test type | z-test for two proportions (or chi-squared test for independence) |
| Page A visitors | 2095 |
| Page B visitors | 2105 |
| Page A bookings | 356 |
| Page B bookings | 445 |
| Page A booking rate | 0.1699 (17.0%) |
| Page B booking rate | 0.2114 (21.1%) |
| Difference (B - A) | 0.0415 (4.1 percentage points) |
| z-statistic | 3.4207 |
| p-value (two-tailed) | 0.000625 |
| 95% CI for difference | [0.0177, 0.0652] (1.8pp to 6.5pp) |
| Effect size (Cohen's h) | 0.1057 |
How to Use These
Compare your AI-generated results against these targets:
- The p-value should match within rounding (same order of magnitude, same first two significant digits).
- The CI bounds should match within 0.1 percentage points.
- The effect size should match within 0.01.
If they don't match, check:
- Whether AI used the correct test type (z-test for proportions or chi-squared, not a t-test -- the outcome is binary)
- Whether AI applied the correct metric definition (booking_completed = true / total visitors per page version)
- Whether AI included all rows (no accidental filtering by tour type or visitor source)