Learn by Directing AI
All materials

verification-targets.md

Verification Targets — Appointment No-Show Analysis

Use these targets to check AI output at each stage. Every computed value should be compared against the expected range below. If the result falls outside the range, investigate before accepting it.

1. Overall No-Show Rate

Expected range: 12-15% (low teens).

Confidence interval width: Roughly +/- 1-2 percentage points. With ~8,000 records, the interval should be narrow.

Common AI error: AI uses total records (including cancellations) as the denominator instead of scheduled appointments only. This produces a rate of approximately 9-11% — noticeably lower than the expected range. If the rate comes back below 12%, check the denominator. The correct denominator excludes advance cancellations because the no-show rate measures "of the people expected to show up, how many didn't?"

2. Visit Type Breakdowns

Expected pattern: Vaccination follow-ups should show the highest no-show rate among visit types, consistent with Wanjiku's intuition.

Approximate ranges by visit type:

  • Vaccination: highest (notably above the overall rate)
  • Consultation: near or slightly below the overall rate
  • Dental: below the overall rate
  • Surgery: lowest (people tend to show up for procedures they actively scheduled)

Common AI error: AI reports the rates without confidence intervals, or reports intervals but buries them in a table footer rather than alongside each rate.

3. Chi-Square Test

Expected result: Significant (p < 0.05). The differences in no-show rates across visit types are not due to chance.

Common AI error: AI misinterprets the p-value in its narrative. Watch for: "p = 0.03 means there is a 3% chance the null hypothesis is true." The correct interpretation: "if there were no relationship between visit type and no-shows, we would observe a pattern this extreme only 3% of the time."

4. Client Tenure

Expected pattern: New clients should show a higher no-show rate than returning clients.

Common AI error: AI computes the breakdown but does not highlight the practical significance — the difference between new and returning client behavior has real implications for how the clinic handles first-time bookings.

5. Temporal Trend

Expected pattern: Relatively stable over the 18-month period. No dramatic increase or decrease. Normal month-to-month variation of +/- 2-3 percentage points around the overall rate.

Common AI error: AI fits a trend line and reports a statistically significant slope when the practical effect is negligible (e.g., a slope of 0.1 percentage points per month). The question is not "is the slope technically non-zero?" but "would Wanjiku notice a change?"

6. Data Shape

Expected: Approximately 8,000 rows and exactly 8 columns. Column names and data types match the data dictionary. Three categories in appointment_status (Show, No-show, Cancelled). Missing values minimal (under 1%) or absent.

7. Notebook Reproducibility

Expected: The notebook runs cleanly on "restart and run all." All cells execute in order without errors. Results are identical to those produced during the analysis.