AI PRODUCT PRE-FLIGHT
AI product launch readiness checklist
An AI launch needs product evidence and model evidence. Verify useful outcomes, controlled failure, privacy boundaries, unit economics, and traceable runs before scaling traffic.
User outcome
- The product defines a measurable successful outcome, not only a model response.
- Representative prompts include happy paths, ambiguity, refusal, and recovery.
- Users can tell what the system did and what they should do next.
Reliability and safety
- Timeouts, provider errors, malformed output, and rate limits have tested fallbacks.
- High-impact actions require the appropriate confirmation or human review.
- Prompt injection and untrusted retrieved content are treated as data, not instructions.
Privacy and data
- The product discloses what user content is sent to model providers.
- Logs and replays redact secrets, credentials, and sensitive input.
- Retention and deletion behavior match the public policy.
Cost and performance
- Token, tool, image, audio, and retry costs are attributed per successful outcome.
- Hard caps prevent one user or loop from creating unbounded spend.
- Latency is measured end to end, including tools and queues.
Observability and evaluation
- Every run has a traceable session, model, version, tools, result, and failure class.
- Quality evaluation uses a named dataset and rubric.
- Synthetic evaluation traffic is labeled separately from real customers.
Audit the public launch surface
LaunchReady checks the live experience and produces prioritized fixes your coding agent can implement.
Run a free AI product launch scan