Purpose of this document: Define the manual and automated test coverage
required for a credible all-platform Steam release of Conversation Simulator.
This is the QA companion to steam-mvp-scope.md (feature
gates) and release-checklist.md (release smoke steps).
Minimum tier: text-only is the default recommendation; app must surface a visible performance warning when first-token latency exceeds 10 s; STT/TTS disabled by default.
Starter tier: app recommends a 4 B model and CPU+GPU split; performance warning at > 5 s.
All tiers: a turn that times out after 60 s must produce an in-app error with a recovery action; the session must remain open for retry.
Out-of-memory: the app must display a readable error message; it must not crash silently or corrupt the session database.
Steam Deck is a required platform for the Stage 4 public release. Valve
must grant Verified (not merely Playable) tier before the public release gate
opens.
Each test case below maps to one or more acceptance gate IDs from
steam-mvp-scope.md and/or steps in
release-checklist.md. Automated means the case is
or should be covered by CI; Manual means it requires a human tester on real
hardware.
Full end-to-end play from launch to quit using the controller alone — no
keyboard or mouse contact. All steps must be completable with D-pad, A/B/X/Y
buttons, left stick, and the on-screen keyboard only. Corresponds to the
controller-only column added to §3 (Steam Deck verification checklist).
#
Step
Pass criterion
Method
11.1
Launch app in Gaming Mode from Steam library
App reaches Home screen; focus is placed on the first interactive element
Manual — Steam Deck hardware
11.2
Navigate Home screen with D-pad / left stick
Every link and status badge reachable; a visible focus ring appears on each element at 1280×800 from couch distance
Manual — Steam Deck hardware
11.3
Navigate to Scenario Library via controller
Focus moves correctly through nav links; Scenarios screen loads
Manual
11.4
Select a scenario pack and choose a scenario
Card selection works; Setup screen reached with no keyboard/mouse
Manual
11.5
Enter a player name or adjust setup field with on-screen keyboard
Steam floating keyboard appears automatically on text field focus; dismissed on confirm
These thresholds are pass/fail criteria for QA sign-off. A platform + tier combination that cannot meet the minimum threshold is a blocker, not a deferral.
Metric
Minimum tier threshold
Starter tier threshold
Recommended tier threshold
Notes
Session start (first NPC line)
≤ 60 s
≤ 20 s
≤ 5 s
Timed from “Start session” button press
First token latency
≤ 15 s
≤ 5 s
≤ 2 s
From player turn submission to first streamed token
Full NPC response
≤ 90 s
≤ 30 s
≤ 10 s
From player turn to complete response
Debrief generation
≤ 30 s
≤ 15 s
≤ 5 s
From “End session” to debrief screen ready
App cold-start to home screen
≤ 10 s
≤ 5 s
≤ 3 s
Model not counted; model load is separate
Model load (first load after download)
≤ 120 s
≤ 60 s
≤ 20 s
One-time; subsequent loads may be faster via cache
When first-token latency exceeds the threshold for the detected tier, the app must surface an in-app performance tip linking to Runtime Settings.
When RAM is insufficient for the selected model, the app must reject the model load with a clear “Insufficient memory” message before any OOM condition occurs.
When VRAM is insufficient for full GPU offload, the app must fall back to CPU+GPU split or CPU-only mode automatically and inform the player.
All degradation messages must be actionable (link to settings or model manager) and must not expose a stack trace to the player.
A minimum of five testers covering the required platforms must complete at
least one full text session before the Stage 3 gate (G3-06) can be declared
PASS.
Tester receives a Steam private beta key from the Outright Mental publishing account.
Tester installs the app from the Steam library and completes the first-run model setup.
Tester completes the session verification steps in TC-03 and TC-06 and files the result in the beta feedback form (link TBD).
Maintainer reviews results; any session-ending, data-loss, or privacy-regression bug blocks the Stage 3 gate.
Note: Machine and tester assignments will be updated in this table when
the Stage 3 beta programme launches. If you are an Outright Mental team member
with access to a listed platform, open a GitHub issue with the label
beta-testing to register.
Copy this template and fill it in for each platform before declaring the
corresponding release gate PASS. One completed sign-off is required per
platform per gate stage.
After all per-platform sign-offs above are complete, combine them into the
aggregate beta verification report using the template in
docs/STEAM_BETA_VERIFICATION_REPORT.md.
The completed report must be attached to the release GitHub issue before the
Stage 3 gate (G3-06) can be declared PASS.
Before opening the Stage 4 gate, the publishing owner must complete the
following Valve-facing sign-off in addition to the platform QA sign-offs above.
All platform QA sign-offs (§7 above) are filed and show PASS for Windows, macOS Apple Silicon, macOS Intel, Linux, and Steam Deck.
Valve has approved the store page for public release (G4-01).
Valve has granted the Steam Deck Verified tier (G4-02).
Steam depot content audit (G3-02) confirmed: no model weights in depot.
Store page reviewed: no claims of AI therapy, diagnosis, or legal advice; local-first copy accurate (G4-04).
All compliance checklist items SR-01 through SR-08 signed off (G3-05).
release-checklist.md — Parts A–D platform smoke matrix; Parts E (Windows), G (macOS), H (Linux/SteamOS), I (Steam Deck), J (sign-off) for Steam beta verification
performance.md — hardware tier definitions and latency guidance
