Purpose of this document: Define the manual and automated test coverage
required for a credible all-platform Steam release of Conversation Simulator.
This is the QA companion to steam-mvp-scope.md (feature
gates) and release-checklist.md (release smoke steps).
Minimum tier: text-only is the default recommendation; app must surface a visible performance warning when first-token latency exceeds 10 s; STT/TTS disabled by default.
Starter tier: app recommends a 4 B model and CPU+GPU split; performance warning at > 5 s.
All tiers: a turn that times out after 60 s must produce an in-app error with a recovery action; the session must remain open for retry.
Out-of-memory: the app must display a readable error message; it must not crash silently or corrupt the session database.
Steam Deck is a required platform for the Stage 4 public release. Valve
must grant Verified (not merely Playable) tier before the public release gate
opens.
Each item must be checked by a tester on physical Steam Deck hardware running
SteamOS 3.x in Gaming Mode before G4-02 can be declared PASS.
Check
Controller-only test
Result
App launches in Gaming Mode from the Steam library without extra setup.
Use controller to launch from library
[ ] PASS / [ ] FAIL
Home, Library, Setup, Conversation, Debrief, Model Manager, Settings, Support, and Workbench are fully navigable with the controller alone.
TC-11 full walkthrough
[ ] PASS / [ ] FAIL
Every interactive element has a visible focus ring at 1280×800 from couch distance.
Inspect each screen during TC-11
[ ] PASS / [ ] FAIL
On-screen keyboard appears automatically for every text input field (no manual invocation needed).
TC-11.5 and TC-11.7
[ ] PASS / [ ] FAIL
R1 (right shoulder) triggers push-to-talk in voice mode.
TC-11.6
[ ] PASS / [ ] FAIL
All text is readable at 1280×800 (native resolution) without zooming or horizontal scrolling.
Inspect transcript, meters, debrief
[ ] PASS / [ ] FAIL
No required action is hidden behind a mouse-only hover state.
All screens — controller only
[ ] PASS / [ ] FAIL
DebugDrawer is excluded from the controller focus ring (dev builds only).
TC-11.11
[ ] PASS / [ ] N/A
Default Steam Input config (steam/controller_config.vdf) is loaded automatically.
Launch game; check Steam Input overlay
[ ] PASS / [ ] FAIL
Offline smoke test passes under SteamOS 3.x (no outbound network during play).
TC-09 on Steam Deck
[ ] PASS / [ ] FAIL
Steam overlay (Shift+Tab) visibly composites over the app and closes without breaking the current session. On Windows, “nothing happened” (no visible overlay) is a FAIL even if Steam reports the player In-Game — see TC-11.12.
TC-11.12
[ ] PASS / [ ] FAIL
Approximate battery impact documented (see §5 performance thresholds).
Each test case below maps to one or more acceptance gate IDs from
steam-mvp-scope.md and/or steps in
release-checklist.md. Automated means the case is
or should be covered by CI; Manual means it requires a human tester on real
hardware.
Full end-to-end play from launch to quit using the controller alone — no
keyboard or mouse contact. All steps must be completable with D-pad, A/B/X/Y
buttons, left stick, and the on-screen keyboard only. Corresponds to the
controller-only column added to §3 (Steam Deck verification checklist).
#
Step
Pass criterion
Method
11.1
Launch app in Gaming Mode from Steam library
App reaches Home screen; focus is placed on the first interactive element
Manual — Steam Deck hardware
11.2
Navigate Home screen with D-pad / left stick
Every link and status badge reachable; a visible focus ring appears on each element at 1280×800 from couch distance
Manual — Steam Deck hardware
11.3
Navigate to Scenario Library via controller
Focus moves correctly through nav links; Scenarios screen loads
Manual
11.4
Select a scenario pack and choose a scenario
Card selection works; Setup screen reached with no keyboard/mouse
Manual
11.5
Enter a player name or adjust setup field with on-screen keyboard
Steam floating keyboard appears automatically on text field focus; dismissed on confirm
Submit at least one text turn using the on-screen keyboard
On-screen keyboard appears on input focus; text submitted; NPC responds
Manual
11.8
End the session and navigate to the Debrief screen
End session button reachable via controller; Debrief loads
Manual
11.9
Navigate Debrief screen and confirm readability
All debrief scores and text readable at 1280×800; no clipped controls
Manual
11.10
Navigate to Model Manager, Settings, Support, and Workbench screens
Each screen fully navigable; no interactive element requires mouse hover
Manual
11.11
Confirm DebugDrawer is excluded from controller focus ring
DebugDrawer (dev-mode only) never receives focus during D-pad navigation
Manual — dev build only
11.12
Open and close Steam overlay (Shift+Tab) during a session
Overlay is visibly drawn over the app and closes; controller focus resumes correctly in the app. Windows: the overlay is a silent no-op unless the app forwards the chord and provides a decoy compositing surface — Steam reports the player In-Game / is_overlay_enabled() true / GameOverlayActivated fired even when nothing is drawn, so treat an invisible overlay as FAIL. Record GPU, Windows build, and whether the surface came up (see Steam integration § Steam overlay (Windows WebView2 caveat)).
These thresholds are pass/fail criteria for QA sign-off. A platform + tier combination that cannot meet the minimum threshold is a blocker, not a deferral.
Metric
Minimum tier threshold
Starter tier threshold
Recommended tier threshold
Notes
Session start (first NPC line)
≤ 60 s
≤ 20 s
≤ 5 s
Timed from “Start session” button press
First token latency
≤ 15 s
≤ 5 s
≤ 2 s
From player turn submission to first streamed token
Full NPC response
≤ 90 s
≤ 30 s
≤ 10 s
From player turn to complete response
Debrief generation
≤ 30 s
≤ 15 s
≤ 5 s
From “End session” to debrief screen ready
App cold-start to home screen
≤ 10 s
≤ 5 s
≤ 3 s
Model not counted; model load is separate
Model load (first load after download)
≤ 120 s
≤ 60 s
≤ 20 s
One-time; subsequent loads may be faster via cache
When first-token latency exceeds the threshold for the detected tier, the app must surface an in-app performance tip linking to Runtime Settings.
When RAM is insufficient for the selected model, the app must reject the model load with a clear “Insufficient memory” message before any OOM condition occurs.
When VRAM is insufficient for full GPU offload, the app must fall back to CPU+GPU split or CPU-only mode automatically and inform the player.
All degradation messages must be actionable (link to settings or model manager) and must not expose a stack trace to the player.
A minimum of five testers covering the required platforms must complete at
least one full text session before the Stage 3 gate (G3-06) can be declared
PASS.
Tester receives a Steam private beta key from the Outright Mental publishing account.
Tester installs the app from the Steam library and completes the first-run model setup.
Tester completes the session verification steps in TC-03 and TC-06 and files the result in the beta feedback form (link TBD).
Maintainer reviews results; any session-ending, data-loss, or privacy-regression bug blocks the Stage 3 gate.
Note: Machine and tester assignments will be updated in this table when
the Stage 3 beta programme launches. If you are an Outright Mental team member
with access to a listed platform, open a GitHub issue with the label
beta-testing to register.
Copy this template and fill it in for each platform before declaring the
corresponding release gate PASS. One completed sign-off is required per
platform per gate stage.
After all per-platform sign-offs above are complete, combine them into the
aggregate beta verification report using the template in
docs/STEAM_BETA_VERIFICATION_REPORT.md.
The completed report must be attached to the release GitHub issue before the
Stage 3 gate (G3-06) can be declared PASS.
Before opening the Stage 4 gate, the publishing owner must complete the
following Valve-facing sign-off in addition to the platform QA sign-offs above.
All platform QA sign-offs (§7 above) are filed and show PASS for Windows, macOS Apple Silicon, macOS Intel, Linux, and Steam Deck.
Valve has approved the store page for public release (G4-01).
Valve has granted the Steam Deck Verified tier (G4-02).
Steam depot content audit (G3-02) confirmed: no model weights in depot.
Store page reviewed: no claims of AI therapy, diagnosis, or legal advice; local-first copy accurate (G4-04).
All compliance checklist items SR-01 through SR-08 signed off (G3-05).
release-checklist.md — Parts A–D platform smoke matrix; Parts E (Windows), G (macOS), H (Linux/SteamOS), I (Steam Deck), J (sign-off) for Steam beta verification
performance.md — hardware tier definitions and latency guidance
