Independent field notes · Automated traversal testing

turbotrolleynebul.living

Route atlas / 04

Turn traces into findings

Keep enough evidence to explain what happened, reproduce the conditions and test a repair. A large log archive is not useful if nobody can connect it to the failed route.

Inspect the testing pathway

Preserve the test contract with the trace

A useful evidence bundle begins with the question the run was meant to answer. Include the route identifier, start and goal, allowed actions, success predicate and termination rules. Add build and map identifiers, agent configuration, platform and relevant world-state inputs. Without that context, an apparently blocked character might be correctly refusing an unavailable interaction, waiting for a scheduled event or pursuing a different destination.

Give every run a stable identifier and use it across telemetry, video and report attachments. Record the harness version as well as the game version. A change in the runner’s success tolerance can alter reported outcomes without any product change. Keep the original failing artifacts intact and attach reduced reproductions separately so investigation does not overwrite the evidence that first revealed the issue.

Use a shared time reference

Log meaningful events against a consistent clock or simulation tick. Path requests, path responses, input commands, interaction events, world-state changes and termination should be orderable. For distributed tests, document clock alignment and any uncertainty. Wall-clock timestamps alone may not reveal whether a door became traversable before or after the agent attempted to pass through it.

OpenTelemetry offers concepts for correlating traces and events across systems; a game-specific harness can borrow those ideas without adopting an external service. Chrome Trace Event format and the Perfetto project provide other useful references for timeline inspection. Choose a format your team can retain and inspect locally. The important property is correlation, not the name of the visualization tool.

Expandable field notes

A trace should expose its limits

These are conceptual reporting examples, not recorded test results or performance measurements.

Observation note — controller requests movement without progress
ROUTE: warehouse-doorway / conceptual
PRECONDITION: door open; standard walking profile
EVENT ORDER:
  route request accepted
  corridor returned through doorway
  forward input applied
  position remains near doorframe
OUTCOME: progress condition expired
MISSING: contact manifold and collision-shape overlay

The note identifies an execution boundary. It does not establish whether the navigation representation, collision geometry or movement settings are incorrect.

Hypothesis note — clearance mismatch

Compare the character’s collision shape with the doorway’s effective clearance and the navigation agent settings. Repeat the approach without changing the original artifacts. Treat a successful smaller character as supporting evidence, not a complete diagnosis.

Verification note — preserve the original approach

Rerun the original route and profile after the proposed repair. Add a reverse approach and a nearby control route. Record both direct completion and recovery actions so a new workaround does not masquerade as a clean repair.

Capture spatial context selectively

Record positions and orientations at a cadence that can resolve the behavior under investigation. Include coordinate conventions and units. A navigation overlay, collision overlay and camera view can make a doorway issue legible, but they should identify the same moment in the run. An unlabeled screenshot of the map is less useful than a small annotated view showing the planned corridor and last progress point.

Use event-triggered detail when continuous high-volume capture is expensive. Retain a bounded history before a stuck condition and enough data afterward to understand recovery. Verify that logging itself does not materially change timing-sensitive behavior. If diagnostic instrumentation changes the outcome, preserve that fact and compare lighter-weight capture rather than silently presenting the instrumented run as equivalent.

Write observations before assigning causes

Structure the finding as expected behavior, observed behavior, reproducibility conditions, impact and evidence. Keep hypotheses in their own section. “The capsule contacts the frame while forward input continues” is more defensible than “the AI is confused.” Assign likely ownership only after comparing the relevant systems. Clear boundaries help navigation, physics, animation and level-design teams investigate without arguing over a premature label.

Deduplicate failures using more than their final coordinates. Different causes can converge on the same doorway, while one underlying defect can appear at many locations. Combine route identity, failure phase, movement profile and a meaningful event signature. Review representative cases before merging them. A neat dashboard is not worth losing a distinct progression blocker inside a broad “stuck agent” cluster.

Verify the repair and retain proportionately

Link the proposed repair to the failing case, then rerun the original conditions and relevant neighboring cases. Compare completion, route choice, timing category and recovery behavior. A pass obtained through an unintended shortcut may not satisfy the original contract. If nondeterminism remains, report the observed attempts and their conditions rather than reducing the result to a single reassuring label.

Retain enough evidence for regression analysis while limiting unnecessary data. Test logs can expose internal map names, unreleased assets, user identifiers or secrets embedded in command lines. Redact before sharing and apply your organization’s retention and access rules. This site’s contact tool is deliberately local; do not paste confidential source code, credentials or proprietary traces into a message without authorization.