Validating Performance Across Target Hardware

Execute automated performance tests on target devices to verify that optimization strategies successfully meet the project's target frame rate and memory budgets.

In this lesson, you will learn how to transition from manual profiling to automated performance testing to ensure your game consistently hits its frame rate and memory targets on actual hardware.

Optimization efforts often fail because they are tested against an "ideal" development machine. To validate performance effectively, you must automate the testing process across your target hardware (e.g., specific console devkits or mobile test devices) to catch regressions early.

The Automation Workflow

The goal is to simulate realistic gameplay scenarios through Functional Testing combined with Automation Spec Runners. By creating automated test maps, you can spawn actors, trigger heavy animation sequences, and force streaming transitions while capturing telemetry data.

Instead of relying on a human to play, you use the Gauntlet automation framework to deploy a build, run a test map, and record performance stats. You can use the Stat system commands to output results to a CSV file automatically during these runs:

// Example of triggering a performance capture via console command
// You can execute this via automation script at specific map coordinates
GetWorld()->Exec(GetWorld(), TEXT("stat startfile"));
// ... run movement/combat logic ...
GetWorld()->Exec(GetWorld(), TEXT("stat stopfile"));

Verifying Against Budgets

Once you have the automated logs, you must compare them against your project’s defined budget—for example, maintaining 33.3ms for 30FPS or keeping the Stat Memory total under your specific platform’s limit.

Baseline Creation: Establish a baseline "gold standard" performance profile on your target device for every major area of the game.
Automated Comparison: Integrate a script into your CI/CD pipeline (e.g., Jenkins or GitHub Actions) that compares the latest build's CSV output against that baseline.
Threshold Alerts: If the automation detects a frame time spike or memory footprint increase beyond a 5% margin, it flags a build failure.

By validating performance programmatically, you remove the guesswork from optimization. You are no longer guessing if a texture group change or a new animation budget setting helped; you have empirical data proving whether the build meets your hardware’s strict performance requirements.

Key Takeaway: Automating performance tests on actual target hardware allows you to catch regressions instantly, ensuring that optimization strategies actually hold up under real-world device constraints.