So you've written a simple functional test and now you want to scale it up to test large parts of your gameplay code.
This tutorial should help you create a suite of simple tests and helper functions that you can compose into complex functional tests for your game.
But first, let's discuss Unreal's two testing frameworks: Functional Tests and Gauntlet.
Functional Tests vs Gauntlet
The Functional Test framework is great for writing small unit tests, and not bad for larger integration tests.
Gauntlet on the other hand, is Unreal's newer testing framework that focuses on testing gameplay rather than individual units.
Some benefits and drawbacks of each:
- Gauntlet only works on full builds, it does not run in the editor. This means every time you update your game and wish to re-run tests, you have to make a full build in order to run the Gauntlet tests.
- Gauntlet tests can be run on any platform; PC, console, mobile. Functional Tests cannot as far as I know.
- Functional Tests work both in-editor and from the command line.
- Because Gauntlet tests run on full builds, they are perfect for tests that measure performance and throw errors when FPS drops below a threshold.
In this tutorial we will focus on using Unreal's Functional Test framework rather than Gauntlet.
Functional Tests for Gameplay Code
In the Introduction to Testing tutorial, we created a simple unit test that instantiated a class, tested it, and cleaned up.
Now we want to test a larger chunk of gameplay code or an entire game using the Functional Testing framework.
Gameplay code does not always execute instantaneously, there are often animations, delayed callbacks and load times. For our tests to work with these delays we will need to change our tests to be able to wait until conditions are fulfilled. Unreal calls these functions latent functions.
Latent Hello World
Previously we could do all our testing in a single function, but now that we need parts of it to wait for gameplay code, we need to change our approach.
AutomationTest.h
has a macro ADD_LATENT_AUTOMATION_COMMAND()
, that creates a new instance of the class in the parentheses, and adds it to a list of commands to be performed in-order.
We will be using it to make sure that each part of the test is only started once the previous one has finished.
Searching the Unreal codebase for uses of ADD_LATENT_AUTOMATION_COMMAND
turns up a lot of interesting things showing how it is used:
// We can make the test wait for a second before calling the next command
const float ActiveDuration = 1.0f;
ADD_LATENT_AUTOMATION_COMMAND(FWaitLatentCommand(ActiveDuration));
const float Delay = 1.0f;
ADD_LATENT_AUTOMATION_COMMAND(FDelayedCallbackLatentCommand([=] {
// Do something after waiting 1 second
}, Delay));
So now that we know how to wait, why don't we just write something like this?
Nuances of Latent Commands
It's worth going into a little more detail on what exactly is happening when we define and run a custom latent command.
In our example below, we use DEFINE_LATENT_AUTOMATION_COMMAND_TWO_PARAMETER
to defines a class called FBUIWaitUntilCountMatches
, with two member variables, Start
and Target
. Note that they are member variables in the class FBUIWaitUntilCountMatches
, not function parameters.
The Update()
function that is implemented immediately after it is called as the test runner executes commands in its queue. If the function returns false
it is re-run again from the start, next frame.
Thus the member variable Start
is incremented over and over, returning false
which prompts the test runner to re-run the latent command again. Until finally Start
matches Target
and the Update()
function returns true;
DEFINE_LATENT_AUTOMATION_COMMAND_TWO_PARAMETER(FBUIWaitUntilCountMatches, int32, Start, const int32, Target);
bool FBUIWaitUntilCountMatches::Update()
{
Start += 1;
if (Start < Target)
return false;
return true;
}
This nuance should be clearer when we see what happens with ADD_LATENT_AUTOMATION_COMMAND
. It is calling the constructor of FBUIWaitUntilCountMatches
, and setting its member variables Start
and Target
to 0 and 100.
IMPLEMENT_SIMPLE_AUTOMATION_TEST(FBUICustomLatentTest, "BUI.CustomCommands.Simple", TestFlags)
bool FBUICustomLatentTest::RunTest(const FString& Parameters)
{
ADD_LATENT_AUTOMATION_COMMAND(FBUIWaitUntilCountMatches(0, 100));
return true;
}
Composing Complex Functional Tests
Now we understand how the Functional Testing framework supports latent functions, let's talk about how we can build a suite of tests in a smart way.
After a bit of experimentation, I found it helpful to break down testing code into three categories:
- Single tests
- Composite tests
- Helper functions
Single Tests
Single tests are exactly what they sound like, they test a single value. They must be implemented as latent commands, so they can be placed in a queue of commands to be executed.
Continuing with our generic RPG example, these could be some simple tests:
- Test that the hero's health matches an expected value
- Test if the hero is poisoned
- Test if the hero's inventory contains a given item
- Test if the hero is dead
- Test if the hero has spoken to a given NPC
In order to be able to call TestEqual
and other test functions, I always pass in a pointer to FAutomationTestBase
as the first parameter.
We can call this function in the following way:
Helper Functions
Helper functions are latent commands that perform no tests. They can be useful for setting the game's state before running tests. For our RPG example they could be things like:
- Add items to the player's inventory.
- Create monsters
- Change the player's stats
As before, these need to be defined as latent commands so they are executed in-order after other latent commands. They are implemented in the same way as simple tests, but there is no need to pass in FAutomationTestBase*
.
Composite Tests
We can now use our Single Tests and Helper Functions to put together more complex integration tests.
Using the examples from Single Test, we could compose a more complex integration test that checks that the poison system works in the game.
- Helper β Create a new hero
- Helper β Shoot the hero with a poison arrow
- Single β Test that the hero is poisoned
- Single β Test that the hero has lost health
- Helper β Wait for 1 seconds
- Single β Test that the hero has lost health
- Helper β Wait for 30 seconds
- Single β Test that the hero is dead
You can see that as you add more and more Single Tests and Helper Functions, you are creating a library of tools that make it easier and easier to compose complex integration tests.
Code
All of the code for this tutorial can be found on GitHub.