In game development, I believe that programmers don’t create the game, they enable designers and artists to create the game. To that end, they must create game logic and tools that that let designers and artists create the game.
What is data-driven game design
Through data-driven game design, programmers can empower designers and artists to create and iterate on games.
Rather than programmers writing custom logic for every part of the game, with hardcoded values, classes and behaviours, programmers instead create a suite of behaviours and tools that designers and artists can experiment with to create the best possible game.
No hardcoded values
Imagine we are making a village management game where the player can create buildings and have their little villagers run around, farming, mining and constructing.
We would expose to designers all of the variables we might associate with such a game:
- The resource cost of each building.
- The text describing each building.
- The Actor to be spawned when the building is created in-world.
No hardcoded classes
However as a programmer you might decide to write all of the logic for each of
the buildings in their own classes.
House.cpp etc. Maybe you use inheritance to share common code, and seems
But what if a designer wants to create a new type of building? They have to come to you, the programmer and ask you to create it. Being a programmer is about being intelligently lazy, so how can we improve this?
Stepping back, what is a building? What can it do? We can see a building as a set of behaviours that can be mixed and matched:
- Create resource behaviour. A farm creates wheat, a fisher hut creates fish.
- Convert resource behaviour. A windmill takes wheat and converts it to flour.
- Store resource behaviour. A windmill can store a small amount of flour that it creates, a barn can store many resources of any type.
- Require Worker behaviour. The building will not work unless all the worker requirements are met.
By decomposing a building into a collection of behaviours, designers will be able to create new types of buildings without programmer intervention. Modders will be able to create new building types very easily, too.
Programmers are only required to create new behaviours. For example designers might want buildings to be able to catch fire. In that case then the programmer can create this new behaviour and let designers add it to the buildings they want to be flammable.
Data-driven design concepts aside, what tools are available to us in Unreal?
- Data Tables
- Data Asset subclasses or Blueprints in general
- Your own custom data structure and loader (XML, JSON, whatever)
After a bunch of investigating, none of these are ideal, They all have significant weaknesses that I will discuss.
UDataTable are designed for large amounts of data and to be
compatible with import/export from JSON and CSV, however they have some
significant drawbacks too.
Your data structure must inherit from
FTableRowBase and must be a struct.
To reference other Data Table rows, use
- Tabular view is good for viewing and comparing large numbers of similar entries. Note however that it is not possible to edit the data.
- Import/export from CSV/JSON could be useful. However comes with a serious caveat (see cons).
- Can to refer to individual rows in specific Data Tables by using
FDataTableRowHandle. Unlike using a raw text format.
- Quick to add rows unlike DataAssets that require creating a new asset for every entry. Imagine managing hundreds of DataAsset classes.
- Data Table row structs cannot contain UObjects. This limitation cannot be overstated. It is possible to reference other blueprint assets on-disk, or contain struct assets, but it is not possible to use UObjects.
- It is not possible to edit the data in the shown table. Designers have to select the row and then edit the properties in the standard vertical interface.
- No parent/child hierarchy possible, unlike with DataAssets. If many rows within the table have the same default value, and then designers want to change that value, they will have to do it one by one by hand, or export to CSV, edit and re-import
- Referencing rows in Data Tables is laborious. You have to select the Data Table, then select the row.
- Subclassing UDataTable a nightmare. You have to create custom editor tools to spawn the right type of
asset, then support all of the CSV/JSON importer/exporter stuff manually
because it only works with raw
- Referencing Data Rable rows is not
FDataTableRowHandlecan point to any kind of Data Table containing any kind of data. So your handle that you think points to a
FPlantDataRowcould instead point to a
FHumanDataRowif the reference is set up badly.
- CSV/JSON import/export creates *two places of authority*. Is the latest data in the CSV file or the imported asset? What if someone edits one but does not import/export to update the other? Changes could be overwritten when someone else imports/exports.
- Annoying C++ interface for finding and getting rows.
- Impossible to reorder entries in the Data Table. Their order is
undefined anyway as it's a
- Data Tables are stored as binary Uassets, meaning diffing is impossible and if you use Perforce, require exclusive locking to edit.
Data Assets are basically just Blueprint subclasses. Create a C++ subclass of
UDataAsset, then create Blueprint subclasses of that in the editor.
- Subclassing is possible, as with all blueprints. So common values can be stored in a base class, and instances can modify those.
- Refering to other assets is fast and
type-safe. Just create a UPROPERTY-exposed
- Can edit many assets and properties in tabular form by using the "Bulk Edit via Property Matrix" tool (shown below). Not all properties are supported (asset references for example), but it can be useful.
- Can contain UObject instances.
- Still annoying managing large numbers of entries. Creating an asset for every single new datapoint could be a nightmare if you have hundreds of items in an RPG for example.
- Managing lists of assets is a pain. If the
designer creates a new
BP_Plantsubclass, they then have to add that to some other array that knows about all class types. It cannot be automatically discovered (as far as I know).
- Binary asset as with all Blueprints, Data Assets are stored in binary meaning diffs and locking are an issue.
Raw text format (XML/JSON etc.)
Alternatively you could just ignore the Unreal editor interface entirely and keep and edit all your data in plain text, in a format of your choosing.
This has its own set of problems, regardless of what text format you choose.
- Nicer to edit large amounts of data in plain text using copy/paste, Excel, some other editor depending on your format.
- Diffs are possible. Yay.
- Completely breaks Unreal's asset cooking tool. Normally Unreal's build tool works based on references. If an asset is referred to, it is included in the build. If all your assets are referred to in a plain text file outside of Unreal, the build tool will not include any of your assets. So you will have to include every single asset in your project, or maintain a separate asset library.
- No validation during editing. For example if a value must be an integer between 0 and 10, users can enter "11" or "0.2" and not know that they are doing something wrong until they run the game.
- Harder to edit for non-technical team-members. I don't care what fancy format you recommend, JSON, XML, YAML, TOML, they are all designed for machines or programmers.
- Extremely brittle. A single quotation mark, tab or colon out of place can break the entire data file and require debugging from a technical team member.
- Does not work with Unreal's reference system. I use the reference viewer all the time to see if an asset is still being used, or what it's being used by. Storing data externally breaks this and means you're never sure if something can be safely deleted.
- Referring to Unreal assets is laborious and not type-safe, error prone and annoying. Finding the exact path of something and typing it is awful compared to just clicking a UI element.
- Text format means merges may be required, something that non-technical members may struggle with.
- Custom data loading may be required depending on your format. Meaning every time you add a property to a structure, you need to add the associated data loading code.
The ideal tool would:
- Allow for editing data in Unreal in a tabular format.
- Use a plain-text file as a single point of authority, letting users edit it directly or use the editor.
- Work with Unreal’s build system to include any referenced assets.
- Support all data types as properties.
None of the tools offered by Unreal fulfill this list.
So choose whichever is the least-bad for you.
Or just use Unity and Odin Inspector.