How to program a text adventure in C

by Ruud Helderman <r.helderman@hccnet.nl>

Licensed under MIT License

17. Test automation

Testing is important in any software development project. Games are no exception. Dutch games developer John Vanderaart made it very clear in an interview in 2007: “test, test, test.”

You could test everything manually; play your own game to see if everything still works. Over and over again. Every change you make, no matter how small, should be tested. This quickly becomes tiresome.

Fortunately, text adventures are very suitable for test automation. In its simplest form, this works a follows.

1. Test the game manually once, while letting the program log every command you enter.
2. Run the game again, but this time, use the log file as input. Capture the output in a file.
3. Whenever you want to test the program, just repeat step 2, and compare the output from both sessions. Any differences that cannot be justified, should be considered bugs.

Step 1 is easy; we implemented this in the previous chapter. All we have to do is run the program with a filename as argument.

Now start testing the game manually. Try as many scenarios as you can. Just a quick walk-through is not enough; you really need to test the dead ends and pitfalls. Try to get a good code coverage; use tooling like gcov where possible. When you are done, exit the program.

Keep the resulting log file in a safe place; it is your test script.

There is one more thing to do here: add the command ‘quit’ to the end of the file.

We are a bit cheating here; normally a log file never contains the command ‘quit’, because that would make it impossible for the player to continue the game. But in an automated test, we really want the program to quit once the test script has been finished.

This test script is full of weird (combinations of) commands. I did my best to let the program give as many different responses as possible. The result is a code coverage of more than 90%, which is pretty good. Please note that this does not imply that we have tested 90% of all possible verb/noun combinations. Code coverage says little to nothing about the number of objects that we interacted with, due to the data-driven nature of our program.

In step 2, we just call the program again, with the same filename as argument. Effectively, this repeats the entire session of step 1, without any user intervention. The difference is, this time we redirect the output of the program to a file.

This file is a transcript of the entire test session. Keep it in a safe place as well; it is our baseline.

We now have two files, the test script and the baseline, that we will be using to test every change we make to the program. Testing is easy:

Above, I used diff to compare the actual output of the game (transcript.txt) with the output that was last considered correct (baseline.txt). Please feel free to use your favorite diff utility instead.

Now there are three possible outcomes.

1. There are no differences. Apparently, your changes did not result in a regression.
2. All differences were intentional. For example, after fixing a typo, the game’s output should only change for the better.
3. There was an unintentional difference. This is a potential bug; it should be analyzed and if necessary, fixed.

Please note that case 2 is a reason for your baseline to be updated; otherwise your intentional differences will keep piling up, making it increasingly hard to spot the unintentional differences. If there are no unintended changes, then updating the baseline is straightforward:

Obviously, changing your test script (e.g. because you introduced new code or new objects not covered by the original test script) will always break the test. This warrants a new manual test, resulting in a new baseline. Of course, you can use (part of) the existing test script as a starting point; rarely will it be necessary to start all over.

Consider including test automation in your build process. Here is an example for make:

Here, make will only execute the final move (mv) after a successful comparison (cmp). That way, an invalid transcript can never be mistaken for a successful update, and make will always re-run a test that failed earlier.

Performance testing

Performance is hardly ever an issue in text adventures, for a number of reasons.

• Most text adventures do not take place in real time. To win the game, the player needs to be clever rather than fast. Most of the time, the program will be waiting for a reply from the user, not the other way around. Even if combat is involved (see chapter 20), it will be turn-based. A response time of 100 milliseconds or more should be totally acceptable.
• Most text adventures are single-player games. There are no scalability issues to be solved.
• And considering the code we have been writing so far: C compiles to native code, which is fast by nature. Especially when there are no performance bottlenecks like network traffic and complex memory management (garbage collections).

Still, it is always good to take performance seriously. Looking back at our code so far, you may have noticed some parts that are potentially suboptimal.

• Each time the player enters a command, the parser loops through all possible command patterns, trying to match each one to the user input.
• Most of these patterns contain 'nonterminals'; blanks that need to be filled in by finding an object that matches the noun entered by the player. The parser will scan through the full set of objects, then for each object, scan through all its tags. That's a loop nested inside a loop.
• Commands like look and inventory use a similar loop to scan through the complete set of objects, in search of the ones relevant at the current location.

In a big adventure, with many locations, items, actors, and with a comprehensive vocabulary, these loops will have to traverse some pretty long arrays. To make matters worse, the 'log and roll forward' technique proposed in the previous chapter means the player will have to wait for hundreds, maybe thousands of commands to be parsed and executed before being able to resume a game saved earlier. How long is that going to take?

Many programmers will not wait for the bomb to burst, and immediately start replacing the loops by more advanced techniques, for example hash tables. This could make your code more efficient, but possibly also more complex. I strongly recommend every developer to take some time to study the rules of optimization first:

• Rule #1: Don't.
• Rule #2: Don't... yet.
• Rule #3: Profile before optimizing.

The simplest way to profile your code, is to run an automated test while keeping track of time. In most operating systems, you can use time for that. If you want a more detailed insight, then you can use gprof; it can tell you exactly what functions eat up the majority of CPU time.

I leave it up to you to decide whether or not to make performance testing an integral part of your test automation.

Next chapter: 18. Abbreviations