Game State Persistence

Sun, Mar 31, 2019

At a high level, Triverse resembles an actor model where independent actors send messages to each other using a message bus. They have no shared state, making them suitable for parallel processing. Actors are reactive in that they are triggered only by incoming messages.

Some actors contain game state. These actors are finite state machines fully determined by the sequence of input messages they receive, and their only output is messages sent to other actors. Actors without game state have more relaxed constraints and may make system calls, either to access the file system or call the rendering engine.

+-----------+                     +-----------+
|           |                     |           |
|  Actor 1  +<------------------->|           |
|           |                     |           |
+-----------+                     |  Message  |
                                  |    Bus    |
+-----------+                     |           |
|           |                     |           |
|  Actor 2  +<------------------->|           |
|           |                     |           |
+-----------+                     +-----------+

Game state actors requiring persistence include proxy, sim, and agent actors. Since they run independently, we need to consider and avoid race conditions to take a consistent snapshot. For example, after the sim actor updates its own state, it then sends observable portions of the world to agent actors. Agents maintain their own world state which should correspond to the same turn as the sim.

Update phases

Updating the simulation occurs in two phases, both originating from the proxy actor:

  1. Solicit actions: In this first phase, the sim actor requests all active agents to submit their actions for the next turn. Once all have completed and the actions are stored within the sim, this phase is complete.
  2. Step simulation: In this phase, the sim actor uses the stored actions to update the world state and distributes observable portions of the updated state to agents.

Originally only one phase was used, where the sim itself would solicit actions, then immediately step once it received actions from all agents. However, this complicates the flow of state change (proxy->sim->agent->sim->agent instead of two-phase proxy->sim->agent) and opens the possibility of race conditions when taking snapshots.

Snapshot flow

The overall strategy is to keep stepping and timing state completely inside a clock actor and rely on a consistent flow of messages through actors with persistable state to avoid more complex coordination. The clock actor dispatches solicitation, stepping, or snapshot messages, which always flow to the proxy, sim, and agent actors in that order. Saving snapshots can be triggered either manually from the UI/controller actor or via autosave in the clock actor itself.

Once a save is triggered, this sequence of messages occurs:

  1. Clock
    1. -> Snapshot: Expect a commit from Proxy
    2. -> Proxy: Take snapshot
  2. Proxy
    1. -> Snapshot: [State messages]
    2. -> Snapshot: Expect a commit from Sim
    3. -> Snapshot: Commit
    4. -> Sim: Take snapshot
  3. Sim
    1. -> Snapshot: [State messages]
    2. -> Snapshot: Expect a commit from each agent
    3. -> Snapshot: Commit
    4. -> Agent: Take snapshot
  4. Agent
    1. -> Snapshot: [State messages]
    2. -> Snapshot: Commit
  5. Snapshot
    1. Wait until all expected commits are fulfilled
    2. Serialize accumulated state and write to file
    3. -> Clock: Save completed
+-----------+                                         +-----------+
|           |                                         |           |
|   Clock   +- Expect ------------------------------->|           |
|           |                                         |           |
+-----+-----+                                         |           |
      |                                               |           |
     Take   +-----------+                             |           |
      |     |           |                             |           |
      +---->|   Proxy   +- State/Expect ------------->|           |
            |           |                             |           |
            +-----+-----+                             |           |
                  |                                   |           |
                 Take   +-----------+                 |           |
                  |     |           |                 |           |
                  +---->|    Sim    +- State/Expect ->|           |
                        |           |                 | Snapshot  |
                        +-----+-----+                 |           |
                              |                       |           |
                             Take   +-----------+     |           |
                              |     |           |     |           |
                              +---->|  Agent 1  +---->|           |
                              |     |           |     |           |
                              |     +-----------+     |           |
                              |                       |           |
                              |     +-----------+     |           |
                              |     |           |     |           |
                              +---->|  Agent 2  +---->|           | 
                              |     |           |     |           |
                              |     +-----------+     |           |
                              |                       |           |
                              |     +-----------+     |           |
                              |     |           |     |           |
                              +---->|  Agent N  +---->|           | 
                                    |           |     |           |
                                    +-----------+     +-----+-----+
                                                            |
                                    +-----------+       Completed
                                    |           |           |
                                    |   Clock   |<----------+
                                    |           |
                                    +-----------+                                    

Design points:

Stored game state

Reducing game state to its bare minimum can benefit design in addition to persistence. For Triverse, several systems such as projectiles and homing mines needed reworking to avoid use of hidden state and instead rely on existing visible state such as velocity and orders. This slightly reduces the size of snapshots, but it also makes the game more understandable since all state which the player should know is visualized in some way.

Here’s what each actor stores:

Proxy:

Sim:

Agent:

Other data such as blueprints created by the player are global and not associated with a particular game world.