Category:AI War 2: All About Multiplayer

Initial Caveats And Related Resources

• Please note that multiplayer is currently in alpha status. If you have not read about that, or the current status of where it is is, please see this page.

• If you are testing during alpha, we have a variety of questions for you that we would definitely appreciate you taking a look at.

• If you want to know what the known outstanding issues are, a pretty good summary of that is here. In some cases, particularly for features we might add, that can be rather vague, so more data is on our our mantis bugtracker.

What Does Multiplayer Alpha Mean?

Multiplayer for the game is currently in alpha status. This means that we do want your feedback on it, and you can play it, but there's an incredibly high chance that you will run into gamebreaking issues that don't allow for you to complete an entire session. If you expect to sit down with your friends and play a multi-hour session, you may find that is impossible due to bugs that you are the first to encounter. Please report them all on our mantis bugtracker, ideally with a savegame and/or reproduction steps, as well as any relevant error logs.

It's really critical to remember that there are a ton of edge cases where bugs can pop up, so even if there are dozens of people playing different multiplayer games during the alpha, you and your group may be the only ones to hit a specific issue. Please actually make reports to us, rather than assuming it will get fixed because someone else reported it. If none of the groups happen to report it, despite knowing about it, then we'll eventually proceed to beta with those bugs still in place because we'll have no idea they exist. The game is just too large, and there are too many things that are situationally-dependent, for us to find everything on our own without user reports.

Beyond that, a key difference between an alpha and a beta is that a beta is (in this case) feature complete, whereas this alpha version is not. All of the parts of the game that exist in single player also exist in multiplayer already, but for a multiplayer experience that doesn't mean it is going to be perfectly balanced or feature complete. There are a lot of features, and many of them may not be properly tuned for multiplayer.

If you are playing together with a group and you find things that are frustrating or unfair with the group, then please don't assume that's how we intend it to be, and equally please don't assume that we will become aware of it it and fix it on our own. It's entirely likely that doing something like hacking an IGC is just giving the hacking player the new capabilities, for instance, and not also giving that to other players. We haven't yet checked. It's possible that hacking an ARS gives a benefit to only one player in a fashion that feels anemic... or maybe, given the extra firepower multiple players automatically have, that actually is appropriate. AI waves might be too few with multiple players, or might be just right.

Certain things, like science points and hacking points and scouting, we intentionally designed for multiplayer and so those should work just great. Most other non-combat mechanics that give a benefit to potentially just one of several players all need to be evaluated by players. We're going to be heavily busy working on optimization and fixes and features that we do know about, so for a few weeks at minimum we won't have time to be testing that sort of thing to form our own opinions. Please let us know where things seem off.

Similarly, there are other features that are nice to have that only have relevance in multiplayer. Chat is one obvious thing, and that is in place and works great. But being able to give a ship line to another player is definitely a nice thing, but not possible (yet -- because we have not had time to add that in and it's not directly something that exists as part of the singleplayer flow). How do outguard summons work with multiple players, for that matter? Are there other things you want to be able to share between players, but cannot? Share some metal or energy income in some fashion, or just dump a big chunk of metal over to another player?

If you find yourself feeling frustrated while trying to communicate or coordinate with other players in your group, then please make a note of that and tell us how we can help. These are the sorts of things we want to iron out during the alpha, and to a lesser extent beta. Thank you!

Also:

• Below you will see a variety of questions aimed at testers.
  • Please see if there are any that you have feelings on, and feel free to offer suggestions ideally on mantis bugtracker, either in new topics or on existing topics that are open that someone else raised.
• Further below, you will see a number of "todo before beta" issues.
  • If an issue that you are running into is mentioned there, then it might not warrant a report, but we'd rather have extra reports than not enough; use your own judgement, and we certainly won't fuss at you for giving us extra data.

Questions For Multiplayer Alpha Testers

• Spectator mode needs investigating further, to make sure it can see all the various notices and warnings we would expect, and chat as normal, etc.
  • To test this at present: simply remove the second human faction that the second player is assigned to, and that player should drop into spectator mode. Later it will be better.

• Two players controlling one faction needs testing, when no empire name selected, to make sure that creates a correct-looking name.
  • Same for exactly three players controlling one faction.
  • Same with four or five players or more controlling one faction.
  • This cannot be tested yet, as we don't have the UI controls yet for this.

• For every possible hack that you can do in the game and its various expansions, does it feel fair what results from the hack?
  • If not, what doesn't feel fair about it and how might we address it? For example, in many cases, just the hacking player expended hacking points (HaP), so just them getting a benefit sort of makes sense. Given that, one potential would be to make those structures able to be hacked once per player, and they each get the benefit if they choose to spend HaP there. That might be the way to retain the most freedom for players, especially since they all get their own "copy" of the HaP (so in a two player game, your team has 2x the total HaP you would have in a solo game).
  • There are, of course, other consequences to various hacks, ranging from exos to other angry AI or faction responses, to even AIP going up. Is there anything we're missing with specific ones that seem off to you?

• For every capturable, such as new fleets/golems/arks/battlestations/citadels/etc in particular, is that feeling like it is fairly distributed and not resulting in too much AIP?
  • The AIP of a multiplayer game should probably be expected to run a bit higher, but you have double or more in terms of metal and energy income, as well as double science and hacking points, as well as a lot of extra firepower in terms of starting fleets, so to some extent that's actually a good thing.
  • But there's a difference from being balanced and feeling balanced. If it doesn't feel right, even if it is right, then we still need to at least look at it.

• Are there any situations that seem to be causing inordinate amounts of lag for you, or giant spikes in amounts of transmitted data (as seen in the escape menu)?
  • Most likely these would be something where there is a large amount of sync correction data being generated because something is getting out of sync and the game is having to repair itself on the fly. If it's something that we can commonly replicate, particularly with a save from you, then we can work on eliminating the initial problem that causes the sync issues in the first place.

• Are you getting any "random" errors popping up, particularly on clients?
  • These usually will be nullref exceptions or out of index exceptions in what may seem like completely random parts of the game code.
  • There is a whole new set of cross-threading conditions that exist only on the clients in a multiplayer game that can cause these, and they are extremely dependent on millisecond-level timings (in some cases, literally nanosecond-level timings). So to say that not everyone will hit all of these is an understatement. If you run into any of these issues, even as a one-off thing that doesn't occur again even for you, please do let us know with a log of the error.

• What features are missing for you sharing things between player factions?
  • You want to give X to your ally, or have them give you X, and you can't do it -- what is X, and is there anything else we know about that?

• What communication features are missing for you working with other players?
  • You're trying to coordinate something, and you wish you could just show them what you are seeing, but that doesn't quite exist. What's happening, and how can we help?

• How does the difficulty feel compared to you playing solo against the same enemies?
  • We don't really have a for-certain goal on this, but probably it would be expected that -- unlike the first AI War -- you need to turn up the AI difficulty to handle more players, or add more AI or other factions.
  • The original AI War was locked to exactly two AI factions, never more or less, so we had to make those scale in difficulty between 1 and 8 players. In this game, you can play against many more AIs, so making the AI automatically scale might be more confusing than anything else. Then again, the fact that you will inherently have a higher AIP will make some scaling inevitable.
  • But that AIP scaling won't apply to other factions, friendly or otherwise, and so they might feel underpowered... or might feel just right. Your reaction may be "there are four of us against the same number of nanocaust enemies, of course we win easily without increasing their intensity."

Reporting Bugs Or Suggestions

• Any bugs or requests should go to our mantis bugtracker
  • If you need to submit log files, those can generally be found under your PlayerData folder in the folder your game is installed in. The most relevant one is called ArcenDebugLog.txt. You can send us the whole thing, or just strip out relevant parts.
  • In rare cases, mainly if your entire game crashes (that almost never happens), we will need your unity player log. That gets overwritten the next time you run the game after a crash, unlike the other log. These can be found here:
    • Windows: C:\Users\username\AppData\LocalLow\Arcen Games, LLC\AIWar2\Player.log
    • macOS: ~/Library/Logs/Arcen Games, LLC/AIWar2/Player.log
    • Linux: ~/.config/unity3d/Arcen Games, LLC/AIWar2/Player.log

• If you have been generating logging information that is networking-specific, then that will also likely be in the PlayerData folder, in some cases in subfolders.
  • Generally with those sorts of logs, we need the equivalent log from the host and at least one client (whichever client had the problem).
  • But, also, we really don't expect you to be generating these sorts of logs for us typically, so these really are going above and beyond if you are sending those in.

• In other cases, if you suspect something is wrong, then opening the escape menu during the game and taking a screenshot of the stats there to send to us is welcome.

Reporting Lag Or Connection Problems, Specifically

• Relating to multiplayer performance problem reports:
  • Knowing how long (in actual realtime, not gametime) you were playing in a given session to go along with the amount of data transmitted and performance you were seeing is useful.
  • Knowing rough distances between the players (intercontinental, same country, same metro area, or same building is also useful if you're having performance-related problems.
  • Similarly, knowing the general strength of the internet connections in question is useful if it's a performance problem, as well as which networking framework (Steam, GOG, or LiteNetLib) was used is also helpful. If one or more of you is running this over a VPN, that's also important for us to know.

Multiplayer Remaining Todo List

Selected Short Term Todo List

Some items from the bugtracker just copied across to here so people know it's coming:

• Some sort of issue with chat

• fleet cloning bug

• client side disconnection

• "Fixed attempt to read more faction data than we had factions" on the client

• General work on some stutter with "waiting for players" that flashes by too frequently on many machine combos.

Before Beta

• Allow clients to be removed and added to factions, either as the secondary controllers of them or the primary. Beyond the auto-adding happening right now.

• "Reset to defaults" on the host causes an error on the client side, and player account destruction.

• After SetUpAnyMissingFactionsForPlayerAccounts(), it doesn't seem to update the client on what their starting planet is.

• There are likely to be some exceptions on sync based on the "create new if not found" code paths not having a "I am from a sync" variable, which was also a problem with ships.

• Sync improvements:
  • Make primary keys now be generated from factions, and thus the amount of space for sync to go wrong a lot smaller.
  • Maybe: Move fireteams into the main codebase a bit more, and have them have IDs issued by the faction also.
    • This may no longer be needed.
  • See if we need to strip out any parts of fleets to make them more efficient or have fewer of them.
  • Investigate regiments aimed at planets.
  • Provide a way for specific ships to specify that they need to sync their externaldata to clients on some periodic time period.
  • Consider removing things like IncomingShots on ships, as we are not generally keeping shots in sync (they last too little time, etc).
  • Do we need to sync journal entries and world history? If so, given them their own time slice, but it seems unlikely.

• Use ObjectDumper or similar to find "runaway lists" on clients, which would basically be a client-only memory leak caused by sync not clearing collections prior to adding to them.

• IncomingShots should be looked at, and potentially refactored out if possible.

• Consider stripping down PlanetFactions in some fashion for performance in general, even in single-player.

• MAYBE: Re-code GameCommands to be more efficient and special-purpose. This is probably a job that is a couple of days long, and will potentially lead to widespread bugs for a week or so after it.
  • This has become a maybe because, so far, the gamecommands seem very efficient as they are. And for the sync data, which was a big concern, that's being handled out of band.

• The other features on multiplayer, mainly regarding things like donating fleets between one another, and/or whatever else we come up with that is desirable.

• Find and fix as many lingering lobby and other UI bugs as possible.

• When a player joins or leaves the game in a 3+ player game, update the other clients to let them know PRIOR to the next map regeneration.
  • Either using OnPlayerJoined() and similar, or SendTheWorldFromTheHostToClients()

• Why are we able to spam "randomize seed" messages and have that get away from us? That seemingly shouldn't happen, but does.

Before Full Launch

• Allow clients to save the game by having a gamecommand created (as always), but having the host then create the save and send it as a package deal to the client, who saves it.

• Get it so that clients are kept up to date on the status of other clients, so that they can make a call similar to how OnServer_GetIsConnected() works on the host.
  • This is only relevant for games with 3+ players, but is a nice convenience. If the host is waiting excessively long a specific client, then it might be nice to send that information, but might be hard to get there in time for most situations.

• Whatever changes we need to make to balance in order to make things "feel right," which will be a matter of working with the multiplayer alpha and beta testers. A lot of things we already did in the past, like making science collection a humanity-wide thing that each player gets a copy of, rather than something people have to do individually (what a pain that was in AIWC). We will have to scale waves like we did in AIWC multiplayer, or in some other fashion. But a lot of the difficulty scaling is inherently handled by AIP being higher when you have to take more planets in multiplayer.

• If we're seeing network degradation or other issues due to the constant need to sync errors, then that will be to be investigated and improved. But those things are most of what the focus of the alpha/beta will be on.

• Host and client are still exchanging a bunch of no-op commands in the lobby for some unknown reason. I guess that really doesn't matter, as they are next to no data. This probably does not matter, but we'll keep an eye on it.

Multiplayer, At A Glance

This game is a client-server model, with one host, and any number of clients who are connected to that host. There are several networking frameworks available to use, but the host and clients must all be using the same one.

About The Host

• If the host severs the connection, then the game is halted for everyone else. A new host is not automatically chosen.
• Only the host can save the game, at present. Even with our improvements to multiplayer sync, clients are too likely to have slightly-garbled versions of the game.
• The host must start up the game first, either in the lobby, or by loading an existing save or quick start. The clients are then able to connect, based on whatever the rules are of the networking framework everyone is using.
• The host should have the most powerful CPU, as it runs a lot of background AI threads that are too time-consuming and long-running to run in the main simulation, and which do not run on clients.
  • This is similar to AI War Classic, but many threads instead of just one.
• The host should also have the most robust network, as they have to send approximately 2n the data that a client does, where n is the number of clients.
  • So in a two player game, the host is probably sending and receiving roughly 4 times as much data as the sole client.
  • In a three player game, the host is sending and receiving roughly 6 times as much data as either of the two clients.
  • A spectator client adds roughly the same amount of bandwidth requirements as a regular player.

About Clients

• Each client connects to a host, and passes the name of their current Player Profile (as configured in AI War 2 itself).
• Clients (and for that matter the host) are linked back to their prior faction based on the player profile's name, so changing that midway through a campaign with friends is not a great idea.
• Clients are in general in a state of slightly-wrong-sync with the host, though the game does its best to keep everything consistent.
  • There's a rolling sync process that sweeps through and corrects inconsistencies within 3-4 seconds on each client, and so in cases of high desync clients may see things jump to correct themselves to match the host.
  • This is not something the host will see, as the host is treated as the "central correct record" for everything.
• It's worth noting that if a client is lagging behind, it will slow the host and other clients to wait for them. BUT if a client is disconnected entirely, then the game will continue without them (though any other player can pause it; they can then save, stop, or continue).
  • It's also worth noting that the host and the other clients do not have to disconnect or restart anything in order for the missing client to connect back. When the missing player returns, there is usually just a 1-5 second pause (depending on the size of the savegame) while the client gets caught back up to speed. But you don't even have to explicitly pause the game.
  • If sync is ever atrociously off on one client, that client can always use this to their advantage to disconnect and reconnect and have perfect sync again (although sync being that bad off is not supposed to ever happen).

Bandwidth/CPU Requirements And Player Counts

• There is a built-in limit of 254 clients connecting to a single host, but for practical purposes there's really not going to be remotely that many players supported.
• How many clients are actually supported is a matter of bandwidth between them and the host, as well as CPU processing power for the resulting game.
  • If there are 40 clients all sharing one faction, then this is far less of a load than 40 clients all with their own starting planet and unique faction. It's several orders of magnitude less CPU processing, to the point where the former might be barely possible on a nice LAN, but the latter is not possible with current hardware.

• As the years pass and multi-core processors become stronger and also add more cores, AI War 2 will naturally spread out to use those capabilities with no code changes required. It would probably take 16 or so cores before you stop seeing any tangible benefit.

• At the moment, with one client and one host in a fairly average starting galaxy with one AI and two player factions, the bandwidth requirements are less than 100KB per second on the host (in many cases closer to 10KB per second). The client ranges from half to a quarter of that amount, depending on what stage of the game we're talking about.
  • With just two players, far more important than the speed of the connection is the reliability of it. Lots of dropped packets and resends will cause delays even if the actual connection is plenty robust.
  • With two clients and one host (aka 3 players), the client requirements stay the same, but the host bandwidth requirements go up to something in the 200KB range.
  • It's actually expected that, for most practical cases with more than 10 total players, the CPU load of preparing all the extra network messages will probably cause more slowdown than the actual sending of the messages. This is something that we have worked around as much as we can, but also not something that more CPU cores will help given the linear nature of the data we are sending.

• The bottom line is that, in most situations, you should have a fine time as long as you have a 10 or fewer players, but we'd be interested in hearing what sort of experiences you have.
  • For practical purposes, the game itself will probably bog down with more than 4-6 human factions on current hardware.

Which Networking Framework To Use?

The game supports three networking frameworks at present, although more could be added in by ourselves or modders.

What Is A Network Framework?

A networking framework is responsible for transmitting data between players, and for getting players connected in the first place, but it is completely unaware of what data is being transmitted.

Conversely, the game itself is responsible for preparing and handing off all the actual messages it wants to orchestrate between players, and it has no idea where those messages go (at a low technical level) or how they get from the client to different hosts.

Some networking frameworks are more efficient than others in terms of how they use CPU and RAM on the client and host, and also in terms of how they use your network card and available bandwidth. They also can differ in what sort of networks they use, and so again can have a big impact from that area alone.

Steam

In this networking framework, you don't have to worry about IP addresses at all. As long as all of the clients are Steam friends with the host, you're good to go. Firewalls and network configurations don't matter much.

The host starts a game while logged into Steam, but they do not have to be in Online status on the Steam Friends section. Even someone who appears Offline in Steam Friends can be a host or client.

The clients will see a list of their Steam friends, sorted first by those who are currently online and playing AI War 2, and then by those who are online, and then those who (at least seem to be) offline. You can click connect next to any of the friends and try to connect to their game, even if they appear to be offline. It won't hurt their machine or yours, and if they have a game open even while Steam Friends is set to offline, you can connect to them. Please do only connect when invited, though!

It's worth noting that all of the network traffic is routed through Valves relay servers at all times. This will typically give you better performance than the general Internet backbone, according to Valve. You CAN also use this while on the same LAN as other players, but your performance and ping will be as if you were on different networks in the same city. Aka, far worse than if you were on the LAN together, but still plenty great for most purposes.

GOG Galaxy

In this networking framework, you also don't have to worry about IP addresses. As long as all of the clients are GOG friends with the host, you're good to go. Here again, firewalls and network configurations don't matter much.

The clients will see a list of their GOG Galaxy friends, sorted first by those who are currently online and have an active AI War 2 multiplayer session open. You will then see the others who are offline, or in game, etc. The various statuses don't really matter except for the ones that show up as purple, who have a game actively open at the moment. You won't see a connect button except for players in that status, so be sure to refresh the list of friends frequently if one of them just opened a game and you want to connect.

We're not sure exactly how the network traffic is routed. There are relay servers involved if a direct connection can't be made, but we're not sure if a direct connection is used in cases where it can be. Performance seems great in general, but definitely is relay-based if you are on the same LAN.

LiteNetLib

If you're on the same LAN as the other players, then this is a no-brainer choice. If you're on different networks connected to the Internet, then it's your choice; to use this in that situation would probably require port forwarding on your routers, and that's something you can easily look up for multiplayer games in general if you want to go that route.

The game's network tab of the settings menu shows you your public internet IP address, as well as your local IPv4 and IPV6 addresses. You can also configure what port the game runs on (it is using UDP). Only the host would need to set up port forwarding for that port if you are playing across the Internet.

The host would need to give their public IP address (for internet play) or their local IP address (for LAN play) to all of the intended clients, and they then type that into the connection textbox and connect to the host after the host has started or loaded a savegame/quickstart or the lobby.

If you are on a VPN network together, then it works much like a LAN does, but whether or not your IP address for the VPN's virtual network adapter will show up in the game interface is dependent on the VPN service itself. If not, then most VPN services prominently show both your virtual IP address as well as the virtual IP addresses of any other folks in the same session with you.

VPN services usually try to forge a direct connection if they can (which is almost as fast as the port forwarding route, but not quite because of encryption), but fall back on a relay server network of their own if that is not possible. The speed of these relay networks varies hugely depending on the VPN service. Some of them are extremely fast and suitable for gaming, others are direly bad.

So What Should I Use?

• Are you playing the Steam version across the Internet? Use the Steam network framework. It's simple and really fast.
• Are you playing the GOG Galaxy version across the Internet? Use the GOG Galaxy network framework. It's also simple and quite fast.
• Are you playing on a LAN? Consider using LiteNetLib, as it's the most direct and will be the fastest by far. If you have trouble, you can fall back to Steam or GOG.
• Do you feel like avoiding Steam or GOG Galaxy, and you're an expert with your router? Use LiteNetLib with port forwarding for maximum speed.
• Do you feel like avoiding Steam or GOG Galaxy, and you're not a networking wizard? Use LiteNetLib with a VPN service like Hamachi or similar. There are many available.

What If I Only Have One Steam Or GOG Galaxy Account?

If you are planning to use Steam networking, or GOG Galaxy networking, you need a unique account on the respective service for each machine that is going to use it. They each need to own the base game, although the DLC can easily be pirated by just copying it from one machine to the other. (Please don't, if you're not a developer testing it, but literally piracy is as easy as copying thing, since we don't believe in DRM).

The only reason that you have to have unique accounts is because the connections in these cases are account-based. And the reason that you have to own those games is that those services check for that when you use them -- the services are paying for the bandwidth usage of games that are run on them, and they'd rather not have games not owned on their service run multiplayer through their relays, which seems reasonable.

Anyhow, so for testing purposes Arcen has a few Steam and GOG accounts to run between multiple computers, as one example. But if we were to play with family members, they would just each need a Steam key or GOG key for their own accounts, and then they're good to go.

The LiteNetLib networking framework has absolutely zero restrictions, so you can put the game on a thumb drive and copy it to 4000 people and they can all play each other in multiplayer without any sort of checks or restrictions. Again, please don't pirate our stuff, but it's stupidly easy to because we're not trying to stop you.

How Do Expansions (DLC) and Mods Affect Multiplayer?

In general, players must have the same expansions and mods turned on when they are playing multiplayer together. If a client tries to connect to a host with not-enough or too-many things installed, it will inform them of the differences and what changes either they or the host should make to bring them in line.

It is our goal that people never be forced to buy DLC/Expansion content in order to be able to play with friends who have that content. However, while you're playing with those friends, they will have to disable the DLC/Expansion content.

You can disabled expansions/DLC or mods, as needed on the client or the host under the Game tab of the settings menu. That then does require an application restart, as a lot of the data that is included in those packages is very tightly integrated into the main game and can alter things in unexpected ways without the restart.

Compatibility?

All official content from Arcen, ranging from the base game to the Expansions/DLC is compatible with multiplayer. If there are any problems with any of it, that's a bug that we will look into and get fixed for you.

Mods, on the other hand, are not something that we can control in this way, and are something of a wild card. This caveat also includes mods that we distribute with the game, as we are making them more accessible to you, but not taking responsibility for them being bug-free.

That said, mods that are just xml or just visual components are pretty much guaranteed to be compatible in multiplayer, as they are using core game engine features that we designed. So, for instance, there's no risk of the Spire Railgun Shop messing up something in multiplayer.

Mods that include code of any sort tend to be the ones at risk of lack of compatibility, and the more complicated the code is, the more likely problems are. Civilian Industries, which we also include with the game, is a great example of this. We do our best to give tools to the mod authors so that they can make sure that their mods work well in multiplayer, but ultimately that still requires testing on their part, or feedback from people playing their mods in multiplayer.

The sync-correction process that is a core part of the game in general will keep things from a mod working smoothly, or at least not diverging too heavily. But it's also possible for a mod to introduce a memory leak (client side only, or else it's a problem with the mod not limited to just multiplayer) by not handling the sync code properly, or to introduce exceptions during sync itself, etc. It's also possible for a mod to fall prey to the same sort of "random threading exceptions" that other parts of the game have to guard against, through no fault of the mod author.

The most likely thing to happen in a mod with some incompatibilities is that certain data in tooltips will be inconsistent or missing on multiplayer clients. For instance, using Civilian Industries as an example again, it may be that the resources collected by various civilian ships are not reflected in the tooltips on the client the way that they are on the host. This is something that we will work with mod authors on to ensure is as correct as possible wherever they are willing and such an issue is noticed.

How Can Players Participate In A Multiplayer Session?

There are several ways that players can join in. In order to discuss this, we need to make three pieces of terminology clear:

• A player is a specific person, at their computer, who is going to be either the host or one of the clients.
• A player faction is kind of like an empire. In single player games, there is always one empire for the one player. But that's not always the case in multiplayer.
• A player profile is the name and preferred-color you give yourself on your machine. You can have multiple of these, with different names and colors, but only one can be active at a given time.

Example time!

If we have BobX as a player profile on the host machine, and he starts a new custom game, he will immediately be put in charge of player faction 1, which will also be called BobX unless he changes it (via the empire name field on the player faction). The faction color will be set to match his preferred player profile color, again unless he changes that on the settings for the player faction he's now in charge of. So far this is just like a single-player game.

Now if we have SallyB as a player profile on a client machine, and she connects into BobX, then a second player faction will be generated, this one for her. It will get the name SallyB, and take on her preferred color, but it does not have to stay that way. And that's where we start getting into a slight bit of complexity.

Everyone With Their Own Faction

This is the simplest and most obvious way to play, and what happens by default. If you have three clients join the host, then you wind up with four total player factions. Each one has a name that matches faction to player, and that's that.

Although, still, there are some questions worth addressing:

• What if several of the players' preferred colors are too similar or the same?
  • This won't affect anything, gameplay-wise. The color is just a color, and won't merge factions or anything like that.
  • From a practical standpoint, it will be confusing, however. So changing the colors up a bit via either the factions tab in the lobby, or the factions screen from the escape menu during gameplay, might be a good idea. You can always change the colors of any of the factions at any time, so it's never to late to shift things up if you're finding it hard to tell someone apart.

• What if players choose the same name?
  • This is not allowed. Out in the world, globally, people can reuse names as much as they want. But in a specific multiplayer campaign, there can only be one "BobX" and one "SallyB"
  • Fun fact, if BobX and SallyB save and quit the game, then rename their player profiles (or switch player to alternate profiles on their machine that have those other names), then they can come back into the game as each other's factions.
    • Put another way: If BobX is now SallyB, and opens up the savegame to host on his computer again, he will now be in control of SallyB's faction. If she tries to reconnect as SallyB, she'll be told that someone else is already playing with that name. If she tries to connect as BobX to that save she and he were previously playing, then now she has his faction, but is still the client.
    • Why you would want to do this is not entirely clear.
  • More realistically, if there is a second BobX trying to connect to a game with BobX and SallyB, then that second Bob will have to come up with a new name. They can either rename their existing player profile, or just create a new one and use that when they are playing with this specific group. Maybe this new person continues to go by the profile name BobX most of the time, but in this campaign they switch over to a profile name of BobZ so that they can be uniquely identified.

• Why not use something like people's IP addresses or Steam or GOG name or something else like that to solve the above conflict?
  • First of all, because the above problem is really kind of an edge case. Most people choose fairly unique names, especially among groups of friends.
  • But secondly, we assume that people may have their IP addresses change, or may switch around what network framework they use, or any number of other factors.
  • And finally, we actually view it as a really nice feature that anybody can impersonate someone else in a savegame, to see what things look like from their point of view.
    • This is useful for us as developers, but it's also neat for anyone who wants to share savegames on the Internet for others to play.
    • If there's a really challenging savegame with three players where people post it going "can anyone beat this??", then they can give the names that people connect as (Be BobX if you want the big empire up nort, but that one is getting hammered by waves. Be SallyB if you want all those upgraded golems in the east. Be BobZ if you want that almost-dead last planet in the center. Can you win where we lost?) That sort of thing is neat.

At Least One Spectator

A spectator is a player who controls no factions, and thus can do nothing except watch what everyone else sees.

Fun fact, it's possible to play this game in multiplayer, but still have it be single-player in all respects that matter. Have the host be an actual player, and the sole client be a spectator, and now you have a second person who is just watching you play but can't do anything. A twitch stream might be a little more straightforward, but a spectator can look at different things from what you are seeing.

The one thing that spectators can do is chat. They can click around to all the planets that any other human players can view, but they don't see anything extra that the others would not. So you can't use a spectator to cheat, but unlike a twitch stream a spectator can look in places you are not seeing and can alert you to threats. They can thumb through intel, journal entries, watch border planets you don't have time to glance at, and so on.

To become a spectator, or make someone else in your lobby become a spectator, simply remove the faction that is tied to that person. It won't assign them to another faction unless one is added back manually for them.

Alpha Note: we will be expanding the interface a bit to make it easier to set yourself up as a spectator, later during the alpha.

Multiple Players On One Faction

Alpha Note: the interface for setting this up is not yet ready.

It's possible to have multiple players share control of a single faction. There's no explicit limit on how many players can be sharing that control, and not all of the players have to be present for gameplay to proceed.

By default if BobX and SallyB share control of a faction, it will name it BobX and SallyB. With three players, it becomes BobX, SallyB, and BobZ. Beyond that, it adds "and others." But really, if you have more than one player controlling a faction, it might be nice to set the empire name to be something thematic like "Empire of the Rising Bobs."

In chat, it will identify players by their profile name, but the color of their faction. That way you can tell if it's BobX or SallyB talking, despite the fact that they are sharing a faction at the moment.

Many, Many Uses For This!

There are NO RESTRICTIONS on what two players can do if they share a faction together. Essentially they both have equal control of the resources and units, and their own independent copy of the interface. If a friend who is new to the game wants to learn, this is a great way to do it. You are essentially playing a "single player game" in terms of balance, but with two brains and four hands and four eyes controlling the shared empire. If you want to set up an arrangement where certain fleets are used by certain players, that needs to just be an agreement between you.

Various folks have asked us about things like "Champions" or "Helpers" from the first AI War, and this ability to share factions really handles the bulk of the use cases of those old roles, but with more flexibility. Essentially if you want to have one friend share in your empire but just drive a Botnet Golem around causing chaos, then that can be their sole responsibility.

Any Combination Is Fine!

You can mix and match this however you like. Two players can share one faction while a third has their own faction. Yet another player can be a spectator. It doesn't matter which one of them is the host (the host can actually remove themselves from the original faction once a second player is in).

Empty (From The Start) Player Factions

It's possible to add an extra player faction that... isn't controlled by anyone right now. It will just kind of hang out on its planet and gather resources, and not have any intelligence behind it. Its ships will fire on things that come into range, but they won't be in pursuit mode by default.

The only real reason to do this seems to be if you have a two-person group today, but next week it will be three people and you don't mind defending that third planet until then.

Alpha Note: it's possible that late-assigning a player to this won't work properly yet, in the alpha.

Abandoned (Forever Or For A While) Player Factions

Last week, perhaps you had three players who wanted to play, each with their own factions. This week, you want to continue that campaign, but you only have two of the players present. You can load up that savegame and continue with the third faction just sitting there doing whatever its last orders were to do, and that won't be a problem.

If next week the third player is back, you can either go back to where you were when they last played, or you can jump them into the future where they are probably behind and disorganized, but hopefully still living.

What Multiplayer-Specific Features Exist?

Generally speaking, if you can do something in single-player, then you can do the same thing in multiplayer. Overall our goal is not a radically different experience when playing multiplayer, but rather "it's AI War 2, but with more people, and fun." That said, to hit the "and fun" bit, certain things had to be taken into account specific for multiplayer.

Alpha Note: We're going to need new things in here based on how answers to our various 'questions to MP alpha testers' come out.

Copies Of Science And Hacking Points

In the original AI War, every player individually had to gather their own knowledge and hacking points. That was actually really annoying and tedious.

In AI War 2, if any player has a unit gathering science or hacking points, then all players get a copy of those points. So there's never a case of us having to coordinate over gathering science from a distant temporary planet we are holding. If you gather it, I get a copy, and if I go there later, there's nothing left for me to gather since I already have it.

Shared Scouting And Intel

Our scouting levels, and our intel sidebar, are all shared. This is the same as the original AI War.

One of the nice things about the intel sidebar in particular, however, is that any player can right-click the items to change their priority from normal to high or low, and then all players see that. So if there are things that everyone is wanting to coordinate, people can adjust those priorities together until they agree upon them. Or one or more players can be designated to evaluate intel and recommend what to pursue and what to ignore for everyone.

See Where Other Players Are On The Galaxy Map

It's really handy to be able to say "see this planet up here where I am?" This is something you can do by simply clicking the planet, and other players will then see it as highlighted. If they mouseover the planet, they can see your player name (since with many players, this can be confusing as to who is where). This is meant as a visual aid to allow players to see where each other are and thus coordinate a bit better.

What Do I Need To Know About Balance In Multiplayer?

Alpha note: this is actively under discussion, and one reason for doing the alpha.

Overall it's worth noting that if you add in multiple players to a single player faction, that's the same as having one player faction in single-player, and it always will be. The game is definitely not meant to react to having more players, ever, but only to having more player factions.

When it comes to having more player factions, right now the big change in balance compared to single player is that in order to support those factions, more planets will need to be taken, and more fleets captured. This is going to raise the AI Progress (AIP) substantially more than in single-player. But since the AI is not inherently scaling up in any other way in response to multiple player factions, this might actually work out just great. At the moment we don't have enough data to know either way.

In the questions to alpha testers section, we have posed some questions about what people would like to see regarding hacks in particular. But bear in mind that in a two-player-faction game, you get double science and double hacking points for free, plus double starting fleets and starting planets and so on.

The original AI War was really rigid in its number of factions (always exactly two AIs), but the sequel is not. To some extent it may be best if we don't do too much scaling to multiple player factions, and instead let them tune to taste. But if we did that, then essentially every quick start would be extremely easy for four players compared to one, as just one example problem. This again is one of those things that we want more data on from people actually playing: how it feels to them, versus what we can guess at on paper.

Technical Explanation Of The Multiplayer Model

We've written about this extensively over time, and done a few videos, but it's nice to collect this into one place in a brief format.

AI War Classic: The Lockstep Model

The original AI War used a lockstep design along the lines of what you can see in that link. That is indeed what multiplayer RTS games have been doing essentially from the dawn of them existing. We used the same networking model in Tidalis and in Skyward Collapse.

We WERE planning on using the same model in AI War 2, but ultimately we found ourselves getting caught out by the extremely multi-threaded nature of this game. We just can't guarantee absolute determinism between clients and the host, or clients and each other, and so in 2018 we decided to stop trying for absolute rigor in that regard.

We do still mostly use the lockstep approach, and we do our best to keep things deterministic as much as possible, but divergences are expected -- unlike with the original AI War. When a desync was detected in the first AI War, it would stop the game, ask you to save and reconnect, and suggest you give a report to the developers. We would then spend sometimes hours or days chasing down the problem, until it could be fixed and run well again.

(It's worth noting that that level of rigor was also something we realized we could never enforce in mods for AI War 2, and that we'd never be able to tell if a desync was a mod or the core game, so we were really going to be messed up if a desync was a crisis like it was in the first game. There were just a whole host of reasons why a pure lockstep model could no longer work for us.)

(As one other aside, we also really needed to use some floating point math for accuracy in certain places, as our fixed-integer math code was showing its age. We needed this for speed and accuracy with certain kinds of calculations in particular. Floating point numbers are a definite source of desync, which is why AI War Classic doesn't use any of them. Here's a fun rabbit hole, if you want to know more.)

Entity Prediction And Extrapolation?

In our titles A Valley Without Wind 1 and 2, we took our existing lockstep code and changed it into something more appropriate for an action game. We added in our own form of entity prediction (on the client and server side), and error correction interpolation. Since it was a co-op game (not competitive), and cheating wasn't a concern, there was a lot that we were able to do to make that go more smoothly. However, this sort of approach really only works when there are small numbers of entities, a few dozen rather than tens of thousands, and only so many states they can have on them (a single ship in AI War 2 can have several dozen states in its data in hundreds of thousands of possible combinations).

So for the most part, the sort of entity interpolation and input prediction that most action games use (that link is an excellent article on how Valve's Source Engine does it) is a complete no-go for us here.

As one aside, however, back when we were first designing AI War 2 at all, we did use concepts from this in order to handle the entire "visual layer" of AI War 2, and this has had many benefits for us in both single-player and multiplayer.

Essentially, we're running our extremely demanding game simulation at a mere 10fps, no matter what, which is very similar to how things are done in most physics simulations in most major games (AAA and indie) regardless of if they are singleplayer of multi-player. Those other games wind up interpolating between physics steps at a higher framerate, and we do the same thing with our visual display and camera and player input reacting at a much higher framerate than our simulation itself.

Essentially we use that same sort of time-slicing that both Valve and Unity discuss (and PhysX does the same thing), but we do it in a slightly novel way to handle how things are drawn. There are probably other RTs games that do the same thing, but mainly it's something that an MMO would do, since a MMO is another rare case of a game showing an individual player just a small part of a larger simulated world.

But we digress. At any rate, this sort of off-main-thread simulation is both exceedingly common but also unusual in how we implemented it, and it buys us a lot of processing power and ability to multithread that lets us also make small corrections "just happen" when there are sync errors. So that's what we'll get into next.

Sync Correction Code

In AI War 2, we're simulating tens or hundreds of thousands of agents, and a lot of other data at a faction or world or arbitrary-modder-data level. So in a time-sliced fashion, we have the host simply go over a lot of the data and forcibly sync that to clients. In other cases, the host sends a few bits of data to clients for them to compare to their own entities and then make requests for full entity resyncs based on any divergences.

This makes it so that our simulation can react to the simulation being out of sync and self-heal, and in cases of fairly minor discrepancies (the sort you might see from a floating point error) it will actually do so in such a way that our existing "visual later interpolation code" will automatically and smoothly handle the interpolation so that you don't even notice it. In other cases, the sync is more "catastophic" in nature, where ship number 345466 is a nanocaust ship on plant 5 on the host, but a BobX player ship on planet 17 on the client. In those cases, it winds up finding that out within a few seconds (on average within 4), and deletes both wrong ships on the client and recreates them in the image that the host has.

If this was a competitive game, or a game where we were worried about malicious hosts cheating, then this would be big trouble. But as this is a co-op game typically played with friends, we're able to take the host in good faith and deal with sync data in a way that assumes lockstep works most of the time (and it does), but non-destructively fixes divergences when those arise (and they do).

In essence, we're combining several different networking models for maximum performance in as large a simulation as we can possible muster.

Alpha Notes: During alpha, the sync code may be triggered too frequently because of things that could better be in lockstep, and we can improve performance in various areas by improving the determinism of the underlying code, or by providing tools for specific factions or mods to note that they need extra sync at times. At any rate, this is one of those things that we can only really address as we see what sort of data we get, so if there's a particular savegame that is throwing off tons of divergences to an unreasonable degree, that's a great one to throw on the bugtracker for us to look at.

Various Logging Tools For Multiplayer

There are several different methods of enabling several different kinds of logging. All of them are aimed at different purposes, and most of them slow the execution of the game down. So you'll want to enable with care. Let's talk about each of them, and when you would use them. You're going to get basically no information of use from most of them, so it's a matter of when you might use them in order to send logs to us.

Log All Decoded Network Sync Data To Disk

This is really only useful if a client and the host both have this on, ideally from the start of the client connecting to the host. If you are seeing repeated errors on the client or the host that pop up and say something about "serialization" or "sync" or something to that effect, then there may be a problem with the sync data serializing properly, and this would be a way to document what is actually happening between computers.

1. First of all, you need to have a savegame that your host is able to load, and that once the client connects to it, it reliably causes the problem within 30 seconds of realtime at an absolute minimum. If it requires unpausing and clicking around in some specific fashion, that's fine, but make sure you document what those steps are and if the client or host has to do it.

2. Now that you have this save in hand, look in your PlayerData folder and see if there is a NetworkSyncMessages subfolder. On both the client and the host, if there is anything in that folder, delete it.

3. Now make sure that this "Log All Decoded Network Sync Data To Disk" setting is on on the host AND the client. It's under the Network tab of your personal settings window.

4. Have the host load the savegame that demonstrates your problem. You will note that nothing gets logged to disk on the host at this point, because there is no client to share sync data with.

5. Have the client connect to the host, and as soon as that connection is complete it will start logging a tremendous amount of data to your disk. It's something along the lines of 200kb every second, which is one reason you really can't let this run for too long or you'll have hundreds of MB of data.

6. Quickly unpause and do whatever it is that causes the problem. Once the problem has happened, have the client and the host both kill the game. If you're running in windowed mode, then hitting the X in the top corner of the window (or red dot on OSX) will do it. On windows, you can use Alt+F4, or on OSX you can use Command+Q.

7. On each of the two machines, zip up the entire contents of the NetworkSyncMessages subfolder, and be sure to label which one was from the host machine and which one is from the client machine. We will later unzip both into separate folders, and compare them with a tool like WinMerge, which will show us all of the differences in what the host thinks it sent and what the client thinks it received (and vice-versa in some cases). Somewhere there is some form of typo or inconsistency, and this will let us find and fix that.

In order for us to verify the fix, we will also need your savegame from the host that you used to reproduce this, and it would probably be helpful to have the ArcenDebugLog.txt file from both the client and the host. But it's really not required.

As far as logging goes, this is pretty intensive, so this is not something we expect most people to do most of the time.

Log Decoded Network Ship Sync Data To Disk

This works exactly the same as the "Log All Decoded Network Sync Data To Disk" above, but this just focuses in on ship syncs themselves. If you see for certain that the bugs are somewhere in the GameEntitySquad serialization, then you could use this one, as it definitely logs less data. But frankly it's going to be hard for you to know when it's only limited to this area, so if you're going to all the trouble of logging, you should probably go ahead and log the whole thing.

Network Debugging Mode

This logs a bunch of stuff to your local ArcenDebugLog.txt file, but outside of testing a new networking framework there's not much purpose for this. This is for testing and demonstrating the reliability of messages, not their contents, and that is critical when implementing a new networking framework (which, as noted, is something a modder or external developer could do if they want support for more than Steam, GOG, and LiteNetLib), but that's when you'd be using this.

Log Network Traffic To Disk

This is a more extreme version of the above, which logs in even more detail. So very useful for implementing new networking frameworks, and for when we were first developing the protocols that AI War 2 uses to talk back and forth in general one layer above that, but it's no longer useful for most situations for folks.

Network Logging Includes Frame Auths

Normally the above logging excludes frame auths, which happen 10x a second, but this turns them back on if they were needed for some reason.

Network Logging Includes Command Batches

These also are so frequent and reliable that they are too much of a clutter, but then aside from sync data this is basically all of the game data that is being passed back and forth. With this off, very little is logged beyond the actual initial connection logic (and now the sync data also).

Network Logging Includes Sync Checks =

Same notes as above, essentially. This is voluminous and frequent data, but then again it's most of the data that makes the game actually work.

Write Network World Serialization Logs

This is something you would use if the client has a corrupt world immediately upon connection to the host. To be useful, both the host and client would need this setting on, and then we'd need the NetworkWorldSerialization.txt from the host, and the corresponding NetworkWorldDeserialization.txt from the client. We would then look at the differences to see what is wrong with that initial world sync.

Show Network Sync Details In Escape Menu

Ah, now this is something useful! It has a very different output on the host and on each client. This data is being collected no matter what (in memory, not on disk), so having it show causes no extra CPU load or slowness. The data is all just aggregated integer counts, so it's very inexpensive for us to collect in the first place. The main reason this field exists at all is to reduce the clutter on the interface.

There are times where we might want to see screenshots of this data from the escape menu on the host and clients, so that we can have a general sense of potentially why performance is bad in some savegame. Basically this is kind of broadly diagnostic, telling us if there are too many syncs happening, but not why. Looking at this data yourself, if you see the PKID or ship sync numbers skyrocketing suddenly, then that might be a good time to make a savegame and send it to us, because something that is breaking lockstep is happening.

You only see these numbers go up AFTER lockstep is broken, however, so it can be a bit too late by the time you think to save (unless it's an ongoing process that is breaking lockstep). But if it's something like "at every wave there's a major amount of desyncs that it has to fix," then you can make that observation yourself and then give us a save from right before a wave unleashes, or something like that.

If you are suspicious that network traffic spikes at certain times, this will also give you hard data to back up your suspicions. You can watch the network traffic totals, and if those spike after certain events, or when you give certain orders or do certain types of battles or something, then that can give you confirmation that your hunch is right, and you can send a savegame to us for analysis.