AI War 2:The Paradigm Shift

Known Issues

• Any bugs or requests should go to mantis: https://bugtracker.arcengames.com/view_all_bug_page.php

• Multiplayer is disabled but coming very soon. We first focused on tightening up the single-player loop (more info here), so thanks for your patience!

Multiplayer Remaining Todo List

Before Alpha

• First handshake messages going back and forth between two computers (since 2018).

• World data passed from one computer to the next (in new format).

• Re-code GameCommands to be more efficient and special-purpose. This is probably a job that is a couple of days long, and will potentially lead to widespread bugs for a week or so after it.

• Get Steam and GOG integrated as networking frameworks, once things are working on Forged.

• Get the basic connection interfaces working, so that you can choose to connect to players via Steam, GOG, or a direct IP address (forged). This will be a moderate challenge, because Steam and GOG both have APIs that I'm pretty unfamiliar with in this exact area. Once the connection is going, it's much easier. But with the new compiler chain, at least I can hopefully get these going in a matter of days or a week rather than it being something that takes multiple weeks.

• V1 of the desync detection and correction code, which should probably only take a few days.

Before Beta

• The other features on multiplayer, mainly regarding things like donating fleets between one another, and/or whatever else we come up with that is desirable.

• Make the desync detection and correction able to correct factions, not just ships.

• The really big one that remains is making sure that the cross-machine identifiers (PrimaryKeyIDs) are consistent between machines. I don't fully have this figured out yet, but I think that the interim state of it will essentially be that there are occasionally too many messages being passed around because of rolling sync errors. I will probably punt this issue into something I look at during the alpha, so that I can gauge what sort of impact it really has on performance, and where the problems are coming from.

• Make sure that the lobby fully works as we expect, and various other small UI systems to get multiplayer basically playable. A lot of work went into the lobby in particular in the last few months to make that as close to as ready to go as possible.

Before Full Launch

• Whatever changes we need to make to balance in order to make things "feel right," which will be a matter of working with the multiplayer alpha and beta testers. A lot of things we already did in the past, like making science collection a humanity-wide thing that each player gets a copy of, rather than something people have to do individually (what a pain that was in AIWC). We will have to scale waves like we did in AIWC multiplayer, or in some other fashion. But a lot of the difficulty scaling is inherently handled by AIP being higher when you have to take more planets in multiplayer.

• If we're seeing network degradation or other issues due to the constant need to sync errors, then that will be to be investigated and improved. But those things are most of what the focus of the alpha/beta will be on.

What's this phase all about?

We've been preparing for this for months, tightening up the codebase and getting everything ready as much as possible. Now it's time to start having machines actually talk to one another, and then refine from there. You can see at the top of this page what the current todo list is. We expect to be into beta of multiplayer in August. Hopefully the alpha and beta periods are both short due to all the work of the last phase, but we shall see how it shakes out.

Once we get into a stable beta period for multiplayer, then we'll let the clock run a bit and Chris will work on adding interplanetary weapons to the base game as a free update (that's the last of the kickstarter stretch goals). This phase should wrap up all our kickstarter promises (including a laundry list of other smaller items).

Badger has already done the bulk of his work for the second expansion, Zenith Onslaught. Chris has done none of his work for that, as yet. Once multiplayer is in the "let it exist in beta and have people bang on it" phase, then Chris can circle in and get his work done for DLC2 along with those base game bits for interplanetary weapons, etc. The full version of multiplayer should launch alongside of DLC2, but we're not sure exactly when. October?

Either way, multiplayer will be in a solid state that is "beta but just because we want more time with more people testing it" for a month or so while the DLC2 work from Chris's end happens. None of the actual multiplayer stuff is delayed in any fashion for DLC2, it's been the other way around.

Version 2.100

(Not yet released -- we're still working on it!)

• Carriage returns and newlines are now supported by the "condensed" string format. This will prevent tutorial messages from turning those characters into question marks when they are dumped to the chat log. Note that this will only affect chat log messages logged in this version and on, not past versions.
  • Thanks to ParadoxSong for reporting.

• The left and right click functions of the metal display have been switched to be more natural.
  • Thanks to Waladil and Galian Gadris for suggesting.

• The "Spire Railgun Shop" mod that comes packaged with the game has been updated to use the icons from their new locations, and will no longer error.

• We've reworked our "Window_PopupScrollingColumnButtonList" a bit to add some new handy features for us.
  • In the factions selection window, when you are adding new factions, it now still lets you see factions that you can't add because there is only one per galaxy, but now it shows them in red, doesn't allow you to select them, and explains why in the tooltip.
  • In the debug menu in the escape menu in the main game, the change music submenu now shows the current selection in red (with the note that you can't change to it because it is already playing), and it shows all of the options sorted alphabetically except for the current track, which it shows as the first item.
    • Also fixed a bug where the music you selected to play would not actually be the track that played; it was choosing a random other track.

Scourge Intelligence Increases

• Fix a bug where the scourge were periodically suiciding Defensive Fireteams into player positions.
  • Thanks to ParadoxSong for the save.

• On intensities >= 5, the scourge can create cloaked "blockade running builders" if the player has tried to blockade the scourge into a small region of the galaxy
  • Thanks to ParadoxSong for the suggestion

• On higher scourge difficulties, the scourge is intended to sneak a builder off to another part of the galaxy at the beginning of the game, to make it much harder for the player to blockade them in. This was not working on some map types, including the X type.
  • Thanks to ParadoxSong for the save that exposed this problem.

• These changes should make the scourge much harder to deal with in general, and on the X map type in particular.

Initial Multiplayer Connection

• AllowOtherPlayersToConnect was a setting on the World object in the past, and thus something that was getting saved into savegames and that dictated how a lot of things worked for player connections.
  • We've removed this concept, and instead are making it based on the menu choices you make on the main menu.
  • We now have a DesiredMultiplayerStatus on ArcenNetworkAuthority, which lets us recall if you wanted to be the client, a host, or just left alone in single player.

• The basic shells of SteamSocket and GOGSocket have been added to the game. The in no way function yet, but this will allow for them to be built out in the future.

• The SpecialNetworkType has been moved into ArcenUniversal, and is now copied onto ArcenSocket along with the InternalName of the row that created the socket.
  • There is now a list of AllPossibleSockets on the ArcenNetworkAuthority to allow it to swap between sockets as needed, and particularly by type.

• When you open the multiplayer menu, if you have that enabled, it will now set a default network framework for you based on the status of your game at that time.
  • If you are logged into Steamworks, it will set steam as the default network.
  • If that didn't work or wasn't true, and you are logged into GOG Galaxy, it will use that as the default network.
  • If none of those worked, then it will pick the first "other" network type, which in this case would be Forged.

• All of the options in the multiplayer menu for hosting a savegame, quickstart, or custom game now properly do those things identically to as if it was the single-player menu, but put you into host mode first.
  • The single-player menu puts you into single-player mode, and makes sure that your ArcenSocket is set to NullSocket.

• When you try to go into one of the client or host multiplayer areas from the multiplayer menu, it now properly activates your most recent chosen network framework, or switches to the first available one that actually can function right now on your system and gives you explanations for why it did that.
  • Essentially there is a fair bit of under the hood plumbing now for the frameworks to register themselves with the network authority class and explain about their availability.

• If you are looking to connect to another machine, then the "connect as client" tooltip now gives you information about what to expect based on your currently-selected framework.
  • Are we connecting by IP? How much of a hassle will that be? Is it going to be by selecting a friend on Steam or GOG? Etc.

• The main menu has been updated so that the "network" button at the top of the multiplayer menu now updates its text to show the current network.

Version 2.099 Last Rabbit Holes

(Released July 15th, 2020)

• There is now a galaxy setting to grant players Watch vision for 1, 2 or 3 hops from all command stations.
  • Thanks to uhamster9 for inspiring this addition.

Fixes And Tweaks

• Improved the way that time-based pools are incremented in time, so that we're never having accidental cases where we miss one.
  • Also made it so that they actually work as intended when the world is being cleared (running 16 seconds' worth of time at once). This then keeps the pools smaller if you repeatedly load savegames.

• On the escape menu, under where it shows you the various numbers of objects in memory, it now shows you the number of galaxy planets and planet links activated and in existence.
  • This lets us tell when these have a memory leak, if there ever is one with them.

• Additionally, galaxy map links are now pooled for the first time ever, as the discarding of them was previously a definite memory leak. Normally this just happened between saves and loads, but it was also possible to happen when nomad planets moved.

• In addition to the existing time based pools, we now have a new ExternalizedPool.
  • This is now used for the galaxy map links, and actually we have stopped the galaxy map planets themselves from using the super-old IArcenGameObjectResourcePoolable and they also now use this.

• The way that planets for the galaxy map are initialized, and the way that their positions are set, is completely overhauled.
  • This is more efficient and properly uses pooling (which never was working properly before, it turns out).
  • It also makes the position of planets automatically react to them having moved in the game sim, without having to do anything special.

• The arcencolors asset bundle has been removed, with its contents simply rolled into arcenui. Ultimately this is a bit faster to load, and saves a bit of disk space and RAM also.
  • Also removed the aiw2squads bundle, and the examples bundle.
    • The examples bundle simply has its example content still there, but not in a bundle (it's to help aspiring models modders).
    • The squads bundle is old data that we've not used in forever, and so has just been cleared out. It wasn't huge, but wasn't worth being there.

• Starting to load a savegame (including for a quickstart or the last settings for a lobby) now writes to the log when you start the process, and at the end of the process writes how long it took.

• Added a new debug setting: "Write Detailed Savegame Timings"
  • When loading a savegame, including the 'last settings' for the lobby or the basis for a quick start, keep track of detailed timing data and write that into the normal debug log so that it's clear what is taking longer and shorter amounts of time.

• It turns out that chasing "why savegames now take 16 seconds to load sometimes" was a snipe hunt. The serialization sizes logging, when enabled, actually was causing that amount of slowness. With that off, it loads in about 2 seconds.
  • The extra instrumentation that we added for the savegame timings is still nice, but ultimately was not a useful bit of time spent at the current moment.

• Fixed an issue that could happen in the lobby in particular where it could not properly save your prior settings because the fleet ID or speed group ID was lass than zero.
  • This pretty much could only happen if you already had another error first.

• Fixed an exception in SeedNormalEntities that could happen if you rapidly tried to regenerate maps in the lobby in just the wrong way.

Under The Hood Improvements For Icon Modding And RAM Usage

• The last of the icons from the ExternalIcons folder have been moved into the unity project that generates the asset bundles, simply to dispel any potential confusion about the fact that they can be edited directly and that have some change on the game without recompiling asset bundles. This was a frequent modder confusion.
  • That said, the ones that were in the Official_1 folder are now in a CentralIconBits, and can now be used in the UI for the first time if we ever want to. That means things like health bars and whatnot could in theory be used in the ui if someone wanted to, whereas before they could only be used on gamespace-level icons.

• Cleaned out a variety of unused icons from the arcenui asset bundle, and in general tidied up some of our organization of working files versus final files.
  • Also standardized the naming of certain things.

• Broke the "official icon dictionary" out into six dictionaries:
  • One is for the "central bits" like health bars and so on, and would be the same for all ships.
  • Two are for the overlays, like for saying what kind of starship or guard post something is. There can be more than one of these, and the fact that these are split out like this demonstrate the modding capabilities and how things can combine.
  • Three are for ship icons and their borders, and there can be more of these modded in as well. As with the overlays, not only does this demonstrate how such mods would work, but it also has the side benefit of slightly less VRAM usage in a few cases.

• The way that GUI icons are discovered from the xml based on their gamespace-icon counterparts is completely revised, and is now based on extra information in the ExternalIconDictionary entries.
  • This is a lot more flexible, and allows us to load icons from multiple asset bundles and/or multiple files within the same bundle.

• The naming for finding icons is now much more complicated, in the sense that they don't all come from a single dictionary called "Official."
  • So there were over 1100 places in the base game and expansions where we've had to modify to point to the new dictionaries. Any mods would also need to be updated to point to the new places, unless they want to start using their own icons (once we put together a tutorial for that).

• The external sprite dictionaries have been updated to be able to properly load in in multiple threads.

• The game is now able to load the xml files for external icons from any mod or expansion, in:
  • Expansions/[ExpansionName]/GameData/ExternalIcons/[thefile].xml
  • XMLMods/[ModName]/ExternalIcons/[thefile].xml
  • XMLMods_NonDistributed/[ModName]/ExternalIcons/[thefile].xml
  • It's worth noting that these are xml files that are generated by Texture Packer, not xml that we create.
  • These files are referred to in the "ExternalIconDictionaries" xml as something along the lines of xml_path="Official_CentralIconBits.xml"
    • It will then search the main game's folder (/GameData/ExternalIcons/[thefile].xml), then all the expansion folders, then any activated mod folders for a file with that name that was specified.
  • This allows for mods to be self-contained when it comes to their icons, and it allows expansions to have their own icons that are not packaged with the main game (not that we have imminent plans to do that).

• On the icons used in the game, we previously were not using any compression, which made them absolutely massive in size (80mb for the main dictionary).
  • This was done in order to preserve quality, but we're also using GIANT icons in order to allow for really super-high-res displays of the future, as well as square ones to allow for ideal mipmaps, etc. So the lack of compression was hugely overkill.
  • We're now using compressed textures, which turns what was 80mb into more like 8mb. This has a very direct impact on performance and VRAM usage, so particularly on low-spec machines they will likely run far better.

• The game now automatically constructs custom "gimbal materials" for the in-gamespace icons, in as many combinations as are needed for the things that it has drawn for you so far.
  • The total number of materials always in the past was "one," but it was using a single large dictionary and a lot more VRAM -- and it had a finite amount of space for things. There was a solid chance we were going to run out of space in that one dictionary as part of implementing DLC2. But either way, mods were not possible. The new number right now is 8, looks like.
  • Now it may make a handful of materials, depending on how many mods you have installed that have custom icons, and several for the core game and expansions itself. But these are vastly smaller, and while this does increase the number of "draw calls," each material is still mostly instanced together (there are a few mesh differences depending on if there are health bars shown or whatever), but the overall load on the GPU pipeline is lower.
  • All of this is automatically handled by the game in as efficient a pattern as possible, and if you're curious how many combination materials it has created, you can see that in the escape menu at the bottom of the list of memory pooling info.

• A new debug setting, "Log Gimbal Icon Material Creation", has been added:
  • When this is on, any 'gimbal icon' material creation events will be logged as to what was created and how long it took to do so.
  • Turns out that this uses an immeasurably small amount of time for each material (less than 1ms each), so that's awesome.

Under The Hood Improvements For Mods With Code

• The game is now able to load dlls from mods or expansions, rather than just from the central game folder.
  • This works just like the external icon files, and basically looks for the dll in the central game folder first (/GameData/ModdableLogicDLLs/[thefile].dll), and then looks at expansions and then installed mods if it can't find them.
  • The paths are:
    • Expansions/[ExpansionName]/GameData/ModdableLogicDLLs/[thefile].dll
  • XMLMods/[ModName]/ModdableLogicDLLs/[thefile].dll
  • XMLMods_NonDistributed/[ModName]/ModdableLogicDLLs/[thefile].dll
  • We really don't have a need to do this with expansions, but the flexibility is nice.
  • With mods that are more than just xml, however, this finally lets a modder distribute just a single folder that has everything in it, and not have to worry about putting some things in central game folders.
  • This is only partially tested, but should work.

• It's worth pointing out that expansions and mods already did (and still do) have the capability to load asset bundles (icons, music, sound effects, models, textures, shaders, etc) from their folders.
  • The structure for those is /GlobalBundles/ in the main folder for anything not platform-specific, /AssetBundles_Linux/, /AssetBundles_OSX/, or /AssetBundles_Win/ for the things that are.
  • Inside the folders for expansions it is: Expansions/[ExpansionName]/[OneOfTheAboveFolders]/
  • Inside the folders for mods it is: XMLMods/[ModName]/[OneOfTheAboveFolders]/ or XMLMods_NonDistributed/[ModName]/[OneOfTheAboveFolders]/
  • None of this is new, but it's worth mentioning for now.
  • This is only partially tested, but should work.

Larger Gamespace Icons And Fixed Galaxy Map Line Offsets

• We also now have a non-billboarding version of our shader for purposes of our icons on the galaxy map.
  • Our core shader does on-GPU (read: hyper efficient) billboarding to always fully face the camera. This is very useful in the main view where you are moving around the camera a lot but always want the icons to face you.
  • The downside, however, is that with a largely-straight-overhead camera like the galaxy map uses, this billboarding would cause there to be a perspective shift where the icons for planets would get offset from the lines leading between planets no matter what we did. It also caused them to move relative to the text that was next to them.
    • This offsetting of the lines to the planets was one of the largest "visual papercut" issues that we've had for a really long time, and it is finally fixed!

• The ship icon scale has been adjusted from a default of 1.5 to 2.2, and will affect all prior settings files.
  • It's description has also been updated for the personal settings tooltip: For the icons in the main display area (not the sidebar), how large should they draw? Default is 2.2. If you go too large, it can be hard to see things because they overlap too much. If you go too small, they can get extremely blurry because of flipping to a lower mipmap. The larger your screen DPI, the smaller you can go without losing clarity.

Prior Release Notes

AI War 2: Building Multiplayer