AI War 2:The Paradigm Shift

Known Issues

• Any bugs or requests should go to mantis: https://bugtracker.arcengames.com/view_all_bug_page.php

• Multiplayer is disabled but coming very soon. We first focused on tightening up the single-player loop (more info here), so thanks for your patience!

Multiplayer Remaining Todo List

Short-Term Work

• Get chat controls working in lobby.
  • Figure out why chat is invisible on factions tab of lobby.

• Client bugs in lobby:
  • Not able to change planet count.
  • Map sub-types not shown correctly after they change.
  • Clicking regenerate map does not work on client.
  • Randomize seed from client does work, but causes host and client to diverge.
  • After several regeneration attempts from client and host, client gets black screen with no responsiveness or errors.
    • Possibly this causes the settings for the game to get completely zeroed out, which then leads to the strange black and white view the next time you log on.

• Show when clients are in spectator mode, or what faction they control.

• Allow clients to assign themselves (or be assigned) to player factions.
  • As many clients can share a faction as want to, no problem.

• Make sure that proper player name is shown for the first player, and not just your own personal name.

Before Alpha

• Re-code GameCommands to be more efficient and special-purpose. This is probably a job that is a couple of days long, and will potentially lead to widespread bugs for a week or so after it.

• Get GOG integrated as a networking framework, now that things are working on LiteNet and Steam.
  • This will be a moderate challenge, because GOG has a different API from either of the other two. Right now the main thing we are waiting to hear back on is fragmentation and recombination logic, which we hear is in there, but that we don't see how to use yet.

• V1 of the desync detection and correction code, which should probably only take a few days.

Before Beta

• The other features on multiplayer, mainly regarding things like donating fleets between one another, and/or whatever else we come up with that is desirable.

• Make the desync detection and correction able to correct factions, not just ships.

• The really big one that remains is making sure that the cross-machine identifiers (PrimaryKeyIDs) are consistent between machines. I don't fully have this figured out yet, but I think that the interim state of it will essentially be that there are occasionally too many messages being passed around because of rolling sync errors. I will probably punt this issue into something I look at during the alpha, so that I can gauge what sort of impact it really has on performance, and where the problems are coming from.

• Make sure that the lobby fully works as we expect, and various other small UI systems to get multiplayer basically playable. A lot of work went into the lobby in particular in the last few months to make that as close to as ready to go as possible.

Before Full Launch

• Whatever changes we need to make to balance in order to make things "feel right," which will be a matter of working with the multiplayer alpha and beta testers. A lot of things we already did in the past, like making science collection a humanity-wide thing that each player gets a copy of, rather than something people have to do individually (what a pain that was in AIWC). We will have to scale waves like we did in AIWC multiplayer, or in some other fashion. But a lot of the difficulty scaling is inherently handled by AIP being higher when you have to take more planets in multiplayer.

• If we're seeing network degradation or other issues due to the constant need to sync errors, then that will be to be investigated and improved. But those things are most of what the focus of the alpha/beta will be on.

What's this phase all about?

We've been preparing for this for months, tightening up the codebase and getting everything ready as much as possible. Now it's time to start having machines actually talk to one another, and then refine from there. At this point, both LiteNetLib and Steam are fully working as communication frameworks, but now the game itself needs to fully handle all its logic properly on top of those. You can see at the top of this page what the current todo list is. We expect to be into beta of multiplayer in August. Hopefully the alpha and beta periods are both short due to all the work of the last phase, but we shall see how it shakes out.

Once we get into a stable beta period for multiplayer, then we'll let the clock run a bit and Chris will work on adding interplanetary weapons to the base game as a free update (that's the last of the kickstarter stretch goals). This phase should wrap up all our kickstarter promises (including a laundry list of other smaller items).

Badger has already done the bulk of his work for the second expansion, Zenith Onslaught. Chris has done none of his work for that, as yet. Once multiplayer is in the "let it exist in beta and have people bang on it" phase, then Chris can circle in and get his work done for DLC2 along with those base game bits for interplanetary weapons, etc. The full version of multiplayer should launch alongside of DLC2, but we're not sure exactly when. October?

Either way, multiplayer will be in a solid state that is "beta but just because we want more time with more people testing it" for a month or so while the DLC2 work from Chris's end happens. None of the actual multiplayer stuff is delayed in any fashion for DLC2, it's been the other way around.

Version 2.113

(Not yet released -- we're still working on it!)

• Fix a null reference exception if the game wants to spawn a Relic but you own basically every explored planet
  • Thanks to Lord of Nothing for reporting

Version 2.112 Steam Networking Complete

(Released July 31st, 2020)

• The description of the Ensnarer Battlestation now explicitly mentions that it can use AI Great-Turrets
  • A number of people have reported this as a bug over the years, so hopefully that won't happen anymore

• All of the Fallen Spire ships, including those in the Spire Railgun Shop optional mod, have had their health and shields doubled, but their firepower halved.
  • This should make battles with other mega-units last longer, which feels more appropriate, but keep their "damage over time" the same.
  • When these were going toe to toe with the AI Exogalactic War Front units, they seemed to disappear too quickly, making them feel weak. This should keep their effectiveness the same as before, but make the battles take about twice as long with those large ships, so that it doesn't feel like instant evaporation at least.
  • The one unit not adjusted is the Spire SuperDreadnought from the Spire Railgun Shop mod. That was already maxed out as a "kills everything and is basically invincible" sort of ship.
  • Thanks to a variety of players for weighing in on the feel of these units, and Badger for suggesting this adjustment.

• Rather than having a slider for the number of planets in the galaxy map view, there is now a dropdown.
  • This prevents the intense "regenerate map spam" and lag that was inherent in using a slider, along with making it easier to choose a certain number of planets.
  • We have made it so that it only gives you multiples of 2 below 60, multiples of 5 below 100, and multiples of 10 above that. This makes it easier to quickly peruse the range.
  • Each entry has some commentary on what it will mean to the feel of the map if you are hovering over it in the dropdown with the number of planets expanded.
  • Actually, it wasn't even regenerating the map after you changed the number of planets, which was really confusing! You had to hit regenerate map. Now there is no confusion.
  • All of this is easier to use as a player in general, on top of being something that is easier to deal with in multiplayer for sync purposes, too.

• Fixed a longstanding bug where dropdown selections that were open could persist when leaving the lobby or changing tabs within the lobby.
  • Thanks to MaverickPenkenn for reporting.

Milestone: Steam Networking Integration Complete

• Fixed an issue where multiplayer clients disconnecting from a host would try and fail to save the last settings from the lobby. Now it does not save the lobby settings from a session you are just a client of.

• All of the networking frameworks have had their "send to anyone here and myself" code consolidated into the network authority, so when an interface is partially implemented it won't wind up stalling out waiting on the host.
  • Less code duplication is also always good, but it wasn't clear that this was a complete commonality until we got partway into the Steam framework implementation.

• A new ArcenSteamClientConnection wrapper has been created to keep track of the Steam connection on servers.
  • This now properly is registered while the steam connection is alive, and all of the "waiting for player to connect" and "player disconnection" events now happen properly.
  • For some reason, we are not getting the "host has accepted you" callbacks on the client end, but that doesn't really matter. As we start actually having our challenge-response data sent via the higher-level networking code, that bit will become irrelevant as it establishes itself within part of one second anyway. It's just a curiosity, or potentially something that is only meant to trigger host-side. Frankly the documentation is sparse in points, both from the Facepunch C# wrapper and the Valve C++ baseline, so this might be working as intended.

• Substantial progress on the Steam network transmission of data. Still not quite seeing it register yet, but we're getting close.

• The full suite of network logging has been integrated into our steam networking wrapper, now.

• When a Steam host shuts down the connection, the steam client computers now definitely notice and react to that. The inverse was already working from the start.

• Figured out the missing information on how to get data from the facepunch steamworks wrapper (call Recieve() to trigger the other events), and are sticking with the derived classes methods rather than the interface method, since for relay connections it seems like the interface method is not fully implemented.
  • With this in place, we're now getting messages on the client machine, and they are properly decoded! The host for some reason still doesn't get messages back, but this is progress.

• Fixed a bug in our connection status window code that was causing it to stop showing a timer on connection stages after the earliest ones.
  • With LiteNetLib, that all flashes past so fast we could not tell that was even happening. With Steam currently stalling out with the host not getting messages, it became apparent.

• Previously it was possible for our socket mode to be off, and silently ignoring messages we were sending.
  • This was initially done to make it so that any lingering messages that would try to be sent after a disconnect would not cause visible errors, but in this case we've wound up with the socket state being off on the steam networking, despite it being active, and this thus resulting in client messages being composed but not actually sent. That was not obvious because of the lack of error messages.

• Figured out a stupid mistake in our own code, indeed a single line of code, which was preventing the client from sending data properly back to the host.
  • Steam networking is now fully functional in terms of sending data back and forth. There is some sort of issue with the world data not getting across properly, but we have to now verify whether that's limited to Steam or also happens on LiteNetLib.

• Added in a substantial amount of extra error tracking in our univeral and game-specific network message handlers.
  • This helps us to find out where errors actually are.
  • In the case of steam, right now we are getting some exceptions when the host tries to tell the client what their player profile number is.

• Fixed a simple typo that was having us send a junk message via both frameworks to the client when they first connected.
  • In both cases, this led to the galaxy map not showing up properly on the very first world connect. But in the case of Steam networking, it also led to an exception because the byte array was too small.
  • This extra smallness is actually more accurate, and basically it was only not erroring with LiteNetLib because the buffer array was larger than needed because they are pooled.
  • This is an interesting difference between these two networking frameworks, but neither one is particularly "wrong. But it does make Steam slightly more sensitive to certain errors in a way that is useful, because it's an error in both cases that manifests in other places.
  • Anyhow, this seems to argue for us to internally use Steam as our main testing framework, additionally because that means we are round-tripping out through the internet and Valve's servers and back rather than having the speed of a LAN.
  • This also means that the Steam networking library is fully completely integrated, which is a very major milestone! Now it's all game logic, regardless of what is going on with the two networks.
  • Also worth noting that it's under 3 seconds to connect a client to the lobby in our test case, using the Steam relay servers, etc. On the LAN it is under one second, so that's pretty killer.

More Multiplayer Work For All Frameworks

• "Planet GalaxyMapVisuals in CenterGalaxyViewOnPlanet null!" will no longer happen. It was a thing relating to trying to center on a planet slightly too early, and the population of the galaxy map still having failed.
  • We are now bypassing that and using the raw internal transform offsets to calculate the correct camera position.

• In response to the initial challenge from the host, the client no longer just sends their profile name. They now also send their list of installed and enabled expansions and mods.
  • The host then compares that list to its own installed and enabled expansions and mods. If there are differences, then it rejects the client connection and does a popup on the client and a chat message on the host side explaining why.
  • It also notes in there that both parties can enable or disable expansions or mods in the Game tab of the Settings menu to get things to match.
  • Please note that, yes, in some cases there will be mods that are enabled but which "wouldn't do any harm" because their functionality won't be triggered in the campaign in question. We don't have any way of knowing that, so we have to err on the side of caution.
  • When it comes to expansions, there are none that "wouldn't do any harm," because all of them add some ships and features to the game in general that all players need to have. For instance, the first expansion adds a bunch of new turret options that are available to choose from in any game you play with that expansion on.
  • At any rate, we've gone out of our way to make it easy for people to disable expansions and mods so that they can play with friends who don't have those same expansions or mods. The last thing we want is for the only solution to this sort of thing to be someone reaching for their wallet or having to faff about in the filesystem.

• Low-level chat messages that are sent to everyone from the host are also now sent to the host themselves. Previously, they could not see messages of this sort, turns out.

Version 2.111 Initial Steam Connection

(Released July 30th, 2020)

• Fix a null reference exception in my new spire relic code
  • Thanks to Chuito12 for reporting

• Nanocaust: Make the AI wait a bit longer before unleashing Exogalactic units on it. Remove some deprecated code
  • Also the exogalactic units won't hang around to kill the entire nanocaust, so there's a chance it can recover.

Milestone: Initial Steam Networking Connection Between Client and Host

• The Steam versions of the game now initialize relay access on startup, to make it so that the first connection using Valve's relay network will be extra fast.
  • If you wind up not playing any multiplayer, this is absolutely harmless. If suddenly it is flagging your firewall on game start on the Steam versions of the game, though, that might be why.

• The basic connection through Steam's newer networking sockets connection is now in place, using their relay servers as recommended. This is the approach that absolutely bypasses NAT traversal and anything else of that sort, and in fact hides your IP address from anyone you are playing with. The claim is that their backbone is faster in most cases than the general internet one.
  • If need be, we can make a "Steam over IP" version that allows for their networking sockets stuff to run more directly over UDP, and only fall back to their relay servers if there is a problem. This would be a lot like LiteNetLib, except with much better NAT punchthrough making use of ICE and STUN and all those things.
  • For now we are sticking with Valve's recommended practices.
  • Beyond just getting the initial connection up, we don't have anything there yet, but on the server we can see the Steam username of the client who requested to connect and who forged the connection.

Version 2.110 AI Exogalactic War Front

(Released July 29th, 2020)

• Spire Relics now have additional text in their descriptions explaining either A. whether the relic is on route to a particular planet , or B. that this relic must build on this planet
  • Thanks to Lord of Nothing for suggesting

• All of the units that were previously named "Galactic War [name]" are now just named "[name]"
  • However, in the tooltips for them, their faction should now say "AI Exogalactic War Front" in whatever the hunter fleet's color is, rather than saying the AI Hunter Fleet.
  • This lets us keep the shorter name, while making their sub-group and source a lot more clear.
  • This does need testing, so please let us know if it doesn't show up correctly somewhere. This is using a new xml feature called "override_faction_name", which lets us use a different faction name for some units in a faction, but without actually creating a new faction.
  • Thanks to Ovalcircle for leading the discussion that led to this.

• Spending a ton of science will now increase a player faction's Overall Power Level very slightly.

• Remove some inaccurate description text for the Rorqual Hejira
  • Thanks to GreatYng for reporting

UI for Steam Networking

• Added the new ArcenNetworkConnectionOption class, which we are using as a basis for things like lists of your Steam friends that you can connect to.
  • On the network authority, there is a new PotentialServerOptions, which is used to list out such things. This can be used for anything that we need to list a variety of to connect to, not just friends in Steam and GOG. This is in contrast to the "type in an IP" method that we use with LiteNetLib.

• Started building out the Steam networking framework layer, currently based on the ISteamNetworking implementation that most games use.
  • We MAY opt to also start supporting the newer ISteamNetworkingSockets framework that Valve has rolled out more recently, but that would be an alternative networking framework in AI War 2 that you could choose between.
  • The fundamental premise of those two Steam networking setups is really different, and while the newer one would in some ways be easier for us to implement, it seems a lot less familiar to the average Steam player, involves IP addresses potentially, and is maybe less battle-tested, but who are we to say.

• Previously it was still possible to have some exceptions happen silently in the Engine_Universal main loop, and they would log to disk but not show up in your face. Fixed.
  • It was also possible to get some silent errors when clicking buttons and other UI elements, turns out. This made the application just seem not to respond to the click, but now it properly shows the error if there was one.
  • Also it was possible for certain errors in the UI call stack to prevent the display of the error window.

• When you try to connect to an IP address, or you change your active networking framework, the game settings now get saved immediately so that that is remembered even if you shut down the program right away.

• There is now a screen that will be used for Steam and GOG connections, from the client side, and in Steam it now:
  • Shows a list of all your Steam friends, with ones that are in AI War 2 at the top, then ones that are online, and then the rest below that.
  • Lets you sort to just ones that are in AI War 2 or online.
  • Lets you refresh the list to check for updated statuses.
  • There is a connect button for each friend, but that is not functional yet. Just haven't gotten to that part yet.

Version 2.108 Galactic War Units And The Exostrike

(Released July 28th, 2020)

• Completely rework the way Exogalatic strikeforce leaders work, to allow per-AI-Type customization. No functional change right now, but intended to support some new AI types for DLC2

• HandleHelperJournals() has been split out into a bunch of sub-methods, since they can error in rare cases and it would be nice to know where the error generally is.

• Fixed an issue where we were not letting negative risk analyzer data be saved properly. Turns out that has a good reason to be negative at times.
  • Thanks to Badger for reporting.

• Added a new UI feature that is a throwback to some of the functionality we had in various past games: the "center screen message"
  • This is basically a message that can show up near the top center of the screen, and stay there for a bit or for as long as needed.
  • It won't block the mouse, and it is over everything except modal popups and tooltips.
  • This isn't something we would want to use all THAT often, but for cases of things like "hey the server isn't responding, why is MP stalled," this is exactly the sort of UI functionality we need.

• Fixed a harmless bug where, for the last few months since we've been doing multithreaded loading of camera xml files, the "PrivateVisExtensions" assembly would sometimes have annoying but unimportant errors appear on startup.
  • Essentially, it turns out that loading classes from a custom dll is not entirely threadsafe. Go figure. The fix for that is simple, and it doesn't slow things down, but now we know.
  • Thanks to GreatYng for the first report of this we'd had outside of ourselves running into it.

• Fix a bug where macrophage achievements weren't triggering if you killed all the Telia
  • Thanks to GreatYng for reporting and StarKelp for the fix.

• The in-game credits have been updated to include two more individuals who are key DLC2 testers.

• Added new GetWasWorldStartedOnGameVersionOlderThan() and GetWasWorldStartedOnGameVersionAtLeastThisVersionOrNewer() to world, which let's people find out in a really efficient way if a world was started before some certain point.
  • We can't be accurate past a month or so ago, because we were not tracking the original savegame version back then. But it's useful for looking at saves older than the more recent ones and altering them if need be.
  • This is now used specifically to fix this macrophage disabled stuff only on older savegames than 2.108.

• Fix a mispelling in the Mesopotamian planet names
  • Thanks to Lord of Nothing for reporting

• Fix a typo, "reciever" to "receiver"
  • Thanks to Apthorpe for reporting

UI Changes

• In the Load Menu, campaigns are now sorted by 'most recent save game in that campaign' rather than alphabetically
  • Thanks to Puppet Master for the suggestion.

• The default sort for the load menu is now by date.

• Rename 'Exogalactic Strikeforce' to 'Exostrike' and 'Extragalactic War Units' to 'Galactic War Units' in UI. Having "ExoGalactic" and "ExtraGalactic" was a perennial source of confusion
  • Note that internally the code uses the old names for the moment
  • Thanks to a discord conversation involving relmz32, Oryutzen and NRSirLimbo

Dyson Sphere Buffs

• The Dyson Sphere's ships should now really level up after the hack
  • Thanks to GreatYng and zeusalmighty for the bug reports

• Dyson Sphere hack descriptions now say "This hack will make the Dyson Sphere very unhappy with you for a while" to clarify the behaviour.
  • Thanks to Apthorpe for suggesting

• The Dyson Sphere now gets additional income (ie it can build ships faster) that scales with AIP. As AIP goes up the Dyson and Antagonized Dyson get stronger

Multiplayer Work

• SendMessageToAllClients() has been split into two methods: SendMessageToAllClientsWhoAreFullyConnected() and SendMessageToAllClientsRegardlessOfConnectionStatus().
  • We really don't need heartbeats and frame auths and so on going to clients who are still connecting in.

• Also added a GetCountOfClientsWhoAreFullyConnected() that lets us know if there are any clients who actually have gotten all the way in (and thus we should send world data)/
  • And added GetCountOfClientsNeedingWorldData() that lets us know we should make everybody wait for a bit until those people connect in. Otherwise everyone won't be on the same page.

• Added a OnServer_IsAtPausedSpotWhereGameWorldCanBeSent() that lets us wait for a break in things happening (usually only 400ms or so at most) to then have a clean slate to send a world state to a connecting client without that state changing between them being sent the state and them getting it.
  • Along with this, if GetCountOfClientsNeedingWorldData() is more than zero, then it now pauses authorizing more simulation frames until the world is fully taken in by the new clients who are joining.

• When a multiplayer game is stalled out for some reason, it now uses that central screen message to let you know what is happening, and if possible, why it is happening.
  • The game itself will have stopped having any action at this point, though you can still scroll around and look at things and give orders. But nothing is proceeding on the enemy side, so you don't have to worry about it being in your way.
  • When a client is first connecting into the lobby, it also uses this, to basically get the host and any other existing players to wait a moment before making more changes, etc.
  • On the host, this is already set up to give some context about the exchange under the hood that is happening with the new client.

Multiplayer Transition To LiteNetLib

• Okay then... we are switching network libraries when it comes to our non-Steam/non-GOG transport layer. Instead of FORGE Remastered, we'll be using LiteNetLib.
  • FORGE has grown a lot over the years, but the version we were using was from 2018...ish. We looked into updating our version of the code, but were not thrilled with how complicated it was going to be. Since we had working AI War 2 multiplayer on FORGE at least as recently as 2017 (in an alpha state), staying with the existing code made a certain amount of sense.
  • However, we started running into really long delays with something as simple as sending 10 bytes to say what our profile name was. This was taking upwards of 14-18 seconds between two machines sitting three inches apart and both within five feet of a powerful wifi router. We verified that the polling was happening hundreds of times per second, and that massive amounts of data was being sent via sockets. But in the internals of the old version of FORGE we have, it was taking it quite a very long time to assemble that into a full message, despite all the churn of hundreds and hundreds of data reads that were apparently all just empty headers and then noise.
  • Time for a new approach. It's worth noting that the current version of FORGE still lists all the most internal classes as "in development and lots of dire warnings," so that was another reason for switching up frameworks.
  • We chose LiteNetLib at this point, and within 40 minutes of dropping that in, we had the first successful connection between the machines on using this framework for the game. It was just a simple "hello client" message in basic unicode, but it arrived extremely instantly, as expected.
  • Next steps are going to be doing some cybernetic implants on LiteNetLib, rather like what we did on FORGE, to make sure that it integrates with AI War 2 properly and our network authority, and that it communicates as directly as possible with our own pre-formatted bitstreams. Most of that should be part of one day of work, at this point. We shall see. Then it's back to the actual larger "build out the multiplayer part of the game" work, but minus the lag that was tripping us up the last day or two.
  • It's also worth noting that, with this switch, we had to unfortunately also lose the awesome NAT punchthrough work that Doug Fields did for us with the FORGE interface back in 2018, but LiteNetLib has some of that of its own. And hopefully most people use Steam or GOG if they are behind firewalls of any complexity (since relay servers then become required, and that's where the free relay servers are).

• NetPeer in LiteNet now inherits from ArcenNetworkClientConnection.
  • For now, we are using our own methods to generate the ConnectionIndex, rather than using the existing peer.Id. There may be some numeric conflicts, otherwise, but we can also change this later.
  • We also are initializing everything on the Arcen-integration side, keeping our own clientsOfServer list for quick access, and then calling the game-layer RespondToNewConnectionAcceptedByTransportLayer() that will kick things off.

• PrepareMessageForSending() has become WriteLogOfSend().

• Added a new SendToSpecificPeer() method onto LiteNet's NetManager. Not sure why that variant didn't seem to exist.

• Wiring back up our LitNetSocket wrapper class:
  • SendMessageToHost_Inner() done.
  • SendMessageToAllClientsRegardlessOfConnectionStatus_Inner() done.
  • SendMessageToAllClientsWhoAreFullyConnected_Inner() done, using the new SendToSpecificPeer() method.
  • SendMessageToSpecificClient_Inner() done, using our own ConnectionIndices for now.

• Booyah! Exclamatory phrases are not the norm in our patch notes, but this is an exciting moment. LitNetLib is sending messages, and doing so with speed and accuracy and a minimum of fuss. The username travels three inches in well under a second, as should be expected.
  • At first, I was getting some very strange data sent across, and I figured that this was going to be an endianness problem, but actually all of the .NET/Mono data is the same endianness from the look of things. Which makes sense, given that otherwise savegames would not work from computer to computer.
  • After spending the night with that rattling around in my brain, this morning I decided to definitely do some profiling of the data before I started messing with the #ifdefs.
  • First thing I noticed was that I'd chosen to use the RawData byte array, and immediately next thought was "I better check there's not a network header in there." Turns out, yes, there's a UserDataOffset. The first four bytes are not my data, and so of course it would give gibberish as if the endianness was wrong. Just starting my reads from the proper offset makes everything work instantly.
  • All of this is from a Mac to a Windows machine at the moment, but my I'll be testing three-way with my linux machine as well. Right now laptop is sitting closed under the Mac.

• LiteNet logging now gets funneled to our own main logging pipeline.

Continued Multiplayer Work

• A fair bit of work has been done to make intentional disconnections from the client or server tell the other side more quickly.
  • This should help to avoid some disconnect-and-reconnect issues that are otherwise inevitable, and it just makes things feel more responsive.

• Additionally, when you use the in-game controls to exit the program as a while, it now takes one extra half second to lets Steamworks and GOG actually do their exit logic properly, rather than rushing out the door.
  • This should hopefully help with any cases of the game still being thought to be running on Steam in particular after it closed.

• When you exit the game, while it is doing its brief exiting stuff it now throws up a center-screen "exiting.." message.

• Got rid of OnServer_WaitingOnConnectionIndices, as this was basically a duplicate of something we are doing better and more cleanly a new way, now.

• UpdateDebugText() and all of its related variables have been removed.
  • This was the sort of thing that, in all of our prior games before Raptor and moving to 3D, you could hit F3 and get output information from. But it has never really been used in AI War 2 properly.

• Got rid of WorldTransmissionMessageType, as it was needlessly complicated.
  • This was essentially us breaking the world transmission data into multiple parts and then sending those in parts. Historically there were two reasons for this:
    • Back in AI War 1, we were using the Lidgren network library, and even though it was trying to work around the MTU (typicaly 1200 bytes) of various networks players might be on (or passing data through indirectly), it was often having bottlenecks and slowdowns with how it sent fragmented messages through the network card. Our theory at the time was that it was flooding the buffers of the NIC, rather than giving it data at a more reasonable pace. Network cards have grown enormously in the last 11 years, as have the OSes that control them, so that's not really a concern now. All of the newer network libraries that we would be using to send data should be able to handle 1MB of data without incident.
    • Secondarily, for AI War 1 and for the A Valley Without Wind games, we were sending enough world data that you could see a noticeable wait time, and by handling this above the network transmission layer we could give you visual feedback on transmission progress (even though the whole act of that actually slowed it down). Eleven years later, we have networks that are vastly faster basically everywhere in the world, and the world savegames for AI War 2 are much more efficiently packed than AI War 1 was. So sending a file like this really is like a midsize image on any website you visit, now. Not really a progress bar sort of situation, especially if the progress bar itself causes slowdown.

• ConnectionStage has moved from being just a UI-only element to being ClientConnectionStage as part of the ArcenNetworkAuthority, so that clients can better keep track of their connection status. Usually this is less than a second or so, anyhow, but we still need to keep track of it in case something goes wrong in particular.

• The host now properly sends the world data to the client, for the first time in three years or so.
  • The client is having some trouble understanding it thus far, so now is a good time to build in error handling (it was just silently failing, until now).

• Added a new SendLowLevelChatMessage() method on the network authority, which is used mainly for sending back and forth arbitrary messages from clients and the host to explain about the status of things. These can be echoed to all other clients or not, as the case warrants.

• If the client fails to load a world sent by the host, then the host and any other clients all get sent a low-level chat message about that, the client gets kicked out (since they never made it fully in), and the client gets a nice error message on their screen and in the log.
  • This way we can solve whatever the problem is, and nobody is left sitting wondering why things are slow (when in fact they errored).

• When you are loading a quick start, or loading a savegame as a template into the lobby, it now clears the chat log. This was definitely not information that needed to persist from a prior game to the new lobby!
  • These sorts of chat log messages, which are kept over the long term for the game (up to a limit), are no longer shown in the lobby at all.
  • Instead, we now are using LocalMomentaryDisplayLog, which is something that doesn't get sent with savegames, and which normally is just "stuff showing on your sidebar during the game for a little while."
    • This is perfect for the lobby, and in the lobby we are just showing the last 100 of these, and not clearing them until you reach the game itself.
    • These things are actually shown by timestamp, not by gamesecond (which isn't ticking yet), so that's just extra perfect for a lot of reasons.
    • This also leads to privacy for people who were talking before someone new joined -- you only see messages that were sent after you were connecting -- although that's not hugely important in most cases.

• Got rid of ArcenUI.Instance.CurrentNowUTC, since that was putting all of our lobby chat messages in the British time zone. We didn't use this anywhere else, and it was puzzling that we used it at all.

• NatPunchEnabled is now true on the server and client for LiteNetLib. We'll see how well that works.

• Added a new debug setting to the network section of the settings menu: Write Network World Serialization Logs
  • When sending a game world across the network to a client, write NetworkWorldSerialization.txt, and when getting one from a host write NetworkWorldDeserialization.txt -- both in the PlayerData folder (but on two different computers). The only real reason to turn this on is if a client is unable to join a host and gets an exception; this lets us do a diff of the two files and figure out what is different about what the host is trying to send and the client is trying to read.
  • Note these logs are often huge, sometimes hudreads of MB each. Compressing them into a zip or tarball and then uploading them somewhere we can download them to find out what is happening is a good idea. Hopefully as a player you will never need this, but this is basically a way for us to test the network world sync process, which otherwise can be incredibly opaque and take a lot of time to hunt down.

• Used our new network world logging capabilities to fix an error in network world transfer where it needed to send the game version in a different format.
  • The client now looks at the game version very early on, and if it does not match it sends a message back to the host telling the host what the problem was.
  • Basically: If the client can't join because of a version mismatch, that gets shown and things happen properly.

Milestone: World Sync Between Host And Client

• Milestone! Initial transmission of the world data from the client to the host now works properly.

• When a local player account is missing the following methods no longer throw errors:
  • SwitchViewToPlanet
  • OnClient_SendClientBatchToServer (actually still errors, because it's useful to have it do so).

• PlayerAccount is now more strongly linked to the new ArcenNetworkClientConnection, and its connected status is now a method called OnServer_GetIsConnected().
  • There is some duplication of concepts here, but when you factor in the way that things like viewing of planets are synced in multiplayer, this is a worthwhile separation of concepts.

• If two players with the same profile name try to connect to a game at the same time, it will now prevent the second one from joining and tell everyone why.
  • Bear in mind that these are AI War 2 profile names, not Steam or GOG or what have you. Everyone needs to have a unique AI War 2 profile name in the current game, but they can change up their names however they want and global uniqueness doesn't matter beyond the current game you are in.

• In situations where a player 3 joins after player 2, and it's the first time player 3 has been in this particular world, player 2 now gets told by the host about player 3.
  • And of course all the more complicated permutations of this. It's the new FromServerToClient_SendNewPlayerProfile.
  • This is needed specifically so that all clients can know which planets are being viewed by other players, so that which planets are "tier 1" processing is honored and identical everywhere. Among other later needs.

• Clients connecting to a host are now properly told what playeraccount they have been put in charge of.
  • This makes PlayerAccount.Local not null on their local machine, and lets them give orders and whatnot from their spot there.

• Errors that happen during the main sim step will now be more visible in multiplayer contexts.

Milestone: Initial Sharing Of Lobby Between Host And Client

• Milestone: Clients are now able to load into the lobby, and they and the host both see most (but not all) changes that are made.
  • Clients are not yet assigned to any faction (making them just spectator-mode for now), and there are some specific bugs with certain data that isn't synced correctly just yet.

• Map generation can no longer happen in the lobby except on the host.
  • We have no real way to be sure that map generation is deterministic between machines, so we need to do a world sync each time anyway.

• When the map is regenerated on the host during a multiplayer lobby, it recreates all of the playeraccounts needed for each of the player connections that exist.

• And when the map is regenerated on the host during a multiplayer lobby, it now re-syncs the entire world data to all of the clients so everyone is up to date.
  • This also solves the "authorized through frame number" issue, which was really just a smaller symptom of this larger issue.

• Both the client and host are now verified able to add, remove, and edit factions and have that properly affect the world.

• Window_ServerMultiplayerConnectionStatus has been removed, as we no longer use that. We've been using the new center-screen messages instead, or the chat sidebar, as needed.

• In multiplayer games, the save button is now disabled and says: Cannot Save Game As Client
  • If you click it, it gives you this message: Unfortunately, only the host can save games in multiplayer. To save on bandwidth as well as on processing power between your machines, there is a LOT that is calculated only on the host machine. If you were to save a copy of the game on your client's end, loading it would lead to a lot of wrong and missing data.

• Additionally, autosave is now something that does not happen on multiplayer clients. Just the host.

• The header of the load quickstart and load saved game menus now make it clear if you are loading for multiplayer or single-player.
  • Also on these screens is a new "[Multiplayer Questions?]" bit of text up at the top that appears only on the multiplayer version, and which has some helpful mouseover text.

• Rather than having an annoying explanatory popup to click through every time you open the various ways of hosting or joining a multiplayer game, it now puts that extra information under the "[Multiplayer Questions?]" section up at the top of those screens.

• At the top of the lobby screen, it now says if you are joining as a client or a host to a multiplayer lobby, as well as having that same "[Multiplayer Questions?]" bit.
  • This includes the following new text in its mouseover text: Except for the host, all players are spectators by default. There is only one human player faction by default. Any number of players can share control of a single faction, or each player can have their own faction that they control on their own. You can mix and match as you like.

• The connect by IP window has been completely reworked visually so that it matches the rest of the game and doesn't look like the old temporary style that we were using in 2017 (because that's what it was).
  • This also lets us put extra information on that screen to help anyone who might be confused.

• The client connections status window is now patterned as a continuation of the connect by IP window, and it's nice and graceful and in the new style.

Version 2.106 Immortals and Unresponsiveness

(Released July 23rd, 2020)

Bugfixes

• Put in field names for the risk analyzers saving, so if they have trouble serializing we can see which field it was.
  • Also then made it so that the total increase and net increase both are automatically clamped to 0 instead of a negative number so that they won't throw an exception when trying to save.
  • Thanks to GreatYng for reporting.

• When stacks are split, they now have their "time on planet" and "time alive" set to be whatever the original stack was. Same with the amount of cloaking points they have lost, and the number of cloaking points they have left.
  • This prevents a variety of abilities from triggering, including the "invincible if only on this planet for x seconds."
  • We also made it so that dying to remains does not reset the "time have been alive or on this planet" counter, for similar reasons.
  • This keeps the vanguard hydra heads from being insanely impossible to kill, among other issues that are probably pretty much all in the player's favor to have resolved.
  • Thanks to crawlers for reporting.

• Added some extra debugging information for the status of gamecommands that are queued, to see if there are clogs happening. At this point, we can't replicate any evidence of that, though we have some reports from multiple players of it. So maybe the problem either takes a bit to manifest, or is in a different area of the code than we expected.

• AIP should no longer increase when things die to remains if you have the "ships of X type die to remains instead of dying entirely" galaxy map option on.
  • This is untested, but should work.
  • Thanks to crawlers for suggesting.

• Found and fixed the core typo from a few builds back where remains that were being rebuilt were being set to have the cost of the rebuilder, not themselves. This was previously massively overcharging players for building most things, but then once we put on a band-aid to fix that, it started giving it to them for basically free.
  • Thanks to Lord Of Nothing for the report and save.

• Fixed a bug from the last few versions that nobody seemed to notice, where the galaxy map was not centering on planets properly when you first went into the game after loading a savegame. This had to do with when we were calculating the position of the planet, and that not always being calculated before we tried to center on it.

• Fixed a bug where it was possible that a client would send more commands to the host than it would tell the host about, and thus the host would not know to read the extra ones (or vice-versa). In single-player you are both the host and client, so it still applies.
  • This was happening because of multiple threads adding to the queues of commands at the same time that the queue was being drawn down.
  • This was actually something that probably could have happened since more or less forever, but it was substantially more likely in the last few versions because of some efficiency improvements we made.
  • Additionally, we have now switched to using a ConcurrentQueue for each of those collections, rather than a regular Queue. This should also help with potential contention.
  • There was also the possibility that we might not be able to get enough commands out of the list for some reason to then send more, and we are also now guarding against that.
  • Previously, this problem was manifesting as orders not being carried out when you gave them, sometimes or all the time. And things like "save my game" and "pause or unpause" not working anymore.
  • We were able to duplicate this in one savegame after some time, but no longer can. Given the intermittent nature of this sort of thing, we can't say for sure that it is solved -- but the logic of what was going on makes sense, and the fix we put in should hold that at bay. It's one of those "can't prove a negative" things, at the moment, so fingers crossed it just never shows up again.
  • The characteristics that we observed while this happens, if it happens to you and you want to see what you can tell us:
    • In the escape menu, under the memory pooling performance, the number of raw commands were steadily climbing. One or two every few seconds. This should never happen in that way. Increases happen, but not a steady drain flow like that.
    • In the escape menu, further down under the Commands Queued By Type, you may see some null warnings up at the top of that list, now. That wasn't there when we observed it, but it would be if it ever happened now.
  • Thanks to zeusalmighty, Puppet Master, Gdrk, and GreatYng for reporting and giving advice on how to reproduce it.

Multiplayer Work

• In our copy of FORGE networking, we've removed some useless extra code that was for the UniversalWindowsPlatform framework, which we won't be ever using.

• In our copy of FORGE networking, we also got rid of a very useless set of "pending" messages on the UDPClient and UDPServer, which were ostensibly to help with reliable messaging, but in reality was just doing some useless event handling and invocation, and some useless cross-threading locks. It just put the things in the basket and took them back out, never checking the basket or doing anything with it. Thankfully, there is a second basket for reliable UDP inside the UDPPacketComposer, so this was just extra redundancy.

• Added a new ArcenNetworkClientConnection abstract class, which we now store on ArcenNetworkAuthority in a ClientConnections list.
  • These are connections, not yet correlated to specific players. But it lets the network pipe connect up a specific connection with a specific player (or reject that connection if it's not okay).
  • Each individual networking framework (ArcenSocket descendent class) is actually responsible for keeping track of these connections and populating them.
  • But the actual network authority does the job of authenticating them into a given game and handling the challenge/response work, figuring out a match to a player slot in the game, etc.
  • With this in place, we can now get rid of GetNumberOfConnections(), because that's now moved out of the socket and into the network authority.
  • This is a substantial refactor, in that it will really make Steamworks and GOG a lot easier for us to work with as networking frameworks, faster.

• NetworkingPlayer in FORGE now inherits from ArcenNetworkClientConnection, and is the first version of a networking framework using this new pattern.
  • On NetworkingPlayer, rather than having a public { get; private set; } for some of the immutable connection things, we now use them as public readonly. This is both more efficient in performance (slightly), but also prevents sync issues between this class and its new underlying abstract class.
  • This class gives the information it needs to the base class, and doesn't bother registering the host NetworkingPlayer as an underlying client connection at all, keeping things simple.

• A bunch of stuff with the FORGE networking logging has been replaced with wrappers around our own logs, so that if there are internal exceptions those don't just go nowhere invisibly anymore.

• Trimmed out a bunch more old FORGE code that has been commented out for years from the look of things.

• RespondToNewConnectionAcceptedByTransportLayer() is definitively game code, not part of the actual networking transport layer, and it has been moved into the central codebase and away from any specific networking framework.
  • "Hello friend, you've connected: what's your profile name you want to play under?" Etc.

• A fair bit of the "initial chatter" of clients connecting to a host and the host responding are now logged to the debug log no matter what.
  • There really is not that much that is said, and this will certainly help someone at some point who is having some sort of connection issue.

• When clients are connected to the host, now the host not only shows the number of clients, but also sees their profile names that they have chosen to connect with.

• A bunch of event-driven stuff on the FORGE networking has been made a bit more inline, improving performance a bit and also reducing certain areas of complexity.

• The initial challenge-response work, and "what's your name" work, is all functional again. This is stuff that will be identical regardless of the network framework.
  • The host can now properly see the profile name of the client, and is poised to send the client back all the world information.

Version 2.105 Selection Hotfix

(Released July 22nd, 2020)

• The game still loads various xml files asynchronously, and loads icons and music and sound in that fashion, but it no longer tries to load the ships, shots, or wormholes in that way.
  • This was just too unreliable, because of some issues in the unity engine. It didn't affect all players, and not all the time -- but enough that it majorly affected the stability of the game for some folks, and that's too much.
  • This also removes the potential for the really long (15-30 second) lag time when you are first opening a new game or the lobby if things didn't load during the loading period.
  • Thanks to Ovalcircle for the most recent set of reports.

• Previously, the game was using a standard List<> object for keeping track of selected ships, and that sometimes ran into inefficiencies and cross-threading issues.
  • In the prior build, we tried to fix some of those cross-threading issues, but wound up making things not actually deselect properly.
  • To fix all of the above, we've now switched to the much more flexible and robust ArcenLessLinkedList<> object, and adjust the code to handle this. This is both more efficient (not that efficiency is a major problem here), and does not fall victim to the various cross-threading woes that the old style could.
  • And, naturally, as part of that, this also resolves the frankly infuriating selection issues in the most recent build of the game. Sorry about that one!
  • Thanks to ctl0ve, CRCGamer, Chuito12, and Burner for reporting.

• Previously, the tech menu only showed techs that would actually benefit a ship you have at present, or which you could capture.
  • This was very confusing to a lot of players, as they would not see the entire tech tree and thus think that something was missing or broken.
  • In some other cases, possibly due to changes in the recent past, you CAN build something (like a command station), but since you have not done so yet, it would not show up.
  • All of the techs now always show up, but the ones that are for ships that would not benefit you right now still show up as red and not researchable.
  • Additionally, the tech color shows up as white when it benefits ships you have, gray when it benefits ships that you don't have right now, and lighter gray when it benefits ships that you could capture.
  • Please note that since citadel upgrades don't benefit battlestations anymore, the citadel tech not showing from the start was working properly.
  • Thanks to Strategic Sage and CRCGamer for reporting the confusion.

Version 2.104 Negative Build Percentage Hotfix

(Released July 21st, 2020)

• Somehow or other, it became possible for some self-building ships to go very negative in their build percentage (instead of counting up to 100%, they would be way in the negative percentages of building process).
  • This seems to be a new bug in the last day or so, based likely around some changes that we made to improve the cross-thread reliability of the game. The problem is, upon manual code review of the areas that would be relevant, we just can't see anything that looks like a bug.
  • Ultimately what is happening is that the SelfBuildMetalRemaining is getting absolutely giant in value, far larger than the original metal cost of the ship that is constructing. This was probably an integer overflow from it going really negative first, and then wrapping back around, but it's hard to be sure.
  • We've put in some extra defensive code to make sure that if it goes negative at all, it marks itself as complete. We've also put in protections so that if you "owe more than it is worth" in general, that it will assume that there was an overflow and just go ahead and finish it now. These two changes should help to keep the problem from happening again, and fix the cases where it was already in progress.
  • While we were already at it, we put in a small general improvement that prevents you from being overcharged for the last tiny percentage of constructing a ship. If you had a bunch of engineers building, and each frame cycle they normally would do 1.1% of the construction of a ship, and charge you accordingly, then you might wind up in a case where you had only 0.2% remaining work to do, and yet got charged the full 1.1% rate. It is now careful to only charge you for what you would owe, so the 0.2% rate in this example.
  • Hopefully there are no other errors in any of the metal flows, with things like drones building, anything engineers would claim or repair, etc. The code looks clean, but then again the code for the self construction also looks clean. In the next couple of days, please let us know if you see anything else that seems amiss with how metal is being charged against you. So far we can't duplicate anything like that anymore, now.
  • Thanks to CRCGamer, deso, and Puppet Master for reporting.

• Some code from yesterday dealing with cross-threading protections let us unfortunately sometimes wind up with runaway DoForSelected loops. This is now fixed, and it has several kinds of self-repair in place to prevent this from happening now.
  • Thanks to Gunner for reporting.

Multiplayer Connection Work

• Reworked the NetworkTrafficLog.txt multiplayer output to actually be a lot more informative.

• GetNumberOfConnectsCurrentlyActive has been replaced with GetNumberOfConnections, which lets the network framework tell us a lot more about the general status of things happening.
  • Instead of just a generic number of players that exist, we can see how many are disconnected, how many if any are the host, how many are clients, etc, etc.

• In the lobby, the host now sees how many connected clients there are, and if there are any disconnected ones, how many disconnected ones there are.
  • Previously it just showed one number that included the host and also disconnected clients all wrapped up into one.
  • It still isn't detecting disconnects properly in FORGE, but we'll get to that.

• The host now only bothers sending heartbeat messages if it has at least one non-host client machine connected.

• Removed some old "desync detected" code that was not fitting with the new plans for multiplayer.

• Figured out some of why the client was not detecting itself as properly connected, and so was not getting messages from the host at all in the last few builds after the connection.

• The way that gamecommands are queued and then given out to player simulations for both single-player and multiplayer is now more efficient.
  • We're using queues now instead of lists, and this gives us a lower chance of accidentally losing a command, as well (not that we know of that ever happening before).
  • This has several performance benefits even for single-player when there are a lot of commands going around.

• Added a new FromServerToClient_AuthorizeThroughFrameNoCommands in addition to the FromServerToClient_SendNextBatchOfCommandsToExecute, to slightly save on space (1 byte per message) and make the logs clearer.

• The network log no longer has a time delay built in where it logs only 100 items at a time. It just immediately dumps items to disk.
  • Realistically there is only substantial traffic every 100ms or so anyhow, and the delay in logging was incredibly confusing as a programmer working with it.

• The network polling interval has been updated from 20 times per second to 100 times per second. In practice it will actually be variable, but the polling is extremely lightweight (the part that is on the main thread), and having absolutely minimal delays in getting and sending messages is what will keep things moving smoothly.

• To make network logging even more efficient and clear, the old ArcenNetworkLogEntry is now ArcenNetworkSendLogEntry, and is now a struct instead of a class. This runs lightning fast and doesn't affect RAM usage.
  • We also now are then logging data that is gotten in (which we call TAKE) along with data that is sent (which we call SEND).
  • This way we can see the conversation between several machines all interleaved on a single machine's log, and thus figure out why they are saying what they do, and if messages are getting there, etc.
  • Surprisingly, the way we were logging things previously really didn't lend itself to that, so it was easy to go down rabbit holes of wrong information (thinking data did not get to a machine when in fact it did).

• Furthering clarity on the networking side, we're now ignoring loopback messages-to-self on the host in our logs, which previously we were not doing.
  • This way we don't get drowned in a sea of stuff that "I'm telling myself in order to be the same as everyone else," and we can just see what is actually passing between machines.
  • The loopback stuff has been demonstrably working fine since more or less forever, unless we broke it today, since single-player relies on that.

• Fun fact, after all these logging changes, we can now actually see how well parts of the networking are already doing.
  • As one client connects, the host is able keep talking to itself and/or others without breaking stride, letting the potential connection wait for the challenge and acceptance (not yet reimplemented) before it loops in the new client to the rest.
    • This is a great example of how we can keep people from accidentally messing with the sessions of friends by trying to join during a game, etc. It's the sort of thing that only matters among friends, but it keeps it clean.
  • On top of that, we can see now that the reason for the host not noticing when a client disconnects is because it's ignoring the client that has not fully gone through the challenge-response process yet, and so the mystery of how it was "sending messages that failed but didn't flag the client as disconnected" is easily solved (answer: it wasn't sending messages to that particular client yet, because it is smart).
  • In general, by improving our logging quite a lot we can see what is actually happening and make sure all of this works properly. The vast majority of this is network-agnostic, which is doubly great. These particular logs will look substantially the same between FORGE and Steam and GOG.

Version 2.103 Hotfix

(Released July 21st, 2020)

• Fixed a bug from the prior version where the dark spire serialization fix was making any savegames with dark spire in them -- new or old -- not able to load.
  • Thanks to valinor000 and stanazolol69 for reporting.

Version 2.102 Digression For Quality

(Released July 20th, 2020)

• Using the "Increase Max Dyson Strength" hack will now also increase the Dyson's general mark level
• Some minor buffs to the Dyson's income, mostly at higher levels
  • Thanks to a discussion started by zeusalmighty

• When picking music (through the debug menu), the hovertext will now give you information on most of the tracks saying the original source

• Improved the speed of some dictionary lookups by precalculating the ArcenAssetBundlePath CombinedPath.
  • This also required us to change their member variables into properties, and in return that meant that the general string fill methods needed to be adjusted to new FillBundle and FillPath methods for xml reading.
  • This doesn't affect as much code as it sounds like.
  • We also adjusted the various debug logging points to use the combinedpath, which is cleaner and easier to read, but not much faster since that almost never happens.

• Fixed a bug that could happen with entity order serialization in rare circumstances if threads were fighting over putting items back in or taking them back out.
  • Since multiple threads do in fact often use orders, at once, this is one of the few collections prone to that.

• Fixed a variety of exceptions that could happen based on cross-threading bad luck during the exit of the game either to the OS or to the main menu, all in the metal expenditures code.

Ship Behavior Improvements

• Fix two problems related to shields recovering from being bumped by Astro Trains
  • Human Home Forcefield Generators can now recover from being bumped.
    • Thanks to Lord Of Nothing for the bug report
  • If a forcefield generator is bumped twice in short order, it will now do a better job of returning to its original location. This isn't perfect, but it's a pretty intense corner case
    • Thanks to Puppet Master for the bug report

• If there are only non-combatant enemies on a planet, ignore any requirement in the targeting code that particular targets need to be combatants. This prevents ships in FRD from discarding reasonable targets incorrectly.
  • Thanks to ParadoxSong for the bug report

• Actually support Targeting tracing now; you need to set Targeting tracing and the Debug setting that lets you set the PrimaryKey for the unit you want to follow, and you'll get some actual logging.
  • Not for the faint of heart; intended only for developers who want to get their hands very dirty.

Bugfixes

• Fix a bug where the 'delete campaign' button wasn't giving the campaign name
  • Thanks to OvalCircle for reporting

• Fixed a rare and harmless (but annoying) cross threading exception that could pop up with argument out of range exceptions in RenderShip.
  • Thanks to CRCGamer for reporting.

• Fixed a rare nullref exception that could happen in DoRemovalChecks() on shots, mainly a cross-threading thing.
  • Thanks to Lord Of Nothing for reporting.

• Fixed two possible spots in DoForSelected() where cross-threading issues could sneak in and cause an exception in rare cases.
  • Thanks to Lord Of Nothing for reporting.

• Fixed an issue when loading savegames that have a missing planet naming scheme. It will now default to the default naming scheme if it can't find the one that the world was started with.
  • Thanks to Oryutzen for a save that demonstrated this.

• Fix a bug where the imperial spire wasn't Watching previously explored planets for you.
  • Thanks to Lord of Nothing for reporting

• Fixed an error in TextMeshPro where it would still give us "Unable to use Ellipsis character since it wasn't found in the current Font Asset" invisible errors that would fill up the debug log even if warnings were disabled.
  • This was, in very long play, a memory leak as well as a performance drain, and it was generally invisible.
  • The ellipsis character was typically something it was trying to find specifically for purposes of showing cut-off text, and some fonts just don't have that in them. So no wonder we couldn't actually find the ellipsis character used in our own text files.
  • Thanks to Puppet Master for providing the log file demonstrating all this.

• Fixed an issue where the stationsRemainingBeforeDepot on astro trains could go arbitrarily negative, thus causing an error on save. It now stops at 0, and thus seems not to cause the issue anymore. But on the change that an astro train does have a data exception during save anymore, it will now log what field is having the problem and we can fix it.
  • Thanks to Lord Of Nothing for the report and save.

• The game has been updated so that the debug logging that is DoNotShow now ONLY goes to the ArcenDebugLog.txt, and not to the unity Player.log. The unity Player.log is kept in memory in a very unfortunate fashion, and in long gameplay this will be extra RAM used that should not be. Probably also extra slowness.
  • We added a new DoNotShowButSendToUnityLogEvenOutsideEditor Verbosity option that lets us have the old behavior of going to both of those logs, but we are not using that anywhere at the present time.
  • All of this is going to make it even more important that we have both kinds of logs from players if they have an outright game crash, now. But generally speaking, if it's an exception popup but not an outright crash, the ArcenDebugLog.txt has always been enough and will continue to be. It's only when the game crashes all the way to the desktop that we need the Player.log, typically, and now we'll just still need the ArcenDebugLog.txt in addition to that. However, any ShowAsInfo or ShowAsError logs to the main ArcenDebugLog will still appear in the Player.log, so potentially even that won't be true.
  • For a developer using the unity editor, it still logs everything to the actual unity log because that's for short testing cases and super useful to see output in realtime.
  • At any rate, we are trying to strike a balance between getting the logs we need, easily on the part of players, but without having any accidental extra usage of memory on very long play sessions.

• All of the dark spire per-unit data is now saved in a format where we can tell which field is having trouble if one does.
  • The dark spire data "lastTimeAttemptedLocus" was trying to save as an int16 and would overflow any time a game saved that was more than about 32000 seconds in. Fixed.
  • Thanks to RockyBst for reporting.

Memory Leak Fix From 2.099

• New setting on the debug menu: Show Pool Counts In Escape Menu
  • Used for searching for memory leaks, particularly between loads of a savegame. If pool counts keep rising after each load, then something isn't getting put back in the pool properly and is instead persisting lost in memory.

• ArcenGameObjectResourcePools must all now have a unique name, and they register themselves in a central place that identifies how many of them there are.
  • These pools also now keep track of how many objects they have ever produced from themselves, so that if we are leaking objects from them we can tell. And we can tell if we are leaking pools themselves.
  • After seeing all of these in the new debug menu option above, we can clearly see that none of these are leaking and they are all behaving just wonderfully. Hmm. But we have a leak somewhere, so let's expand this...

• Created a new CountedPoolBase abstract class that sits under the ArcenGameObjectResourcePool, and takes its counting capabilities and puts them there so that we can also use them for some other pool types.
  • This is also now used on a new "Shot Instance Renderer Pool Of Pools" so that we can tell if we are leaking there. And now ships and squads, too.
  • Also this is now used on LoosePool, which is the underlying basis for ship, shot, and squad visualizers (different from instanced renderers) now.
  • Despite all this, no memory leak was found in any of these pools. They are all working wonderfully efficiently.

• ExternalizedPool now inherits from CountedPoolBase.
  • Same for TimeBasedPoolBase (and thus TimeBasedPool itself).
  • And same for BasicPool.
  • And holy cow, there's some sort of memory leak with the time-based pools, looks like, based on this. That makes sense, as we recently reworked some of their functionality to fix a bug with them. Apparently we introduced a new bug at the same time.

• Added a new setting to the debug menu: Show Details Of Time-Based Pools In Escape Menu
  • Used for searching for memory leaks or other bad behavior on a certain very high-turnover series of pooled objects. If pool counts keep rising, or other numbers seem off, then we know we have a memory leak or a performance problem.
  • Thanks to Puppet Master, RockyBst, Lord Of Nothing, and NRSirLimbo for reports that led to us finding this.

• Fixed a MAJOR bug from version 2.099 that was leading to a memory leak in general, and performance problems as well (the longer you played the slower it would get).
  • Our entire "time based pool" logic was not actually properly processing because we forgot a single line of code, go figure.

Slow Load Or CTD Fix

• Since we started using the asynchronous loading of asset bundle items using unity (the last few weeks), there have been some intermittent problems with certain random items not loading in properly. This is some sort of funky bug in Unity, we're pretty positive.
  • We solved that problem a week or so ago by checking the async request's asset property, and if it was not invalid, forcing it to finish loading even though it was stuck.
  • On most machines, this was causing a really annoying lag of 10-25 seconds while Unity did whatever it was doing to finish that load. That was really annoying, but only happened the first time you loaded a savegame or quickstart or custom lobby after starting the program, and not every run of the program, and not on all machines.
  • More recently, we discovered that the very act of checking the async request's asset property would cause an unrecoverable error deep in unity and an immediate crash to the desktop.
  • Killing two birds with one stone, we're no longer checking the async request asset property at all -- thus avoiding the crash -- and instead just do a normal synchronous load when we detect a failed load. This leaves the one async request in limbo kind of forever, but that shouldn't be a problem; if it ever decides to complete, it will see that it was already completed and just do nothing.
  • In the meantime, not only should this avoid the crash, but it should also avoid that awful lag spike that happened for those for whom it didn't crash.
  • The downside is that on our dev computers we haven't been able to replicate the error at all today, so we can't verify that this 100% works. It should work, and you should see in the arcen debugging log slightly differently that says "PrototypeObject failed async load and so loaded synchronously for [some stuff] (this is not a problem, but is odd)"
    • If you're one of the people who were frequently seeing either the crash, or seeing the older "PrototypeObject fixed and loaded late for [some stuff] (this is not a problem, but is odd)" message, then we'd love to have a confirmation of you seeing the new message and all being well.
  • Thanks to Sol and RocketAssistedPuffin for reporting.

Version 2.101 Connection Status: Confirmed

(Released July 17th, 2020)

• Carriage returns and newlines are now supported by the "condensed" string format. This will prevent tutorial messages from turning those characters into question marks when they are dumped to the chat log. Note that this will only affect chat log messages logged in this version and on, not past versions.
  • Thanks to ParadoxSong for reporting.

• The left and right click functions of the metal display have been switched to be more natural.
  • Thanks to Waladil and Galian Gadris for suggesting.

• The "Spire Railgun Shop" mod that comes packaged with the game has been updated to use the icons from their new locations, and will no longer error.

• We've reworked our "Window_PopupScrollingColumnButtonList" a bit to add some new handy features for us.
  • In the factions selection window, when you are adding new factions, it now still lets you see factions that you can't add because there is only one per galaxy, but now it shows them in red, doesn't allow you to select them, and explains why in the tooltip.
  • In the debug menu in the escape menu in the main game, the change music submenu now shows the current selection in red (with the note that you can't change to it because it is already playing), and it shows all of the options sorted alphabetically except for the current track, which it shows as the first item.
    • Also fixed a bug where the music you selected to play would not actually be the track that played; it was choosing a random other track.

Scourge Updates

• The hovertext for the Scourge "Strength" in the game lobby now tells you what the strength does (more scourge ships/structures, and faster).

• Add a "Sandbox: Extra Strong Mode" for the scourge
  • This is not balanced for regular play, but is intended as a sandbox thing.
  • Thanks to Avenger1649 for suggesting

• Fix a bug where the scourge were periodically suiciding Defensive Fireteams into player positions.
  • Thanks to ParadoxSong for the save.

• On intensities >= 5, the scourge can create cloaked "blockade running builders" if the player has tried to blockade the scourge into a small region of the galaxy
  • Thanks to ParadoxSong for the suggestion

• On intensities >= 6, the scourge is intended to sneak a builder off to another part of the galaxy at the beginning of the game, to make it much harder for the player to blockade them in. This was not working on some map types, including the X type.
  • Thanks to ParadoxSong for the save that exposed this problem.

• These changes should make the scourge much harder to deal with in general, and on the X map type in particular.

Initial Multiplayer Connection

• AllowOtherPlayersToConnect was a setting on the World object in the past, and thus something that was getting saved into savegames and that dictated how a lot of things worked for player connections.
  • We've removed this concept, and instead are making it based on the menu choices you make on the main menu.
  • We now have a DesiredMultiplayerStatus on ArcenNetworkAuthority, which lets us recall if you wanted to be the client, a host, or just left alone in single player.

• The basic shells of SteamSocket and GOGSocket have been added to the game. The in no way function yet, but this will allow for them to be built out in the future.

• The SpecialNetworkType has been moved into ArcenUniversal, and is now copied onto ArcenSocket along with the InternalName of the row that created the socket.
  • There is now a list of AllPossibleSockets on the ArcenNetworkAuthority to allow it to swap between sockets as needed, and particularly by type.

• When you open the multiplayer menu, if you have that enabled, it will now set a default network framework for you based on the status of your game at that time.
  • If you are logged into Steamworks, it will set steam as the default network.
  • If that didn't work or wasn't true, and you are logged into GOG Galaxy, it will use that as the default network.
  • If none of those worked, then it will pick the first "other" network type, which in this case would be Forged.

• All of the options in the multiplayer menu for hosting a savegame, quickstart, or custom game now properly do those things identically to as if it was the single-player menu, but put you into host mode first.
  • The single-player menu puts you into single-player mode, and makes sure that your ArcenSocket is set to NullSocket.

• When you try to go into one of the client or host multiplayer areas from the multiplayer menu, it now properly activates your most recent chosen network framework, or switches to the first available one that actually can function right now on your system and gives you explanations for why it did that.
  • Essentially there is a fair bit of under the hood plumbing now for the frameworks to register themselves with the network authority class and explain about their availability.

• If you are looking to connect to another machine, then the "connect as client" tooltip now gives you information about what to expect based on your currently-selected framework.
  • Are we connecting by IP? How much of a hassle will that be? Is it going to be by selecting a friend on Steam or GOG? Etc.

• For IP-address-based networking frameworks, either starting a host mode or client mode event now pops up with a message explaining what information you need to give to the client players or get from the host.
  • This is not really the preferred networking method unless you're having a LAN party, and the game makes that clear, but it also goes out of its way to address the most likely areas of potential confusion for people who are using these frameworks.

• Fixed a few bugs, and added a bit more debug logging clarity, so that the host can once again properly open a socket and wait for connection.
  • This was working up until a week or so ago, and was simply messed up a bit when we reworked things to support multiple networking frameworks. There were lots of lonely hosts never getting any client requests ever since the game has been in public beta.

• When you quit out of the multiplayer lobby back to the main menu, it now immediately disconnects you as a host or client, to avoid confusion of lingering connections.

• When you are in the lobby in multiplayer, the chat sidebar/log now appears again. At the top it says if you are hosting multiplayer or a multiplayer client.

Choosing A Networking Framework

• The main menu has been updated so that the "network" button at the top of the multiplayer menu now updates its text to show the current network.

• The network button at the top of the multiplayer menu on the main menu is now fully functional. It shows available network frameworks, and gives you explanations for each one, as well as showing unavailable ones in red with an explanation for why they are not available right now. It lets you select from available ones, which then get set as your current socket when you try to host or join a multiplayer game.

What Is My IP Address If I Need It?

• In the networking section of the settings menu, it now has helpful informational displays of your local IP addresses and your public IP address.
  • To get your public IP address, we have to make a public call to the url http://checkip.dyndns.org, so this may flag your software firewall to ask permission when you go to the networking tab now.
  • Our goal is of course for you to not have to use your direct IP addresses at all in this game, but if you're playing on a LAN then that would be one great example of when you'd want to use this.
  • At any rate, each IP address is stored in a textbox that you can copy-paste from, which is very handy when sharing with friends.
  • There are also lengthy explanations about what the public IP address is for, and under what circumstances to use it, as well as the local IP addresses, plus special notes for any IPV6 addresses that it detects.
  • Getting this data is time-consuming for your computer, so you will notice micro-lags every few seconds while you are on the networking tab, now. This is normal and should not impair your use of the functionality.

Milestone: Successfully Connecting A Client To Host

• The old interface for network connections by IP address is restored and works again. We'll have a different one for connecting via friends list on Steam and GOG, later, but this is the one that gets used for Forge and any other IP-based network frameworks.

• Connections are now able to be established from one computer to another, for the first time since 2018!
  • We added the ability to see how many clients there are connected to a host in the lobby (including the host), so you can see the connection tick up.
  • Next up is them actually exchanging enough information to get the client into the lobby and have them see shared world info and be able to chat and all that. But this is milestone one!

Version 2.099 Last Rabbit Holes

(Released July 15th, 2020)

• There is now a galaxy setting to grant players Watch vision for 1, 2 or 3 hops from all command stations.
  • Thanks to uhamster9 for inspiring this addition.

Fixes And Tweaks

• Improved the way that time-based pools are incremented in time, so that we're never having accidental cases where we miss one.
  • Also made it so that they actually work as intended when the world is being cleared (running 16 seconds' worth of time at once). This then keeps the pools smaller if you repeatedly load savegames.

• On the escape menu, under where it shows you the various numbers of objects in memory, it now shows you the number of galaxy planets and planet links activated and in existence.
  • This lets us tell when these have a memory leak, if there ever is one with them.

• Additionally, galaxy map links are now pooled for the first time ever, as the discarding of them was previously a definite memory leak. Normally this just happened between saves and loads, but it was also possible to happen when nomad planets moved.

• In addition to the existing time based pools, we now have a new ExternalizedPool.
  • This is now used for the galaxy map links, and actually we have stopped the galaxy map planets themselves from using the super-old IArcenGameObjectResourcePoolable and they also now use this.

• The way that planets for the galaxy map are initialized, and the way that their positions are set, is completely overhauled.
  • This is more efficient and properly uses pooling (which never was working properly before, it turns out).
  • It also makes the position of planets automatically react to them having moved in the game sim, without having to do anything special.

• The arcencolors asset bundle has been removed, with its contents simply rolled into arcenui. Ultimately this is a bit faster to load, and saves a bit of disk space and RAM also.
  • Also removed the aiw2squads bundle, and the examples bundle.
    • The examples bundle simply has its example content still there, but not in a bundle (it's to help aspiring models modders).
    • The squads bundle is old data that we've not used in forever, and so has just been cleared out. It wasn't huge, but wasn't worth being there.

• Starting to load a savegame (including for a quickstart or the last settings for a lobby) now writes to the log when you start the process, and at the end of the process writes how long it took.

• Added a new debug setting: "Write Detailed Savegame Timings"
  • When loading a savegame, including the 'last settings' for the lobby or the basis for a quick start, keep track of detailed timing data and write that into the normal debug log so that it's clear what is taking longer and shorter amounts of time.

• It turns out that chasing "why savegames now take 16 seconds to load sometimes" was a snipe hunt. The serialization sizes logging, when enabled, actually was causing that amount of slowness. With that off, it loads in about 2 seconds.
  • The extra instrumentation that we added for the savegame timings is still nice, but ultimately was not a useful bit of time spent at the current moment.

• Fixed an issue that could happen in the lobby in particular where it could not properly save your prior settings because the fleet ID or speed group ID was lass than zero.
  • This pretty much could only happen if you already had another error first.

• Fixed an exception in SeedNormalEntities that could happen if you rapidly tried to regenerate maps in the lobby in just the wrong way.

Under The Hood Improvements For Icon Modding And RAM Usage

• The last of the icons from the ExternalIcons folder have been moved into the unity project that generates the asset bundles, simply to dispel any potential confusion about the fact that they can be edited directly and that have some change on the game without recompiling asset bundles. This was a frequent modder confusion.
  • That said, the ones that were in the Official_1 folder are now in a CentralIconBits, and can now be used in the UI for the first time if we ever want to. That means things like health bars and whatnot could in theory be used in the ui if someone wanted to, whereas before they could only be used on gamespace-level icons.

• Cleaned out a variety of unused icons from the arcenui asset bundle, and in general tidied up some of our organization of working files versus final files.
  • Also standardized the naming of certain things.

• Broke the "official icon dictionary" out into six dictionaries:
  • One is for the "central bits" like health bars and so on, and would be the same for all ships.
  • Two are for the overlays, like for saying what kind of starship or guard post something is. There can be more than one of these, and the fact that these are split out like this demonstrate the modding capabilities and how things can combine.
  • Three are for ship icons and their borders, and there can be more of these modded in as well. As with the overlays, not only does this demonstrate how such mods would work, but it also has the side benefit of slightly less VRAM usage in a few cases.

• The way that GUI icons are discovered from the xml based on their gamespace-icon counterparts is completely revised, and is now based on extra information in the ExternalIconDictionary entries.
  • This is a lot more flexible, and allows us to load icons from multiple asset bundles and/or multiple files within the same bundle.

• The naming for finding icons is now much more complicated, in the sense that they don't all come from a single dictionary called "Official."
  • So there were over 1100 places in the base game and expansions where we've had to modify to point to the new dictionaries. Any mods would also need to be updated to point to the new places, unless they want to start using their own icons (once we put together a tutorial for that).

• The external sprite dictionaries have been updated to be able to properly load in in multiple threads.

• The game is now able to load the xml files for external icons from any mod or expansion, in:
  • Expansions/[ExpansionName]/GameData/ExternalIcons/[thefile].xml
  • XMLMods/[ModName]/ExternalIcons/[thefile].xml
  • XMLMods_NonDistributed/[ModName]/ExternalIcons/[thefile].xml
  • It's worth noting that these are xml files that are generated by Texture Packer, not xml that we create.
  • These files are referred to in the "ExternalIconDictionaries" xml as something along the lines of xml_path="Official_CentralIconBits.xml"
    • It will then search the main game's folder (/GameData/ExternalIcons/[thefile].xml), then all the expansion folders, then any activated mod folders for a file with that name that was specified.
  • This allows for mods to be self-contained when it comes to their icons, and it allows expansions to have their own icons that are not packaged with the main game (not that we have imminent plans to do that).

• On the icons used in the game, we previously were not using any compression, which made them absolutely massive in size (80mb for the main dictionary).
  • This was done in order to preserve quality, but we're also using GIANT icons in order to allow for really super-high-res displays of the future, as well as square ones to allow for ideal mipmaps, etc. So the lack of compression was hugely overkill.
  • We're now using compressed textures, which turns what was 80mb into more like 8mb. This has a very direct impact on performance and VRAM usage, so particularly on low-spec machines they will likely run far better.

• The game now automatically constructs custom "gimbal materials" for the in-gamespace icons, in as many combinations as are needed for the things that it has drawn for you so far.
  • The total number of materials always in the past was "one," but it was using a single large dictionary and a lot more VRAM -- and it had a finite amount of space for things. There was a solid chance we were going to run out of space in that one dictionary as part of implementing DLC2. But either way, mods were not possible. The new number right now is 8, looks like.
  • Now it may make a handful of materials, depending on how many mods you have installed that have custom icons, and several for the core game and expansions itself. But these are vastly smaller, and while this does increase the number of "draw calls," each material is still mostly instanced together (there are a few mesh differences depending on if there are health bars shown or whatever), but the overall load on the GPU pipeline is lower.
  • All of this is automatically handled by the game in as efficient a pattern as possible, and if you're curious how many combination materials it has created, you can see that in the escape menu at the bottom of the list of memory pooling info.

• A new debug setting, "Log Gimbal Icon Material Creation", has been added:
  • When this is on, any 'gimbal icon' material creation events will be logged as to what was created and how long it took to do so.
  • Turns out that this uses an immeasurably small amount of time for each material (less than 1ms each), so that's awesome.

Under The Hood Improvements For Mods With Code

• The game is now able to load dlls from mods or expansions, rather than just from the central game folder.
  • This works just like the external icon files, and basically looks for the dll in the central game folder first (/GameData/ModdableLogicDLLs/[thefile].dll), and then looks at expansions and then installed mods if it can't find them.
  • The paths are:
    • Expansions/[ExpansionName]/GameData/ModdableLogicDLLs/[thefile].dll
  • XMLMods/[ModName]/ModdableLogicDLLs/[thefile].dll
  • XMLMods_NonDistributed/[ModName]/ModdableLogicDLLs/[thefile].dll
  • We really don't have a need to do this with expansions, but the flexibility is nice.
  • With mods that are more than just xml, however, this finally lets a modder distribute just a single folder that has everything in it, and not have to worry about putting some things in central game folders.
  • This is only partially tested, but should work.

• It's worth pointing out that expansions and mods already did (and still do) have the capability to load asset bundles (icons, music, sound effects, models, textures, shaders, etc) from their folders.
  • The structure for those is /GlobalBundles/ in the main folder for anything not platform-specific, /AssetBundles_Linux/, /AssetBundles_OSX/, or /AssetBundles_Win/ for the things that are.
  • Inside the folders for expansions it is: Expansions/[ExpansionName]/[OneOfTheAboveFolders]/
  • Inside the folders for mods it is: XMLMods/[ModName]/[OneOfTheAboveFolders]/ or XMLMods_NonDistributed/[ModName]/[OneOfTheAboveFolders]/
  • None of this is new, but it's worth mentioning for now.
  • This is only partially tested, but should work.

Larger Gamespace Icons And Fixed Galaxy Map Line Offsets

• We also now have a non-billboarding version of our shader for purposes of our icons on the galaxy map.
  • Our core shader does on-GPU (read: hyper efficient) billboarding to always fully face the camera. This is very useful in the main view where you are moving around the camera a lot but always want the icons to face you.
  • The downside, however, is that with a largely-straight-overhead camera like the galaxy map uses, this billboarding would cause there to be a perspective shift where the icons for planets would get offset from the lines leading between planets no matter what we did. It also caused them to move relative to the text that was next to them.
    • This offsetting of the lines to the planets was one of the largest "visual papercut" issues that we've had for a really long time, and it is finally fixed!

• The ship icon scale has been adjusted from a default of 1.5 to 2.2, and will affect all prior settings files.
  • It's description has also been updated for the personal settings tooltip: For the icons in the main display area (not the sidebar), how large should they draw? Default is 2.2. If you go too large, it can be hard to see things because they overlap too much. If you go too small, they can get extremely blurry because of flipping to a lower mipmap. The larger your screen DPI, the smaller you can go without losing clarity.

Prior Release Notes

AI War 2: Building Multiplayer