AI War 2:The Paradigm Shift

Known Issues

• Any bugs or requests should go to our mantis bugtracker
  • If you need to submit log files, those can generally be found under your PlayerData folder in the folder your game is installed in. The most relevant one is called ArcenDebugLog.txt. You can send us the whole thing, or just strip out relevant parts.
  • In rare cases, mainly if your entire game crashes (that almost never happens), we will need your unity player log. That gets overwritten the next time you run the game after a crash, unlike the other log. These can be found here:
    • Windows: C:\Users\username\AppData\LocalLow\Arcen Games, LLC\AIWar2\Player.log
    • macOS: ~/Library/Logs/Arcen Games, LLC/AIWar2/Player.log
    • Linux: ~/.config/unity3d/Arcen Games, LLC/AIWar2/Player.log

• Multiplayer is in public alpha, as noted below. There is a detailed multiplayer guide that we are working on building up.

• Feel free to join discussions on discord!

What Does Multiplayer Alpha Mean?

Please see this link for details on multiplayer. This wound up taking up too much space in this document, so all of the multiplayer-relevant bits have been moved to the other page.

What's this phase all about?

We've been preparing for this for months, tightening up the codebase and getting everything ready as much as possible. Now it's time to start having machines actually talk to one another, and then refine from there. At this point, LiteNetLib, Steam, and GOG Galaxy are all fully working as communication frameworks, and the game is in an alpha state (as of September 9th) where not all of the functionality is there, but the game can be played together.

It's important to remember what an alpha is! There are glitches and inefficiencies, and we'd like to know more about those. There are also glaringly missing features. Beta means feature-complete but not bug-free, and we're not there yet. Please don't assume that we will implement some feature you want, like the ability to share X between players. If you're testing the alpha and want something, please do drop us a message and ask for it, just in case.

We expect to be into beta of multiplayer in late September. Hopefully the alpha and beta periods are both short due to all the work of the last phase, but we shall see how it shakes out.

Once we get into a stable beta period for multiplayer, then we'll let the clock run a bit and Chris will work on adding interplanetary weapons to the base game as a free update (that's the last of the kickstarter stretch goals). This phase should wrap up all our kickstarter promises (including a laundry list of other smaller items).

Badger has already done the bulk of his work for the second expansion, Zenith Onslaught. Chris has done none of his work for that, as yet. Once multiplayer is in the "let it exist in beta and have people bang on it" phase, then Chris can circle in and get his work done for DLC2 along with those base game bits for interplanetary weapons, etc. The full version of multiplayer should launch alongside of DLC2, but we're not sure exactly when. October?

Either way, multiplayer will be in a solid state that is "beta but just because we want more time with more people testing it" for a month or so while the DLC2 work from Chris's end happens. None of the actual multiplayer stuff is delayed in any fashion for DLC2, it's been the other way around.

Version 2.508

(Not yet released -- we're still working on it!)

• Fixed a couple of places in the metal flows tooltip where it was stating the left and right mouse buttons backwards. (There were three places in all, and we had only changed one of them).
  • Thanks to Daniexpert for reporting.

• Fixed a rare exception that could happen inside GetControllingOrInfluencingFaction() when exiting the game either to the OS or to the main menu.

• Slight tweak to the way the Dark Spire chooses targets

• Fix a bug where waves were not spawning any units. This affected regular waves and hacking responses.
  • Thanks to weapon master, CRCGamer, DaniExpert, ArnaudB, Gott365 and probably others for reporting

• The Zenith Trader no longer sells AIP-increasing black hole machines to the player
  • Thanks to TechSY730 for reporting

• The Zenith Trader will no longer sell things to dead AI factions
  • Thanks to GreatYng for reporting

• Really fix that Macrophage null reference in LRP bug that's been annoying badger

• Some performance improvements for fireteams in general (and some extra improvements for hunters) in late game scenarios
  • Thanks to a giant save from crawlers for letting me see what was going on

• When you see the name of a song in the Escape Menu, it should now be formatted in a more readable way. For example, it now shows "Waking To Darkness" instead of "WakingToDarkness"

Dark Spire Tweaks

• Fix a bug that was preventing the Dark Spire from being suitably aggressive in late game situations
  • Thanks to crawlers and Gott365 for the bug reports

• Other factions will now actively go after Dark Spire Conquest
  • This should give some counterplay against the Dark Spire

• To compensate for the fact that Conquest Mode VGs will be targeted more, they are also tankier

Version 2.507 Bugfixes

(Released September 16th, 2020)

• Fix a bug where player-allied nanocausts that aren't allowed to kill command stations weren't properly expanding. Add a journal entry to point out to the player that this leaves their allied nanocaust vulnerable
  • Pointed out to me on discord

• Fix a bug where the render vengeance generator hack wasn't completing
  • Thanks to a number of people for reporting, including jradishurr, Magiik and Gott365

• Fixed several cases where cross-threading issues could lead to some issues in the selected-ships counter.
  • Thanks to MasterGeese and GreatYng for reporting.

• Fixed several cross-threading bugs that could happen in GetIsWithinRangeOf() or GetIsWithinRangeOf_VeryBasicCheckOnly().
  • Thanks to GreatYng for reporting.

• Fixed a bug introduced in the prior build that made it impossible to save most games that had outguard in them (all of them...?), and probably was messing with initial multiplayer sync, too. This was not seeming to affect every last save, but nonetheless was not good.
  • Thanks to Badger for reporting.

• The "long range planning" code for the Macrophage now includes a lot more instrumentation for more detailed error messages. We started having some sort of error in at least some games starting the last release.
  • But after putting in the instrumentation (and having fixed the outguard serialization bug that also manifested in the same savegame), we no longer can duplicate the problem. So for now there's nothing more to do, but if it comes up again we'll at least know where it is.
  • Thanks to Badger for reporting.

Beta 2.506 ExternalData Overhaul Two

(Released September 15th, 2020)

This is kind of a nasty one, in that we're fixing a ton of things but potentially also breaking more things. External code-based mods are definitely broken again (sigh). But a bunch of things that were question marks for multiplayer are either now working properly for the first time (a few dozens of things), or are now verified-correct (even more dozens of things). So we're dropping back to beta briefly, to let people test things and let us know if there are any bugs that make this unplayable for some reason. Once folks have been through it sufficiently, we'll move back to the non-beta branch.

• Previously, the dark spire ships that you could hack for had a starting mark level of X, and a max mark level of 4. Now they all start at mark 1 and level up like normal, but have a max mark level of 7 (because why not).
  • Essentially, tech unlocks for players probably did not work terribly well when the starting mark level of a player unit is greater than 1. And having a low mark cap on top of that can be frustrating for folks.
  • Specter used to start at mark3, now is 1.
  • Phantom was already mark1 at the start, but still is.
  • Okay, wraith was already mark1 at the start, and still is.
  • Anyhow, the level caps were a big part of what was messing with people, probably.
  • Thanks to crawlers and CRCGamer for reporting.

• Fixed a potential nullref that could happen in CheckForInternalShipDeployment_DroneProducers_FromSimBGThread(), probably more likely on multiplayer clients than elsewhere, but definitely possible anywhere.
  • Thanks to Puffin for reporting.

• Fixed a strange capitalization issue in the logging for faction deserialization where MinFIreteam and MaxFIreteam would thus show up as diffs.
  • Thanks to Puffin for the report.

• New debugging command added: dump data tables
  • This causes all computers involved in the current game (so not just the local machine in multiplayer) to immediately dump all their data in the same way that happens from "Dump All Data Tables After Load" from the debug section of the settings menu.
  • Please be advised that this will cause all of the computers involved in this game to freeze for something like 20-60 seconds, depending on their relative speeds, how many expansions and mods are installed, and so on. It's a good idea to warn people before you run this comment.'
  • Aka, this writes a text file for each in-memory data table into a DataTableExports subfolder in the PlayerData folder.
    • The purpose of this is mainly to use with diffing tools between one run of the game and another, to see what sort of data changes happened. If you are modding and made changes to some ships and want to see how those changes cascaded, this would be one way to do that. This is also a way for us to verify correctness when we make structural changes internally.
    • In multiplayer, this can also be used to compare the contents of these folders between the host and various clients, to check for things that might be amiss thanks to mods or other local changes.
    • To compare folders at a time, you'd need to share the contents of these folders between machines (zip and email, etc), and then run a folder-wide diffing tool like WinMerge to find any discrepancies.
    • Lastly, if you're a mod author and concerned that your mod is adding to the data tables incorrectly over time, then you can use this to get snapshots of what those look like as you are playing. Normally this data should not be changing after the initial load of the application.
  • Never considered a cheat.

Multiplayer And ExternalData Continued Improvements (Breaks Code-Style Mods Temporarily)

• There are some issues with the data table dumps now actually dumping some game data.
  • To combat this, the ArcenOverLinkedList and ArcenLessLinkedList both no longer dump their firstItem and lastItem fields.
  • Hopefully we will still see the count of items in these, but that's really all we can do right now without getting into circular references by the very nature of these linked lists.

• This also finally pushed us over the edge when it came to IArcenExternalDataPatternImplementation, and this is now becoming an ArcenExternalDataPatternImplementationBase abstract class that we can control a bit better in several different ways.
  • In general this will be a help for us with multiplayer sync, among other issues.
  • This class should never be directly inherited-from by things in External or mods, however.
  • Added a new ArcenExternalDataPatternImplementationBase_World, which is for use with things being attached to worlds.
  • And also a new ArcenExternalDataPatternImplementationBase_Faction, for things being attached to factions.
  • And a new ArcenExternalDataPatternImplementationBase_Squad for things being attached to ships.

• Took this chance to fix a number of data structures that might not be multiplayer client-sync safe based on the fact that MP sync data might be missing and might need a catch-up:
  • AIReservesData (constructor updates, DeserializeExternalData updates).
  • Okay, 33 other classes needed the same update pattern, so we'll save you the list of going through all of those again.
  • Added a new ArcenExternalSubManagedData class, which all 34 of those classes now inherit from.
    • This lets the increasingly-complex central logic from DeserializeExternalData be written only once, not 34 times.
      • This is all handled, for the default case, via the new generic method DeserializeExternalDataAsArcenExternalSubManagedData().
    • Also worth noting, we can no longer use constructors with arguments on this, because of how the logic is centralized. The default constructor is always called now, and then DeseerializeIntoSelf() is called after that if needed.
      • This is doubly worth noting, since it makes sure that there are no cases where object initializations from the default constructor are not called.

• While we are at it, we're marking the various lists and other collections as readonly in ArcenExternalSubManagedData-inheriting classes.
  • This helps us avoid ambiguity and extra code, and is a double reminder for us to review places where we could get into infinitely-expanding lists on multiplayer clients.
    • Fixed the following infinite-expanding collections for multiplayer clients:
      • The FinalComposition dictionary on PlannedWaves was going to infinitely expand while waves were incoming.
    • Fixed the following "accidental data churn from throwing away collections on multiplayer clients":
      • The DZPerUnitData ConversionList and ConversionBag.
      • The metalByTier on DysonData.

• Added a new ArcenExternalSubSubUnmanagedData abstract class for classes that is for the sub-data types that a few things use, like DarkSpirePerPlanet.
  • This adds some structure, but is not as fully managed as the main classes above it. These again help us to avoid some bad habits that can lead to memory leaks on clients.

• Fixed a general cross-threading issue that could happen with Loci_ForUI in the dark spire. This could happen in single player or multiplayer, and we found it because of the new code rigor enforced by these new abstract classes.

• The DeserializeDictionary utility method, and a few similar ones, no longer take their dictionaries as a ref parameter. That's now allowed syntactically for readonly dictionary fields, and it's not needed logically at all for dictionaries as a whole, which are reference objects and thus always passed by reference no matter what.
  • Same deal with ArcenRandomDrawBags, for the same reasons.
  • And same for some various working methods that were passing generic List<>s with ref explicit. It doesn't matter if the contents of the list are a value type, the list itself is still a reference type.

• Fixed a client-only bug where certain things were being cleared incorrectly on the DLC2 "ZM" faction data object during faction sync, but now are not.
  • In that same faction, fixed another of the "ForUI" list cross-threading issues.

• Added a ClearAndCopyFrom() method onto ArcenSparseLookup, which does what it says on the tin.
  • This is useful in places where we are no longer shuffling around sparse lookups by reference (think pointer), but instead are now keeping the lookups but needing to copy from one to another.
  • The need for this is pretty rare, but it's used in wave planning.

• Discovered a rather dire issue from the last week or so, and have fixed it.
  • Essentially, the objects that previously inherited from IArcenExternalDataPatternImplementation (and now from a descendant of ArcenExternalDataPatternImplementationBase) are NOT UNIQUE per object.
  • So the fact that we were starting to store the ParentWorld and ParentSquad and so on on those was actually a major problem, and probably a source of some sort of random bugs in single player and multiplayer.
  • We were mostly saved by the fact that we ignored those properties and were using the Source object cast properly, which is what was supposed to happen.
  • But upon refactoring things, we ran into increasing problems with this. It has now been corrected, and this should hopefully also prevent us (or a modder) from making the same error in the future.

• With the new way that we are using readonly for collections on the ArcenExternalSubManagedData and ArcenExternalSubSubUnmanagedData, we have to initialize all those collections properly in the constructor, which was not always happening before.
  • We now are just initializing them as part of their definition itself, which is easier to check for correctness and has the same end result and performance.
  • If we missed any of these, or heck if we made any typos in the many many other changes list above, then there will be errors when loading certain savegames or when during certain parts of gameplay.
  • We should be able to fix such issues very quickly, and for now we're not seeing any issues, but we'd definitely appreciate testers in both singleplayer and multiplayer.

• It goes without saying, perhaps, that this breaks all of the code-style external mods again. We will probably break those a few more times in the coming weeks as we get things multiplayer-safe, and then will stop that.

Version 2.505 Multiplayer Alpha Starts, Take Two

(Released September 14th, 2020)

Let's try this again! This is one big reason we did a "soft launch" into MP alpha. If anyone is able to test tonight and let us know how it goes for you, then we will probably make the official "hey there's multiplayer alpha now!" posts tomorrow, unless new showstoppers are found. At this point there's the periodic minor "waiting for players" micro-hitching annoyance, but beyond that being a small thing, Chris was able to play for 18 minutes between two computers across Valve's network tonight. No permanent sync issues, no degradation in performance, nothing that seemed glaringly off. There will be more bugs, but the core stuff -- knock on wood -- seems to be working. There's a good list of things that need to be done in the near term here, and that's just in terms of general functionality, not MP-specific features.

• Fixed a bug where "??? Name" was showing up incorrectly in the factions list in the prior build.
  • Thanks to Badger for finding the typo.

Multiplayer Sync Initial Implementation Now Tests Out Initially Working

• Followed several paths of code trying to fix the ship sync, but ultimately wound up reverting a lot of that code after we found the actual cause.

• Found a one-line typo that was leading to infinite duplication of every fleet on the client every few seconds.
  • Essentially every fleet was trying to match to fleet 0, and never finding it, and then creating a new one.
  • This was leading to a really devastating amount of duplicate data, and confusing everything on the client.

• Manually reviewed the code for all similar things, including factions and planets, and all of those seem good for now.

• Discovered that it was possible for extra fleets to wind up on the client just hanging around after a full fleet sync. Those extras (usually were only one or two) now delete themselves.

• The clients no longer automatically add 1000 to all of the PKIDs that they are syncing from the host. We welcome the extra conflicts at this point... maybe... as it will also lead to some sometimes-correct things that don't need to sync.
  • This was something we were doing as of a couple of versions ago in order to try to prevent things from fighting over the same dictionary spot.

• At this point we have far fewer syncs needing to happen between the host and client, which is awesome.
  • The number of ships and fleets is consistent between the client and host, and the energy usage is also consistent, and so on (though energy usage may be misleading since that gets synced from the host as part of the ultra-frequent processing).
  • A bunch of ships are showing up as invisible on the client for whatever reason, we know for certain, but those ships that are showing up maintain sync properly.
  • There are still some extra ships in ship fleet lines on the client that should not be there, and may just be duplicate references to existing ships.
  • Overall the total stats on the number and size of messages sent and received on the clients and the server seem to be incorrect, although sub-parts are often correct.

• The "Deletion Time!" header in the ship sync details log now only appears when there is actually something to delete, which is not all that frequently anymore.

• Fixed a bug on the client and host in multiplayer where the total messages received, and the total bytes of them that were received, were far too low. A filter based on message type was incorrectly being applied to these overall counts, where no filter should have been applied.

• Fixed the remainder of the network messages still not being properly logged, in size or number. There are two places where filters applied, and we missed the second one at first.

• Fixed an issue where ships that were network-synced were properly added to fleets and the global registry, but NOT to the planet faction (and thus faction) registries. So they were showing up as invisible, despite syncing properly as of today.
  • Now the ships show up just fine!

• Previously we had put some logic in place to not delete ships too fast on the client if they were not discussed by the host in a sync pass. But now that the sync passes are working, this is no longer needed (and in fact actively makes things look wrong for seconds at a time).
  • The game now does a much quicker job of removing these extras, and consequently there are not as frequently extra ships that should not exist in the fleet memberships sitting around.
  • It's worth noting that for 3ish seconds at a time right now, it is possible for these ships to hang around (but they only appear on the client, and can't be given any orders, which is confusing and annoying but a comparably minor thing and will be tamed soon when we improve how PKIDs are allocated so that fewer clashes happen.

Version 2.504 Clarity And Fair Play

(Released September 14th, 2020)

We're no longer on the beta branch, because we don't think that this will negatively affect the single player experience. However, multiplayer's alpha is still not ready to go. This build is focused on fixing things up to get things back working properly for single player (which of course also affects MP), and we'll be resuming work on getting MP ready now that this is out.

• If a player-allied faction kills something that would grant rewards (science, hacking, metal), the game now sees if there were player ships on the planet too. If so then it grants the rewards to the strongest player on the planet.
  • This seems the path of least frustration for players
  • Thanks to Ymir for the bug report, and others for a spirited discussion on steam.

Bugfixes

• When placing wormholes on a planet, the game now makes an effort to make sure they aren't on top of eachother. In very rare cases, some map types could wind up with wormholes that basically overlapped eachother, which was very hard to work with.
  • Thanks to Mac for pointing this out.

• Fixed a longstanding issue with the number of entities contained within a ship where it was showing much lower numbers than appropriate, because we were indexing into the wrong array of display numbers. Above 100 units, it should now show much more appropriate numbers.
  • Thanks to TechSY730 for the report that finally tipped us off to the potential cause of his strange area.

• Fixed an issue that could cause exceptions in the target list sorting if ships died at just the wrong moment.
  • Thanks to CRCGamer for reporting.

• Fixed a very unexpected bug that our recent extra-strict ExternalData checking logic let us find. Essentially it was somehow possible to get a second Praetorian Guard faction for a single AI, although we're not quite sure how. The game now actively clears those out when it is loading a savegame or settings as a template, to keep you from having double-strength PG forces for no apparent reason.
  • Thanks to Deathlymad and Metrekec for reporting, and for providing the savegame that let us practice testing and fixing the issue.

• Fixed an exception that could happen at various times if the spire debris data was null when it was thinking about making notifications relating to them.
  • Thanks to jradishurr for reporting.

• Fixed a minor visual error where it would say "ERROR: my FleetMembership is null!" if you were viewing a tooltip or popup window with a ship in it that had died. It now will say "This ship has died." instead, if the ship properly died (which is likely to be the case).
  • Thanks to GreatYng for reporting.

• Preventatively 'fixed' theoretically-possible-but-not-encountered bugs (that might not actually have been bugs in the end relating to the following, all based on the new ExternalData changes:
  • Winning a game and getting a superterminal achievement (probably not likely).
  • Exceptions in notifications about wormhole invasions (this one probably would have been hit eventually).
  • Exception in notification tooltip about spire debris (again was likely to be hit).
  • Exception in notification tooltip about instigators (very unlikely).
  • Two exceptions in notification tooltip about astro trains (somewhat likely).
  • Exception in notification tooltip about relic trains (unlikely except maybe for multiplayer client).
  • Exceptions in five tooltips for DLC2 (likely to be hit).
  • One other exception relating to DLC2.
  • Possible exception with the Shark B plot on.
  • Several possible exceptions in writing out the list of AI factions and their subtypes (most were very unlikely, except maybe in multiplayer on clients).

• Fixed a pretty huge bug where if certain entities died to remains (rather than permadeath), they would give AIP every time they died to remains, but when the normal permadeath setting was on they gave no AIP. This was the opposite of our intention and has apparently been incorrect since July 23rd.
  • This meant that things like GCAs, etc, would not give their proper AIP on death.
  • Thanks to ParadoxSong, Strategic Sage, crawlers, Isiel, and Puffin for reporting and helping track this down.

• Regular engineers are no longer part of the fleet membership selections for anything that you can get via battlestations or citadels. Instead, combat engineers are used for those cases.
  • This will not affect existing savegames, only new ones.
  • Thanks to ultamashot for the report.

• On September 6, we added the following: "QoL tweak for repair. Any ships on a planet where you outnumber the enemies 10 to 1, or the enemies have less that .2 strength are eligible to be repaired immediately. The previous rule was 'Any ships on a planet without enemies'"
  • This now only applies to ships and structures of factions that are friendly to the humans. Any neutral or enemy factions will now use the old style of logic that existed prior to September 6th.
  • This prevents the AI and Exos in particular from developing invincible shields when they have overwhelming firepower against you, among some other unintended consequences.
  • Thanks to TechSY730, crawlers, and MasterGeese for reporting.

Fireteam dynamic resizing

• In the late game, powerful factions can have hundreds or thousands of fireteams, which can make LRP threads run very slowly.
• Each faction using fireteams has some variables that control 'how large should my fireteams be'.
• Now for every 50 fireteams a faction has, it will require future fireteams be larger 1.5x larger. If a faction starts losing fireteams, the size requirements will go back down
  • The end result is fewer but larger fireteams, which should be a nice performance boost. Note this won't really help existing games as much as it will help new games to never have that many fireteams at all. Applies to all factions that use fireteams except the Imperial Spire, since that ends the game quickly

AI Difficulty Description Rewrites

• The tooltip descriptions for AI difficulties have been updated after folks have given us indication that there was some confusion based on the old ones (particularly sometimes players felt bad for playing too low, or did not understand the true purpose of difficulty 10, etc):
  • Difficulty 1:
    • Old: So easy it might be asleep.
    • New: The AI is effectively dormant; this difficulty is good for practicing the game without any real opposition.
  • Difficulty 2:
    • Old: Actively does stupid things.
    • New: The AI will only utilize its most basic subroutines. Retaliation will be minimal in response to your efforts.
  • Difficulty 3:
    • Old: Excellent if you're just wanting to mess around with no stress and you've never seen an RTS, ever, before.
    • New: The AI is conscious, but will operate primarily in a reactive capacity. Most of your opposition will be in the form of defensive AI forces in enemy territory.
  • Difficulty 4:
    • Old: If you're only vaguely familiar with RTS games, this might give you an interesting time. But bear in mind all the cool and clever tricks aren't remotely here for the AI yet.
    • New: If you’re ready for a little push-back from the AI and some confrontation on the homefront, this will be a good introduction to some of the behavior you can expect from higher difficulties.
  • Difficulty 5:
    • Old: If you're here for the first time, but know your way around RTS games, this isn't an awful place to start. The AI is partially brain-dead still, but it should give you an interesting time. But if the game seems too easy, you'll know why.
    • New: The AI takes your presence a bit more seriously. You may find your attention split from time to time and will have to more carefully consider the repercussions of your choices and how the enemy will react to them. Ongoing reckless expansion on your part has the potential to incite a deadly response from AI.
  • Difficulty 6:
    • Old: If you're solid at RTS games, then this is what would typically be considered 'Normal' for you... potentially. The AI doesn't have its full bag of tricks yet, but it's still pretty crafty and is starting to have some of its tricks. This difficulty is also great to use if you're really wanting to have the AI be there, but some other factions be the MAIN threat. This will keep the AI interesting but not on your back all the time.
    • New: Against a Difficulty 6 opponent, your choices are really starting to matter. You will notice more devious tactics being used by the AI and will need to watch your back as you claim assets and territory. Other factions have the potential to be more threatening depending on how you set them up, but the AI will continually make its presence known.
  • Difficulty 7:
    • Old: Players in the first game had a saying, that 'the real game starts at difficulty 7.' And while that is kind of... rude?... there is also some truth to that, in that the AI is finally fully unshackled. There are a few truly nasty things for the AI that you would still need to turn on in the AI Behaviors section of the Options tab if you really want full pain, but this is a fully competent AI. This is also the difficulty that Chris, the original developer of the games, plays at. It's fun and he can play in a relaxed fashion and not be super focused on every detail, but still win only about a third of the time playing that way. So if that gives you some measure. He plays most other RTS games on their max difficulty or close, not that that's always saying much.
    • New: This is where the AI stops holding back and utilizes all of the strategies and resources at its disposal to try and keep you at bay as your power grows. Its economic power is still somewhat limited, and while you can still get away with being a bit of a nuisance early on, the enemy won't hesitate to put you down if you make yourself too much of a threat.
  • Difficulty 8:
    • Old: This is the sweet spot for veteran players. The AI has all its tricks, and has a bit of a stronger economy and stronger responses, too. You COULD make the argument that the economy of the AI at difficulty 7 is a bit on the weak side, so it's not that this is a cheaty AI or something. For players who are really paying attention and managing all the small things and want to have a drag-out battle, this is the difficulty of choice. If Chris really wanted to white-knuckle it, he could probably maintain his 30% winning streak at this level. But he's not into that kind of stress. There's a good chance you're better at RTS games than he is, though, if you're a veteran in particular, so maybe that's not so much stress for you.
    • New: If you take the fully-functional Difficulty 7 AI, provide it with a bit more fair allotment of assets, and make it more sensitive to your actions, this is where you end up. Ideal for veteran players really looking to test themselves.
  • Difficulty 9:
    • Old: Are you super awesome at RTS games in general, and this one in particular? Difficulty 8 is just proving too passe? This keeps all the tricks from before, but cranks up the economy and some of the frequency with which the AI will harass you. You could argue that at this point it is getting to be slightly a 'cheating AI,' but frankly things are so lopsided it is hard to make that kind of distinction. If you're so much better than the average player at managing your empire, then... I guess this is kind of taking away that advantage that would otherwise let you roll the 'fair' AI? We get into murky waters here, but a subset of veteran players enjoy this difficulty, so here it is.
    • New: Making for an exceptionally tough adversary, Difficulty 9 stacks the odds in favor of the AI. Armed with a robust economy, the enemy will be ruthless in their attacks and relentless in their harassment of your defenses. To overcome this challenge, you will always need to be on full alert to prevent the AI from exploiting any weakness in your strategy.
  • Difficulty 10:
    • Old: This difficulty level is not meant to be fair. It's using all of the AI's legitimate tricks, and it has its economy and whatnot cranked up to 11. If you win against this level of AI, that's something that we traditionally consider 'a bug.' You are supposed to lose, every time. Please file a bug report with how you won so we can fix it. So why do we have this difficulty level? There are some players who are just THAT GOOD, and they spend their time trying to find weaknesses in the AI despite its unfair advantage. A lot of improvements to the AI have come about because of players on difficulty 10 telling us how they won. We usually adjust unit balance or AI logic in response to difficulty 10 victories, we don't just make difficulty 10 harder. That would defeat the point.
    • New: Economically, the 'SuperCat' AI gets a substantially unfair advantage. Militarily, they still play fair but will be able to bring overwhelming force against you in most every situation, all the time. A subset of dedicated players enjoy this difficulty level, but it is not fun for most. Beating the SuperCat even once is considered a crowning achievement, and usually involves a player discovering an unbalanced tactic. If you find yourself in the enviable position of being able to have your name added to those who win a 'pure' difficulty 10 win, please file a bug report with any tips you have on how we can (legitimately) upgrade the AI to counter your winning strategy. This back-and-forth arms race between the players and the developers has led to some of the most interesting innovations for all levels of AI for the game (and its predecessor), and it is also a very specific style of brutal gauntlet that certain dedicated individuals enjoy throwing themselves into.
  • Thanks to Tzarro for writing the new descriptions for us, except for the new difficulty 10 description.

Beta 2.503 Multiplayer Not Quite Ready

(Released September 11th, 2020)

This remains on the beta branch in case there are more singleplayer issues. As far as multiplayer goes, things are not stable enough to do any real testing. There is an issue of some sort that causes fleets to infinitely multiply copies of themselves on the client, which leads to a lot of lag and other problems. There's also an issue with various ships syncing in and out of existence in funky ways, which essentially makes multiplayer unplayable. This is something that is going to take more hunting, but it's all one cluster of issues, and once it's resolved multiplayer should be pretty ready to go. Really glad we did a soft launch! It would have been awesome to have people able to play multiplayer over the weekend, but we're just not quite there at the moment. Hopefully in the first couple of days of next week we can figure this out, but for the moment our brains are fried. More to come soon!

• There's now extra error handling in the "stage 3" faction logic, to keep that from ending the entire game when those happen. It reads:
  • DoPerSecondLogic_Stage3Main_OnMainThreadAndPartOfSim Error for faction [name] (index [a number]). Consider restarting the game, as many other things may now go wrong because of this first error. And please report this! Error: [error text]
  • This will probably solve things like paralysis counters not counting down when there are such errors, but you're still likely to have a strange time.
  • Thanks to StarKelp, Mac, and gigastar for reporting.

• Fixed a bunch of places in notifications where it was still looking to the old locations for an out of date icon. We will have real icons in the future for those, but for now the placeholder will no longer throw an exception.
  • Thanks to jrad for reporting.

• Fixed an exception that could happen when killing instigator bases, in the notification tooltip for them.
  • This is another regression... kind of... from the last few betas. This actually will make clients more robust in multiplayer, though, so it's better than just a straight regression fix.
  • Thanks to TechSY730 and jradishurr for reporting.

• The Civilian Industries mod has been updated to be compatible with the new versions of the game.

Multiplayer Ship Sync Fixes

• Added a setting to the networking section of the settings menu: Log Human-Readable Ship Network Syncs To Disk
  • Only relevant on multiplayer clients, not the host. Will majorly slow the game down, but dumps a huge amount of networking info to the disk in the NetworkHumanReadablyShipSyncLog.txt file in the PlayerData folder. This gives a message for each ship that was changed or deleted and why that was done, as well as what planet it was on and who owned it. When syncs are doing funky things, this is a way to manually review it and find out why.
  • This required us putting in a LOT of extra code to handle log

• The current sync cycle is now sent with the divergent data sets, hopefully leading to clients not deleting things they just fixed (because that was totally happening before. Logs are a magical thing.).

• Discovered a major oversight in our sync code, where basically once something had been synced once it was then being synced every sync cycle forever after that.
  • This was a huge cause of lag for multiplayer, and made things disappear inappropriately, probably.
  • There are still other things wrong with the ship sync, though, that we can see in the new logs.

• The sync code has been made more robust in general, now syncing the type of entities as well as what faction they belong to.
  • These are often incorrect, when there are catastrophic different uses of primary key IDs anyhow, so getting those right is a good thing.
  • It occurs to us that maybe this is overkill, since if these are wrong then probably the planet or at least the location would also be wrong? But... let's leave it in for now, and maybe make an option later for skipping it. It's not a lot of data to transmit or a lot of extra processing.

• Fixed a couple of issues with reusing entities as different ship types, where it will now properly add or remove or reassign system types on them.

• Fixed some issues with reusing entities during sync where they were not being added back to the central lookup of entities, leading to future syncs to fail on them forever.

• Put in some logic to improve the number of loops before we remove something that isn't being synced properly, to make it based on the number of loop counts rather than the amount of time something has been alive (since things that have been alive for a while can get fully synced later).

• Fixed some conflicts that could happen with entities in the central lookup on the client in multiplayer, if the order of registering and unregistering entities gets out of sync.

• Intentionally desyncing the PKIDs of ships, wormholes, and fleets on clients compared to the host.
  • Trying to reuse entity IDs was often causing some notable problems, and they were rarely in sync as it was.
  • This forces a sync cycle for every new ship and fleet that is created, but makes conflicts on the client a lot less likely.
  • This may not be a good long-term solution, but a refactor to avoid the conflict space is potentially where we're at, honestly. That will take days, and add a lot of bugs temporarily, so it would be great to avoid it.

• At the moment the game creates infinite extra fleets on the client, slowing it down progressively. This makes it effectively unplayable.
  • There are also about a thousand ships that it will routinely delete from the client, then re-add, then delete again. Also making things unplayable. Still looking into it, but this is likely to be a Monday thing, as this is not a simple matter.

Beta 2.502 Regression Fixes

(Released September 11th, 2020)

Note that this build is still in beta in order to let people have a chance to run into any more regressions that we may have introduced. So far none of the regressions have indicated any errors in older builds, which is a nice thing. Though in the process of making the prior beta build, we did fix a number of "useless extra data" things, so even if we don't find any new bugs that were older, these regressions were still worth it to ensure data accuracy. It is quite annoying in the short term, though, and we really appreciate the folks taking the time to test it out.

• Add some new voice lines when the player is attacked by the Nanocaust

• Hopefully Zenith Forcefield Generators will now return after being pushed
  • Thanks to GreatYng for reporting, and Puffin for reminding us of the fix

• Fixed a bug from the last couple of builds where chat commands were failing in general.
  • Thanks to Magiik, MasterGeese, gigastar, and Richard333 for reporting.

Fixes To Regressions From Prior Beta Build

• Fixed a bug in the latest beta build that was causing new games to constantly error out with autosave data not being added properly.
  • This is one of those "externaldata is now more explicit" things, but this is an example of us choosing the wrong explicit option this time around.
  • Thanks to Puffin for reporting.

• Fixed an exception that could happen when trying to generate notifications about AI reserves before their data was initialized. This was caused by the last beta, but is better to have it handled this new way (so, yay error, this time).

• When there are errors in generating a galaxy, it now no longer lets you into the galaxy to play (since that leads to many random errors after the real one with the map not generating).
  • Random things might include lots of planets belonging to no one, among other things.

• SeedStartingEntities_LaterEverythingElse during mapgen now has a lot of new exceptions it can throw if various data is missing that it is supposed to have.
  • Mostly these are related to the AI and its subfactions (hunter, warden, and PG), but there's also one for if risk analyzer data is missing.
  • Prior to the last beta, it seemed to have been just using blank data, although it's impossible to be sure. And in the last beta, it was throwing an exception.
  • We then tested all of the factions in the game and the first two DLC, and the only three types that threw exceptions in this area were the AI subfactions (hunter, warden, and PG).
    • We could use more robust testing of the game starting with various factions and them having various settings, though, to make sure that it's all going across properly (this is just in single player, not even multiplayer).
  • Thanks to Puffin and jradishurr for reporting.

• Fixed four typos in the last beta that were not properly having the "related AI and subfactions" properly reach for one another, which led to missing or misapplied settings on game start.
  • This then left the Praetorian Guard still not working properly, because they have no custom fields in the lobby and thus were never having a reason to initialize. That probably was not a problem, but to keep things consistent and safe we are now initializing it when the general "AIDifficulty" field is processed.
  • For the record, prior to the most recent beta this means there were not actually any errors, but now we do have protection in place in case such an error pops up in the future. But for the time being, these errors were just in the most recent beta, which is gratifying to know.

• Fixed a couple of logic errors with risk analyzers in the most recent beta that was causing their notification to show up wrong as well as also firing them immediately on game start.
  • They now should work properly again, and if the notification is going to be wrong, it will be more informative in its wrongness.
  • Also fixed the fact that this could happen when AI Risk Analyzers were not even enabled as a faction.
  • Thanks to Puffin for reporting.

Beta 2.501 First Raft Of Multiplayer Fixes

(Released September 10th, 2020)

This one is on the beta branch on Steam and GOG because of how much we changed with the "ExternalData Accidental Creation Avoidance" section. There may be legitimately new bugs that we introduced from that, or there may be old bugs that now simply show themselves with error messages. Either way, we don't want to inflict that on everyone, so please use the beta branch to help us test this one out.

• There is a list of known issues with multiplayer here: https://wiki.arcengames.com/index.php?title=Category:AI_War_2:_All_About_Multiplayer#Selected_Short_Term_Todo_List
  • It's debatable how playable the alpha is at the moment, given the issue with the client ships deleting themselves rather than syncing, and given some of the (essentially) client-side memory leaks. Those two things will be my main priority tomorrow, unless something else more pressing comes up (like fixes to things broken by the ExternalData stuff).

• A new button has been added on the main menu above the forum link that links directly to the AI War 2 discord channel.
  • Thanks to Metrekec for suggesting.

Better Default Screen Resolutions!

• Fixed a bug where the fullscreen resolution was still being saved into newsettings.dat rather than graphicssettings.graphics.
  • This meant that the fullscreen resolution was being sent over cloud sync, when really that should not be, since that is a machine-specific setting.
  • This may wipe out your prior values for the fullscreen resolution, requiring you to set it again.
  • Thanks to jrad for reporting.

• At long last, added a startup feature that folks have been wanting for a while: better default screen resolution, for a better first impression as people start the game for the first time.
  • Previously, this particular game was just having a default of opening in windowed mode and 1024x768px, since that will fit on most monitors.
  • One of the reasons for this is that opening directly in fullscreen mode can cause bugs, particularly when settings are copied from one machine to another. But we are using fullscreen windowed mode, which reduces the chance of fullscreen bugs, and we also have our graphics settings not set to cloud sync for the last year or so.
  • With that said, now the game will automatically (first time opening it on this or future versions on any given computer) set itself to be fullscreen mode with your desktop resolution.
    • It also will set your windowed mode defaults to be your desktop resolution minus 80px width and 100px height. So if you do flip it back to windowed mode, it should be at a size that feels sensible for your machine.
  • If you are using a very high-DPI monitor on an underpowered machine (for instance how Chris is testing with a late 2013 MacBook Pro 15" that is below minimum specs for the game), then you may want to lower the fullscreen resolution to something that machine can handle more gracefully (for instance, with ship graphics off and only running at 720p, that below-specs machine runs the game great).
  • Note: if you are already in fullscreen mode, then unity just reports what your current fullscreen resolution is. So it will save itself to that, and then set your windowed mode resolutions to be a bit smaller than the current fullscreen mode. Normally this game opens for the first time in windowed mode until this logic kicks in, so this only applies if you already have a current fullscreen setting active.

Multiplayer Fixes From First Alpha

• If there is a null result from FindArcenSteamClientConnectionByConnectionID in OnMessage on a Steam host, it now will write a more detailed and informative message as to why.
  • Additionally, in general OnMessage on the Steam server and client is now far more instrumented if those wind up with issues.
  • This won't actually solve any problems, but for the case where there was a client disconnect on connect, this should tell us what is going on more.
  • Thanks to StarKelp for reporting.

• The numerical order of the detailed networking sync logs is now better regardless of OS file sorting, as it gives leading zeroes where needed.

• While in the lobby, or while players are still connecting, the game no longer tries to run the general sync-correction code. This was an oversight in general, and was leading to various errors that would keep popping up until the host saved in-game and clients disconnected and reconnected.
  • This was the chief cause of the ""Fixed attempt to read more faction data than we had factions" error on the client.
  • Sync was already being handled as well as it needed to be in the lobby in particular: all it needs to do is make sure that your UIs are consistent, which it does. The actual underlying data about incomplete factions and such that can't be seen yet are really quite irrelevant at that point, and so it actually skips a lot of that data, which was incompatible with the full main-game-style sync. As soon as you start a game from the lobby, it already doing a much more robust generation and transmission of the data.
  • After loading into the game from the lobby, however, it seems that that initial sync is not as complete as we had hoped, so that's another area for us to now investigate.
  • Thanks to StarKelp and his play group for reporting.

• Fixed a bug in deserializing player accounts, which was something that was leading to the immediately-after-lobby sync being broken.

• Fixed a harmless extra blank fleetID that was being sent as part of the PeriodicWorldExtrasSync, which made them look inconsistent but did not cause other problems.

• Fixed a larger bug that was actually preventing deserialization of divergent ships from working properly if the ship did not already exist on the client.
  • We had fixed this the other day, but made a mistake in the fix and had to do it over. Now it works!

• There are still some more issues with syncing divergent ships that have external data on them, and we're not sure why yet.
  • The deserialization code for external data now has extra error handling in general, so that we can be more informed if something like that happens during sync or a load off of disk.

• On the ArcenSerializationTester, added AppendIfActive() and AppendLineIfActive(), which are basically like Append and AppendLine().
  • These return a ArcenSerializationTesterWriter so that they can be chained into concisely readable calls like other parts of the code.
  • We want to be able to write more complex data in for informational purposes without it being the full WriteHeaderStringIfActive(), and without having to do string concatenations that hit the GC.

• Several pieces of new logging are now in place to help us more easily identify problems syncing unit data, and external data in general.
  • We were seeing some mighty funky stuff on ExternalData syncs failing in multiplayer, and are trying to understand what is happening and why.

• A WHOLE lot of extra error handling and instrumentation has been put in place around ships and externaldata in general.
  • Basically if something goes wrong, we don't want it to do so semi-silently. The various problems that most people were seeing in the first multiplayer alpha version were really downstream issues from the real errors, which were largely silent.

• Found and fixed a part of the externaldata serialization that could be null in some cases, requiring us to instantiate it even on a partial sync.
  • This is something that we had already guessed we would have to do, based on us doing the same thing with ships themselves last version, but the silent errors happening here took us several hours to figure out what was going on. It's fixed now, but there will be more cases of this, probably.
  • With this fixed, the divergent ships no longer throw any errors. However, there are still some major differences between the client and the host that need to be looked at.

ExternalData Accidental Creation Avoidance

• Put in several fixes to potentially remove ScourgePerUnitExternalData from accidentally being created on any unit that was part of a fireteam.

• This is going to break code mods temporarily.
  • GetCollectionByPatternIndex() on externaldata now takes a new ExternalDataRetrieval enum, which can either be CreateIfNotFound or ReturnNullIfNotFound.
  • The default used to always be CreateIfNotFound, which was nonobvious and was causing things like scourge data to appear on non-scouge units. But very likely it was also causing all sorts of other data to be erroneously initialized. This wouldn't have broken anything, but was certainly bloating savegames prior to this version.
  • For reference, all of the methods for getting these are expected to follow this sort of pattern: GetScourgePerUnitDataExt( this GameEntity_Squad ParentObject, ExternalDataRetrieval RetrievalRules )
    • In the main game and first two expansions, this led us to having to correct 112 locations in 26 files. That in turn required another 705 secondary fixes.
      • This may seem excessive, but being able to verify that we are correctly initializing data only when needed is a worthwhile goal, and it makes code clarity so much greater.
    • There are a variety of places that this may make a difference based on our changes thus far (aside from whatever bugs we have introduced):
      • Reinforcement spawning may be more correct now. There was previously some logic that may never have been hit if it was originally pointed at a non-sentinels faction, but now it will hit it.
      • AI sentinels data will no longer be put on every faction during the post-victory achievement check. That was likely causing some problems (which someone had put a mantis report about post-victory slowdown and exceptions, so it's possibly related).
      • When checking faction intensity in general, a whole host of wrong data collections are no longer created on random factions.
      • Several things about astro trains can no longer cause accidental data on wrong factions.
      • If astro trains that should be spawning AI waves or adding to the AI budget are pointed at a wrong faction, the data no longer goes into the void but instead an actual error pops up.
      • Several hacks will now show exceptions rather than throwing their data into the void if they are pointed at wrong factions.
      • There are a wide variety of places where the AI difficulty or AIP of an AI were referenced, and which might now throw a nullref exception. If any of those DO, then that is actually a good thing, because in the past those have silently been returning 0 for both rather than using real numbers. If neither of those cause any errors, then that's even better because we know our other code has been correct already. Fingers crossed for more of the latter than the former.
      • Any time any fireteam was disbanded, all of its units were assigned blank scourge external per-unit data. Fixed. Oh, actually every time it looped over the units in a fireteam. Fixed that, too.
      • ExoData is no longer added to every last faction in the game (notifications checks were causing that).
      • It's possible that some tooltips or notifications might throw exceptiosn now, particularly if you load a savegame and mouseover them before unpausing. If these throw exceptions, then basically this is a case where it would have been gibberish data previously, so it's still useful.
      • Various pieces of code like "don't run the nanocaust info if it's not set up" will now work as their programmers likely originally intended.
  • This probably introduced a number of new bugs from typos, along with whatever bugs it uncovered, so this is why we're heading back into the beta branch temporarily.

Version 2.500 Multiplayer Alpha Begins Now!

(Released September 9th, 2020)

• Removed some extra code that was accidentally included that was preventing the new findp command from being able to cycle through planets properly if there were multiple matches.
  • Thanks to cml for reporting.

• PlayerAccounts are now also passed to cheats/commands (not to be confused with gamecommands), so that now if there are multiple people in charge of a single faction, commands that are sent can affect just one of the players if needed.
  • This is now used for the findp command, which lets two players share control of a single faction without the findp of one player affecting the other.

Multiplayer Readiness In The UI

• The multiplayer button on the main menu now has a small bright "Now in public alpha!" tag on it, to make sure no one misses that.

• In the multiplayer section of the main menu, a new "Alpha Testers: Please Read!" button has been added.
  • This has a tooltip that says:
    • Click to open a web browser that explains the current state of the multiplayer alpha (that changes almost by the day), as well as questions for testers (also get frequently updated), a history of recent improvements to multiplayer categorized by date, and a list of work items that are upcoming.
    • Most important of all: if you are running into problems, please take the time to report them to us, rather than assuming someone else will report it. That is absolutely the most helpful thing we can ask for.
  • And its link goes here: https://wiki.arcengames.com/index.php?title=AI_War_2:Multiplayer_Alpha_And_Beta#What_Does_Multiplayer_Alpha_Mean.3F

• At the top of the networking section of the personal settings menu, there used to be an option called "Enable Multiplayer Alpha"
  • This had warnings about how multiplayer alpha was not ready, but that you could click this to enable it anyway.
  • This option has, happily, been removed!
  • Previously, if that option was not enabled, then clicking the Multiplayer button on the main menu would show a popup with a message from Chris explaining the current state of multiplayer and projected timelines.
    • That message has been removed, and clicking the multiplayer menu now just opens it, again, happily.

Included Mod: Civilian Industries Pre-Multiplayer Finalizations

• Final optimizations for Multiplayer, touching up loose ends.

• Many nerfs in regards to AI Raids.
  • A 20-40%, based on intensity, reduction to AI Raid strength.
  • A roughly 100% longer warning before they fire.
  • They now only spawn a singular wormhole per planet.

• Large scale Trade optimization
  • The Grand Station will now output a larger amount of cargo ships on demand.
  • Many more Cargo Ships can accept trade routes every second.
  • Trade Stations will no longer be built on planets that construction ships couldn't safely reach from the Grand Station's planet.
  • They are now much smarter about stockpiling resources for local militia use before exporting them.

• Additionally, a large number of undocumented bugfixes that were lost in the surge of all these multiplayer-based updates.
  • The most notable fix is a fix to the Exception that could pop up when mousing over the Raid notification.

Multiplayer Sync V1.0

• Added AddBytesWithFormatAndColor() and AddBytesWithFormat() for writing bytes nicely to an ArcenDoubleCharacterBuffer.
  • This cuts down on our repeat code for easy formatting of things that might be in bytes, KB, or MB.

• There is now some basic sync stats that will show in the escape menu during multiplayer, and then more detailed sync stats that can be turned on at will.

• New setting in the network section of personal settings: Show Network Sync Details In Escape Menu
  • Only applies when you are in multiplayer, and has different outputs on the host versus clients. Gives statistics on how much the game has had to correct in terms of divergent data between the client and the host.
  • Note: The PKIDs and squads are actually checked for divergences, and those are shown in more detail on the client than the host. All the other things are sent in a periodic overwriting fashion, without checking for divergences, as the bandwidth used (which you can see with this setting on) is less disruptive than the CPU cycles required to do divergence checks.

• The width of the columns in the escape menu are now a little wider.

Fixes Based Off Testing Initial Implementation

• Fixed a bug where the client in multiplayer was incorrectly calling a host-style method when trying to tell the host about divergences in squad sync.

• Fixed a bug in multiplayer sync where the squad sync stage would never complete or advance beyond the first 20th of the squads, instead just doing those ones over and over again.

• Fixed a minor bug in multiplayer where the client was telling the server about mismatches even when there were none (just sending an empty list to them).

• Fixed a one-line bug in multiplayer sync that hilariously just caused all of the ships on the client to be deleted within a few seconds of playing.

• Put in extra debugging for Client_AcceptDivergenceDataFromHost(), since it was having some exceptions happen.
  • In general at the moment, actual divergent ship data is coming across slightly garbled.

• Temporarily disabled the planet and world-extras sync steps in the multiplayer sync code, as those both were throwing errors on the client side.
  • With those disabled, we can see that the ship sync, fleet sync, and "ultra frequent" sync all seem to be not only working great, but also not sending too much data and not slowing the game simulation down.
  • The garbled divergent ship data is a bummer and does mess with things, but in general it just speaks to our need for better instrumentation in that area, and is not entirely unexpected.

• A new setting has been added to the networking section of personal settings: Log All Decoded Network Sync Data To Disk
  • Will majorly slow the game down, but dumps decoded sync messages to the disk in files inside the PlayerData/NetworkSyncMessages folder. This only covers sync-style data, but with this logging enabled on both the client and host, this is a great way to see why serialization is failing, if it is.
  • Enable with care! Since this data is being actively decoded as it is written, it is far larger (and in plain text) than the actual data being sent across the network. A typical ratio might be 40MB of decoded data for 500kb of actual network traffic. You can have a few GB of data on your disk after just a few minutes of letting the game run in this fashion.

• Using this hefty new tool and some extra instrumentation, we were able to locate and fix the problem with the sending of divergent ship data from the host to the client.
  • We were simply omitting the primarykeyID for entities that were different, not deleted. Things like that are easy to miss but almost impossible to find without a logging mechanism like this. Manual code review just leads you to look right past it, especially if you've been staring at it for days.

• Did a bit of an overhaul on how some of the network sync data is logged, so that we can do a folder-at-a-time comparison to find differences where any may exist.

• Server_SendBatchOfSyncsBasedOnCurrentSyncStage() now has extra debugging going on in itself, because we're experiencing some sort of errors in there on the host after a certain amount of time in multiplayer, now.
  • Thus narrowed the problem down to Server_SendPeriodicFactionSyncDataThatJustOverrides(), and so instrumented that a lot more thoroughly... except actually the problem was in Server_SendBatchOfDivergentSquadsToFix(), we just read it wrong. So that also got extra instrumentation.
    • And that was, in turn, just a simple typo on one line. Logging and instrumentation makes the impossible and time-consuming very fast and easy -- after the initial setup time cost. Very much worth it.

• Found ourselves back with another error in Client_AcceptDivergenceDataFromHost-DivergencesSection. But we can't quite use the logging as it currently exists to easily diff these on a folder level.
  • In order to make that possible, we are switching away from using a central LogIndexThisSession Int64 for all message types being sent to StartNetworkSyncDataMessageLoggingIfNeeded, and instead are having an individual index for each FilenameBase string.
    • This puts things slightly out of chronological order when we're talking global messaging, but that's because the client and server can be talking back and forth and have some different ordering between message types. We need the message types to always match up, and now they will.

• Discovered that we were not properly sending the TypeData of the ships in the divergent ship fixes messages from the host. This was then causing nonsensical errors later in the deserialization of those messages.
  • Another great example of something we would not have found, potentially at all, without the new instrumentation.

• Discovered a harmless logging artifact that we introduced into the last few versions where the header for fireteam data was behind its id on one machine, but in front of it on the other.

• Thanks to the logging, also discovered an inconsistency in how fleet data was being updated, which was causing those to choke and die on the client side.

• Went ahead and re-enabled the world-extras and planet syncing, since those go wrong so quickly but we now have logging on for them.
  • That immediately errored, but that is just fine for our purposes. Can you tell logging makes us happy?
  • This quickly revealed that there was something funky and inconsistent with how we were reading the planet's type. That has been fixed to be consistent.
  • That then revealed something a lot more complicated was wrong with how the external data patterns were being read in general, most notably on the world but definitely not limited to that.
    • We improved the logging and instrumentation on the external data pattern serialization in general, and this then solved the issue on the world and the larger issue as a whole that would have come back to bite us in many other areas later.
    • Then put in some more logging related to MDCExoDataExt, since it seemed to have an inconsistency. We now are marking when ExternalData is complete in the log, to make that mistake harder to make.
  • Then found and fixed yet more inconsistencies in how we were syncing squads.
    • Also we then stopped syncing some stuff that was set to "only save on the network" (and not to disk) for squads, but it was data that really really should not have been synced because it was working fields from other threads.
  • And then also fixed a random extra float that we were sending after "world extras" sync. It's the sort of thing you only see in a log, as it was invisible in the code.

• We are no longer syncing the IncomingShots on entities except during the initial world sync, because in general we are not syncing shots.
  • This is going to take some refactoring for targeting, possibly, but then again existing code will very possibly just adapt to it.
    • "Catastrophic" or "planet switch" styles of sync fixes are probably not frequent enough during combat for it to be too very wrong, but either way things will fix themselves within a rolling 3-4 second window. It's interesting, because its constant attempts to heal itself in various places, while "in battle" with attrition is reminiscent of a scene with Wolverine in X-Men 3. That's an amusing thing to realize about one's code. (Let the record show that the devs really don't like X-Men 3, but that one scene was visually neat).

• Fixed some potential nullref exceptions in CalculateSpeed() on ships, which really at this point mainly would happen when ships or fleets or speedgroups or some combo of that were being updated at the same time as they had something going on in the background threads.
  • Basically there's a whole new nest of cross-threading errors that we can run into, but there's not much we can do about them except to harden individual methods as they come up with problems, and in general also add extra instrumentation to methods if we think they have the potential to be a problem again (this one does).
  • This is kind of an unavoidable element of the loose way that we handle threading, which is needed in order to get the maximum speed out of the simulation and all the various threads that are involved. The client side of network sync is now a new source of these issues, but it's not remotely the first of them. We'll probably be swatting this sort of thing down throughout the alpha and beta of multiplayer. And then SirLimbo will somehow find a way to run into exceptions that no one else sees for the next two years. ;)

• At this point we can confirm, based on our detailed logging, that multiplayer is able to successfully run for a solid four or five minutes of gametime (at 5x sim speed) and come out with consistent results the entire time.
  • This leads to about 2600 network log files for the sync messages, which are about 100MB per computer, and this substantially slows down the game from running at an unencumbered pace (the disk writes are heavy).
  • Of course, as soon as we turned off the logging, we ran into some sort of new gameplay thing which caused issues in the ship sync code. So there are still some goblins in there, probably related to certain types of data that are not on every ship.

• Added a new setting to the network section of personal settings: Log Decoded Network Ship Sync Data To Disk
  • Same as the 'Log All Decoded Network Sync Data To Disk' setting, but only writes data for actual divergent ship fixes. Since that is one of the more-likely-to-break areas of sync, but only something like 1/15th of the actual data being passed around, being able to isolate it to this is helpful.

• Fixed an issue where the current galaxy map display mode, and the current planet index you were viewing, would be reset by the sync data on clients in multiplayer.
  • It now takes in that sync data only for non-local player accounts. Clicking to another planet only to have your view bumped back a second or two later was MILDLY frustrating, heh. And also slightly hilarious.

• Discovered and fixed an issue where if a partial sync was sent from the host to the client about a ship that the client had no knowledge of, then the client would try to read it as if it was an off-disk sync and thus read things incorrectly and fail.
  • This will probably be elsewhere in the code as well, so we will potentially need to review a lot of other data structures.

Beta 2.134 Searching For Planets

(Released September 7th, 2020)

This one is still on the beta branch on Steam and GOG, since we have continued making lots of substantial changes to the central serialization logic for the game in service of the multiplayer sync code. We're now done with what would constitute V1 of the MP sync code, so we can come out of the beta branch tomorrow so long as no one runs into any problems with single player on this build. As for multiplayer itself, we have a fair bit of testing to do to make sure that the MP sync code is doing its job and not exploding RAM usage on the clients or harming performance, etc. Once that's verified and all the bugs we find in testing the MP sync code are fixed, then we're officially into an alpha status with multiplayer. Given that the V1 of our sync code turned out to be far more thorough than we originally planned, the multiplayer alpha period should hopefully be shorter than it would have been.

• Extragalactic war ships are no longer classed as 'small' when hovering the Planets Under Attack notification
  • Thanks to crawlers for the bug report

• The Dyson Sphere produces a small number of drones, as well as its usual ships. Those few drones no longer leave the sphere's planet
  • Thanks to GreatYng for reporting

• Update the time estimates for releasing multiplayer and DLC2
  • Thanks to ParadoxSong for reminding

• There's now a Journal entry for when the AI can send extragalactic war units, so they won't come as a surprise
  • Suggested by zeusalmighty

• Fix a bug where the Helping Hands 2 quickstart was broken
  • Thanks to Spook for reporting

• QoL tweak for repair. Any ships on a planet where you outnumber the enemies 10 to 1, or the enemies have less that .2 strength are eligible to be repaired immediately.
  • The previous rule was 'Any ships on a planet without any enemies' which could be frustrating if you were trying to hunt down a few cloaked ships.
  • Thanks to crawlers and Isiel for suggesting

• Some minor tutorial tweaks
  • Thanks to MasterGeese for reporting

• The extragalactic war spawning notification message now uses 'a' and 'an' appropriately
  • Thanks to Ovalcircle for reporting

• Update the Description for the snake map to mention that it's very hard
  • Prompted by a discussion on the forums

• When you have the galaxy setting for Adjacent Planets Watched, it now takes that as the baseline and is increased by things like Economic Command Stations and Spy Cradles.
  • So if you have "Watch 1 adjacent planet" then a military command station will watch 1, but an economic will watch 2
  • Thanks to ParadoxSong for suggestion

• Small update for MoreStartingOptions by AraudB:
  • Inordinate had 10 plasma turrets rather than 2. Fixed the number of stations in the mod description too.

• Removed a blank "Fleet Experience" section from the tips window.
  • Thanks to Isiel for reporting it.

• Several different spots that could throw errors if fleet membership was null now no longer do.

• DoEntityStepLogic_Ship now has better debug logging.

Multiplayer: Sync Correction V0.90

• Added a new SerializationCommandType, which has three values:
  • NormalFullType, Network_ContinuousPersistentSync, and Network_DuringDetailedNetworkSyncStage.
  • This is now used on Factions in order to differentiate the three major types of serialization that we might be doing.

• The ExternalData on factions is no longer synced as part of the "every frame" IsForNetworkContinuousPersistentSync.
  • And actually a bunch of other stuff on factions is now limited out so that it's not over-saturating certain data while it IS still sending other things every frame auth.
  • This is also of relevance because it means that any faction externaldata that is not fully optimized to not thrash the GC will inherently do less thrashing of the GC. So that makes mods a little more safe in general in multiplayer.

• Fixed several parts of the faction object to be efficient and not thrash the GC in the way that we were doing for the ExternalData last build:
  • TechHistoryEvent, HackingEvent.

• FromServerToClient_SendAllOtherSyncDataThatJustOverrides has been renamed to FromServerToClient_SendUltraFrequentSyncDataThatJustOverrides, because it no longer is "all the other data." It's just the most common stuff.
  • Added new new messages for time-sliced sending of the following: FromServerToClient_PeriodicFactionSyncDataThatJustOverrides, FromServerToClient_PeriodicFleetSyncDataThatJustOverrides, FromServerToClient_PeriodicPlanetSyncDataThatJustOverrides, and FromServerToClient_PeriodicWorldSyncDataThatJustOverrides.

• The full-faction-data sync even has been implemented on the client and the host, and now includes speed groups (which previously we felt like were too heavy to send on the ultrafrequent channel -- which was correct).
  • This also means that we needed to make the SpeedGroups be nice to the GC on deserialization, so that's done.
  • Also fixed several oversights that would have led to hard-to-diagnose endless inflation of array contents for various faction items. Yow. These sorts of things will be hard to find any other way than manual code review, and can be in mods or the main game and can cause massive slowdowns with no central way to diagnose it. These will be "fun" to find in the future.
    • Essentially lists like FactionIndicesIAmAlliedWith were being added to each frame without ever being cleared before the new set of data was put in. This would only happen during multiplayer, and only on the client, but would start causing all sorts of problems.

• Planets and planetfactions and their externaldata have now gotten the same treatment that factions did.
  • Here again there were a couple of infinitely-expanding lists that are now fixed.

• Whew, okay, fleets and their memberships have also been updated to the new style.
  • This... is going to be very problematic, the way it currently exists. This first checkin of the fleet memberships in particular is going to be very problematic indeed, for a whole lot of reasons, but it's a good start for refactoring.
  • Essentially, fleets are so bloody complicated the way that they exist right now (in terms of data structure under the hood) that they can't work cleanly in multiplayer without a bit of refactoring. The positive news is that we can make potentially even single-player slightly more efficient with this refactor, but it's going to require redoing drones and a few other things along those lines.
  • Edit: the below solves our issue for the time being, although we may still make some changes in this area in the future to make things easier. We're no longer FORCED to, though, which is nice.

• Came up with what is hopefully a clever use of our existing sync infrastructure to handle the cascading wrong data that can happen from the current fleet structure. This was something that has us stumped for a bit, because doing the most efficient sync style for fleet memberships could lead to units that were in the wrong membership after that.
  • The simple solution is the fact that we do know when we are hitting such a case, and so we can just tell all those units to kill themselves on the client. They will then naturally and fully re-sync from the host within 2-3 seconds.
  • This is the sort of thing that we want to minimize, of course, but the worst case is having something be permanently wrong in a MP game and us not being able to fix it. Later logic can always do something to sync these back quicker, or to minimize the number of times these are being hit if it is frequent, etc, etc.
  • For now we're just monitoring (in the UI) how often it happens, and we can make decisions based off of that... as people choose to share that data about their sessions with us. We may implement some sort of automated data reporting that people can opt into if things seem problematic, or if we think we're not getting reports on things like this but it does seem to be a problem for people.
  • At any rate, priority one is making it work properly in the long term of a game even if there is a snafu for a few seconds on a client, and priority two is minimizing or erasing the snafus.

• Various central world data is now synced from the host to clients every few seconds/
  • Among other things that were more intentional, this actually lets clients know the status of other clients pretty rapidly (aka, seeing that someone else is disconnected or connected).
  • This should have all the central items of relevance nicely synced, although we are skipping the "world history" and "journal history." For those, if they are really needed, we may just introduce a new sync stage to time-slice the processing of those. But frankly they are not likely to get out of sync.

• At this point it will be time for us to test our sync code, which is utterly untested at the moment. But the design and implementation of V1, except for actually testing it, is complete.

"Find Planet" Command

• Added a new extension in our central ArcenStrings which lets us do a case-insensitive Contains() call (for purposes of searches, etc).
  • This is useful for a variety of purposees, and lets us do partial comparisons (not just Equals()) calls) without having to cast to lowercase.

• Implemented a new command, "findp", which lets you search for planets via the chat command:
  • Formatting is like this: "cmd:findp,gear" (minus the quotes).
    • This example would search for the text "gear," and would bring up results "Geary" and "Gearworld" and "geaRson" if all three of those existed in one galaxy.
  • It tells you how many results there are, and if there is more than one result, then repeat entering of the same command (just press up and enter) will cycle you through them. It tells you that it will cycle you through them if there are more than one.
  • If you are on the planet view, it switches you to the planet view of that other planet. If you are on the galaxy map view, it centers the galaxy map on that planet.
  • Thanks to cml for the initial implementation of this as a mod.

• Added a new "Quick Tip" item, which shows up randomly on the starting screen as well as in that section in the tips section: Searching For Planets By Name
  • During the game, if you hit enter/return to bring up the chat window, and then type 'cmd:findp,yoursearchhere', you can search for planets by name. If you type part of the name of a planet, then you can hit up and enter to issue the command repeatedly and cycle through multiple hits if there are any.

Beta 2.133 Hotfixes

(Released September 4th, 2020)

This one is on the beta branch on Steam and GOG, since there were a number of problems with the prior build that prevented adequate testing. Once we have confirmation of more people able to play this new version without incident, we can move back out of the beta branch.

• Fix typo in Full Badger tooltip
  • Thanks to Breach for reporting

• Nanocaust now uses a darker border colour
  • I forget who mentioned that this looks much better

• The Nanocaust might now play a bit of defense

• Improve the way a player's Overall Power Level is calculated
  • A player can now have a high enough Overall Power Level to trigger Extragalactic War Units without allies if they are on difficulty >= 7

• Dark Spire ship line hacks no longer destroy the VG
  • Thanks to Sol for reporting.

• In Steam, the default launch style is now OpenGL, rather than Vulkan.
  • As noted, this is the preferred launch style, and more stable.
  • Thanks to TechSY730 for suggesting.

Fixes Relating To Prior Beta

• Fixed several areas where trying to check a non-existing setting (potentially because it was from a mod or expansion you don't have installed) was causing issues.
  • This is a problem dating back months to when we changed how lookups to missing settings worked, but we're only now seeming to hit it in the most recent betas.
  • Thanks to Badger and others for reporting.

• A variety of extra debugging info has been put into some of the high-level world serialization areas for our logs when problems happen.
  • Also put in a bunch more into fleets and fleet memberships for debugging purposes.

• Identified two fields from fleet memberships that were not properly translated over into the new format in the last beta build, causing all new savegames in that beta to fail to load.
  • Now that this is fixed, the broken savegames all seem to load just fine, which is always nice when that happens.
  • Thanks to StarKelp, Sol, and ArnaudB for reporting.

• The "Civilian Industries" mod by StarKelp has been updated to function in the new code framework we introduced last build.
  • All of the mods we distribute with the game are again working at the moment, to our knowledge, now. Code-style mods from other sources (forums, etc) may not be updated yet to compatibility with this latest set of builds.
  • Please note that we broke compatibility on purpose (not a typical thing), to add multiplayer functionality to mods and make sure that no mods would cause unpleasant spikes in memory usage on client machines in multiplayer.

Beta 2.132 Exterior Leviathan

(Released September 3rd, 2020)

This one is on the beta branch on Steam and GOG, because we made so many changes to some fundamentals of how the game works. These are driving at improvements for multiplayer, for the partial syncs to be able to happen, but they break code-style mods for the game (including Civilian Industries as included in this version), and it's possible that they temporarily are breaking something else since we've changed several thousand lines of code in a few dozen files and may have made an error in any one of a number of places. Some bits of multiplayer sync are a bit more hefty than we'd been thinking they might be for V1 of that, but then again we're getting more of it done upfront rather than later. We'll be able to start actually testing the sync maybe tomorrow, unless a bunch of other things are broken by our changes.

• Dyson Spheres now must kill all the guard posts on a planet before they can kill a reconquest command station. This should prevent a buildup of guard posts.
  • Thanks to TechSY730 and Khankar for reporting

• Fix some bugs with Usurpers owned by praetorians or other AI subfactions
  • Reported by GreatYng

• Try harder to make sure waves against minor factions have ships
  • Reported by GreatYng

• Fixed some minor typos
  • Reported by GreatYng

Multiplayer: Sync Correction V0.75

• New setting in the network tab of personal settings: Network Logging Includes Sync Checks
  • During gameplay there are messages sent by the network as frequently as several every 100ms, and these are one of the largest sources of data usage. This lets you see the timing and payload size of those messages, if you think that potentially they are what is slowing things down in a networked environment.

• Stripped out a lot of old code that was related to an old style of sync (aimed at desync detection), and started building in the new sync framework (aimed at detecting micro desyncs and correcting those).
  • Decided to move this to a new AIWar2NetworkSync static class in its own file, as after just getting about a third of the way through this logic, it was clear that it was pretty sizeable and overwhelming the rest of the networking code's readability.

• Added in the logic for the server keeping a list of ships that the various clients have told it are divergent.
  • The server could be getting overlapping reports from several clients, but we need only one copy of this, because we're going to err on the side of caution and tell every client about each bit of sync data that needs fixing.
  • We could always change this in the future if there seems to be a bandwidth benefit to this, but it would probably just mean extra processing on the host for more unique sends per-client. At the moment we judge that to be the resource that is more scarce.

• The logic for removing ships on a client that were not part of the last sync cycle, but also not new during that cycle, has been put in place.
  • The idea here is that these ships probably died on the server, and the client didn't know for some reason.

• The logic for having the server do send check data for 1/20th of its ships (or stacks or whatever, as the case may be), but no more than 9000 per 100ms, is now in place.
  • This is the meat of the desync detection code, and it is based around only a very few factors:
    • Health lost, shields lost, extra ships contained within (of ANY fashion), current planet, and current location.
    • We are generally making the assumption that if all of those things match between the client and the host, then the ship is probably in sync.
      • In the case of certain faction ships that do things like gather resources or whatnot, that will obviously be very incorrect, but it is correct for most combat ships.
      • For those cases where faction-specific data is potentially going to be wrong, it would be prohibitively expensive to add a bunch of checks based on that extra faction data, so we will build in separate tools for forcing sync of these non-combat factors.
      • Frankly, other things like fireteam history, which would be missing on the clients, could be synced via other methods later on if we really want to. Though that's generally only used for debugging in most cases, so seems not to be worth doing.

• The logic for clients checking for mismatches with their local data compared to the server is now in, and it sends back the requests to the server to have divergent units corrected.

• On PlanetFactions, there is now FactionIndex and PlanetIndex directly in order to make it easier to quickly find those in case they have been reassigned.
  • Additionally, their Faction and Planet member variables are now exposed as get-only properties to keep unexpected things from happening there.

• The way that squads deserialize has been updated substantially to allow for several levels of partial deserialization for various sync scenarios.
  • The sync fix code allows for normal-level same-planet sync fixes (which should hopefully look very nice with things sliding visually into new positions if positions differ).
  • The sync fix code then also allows for normal-level different-planet sync fixes, which instantly disappears the old spot and puts it somewhere new.
  • And finally, the sync fix code allows for catestrophic-level sync fixes, where the two entities aren't even the same (not matching faction or type), and it destroys the old one instantly and as invisibily as possible, and then puts in the new one where it should be.
    • This last category is for basically when the primary key ID generators are out of sync between the host and one or more clients. These will be hit with some regularity, but they should not be hit over and over for the same unit.

• The deserialization logic for "fleet memberships" has been split in such a way that we can now sync just some of the data into them (for network purposes).
  • We are not using this yet, but will once we get into the fleet sync phase of the sync cycle (outlined below).

• CalculateContentsCount() on squads now takes a required parameter IsForNetworkSyncCheck.
  • If that is true, then it ignores any data that is not on the squad (but is usually on the fleet or the "fleet membership" areas.
  • This is things like drones or transported ships, and city sockets. These will be something we sync at the fleet level.

• Network_CurrentSyncCycleNonSer has been moved off of the World_AIW2 object and into the AIWar2NetworkSync class.
  • Same with the OnServer_SquadsNeedingSyncFixes dictionary.

• The server now cycles through a series of "sync stages," where it will sync different things other than just squads.
  • This is useful for us to be able to time-slice syncing of data of different sorts... for instance fleets and fleet memberships data.
  • We actually are now going to use this for separating out the squad sync checks and the divergent squad fixes, too, since time-slicing that is good for smooth gameplay as well as also making sure that multiple clients with partially bad sync cause as few excess sends as possible (that's a ms timing thing).
    • The code for syncing the squads that were divergent is now in, as noted above.

• The data for factions, and for some of the Primary Key info, is now synchronized from the host to all of the clients every authorized network batch.
  • On average this is probably anywhere from 10kb to maybe 40kb, at most. It's easier just to keep this in sync.
  • At the moment, for ExternalData on factions and squads, there is a huge amount of inefficiency on the clients in that it is completely recreating objects every network sync right now. We are looking into various refactoring options.

• At the moment we are explicitly not syncing SpeedGroups, since those are complicated and may tend to diverge.
  • We are probably going to wind up refactoring these to be by faction, or something along those lines. In fact, there are several major refactors coming, and one of those may be that a lot of the PKIDs may become faction-specific in general.

• Turns out that adding the NetworkSyncStage as a concept was a really good idea, because we not only need to sync squads and fleets, but also planets and some data off the world, too.
  • The faction sync is happening once every sync frame, but it's possible we might back off the frequency of some of that if it becomes too harsh. Mainly for ExternalData stuff. But for now we'll see how it does.

• Planet has now had its data split out like squads and fleet memberships and factions and some other classes to have its own DeserializedIntoSelf() method.
  • Also PlanetFaction. Whew, wow. For this it doesn't sync the entities during the partial sync, of course, unlike during disk or main network sync.

"ExternalData" Updates That Affect Mods

• The ExternalData framework has gotten a few updates to allow for us to do partial-syncs like we now do with squads and such, versus always just creating something new.
  • Any mods that use ExternalData will need to be updated to use the new pattern, although it's not a huge change.

• FireteamRequiredTarget has been updated to have a DeserializeNewFrom and DeserializedIntoSelf, to ensure that what we are doing is intentional.
  • Since this is a class, not a struct, it's highly efficient for us to be able to write to an existing class rather than always replacing it, even in single-player.

• The following "external data" has been updated to be dramatically more efficient with syncs during multiplayer, employing a variety of strategies that we're using in the main game stuff:
  • WormholeInvasionDataExternal (this one is basic and simple)
  • AI_PlannedWave_Data (this one is complex in that it has a list of sub-objects and shows how to handle that properly. It even gets rid of theoretical extras on the client machines)
  • DoomData (this is really old style in terms of the data, and is not used actively, but does show a way to handle the multi-part sub-data well).
  • ExternalData_AIFactionCommon (this one is ridiculously complicated and used for a couple of factions)
    • ExtragalacticBudget and AIPChange as a part of this, and are good simple examples. These don't even have the new bool passed to their sub-objects, because both the DeserializeNewFrom and DeserializedIntoSelf() methods can both validly be used as part of one during-game sync (depends on relative list length, see code).
    • StoredAIPurchaseCostByBudgetForSpecificUnits is a good example of where it's better to just use a bit of extra GC (especially with the new time-sliced GC unity now has) instead of doing extra CPU processing to be a hair more efficient with that memory.
    • Fireteams themselves have changed from having a constuctor for deserialization to having a DeserializeNewFrom method that makes it a bit more explicit what is happening. We also pulled out the id deserialization so that we could reuse Fireteam objects as much as possible if they are being updated.
  • DysonExternalData and DysonAntagonizerExternalData (another two examples of the simplest possible cases for updating a modder might have)
  • MacrophageGlobalData, MacrophagePerTeliumExternalData, MacrophagePerSporeExternalData, and MacrophagePerHarvesterExternalData, which are also very simple examples.
  • RiskAnalyzerData is even simpler, with no sub-objects to modify.
  • AIReservesDataExt is seemingly simple, but is dodging accidentally creating two sparse lookups internally on each sync.
  • AstroTrainsGlobalExternalData, AstroTrainsPerDepotExternalData, and AstroTrainsPerTrainExternalData are all pretty straightforward, although we saved ourselves a lookup load on AstroTrainsPerDepotData.
  • AutosaveDataExt, even though it's not really even needed on the clients at all. Since it has some sub-data that is in a list, and in string format, we're just not bothering to send that to clients at all during syncs since it is a waste of bandwidth.
  • DarkSpireData, which has some complexities with the sub objects DarkSpirePerPlanet. It has special notes in the code for any similar cases with a sparse lookup and sub-objects.
  • Two "DZ" items for DLC2, which was wildly complicated to convert compared to most things. It involved sub objects DZResourceConversion, and DZUpgrade, among other nested objects. Mostly it was a good example of just following the existing principles on down, but there is one ConversionBag that was tricky enough that we're just clearing and refilling it in a somewhat GC-wasteful way rather than going into the extra complexity and perhaps incorrectness of doing it another way.
  • AntiMinorFactionWaveDataExt, ExoDataExt, and MDCExoDataExt, which thankfully were all trivial examples of the common case.
  • ExternalData_GroupTargetSorting, which has nothing formal to save or send.
  • InstigatorPerUnitDataExternal and InstigatorDataExternal, which again the simple common case.
  • SpawnAnimationData and DespawnAnimationData were very old and probably not used, but simple to upgrade.
  • MarauderOutpostRaiderSpawnData is again the simple common case.
  • MercenaryUnitExternalData and MercenaryGlobalExternalData, which had some more notable complexity. MercenaryBeaconState upgraded with WorkingHasBeenUpdatedThisDeserializationCycle to handle its situation. MercSpawnRequest just does it the inefficient way since these rarely exist, and only exist for a short period when they do. SingleMercenaryGroupState is a strange bit of data, and gets handled in a novel way. Thankfully MercenaryUnitData was a trivial case.
  • ExternalData_FactionCommon is old and strange, but was not too hard to update.
  • ExternalData_MinorFactionCommon was super old in its styling, but already was efficient enough.
  • NanocaustMgrData, NanocaustPerUnitDataExt, and LastSpawnTimestamp are all extremely simple. The old FrenzyFleet stuff is still kinda-sorta there, but never has data since it is legacy, so that shouldn't cause any GC churn.
  • FallenSpireGlobalExternalData and FallenSpirePerUnitExternalData had a few quirks, but mostly were the common case with some minor array clearing and preservation.
  • One "NP" item from DLC2 required pretty minimal updates to work with this, hooray.
  • ScourgeGlobalExternalData and ScourgePerUnitExternalData were both the common case with a tiny bit of list/dict management.
  • Two "ZA" items from DLC2 are mostly the common case but with some collection management added in.
  • Two "ZM" items from DLC2 are also mostly the common case with a bit of collection management.
  • HRFDataExt and MarauderDataExt both were also mostly the common case with a bit of list management. There is a question mark about the regiments aimed at planets, on this and a few other locations, but they are at worst going to be inappropriately blanked out over and over again on clients in MP, which we can fix later.

• In general, any of the usages of IArcenExternalDataPatternImplementation in mods should be updated to follow the new examples in the main code. The changes are not severe.
  • The most important changes are the ones above, relating to SerializeExternalData() and DeserializeExternalData() taking an extra parameter, and then each handling a graceful DeserializedIntoSelf() sub-method (and choosing if there's some data not to sync across during gameplay if required). But beyond that, there are some more changes:
  • RelatedParentTypeName has been removed, as it was never used. Same with GetShouldInitializeOn.
    • During InitializeData, the object ParentObject is being cast into an appropriate-type Parent object for general usage, (aka GameEntity_Squad, Faction, whatever) that people can then later use. If the type is wrong, it will just come across as null, and we have it complaining if that's the case. You can do the same or omit that part if you don't care.

Third Off-By-Default Mod: More Starting Options By ArnaudB

• The More Starting Options mod by ArnaudB is now included with the game by default (with permission), but disabled.
  • Requires The Spire Rises expansion (DLC1).
  • Pick between seven starting fleets and twenty starting battlestations for your custom games. These choices are balanced for regular play and are acceptable to use for achievements.
  • The eight starting fleets will offer you a more varied combination of ships so you may explore more varied combinations of technologies and playstyles than those incentivised by the default options.
  • The twenty starting battlestations will let you play and test every of the many turrets brought by The Spire Arise DLC, which are otherwise completely absent from the default starting options. You’ll find multiple combinations for both offensive and defensive playstyles, balanced for your enjoyment and giving you hints as to what synergies you could make with a few turrets or ships.
  • This mod is off by default as to avoid paralyzing the player by offering too many choices.
  • You can activate this as a new player unsatisfied with the default options, or as a veteran wishing to vary your experience of the game.

Mod: Civilian Industries Updates

• Civilian Industries 0.6.4
  • Full update notes may be found on the forums: https://forums.arcengames.com/ai-war-ii-modding/civilian-industries/msg222774/#msg222774
    • A large number of code optimizations, directly mostly at performance and multiplayer readiness.
    • An overall nerf to Mobile Ship counts, but a simplified formula will allow easier future modification.
    • A rework to the Barracks. Now scales with Mark Level, Command Tech, and can even let you get more Protectors at mark 7.
    • Buffs to many Protectors. Balance for them is still being worked on.
    • Resources renamed across the board, and structures now state what technology works for what resource.
    • A notification will now show when a Raid is about to fire against your civilians, thanks to Chris' recent changes on the notifications front.

Version 2.130 Redux: Dutch Cities and Softer Eyeballs

(Released August 31st, 2020)

We accidentally forgot to increment the version number in game. So this would have been 2.131, but is just called 2.130 in the game. Sorry about that, this is perhaps a first?

• New planet naming style added, with over 4000 names in it: Cities (Dutch)!
  • Most Dutch cities, villages, and hamlets shorter than 10 characters; also includes many local spelling variations. Created by GreatYng.
  • Thanks to GreatYng for creating!

• Added a new xml tag, external_invulnerability_unit_required_count, which works with external_invulnerability_unit_type or external_invulnerability_unit_tag.
  • This basically lets us say that a unit of whatever sort is still protected by others (such as guard posts), but only if there are more than the number specified.
  • The default number is 0, which means the previous behavior of "any guard posts present make this invulnerable."
  • For AI Eyes, the number is now 3, which means that if there are not at least 4 guard posts remaining on a planet, you can go kill the eye.
    • This is pretty huge, because you can now kill just part of the planet and then kill the eye before going to kill everything else.
  • If we need to make further adjustments to the Eyes based on the relative ratios of strength of them versus enemies, we can do that in the future, but for now this is a solid start.
  • Thanks to Lord Of Nothing and CRCGamer for reporting.

• Further balance adjustment: all AI Eyes are no longer Mark 7 at minimum. They are now Mark 5 at minimum, and if their planet would have a higher mark (6 or 7) then they will instead have those higher marks.
  • This should help to make a bit more variance with them, and eyes to be a bit less of a pain in general.

Bugfixes

• Fixed HRF Nucleophilic Defenders as well as other defender types not being loadable into transports. Additionally they now count as Frigates and benefit from their hull/shield/damage settings.
  • Thanks to Darkshade for reporting and -NR- SirLimbo for fixing.

• Fixed a couple of issues relating to getMostCommonPlanetForRegiment() with fire teams, which were likely leading to who knows what other errors.
  • The PlanetsForRegiment Dictionary was declared as static, and so all of the regiments were trying to use the same dictionary at once, leading to exceptions as well as probably wrong results. Exactly how much chaos this could have caused is unclear, but with some of the LRP threads stalling out this is certainly not going to help (fingers crossed it was the whole thing).
  • Now if getMostCommonPlanetForRegiment() is called again on the same regiment before it finishes, it will throw the error "BUG: called getMostCommonPlanetForRegiment() again for a fireteam regiment before the last call finished. Will break both calls." Hopefully we don't have people getting this, but if we do we need to know about it to fix that. In that case, the workingPlanetsForRegiment dictionary can't work as a member variable at all.
  • Additionally, if there's an exception in getMostCommonPlanetForRegiment() it will now throw an exception but then exit more gracefully, letting whatever calling code keep doing its thing.
  • Thanks to CRCGamer and TechSY730 for reporting.

• A few versions back, we made some changes to how the icons are rendered on the galaxy map, including planet icons. We made them keep their orientation better, but one of the things that was an unfortunate side effect of this is that the colliders on the lines between planets were often taking precedence over the actual planet or the ships at the planet. That is now fixed.
  • Thanks to TechSY730 and CRCGamer for reporting.

• Heavily reworked how Journal_TooMuchAIP calculates both how many planets you have paid AIP for, as well as how many planets are neutered. It was triggering the journal entry far too early, previously.
  • Please see the code or here for details: https://bugtracker.arcengames.com/view.php?id=23609
  • Thanks to Sigma7 for reporting, and Badger for helping figure out what was wrong.

• Fixed old wording on the AI Eyes that still said "when there are enough fleets present on this planet" to instead read "when it is sufficiently outnumbered on this planet."

Version 2.130 Civilian Industries

(Released August 29th, 2020)

• For determining what your performance is like, the game now uses only one second of data instead of ten seconds.
  • This makes the sim speed average a lot more responsive and natural-feeling.

• The "suppress upgrade prompt" setting and keybind now both work on fleet upgrades and not just science tab upgrades.
  • There is some extra info in the tooltips and popups to reflect this, as well.
  • Thanks to Fritz1776 for suggesting.

Second Off-By-Default Mod: Civilian Industries By StarKelp

• The Civilian Industries mod by StarKelp is now included with the game by default (with permission), but disabled.
  • This lets people simply enable the mod by clicking a button in the settings menu, rather than having to find and download the mod from elsewhere.
  • StarKelp has been a volunteer contributor to the main codebase off and on for a while, and this mod contains both code and xml.
  • Full description of the mod is included in the tooltips in the game, but also here:
    • Every empire is only as strong as the people within.
      • Protect your people, and they will help your empire flourish.
      • From collecting and processing various resource around the galaxy to bolster the economy, to taking up arms on their own to aid your survival, you'll find their help invaluable in the long term.
      • As they flourish, so do you.
    • Adds in the Civilian Industry faction, a potent defensive faction that can be allied to either a player, or a faction team.
      • They will expand alongside their respective ally, and produce a large number of defensive forces to aid their team.
      • They gain a new resource type per planet they expand to, and will work diligently to spread these resources throughout their empire.
    • When allied to the player; you will find your command station and battlestations have access to a few new structures that you can invest in to greatly incresae the power of your civilian economy.
    • Please check Galaxy Settings for many tweaks in regards to how they behave.

Bugfixes

• Fixed a bug in GetIsWithinRangeOf() that could happen in the selection logic (and probably other places) if a background thread marked a unit as deleted as it was doing the range check.
  • Thanks to CRCGamer for reporting.

• It is now possible that your logs will wind up having the message "BUG: overriding speed limit for group X to [a negative number below -1]. Overflow? Setting to -1 instead."
  • If this is frequent, it may slow down your game, but the stack traces will let us know where we are going wrong and causing this to happen.
  • We fixed one area where MAYBE this could be caused, but probably that was not possible in the place we fixed it. Most likely the problem is in something that is calling GameCommand_CreateSpeedGroup with a command.RelatedIntegers[0] that is greater than the 32767.
    • With that in mind, anything that is larger than that being passed into that method is now automatically corrected down to 32766. This may lead to some turbo speed groups, not sure, but at least they won't be erroring... and that IS what the calling code is asking for, so maybe it's intentional?
  • At the same time, in some other places in the code where it was checking to apply the speed only if it was "not -1", it is now checking to apply the speed limit if it is "greater than 0." This seems more like the correct intent, and hopefully won't break anything.
  • And beyond that, prior to serialization, any speed groups with an OverrideSpeedLimit less than -1 will automatically just set their value to -1 silently and thus avoid popup errors that could be really annoying in savegames for folks. This has been an issue on and off since July 24th, apparently.
  • Thanks to Lord Of Nothing (twice over), TechSY730, GreatYng, and Apthorpe for reporting.

• Add a new time_must_have_been_on_planet_before_transforming_if_outnumbered_or_outnumbering, for use with transform_when_outnumbered and transform_when_outnumbering.
  • This defaults to 0 seconds, which has been the behavior in the past.
  • But now for all of the ships (mostly AI Eyes, but also some guardians) that use transform_when_outnumbering, it now has a value of 2.
  • For those that use transform_when_outnumbered, it has a much higher value of 30, to keep them from downgrading their alertedness too easily.
  • The AI eyes will still switch between alerted and unalerted status every so often in a general battle, since their alerted status increases the balance of battle in their favor so much, but now it won't be a constant thing and ships can continue targeting them in a reasonable fashion.
  • Thanks to whakomatic, Ovalcircle, MasterGeese, Gdrk, and GreatYng for reporting.

• Additionally, previously anything that was transforming via one of those two methods was having its health and shields and cloaking points all reset to full.
  • Now, however, when transformed via these two cases ONLY, it keeps the health and shields and cloaking point losses after transformation.
  • This makes it so that battles with AI eyes are actually something you can win, even as they cycle back and forth between statuses. And in general this just makes sense, and is important on the dire guardians that use this same sort of mechanic, too.
  • Thanks to whakomatic, Ovalcircle, MasterGeese, Gdrk, and GreatYng for reporting.

Multiplayer Work

• GameCommandTypes are now serialized by index rather than name, which chops off the vast bulk of size from most GameCommands, leading to even snappier networking.
  • Honestly this is a very small boost in most cases, but this also took like two minutes to add.

• The network traffic logs have all been updated to be able to actually report how many bytes were read in for each message that the host or clients receives. Before it was saying ?? because we were checking prior to doing the reading.
  • Note that depending on the networking platform in use, there is sometimes extra buffer data that was read from the socket, but if it wasn't part of the game, we just ignore all that. This is also ignoring packet header sizes, so we're just talking payloads here (but that's consistent with the send records, too.

• In the escape menu, on clients and the host, it now has a new Multiplayer Stats section when you are in multiplayer mode only.
  • At the moment, this shows how many messages have been sent or received in total, and how many bytes/kb/mb have been sent.
    • These all apply to the current session only. So if you are the host, it is the stats since you loaded up the server and people started connecting. If clients come and go, it's including all of that. If you're the client, it's the amount since the last time you connected; if you disconnect and then come back, it starts over.
  • This is pretty important information in terms of figuring out just how much data is actually being sent back and forth, and how it changes during gameplay, so that we can react to any unexpected spikes if there are any.

Version 2.129 No Shrooms For Ships

(Released August 28th, 2020)

• Nanocaust now prioritize holding planets adjacent to their homeworld
  • Thanks to crawlers for reporting

• The nanocaust units now prefer to spawn a bit further from their centers; it looked kinda weird when a giant Nanocaust ship would spawn right on top of a nanobot center.

• Added a lot of extra instrumentation to the deserialization and serialization of factions, since there seems to be some sort of error in there with savegames from specific versions.
  • Essentially with this sort of thing on, we can now manually look through the deserialization log and see if we notice where field data stops making sense. It was obvious that it was getting something like "I am hostile to 20 thousand factions," which is nonsense data, but it was not clear where the data was going wrong prior to that.
  • From this, we could then see that most of the data reads in the player faction seemed reasonable, but the faction right after them was immediately messed up. This is after a lengthy code review turned up nothing wrong, so it's always possible that it's some sort of edge case that has been around for longer than we thought.
  • Added in code to make it so that if an exception is thrown while deserializing a faction, we will now repeat that error better and stop doing further deserialization attempts. It turns out that was what was happening, and a more careful reading of the log could have saved about an hour and a half of work (oh well, things in the future will be easier to debug).
  • Thanks to Lord Of Nothing for a very challenging save that led to these changes.

• Fixed an issue with some savegames that had hacked dyson or dark spire ships, where the HackToGrantShipLine_DontDestroyTarget hack had been removed and split into several versions a few versions back.
  • We now have added HackToGrantShipLine_DontDestroyTarget back in, in a deprecated fashion, so that old savegames with this data in it can load.
  • Thanks to Lord Of Nothing for the save that let us find this.

• Fixed a bug in the prior build where the "flair" overlays for ships in the main view were either black (some machines) or a rotating color set of insanity (others).
  • One line was accidentally omitted in the new build of the shader that we made.
  • We also updated some calling code to the shader in a way that is harmless but not actually needed for this fix.
  • Thanks to Strategic Sage, Isiel, CRCGamer, and TechSY730 for reporting.

• The "Icons Disappear When Camera Lower Than" setting in the main camera window has been updated as follows to have this note
  • Note: this setting does nothing (icons always draw) if you have 'Skip Drawing Ship Models' turned on in the Performance section.
  • (Also, this is how it actually now works).

• If a player is already on a planet viewing that planet, and a further command was sent to view that planet again (as if they were not already there), then funky stuff happened in various places, including the wormholes getting half-initialized and not drawing properly in scale.
  • To solve this, it simply doesn't do anything if you try to re-open your view to the existing planet you are already on.
  • Thanks to StarKelp for reporting.

Version 2.128 Don't Lie About Distant Future Waves, AI

(Released August 27th, 2020)

• Fix a bug where the scourge would sometimes suicide into multiple layers of defensive planets in the (foolish) hope of hitting the weak target on the other side.
  • This is an intelligence improvement for all factions using fireteams (Hunter, Warden, Nanocaust, Scourge, etc etc).
  • Thanks to Crawlers for reporting

• Add some defensive code to the Fallen Spire.

• Put in extra logging for the SpeedGroup serialization and deserialization, to figure out where there are occasional exceptions being logged.
  • Thanks to TechSY730 for reporting.

• Sniper Energy Wave on the Dark Spire Eidolon has been renamed to just be Energy Wave, since this is not an infinite-range weapon.
  • Thanks to crawlers for reporting.

• DoForKnownWavesAgainstHumanHomeworlds in WaveUtils has been renamed to DoForKnownWavesAgainstHumanWorlds, which is what is accurate.

• A whole bunch of extra array indexing calls into QueuedWaves have been made more efficient in the AI HandleWavesForAI() method.

Interface Improvements

• The planet's tooltip has been updated to show the owner on the same line as the planet name, but positioned way over to the right.
  • This keeps things pretty condensed, but avoids some clashes of words where it seemed to be saying something like "Permanently Watched Owner:" as one phrase.

• Previously, there was a chat message going out as soon as a wave was queued, but before it would become visible to the player in the actual interface (based on their chosen galaxy map settings for how much warning to get).
  • Now it only shows the actual chat message from the AI as the wave is being revealed to players.
  • This also fixes an issue where it was for some reason actually giving the wrong planet name that it was going to attack in some cases, since the announcement came before it finished changing its mind about the real target it wanted to hit.
    • This latter one may have been more common in multiplayer, but it's hard to be sure. At any rate, it just tells you real information now.

• The second chat message that goes out (with the audio taunt) when waves are launched now also happens when they are revealed, and no longer duplicates when in multiplayer mode.

Mod Compatibility Improvements

• There is a new DoPerSecondNonSimNotificationUpdates_OnBackgroundNonSimThread_NonBlocking() method that gets called on each faction each frame, and which takes in a note which says if it is the first faction of its type this frame or not.
  • This lets us move notifications for factions into their own files, which makes notifications modding-compatible.
  • There are some other supporting methods that have been added to handle this, but understanding them is not required for using this,.
  • It is worth noting that ALL of the existing DoPerSecondNonSimNotificationUpdates_OnBackgroundNonSimThread_NonBlocking() calls do happen in a loop prior to the new DoPerSecondNonSimNotificationUpdates_OnBackgroundNonSimThread_NonBlocking() being called.
  • As an interesting side effect, this actually makes it so that notifications will now work for people in spectator mode in multiplayer for the first time.
  • The nanocaust data has been moved out of the human file as an example of how to do the others.
    • Same for antagonized dyson spheres, and the dark spire. It would be great to get the other factions moved over to their own files for the sake of cleanliness, but functionally there will be no difference for our things in the main game and expansions now.
  • Thanks to StarKelp for requesting the ability to add notifications in mods.

• Added a new method DoPerSecondLogic_Stage0Clearing_OnMainThreadAndPartOfSim_OncePerFactionTypeEvenForFactionsNotInGame().
  • This is called on the GetDefaultImplementationForLobbyOrClearingOnly() from the faction TYPES, which means it happens even for the factions not in a game. Great and easy place to clean up, and we know it happens only once per cycle per faction type, and we also know that it happens even if a faction is not in the current campaign.
  • This is useful for factions that had some data in a prior campaign, but the player loaded a new savegame and they are not in this one.
  • All of the various data from these that was previously in the Human class and in DoPerSecondLogic_Stage1Clearing_OnMainThreadAndPartOfSim() has been moved to the new stage0 for each of the specific class types, which again is more mod-friendly and should prevent the issue of certain mods being unable to clean themselves up between campaign loads.

Multiplayer Work

• When the host is waiting on a client that is behind, it now shows a message saying which clients are behind and being waited on.

Bugfixing

• The galaxy map spacebox background is no longer serialized into savegames. Instead it is deterministically calculated based on the map seed.
  • This makes very little difference for single player, although you will notice that reloading the game or going back and forth to the map, or hitting regenerate map on the lobby no longer changes the space background.
  • But for multiplayer, this keeps things consistent between all players.
  • For existing savegames, you will notice that the spaceboxes become something new and stay that.

• It turns out that the planet spaceboxes were not being serialized at all in savegames before, but their rotations were.
  • This was leading to very different planet background views in multiplayer, to a very distracting degree.
  • This has been made deterministic, and we no longer store even the rotations in savegames.
  • For single player there is no real change except that the savegames are a tiny bit smaller.
  • For existing savegames, you will notice that the spaceboxes become something new and stay that.

• Fixed inconsistent sorting between runs of the game (and between differnt clients) for rows in the following tables:
  • AIGuardPostAndCommandPlacerTable, FleetDesignTemplateTable, AIShipGroupCategoryTable, and SpaceboxDefinitionTable.
  • These led to various MP inconsistencies, including the spaceboxes still being divergent on planets despite the math lining up.

• Fixed an issue where the home planets of other players in multiplayer games just looked like normal planets instead of homeworlds.

• There were a bunch of cases where human factions were still named wrong during multiplayer (and differently on the host and client), saying the wrong player's name or Unknown Player in various cases. Fixed all of those.
  • Additionally, in the case of multiple players owning a faction, if they did not choose an empire name, it now says all of their names together (but no more than three) as the faction name.
  • If there are more than three players controlling a single faction, that gets too wordy, so it just says "and X more".
  • When loading the single player games of other players, you will now see their chosen account names (from in AI War 2, not Steam or whatever) instead of your personal profile name.

Seeing Where Other Players (Or Spectators) Are Looking

• The way that the gimbal icons decide which diffuse color to use is now more efficient on the GPU (this is minor, but hey).

• It is now possible to have a secondary diffuse color for the flair (really used for selection circles now), which is multiplied by the main "other diffuse color" of the icon.
  • This lets us do things like change the opacity or color of the selection ring around a ship or planet.

• When you are viewing a planet, the game's way of showing the selection circle around it is the same as it always has been.
  • However, when someone else in multiplayer is viewing a planet other than the one you are viewing, it now shows a dimmer and more-transparent selection circle to show you that someone else is looking there.
  • If someone else is looking at the same planet you are looking at, then it will show the normal bright selection circle in a faint green rather than white.
  • This is something that is deceptively important, because we need to make sure that we know which planets are "tier 1" planets on clients and the host, which lets us verify that ViewedByPlayerAccounts_DuringGame is being set properly on all machines (it is). This keeps the game in the sync, which we can now tell is working properly.
  • But also this is just a helpful indicator for players to help with coordination. ("Check out this planet I am looking at in the northeast" is so much easier to say than trying to direct them to a name).

• As an added help, in the galaxy map view when you hover over a planet, it tells you which other players are viewing a planet. This may be players in exclusive control of a faction, or one of several player players controlling a faction, or just a spectator, so it shows you their name as it would appear in chat.
  • For games with more than two players, this is fairly important for purposes of clarity. In a two player game it's easier to just infer what is going on.

• When a client is disconnected, other clients and the host can still see what planet that client WAS looking at, and that planet still counts as a Tier 1 planet for processing, but hovering over it will show the name of the player with (Disconnected) behind it on the host.
  • Presently clients cannot see the status of other clients.

Disconnection And Re-connection Verified

• We can now verify that after a client disconnects in a disorderly way, the host notices a bit later and then continues on without them (they can choose to pause or not).
  • We can FURTHER verify that the client can reconnect to that running game with only a few-seconds interruption to other players who are still going. Even if the client had crashed and had to restart the game or their computer, they are able to get right back in there.
  • The original AI War did not remotely have any of these capabilities, but this works on all three networking platforms.

2.127 Cranky AI Exceptions

(Released August 26th, 2020)

This was originally released as a beta on the 26th, but then changed to a main branch release on the morning of the 27th.

• The Dark Spire now responds properly to being hacked by being cranky
  • Thanks to crawlers for the bug report

• Fix a bug where the AI was incorrectly combining some Hostile To All factions when spawning extragalactic war units.
  • This was leading to more extragalactic war units than one would want.

Exception Fixes

• Added new debugging instrumentation into ImportIntoDynamicTable_XMLDirectory, so that we can find errors that may happen in there from time to time on some machines.
  • Also put in some code that should make it load faster for mods and expansions that don't have certain folders.
  • Thanks to Ovalcircle for the report of something mysterious in this area on his system.

• Also expanded the debugging instrumentation on ReadXmlFileIntoBatchProcessingLists, same reason.
  • Ditto DoForXmlFolders.
  • And finally, ditto ActuallyImportAllOfTheBatchedRows.
  • Thanks to Ovalcircle for the report of something mysterious in this area on his system.

• ArcenXMLElement.GetNewFor has had its debugging instrumentation greatly expanded, and also has been split into one version that expects a data table, and a new GetNewForNonTable variant that does not expect a table.
  • The idea here is to catch various forms of errors early.
  • This in turn caused a cascade of other changes to other methods needing variants of each other along these same lines, and having better instrumentation.
  • Thanks to Ovalcircle for the report of something mysterious in this area on his system.

• Fixed another rare race condition that could happen with the vis code, this time in SetFormationPositionFromSquad.
  • Thanks to NRSirLimbo for reporting.

• If for whatever reason the game was not able to proeprly write your settings data, it was still overwriting your settings file previously. It will no longer do that.
  • Additionally, a lot of extra instrumentation has been put into GameSettting.SerializeGraphicsTo and GameSettting.SerializeTo, so that we will know better what is wrong if it is trying to save improperly. These should also be more resistant to saving incorrectly in the first place.
  • Thanks to Isiel for reporting.

• If a savegame has somehow been made that has a blank savegame in it, then it will try to load it with the info of the current version of the game, with a warning in the log that this may fail very badly.
  • Thanks to Isiel for reporting, and providing a save that was probably made while the game was in an invalid state.

• Fixed a case where trying to load certain saves could fail silently as far as the interface was concerned. Now it actually shows you a proper error message.
  • And then fixed a whole bunch of cases where certain code paths on almost-certainly-broken saves would return very genric and unhelpful errors when you tried to load them.
  • This doesn't really help us actually load the savegames in question, but tells us more about what is wrong with them.
  • Thanks to Isiel for reporting, and providing a save that was probably made while the game was in an invalid state.

• Suppressed a harmless popup that could happen in ReactToShotHittingSquad during cross-thread race conditions. Now it just fixes the data and moves on.
  • Thanks to CRCGamer for reporting.

Beta 2.126 Dark Spire, Eyes, and Fixes

(Released August 22nd, 2020)

To play this version, please be sure to choose the current_beta branch on Steam. The central game loop changes we made in the prior version did not blow up, but a few other things did, so we're glad we went the beta route. If for some reason this beta has problems but the prior one worked for you, we have temporarily made the most_recent_stable_beta active and pointing to 2.124. But this version should actually be more stable (knock on wood). We'll be back out of beta by mid-week the week of the 23rd.

• Fix a bug with DS Conquest VGs and the new hack
  • Thanks to gigastar for reporting

• Fix a bug with the Full Metal notification
  • Thanks to a screenshot from Oval for making me aware of this

• The AI should now attempt to play voice lines for Exogalactic Strikeforces, and when you spot the Nanocaust.

• The DS is no longer allowed to spawn Loci on planets with Dark Spire Wards
  • Thanks to crawlers for reporting

• Make the Dark Spire more aggressive about spawning Loci
  • Thanks to crawlers for suggesting

• The player can now choose the starting armory for player-allied scourge
  • Thanks to crawlers for suggesting

• Improve the hovertext for some scourge units just a bit.

• Fix a bug where new games in the beta weren't starting correctly
  • Thanks to Keith and Tynendir for reporting

• Capturing an AI homeworld no longer generates AIP. This only applies to games creates on 2.125 or above.
  • Thanks to GreatYng for suggesting

• AI Eyes now trigger based on whether you have more Strength on a planet, not off the number of fleets
  • This is not necessarily the final state for Eyes, but having them trigger off of 'number of fleets' didn't make any sense anymore. This is a functional stopgap until someone has a better idea
  • Thanks to a number of people for discussing, including Flypaste, Asteroid, Democracy, GreatYng and others

• Fixed typo in multiplayer message.
  • Thanks to fwiffoforce for reporting.

Beta 2.124 HRF Pacekeeping

(Released August 18th, 2020)

To play this version, please be sure to choose the current_beta branch on Steam. If this isn't blowing up in major ways, we'll move it back to the main branch within a few days. If it is blowing up a lot, then it may be into next week before we come back from beta. We changed too many fundamental things with the game loop to just release this with no beta, even though we were careful.

• Minor rework to minor faction hacks. Its now more expensive (in line with the previous hacks). The description no longer mentions an AI response. The DS and Dyson units are now upgradable, but not very much. These are intended to be 'cool little tidbits' but nothing to really swing the game
  • Thanks to GreatYng and Lord of Nothing for bug reports and comments

• Add a 'Full Metal' indicator in the metal bar
  • Thanks to Nyarlathotep. Iä! Shub-Niggurath!

• The hovertext prompt that describes when swapping ship lines will scrap your units has been modified to describe the current behaviour correctly ("Your ships are scrapped if they are loaded, or the flagships are on different planets").
  • Thanks to StrategicSage and Ubifan for reporting

HRF Changes

• The HRF ships now have some unique names.

• The HRF now create a structure at game start time that you can hack for a new ship line (using the backported DLC2 tech) and for 1K science

• Fix a longstanding bug where the Human Resistance Fighters would only say their 'defeat' voice lines, not their 'victory' voice lines.
  • Thanks to Nyarlathotep. Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn.

• The HRF should now pack a bigger punch when they arrive, but they are a bit less likely to show up.

Bugfixes And Tweaks

• Some minor improvements to the ability of vulnerable Vengeance Generators to be killed
  • Hopefully this helps. Thanks to Lord of Nothing for reporting, and Puffin for suggesting the improvements

• Better enforce the 'can't repair ImmuneToRepairs' unit interactions. Still a bit weird.
  • Thanks to MasterGeese for reporting

• SerializeByInternalName and SerializeByIndex both now accept FieldName as an optional parameter so that we can see what is happening within them from a serialization logging standpoint.

• A bunch of extra serialization logging has been added into the world and the faction configuration objects, to make finding errors there easier.
  • However, it turns out that there were no public errors on this, just an error in a working copy on one developer machine that had a save from between two updates from that same developer.

Milestone: Multiplayer Pacekeeping Functional

• In the event of a campaign switching which player is the host (this probably means one player manually emailing the game to another), the game now tries to do a better job of handling that.
  • In the end this should work fine, but we haven't bothered testing this yet as this is probably kind of an edge case compared to most other things we're working on right now.

• All of the "frame number" information for sim frames is now just serialized on the network and never to disk.
  • This information is really only useful within a single run of a campaign (aka after loading it off disk and playing until you exit to the main menu or the OS).
  • Even if you go just back to the main menu and load back into the game directly after, there's no reason not to start those numbers off fresh again.
  • We've thus switched to a number format for serializing these that will "only" allow for 160 hours of continuous realtime play in a given campaign, without exits to the main menu. This is the definition of excessive, and could always be changed in the future if it was needed.

• A variety of variables that really are only for the network, and serialized but not to disk, now have the prefix Network_ so that we can tell those apart with ease.

• In multiplayer, the basics of "host waiting for clients to catch up" is now in place. This prevents the strange lag that clients were seeing, which was happening because they were back in time.
  • However, also in multiplayer, unlike the original AI War, it will not definitely wait around for players who are missing. If those players are not connected, then you can unpause and play without them until they arrive. Only once they are present do you have to wait for them to sync up.
    • If this latter choice turns out to be problematic for some reason, we can always add a toggle for that or change it. But it seems friendly. Chris remembers nights where 1 person out of 4 player game couldn't make it, so just keeping on playing without them would have been so nice versus having to go in and change the status of their account before playing.

• Various of our "simulation profile" variables that were set a loooong time ago in a different way for multiplayer and single player were making multi-player feel sluggish. We've tightened up this some in order to feel better under a pretty general set of networking conditions.
  • We may need to revisit this in the future (during alpha or beta) to make these things that people can configure if they are in a situation of, for instance, high packet loss or extremely high or low pings, to get the best possible experience at any given time.

• Various other changes have been made in order to make the client send over commands to the host as soon as possible, and then to make sure that the host causes those commadns to be executed on the FIRST simulation frame of the next batch, not the last frame of that next batch.
  • All in all, the game now stays in time sync, across Valve's server relays and back, and things are snappy and responsive on both the client and the host. There are moments of delay that cause a brief "waiting" message to flash up for maybe a quarter of a second every so often, but it's really a minor thing and is something that we can improve with tuning in the future.
  • The overall functionality is there to the point that if background threads and floating point math were not knocking things out of sync, this would be a perfectly playable alpha. As it stands, it's quite playable for a few minutes at a time, at least on Chris's two test machines, without things becoming noticeably off.
  • It's also worth noting that the simulation speed on the host shows up as being something like 80% right after starting the game, which is misleading because it had one very long frame where it was waiting on the client to connect and sync up. That will be fixed in the future, but it self-resolves the visual artifact within something like 20 seconds of starting, anyhow.

Version 2.122 Empire Names

(Released August 17th, 2020)

• *Makeshift Drones now once again self-destruct when attacking instead of only self-attritioning slowly.
  • Thanks to NR SirLimbo for fixing.

• The lobby now once again allows for you to zoom in and pan around, since some maps can be larger than you can comfortably see on a single screen.
  • However, every time you generate a new map, it will default to showing you the entire map completely zoomed out.
  • It also no longer EVER allows edge-panning in the lobby, as just general usage of the lobby will wind up with you accidentally panning the lobby around if that is in place.
  • Thanks to CRCGamer for confirming our suspicion that we'd need to do this.

• Fixed ANOTHER bug with the lines being offset from the planets in the galaxy map, this time when you were zoomed in. The more zoomed in you were, the more off they got.
  • This bug was caused by our latest fix to the lines being off when you were zoomed out. Now it should work at all zoom levels. This was an annoying one, to be sure, but we're glad to have it finally lining up all the time.

• Fix a bug where player-allied scourge weren't building properly

Multiplayer Progress

• Added a new Human Empire Name setting on player factions:
  • If left blank, will be the name of the first player controlling this faction. If you want your empire to have a different name than your personal name, you can do that here. If multiple players are sharing a faction, then they can give it a name that represents their shared interest in it.
  • This can be edited during the game (like player colors can be), or in the lobby.

• When a new player joins as a client to a game, it now creates a slot for them and puts them in charge of it instead of them defaulting to spectator mode.
  • It assigns their colors properly, and all of that sort of thing, too.

• Player accounts beyond the first are now removable, which lets players go back into spectator mode, or (later) control a single faction via multiple players helping.
  • This makes the default case (everybody has their own faction) set itself up by default, but then lets players back things out into the less-usual cases (shared factions, spectator mode, etc).

• Players beyond the first are now auto-assigned a starting planet somewhat at random, letting them choose something on their own later.
  • A whole lot of logic that was very not-MP-safe is now MP-safe.

• When loading a savegame or quickstart into the lobby, or loading the last settings for the lobby, it strips out all human players beyond the first.
  • This is consistent with it then recreating player accounts as those come in, too.

Version 2.121 Savegame Hotfix

(Released August 17th, 2020)

• Fixed a deserialization bug in the prior version that made any games saved with a dyson sphere in them not able to be loaded if they were saved in the prior version.
  • The nature of the issue was such that those savegames will now load without issue, thankfully.
  • Thanks to Lord Of Nothing, CRCGamer, fatcat__25, GreatYng, Djri123, Djri123, mekolab, and goz for reporting.

Version 2.120 Improved Lobby Map Experience

(Released August 14th, 2020)

• Fix a bug where sometimes the AI's ships wouldn't be allowed to retreat
  • Thanks to Sigma7 for reporting

• Hopefully fixed a problem where sometimes allied scourge could kill AI Command Stations immediately after a game load
  • Thanks to GreatYng and ZeusAlmighty for reporting

• Fix a typo with the Spire and Marauder Beacons
  • Thanks to Apthorpe and Ovalcircle for reporting

• Fix a bug where hovering the Hacking resource bar entry would give a negative number during the superterminal hack
  • Thanks to GreatYng for reporting

• Improve the Hacking Resource bar hovertext for the case where you have multiple hackers. Note it still doesn't show hacks against planets.
  • Thanks to Lord of Nothing for reporting

• The MDC hovertext no longer mentions Exos
  • Thanks to Strategic Sage for reporting

• Exogalactic Strikeforce speed updates:
  • Previously, Exos had travelled at 1800 speed (as of a few patches ago. Before that they didn't have a consistent speed).
  • Now Exos travel at "Average speed of their units + 20%" or "800", whichever is higher. This should feel less like they are speed-racing to your homeworld. For speed-context, 800 is the speed of a Parasite.
    • Thanks to GreatYng and Lord of Nothing for discussion

• Fix a typo in the hovertext for larger numbers of planets
  • Thanks to Lord of Nothing for reporting

• Remove a mention of Fleet Exp in a tooltip
  • Thanks to Lord of Nothing for reporting.

• Take safest path through galaxy now uses the modifier key alt, while take shortest path now uses the hotkey ctrl.
  • The prior hotkeys of X any Y conflicted with other functions and changed your unit stances or sidebar status, etc.
  • Thanks to Sigma7 for reporting.

Ability To Reset Camera View To Default

• New hotkey in the camera section: Reset Camera Rotation And Tilt
  • Default ctrl+R.
  • Press this button combo to reset the camera's rotation and tilt to the default values.
  • Thanks to dv, Ovalcircle, Bobtree, and many others for requesting.

DLC2 Backports

• The Dyson Sphere Golem can now be hacked like an ARS for one of a small selection of Dyson ships. You only can use this hack once.

• The Dark Spire Vengeance Generators can now be hacked like an ARS as well

• This is a backport of a DLC2 feature that allows for a minor faction to have a structure which can be hacked like an ARS for unique ships. This is usable by modders with only some XML.

• Approximate usage: in the minor faction GameEntity XML, for the ARS-equivalent, set the following
  • Assign it the HackForShipType tag
  • Make it eligible for the HackToGrantShipLine_DontDestroyTarget or HackToGrantShipLine hacks
  • Then add the following additional fields
    • grants_stuff_to_be_added_to_player_fleets="true"
    • grants_stuff_to_be_added_to_player_fleets_strikecraft_options="X"
    • grants_stuff_to_be_added_to_player_fleets_frigate_options="Y"

• Then for the ships you want to have available, set one of these
  • <fleet_membership name="AddedToFleet_MinorFaction" ship_cap_group="Strike" weight="100" min="30" max="35" faction="factionName"/>
  • <fleet_membership name="AddedToFleet_MinorFaction" ship_cap_group="Frigate" weight="100" min="1" max="1" faction="factionName"/>
    • Note the FactionName in the fleet_membership must match the name of the faction (I believe it checks the TracingName for the faction, which is set in C#)

Dark Spire Improvements

• There's now a preliminary Journal Entry for the Dark Spire when you first see them.

• There's now a Journal Entry for when the Dark Spire enter conquest mode (ie from the Fallen Spire campaign).
  • This explains what Conquest Mode is, and it's much harder to miss.

• The tooltip for the Locus now prints different text in Conquest Mode.

• Thanks to crawlers for the bug report that prompted these enhancements

Galaxy Map And Camera Display Work, Lobby Map Overhaul

• The galaxy object now has a a FindSpatialCenterOfGalaxyInGalaxyCoordinates() method, which lets us find the center point of the galaxy based on however the planets were seeded.
  • Also added FindSpatialBoundariesOfGalaxyInGalaxyCoordinates() to let us find the overall bounds easily.
  • But actually we're going to not bother using this, turns out.

• We were also working on a CenterGalaxyOnAllPlanets() method, but we've abandoned that for a different approach. The code is mostly complete and left in for now so that we can revisit it if we find a use for it.

• The old-style cameras have been disabled for a long time (years) in xml, but we've now removed them from the code as well.

• Added extensions GetWorldSpaceBottomLeftCorner and GetWorldSpaceTopRightCorner for UI RectTransforms.
  • The chat window in the right hand side of the lobby in multiplayer now uses this and a world space to camera transform to calculate its ScreenSpaceLeft, which lets us, regardless of your screen resolution, DPI, aspect ratio, or custom settings in this game itself, figure out where the UI ends and avoid underlappig it.
  • Same thing with a ScreenSpaceRight on the map tab left sidebar.
  • Same with ScreenSpaceTop on the footer controls, and ScreenSpaceBottom on the header controls.

• For the galaxy map camera, the zoom now lets you go out farther in the lobby, and even farther in the multi-player lobby, so that it can always fit the map entirely in the main view screen.
  • Tested this with the game at 900px wide, 1800px wide, and 2560 pixels wide. In all cases we now come out with a consistent result.

• Fixed up a bit of logic for the dynamic zoom for resolutions smaller than 1200px x 900px.

• In the lobby, the galaxy map camera no longer responds to mouse or keyboard input at all. This is an overhaul that has been requested for years and years.
  • Now the lobby always has the map as zoomed out as it can.
  • It is also centered in whatever the available space is (depending on if the right-hand sidebar is there or not, that may or may not be centered on the actual screen).
  • This is extraordinarily useful, in that it lets you cycle through maps with ease, and there is typically not too much detail that you need to see in them like you would during an actual game.

• The math for how much the galaxy camera should be able to zoom out has been entirely re-coded from scratch, and is now vastly more efficient as well as something we can actually follow.
  • Instead of taking something like a thousand loops to get a kinda-bad answer, it now takes typically under 60 loops (often under 20) to get a highly accurate answer.

• In the galaxy map display calculations, and number of strange magic numbers were removed, now that we have the proper math in here. Wow this was unexpectedly hard.

• At very long distance zooms, the planets were STILL being visually offset from their lines that they were above, it turns out.
  • This had to do with how we were offsetting the icons in order to avoid z-fighting, and how that was getting magnified by us scaling up the planets at further zoom distances.
  • The fix for this might introduced some z fighting, but probably will not because of the way that we are rendering these as "cutout" style transparency, and with GPU instancing determining the order of draw.

• In the lobby, the planet you own is no longer ringed in white selection, as the in-game "planet you are on" is done. Now it just shows the planet color, plain and simple.

• In the lobby, if you have the setting "always show planet names" on, it still won't do that, because that will make the fixed-size galaxy map potentially impossible to read.

• The galaxy map in general shows planet names that are un-owned as being far less blinding white in their text.

• In the lobby, it no longer shows the first owning player name under the name of the planet.

• The visual scale of the actual planet spheres in the lobby is now about a two thirds what it previously was, but the text is almost as large as it was before. If the text is too small, we can adjust it up.

• In the lobby, hovering over a galaxy map link no longer highlights the two planets adjacent to it, since that is visually confusing in that context only.

• Added ShowTooltipNarrow() and ShowTooltipWide() to the IPresentationLayer and IBattlefieldHandler, so that we can trigger tooltips super-directly from deeper-in code.

• In the lobby, hovering over planets now makes it show a tooltip at the mouse cursor that says the name of the planet (since that might be obscured by being very near the top of the screen, now), but also some other information.
  • It says how many wormholes there are, and gives a comment if that will be extra hard to defend.
  • It also shows the name of any human players who are going to have that as their starting planet, in their appropriate colors based on their faction color.
  • There was some funky stuff in the hover logic for the galaxy map that made this super difficult.

• Fixed a funky bug relating to tooltips that were showing the same text as before, but had themselves cleared and then wouldn't keep showing the text anymore until it was given different text.
  • The first time this ever manifested was in giving tooltips on the lobby galaxy map, but it's an old old bug.

• Fixed an issue in the lobby where it was not showing the names of planets automatically for planets owned by players other than yourself.

• The map types list in the lobby has always been kind of randomly organized, which was annoying. More recently, it has been in a different order every time you start the game.
  • Now it is in the order Realistic, Simple, and then all the others alphabetically.

• Fixed a bug when loading directly into the game and not visiting the lobby where it did not set the dynamic zoom properly in the last few internal builds.

• Also fixed a boneheaded typo in the very most recent internal versions that was making the height of the dynamic zoom calculation slightly off.
  • There is still something a bit off with the squares map type and how it displays in the lobby (it is offset upwards and also shown too small), but all the other map types work and so this seems to be some sort of data issue with that map type and not a problem with our central algorithm.

Version 2.117 Ion Cannons And Melee Units

(Released August 11th, 2020)

Balance Work

• Ion Cannon Shots can now:
  • Damage both the personal shields and hull of a target in one shot (including killing it outright).
  • Kill multiple ships in the same stack in one shot.
  • Thanks to GreatYng for pointing out the oddity of Ion Cannons not being able to do these things.

• AOE shots can now hit any enemy bubble shield overlapping the explosion, rather than only being able to hit the shield if the generator (or something it's protecting) were physically within the explosion.
  • Thanks to NRSirLimbo for reporting and RocketAssistedPuffin for the save.

• Crippled units (i.e. player flagships) can now pass through enemy bubble shields and traverse wormholes covered by them, to avoid being stranded in hostile territory.
  • Thanks to tadrinth for reporting and RocketAssistedPuffin for the save.

• Melee units in attack-move mode will now go after units within a limited range (equivalent to a weapon with range=Normal1), so you have a middle-ground between "only attack your direct target or things actually touching you" and "pursue anything on the planet".
  • Also, even when not in attack-move mode, player melee units without orders (and not in hold-fire mode) now automatically move to attack nearby targets. If all such targets die or move away, the melee unit goes back to where it was.
  • Thanks to OzoneGrif, wm46, RocketAssistedPuffin, and Ubifan for the suggestions and saves leading to this change.

• Fixed a bug where deathgrip tackle drones had a gun that could shoot immobile targets, greatly confusing their logic. That gun now has the only_targets_mobile_units flag.
  • Also fixed a bug where tackle-drone-launchers could give their spawned drones a direct attack order to go after an immobile unit.
  • Thanks to tadrinth and RocketAssistedPuffin for the reports leading to these discoveries.

Bugfixes

• Scourge: fix a bug where player-allied scourge were building tons of Builders but no Fortresses (they were mistakenly thinking they were minor faction allied)
  • A number of people on discord noticed the lack of Fortresses

• Potentially fix a bug where Regenerator Golems were regenerating things while crippled
  • Thanks to crawlers for reporting.

• AI Reserves now will create anti-player zombies.

• Take another stab at 'adjacent planets watched'. Sigh.

• Fixed an issue where on computers without GOG Galaxy installed, they would get a harmless-but-annoying error message at game startup.
  • Thanks to Badger for reporting.

• Minor tweak to target list planning; use a case insensitive comparison for zombification.

• The 'total units' count should now include stacks.

• The AI "Great-Turrets" are now simply called "Turrets" again, as at this point balance-wise the player turrets and AI turrets are similar.
  • Thanks to GreatYng and RocketAssistedPuffin for bringing this to our attention.

• Exogalactic War Units now use a better spawn-in message than before
  • Thanks to zeusalmighty for reporting

• Fix a problem with the Custodian description text
  • Thanks to GreatYng for reporting

• The semicolon was previously not supported by our chat log format (our "condensed ascii"), but now is.
  • Thanks to Badger and Sigma7 for reporting.

Multiplayer And Lobby Work

• Heartbeat messages from the client to the server and vice-versa are now omitted from all of the networking logs. They were making things far too cluttered for us to be able to see real information when we had network logging of various detail levels turned on.

• Some excessive logging on GOG and Steam with regard to their send result statuses has been removed, as again this was clogging up our network logs.

• The clients now have a secondary FromClientToServer_SignalDoneButHasNoCommands that they can send in place of the FromClientToServer_SendMyNextCommandBatch when they don't have any commands to send (which is most of the time, statistically).

• GameSpecificNetworking now has a DoPerFrameLogic() method, which lets us efficiently set some per-game settings like we do for the arcen debugging stuff.

• New setting in the Network section of the settings menu: Network Logging Includes Frame Auths.
  • In the lobby and during gameplay, there are messages sent by the network every 100ms or so to keep themselves in sync, and if you have them included they will tend to crowd out the rest of the messages that are sent. So by default these are disabled.

• New setting in the Network section of the settings menu: Network Logging Includes Command Batches.
  • In the lobby and during gameplay, there are messages sent by the network as frequently as every 100ms, but more often some sporadic multiple of that. This is how factions tell everyone what they want their ships to do, and how human players say the same. There's very little that we can say about this in a meaninful way in the networking log level, and again this will tend to crowd out the other messages that happen. So by default these are disabled, but bear in mind that most things that players actually cause to happen are being caused through these batch messages.

• Found and fixed a general issue where the player accounts were not properly being cleared (and fully resetting the player account IDs) during all cases where a game is loaded into the lobby. At a bare minimum, it was affecting the "open a custom game with my last settings."
  • That would have eventually led to out of bounds exceptions, over time, and in general was cluttering up some areas important for multiplayer, but it had not had a negative impact on anyone so far.

• Fixed a bug that was causing all sorts of craziness on clients in multiplayer in the lobby, with them not getting their galaxies full cleared.
  • This was leading to things not being properly in sync between the client and host, like planet names or locations, and things like extra planets or planets in old locations, or noninteractive planets at a whole other scale visually from the real planets. It was quite messy! All of that is now corrected, thankfully.

• In the lobby, when you change the number of human factions, it now regenerates the map immediately. For other factions it does not bother, never has, because there's nothing different to see at this point in the galaxy.

• The WorldSetup objects now have a readonly SetupTypeName on themselves, so that we can tell what they are.
  • ChangedSinceLastMapGenCall on this same object is now a property, which lets us throw out debugging messages when it is being triggered for whatever reason.
  • Something has been causing extra calls to be sent through the map generation pipeline in a redundant fashion that makes everything slower than it needs to be.

• Removed the ability to send a WorldSetup object via GameCommand, as that is actually highly antithetical to the way we edit that data as of the last year and a half or so.
  • We specifically try never to adjust the entire thing, because several players at once could be editing different parts of it and we want those bits to all merge at once.
  • Got rid of the SetupOnly_ChangeSetup GameCommand, which was set to use this sort of thing.
  • Got rid of the LobbyWorking_HasBeenChangedAndShouldTransmitChanges() method on WorldSetup, which was being used by a loop in the footer to keep clients up to date.
    • The problem with this is that this was far older code, back from perhaps 2017, and it was using an older and more sledgehammer-y approach to try to keep clients and the host in sync. When the lobby was redone in 2019, this became obsolete but was still hanging around and so caused some nontrivial chaos.
  • LobbyWorking_HasBeenChanged() has also been removed, as it seemed duplicative and was no longer in use perhaps?
    • Oh goodness, this was actually how we were ever updating the long-term stored version of the game (as opposed to the lobby working copy). This was not a good way to do it, but it was not quite as antiquated as we thought.

• Fixed a rare nullref exception that could happen in the ships sidebar if you were not properly on an actual planet.

• Fixed an issue from the last few months (probably) where if you had an error during map generation, it would still try to dump you into the world and then give you tons of other errors. It should have been stopping you and making you fix whatever was wrong.
  • This is most common when there are not enough planets for all the factions you have chosen, or a few other rare cases. One such case is just discovered internally, and that's you having a planet starting index that is higher than the number of planets currently in the world.

• Helper_AssignHumanHomeworld() and the AI variant are now Helper_TryAssignHumanHomeworld() and an AI variant.
  • It was already quite possible for these to fail if the player had selected something invalid, such as previously having clicked a planet with an index above 40 and then choosing a map size of 40 planets.

• The game now auto-assigns humans a planet that is valid if they don't have a valid option, and it does so in a multiplayer-compatible fashion.
  • Previously this tried to do the assignment, but did not do so in a way that worked, and even if it had worked it would have only worked for one player.
  • This also includes a new sub-gamecommand, StartingIndex_FromAutoAssignInMapgen, which efficiently lets the UI know what is happening without triggering an extra map generation cycle or accidentally overriding a distinct change made by the player.

• The lobby is now substantially more responsive even in single player, although it seems to be having some extra calls that we are still hunting down.

• Fixed an annoying issue that was causing an extra regeneration of the map to happen when you first went into the lobby in single player or as the host, leading to the lobby load time being half a second or so longer than it should have been.

• Fixed an issue where if one person in the lobby adjusted the following fields, it would not be reflected for the other players in the GUI (but would in the map itself):
  • Planet naming style.
  • Map type.
  • All of the other map sidebar controls were showing things properly and also able to be controlled from any client or the host.
    • Do bear in mind that if you manually enter a map seed, it won't appear for the other players until you hit "Regenerate Map," but given that does not actually affect the map until that button is pressed, this should not be overly surprising.
  • All of the faction controls and their sub-controls have been workign properly from the start, and still seem to be.
  • Same thing with the galaxy option controls (all of them), which is pretty exciting.

• The "Campaign Name" is something that is only for savegame management, and thus is only relevant on the host side. The clients now see "Will be set by host" as the text in that box, and cannot change it.

• The "Start Game" button is also something that we are limiting to be host-only, for a variety of reasons. The biggest is that they need to set a campaign name, but that's entirely host-side.
  • Clients clicking this button now get a message that asks them to wait for the host to do it.
  • Actually, for the sake of ease of use, the "Reset To Defaults" button and this one are just hidden on clients in multiplayer. Clicking Reset to Defaults on a client results in an error for a lot of good reasons based on how it is trying to accomplish that.

• All of the lobby controls now seem to work properly for both the host and client. There are still some things in chat that are messed up, and we're missing controls for making clients not always be spectators, but we can also get into the game.

• Fixed the harmless but annoying "StartingIndexChanges: factionIndex 1 passed when faction count was 0" error message that would happen on clients after a host gave orders to start.

• The campaign name and its textbox is now just plain invisible for clients in a multiplayer game, again to avoid confusion.

• The right-hand sidebar in the lobby in multiplayer is now split into two scrolling parts.
  • The top part has information about the connected players and the factions that they are a part of, and never auto-scrolls around.
  • The bottom one has the actual chat messages, and auto-scrolls to the bottom when new messages come in.
  • The fonts of both are a bit smaller, from size 12 down to 10.

• From past changes, we can now verify that the proper name shows up for clients on the host, instead of it just saying "Player 2" for them.
  • However, text messages from the client in the lobby show up with the color of the client but the name of the host?

• Hitting enter while in the chat textbox in the lobby now actually sends the chat properly, as already worked in the main game.

• In the multiplayer lobby, the top panel now shows the status of the various PlayerAccounts that are in a game. Specifically it shows any who are connected or disconnected, or which ones are you, etc.
  • It also shows players that have an account in this game at the moment, but who are absent.

• Fixed a bug where player accounts were being created with the name of the host, not the connecting client, which was confusing all sorts of things.

• The preferred colors from the player profile on each local machine is now sent over and applied to their player account for later reference. This was also causing endless new player accounts to be created, since there was never a match to the connecting client.
  • A variety of pieces of code had to be updated to support this.
  • Apart from them selecting a color for their faction, this gives them a chosen color in text chat.

• Added a new OnEndEdit() event for text input boxes, which lets us trigger things when a player clicks out of a textbox.
  • This now happens for the map seed in the galaxy map, same as with hitting enter, as both now trigger an immediate regeneration of the map (without you having to hit regenerate map manually).

• Faction colors for player factions now come from the network data rather than just the local data, which is important for multiplayer.

• Fixed the bugs with lobby chat messages not appearing properly on the client and host in various ways.

Version 2.116 GOG Networking Done

(Released August 6th, 2020)

• Fix a bug where the 'Watch Extra Hops Worth Of Planets' code was being applied incorrectly
  • Thanks to Tynendir for reporting.

• Fixed a bug where Macrophage Spores don't move to new planets when loading a second save game.
  • Thanks to GreatYng for reporting, and StarKelp for fixing.

Milestone: GOG Galaxy Networking Integration Complete!

• The Arcen GOG Wrapper has been majorly updated to get your GOG Galaxy Friends list and the status of them so that you can play multiplayer with them if you want to.

• When you are in the GOG networking mode, it now shows the list of your friends, just like it does for Steam.
  • Basically all of the functionality is identical here, in terms of how it shows those who are online, offline, and in-game, and lets you filter between them, etc.
  • The one difference is that in Steam we show the "in this same game" status as a light green (like Steam does), and in the GOG version we show that status in a purple color (to match their color theme).

• The host computer now is able to open up a lobby for friends when it says "open server."
  • This works with all the various NAT punchthrough that GOG Galaxy supports, which we are not entirely clear on the details of. This may be as robust as Steam, or slightly less so, but we honestly have no idea. We'll be interested to hear from you! But either way, it's out of our hands at that point, so it's just information gathering.
  • The host is also now properly destroying the lobby when it shuts down, so that others won't try to connect to it and not see it.

• When clicking the Join menu in GOG, the client now sees lists of their friends with four categories instead of just three.
  • Friends who are in-game but not in a lobby right now are showed in a dull light purple. Friends who are in game and have an open lobby show up with a bright purple status.
  • You can also cycle through all friends, open lobbies, friends in this game, and online friends.

• In the listing of connections, for GOG it now shows a button that says "closed" instead of "connect" next to each friend who does not have an open lobby.
  • With Steam, someone could be in offline status but still let you connect to them. With GOG, we have to have a lobby ID to connect to, so there's no way to hide that and still even attempt a connection to someone.

• GOG clients are now able to search for lobbies and correlate those to their friends list, which under the hood is a several-step process.
  • We are filtering out any potential rogue lobbies created by bad actors, and correlating them specifically to those owned by your friends.
  • It's possible that a friend might jump from the status of being offline (prior to opening the lobby) to showing up as someone you can connect to (when they open the lobby), actually, so potentially this actually does allow for the same sort of quasi-stealth game connections that people can do using Steam.
  • That said, since lobbies have to be advertised (to your friends) and we can't have a client just blindly try connecting by userID like in Steam, if you open a lobby and your other friends are actively in GOG looking to join a game of AI War 2, they would see that. Not a big thing, probably, but it's something we've noticed about the nature of these two differing systems.
  • You may find yourself having to hit refresh a few times on the client side in GOG, as there is a few seconds of lag before the lobby that the host creates shows up. It seems to be under 4 seconds at the most in current tests.

• GOG clients can now actually initiate connections to a host lobby. And it works!
  • The client is aware of getting into the lobby, although it takes a few seconds.
  • The client also does gracefully tell them that they are leaving the lobby, and the GOG servers and the host are both finding out about that properly.

• The GOG Galaxy networking client is absolutely fully implemented, now!
  • A huge amount of the actual send/read logic was able to be pretty much identical to Steam, which was not super far off from what we did with LiteNetLib.
  • Pro tip: if any other developer is ever wondering about SendP2P messages on the GOG network, those can take any size, not just stuff under 1200 bytes. It handles all of the fragmentation and recombination extremely well, and just as fast as Steam or other services.
  • The initial connection process to a lobby with GOG is a bit slower from North Carolina in the US than what we are seeing to Steam, ranging from 4-6 seconds on the first creation of a lobby on the run of a game to 2 seconds for ones after that. But once the connection is established and brokered, it's easily as responsive for us as Steam.
  • For us, this is routing through the internet and some firewalls, so the NAT punchthrough or relay servers or whatever it is in this case seem to be doing their job. We certainly did not mess with any ports.
  • Because of the nature of how data is read in from the buffers in GOG, we actually were able to get slightly more efficiency and speed with how we manage the heap compared to Steam or LiteNetLib. It's probably not usually a huge advantage, but it is something that we appreciate the efficiency of.
  • Essentially all three of the networking options have their own pros and cons a bit, but are all extremely good. That's a real relief to see, and to see them all working.
  • At this point it is now down to making the game networking actually work properly to have the game run, and that's a networking-framework-agnostic process.
    • In a lot of respects it feels like after a large move of house, where all our stuff is now out of the old house and into the new house, but we still have a lot of unpacking and arranging to do. It still feels excellent to be done with the movers and have everything in one location now.

More Multiplayer Work

• Removed some old and unused code that is from the Valley Without Wind versions of multiplayer, checking for disconnections due to inactivity. We are handling that at a lower level, now.

• Got rid of some old code that was mainly in the older version of our engine pre-Raptor, which was checking for "pthread exceptions" that was never something that could happen in the newer engine that we've replaced it with.

• The game now has some self-checking code in there for giving us some actual feedback when it decides that it can't run the simulation or show visuals for more than 3 seconds at a time.
  • This is a pretty rare case, but it was the "too many changes and the client machine goes to black screen and unresponsive" issue.
  • This may have fixed the issue, although there also seemed to be another issue that was related to this.

• Took out some other unused and old data.

• The "UseSeed" command in the lobby no longer triggers the gamesettings to be saved. That was not useful, and slowed it down for no good reason.
  • It turns out that this was blanking out the settings data in a strange way a lot of the time, and causing the black and white view of the game after you reloaded. It also was apparently directly the cause (somehow) of the infinite black screen on the client after this was triggered.

• Put in a lot of extra error handling for the steamworks networking in particular in case it has internal exceptions while trying to send something.

• The cumulative effect of a lot of these changes has basically fixed a ton of issues in the lobby, including:
  • Clients not being able to use certain buttons or dropdowns.
  • Certain dropdowns not showing the right values to clients.
  • The client and the host getting a different view of things after certain buttons or dropdowns were used.
  • For a while there I was thinking that I was maybe going to need to make some of the controls for the lobby host-only, but now that's definitely not the case!

Version 2.115 Imperial Summons

(Released August 4th, 2020)

• You can now have multiple copies of the Devourer and Zenith Trader in a game
  • There's no technical reason for this limitation, and we're encouraging Zenith stuff because of the coming Onslaught

• Quickstarts are sorted alphabetically again, instead of when the quickstart was created
  • Thanks to ArnaudB for the bug report

• Add some defensive code to the Dyson Sphere
  • Thanks to GreatYng for reporting.

• Fixed a cross-threading nullref exception that could happen during unit unselection (probably due to it dying).
  • Thanks to GreatYng for reporting.

Imperial Fleet Summoning Changes

• The AI now generates some weaker bonus exos against miscellaneous player targets (energy producers, MDCs, GCAs, etc), just to keep things exciting.
• Fix a bug where Exos were trickling into their targets instead of moving as groups.
• Start the Exos the moment you finish the hack, instead of some random time < 30 seconds from when the transceiver finishes.
  • Blame MasterGeese for these changes

Multiplayer And General Lobby Work

• The "partial map generation" code for giving a preview in the lobby now does even less work, bringing our mapgen times down to 200ms or so on average.
  • Because of this, we've also moved the mapgen code during partial-map-generation onto the main thread, to keep things as fast as possible. The full map generation still happens on a dedicated background thread.

• During the lobby, the way that we handle the execution frame loops is now entirely new, and basically skips over a lot of things that were artificially introducing lag.
  • This makes the lobby perform far better in general, and also makes the map generation times absolutely flyyyy compared to what they were before, on single or multiple player maps.

• This is leading to a number of substantial bugs in the lobby, temporarily, but will make the lobby performance far superior in both singleplayer and multiplayer as we get those resolved.

• The silent "generate partial map" or "generate full map" complete messages now tells you how long it took.

• The way that mapgen works, it now has a couple of different contexts that it passes in, so that each one has its own random number generator that is not dependent on the others.
  • This lets us ensure that there is consistency between runs of the same code on different machines, and on the detailed version (for play) and the non-detailed version (for viewing in the lobby).
  • This is largely solving the issue that we just have started having in this latest round of internal builds with a difference in the final map and the original map.

• Renamed the Generate() method on map generators to GenerateMapStructureOnly() to be more clear, and also make our code more eaily searchable.

• Adding a planet to a galaxy no longer requires passing in a context object at all. It now generates its own internal random number generator for its initial variables based on the galaxy map seed, its index, and its original position.
  • One of the side effects of this is that now when you change which planet you want to be your starting planet, it will keep the name the same instead of shuffling them around.
  • This sort of name-stability has actually never been in this game before, and always was mildly annoying. But in multiplayer lobbies it was going to be even more annoying. "I'll take Adam." "Okay, sure." "Well, I clicked it and now it is called Brian, but that's the one I meant." "Wait, what?"

• When you are loading a savegame or a quick start as a template into the lobby, it now immediately regenerates the map rather than using the old map, because it's highly likely that the new structure will be different in some fashion.
  • This then also solves the problem of clients being different from the host on first connect, probably; that was kind of a compound problem in general.

• Fixed an order of operations issue that could lead to the occasional "No human players found to seed homeworlds for!" error in the lobby in general, but more recently in multiplayer more frequently.

• Possibly fixed the issue with later player accounts in multiplayer being renamed incorrectly to "player 2" and such.
  • There is still a fair bit of duplicate data happening between data structures, and some lobby/startup code that is blatantly single-player-specific that we need to get sorted out.

• Dramatically cut down the time it takes to generate planet random seeds, but still with keeping them consistent as you click around the galaxy, by pre-generating the random seeds as the first 300 calls to the "broad random" for the galaxy. Any seeds needed after that will be generated the slow-but-unique way. This means mainly things like nomad planets or other things that are late additions. When just adding a planet or two, it's not so slow as to be a problem.

• Until now, if you loaded an existing savegame it would automatically put you in control of the first player account, even if your profile name did not match.
  • It now continues to do that when you are loading as a single player game, which is enormously helpful for testing and just in general, really. But in multiplayer, it now tells you a list of what the available profiles are,
  • In general, if you are loading something as a template, it now clears all the old player account data and loads in your own current name as the new first player account (and further players are already added properly as they connect).

• Fixed up the way that planets are shown as colored in the lobby galaxy map, so that it will work well with multiplayer and also with the new faster single-player-generated stuff.

• Improved the logic for it properly setting you to be viewing the new planet you've selected after you make a selection in the lobby. Another thing that did work, but with the revised logic had to be made to work all over again.
  • This concludes the fixes to all the known bugs that we are aware of that we've introduced into the lobby today.

• Fixed another nullref that was happening only on the multiplayer custom games after today's changes.

Version 2.113 Dyson Growth

(Released August 3rd, 2020)

• Fix a null reference exception if the game wants to spawn a Relic but you own basically every explored planet
  • Thanks to Lord of Nothing for reporting

• Fix a bug with Dyson Sphere units not marking up
  • Thanks to GreatYng for reporting

• Fix a typo in a Battlestation description
  • Thanks to ovalcircle for reporting

Multiplayer Work

• Fixed one oversight that was causing the factions tab chat to not show its chat text properly, unlike map and options in the lobby.

• As text messages come in inside the lobby, it now scrolls to the bottom of the chat window, as you would expect.

• The maximum length of chat messages in the lobby is now 1000 characters.

• GetLocalPlayerFaction() is now GetLocalPlayerFactionOrNull(), to support spectator players.
  • Additionally GetFirstPlayerFactionOrNull() has been added, and GetIsFriendlyToLocalFaction() and GetIsHostileToLocalFaction() have been updated to work based on that in order to extend support for spectators even further.

• Planet visibility is no longer determined from your local faction (since for spectators that might be null, and for all humans it is identical anyhow). It now just uses GetFirstPlayerFactionOrNull(), since that's slightly more effient and supports spectators.

• Huge areas of the AI have been updated to allow for spectators, generally giving them information that mirrors that of what the first player is seeing (objectives-wise and so on).
  • In other areas, it simply updates things to not erorr out when you hit hotkeys, etc.
  • Spectators also can't run "cmd" messages from chat; it will just show the text as if it were not a command.

• GetLocalPlayerFaction() on the planet object has been removed, as that did some risky caching for multiplayer in general, and also was not really compatible with spectators.

• Fixed a couple of errors where non-sim-but-background-fation-AI code was referring to the local player faction. It now refers to the first player faction.
  • Probably this would not have made much difference, unless the game switched hosts partway through or something like that.

• The chat controls in the lobby now work, though there are some bugs in them still.

• The correct name for each player faction now shows up for even for players who are in spectator mode.

• Player factions in spectator mode now see in the lobby chat area that they are in spectator mode.

• Both the lobby chat and main game chat now support more characters in their text -- basically anything that our "condensed" ascii format is allowed to display.

Version 2.112 Steam Networking Complete

(Released July 31st, 2020)

• The description of the Ensnarer Battlestation now explicitly mentions that it can use AI Great-Turrets
  • A number of people have reported this as a bug over the years, so hopefully that won't happen anymore

• All of the Fallen Spire ships, including those in the Spire Railgun Shop optional mod, have had their health and shields doubled, but their firepower halved.
  • This should make battles with other mega-units last longer, which feels more appropriate, but keep their "damage over time" the same.
  • When these were going toe to toe with the AI Exogalactic War Front units, they seemed to disappear too quickly, making them feel weak. This should keep their effectiveness the same as before, but make the battles take about twice as long with those large ships, so that it doesn't feel like instant evaporation at least.
  • The one unit not adjusted is the Spire SuperDreadnought from the Spire Railgun Shop mod. That was already maxed out as a "kills everything and is basically invincible" sort of ship.
  • Thanks to a variety of players for weighing in on the feel of these units, and Badger for suggesting this adjustment.

• Rather than having a slider for the number of planets in the galaxy map view, there is now a dropdown.
  • This prevents the intense "regenerate map spam" and lag that was inherent in using a slider, along with making it easier to choose a certain number of planets.
  • We have made it so that it only gives you multiples of 2 below 60, multiples of 5 below 100, and multiples of 10 above that. This makes it easier to quickly peruse the range.
  • Each entry has some commentary on what it will mean to the feel of the map if you are hovering over it in the dropdown with the number of planets expanded.
  • Actually, it wasn't even regenerating the map after you changed the number of planets, which was really confusing! You had to hit regenerate map. Now there is no confusion.
  • All of this is easier to use as a player in general, on top of being something that is easier to deal with in multiplayer for sync purposes, too.

• Fixed a longstanding bug where dropdown selections that were open could persist when leaving the lobby or changing tabs within the lobby.
  • Thanks to MaverickPenkenn for reporting.

Milestone: Steam Networking Integration Complete

• Fixed an issue where multiplayer clients disconnecting from a host would try and fail to save the last settings from the lobby. Now it does not save the lobby settings from a session you are just a client of.

• All of the networking frameworks have had their "send to anyone here and myself" code consolidated into the network authority, so when an interface is partially implemented it won't wind up stalling out waiting on the host.
  • Less code duplication is also always good, but it wasn't clear that this was a complete commonality until we got partway into the Steam framework implementation.

• A new ArcenSteamClientConnection wrapper has been created to keep track of the Steam connection on servers.
  • This now properly is registered while the steam connection is alive, and all of the "waiting for player to connect" and "player disconnection" events now happen properly.
  • For some reason, we are not getting the "host has accepted you" callbacks on the client end, but that doesn't really matter. As we start actually having our challenge-response data sent via the higher-level networking code, that bit will become irrelevant as it establishes itself within part of one second anyway. It's just a curiosity, or potentially something that is only meant to trigger host-side. Frankly the documentation is sparse in points, both from the Facepunch C# wrapper and the Valve C++ baseline, so this might be working as intended.

• Substantial progress on the Steam network transmission of data. Still not quite seeing it register yet, but we're getting close.

• The full suite of network logging has been integrated into our steam networking wrapper, now.

• When a Steam host shuts down the connection, the steam client computers now definitely notice and react to that. The inverse was already working from the start.

• Figured out the missing information on how to get data from the facepunch steamworks wrapper (call Recieve() to trigger the other events), and are sticking with the derived classes methods rather than the interface method, since for relay connections it seems like the interface method is not fully implemented.
  • With this in place, we're now getting messages on the client machine, and they are properly decoded! The host for some reason still doesn't get messages back, but this is progress.

• Fixed a bug in our connection status window code that was causing it to stop showing a timer on connection stages after the earliest ones.
  • With LiteNetLib, that all flashes past so fast we could not tell that was even happening. With Steam currently stalling out with the host not getting messages, it became apparent.

• Previously it was possible for our socket mode to be off, and silently ignoring messages we were sending.
  • This was initially done to make it so that any lingering messages that would try to be sent after a disconnect would not cause visible errors, but in this case we've wound up with the socket state being off on the steam networking, despite it being active, and this thus resulting in client messages being composed but not actually sent. That was not obvious because of the lack of error messages.

• Figured out a stupid mistake in our own code, indeed a single line of code, which was preventing the client from sending data properly back to the host.
  • Steam networking is now fully functional in terms of sending data back and forth. There is some sort of issue with the world data not getting across properly, but we have to now verify whether that's limited to Steam or also happens on LiteNetLib.

• Added in a substantial amount of extra error tracking in our univeral and game-specific network message handlers.
  • This helps us to find out where errors actually are.
  • In the case of steam, right now we are getting some exceptions when the host tries to tell the client what their player profile number is.

• Fixed a simple typo that was having us send a junk message via both frameworks to the client when they first connected.
  • In both cases, this led to the galaxy map not showing up properly on the very first world connect. But in the case of Steam networking, it also led to an exception because the byte array was too small.
  • This extra smallness is actually more accurate, and basically it was only not erroring with LiteNetLib because the buffer array was larger than needed because they are pooled.
  • This is an interesting difference between these two networking frameworks, but neither one is particularly "wrong. But it does make Steam slightly more sensitive to certain errors in a way that is useful, because it's an error in both cases that manifests in other places.
  • Anyhow, this seems to argue for us to internally use Steam as our main testing framework, additionally because that means we are round-tripping out through the internet and Valve's servers and back rather than having the speed of a LAN.
  • This also means that the Steam networking library is fully completely integrated, which is a very major milestone! Now it's all game logic, regardless of what is going on with the two networks.
  • Also worth noting that it's under 3 seconds to connect a client to the lobby in our test case, using the Steam relay servers, etc. On the LAN it is under one second, so that's pretty killer.

More Multiplayer Work For All Frameworks

• "Planet GalaxyMapVisuals in CenterGalaxyViewOnPlanet null!" will no longer happen. It was a thing relating to trying to center on a planet slightly too early, and the population of the galaxy map still having failed.
  • We are now bypassing that and using the raw internal transform offsets to calculate the correct camera position.

• In response to the initial challenge from the host, the client no longer just sends their profile name. They now also send their list of installed and enabled expansions and mods.
  • The host then compares that list to its own installed and enabled expansions and mods. If there are differences, then it rejects the client connection and does a popup on the client and a chat message on the host side explaining why.
  • It also notes in there that both parties can enable or disable expansions or mods in the Game tab of the Settings menu to get things to match.
  • Please note that, yes, in some cases there will be mods that are enabled but which "wouldn't do any harm" because their functionality won't be triggered in the campaign in question. We don't have any way of knowing that, so we have to err on the side of caution.
  • When it comes to expansions, there are none that "wouldn't do any harm," because all of them add some ships and features to the game in general that all players need to have. For instance, the first expansion adds a bunch of new turret options that are available to choose from in any game you play with that expansion on.
  • At any rate, we've gone out of our way to make it easy for people to disable expansions and mods so that they can play with friends who don't have those same expansions or mods. The last thing we want is for the only solution to this sort of thing to be someone reaching for their wallet or having to faff about in the filesystem.

• Low-level chat messages that are sent to everyone from the host are also now sent to the host themselves. Previously, they could not see messages of this sort, turns out.

Version 2.111 Initial Steam Connection

(Released July 30th, 2020)

• Fix a null reference exception in my new spire relic code
  • Thanks to Chuito12 for reporting

• Nanocaust: Make the AI wait a bit longer before unleashing Exogalactic units on it. Remove some deprecated code
  • Also the exogalactic units won't hang around to kill the entire nanocaust, so there's a chance it can recover.

Milestone: Initial Steam Networking Connection Between Client and Host

• The Steam versions of the game now initialize relay access on startup, to make it so that the first connection using Valve's relay network will be extra fast.
  • If you wind up not playing any multiplayer, this is absolutely harmless. If suddenly it is flagging your firewall on game start on the Steam versions of the game, though, that might be why.

• The basic connection through Steam's newer networking sockets connection is now in place, using their relay servers as recommended. This is the approach that absolutely bypasses NAT traversal and anything else of that sort, and in fact hides your IP address from anyone you are playing with. The claim is that their backbone is faster in most cases than the general internet one.
  • If need be, we can make a "Steam over IP" version that allows for their networking sockets stuff to run more directly over UDP, and only fall back to their relay servers if there is a problem. This would be a lot like LiteNetLib, except with much better NAT punchthrough making use of ICE and STUN and all those things.
  • For now we are sticking with Valve's recommended practices.
  • Beyond just getting the initial connection up, we don't have anything there yet, but on the server we can see the Steam username of the client who requested to connect and who forged the connection.

Version 2.110 AI Exogalactic War Front

(Released July 29th, 2020)

• Spire Relics now have additional text in their descriptions explaining either A. whether the relic is on route to a particular planet , or B. that this relic must build on this planet
  • Thanks to Lord of Nothing for suggesting

• All of the units that were previously named "Galactic War [name]" are now just named "[name]"
  • However, in the tooltips for them, their faction should now say "AI Exogalactic War Front" in whatever the hunter fleet's color is, rather than saying the AI Hunter Fleet.
  • This lets us keep the shorter name, while making their sub-group and source a lot more clear.
  • This does need testing, so please let us know if it doesn't show up correctly somewhere. This is using a new xml feature called "override_faction_name", which lets us use a different faction name for some units in a faction, but without actually creating a new faction.
  • Thanks to Ovalcircle for leading the discussion that led to this.

• Spending a ton of science will now increase a player faction's Overall Power Level very slightly.

• Remove some inaccurate description text for the Rorqual Hejira
  • Thanks to GreatYng for reporting

UI for Steam Networking

• Added the new ArcenNetworkConnectionOption class, which we are using as a basis for things like lists of your Steam friends that you can connect to.
  • On the network authority, there is a new PotentialServerOptions, which is used to list out such things. This can be used for anything that we need to list a variety of to connect to, not just friends in Steam and GOG. This is in contrast to the "type in an IP" method that we use with LiteNetLib.

• Started building out the Steam networking framework layer, currently based on the ISteamNetworking implementation that most games use.
  • We MAY opt to also start supporting the newer ISteamNetworkingSockets framework that Valve has rolled out more recently, but that would be an alternative networking framework in AI War 2 that you could choose between.
  • The fundamental premise of those two Steam networking setups is really different, and while the newer one would in some ways be easier for us to implement, it seems a lot less familiar to the average Steam player, involves IP addresses potentially, and is maybe less battle-tested, but who are we to say.

• Previously it was still possible to have some exceptions happen silently in the Engine_Universal main loop, and they would log to disk but not show up in your face. Fixed.
  • It was also possible to get some silent errors when clicking buttons and other UI elements, turns out. This made the application just seem not to respond to the click, but now it properly shows the error if there was one.
  • Also it was possible for certain errors in the UI call stack to prevent the display of the error window.

• When you try to connect to an IP address, or you change your active networking framework, the game settings now get saved immediately so that that is remembered even if you shut down the program right away.

• There is now a screen that will be used for Steam and GOG connections, from the client side, and in Steam it now:
  • Shows a list of all your Steam friends, with ones that are in AI War 2 at the top, then ones that are online, and then the rest below that.
  • Lets you sort to just ones that are in AI War 2 or online.
  • Lets you refresh the list to check for updated statuses.
  • There is a connect button for each friend, but that is not functional yet. Just haven't gotten to that part yet.

Version 2.108 Galactic War Units And The Exostrike

(Released July 28th, 2020)

• Completely rework the way Exogalactic strikeforce leaders work, to allow per-AI-Type customization. No functional change right now, but intended to support some new AI types for DLC2

• HandleHelperJournals() has been split out into a bunch of sub-methods, since they can error in rare cases and it would be nice to know where the error generally is.

• Fixed an issue where we were not letting negative risk analyzer data be saved properly. Turns out that has a good reason to be negative at times.
  • Thanks to Badger for reporting.

• Added a new UI feature that is a throwback to some of the functionality we had in various past games: the "center screen message"
  • This is basically a message that can show up near the top center of the screen, and stay there for a bit or for as long as needed.
  • It won't block the mouse, and it is over everything except modal popups and tooltips.
  • This isn't something we would want to use all THAT often, but for cases of things like "hey the server isn't responding, why is MP stalled," this is exactly the sort of UI functionality we need.

• Fixed a harmless bug where, for the last few months since we've been doing multithreaded loading of camera xml files, the "PrivateVisExtensions" assembly would sometimes have annoying but unimportant errors appear on startup.
  • Essentially, it turns out that loading classes from a custom dll is not entirely threadsafe. Go figure. The fix for that is simple, and it doesn't slow things down, but now we know.
  • Thanks to GreatYng for the first report of this we'd had outside of ourselves running into it.

• Fix a bug where macrophage achievements weren't triggering if you killed all the Telia
  • Thanks to GreatYng for reporting and StarKelp for the fix.

• The in-game credits have been updated to include two more individuals who are key DLC2 testers.

• Added new GetWasWorldStartedOnGameVersionOlderThan() and GetWasWorldStartedOnGameVersionAtLeastThisVersionOrNewer() to world, which let's people find out in a really efficient way if a world was started before some certain point.
  • We can't be accurate past a month or so ago, because we were not tracking the original savegame version back then. But it's useful for looking at saves older than the more recent ones and altering them if need be.
  • This is now used specifically to fix this macrophage disabled stuff only on older savegames than 2.108.

• Fix a mispelling in the Mesopotamian planet names
  • Thanks to Lord of Nothing for reporting

• Fix a typo, "reciever" to "receiver"
  • Thanks to Apthorpe for reporting

UI Changes

• In the Load Menu, campaigns are now sorted by 'most recent save game in that campaign' rather than alphabetically
  • Thanks to Puppet Master for the suggestion.

• The default sort for the load menu is now by date.

• Rename 'Exogalactic Strikeforce' to 'Exostrike' and 'Extragalactic War Units' to 'Galactic War Units' in UI. Having "ExoGalactic" and "ExtraGalactic" was a perennial source of confusion
  • Note that internally the code uses the old names for the moment
  • Thanks to a discord conversation involving relmz32, Oryutzen and NRSirLimbo

Dyson Sphere Buffs

• The Dyson Sphere's ships should now really level up after the hack
  • Thanks to GreatYng and zeusalmighty for the bug reports

• Dyson Sphere hack descriptions now say "This hack will make the Dyson Sphere very unhappy with you for a while" to clarify the behaviour.
  • Thanks to Apthorpe for suggesting

• The Dyson Sphere now gets additional income (ie it can build ships faster) that scales with AIP. As AIP goes up the Dyson and Antagonized Dyson get stronger

Multiplayer Work

• SendMessageToAllClients() has been split into two methods: SendMessageToAllClientsWhoAreFullyConnected() and SendMessageToAllClientsRegardlessOfConnectionStatus().
  • We really don't need heartbeats and frame auths and so on going to clients who are still connecting in.

• Also added a GetCountOfClientsWhoAreFullyConnected() that lets us know if there are any clients who actually have gotten all the way in (and thus we should send world data)/
  • And added GetCountOfClientsNeedingWorldData() that lets us know we should make everybody wait for a bit until those people connect in. Otherwise everyone won't be on the same page.

• Added a OnServer_IsAtPausedSpotWhereGameWorldCanBeSent() that lets us wait for a break in things happening (usually only 400ms or so at most) to then have a clean slate to send a world state to a connecting client without that state changing between them being sent the state and them getting it.
  • Along with this, if GetCountOfClientsNeedingWorldData() is more than zero, then it now pauses authorizing more simulation frames until the world is fully taken in by the new clients who are joining.

• When a multiplayer game is stalled out for some reason, it now uses that central screen message to let you know what is happening, and if possible, why it is happening.
  • The game itself will have stopped having any action at this point, though you can still scroll around and look at things and give orders. But nothing is proceeding on the enemy side, so you don't have to worry about it being in your way.
  • When a client is first connecting into the lobby, it also uses this, to basically get the host and any other existing players to wait a moment before making more changes, etc.
  • On the host, this is already set up to give some context about the exchange under the hood that is happening with the new client.

Multiplayer Transition To LiteNetLib

• Okay then... we are switching network libraries when it comes to our non-Steam/non-GOG transport layer. Instead of FORGE Remastered, we'll be using LiteNetLib.
  • FORGE has grown a lot over the years, but the version we were using was from 2018...ish. We looked into updating our version of the code, but were not thrilled with how complicated it was going to be. Since we had working AI War 2 multiplayer on FORGE at least as recently as 2017 (in an alpha state), staying with the existing code made a certain amount of sense.
  • However, we started running into really long delays with something as simple as sending 10 bytes to say what our profile name was. This was taking upwards of 14-18 seconds between two machines sitting three inches apart and both within five feet of a powerful wifi router. We verified that the polling was happening hundreds of times per second, and that massive amounts of data was being sent via sockets. But in the internals of the old version of FORGE we have, it was taking it quite a very long time to assemble that into a full message, despite all the churn of hundreds and hundreds of data reads that were apparently all just empty headers and then noise.
  • Time for a new approach. It's worth noting that the current version of FORGE still lists all the most internal classes as "in development and lots of dire warnings," so that was another reason for switching up frameworks.
  • We chose LiteNetLib at this point, and within 40 minutes of dropping that in, we had the first successful connection between the machines on using this framework for the game. It was just a simple "hello client" message in basic unicode, but it arrived extremely instantly, as expected.
  • Next steps are going to be doing some cybernetic implants on LiteNetLib, rather like what we did on FORGE, to make sure that it integrates with AI War 2 properly and our network authority, and that it communicates as directly as possible with our own pre-formatted bitstreams. Most of that should be part of one day of work, at this point. We shall see. Then it's back to the actual larger "build out the multiplayer part of the game" work, but minus the lag that was tripping us up the last day or two.
  • It's also worth noting that, with this switch, we had to unfortunately also lose the awesome NAT punchthrough work that Doug Fields did for us with the FORGE interface back in 2018, but LiteNetLib has some of that of its own. And hopefully most people use Steam or GOG if they are behind firewalls of any complexity (since relay servers then become required, and that's where the free relay servers are).

• NetPeer in LiteNet now inherits from ArcenNetworkClientConnection.
  • For now, we are using our own methods to generate the ConnectionIndex, rather than using the existing peer.Id. There may be some numeric conflicts, otherwise, but we can also change this later.
  • We also are initializing everything on the Arcen-integration side, keeping our own clientsOfServer list for quick access, and then calling the game-layer RespondToNewConnectionAcceptedByTransportLayer() that will kick things off.

• PrepareMessageForSending() has become WriteLogOfSend().

• Added a new SendToSpecificPeer() method onto LiteNet's NetManager. Not sure why that variant didn't seem to exist.

• Wiring back up our LitNetSocket wrapper class:
  • SendMessageToHost_Inner() done.
  • SendMessageToAllClientsRegardlessOfConnectionStatus_Inner() done.
  • SendMessageToAllClientsWhoAreFullyConnected_Inner() done, using the new SendToSpecificPeer() method.
  • SendMessageToSpecificClient_Inner() done, using our own ConnectionIndices for now.

• Booyah! Exclamatory phrases are not the norm in our patch notes, but this is an exciting moment. LitNetLib is sending messages, and doing so with speed and accuracy and a minimum of fuss. The username travels three inches in well under a second, as should be expected.
  • At first, I was getting some very strange data sent across, and I figured that this was going to be an endianness problem, but actually all of the .NET/Mono data is the same endianness from the look of things. Which makes sense, given that otherwise savegames would not work from computer to computer.
  • After spending the night with that rattling around in my brain, this morning I decided to definitely do some profiling of the data before I started messing with the #ifdefs.
  • First thing I noticed was that I'd chosen to use the RawData byte array, and immediately next thought was "I better check there's not a network header in there." Turns out, yes, there's a UserDataOffset. The first four bytes are not my data, and so of course it would give gibberish as if the endianness was wrong. Just starting my reads from the proper offset makes everything work instantly.
  • All of this is from a Mac to a Windows machine at the moment, but my I'll be testing three-way with my linux machine as well. Right now laptop is sitting closed under the Mac.

• LiteNet logging now gets funneled to our own main logging pipeline.

Continued Multiplayer Work

• A fair bit of work has been done to make intentional disconnections from the client or server tell the other side more quickly.
  • This should help to avoid some disconnect-and-reconnect issues that are otherwise inevitable, and it just makes things feel more responsive.

• Additionally, when you use the in-game controls to exit the program as a while, it now takes one extra half second to lets Steamworks and GOG actually do their exit logic properly, rather than rushing out the door.
  • This should hopefully help with any cases of the game still being thought to be running on Steam in particular after it closed.

• When you exit the game, while it is doing its brief exiting stuff it now throws up a center-screen "exiting.." message.

• Got rid of OnServer_WaitingOnConnectionIndices, as this was basically a duplicate of something we are doing better and more cleanly a new way, now.

• UpdateDebugText() and all of its related variables have been removed.
  • This was the sort of thing that, in all of our prior games before Raptor and moving to 3D, you could hit F3 and get output information from. But it has never really been used in AI War 2 properly.

• Got rid of WorldTransmissionMessageType, as it was needlessly complicated.
  • This was essentially us breaking the world transmission data into multiple parts and then sending those in parts. Historically there were two reasons for this:
    • Back in AI War 1, we were using the Lidgren network library, and even though it was trying to work around the MTU (typicaly 1200 bytes) of various networks players might be on (or passing data through indirectly), it was often having bottlenecks and slowdowns with how it sent fragmented messages through the network card. Our theory at the time was that it was flooding the buffers of the NIC, rather than giving it data at a more reasonable pace. Network cards have grown enormously in the last 11 years, as have the OSes that control them, so that's not really a concern now. All of the newer network libraries that we would be using to send data should be able to handle 1MB of data without incident.
    • Secondarily, for AI War 1 and for the A Valley Without Wind games, we were sending enough world data that you could see a noticeable wait time, and by handling this above the network transmission layer we could give you visual feedback on transmission progress (even though the whole act of that actually slowed it down). Eleven years later, we have networks that are vastly faster basically everywhere in the world, and the world savegames for AI War 2 are much more efficiently packed than AI War 1 was. So sending a file like this really is like a midsize image on any website you visit, now. Not really a progress bar sort of situation, especially if the progress bar itself causes slowdown.

• ConnectionStage has moved from being just a UI-only element to being ClientConnectionStage as part of the ArcenNetworkAuthority, so that clients can better keep track of their connection status. Usually this is less than a second or so, anyhow, but we still need to keep track of it in case something goes wrong in particular.

• The host now properly sends the world data to the client, for the first time in three years or so.
  • The client is having some trouble understanding it thus far, so now is a good time to build in error handling (it was just silently failing, until now).

• Added a new SendLowLevelChatMessage() method on the network authority, which is used mainly for sending back and forth arbitrary messages from clients and the host to explain about the status of things. These can be echoed to all other clients or not, as the case warrants.

• If the client fails to load a world sent by the host, then the host and any other clients all get sent a low-level chat message about that, the client gets kicked out (since they never made it fully in), and the client gets a nice error message on their screen and in the log.
  • This way we can solve whatever the problem is, and nobody is left sitting wondering why things are slow (when in fact they errored).

• When you are loading a quick start, or loading a savegame as a template into the lobby, it now clears the chat log. This was definitely not information that needed to persist from a prior game to the new lobby!
  • These sorts of chat log messages, which are kept over the long term for the game (up to a limit), are no longer shown in the lobby at all.
  • Instead, we now are using LocalMomentaryDisplayLog, which is something that doesn't get sent with savegames, and which normally is just "stuff showing on your sidebar during the game for a little while."
    • This is perfect for the lobby, and in the lobby we are just showing the last 100 of these, and not clearing them until you reach the game itself.
    • These things are actually shown by timestamp, not by gamesecond (which isn't ticking yet), so that's just extra perfect for a lot of reasons.
    • This also leads to privacy for people who were talking before someone new joined -- you only see messages that were sent after you were connecting -- although that's not hugely important in most cases.

• Got rid of ArcenUI.Instance.CurrentNowUTC, since that was putting all of our lobby chat messages in the British time zone. We didn't use this anywhere else, and it was puzzling that we used it at all.

• NatPunchEnabled is now true on the server and client for LiteNetLib. We'll see how well that works.

• Added a new debug setting to the network section of the settings menu: Write Network World Serialization Logs
  • When sending a game world across the network to a client, write NetworkWorldSerialization.txt, and when getting one from a host write NetworkWorldDeserialization.txt -- both in the PlayerData folder (but on two different computers). The only real reason to turn this on is if a client is unable to join a host and gets an exception; this lets us do a diff of the two files and figure out what is different about what the host is trying to send and the client is trying to read.
  • Note these logs are often huge, sometimes hudreads of MB each. Compressing them into a zip or tarball and then uploading them somewhere we can download them to find out what is happening is a good idea. Hopefully as a player you will never need this, but this is basically a way for us to test the network world sync process, which otherwise can be incredibly opaque and take a lot of time to hunt down.

• Used our new network world logging capabilities to fix an error in network world transfer where it needed to send the game version in a different format.
  • The client now looks at the game version very early on, and if it does not match it sends a message back to the host telling the host what the problem was.
  • Basically: If the client can't join because of a version mismatch, that gets shown and things happen properly.

Milestone: World Sync Between Host And Client

• Milestone! Initial transmission of the world data from the client to the host now works properly.

• When a local player account is missing the following methods no longer throw errors:
  • SwitchViewToPlanet
  • OnClient_SendClientBatchToServer (actually still errors, because it's useful to have it do so).

• PlayerAccount is now more strongly linked to the new ArcenNetworkClientConnection, and its connected status is now a method called OnServer_GetIsConnected().
  • There is some duplication of concepts here, but when you factor in the way that things like viewing of planets are synced in multiplayer, this is a worthwhile separation of concepts.

• If two players with the same profile name try to connect to a game at the same time, it will now prevent the second one from joining and tell everyone why.
  • Bear in mind that these are AI War 2 profile names, not Steam or GOG or what have you. Everyone needs to have a unique AI War 2 profile name in the current game, but they can change up their names however they want and global uniqueness doesn't matter beyond the current game you are in.

• In situations where a player 3 joins after player 2, and it's the first time player 3 has been in this particular world, player 2 now gets told by the host about player 3.
  • And of course all the more complicated permutations of this. It's the new FromServerToClient_SendNewPlayerProfile.
  • This is needed specifically so that all clients can know which planets are being viewed by other players, so that which planets are "tier 1" processing is honored and identical everywhere. Among other later needs.

• Clients connecting to a host are now properly told what playeraccount they have been put in charge of.
  • This makes PlayerAccount.Local not null on their local machine, and lets them give orders and whatnot from their spot there.

• Errors that happen during the main sim step will now be more visible in multiplayer contexts.

Milestone: Initial Sharing Of Lobby Between Host And Client

• Milestone: Clients are now able to load into the lobby, and they and the host both see most (but not all) changes that are made.
  • Clients are not yet assigned to any faction (making them just spectator-mode for now), and there are some specific bugs with certain data that isn't synced correctly just yet.

• Map generation can no longer happen in the lobby except on the host.
  • We have no real way to be sure that map generation is deterministic between machines, so we need to do a world sync each time anyway.

• When the map is regenerated on the host during a multiplayer lobby, it recreates all of the playeraccounts needed for each of the player connections that exist.

• And when the map is regenerated on the host during a multiplayer lobby, it now re-syncs the entire world data to all of the clients so everyone is up to date.
  • This also solves the "authorized through frame number" issue, which was really just a smaller symptom of this larger issue.

• Both the client and host are now verified able to add, remove, and edit factions and have that properly affect the world.

• Window_ServerMultiplayerConnectionStatus has been removed, as we no longer use that. We've been using the new center-screen messages instead, or the chat sidebar, as needed.

• In multiplayer games, the save button is now disabled and says: Cannot Save Game As Client
  • If you click it, it gives you this message: Unfortunately, only the host can save games in multiplayer. To save on bandwidth as well as on processing power between your machines, there is a LOT that is calculated only on the host machine. If you were to save a copy of the game on your client's end, loading it would lead to a lot of wrong and missing data.

• Additionally, autosave is now something that does not happen on multiplayer clients. Just the host.

• The header of the load quickstart and load saved game menus now make it clear if you are loading for multiplayer or single-player.
  • Also on these screens is a new "[Multiplayer Questions?]" bit of text up at the top that appears only on the multiplayer version, and which has some helpful mouseover text.

• Rather than having an annoying explanatory popup to click through every time you open the various ways of hosting or joining a multiplayer game, it now puts that extra information under the "[Multiplayer Questions?]" section up at the top of those screens.

• At the top of the lobby screen, it now says if you are joining as a client or a host to a multiplayer lobby, as well as having that same "[Multiplayer Questions?]" bit.
  • This includes the following new text in its mouseover text: Except for the host, all players are spectators by default. There is only one human player faction by default. Any number of players can share control of a single faction, or each player can have their own faction that they control on their own. You can mix and match as you like.

• The connect by IP window has been completely reworked visually so that it matches the rest of the game and doesn't look like the old temporary style that we were using in 2017 (because that's what it was).
  • This also lets us put extra information on that screen to help anyone who might be confused.

• The client connections status window is now patterned as a continuation of the connect by IP window, and it's nice and graceful and in the new style.

Version 2.106 Immortals and Unresponsiveness

(Released July 23rd, 2020)

Bugfixes

• Put in field names for the risk analyzers saving, so if they have trouble serializing we can see which field it was.
  • Also then made it so that the total increase and net increase both are automatically clamped to 0 instead of a negative number so that they won't throw an exception when trying to save.
  • Thanks to GreatYng for reporting.

• When stacks are split, they now have their "time on planet" and "time alive" set to be whatever the original stack was. Same with the amount of cloaking points they have lost, and the number of cloaking points they have left.
  • This prevents a variety of abilities from triggering, including the "invincible if only on this planet for x seconds."
  • We also made it so that dying to remains does not reset the "time have been alive or on this planet" counter, for similar reasons.
  • This keeps the vanguard hydra heads from being insanely impossible to kill, among other issues that are probably pretty much all in the player's favor to have resolved.
  • Thanks to crawlers for reporting.

• Added some extra debugging information for the status of gamecommands that are queued, to see if there are clogs happening. At this point, we can't replicate any evidence of that, though we have some reports from multiple players of it. So maybe the problem either takes a bit to manifest, or is in a different area of the code than we expected.

• AIP should no longer increase when things die to remains if you have the "ships of X type die to remains instead of dying entirely" galaxy map option on.
  • This is untested, but should work.
  • Thanks to crawlers for suggesting.

• Found and fixed the core typo from a few builds back where remains that were being rebuilt were being set to have the cost of the rebuilder, not themselves. This was previously massively overcharging players for building most things, but then once we put on a band-aid to fix that, it started giving it to them for basically free.
  • Thanks to Lord Of Nothing for the report and save.

• Fixed a bug from the last few versions that nobody seemed to notice, where the galaxy map was not centering on planets properly when you first went into the game after loading a savegame. This had to do with when we were calculating the position of the planet, and that not always being calculated before we tried to center on it.

• Fixed a bug where it was possible that a client would send more commands to the host than it would tell the host about, and thus the host would not know to read the extra ones (or vice-versa). In single-player you are both the host and client, so it still applies.
  • This was happening because of multiple threads adding to the queues of commands at the same time that the queue was being drawn down.
  • This was actually something that probably could have happened since more or less forever, but it was substantially more likely in the last few versions because of some efficiency improvements we made.
  • Additionally, we have now switched to using a ConcurrentQueue for each of those collections, rather than a regular Queue. This should also help with potential contention.
  • There was also the possibility that we might not be able to get enough commands out of the list for some reason to then send more, and we are also now guarding against that.
  • Previously, this problem was manifesting as orders not being carried out when you gave them, sometimes or all the time. And things like "save my game" and "pause or unpause" not working anymore.
  • We were able to duplicate this in one savegame after some time, but no longer can. Given the intermittent nature of this sort of thing, we can't say for sure that it is solved -- but the logic of what was going on makes sense, and the fix we put in should hold that at bay. It's one of those "can't prove a negative" things, at the moment, so fingers crossed it just never shows up again.
  • The characteristics that we observed while this happens, if it happens to you and you want to see what you can tell us:
    • In the escape menu, under the memory pooling performance, the number of raw commands were steadily climbing. One or two every few seconds. This should never happen in that way. Increases happen, but not a steady drain flow like that.
    • In the escape menu, further down under the Commands Queued By Type, you may see some null warnings up at the top of that list, now. That wasn't there when we observed it, but it would be if it ever happened now.
  • Thanks to zeusalmighty, Puppet Master, Gdrk, and GreatYng for reporting and giving advice on how to reproduce it.

Multiplayer Work

• In our copy of FORGE networking, we've removed some useless extra code that was for the UniversalWindowsPlatform framework, which we won't be ever using.

• In our copy of FORGE networking, we also got rid of a very useless set of "pending" messages on the UDPClient and UDPServer, which were ostensibly to help with reliable messaging, but in reality was just doing some useless event handling and invocation, and some useless cross-threading locks. It just put the things in the basket and took them back out, never checking the basket or doing anything with it. Thankfully, there is a second basket for reliable UDP inside the UDPPacketComposer, so this was just extra redundancy.

• Added a new ArcenNetworkClientConnection abstract class, which we now store on ArcenNetworkAuthority in a ClientConnections list.
  • These are connections, not yet correlated to specific players. But it lets the network pipe connect up a specific connection with a specific player (or reject that connection if it's not okay).
  • Each individual networking framework (ArcenSocket descendent class) is actually responsible for keeping track of these connections and populating them.
  • But the actual network authority does the job of authenticating them into a given game and handling the challenge/response work, figuring out a match to a player slot in the game, etc.
  • With this in place, we can now get rid of GetNumberOfConnections(), because that's now moved out of the socket and into the network authority.
  • This is a substantial refactor, in that it will really make Steamworks and GOG a lot easier for us to work with as networking frameworks, faster.

• NetworkingPlayer in FORGE now inherits from ArcenNetworkClientConnection, and is the first version of a networking framework using this new pattern.
  • On NetworkingPlayer, rather than having a public { get; private set; } for some of the immutable connection things, we now use them as public readonly. This is both more efficient in performance (slightly), but also prevents sync issues between this class and its new underlying abstract class.
  • This class gives the information it needs to the base class, and doesn't bother registering the host NetworkingPlayer as an underlying client connection at all, keeping things simple.

• A bunch of stuff with the FORGE networking logging has been replaced with wrappers around our own logs, so that if there are internal exceptions those don't just go nowhere invisibly anymore.

• Trimmed out a bunch more old FORGE code that has been commented out for years from the look of things.

• RespondToNewConnectionAcceptedByTransportLayer() is definitively game code, not part of the actual networking transport layer, and it has been moved into the central codebase and away from any specific networking framework.
  • "Hello friend, you've connected: what's your profile name you want to play under?" Etc.

• A fair bit of the "initial chatter" of clients connecting to a host and the host responding are now logged to the debug log no matter what.
  • There really is not that much that is said, and this will certainly help someone at some point who is having some sort of connection issue.

• When clients are connected to the host, now the host not only shows the number of clients, but also sees their profile names that they have chosen to connect with.

• A bunch of event-driven stuff on the FORGE networking has been made a bit more inline, improving performance a bit and also reducing certain areas of complexity.

• The initial challenge-response work, and "what's your name" work, is all functional again. This is stuff that will be identical regardless of the network framework.
  • The host can now properly see the profile name of the client, and is poised to send the client back all the world information.

Version 2.105 Selection Hotfix

(Released July 22nd, 2020)

• The game still loads various xml files asynchronously, and loads icons and music and sound in that fashion, but it no longer tries to load the ships, shots, or wormholes in that way.
  • This was just too unreliable, because of some issues in the unity engine. It didn't affect all players, and not all the time -- but enough that it majorly affected the stability of the game for some folks, and that's too much.
  • This also removes the potential for the really long (15-30 second) lag time when you are first opening a new game or the lobby if things didn't load during the loading period.
  • Thanks to Ovalcircle for the most recent set of reports.

• Previously, the game was using a standard List<> object for keeping track of selected ships, and that sometimes ran into inefficiencies and cross-threading issues.
  • In the prior build, we tried to fix some of those cross-threading issues, but wound up making things not actually deselect properly.
  • To fix all of the above, we've now switched to the much more flexible and robust ArcenLessLinkedList<> object, and adjust the code to handle this. This is both more efficient (not that efficiency is a major problem here), and does not fall victim to the various cross-threading woes that the old style could.
  • And, naturally, as part of that, this also resolves the frankly infuriating selection issues in the most recent build of the game. Sorry about that one!
  • Thanks to ctl0ve, CRCGamer, Chuito12, and Burner for reporting.

• Previously, the tech menu only showed techs that would actually benefit a ship you have at present, or which you could capture.
  • This was very confusing to a lot of players, as they would not see the entire tech tree and thus think that something was missing or broken.
  • In some other cases, possibly due to changes in the recent past, you CAN build something (like a command station), but since you have not done so yet, it would not show up.
  • All of the techs now always show up, but the ones that are for ships that would not benefit you right now still show up as red and not researchable.
  • Additionally, the tech color shows up as white when it benefits ships you have, gray when it benefits ships that you don't have right now, and lighter gray when it benefits ships that you could capture.
  • Please note that since citadel upgrades don't benefit battlestations anymore, the citadel tech not showing from the start was working properly.
  • Thanks to Strategic Sage and CRCGamer for reporting the confusion.

Version 2.104 Negative Build Percentage Hotfix

(Released July 21st, 2020)

• Somehow or other, it became possible for some self-building ships to go very negative in their build percentage (instead of counting up to 100%, they would be way in the negative percentages of building process).
  • This seems to be a new bug in the last day or so, based likely around some changes that we made to improve the cross-thread reliability of the game. The problem is, upon manual code review of the areas that would be relevant, we just can't see anything that looks like a bug.
  • Ultimately what is happening is that the SelfBuildMetalRemaining is getting absolutely giant in value, far larger than the original metal cost of the ship that is constructing. This was probably an integer overflow from it going really negative first, and then wrapping back around, but it's hard to be sure.
  • We've put in some extra defensive code to make sure that if it goes negative at all, it marks itself as complete. We've also put in protections so that if you "owe more than it is worth" in general, that it will assume that there was an overflow and just go ahead and finish it now. These two changes should help to keep the problem from happening again, and fix the cases where it was already in progress.
  • While we were already at it, we put in a small general improvement that prevents you from being overcharged for the last tiny percentage of constructing a ship. If you had a bunch of engineers building, and each frame cycle they normally would do 1.1% of the construction of a ship, and charge you accordingly, then you might wind up in a case where you had only 0.2% remaining work to do, and yet got charged the full 1.1% rate. It is now careful to only charge you for what you would owe, so the 0.2% rate in this example.
  • Hopefully there are no other errors in any of the metal flows, with things like drones building, anything engineers would claim or repair, etc. The code looks clean, but then again the code for the self construction also looks clean. In the next couple of days, please let us know if you see anything else that seems amiss with how metal is being charged against you. So far we can't duplicate anything like that anymore, now.
  • Thanks to CRCGamer, deso, and Puppet Master for reporting.

• Some code from yesterday dealing with cross-threading protections let us unfortunately sometimes wind up with runaway DoForSelected loops. This is now fixed, and it has several kinds of self-repair in place to prevent this from happening now.
  • Thanks to Gunner for reporting.

Multiplayer Connection Work

• Reworked the NetworkTrafficLog.txt multiplayer output to actually be a lot more informative.

• GetNumberOfConnectsCurrentlyActive has been replaced with GetNumberOfConnections, which lets the network framework tell us a lot more about the general status of things happening.
  • Instead of just a generic number of players that exist, we can see how many are disconnected, how many if any are the host, how many are clients, etc, etc.

• In the lobby, the host now sees how many connected clients there are, and if there are any disconnected ones, how many disconnected ones there are.
  • Previously it just showed one number that included the host and also disconnected clients all wrapped up into one.
  • It still isn't detecting disconnects properly in FORGE, but we'll get to that.

• The host now only bothers sending heartbeat messages if it has at least one non-host client machine connected.

• Removed some old "desync detected" code that was not fitting with the new plans for multiplayer.

• Figured out some of why the client was not detecting itself as properly connected, and so was not getting messages from the host at all in the last few builds after the connection.

• The way that gamecommands are queued and then given out to player simulations for both single-player and multiplayer is now more efficient.
  • We're using queues now instead of lists, and this gives us a lower chance of accidentally losing a command, as well (not that we know of that ever happening before).
  • This has several performance benefits even for single-player when there are a lot of commands going around.

• Added a new FromServerToClient_AuthorizeThroughFrameNoCommands in addition to the FromServerToClient_SendNextBatchOfCommandsToExecute, to slightly save on space (1 byte per message) and make the logs clearer.

• The network log no longer has a time delay built in where it logs only 100 items at a time. It just immediately dumps items to disk.
  • Realistically there is only substantial traffic every 100ms or so anyhow, and the delay in logging was incredibly confusing as a programmer working with it.

• The network polling interval has been updated from 20 times per second to 100 times per second. In practice it will actually be variable, but the polling is extremely lightweight (the part that is on the main thread), and having absolutely minimal delays in getting and sending messages is what will keep things moving smoothly.

• To make network logging even more efficient and clear, the old ArcenNetworkLogEntry is now ArcenNetworkSendLogEntry, and is now a struct instead of a class. This runs lightning fast and doesn't affect RAM usage.
  • We also now are then logging data that is gotten in (which we call TAKE) along with data that is sent (which we call SEND).
  • This way we can see the conversation between several machines all interleaved on a single machine's log, and thus figure out why they are saying what they do, and if messages are getting there, etc.
  • Surprisingly, the way we were logging things previously really didn't lend itself to that, so it was easy to go down rabbit holes of wrong information (thinking data did not get to a machine when in fact it did).

• Furthering clarity on the networking side, we're now ignoring loopback messages-to-self on the host in our logs, which previously we were not doing.
  • This way we don't get drowned in a sea of stuff that "I'm telling myself in order to be the same as everyone else," and we can just see what is actually passing between machines.
  • The loopback stuff has been demonstrably working fine since more or less forever, unless we broke it today, since single-player relies on that.

• Fun fact, after all these logging changes, we can now actually see how well parts of the networking are already doing.
  • As one client connects, the host is able keep talking to itself and/or others without breaking stride, letting the potential connection wait for the challenge and acceptance (not yet reimplemented) before it loops in the new client to the rest.
    • This is a great example of how we can keep people from accidentally messing with the sessions of friends by trying to join during a game, etc. It's the sort of thing that only matters among friends, but it keeps it clean.
  • On top of that, we can see now that the reason for the host not noticing when a client disconnects is because it's ignoring the client that has not fully gone through the challenge-response process yet, and so the mystery of how it was "sending messages that failed but didn't flag the client as disconnected" is easily solved (answer: it wasn't sending messages to that particular client yet, because it is smart).
  • In general, by improving our logging quite a lot we can see what is actually happening and make sure all of this works properly. The vast majority of this is network-agnostic, which is doubly great. These particular logs will look substantially the same between FORGE and Steam and GOG.

Version 2.103 Hotfix

(Released July 21st, 2020)

• Fixed a bug from the prior version where the dark spire serialization fix was making any savegames with dark spire in them -- new or old -- not able to load.
  • Thanks to valinor000 and stanazolol69 for reporting.

Version 2.102 Digression For Quality

(Released July 20th, 2020)

• Using the "Increase Max Dyson Strength" hack will now also increase the Dyson's general mark level
• Some minor buffs to the Dyson's income, mostly at higher levels
  • Thanks to a discussion started by zeusalmighty

• When picking music (through the debug menu), the hovertext will now give you information on most of the tracks saying the original source

• Improved the speed of some dictionary lookups by precalculating the ArcenAssetBundlePath CombinedPath.
  • This also required us to change their member variables into properties, and in return that meant that the general string fill methods needed to be adjusted to new FillBundle and FillPath methods for xml reading.
  • This doesn't affect as much code as it sounds like.
  • We also adjusted the various debug logging points to use the combinedpath, which is cleaner and easier to read, but not much faster since that almost never happens.

• Fixed a bug that could happen with entity order serialization in rare circumstances if threads were fighting over putting items back in or taking them back out.
  • Since multiple threads do in fact often use orders, at once, this is one of the few collections prone to that.

• Fixed a variety of exceptions that could happen based on cross-threading bad luck during the exit of the game either to the OS or to the main menu, all in the metal expenditures code.

Ship Behavior Improvements

• Fix two problems related to shields recovering from being bumped by Astro Trains
  • Human Home Forcefield Generators can now recover from being bumped.
    • Thanks to Lord Of Nothing for the bug report
  • If a forcefield generator is bumped twice in short order, it will now do a better job of returning to its original location. This isn't perfect, but it's a pretty intense corner case
    • Thanks to Puppet Master for the bug report

• If there are only non-combatant enemies on a planet, ignore any requirement in the targeting code that particular targets need to be combatants. This prevents ships in FRD from discarding reasonable targets incorrectly.
  • Thanks to ParadoxSong for the bug report

• Actually support Targeting tracing now; you need to set Targeting tracing and the Debug setting that lets you set the PrimaryKey for the unit you want to follow, and you'll get some actual logging.
  • Not for the faint of heart; intended only for developers who want to get their hands very dirty.

Bugfixes

• Fix a bug where the 'delete campaign' button wasn't giving the campaign name
  • Thanks to OvalCircle for reporting

• Fixed a rare and harmless (but annoying) cross threading exception that could pop up with argument out of range exceptions in RenderShip.
  • Thanks to CRCGamer for reporting.

• Fixed a rare nullref exception that could happen in DoRemovalChecks() on shots, mainly a cross-threading thing.
  • Thanks to Lord Of Nothing for reporting.

• Fixed two possible spots in DoForSelected() where cross-threading issues could sneak in and cause an exception in rare cases.
  • Thanks to Lord Of Nothing for reporting.

• Fixed an issue when loading savegames that have a missing planet naming scheme. It will now default to the default naming scheme if it can't find the one that the world was started with.
  • Thanks to Oryutzen for a save that demonstrated this.

• Fix a bug where the imperial spire wasn't Watching previously explored planets for you.
  • Thanks to Lord of Nothing for reporting

• Fixed an error in TextMeshPro where it would still give us "Unable to use Ellipsis character since it wasn't found in the current Font Asset" invisible errors that would fill up the debug log even if warnings were disabled.
  • This was, in very long play, a memory leak as well as a performance drain, and it was generally invisible.
  • The ellipsis character was typically something it was trying to find specifically for purposes of showing cut-off text, and some fonts just don't have that in them. So no wonder we couldn't actually find the ellipsis character used in our own text files.
  • Thanks to Puppet Master for providing the log file demonstrating all this.

• Fixed an issue where the stationsRemainingBeforeDepot on astro trains could go arbitrarily negative, thus causing an error on save. It now stops at 0, and thus seems not to cause the issue anymore. But on the change that an astro train does have a data exception during save anymore, it will now log what field is having the problem and we can fix it.
  • Thanks to Lord Of Nothing for the report and save.

• The game has been updated so that the debug logging that is DoNotShow now ONLY goes to the ArcenDebugLog.txt, and not to the unity Player.log. The unity Player.log is kept in memory in a very unfortunate fashion, and in long gameplay this will be extra RAM used that should not be. Probably also extra slowness.
  • We added a new DoNotShowButSendToUnityLogEvenOutsideEditor Verbosity option that lets us have the old behavior of going to both of those logs, but we are not using that anywhere at the present time.
  • All of this is going to make it even more important that we have both kinds of logs from players if they have an outright game crash, now. But generally speaking, if it's an exception popup but not an outright crash, the ArcenDebugLog.txt has always been enough and will continue to be. It's only when the game crashes all the way to the desktop that we need the Player.log, typically, and now we'll just still need the ArcenDebugLog.txt in addition to that. However, any ShowAsInfo or ShowAsError logs to the main ArcenDebugLog will still appear in the Player.log, so potentially even that won't be true.
  • For a developer using the unity editor, it still logs everything to the actual unity log because that's for short testing cases and super useful to see output in realtime.
  • At any rate, we are trying to strike a balance between getting the logs we need, easily on the part of players, but without having any accidental extra usage of memory on very long play sessions.

• All of the dark spire per-unit data is now saved in a format where we can tell which field is having trouble if one does.
  • The dark spire data "lastTimeAttemptedLocus" was trying to save as an int16 and would overflow any time a game saved that was more than about 32000 seconds in. Fixed.
  • Thanks to RockyBst for reporting.

Memory Leak Fix From 2.099

• New setting on the debug menu: Show Pool Counts In Escape Menu
  • Used for searching for memory leaks, particularly between loads of a savegame. If pool counts keep rising after each load, then something isn't getting put back in the pool properly and is instead persisting lost in memory.

• ArcenGameObjectResourcePools must all now have a unique name, and they register themselves in a central place that identifies how many of them there are.
  • These pools also now keep track of how many objects they have ever produced from themselves, so that if we are leaking objects from them we can tell. And we can tell if we are leaking pools themselves.
  • After seeing all of these in the new debug menu option above, we can clearly see that none of these are leaking and they are all behaving just wonderfully. Hmm. But we have a leak somewhere, so let's expand this...

• Created a new CountedPoolBase abstract class that sits under the ArcenGameObjectResourcePool, and takes its counting capabilities and puts them there so that we can also use them for some other pool types.
  • This is also now used on a new "Shot Instance Renderer Pool Of Pools" so that we can tell if we are leaking there. And now ships and squads, too.
  • Also this is now used on LoosePool, which is the underlying basis for ship, shot, and squad visualizers (different from instanced renderers) now.
  • Despite all this, no memory leak was found in any of these pools. They are all working wonderfully efficiently.

• ExternalizedPool now inherits from CountedPoolBase.
  • Same for TimeBasedPoolBase (and thus TimeBasedPool itself).
  • And same for BasicPool.
  • And holy cow, there's some sort of memory leak with the time-based pools, looks like, based on this. That makes sense, as we recently reworked some of their functionality to fix a bug with them. Apparently we introduced a new bug at the same time.

• Added a new setting to the debug menu: Show Details Of Time-Based Pools In Escape Menu
  • Used for searching for memory leaks or other bad behavior on a certain very high-turnover series of pooled objects. If pool counts keep rising, or other numbers seem off, then we know we have a memory leak or a performance problem.
  • Thanks to Puppet Master, RockyBst, Lord Of Nothing, and NRSirLimbo for reports that led to us finding this.

• Fixed a MAJOR bug from version 2.099 that was leading to a memory leak in general, and performance problems as well (the longer you played the slower it would get).
  • Our entire "time based pool" logic was not actually properly processing because we forgot a single line of code, go figure.

Slow Load Or CTD Fix

• Since we started using the asynchronous loading of asset bundle items using unity (the last few weeks), there have been some intermittent problems with certain random items not loading in properly. This is some sort of funky bug in Unity, we're pretty positive.
  • We solved that problem a week or so ago by checking the async request's asset property, and if it was not invalid, forcing it to finish loading even though it was stuck.
  • On most machines, this was causing a really annoying lag of 10-25 seconds while Unity did whatever it was doing to finish that load. That was really annoying, but only happened the first time you loaded a savegame or quickstart or custom lobby after starting the program, and not every run of the program, and not on all machines.
  • More recently, we discovered that the very act of checking the async request's asset property would cause an unrecoverable error deep in unity and an immediate crash to the desktop.
  • Killing two birds with one stone, we're no longer checking the async request asset property at all -- thus avoiding the crash -- and instead just do a normal synchronous load when we detect a failed load. This leaves the one async request in limbo kind of forever, but that shouldn't be a problem; if it ever decides to complete, it will see that it was already completed and just do nothing.
  • In the meantime, not only should this avoid the crash, but it should also avoid that awful lag spike that happened for those for whom it didn't crash.
  • The downside is that on our dev computers we haven't been able to replicate the error at all today, so we can't verify that this 100% works. It should work, and you should see in the arcen debugging log slightly differently that says "PrototypeObject failed async load and so loaded synchronously for [some stuff] (this is not a problem, but is odd)"
    • If you're one of the people who were frequently seeing either the crash, or seeing the older "PrototypeObject fixed and loaded late for [some stuff] (this is not a problem, but is odd)" message, then we'd love to have a confirmation of you seeing the new message and all being well.
  • Thanks to Sol and RocketAssistedPuffin for reporting.

Version 2.101 Connection Status: Confirmed

(Released July 17th, 2020)

• Carriage returns and newlines are now supported by the "condensed" string format. This will prevent tutorial messages from turning those characters into question marks when they are dumped to the chat log. Note that this will only affect chat log messages logged in this version and on, not past versions.
  • Thanks to ParadoxSong for reporting.

• The left and right click functions of the metal display have been switched to be more natural.
  • Thanks to Waladil and Galian Gadris for suggesting.

• The "Spire Railgun Shop" mod that comes packaged with the game has been updated to use the icons from their new locations, and will no longer error.

• We've reworked our "Window_PopupScrollingColumnButtonList" a bit to add some new handy features for us.
  • In the factions selection window, when you are adding new factions, it now still lets you see factions that you can't add because there is only one per galaxy, but now it shows them in red, doesn't allow you to select them, and explains why in the tooltip.
  • In the debug menu in the escape menu in the main game, the change music submenu now shows the current selection in red (with the note that you can't change to it because it is already playing), and it shows all of the options sorted alphabetically except for the current track, which it shows as the first item.
    • Also fixed a bug where the music you selected to play would not actually be the track that played; it was choosing a random other track.

Scourge Updates

• The hovertext for the Scourge "Strength" in the game lobby now tells you what the strength does (more scourge ships/structures, and faster).

• Add a "Sandbox: Extra Strong Mode" for the scourge
  • This is not balanced for regular play, but is intended as a sandbox thing.
  • Thanks to Avenger1649 for suggesting

• Fix a bug where the scourge were periodically suiciding Defensive Fireteams into player positions.
  • Thanks to ParadoxSong for the save.

• On intensities >= 5, the scourge can create cloaked "blockade running builders" if the player has tried to blockade the scourge into a small region of the galaxy
  • Thanks to ParadoxSong for the suggestion

• On intensities >= 6, the scourge is intended to sneak a builder off to another part of the galaxy at the beginning of the game, to make it much harder for the player to blockade them in. This was not working on some map types, including the X type.
  • Thanks to ParadoxSong for the save that exposed this problem.

• These changes should make the scourge much harder to deal with in general, and on the X map type in particular.

Initial Multiplayer Connection

• AllowOtherPlayersToConnect was a setting on the World object in the past, and thus something that was getting saved into savegames and that dictated how a lot of things worked for player connections.
  • We've removed this concept, and instead are making it based on the menu choices you make on the main menu.
  • We now have a DesiredMultiplayerStatus on ArcenNetworkAuthority, which lets us recall if you wanted to be the client, a host, or just left alone in single player.

• The basic shells of SteamSocket and GOGSocket have been added to the game. The in no way function yet, but this will allow for them to be built out in the future.

• The SpecialNetworkType has been moved into ArcenUniversal, and is now copied onto ArcenSocket along with the InternalName of the row that created the socket.
  • There is now a list of AllPossibleSockets on the ArcenNetworkAuthority to allow it to swap between sockets as needed, and particularly by type.

• When you open the multiplayer menu, if you have that enabled, it will now set a default network framework for you based on the status of your game at that time.
  • If you are logged into Steamworks, it will set steam as the default network.
  • If that didn't work or wasn't true, and you are logged into GOG Galaxy, it will use that as the default network.
  • If none of those worked, then it will pick the first "other" network type, which in this case would be Forged.

• All of the options in the multiplayer menu for hosting a savegame, quickstart, or custom game now properly do those things identically to as if it was the single-player menu, but put you into host mode first.
  • The single-player menu puts you into single-player mode, and makes sure that your ArcenSocket is set to NullSocket.

• When you try to go into one of the client or host multiplayer areas from the multiplayer menu, it now properly activates your most recent chosen network framework, or switches to the first available one that actually can function right now on your system and gives you explanations for why it did that.
  • Essentially there is a fair bit of under the hood plumbing now for the frameworks to register themselves with the network authority class and explain about their availability.

• If you are looking to connect to another machine, then the "connect as client" tooltip now gives you information about what to expect based on your currently-selected framework.
  • Are we connecting by IP? How much of a hassle will that be? Is it going to be by selecting a friend on Steam or GOG? Etc.

• For IP-address-based networking frameworks, either starting a host mode or client mode event now pops up with a message explaining what information you need to give to the client players or get from the host.
  • This is not really the preferred networking method unless you're having a LAN party, and the game makes that clear, but it also goes out of its way to address the most likely areas of potential confusion for people who are using these frameworks.

• Fixed a few bugs, and added a bit more debug logging clarity, so that the host can once again properly open a socket and wait for connection.
  • This was working up until a week or so ago, and was simply messed up a bit when we reworked things to support multiple networking frameworks. There were lots of lonely hosts never getting any client requests ever since the game has been in public beta.

• When you quit out of the multiplayer lobby back to the main menu, it now immediately disconnects you as a host or client, to avoid confusion of lingering connections.

• When you are in the lobby in multiplayer, the chat sidebar/log now appears again. At the top it says if you are hosting multiplayer or a multiplayer client.

Choosing A Networking Framework

• The main menu has been updated so that the "network" button at the top of the multiplayer menu now updates its text to show the current network.

• The network button at the top of the multiplayer menu on the main menu is now fully functional. It shows available network frameworks, and gives you explanations for each one, as well as showing unavailable ones in red with an explanation for why they are not available right now. It lets you select from available ones, which then get set as your current socket when you try to host or join a multiplayer game.

What Is My IP Address If I Need It?

• In the networking section of the settings menu, it now has helpful informational displays of your local IP addresses and your public IP address.
  • To get your public IP address, we have to make a public call to the url http://checkip.dyndns.org, so this may flag your software firewall to ask permission when you go to the networking tab now.
  • Our goal is of course for you to not have to use your direct IP addresses at all in this game, but if you're playing on a LAN then that would be one great example of when you'd want to use this.
  • At any rate, each IP address is stored in a textbox that you can copy-paste from, which is very handy when sharing with friends.
  • There are also lengthy explanations about what the public IP address is for, and under what circumstances to use it, as well as the local IP addresses, plus special notes for any IPV6 addresses that it detects.
  • Getting this data is time-consuming for your computer, so you will notice micro-lags every few seconds while you are on the networking tab, now. This is normal and should not impair your use of the functionality.

Milestone: Successfully Connecting A Client To Host

• The old interface for network connections by IP address is restored and works again. We'll have a different one for connecting via friends list on Steam and GOG, later, but this is the one that gets used for Forge and any other IP-based network frameworks.

• Connections are now able to be established from one computer to another, for the first time since 2018!
  • We added the ability to see how many clients there are connected to a host in the lobby (including the host), so you can see the connection tick up.
  • Next up is them actually exchanging enough information to get the client into the lobby and have them see shared world info and be able to chat and all that. But this is milestone one!

Version 2.099 Last Rabbit Holes

(Released July 15th, 2020)

• There is now a galaxy setting to grant players Watch vision for 1, 2 or 3 hops from all command stations.
  • Thanks to uhamster9 for inspiring this addition.

Fixes And Tweaks

• Improved the way that time-based pools are incremented in time, so that we're never having accidental cases where we miss one.
  • Also made it so that they actually work as intended when the world is being cleared (running 16 seconds' worth of time at once). This then keeps the pools smaller if you repeatedly load savegames.

• On the escape menu, under where it shows you the various numbers of objects in memory, it now shows you the number of galaxy planets and planet links activated and in existence.
  • This lets us tell when these have a memory leak, if there ever is one with them.

• Additionally, galaxy map links are now pooled for the first time ever, as the discarding of them was previously a definite memory leak. Normally this just happened between saves and loads, but it was also possible to happen when nomad planets moved.

• In addition to the existing time based pools, we now have a new ExternalizedPool.
  • This is now used for the galaxy map links, and actually we have stopped the galaxy map planets themselves from using the super-old IArcenGameObjectResourcePoolable and they also now use this.

• The way that planets for the galaxy map are initialized, and the way that their positions are set, is completely overhauled.
  • This is more efficient and properly uses pooling (which never was working properly before, it turns out).
  • It also makes the position of planets automatically react to them having moved in the game sim, without having to do anything special.

• The arcencolors asset bundle has been removed, with its contents simply rolled into arcenui. Ultimately this is a bit faster to load, and saves a bit of disk space and RAM also.
  • Also removed the aiw2squads bundle, and the examples bundle.
    • The examples bundle simply has its example content still there, but not in a bundle (it's to help aspiring models modders).
    • The squads bundle is old data that we've not used in forever, and so has just been cleared out. It wasn't huge, but wasn't worth being there.

• Starting to load a savegame (including for a quickstart or the last settings for a lobby) now writes to the log when you start the process, and at the end of the process writes how long it took.

• Added a new debug setting: "Write Detailed Savegame Timings"
  • When loading a savegame, including the 'last settings' for the lobby or the basis for a quick start, keep track of detailed timing data and write that into the normal debug log so that it's clear what is taking longer and shorter amounts of time.

• It turns out that chasing "why savegames now take 16 seconds to load sometimes" was a snipe hunt. The serialization sizes logging, when enabled, actually was causing that amount of slowness. With that off, it loads in about 2 seconds.
  • The extra instrumentation that we added for the savegame timings is still nice, but ultimately was not a useful bit of time spent at the current moment.

• Fixed an issue that could happen in the lobby in particular where it could not properly save your prior settings because the fleet ID or speed group ID was lass than zero.
  • This pretty much could only happen if you already had another error first.

• Fixed an exception in SeedNormalEntities that could happen if you rapidly tried to regenerate maps in the lobby in just the wrong way.

Under The Hood Improvements For Icon Modding And RAM Usage

• The last of the icons from the ExternalIcons folder have been moved into the unity project that generates the asset bundles, simply to dispel any potential confusion about the fact that they can be edited directly and that have some change on the game without recompiling asset bundles. This was a frequent modder confusion.
  • That said, the ones that were in the Official_1 folder are now in a CentralIconBits, and can now be used in the UI for the first time if we ever want to. That means things like health bars and whatnot could in theory be used in the ui if someone wanted to, whereas before they could only be used on gamespace-level icons.

• Cleaned out a variety of unused icons from the arcenui asset bundle, and in general tidied up some of our organization of working files versus final files.
  • Also standardized the naming of certain things.

• Broke the "official icon dictionary" out into six dictionaries:
  • One is for the "central bits" like health bars and so on, and would be the same for all ships.
  • Two are for the overlays, like for saying what kind of starship or guard post something is. There can be more than one of these, and the fact that these are split out like this demonstrate the modding capabilities and how things can combine.
  • Three are for ship icons and their borders, and there can be more of these modded in as well. As with the overlays, not only does this demonstrate how such mods would work, but it also has the side benefit of slightly less VRAM usage in a few cases.

• The way that GUI icons are discovered from the xml based on their gamespace-icon counterparts is completely revised, and is now based on extra information in the ExternalIconDictionary entries.
  • This is a lot more flexible, and allows us to load icons from multiple asset bundles and/or multiple files within the same bundle.

• The naming for finding icons is now much more complicated, in the sense that they don't all come from a single dictionary called "Official."
  • So there were over 1100 places in the base game and expansions where we've had to modify to point to the new dictionaries. Any mods would also need to be updated to point to the new places, unless they want to start using their own icons (once we put together a tutorial for that).

• The external sprite dictionaries have been updated to be able to properly load in in multiple threads.

• The game is now able to load the xml files for external icons from any mod or expansion, in:
  • Expansions/[ExpansionName]/GameData/ExternalIcons/[thefile].xml
  • XMLMods/[ModName]/ExternalIcons/[thefile].xml
  • XMLMods_NonDistributed/[ModName]/ExternalIcons/[thefile].xml
  • It's worth noting that these are xml files that are generated by Texture Packer, not xml that we create.
  • These files are referred to in the "ExternalIconDictionaries" xml as something along the lines of xml_path="Official_CentralIconBits.xml"
    • It will then search the main game's folder (/GameData/ExternalIcons/[thefile].xml), then all the expansion folders, then any activated mod folders for a file with that name that was specified.
  • This allows for mods to be self-contained when it comes to their icons, and it allows expansions to have their own icons that are not packaged with the main game (not that we have imminent plans to do that).

• On the icons used in the game, we previously were not using any compression, which made them absolutely massive in size (80mb for the main dictionary).
  • This was done in order to preserve quality, but we're also using GIANT icons in order to allow for really super-high-res displays of the future, as well as square ones to allow for ideal mipmaps, etc. So the lack of compression was hugely overkill.
  • We're now using compressed textures, which turns what was 80mb into more like 8mb. This has a very direct impact on performance and VRAM usage, so particularly on low-spec machines they will likely run far better.

• The game now automatically constructs custom "gimbal materials" for the in-gamespace icons, in as many combinations as are needed for the things that it has drawn for you so far.
  • The total number of materials always in the past was "one," but it was using a single large dictionary and a lot more VRAM -- and it had a finite amount of space for things. There was a solid chance we were going to run out of space in that one dictionary as part of implementing DLC2. But either way, mods were not possible. The new number right now is 8, looks like.
  • Now it may make a handful of materials, depending on how many mods you have installed that have custom icons, and several for the core game and expansions itself. But these are vastly smaller, and while this does increase the number of "draw calls," each material is still mostly instanced together (there are a few mesh differences depending on if there are health bars shown or whatever), but the overall load on the GPU pipeline is lower.
  • All of this is automatically handled by the game in as efficient a pattern as possible, and if you're curious how many combination materials it has created, you can see that in the escape menu at the bottom of the list of memory pooling info.

• A new debug setting, "Log Gimbal Icon Material Creation", has been added:
  • When this is on, any 'gimbal icon' material creation events will be logged as to what was created and how long it took to do so.
  • Turns out that this uses an immeasurably small amount of time for each material (less than 1ms each), so that's awesome.

Under The Hood Improvements For Mods With Code

• The game is now able to load dlls from mods or expansions, rather than just from the central game folder.
  • This works just like the external icon files, and basically looks for the dll in the central game folder first (/GameData/ModdableLogicDLLs/[thefile].dll), and then looks at expansions and then installed mods if it can't find them.
  • The paths are:
    • Expansions/[ExpansionName]/GameData/ModdableLogicDLLs/[thefile].dll
  • XMLMods/[ModName]/ModdableLogicDLLs/[thefile].dll
  • XMLMods_NonDistributed/[ModName]/ModdableLogicDLLs/[thefile].dll
  • We really don't have a need to do this with expansions, but the flexibility is nice.
  • With mods that are more than just xml, however, this finally lets a modder distribute just a single folder that has everything in it, and not have to worry about putting some things in central game folders.
  • This is only partially tested, but should work.

• It's worth pointing out that expansions and mods already did (and still do) have the capability to load asset bundles (icons, music, sound effects, models, textures, shaders, etc) from their folders.
  • The structure for those is /GlobalBundles/ in the main folder for anything not platform-specific, /AssetBundles_Linux/, /AssetBundles_OSX/, or /AssetBundles_Win/ for the things that are.
  • Inside the folders for expansions it is: Expansions/[ExpansionName]/[OneOfTheAboveFolders]/
  • Inside the folders for mods it is: XMLMods/[ModName]/[OneOfTheAboveFolders]/ or XMLMods_NonDistributed/[ModName]/[OneOfTheAboveFolders]/
  • None of this is new, but it's worth mentioning for now.
  • This is only partially tested, but should work.

Larger Gamespace Icons And Fixed Galaxy Map Line Offsets

• We also now have a non-billboarding version of our shader for purposes of our icons on the galaxy map.
  • Our core shader does on-GPU (read: hyper efficient) billboarding to always fully face the camera. This is very useful in the main view where you are moving around the camera a lot but always want the icons to face you.
  • The downside, however, is that with a largely-straight-overhead camera like the galaxy map uses, this billboarding would cause there to be a perspective shift where the icons for planets would get offset from the lines leading between planets no matter what we did. It also caused them to move relative to the text that was next to them.
    • This offsetting of the lines to the planets was one of the largest "visual papercut" issues that we've had for a really long time, and it is finally fixed!

• The ship icon scale has been adjusted from a default of 1.5 to 2.2, and will affect all prior settings files.
  • It's description has also been updated for the personal settings tooltip: For the icons in the main display area (not the sidebar), how large should they draw? Default is 2.2. If you go too large, it can be hard to see things because they overlap too much. If you go too small, they can get extremely blurry because of flipping to a lower mipmap. The larger your screen DPI, the smaller you can go without losing clarity.

Prior Release Notes

AI War 2: Building Multiplayer