AI War 2:Building Multiplayer

Next Release Notes

AI War 2: Sunset of The Badger Era

What was this phase all about?

Once we had the first expansion under our belt, and the game was feeling large and solid at the equivalent of "first AI War plus four expansions," we were ready to tackle the next big thing: multiplayer!

What wound up happening during this particular phase, however, is perhaps best described as "measure twice, cut once." A whole lot of things that would have come up to bite us during multiplayer alpha and beta were instead dealt with during this phase. Perhaps chief among those was a dramatic restructuring of our data for savegames and for multiplayer messages, to keep network bandwidth and processing low while allowing us to pass around the massive amounts of data that we have in mind. Rather than waiting until we had a problem, later, we preemptively dealt with that in a way that makes things smoother even for single-player folks.

While Chris was mostly working on multiplayer prep, plus some base game fixes, Badger was doing the bulk of the work on the second expansion, Zenith Onslaught. That will eventually launch around the same time as multiplayer, and by the end of this phase, Badger's work was mostly complete.

Version 2.097 Below Minimum

(Released July 13th, 2020)

We wrote up how the AI works, if you'd like to read that.

• The way that savegame deletion worked was really confusing, as it put you into "delete mode" and then let you click specific items.
  • It was done this way because, when this feature was added, we didn't have the ability to select a savegame without loading it immediately.
  • Now it works like "delete campaign" does, and just deletes the selected savegame or tells you to select a savegame to delete it.

Multiple Player Profiles

• In the Other submenu on the main menu, the Change Profile button has been altered to have this tooltip:
  • Switch which profile to use, or edit or create a new profile. When connecting to a multiplayer game, the name of your profile must match the name of the player slot in the game.

• The main profile button on the main menu that shows you current profile name also now has that same tooltip.

• Both buttons now open a new Profile List window.
  • You can view all the profiles, delete ones you don't want, add new ones, and edit existing ones.
  • Previously you could just edit and create a single one, but if you renamed it you would get a new one.
  • Being able to have multiple profile names is particularly important for us as developers when testing multiplayer savegames, when we want to go in as a specific player.

• Player profiles can now be 30 characters, not just 10.

Fixes and Tweaks

• The game now goes out of its way to throw exceptions on load if there are calls to create async asset requests from background threads.

• When you load any savegame, the game now immediately does its check for any sort of missing visuals and should show errors about any and all of them that exist, rather than it waiting until you would be using them and then throwing errors every frame.
  • This lets you know sooner if there is a problem, and makes the per-frame drawing checks more efficient as well as not spamming the log if there is something invisible.

• The async request logic has been updated some so that if for some reason the "completed" event has not fired, but the asset is not null, then it will try to latently load the object.
  • This is one of those "things that should not happen," but we have it directly from a recent log from Lord Of Nothing that this was happening (and on windows, so it's not just a platform-specific thing), so we now have some code in place to handle it.
  • Overall this looks like some sort of bug in unity, because AsyncProgress is 0 when it should be 1, but it's vaguely possible that it's simply loading the asset so fast that the event handler simply didn't get assigned. Since the asset does seem to be present, we're just working around that. If the asset is present but in an invalid state, then this will still fail, but knock on wood it should be okay.
  • Thanks to Lord Of Nothing for the latest report.

• The status of the mods and expansions can now be seen during gameplay, but can't be altered. The tooltip notes that you have to go to the main menu to do that.
  • Additionally, when you make any changes to the mod or expansion statuses, it now pops up a yes/no prompt that either makes you not commit those changes, or makes you close the game and then restart it yourself (so that overlays will hook in properly).
    • This prevents any confusion with things seeming to be disabled, but not really being disabled, etc.
  • Thanks to ArnaudB for reporting.

Ability To Run The Game Under Minimum System Requirements

• The following settings have been moved from the Display tab to the Performance tab:
  • Skip Drawing Ship Models
    • The description has been expanded to also include: If you are below the minimum system requirements, this can actually let you play the game at full speed. If you restart the program with this enabled, then it won't even bother loading these ships into RAM, thus even further lowering the requirements (and making loading faster if you're on a really old machine).
    • Please note that if this setting was on when the application was started, then no ships will be drawn until you turn off this setting and then restart the application.
  • Do Not Show Shot Visual Effects
  • Do Not Show Explosion Visual Effects

• The new functionality for Skip Drawing Ship Models now actually works, in terms of doing far less loading of ships, and not causing a bunch of RAM usage when it is on from game start.
  • This is pretty key for Chris's two MacBooks, as both are below the minimum system requirements for the game but he wants to use them for multiplayer testing.
  • This gets loading for the second machine down from 40s to 20s. Compared to 9s down to 7.5s on the main dev machine.
  • Chris's linux machine is a plenty fine beast in terms of hardware, so it should be a good mix for testing.

Version 2.096 Space Engineering

(Released July 10th, 2020)

• Fixed several places in SetSpriteIndices() where there could be nullref exceptions if threads disagreed at just the wrong times (one tearing down while one draws).
  • Thanks to Badger for reporting.

• Improved the speed of generating two of the types of sidebar objectives (hack for ship type, hack for ship line size increase), and also included extra debugging for cases where they may have exceptions so we can fix them if they do happen.
  • This doesn't fix any exceptions, but will make it a lot less destructive to the rest of the program if one happens; the one in GenerateShipLineCapIncreasingStructureObjectives was probably a thread race condition where the sidebar was examining a ship that was in the process of dying.
  • Thanks to Fayte for reporting.

• When several fireteams are going to attack, the game makes them all move at about the same speed, so they don't trickle in. Previously, ships always moved at AboveAverage6, which could be really scary for some usually lumbering units.
  • I have modified it so that the fireteams instead move at their overall average speed (plus a bit). This makes fireteams still an intimidating foe, but a bit less crazy-fast.
  • Thanks to GreatYng for the bug report.

• Rather than throwing a scary error message when it can't load your player profile, it now silently logs the detailed error (just in case) and instead gives you a far friendlier popup that says this:
  • The player profile data you had previously could not be loaded. We changed formats on that in March of 2020, and that's probably all it is. Apologies for the inconvenience! It just had your chosen display name and default player color.
  • The game will now ask you to fill in those again, and you should be good to go indefinitely after that.

• Fixed a rare exception that could happen in GetIsWithinRangeOf().
  • Thanks to Lord Of Nothing for reporting.

• Fixed a couple of bugs that were causing uncaptured fleets to show as zero strength. These were related to the many fleet strength calculation reworks that have happened in the last few weeks.
  • Thanks to Admiral for reporting.

• On the main menu, under the single-player menu there is now another line item for "tutorials."
  • This was already on the right-hand side of the screen, next to the tips and details over there. But it could do with being a bit more obvious.

• While we were in the UI asset bundle, we went ahead and got the credits for DLC2 in there (thanks Badger for putting those together), and corrected Chris's name to his new one.

• Fixed an issue from the last week or so where you could not properly build the AIW2ModdingAndGUI asset bundles because of them missing some dlls.
  • This was not something we noticed, since it's rare that we're in there, and even more rare that modders or similar are in there.

• Split the staff and kickstarter credits out into two separate windows.
  • They now each load faster, and we can show the base game and expansion credits in two columns, now.
  • Additionally, for the kickstarter credits (both columns), we fixed a bug where apparently you could not quite scroll all the way down to the bottom of either one. Yikes!

Ship Intelligence Improvements

• When a crippled flagship retreats to a planet with FRD engineers, engineers are now able to prioritize repairing it over doing anything else; typically that's the first priority the player wants.
  • Thanks to Keith for this fix, and thanks to corfe83 for the bug report

• There is a new method FindStrongestEnemyPlanetOrNull() that lets us look for the strongest enemy planet for a faction.
  • In the absence of an AI king to go fight, the outguard will now pursue the strongest enemy planet when it is in attack mode.
  • In the absence of any strongest planet to go fight, the outguard will now just patrol friendly planets.
  • Previously, if there were no AI kings left on the board, the outguard was explicitly told to just chill out, which was definitely strange-seeming.
  • Thanks to GreatYng for reporting.

• Back in May, we made an adjustment to no longer have decloaking ships work their magic against units that CAN cloak, but which are not currently cloaked.
  • This was an enormous performance improvement in areas where there were tons of decloaking lines that needed to be drawn, most of them to units that were already decloaked anyhow.
  • But this did cause an unintended stutter to decloakers, however, where ships were being allowed to re-cloak intermittently.
  • Now it has been altered to keep doing the under-the-hood decloaking the entire time, and thus remove the stutter, but it only draws the decloaker lines against ships that it has not yet decloaked.
  • Thanks to DEMOCRACY_DEMOCRACY and GreatYng for reporting.

New QoL

• Left clicking on the Metal Income icon will now give you some stats for units you've killed (Right clicking shows the old flows data). This is a sneak preview: the UI is definitely not where we want it to be, but it's cooler than having nothing.
  • Thanks to Mckloshiv for reporting and to others for a discord conversation.

Multiplayer Frameworks (MP Todo Items 2 and 3 of 9)

• Previously, the game only supported a single networking framework, which could be swapped out via the "external constants" xml.
  • This was flexible in the sense that it would let you add in a completely new network handler, which is great in some respects, but the lines of what was "part of the network framework" versus "part of the game logic" were not really drawn as well as they could have been. Too many edge bits that were actually part of the game logic were included in there.
  • That previous linkage has been completely removed, and there is a new NetworkingFramework xml directory which populates a table that supports any number of networking frameworks.
  • The three that we are starting to focus on are Forge, Steam, and GOG. Previously we just had Forge in there.
    • Surprisingly, though, we also are including a new Null option, which basically ignores networking stuff in single player mode.
    • In the past, even in single player games we have always opened a port on the local network via Forge, in theory allowing people to connect even if there was not room (this being only on your local network, however).
    • This sort of "always open a port at least locally" logic goes all the way back to AI War Classic, but the interface for this game is distinctly broken out into single-player and multi-player sections, unlike the first game, so not opening a port when we don't have any reason to is actually a nice thing. It wasn't a security risk or anything like that, but it's just... cleaner this way. And possibly very slightly faster.

• As part of the move to support multiple networking frameworks, we had to completely restructure where the line is between "logic that is the same for passing data and such across any network" and "code for managing a very specific network but not knowing what it is actually passing around."
  • The concept is that basically we want to redo as little logic as possible between Steam, GOG, and Forged -- and that if later someone wants to mod in a new or alternative network library, they are not having to duplicate ANY game code to do so.
  • We previously had a bunch of logic handling (talking about the content of messages) mixed in with the network sockets stuff (talking about how a message gets from here to there), and that has been split out completely differently now.
  • A lot of pieces of code have been moved to a new ArcenNetworkAuthority class, which exists in ArcenUniversal and collects things that were previously scattered around various places.
    • This then has its "UniversalMessageHandler", which is for handling the general content of messages that are like "hello, I'm a client who would like to connect" and "okay, I'm a server, here's your client ID," etc. This stuff will be the same regardless of network pipe in place, and is just part of the core framework. All of this code goes back to 2017 or even before, and we'll be testing it out over the next few weeks to make sure it still works.
    • It also has its "GameSpecificNetworking", which is basically for all the AI War 2 specific stuff like "here's some commands to execute" or "here's sync data" or whatever. This also has existed since 2017, and most of it is probably fine although we know there are a couple of problem areas.
    • And then finally it now has a new ArcenSocket set as "ActiveSocket" on there, and this is where we start wrappering whatever the "network pipe" is that we'll be using.
      • We have a new "Null Socket" that does nothing, as described above, and this is what is now used by default in single-player games. It works just fine.
      • We also have our existing "ForgeSocket" for Forge Networking Remastered, which is based on code from 2017 and 2018. It has a lot of custom bits in there, and they've evolved their framework fairly substantially since we last pulled from their repository. But it doesn't look like their changes actually fundamentally benefit the sort of low-level networking we do, and in fact after pulling from their git repository a couple of times and trying to reconcile things, it seems like their stuff is almost entirely focused on higher-level networking and the low-level bits we care about are identical in all meaningful ways. So for now we're going to stick with our old version unless problems arise.
  • We started looking around for other lightweight "Reliable UDP" networking packages, which could now be implemented via ArcenSockets kind of at-will if we need to.
    • Right now, we will probably just stick with the three we already have planned (Steam, GOG, and our old version of Forged) unless there are problems with Forged.
    • However, RevenantX / LiteNetLib has caught our eye as a very distinct possibility, since it matches a lot of what we're looking for in terms of being lightweight, fast, and native.
    • We did look into the various versions of ENet CSharp, but all of those wrapper native C/C++ libraries, which we're hugely inclined to avoid using. Steam and GOG do the same thing, but when it comes to something not using a large external framework like those clients, we don't see a reason to do that.
    • We also did look back into the Lidgren Networking Library, which is what we used on all our older games that have multiplayer (AI War Classic included), and there have been some substantial updates to that since we last had our own fork of it. But there were some bugs that we had to work around with it back in the day, and we don't know if those are still there, and the library has not been much updated in the last decade, so we'll avoid for now.
    • Overall we expect people to mainly use either Steam or GOG anyhow, since that handles NAT punchthrough and relay servers so brilliantly in both cases, and is just easier. So we're really only talking about for LAN play or people playing over VPN or people who really don't want to use one of those two larger services to help them connect. For now, Forge ought to serve those folks really well, and we've got our list of potential alternatives mainly for if something collapses there.
    • A really big goal for us is simplicity -- we are passing around literal big blobs of binary data, which is much lower-level than what pretty much every networking library wants to help you do. The benefit for us of doing it this way is speed and cross-compatibility, but it also means that most every networking library other than the ones we've listed here (and including the newer versions of Forge) has a TON of extra features that bloat them up and that we'd never dream of using.

• Our network library is now part of our main compiler chain (A_MonoBuild_All.bat, etc), and we no longer track the resulting dll and pdb directly in svn.
  • For anyone not using the windows version of this (aka Badger), they will need to be sure to add this to their own compiler chain or else they won't be able to see the game run anymore.
  • This whole bit just makes sure our stuff is as up to date as possible at any given time, while saving a bit of space in our svn repository as we make future changes.

• Added back in logic for client and server heartbeats, which we had taken out long ago and just never put back in.
  • These keep the game alive in the absence of anything else happening.

• Some rather complicated but not-needed logic relating to the "most recent history" of messages between clients and hosts has been removed.

• New setting added to the Network section of settings: Network Debugging Mode
  • Will majorly slow the game down, but dumps a bunch of networking info to the disk as part of the ArcenDebugLogging.txt file in the PlayerData folder. Data is brief, but written as it comes in or goes out for the game.

• New setting added to the Network section of settings: Log Network Traffic To Disk
  • Will majorly slow the game down, but dumps a huge amount of networking info to the disk in the NetworkTrafficLog.txt file in the PlayerData folder. The data is exhaustively complete, and written in batches after every 100 messages.

• New setting in the network section of settings: Enable Multiplayer Alpha
  • Multiplayer is not yet ready! But we are in the process of coding it in. If you turn this on and mess around in the multiplayer section of the game, you will almost assuredly have a very bad time.
  • We will update this message as multiplayer progresses, but right now this is just here so we can build out the framework while still giving you single-player updates and fixes.

• The "multiplayer is coming" message that you get if you have NOT enabled the multiplayer alpha has been updated to be more concise and have the latest info.

• In the multiplayer menu, it no longer just says "host new game," but rather says "host custom start" or "host quick start."
  • The existing "host saved game" and "join game" both remain as well.
  • Also in this menu is a new button that will let you choose what kind of network you'll be using to either make yourself available for connection or connect to others over.
  • Fair warning none of this works yet, it's stuff we have to hook all up again. So enabling the alpha to try these out is not going to work just yet. ;)

Version 2.094 Devourers Everywhere

(Released July 8th, 2020)

• Added a DoForEntities_DebugOutput( string tag ) method for letting us find out what is going wrong when something goes wrong.
  • Used this to find a major oversight in the new entity search by tag code in the last version, which basically made it all not work (we thought we tested it and it was okay??).
  • At any rate, we were getting zero results ever from tag searches of the new kind, so it was doing things like adding the devourer over and over again to the galaxy.
  • Thanks to Mckloshiv for reporting.

Faster Rollup Addition and Removal

• GetNumberIn() no longer takes a faction that is required, because it is inherently always from one faction.
  • Same for all the various versions of GetFirstMatching().

• There is no longer an EntityCollection at the faction level -- that was duplicative of the Entities list on PlanetFaction, and caused all the rollups and calculations to do twice as much work for definitely not enough benefit.
  • This made entities being created slower to add, entities moving between planets slower, and savegames slowr to load. Also maps slower to generate.
  • Now, instead, the game keeps track of all the child entity collections and loops over them very efficiently, and made it not much extra processing for overall much lower rollup load.

• SquadCountsByType was almost never used, and has been remove.
  • GetNumberIn() has been extended to make it possible to use this same functionality.

• There is no longer a generic "count" on entity collections. It now is broken out by squad, shot, and other.
  • This makes some AI math actually correct for the first time ever, although it's rarely-used AI math.

Version 2.093 Hotfixes and New Player Guidance

(Released July 8th, 2020)

• Fixed faulty integer math in the code for lighting up too dark colors for factions, such as various Zombie types and the Devourer Golem. Text written in their colors should now be visible much better.
  • Thanks to NR SirLimbo for fixing.

• Previously, it was possible for the predicted benefits of a tech upgrade to try to tell you about a higher mark level than the max mark level of the unit.
  • Most units have a max mark level of 7, and if that was the case then it would throw index out of bounds exceptions when you c-clicked the upgrade, and generally break.
  • For units with a lower max mark level, which are rare, it would tell you about a benefit that the unit would never actually get (it would be forced to stay at a lower mark level despite the upgrade).
  • Thanks to GreatYng for reporting.

Reactive Guidance For New Players

• Start adding some 'Beginner' journal entries to catch some places where a player might shoot themself in the foot. So far the cases are as follows:
  • A player is trying to take too many planets early (the check is 'if the player has captured the majority of planets they've defeated the defenses on'), the player is reminded that this is a game of guerrilla warfare.
  • If the player has easy access to >= 2 ARSs but hasn't hacked them, the player is reminded that hacking ARSs is very good.
  • If the player has < 10000 metal and hasn't upgraded their home command station or Metal Generation, they are reminded about those mechanics for increasing metal production.
  • If the player has captured at least 4 planets and has no GCAs then the player is reminded to capture some GCAs (and the entry notes that hacking GCAs will keep them safe)
  • If the player has had flagships repeatedly crippled and hasn't used science to upgrade them much, the player is reminded they can use science to make their flagships tankier
  • If the player hasn't used Load Mode more than 3 times in the first 30 minutes of the game, remind the player that Load Mode is a thing.
  • If the player has captured 2 planets but left some warp gates adjacent to their homeworld, remind the player that Gate Raiding is a thing.
  • If the player hasn't been spending science, remind the player to do so
  • If the player hasn't built 10 turrets by 20 minutes into the game, remind the player to build turrets
    • The goal is to make it a bit harder for novices to bounce off the game by giving them real-time feedback on things. More entries will come as I or others have the time or inclination. Note that these messages can be disabled in game via a setting, but I suspect that most experienced players will never hit those conditions.
    • Thanks to Starkelp, ANGRYABOUTELVES, Mac and CRCGamer for suggestions thus far

More Efficient Searching For Entities

• A variety of data inside EntityCollection is no longer initialized until is actually needed.
  • There were thousands upon thousands of data lists being created in large games with many factions where they often were never filled at all. Now it should fill them just as needed.
  • This should speed up loading a savegame or starting a new game.

• As there has long been a EntitiesByRollup, there is now a new EntitiesByType.
  • Both are now lazy-instantiated as needed, having a very low footprint overall.
  • They have also both been made private, as the wrapper of GetListOfEntitiesByRollupAndAddIfMissing() and GetListOfEntitiesByEntityTypeAndAddIfMissing(), which handle the lazy-loading.
  • There is also a more direct GetListOfEntitiesByRollupOrNull() and GetListOfEntitiesByEntityTypeOrNull(), which return null if the list is not present. But either way, it's very clear what will happen from either code path.

• EntityRollupType.SpecialTypes has been removed, as any usage of it would have been definitely inefficient at this point.

• DoForEntities( SpecialEntityType SpecialType ) no longer uses the internal rollip of EntityRollupType.SpecialTypes.
  • It instead uses an inner nested loop of RowsBySpecialType, which sorts through far fewer entities, combining that wuth the new GetEntityByTypeListOrNull().
  • Same for GetFirstMatching( SpecialEntityType SpecialType ).

• DoForEntities( GameEntityTypeData EntityType ) is now vastly more efficient under the hood, now just using the GetEntityByTypeListOrNull() rather than the list of ALL entities in that context.
  • Same with GetFirstMatching( GameEntityTypeData TypeData ).

• GetAllRowsWithTag() is now GetAllRowsWithTagOrNull().

• EntityRollupType.Tagged has been removed, as any use of the rollups based on this would have been VERY inefficient. And adding them to a pointless list like this is really pointless for our purposes now.

• GetFirstMatching( string Tag ) has gotten the same treatment that its equivalent with GetFirstMatching( SpecialEntityType SpecialType ) did.
  • This is incredibly more efficient compared to before.
  • Added a new DoForEntities( string Tag ), which is highly efficient and a brand new thing.
  • ALSO added a new DoForEntities( params string[] Tags ), which has a different order for the tags (after the processor because of the params keyword), but lets the programmer specify multiple at once.
    • This is highly efficient compared to the alternatives. but not really as efficient as it could be if you're going to be calling GetHasTag repeatedly inside your processor.
    • But this does give us flexibility, and compared to looping over all tagged entities, or all entities in general in a faction or whatever, it's not even close.
  • Versions of both of these have also been put on the planet and world objects, to match how we would use the entity rollup DoFor on those objects.

• Added GetCountFromListOfEntitiesByRollup() and GetCountFromListOfEntitiesByEntityType(), so that we can do null/zero checks very very efficiently where needed (such as in the gravity planning code -- yet more efficiency, although comparably little here).
  • Also helps some on tachyon planning, tractor planning, and metal flow planning. But again, the help here is miniscule, overall. These were already about as efficient as they can be in this particular area.

• Improved the use of GetHasTag() in a variety of places:
  • In guard post placers during mapgen, for guarding metal generation spots (aka faster mapgen).
  • Finding NanocaustBeacon is now vastly more efficient.
  • Finding the Devourer on the "summon devourer golem" is now vastly more efficient. It's not exactly a common case, though.
  • GenerateProgressReducerObjectives() is now far more efficient, and this runs next to constantly on the main thread.
    • Ditto GenerateEngineerRelatedObjectives() and GenerateBeaconObjectives(). And the scourge spawners and armories. And spire debris, research labs, and citadels. And spire archives, tech vaults, zenith power generators and matter converters. And devourers. And things you can hack for ship types. And exogalactic wormhole warnings. And instigator base warnings, and alerted eye warnings. And devourer warning, zenith trader warning, astro train warning, enraged harvester warnings,
      • Same style of logic improvement for the AI fleet looking for warden bases. And exogalactic wormholes. And astro train stuff. And various dark spire stuff. Huge amount of dark zenith stuff. And fallen spire bits. And various instigator stuff. Tons of macrophage logic. Some mercenary logic.
    • Ditto dyson antagonizers, although they use the new params version. Same for IGCs. And antagonizer warnings.
  • Did not bother adjusting marauders, since they are undergoing a giant revision.

• Added a new EntityRollupType.GrantsStuffToPlayers, which lets us vastly more efficiently find things that are either GrantsStuffToBeAddedToPlayerCommandStations or GrantsStuffToBeAddedToPlayerFleets.
  • The CalculateAndSortShipsThatYouCanCapture method now is much more efficient, thanks to that. Previously it was iterating over every unit in the game with any tag.
    • Same for the science sidebar on each tech shown there. This was surprisingly heavy of a check to do.

• The rollup GeneratesBonusHunterShips has been removed, as this was something that was used to narrow down the field of targets, and then have a secondary tag search. Now it just does a direct tag search.

• Updated the nanocaust a minor bit with the new logic, but they couldn't benefit much from it.
  • Same with nomad planets.
  • Risk analyzers got more substantial benefits, though they don't really have that many units.
  • A bunch of adjustments made to the scourge to make those more efficient with these methods, although there's still a lot of complexity there.
  • A few things for architrave and miners.
  • Nothing can really be done of substance to tutorials.
  • So that's everything!

Hotfixes Related to 2.090

• Previously, if you had the spire railgun shop mod enabled but did not have DLC1, then it would throw errors on game start that were impossible to bypass.
  • Now it throws more friendly errors and doesn't outright die. This should help in a variety of situations where there are missing dependencies between mods and each other or dlc, etc.
  • Specifically if sidebar categories are requested that can't be found, then it tells you but doesn't die outright.
  • And if visuals are requested that can't be found (either their bundles can't be found or whatever else), it doesn't die outright, either.

• All of the various faction difficulty levels, including the AI and its sub-components, now read in the old way, in linear xml order, rather than in parallel threads.
  • We couldn't duplicate the issue of the levels being misordered, but were not surprised to see that that was happening.
  • Thanks to Mckloshiv for reporting.

• The game now is more explicit about what has happened if there are missing "prototype" objects from asynchronous load failures.
  • It has some extra debugging info, and tells us how many have loaded versus how many failed, etc.
  • Additionally, at all times in the escape menu it now has the number of async load fails. Which should always be zero, or it's a problem. It shows up with a warning color in there if it's nonzero.
  • Thanks to mountainamoeba for reporting some issues on their machine.

Quick Start Loading

• Fixed an issue in the new versions of the game where any quickstarts that had expansion factions in them (typically for beacons) were no longer able to be loaded by people without the expansions.
  • The game now properly just discards those expansion factions from the quick start. This has been a hard thing to maintain, but goes back to us not being able to selectively disable expansions when we were originally creating the quick starts.

• All list-style xml attributes can now have [attributename]_ignore_errors="true" set on them in order to avoid any exceptions that would happen from missing or malformed data in them.
  • Turns out this was a dead end and we don't need this right now, but it's useful to have in the future just in case.

• Prior to the recent changes to how we deserialize worlds, we had some logic in place for "if you can't find an entity to load as, then just load as a VWing" or something similar.
  • This was a way to load incompatible savegames, but mostly it was a way to load them as quickstarts -- aka, loading a quickstart that has some DLC or mod content in there that your current configuration of the game doesn't have installed or active.
  • We now have brought that general sort of logic back, although ONLY for purposes of loading quick starts. When loading a proper savegame, it should fail and tell you the content is missing, because otherwise you're just asking for strange trouble. A quickstart is different because it's about to regenerate the world and toss all the entities out anyway.

• UpdateVisualObj on Planet objects now catches exceptions when it tries to draw a planet and gives a debugIndex for useful debugging. This both prevents the game from stalling out when the data is bad, as well as gives us the information to fix whatever is wrong at a given time.
  • This area has been very rare for errors but it has come up prior to now.
  • Put in a bunch of fixes to places where the display mode could be null if the game only partially loaded properly, thus preventing error loops.

• In some of the various loading code that we have for "mod or dlc is not installed," we show a nice clean message without stack traces to the user.
  • This is great, and is new and is a lot more clear to players. But when there is an actual problem (like the quickstarts thing), we need to actually see those stack traces and similar. It now does the full stack trace in the log, just not visibly to the player.

• Fleet memberships now use the same sort of "load as a vwing if we can't find the type we're talking about and this is a quick start load" logic that game entities now do.
  • This was never done before, nor needed before, but with the stricter rules (read: more correctness, and better for multiplayer, under normal circumstances) on deserialization, it is now needed.

• Three other places that were deserializing entity types now just don't frankly care if they fail, since they have little consequence if they do.
  • This prevents quick starts from failing to load if they had some extra dlc or mods around when they were saved, but also slightly malformed savegames (of which there should be none, but there's no worth to this particular distinction here).

• A smaller subset of quick starts couldn't load because of not finding an entity system by name. These are also now fixed to not care.
  • In the end we've had to load every quick start and make sure that they all function.
  • By now, all but one of them did. The Buggalactic War quick start had one more error that was an old form of viral shredders being present in the "drones that get built" part of fleets code. Things have been updated to not care about that, too, so now all of the quick starts load properly again.
  • Thanks to Glowsun and others for reporting these issues.

2.091 Multiplayer APIs Move Pt 1

(Released June 6th, 2020)

This was briefly in beta, but then we confirmed it's working on linux and OSX, so we're good to go.

• The error "Null ArcenXmlElement in AllElements, somehow!" no longer will ever pop up. It wasn't actually an item of concern, but is more of an occasional curiosity of threading.

Code Reorganization For MP Todo Item 1 of 9

Referring to the todo list here: https://steamcommunity.com/games/573410/announcements/detail/2501135802570960554

• All of the AI War 2 and ArcenCore projects now use .NET 4.7.1 rather than 4.6.2 as their framework. This enables some new language features that still were needing but lacking for certain external libraries.

• Additionally, support for Visual Studio 2015 is now officially being dropped for our developers and modders.
  • Only Visual Studio 2017 and later has support for all of the C# langauge features used the various external libraries that go as far back as ArcenUniversal.sln, now.
  • It's recommended to use Visual Studio 2019, as that compiles faster anyhow and has all the latest features.

• The UnitTest projects in the ArcenUniversal and ArcenAIW2Core solutions now compile again.
  • We used to use these for CPU usage profiling and memory usage profiling, but it's really not needed anymore. We have better ways of doing that sort of thing now, but having the unit tests compile is not a bad idea.
  • It's worth noting that at the moment they still would not execute correctly, but we have literally no reason to do that, so it's no big deal.
  • And... they're just a pain in the rear. After getting them compiling, they still waste our time and energy. We've removed them from the solutions for now. These were vital a few years ago, but since the advent of better built-in tools they've just become obsolete.

• DiffLib has been moved out of the central AI War 2 project and into ArcenUniversal.
  • So have DifferController and DynamicLinq.

• LINQ_Any, LINQ_ToList, and LINQ_Where have been removed from the "front end link" and presentation layer classes.
  • The version of .NET in use now allows for direct LINQ calls.

• ReplaceMacro, GenerateDiff, and DiffStringArrays have all been removed from those same clases.
  • ReplaceMacro now exists as ArcenStrings.ReplaceMacro in ArcenUniversal.
  • The other two are on the DifferController class in ArcenUniversal.

• Facepunch.Steamworks has been updated to the latest version from their git repository.

• Fixed a minor issue where when we were using the unity editor for internal development, it wouldn't allow us to log into GOG galaxy.

• The steam redistributables have been updated to the latest versions for windows, linux, and osx.

• The facepunch steamworks dlls have all been custom-compiled now into two separate folders (Posix and Win), with their dlls being the same name now (Facepunch.Steamworks.dll) rather than two different dll names.
  • Now a single dll of ours can link against either one of their dlls and then it will work in a platform-agnostic way.
  • This is how GOG Galaxy does it, which is better, and it's also how the old Steamworks.NET wrapper library did it. Also better.

• A version of GalaxyCSharp.dll is now in ReliableDLLStorage\ForLinkingOnly, but this ONLY for linking against, not actually for executing.
  • The originals are still in the unity project, and are required to be there so that they can handle their COM Interop properly.
  • There are different versions used there depending on the OS, so that they will work properly with any supported OS.
    • But the external exposed methods are identical on all of them, so we can just compile against the Win64 version and have it work fine on OSX, etc.
    • That said, since Chris is developing on Win64, it's better to go ahead and use the OSX version so that if there were any problems it would become apparent immediately on the primary development machine.

• All of the members of the ArcenGOG namespace have been moved into ArcenAIW2Core.

• The ArcenGOGWrapper has moved with them.

• So has TestingArea.

• So has ArcenSteamWrapper.

• Same logic for the GOG Galaxy dll is now applied to Facepunch.Steamworks.dll, and that works now that it has been renamed.

• There is now a new ArcenGameControllerBase that is in ArcenAIW2Core, which ArcenGameController inherits from in the main unity project.
  • As much logic as humanly possible has moved from the original class to the new base class.
  • Really, there is very little left in the unity project, now, out of all the stuff that we used to have to call from within there only.
  • This will let us rapidly code against both the Steam and GOG APIs with very short compile times (seconds) now instead of 11+ minute compile times.

• All that remains of item 1 is to get the Forge Networking Remastered code back in there, and the network loop (though technically doing both of those things will also complete item 2 off the list).
  • All things being equal, this first step has gone a bit better than expected, knock on wood.
    • We do need testers on Steam on linux and osx to verify that they still get their username showing in the bottom right of the main menu, but there's no reason to believe they would not.
    • We also need to test on GOG galaxy on osx to make sure that the username from THERE shows up in the bottom right corner of the main menu, but once again there's no reason to believe it would not.
    • The approach we took here was to make windows the odd one out in terms of platforms, and it works, so the other should as well.

2.090 Returning From Beta

(Released July 3rd, 2020)

• RowIndex on ArcenDynamicTableRow is now renamed to RowIndexNonSim. It is also no longer included in the object dump files that we use for detecting differences that may or may not be appropriate after code changes.
  • Essentially these RowIndex values are something that are stable while the game is running, but not between runs or between clients in multiplayer.
  • Thanks to good habits and programmer education, this hasn't been a problem in the code so far. But this rename of the variable did force us to re-evaluate all of the places where that field is used, and it also makes it a lot more obvious what is happening and why as more modders get involved.

• Improved the dump code to no longer include a number of fields that could validly be different between runs of the game.
  • This gets rid of any "noise" that prevents us from seeing actual unintended changes that happen between versions of the game, which is the whole point of the dumps.

• Fixed an issue where the order of certain data table subsets could be inconsistent between runs of the program.
  • This probably did not affect the game, but did affect the accuracy of our dumps.

• Improved the consistency of sorting for tables that use a sort order, but use that sort order for categories rather than individual items.

• Fixed the order of the colors for team choice being a bit off in the last few versions. The order in the document actually matters for these, so we can't process those in parallel.

• We now have confirmed consistency between runs of the program in terms of things being the same each time despite reading things in in a parallel fashion.
  • There were a variety of lists and things that were still not quite right, previously.

• Rows internally no longer have their CopiedFrom set to themselves if they aren't copied from something else. Instead that field is null in those cases.

• It is now possible to use icon_overlay="" on a copy_from entity in order to blank out the overlay on the new entity.
  • Thanks to NRSirLimbo for suggesting.

• Fixed a longstanding and possibly-trivial issue where when you had one entity copy_from another, it would invert the order of any child nodes.
  • So if it was Gun1, Gun2, etc, then it would be Gun2, Gun1 on the second entity.
  • Most entities have only one gun, so that was not very visible on th UI, but it was misordering all their various other systems like cloakers, tachyon or tractor beams, etc.

• When there are multiple levels of copy_from, the game now does a more thorough recursive copy to the entire chain, with both attributes and child nodes.
  • This has zero impact on the existing ships in the game, which is what we had hoped for.
  • However, there were many funky errors that could happen to suitably complex chains of copy_froms, mostly in mods, but not limited to that. These are all finally fixed!
  • Thanks to NRSirLimbo for going so far as to even create a demonstration mod that broke in many glorious ways until now.

• Fixed what appears to have been a relatively old bug (somehow?) where the max strength of a lot of fleets was not being properly calculated. It seems like this may have slipped by for a few months, or since a bit before DLC1 somehow.
  • Also fixed a bug that probably goes back to the start of fleets, where instead of the current strength of fleets on the fleets sidebar, it was showing their total strength. For command station fleets, it was showing a max that they could not even hit, too.
  • Thanks to jkistre13 for reporting.

Beta 2.089 Fireteam Declarations Of Victory Before Showing Up

(Released July 1st, 2020)

• Fix a problem with my 'Hunter ships going after specific targets' code that was causing A. way too many ships to be generated and B. a depressing memory leak
  • Whoops! This was spotted by ArnaudB, Metrekec and tadrinth.

• Fix a fun where the Nanocaust's ships would be unable to decide between targets; they would get partway to one target, then turn around as if going to the other target. This looked like a Buridan's ass style problem with the targeting code.
  • Some judicious sleuthing though eventually figured out it was a Fireteam problem, where Fireteams were incorrectly declaring themselves winners of a battle without showing up.
  • After fixing this, I then uncovered a bug where my "Nanocaust isn't allowed to rebuild immediately" code was preventing them from building at all.
    • Thanks to the almighty zeus for leading me on this voyage of discovery.

• Added a new setting in the debug section: Dump All Entity Tag Data
  • When the program is loaded, or after the xml files are reloaded into the in-memory data tables, write EntityTagDataDumps.txt in the PlayerData folder.
  • The purpose of this is to figure out what might be wrong if you are getting unexpected results as you mod things with tags.

• Fixed a bug from the last few betas where we were not properly checking the attributes on all of the xml nodes that we read in.
  • This had to do with a mistake in our use of Parallel.For().
  • There wasn't actually any bad data of this sort during this time, as it's only been a few days, but it's good to have it working again.

• In addition to doing validation on xml nodes for unrequested attributes, the game now also looks for unrequested child nodes.
  • This helps us to identify xml that is valid structurally but malformed in how it is logically supposed to exist.
  • In this particular case, we had already discovered that ArchitraveCrupellarii and ArchitraveCrupellariiSpire were not being read in properly because of an end-entity tag in the wrong spot for ArchitraveCestus.
    • Previously this sort of error was just silent and it was hard to find unless the units misbehaved or were clearly missing. Now it errors on initial load of the game, to make it clear that these need to be fixed.
  • Happily, nothing else was incorrect in terms of child nodes in the xml, so all is well now.
  • Thanks to zeus and Badger for reporting.

• Fixed some potential divide by zero exceptions that could happen if ships were being shot at with certain damage bonus combinations. This would either happen with malformed ship data, or most likely the case in question here was a bonus against "percentage of remaining shields" for a ship with no shields.
  • Thanks to NRSirLimbo for reporting.

Beta 2.088 Banishment of Undead AIs

(Released June 30th, 2020)

• Fixed a case of the GameSettings directly referencing RowsByName, which should never happen. Things should no longer error out if settings are missing.
  • Thanks to Oryutzen and Badger for reporting.

• Added a new PermanentSerializationIndicesForThisWorld Dictionary<string, List<string>> on the world object, a bit earlier than we were planning on it.
  • This is stored from in SerializeAllAllInternalNames and similar, and should be kept forever. These are not ever replaced during the life of a savegame and its descendants. Instead the things are simply added to.
  • This was already going to be needed for multiplayer, but in the last few days we've realized it's also needed for another reasn for single-player.
  • This central dictionary allows for two critical features:
    • 1. GameCommands to use efficient non-string-based indices when passing data back and forth in multiplayer, which otherwise they cannot do.
    • 2. Single-player games and multi-player games to use consistent indices from here to store the data for "external data," which otherwise can get scrambled.
  • This also means we now have a new NameForIndexLookups on ArcenDynamicTable. This in turn means that without some other code changes, a table cannot be renamed if it is using ByIndex serialization, or it will cause all older saves to stop working. There are ways we can get around it if we ever have to, but there should be no reason for that.
  • And then also this means goodbye to the old rowNamesByOldIndex on ArcenDynamicTable, as this is now wrapped into PermanentSerializationIndicesForThisWorld instead.
  • SerializeAllAllInternalNames and DeSerializeAllAllInternalNames are also gone, and things are now handled centrally by World, based on a new list of tablesThatAreReadyAndAreSerializedByIndex.
    • If you are a modder making your own tables, please be sure to only use SerializedBy.Index for SerializedRules if you actually are going to use that functionality. Otherwise it just bloats every savegame with your mod for no reason. Very very few tables actually need that flag to be true.
  • The complexity of this particular rework made our heads spin a bit, but it's a major pillar of efficient and accurate multiplayer, so it's very exciting to have it done. The fact that it will also prevent certain kinds of mod errors is a bonus.

• RowsByName is no longer directly accessible on arcen dynamic tables. You instead need to call one of the three GetRowByName versions, each of which behave differently but correctly depending on which behavior you need.

• Fixed several different compiler chain issues that could lead to copying a silly number of dlls to the GameData\ModdableLogicDLLs folder, and in a couple of cases could then also lead to us not getting all of the code actually where it needed to go.
  • One result of this was in the prior beta version, there were some MethodNotFound exceptions because outdated versions of the external dlls were in place.
  • Thanks to relmz32, marty651, and ANGRYABOUTELVES for reporting.

• Fixed a bug introduced in version 2.082 that made it so that any game with the nanocaust in them would not be able to deserialize.
  • Also added extra serialization logging info to nanocaust while we were at this.

• dropdown_sub_type is an ancient concept that was used for only four fields in the settings window, but which was always really brittle and hard to use.
  • For some reason, in the last few versions it has stopped working, and so you couldn't set the fullscreen size of your game.
  • Time for a replacement with something more flexible, to fix this while we're at it.
  • There is now dropdown_filler_dll_name and dropdown_filler_type_name, and these must point to a new IDropdownFiller (which works a lot like the IInputActionHandler handlers on input actions, as one of many examples).
  • The existing four fields have been converted over to this, and they all work brilliantly in terms of being something that modders can now add their own of, and being easier to understand in general.
    • However, it did not solve the fullscreen saving problem at all. Huh.

• Added a new do_not_set_my_values_directly_because_managed_by_another_field that is now set to true on the invisible FullscreenWidth and FullscreenHeight fields.
  • This prevents them from accidentally overwriting the values that the fullscreen resolution dropdown is setting on them.
  • The fullscreen dropdown now once again functions properly.
  • Thanks to ANGRYABOUTELVES and Gdrk for reporting.

• Fixed a place where the game was spamming some errors if you were playing with the fallen spire on, multiple AIs, and one of the AIs was dead. It was acting like the death of one AI king was a bug in data, when in reality that is a very reasonable thing to have happen. The dead AI is unable to launch extragalactic forces, now (it already was, but it was spamming your error log hard during that previously).
  • Thanks to GreatYng for reporting.

• It turns out there were some sort of edge cases where the AI king could be dead but its faction would not have gone through the "we have lost" logic. We don't yet know how that could possibly be possible, but we now have a safety check from then on in the future that makes sure that even if the kings death was somehow missed when it happens, it now recognizes that it happened in the next few seconds and will mark the faction as defeated.
  • This lets you actually win the game in events where that was going on, and properly turns that AI against the other AIs if that's the setting that is on, etc.
  • Thanks to GreatYng for the save that had a dead AI that somehow didn't know it was dead.

• Fixed a very rare race condition that could happen in LazyLoadSquadWrapper.GetSquad(). Essentially one thread could be marking it as dead and thus the reference goes away, while another thread was still trying to read its info. By using local variables this should now be properly threadsafe, and it actually let us remove a try/catch block, which improves speed of access slightly (actually the part inside that try/catch was getting hit more frequently prior to now, but we were using that to suppress it in a nonideal way).
  • Thanks to Kalieum for reporting.

• Fixed a SUPER longstanding bug where it would sometimes write "Was looking for a wormhole (some stuff) but couldn't find one."
  • This was causing certain orders to not be followed by some ships, usually in the hands of the AI or another faction. It generally self-corrected, but the invisible error log spam was not welcome and it may have caused larger problems at times (it's hard to say).
  • It turns out the issue dates back to when we started pooling EntityOrders a few years ago, and a single value that wasn't getting properly discarded on wormhole transit orders.
  • Thanks to a few people for bringing it up over the last three years, although we can't seem to locate those reports at the moment.

Beta 2.086 Tell Me About The Zenith, Voice Lady

(Released June 29th, 2020)

• The game now has a Notification for when Dark Spire Loci are warping in
  • Thanks to GreatYng for requesting

• Savegames from the last four beta branch versions (since 2.080) may or may not load for you, apologies for that.
  • Some of them were already not loading, and were loading very odd data onto units and factions, but now new savegames should work properly again (and all older savegames are still fine).
  • Essentially, ExternalData was getting saved in the wrong fashion. There is still some brittleness here that doesn't exist in any other part of the game, and we will see if we can fix that separately so that this never happens again.
  • Thanks to relmz32, zeusalmighty, and Badger for reporting.

• The way that external data logging is handled in the serialization logs is now a lot more clear, so we can easily go in and search for things that may be funky.

• Fixed a bug where off-by-default mods that were supposed to be auto-included were not actually being included.
  • Thanks to Lord Of Nothing for catching this oversight!

• Improved the logging of errors when you try to load a savegame that has a missing mod or expansion in it.
  • There was a regression in the clarity there two versions ago.

• Fixed an issue with loading some very old savegames or quick starts.
  • Until the recent betas, the quick starts were loading fine, and things were regenerated anyhow so it made no difference.
  • But the savegames were loading fine, but then having waves or reinforcements that would not happen because of a missing AI Ship Groups (that we intentionally took out but now were recreated temporarily and then were just empty).
  • Having the data reads be more strict has let us find this problem and then fix it so that both cases now work perfectly, and the new-style AI ship groups (that have been added and are much more varied and detailed now, hence the removal of some old ones) automatically slip in on those older savegames that might have been wave-less for some planets prior to now.

• The performance of GetHasTag should be vastly better now.
  • We now have a TagsList and a TagsDict on each GameEntityTypeData, as well.

New voice lines

• Tentatively add support to some new voice lines for Zenith Trader (ZT respwawns, ZT ready to trade, ZT destroyed) and Devourer (player gets first vision of Devourer)

• Add support for the voice line 'marauders attacking neutral planet' and 'nanocaust hive hack finishes'

Beta 2.084 Freedom From The Compiler

(Released June 26th, 2020)

• Significant nerfs to the hunter fleet bonus income against MDCs
  • Thanks to ANGRYABOUTELVES for the feedback

• Cross planet waves against a specific faction now will stay focused on that faction (and not go off and attack the player)
  • Hopefully this will fix some weirdnesses NRSirLimbo was seeing.

• The "brighter color" hex and color for faction teams now uses new logic by -NR-SirLimbo to make sure that the minimum brightness is 300 (out of the max of 765).
  • We previously had logic in place to try to help make sure that faction names were always visible in text, but in many cases where they were almost black, they would still just be invisible.
  • Huge thanks to -NR-SirLimbo for the implementation!

Exporting Game Data To Examine It

• Our object dumper code is now fully functional, although it takes a fair bit of hinting on a variety of tables to make it so that we don't get things like recursive outputs that spiral into multiple GB of data for one text file.
  • The two main hints that are used are the new custom attributes [NotForDumping], which should go over anything that we don't want to see at all in the dump files (working variables, things that are redundant, things that would lead to an infinite loop, etc), and [DumpInternalNameOnlyForArcenDynamicTableRow] for ArcenDynamicTableRows where we want to show just the InternalName from that field to know to what we are referring, but not include all the contents of what we are referencing.
  • Overall this lets us have a very complete picture of all the data that we're reading in from xml, and what final form it has taken after all the copy_froms, is_partial_records, and just general math that happens during load.
  • We have our own short term needs for this, but this is a feature that we think modders in particular may find useful in the future if they are wanting to make a change to some data and then see what cascading effects, if any, came from that change.

• Put in some changes to no longer initialize EVERY big list on all of the game entity types, but instead to just initialize them as-needed. This makes game entity types a bit faster to generate on initial load, and also cuts down on their size with some useless bits on data dumps.

Whole New Code Compilation Pipeline

• Okay... so we've completely changed how we compile our code for the game.
  • We've done this a few times before, but the most recent time was roughly a year ago when we switched to the more modern Roslyn compiler.
  • In the past, regardless of which compiler we used, we have always been directly compiling our code via having the compiler look at the raw source files and build them from there.
  • This has proved extremely problematic with certain pieces of code, most notably LINQ, which is used in a lot of libraries and we've had to work around the lack of that by making a hacky way for libraries to call up into our main unity project (which is compiled by unity) and not have it in our external dlls.
  • This was absolutely going to be untenable for multiplayer, and so this issue finally had to be fixed so that we can properly link against the various network libraries that we will be using. Overall turnaround time for each piece of work, not to mention just plain-out compatibility, depended on this.
  • Badger has actually been compiling the game himself on linux using his own custom scripts, since our windows ones don't work on that platform, and what struck Chris was that he was literally using the csproj files to directly do the compilation.
  • So, at this point we have now switched over to something that also follows that pattern. We are now using MSBuild behind our windows scripts, and that parses our csproj files as needed and calls the nice modern Roslyn compiler for us, basically getting us out of dependencies hell.
    • Unity, it's worth noting, has dozens of copies of their mono dlls, for various platforms and purposes, and it's super hard to figure out what is what and make it all fit together. We were able to get that working with most everything but LINQ, but that one aspect was the big one that has been elusive for the last three years.
  • So the problem is solved! This opens up a ton of doors and a whole lot of easier ways for us to link our code together with the networking libraries and similar. The amount of head-smashing-into-walls this has been for first Keith, and then later Chris, for years and years is something we are relieved to have behind us finally after one last long session of that.
  • If you are a windows developer or modder working with our internal build scripts, you don't have to do anything new. It all "just works."

• We also took this chance to clean out a bunch of old script files that really are not needed or used anymore.

Beta 2.083 External Data Hotfix

(Released June 26th, 2020)

• Fixed a bug in the last few versions (since parallel load) that was causing exceptions in trying to cast to external data for various factions.
  • We were not initially seeing this much, for some reason, but StarKelp was running into it internally with his own mod doing it intermittently. Now it's happening regularly after we put in some enforced sorting on these rows.
  • For now we've just fixed the problem with the existing data, but this whole thing exposes an underlying weakness in the way that we link external data into the game. If people later are playing with multiple mods and change their mod order, they will potentially wind up with this bug without us being able to fix it (as far as their existing campaigns go). If both mods both have external data on the same units, which is actually not the majority case at all. So it's probably an outside chance of it happening naturally in playing. At any rate, it's on our list of things to refactor sometime after multiplayer, to be a bit more flexible in general, just in case.
  • Thanks to StarKelp, ANGRYABOUTELVES, and NRSirLimbo for reporting.

Beta 2.082 Are Wormholes Visible To The Naked Eye?

(Released June 25th, 2020)

• The Nanocaust and Dark Spire shouldn't be able to rebuild a Nanobot Center or a Locus for at least a few minutes after a previous one on that planet was destroyed

• Fixed a one-line bug in the last couple of versions that were causing wormholes to be invisible.
  • Also fixed an oversight that could let something be invisible without throwing an exception that the player can see. If that ever happens again, it won't just be silently wrong.
  • Thanks to zeusalmighty for the report.

• For some reason, the tech upgrades were never using their proper sorting information to get themselves into the correct order. They were simply using the xml order, which was problematic for anyone modding in new techs. And now that we read these into the game in parallel, they were in effectively random order. Two fixes with one discovery, at least!
  • Thanks to ANGRYABOUTELVES for reporting the new random order, and someone else for mentioning the mods problem, although we can't find that mantis report at the moment.

• New "Dump All Data Tables After Load" setting in the debug section of the settings menu:
  • When the program is loaded, or after the xml files are reloaded into the in-memory data tables, write a text file for each one into a DataTableExports subfolder in the PlayerData folder.
  • The purpose of this is mainly to use with diffing tools between one run of the game and another, to see what sort of data changes happened. If you are modding and made changes to some ships and want to see how those changes cascaded, this would be one way to do that. This is also a way for us to verify correctness when we make structural changes internally.
  • This has some issues at the moment, but is something we're working on getting to be fully functional.

• The game settings and galaxy settings no longer load in parallel, but only linearly. Their order was meant to be pulled from the xml, even though we don't usualyl do that.

• More thoroughly fixed things so that if ANY error happens during loading of the game, it should not be able to just sit there as if it is loading something but really be hung.

Mod Capability Improvements

• Fixed a bug that was causing the disabled expansion status and the toggled (whichever is not default) status for mods to not work properly in the last few versions. This was one of those things that we just hadn't tested well enough yet because it was a beta.

• Mods are normally named by the folder they are in. But what happens if you need to rename a mod? Any savegames that used the old name will not be able to find it, and thus will break.
  • Therefore, a new feature has been added. If you put a ModAlternateNames.txt file in your mod folder, then you can specify alternative names that it can be found by (for purposes of linking up correctly to old savegames).
    • Inside this file, each line should simply have the other names that the mod was once referred to as.
  • If an alternative name would conflict with the name of an actual mod installed on the system (aka you set an alternative name of BeefMod and there is already an actual BeefMod present), then the actual mod will take precedence and the alternative mod name will not be in effect.
  • These alternative names don't actually show up anywhere, but basically just are ways to link to old names, so please don't go nuts with adding stuff to it.
  • In the case of the SpireRailgunShop mod, Chris had originally been using it as LordONothing_MoreSpireShips, and this entry now lets him load those old internal savegames.
    • As a one-off, this would not have been worth the feature, but it's likely to be a thing that is needed in the future. We've noticed some mod authors have multiple folders with different versions of their mods, and being able to cross-load between those in sensible fashions will now be possible. Prior to two versions ago it did not do a mod name check on them, so they never ran into trouble, but now they would. So here's the tool to get around it.

• The TLDR of one of the xml reading improvements below is that not a mod can change a base game unit with a partial record (for instance, alter the stats of a Raider), and then any descendants of that unit even in the base game (for instance, a Dagger) will inherit the modded values. Expansions can also do this.

Xml Reading Improvements

• Fixed a timing bug with rows that were being read in where if there were a lot of copy_froms then it was kind of at the mercy of luck as to whether or not it would load it properly. Basically since it's reading them in parallel, if it had started reading it but not finished it, it would still try to copy it over and thus of course have many problems. Now it fully waits not just until it has started reading, but until it is finished reading them.
  • This was the cause of the intermittent ghost turrets not loading properly in the last two versions of the game.

• ImportIntoStringList_Directory() has been moved from ArcenXML to ArcenStrings, since it makes a lot more sense there (it has nothing to do with xml).

• The way that the "what file is an error in" tracking for ArcenXML works has been redone heavily, so that we can now keep track of that on a per-node basis.
  • We're going to be processing nodes from a bunch of files at once, so that's a needed step.

• Looking into a couple of methods that we probably won't touch right now, there are:
  • ImportAndProcessDirectory() is not actually used at the moment.
  • ImportAndProcessXmlFile() is used for reading the sprite dictionaries that TexturePacker generates for us.
    • We will keep this linear, even if later we have to extend this to handling more of them.
  • ReImportSelectIntoDynamicTable, ReImportSelectIntoDynamicTable_XMLDirectory, and ReloadSelectData are all used ONLY for super limited cases.
    • Specifically these are used to reload the LOD overrides for ships, which lets us design the LODs in-game for us with ships.
    • Given the limited nature of this, and how this isn't really something that needs to be as robust as the real game processing (since it's only used for dev purposes anyhow), we'll leave this alone for now.
  • The other xml parsing paths all pretty much go through one set of methods which is going to get a hefty rework.

• Dynamic tables no longer use an ArcenSparseLookup for their RowsByName, but now use the threadsafe ConcurrentDictionary instead.
  • The lack of this was quite possibly the source of some very confusing bugs in the last few beta versions, potentially including StarKelp getting "DoomData'd."

• The way that dynamic tables are read in has been completely overhauled.
  • Previously, it read in data one file at a time, first from the base game, then expansions in order, then mods in order.
  • Last release we improved it so that the order of operations did not matter between items in a single file, so if a partial record came before its original, or a copy_from came before its original, that was fine.
  • We now have extended that substantially, however, not just to be cross-file, but actually to improve the loading order and give new power to mods and expansions to change the underlying game.
  • The new loading order, with thanks to NSirLimbo, is as follows:
    • First load every item that isn't a copy_from or partial. The order is irrelevant for these, so just do them all.
    • Now do all the partials from the base game, hanging onto any that fail. Then partials from each expansion in order, and then each mod in order.
    • Now do all the copy_froms from the base game, hanging onto any that fail. Then copy_froms from each expansion in order, and then each mod in order.
      • After each batch of copy froms (base game, each expansion, each mod, etc), retry any former failures from partials or copy_froms, in the proper batch orders from before.
  • The way this works out, things are still all loaded in exact order from base game, to expansions in order, to mods in order, BUT with partial records applying before copy_froms in ever possible case.
    • This means that if two mods change the same field, the mod that is lower in the list will be the one that always applies, for each player and each time you run the game. This isn't new, but it's something that was hard to maintain while we added the following:
    • This also means that if a mod or expansion changes a unit that is later copied -- say, perhaps the Raiders is changed by a mod -- then the changes will propagate to all the descendants of the original (so, if you mod the Raider, then the Dagger will inherit those modded changes).
    • This is really useful for mods, but potentially also useful for expansions if we are adding content that goes onto existing ships for whatever reason.
  • We had also hoped that this would help solve our copy_from woes that mods were experiencing, but it turns out that is just a fundamental problem with copy_from (which we now need to investigate but can do with clarity on where and why the issue is) and is not a problem with mods or expansions. It's just by chance we haven't triggered it in the base game, so let's get that fixed!

Beta 2.081 Still-Broken Hotfix

(Released June 24th, 2020)

• Fixed a bunch of issues that were making the game not able to generate new savegames. A couple of minor code changes led to a bunch of things not being able to calculate/recalculate properly.
  • This may or may not work with mods, it's hard to be sure right now.
  • If you get any exceptions during initial startup of the game, then it won't work correctly during that run. Close the game and reopen it and you may get no exceptions, after which it will run just fine.
  • There's an issue with the copy_from field not working entirely accurately that we still need to fix, and having mods on seems to exacerbate it but is not the root problem. Having it fixed will open a lot of doors for mods, though.

Beta 2.080 Fast Loading

(Released June 24th, 2020)

Note: at the moment in this beta, to enable or disable a mod or expansion, you have to do that in the settings screen and then restart the program. The restart of the program should become unneeded soon. But it's not for sure that this is actually working with expansions at the moment even with a restart, just yet.

Note 2: If you have problems running this version (some people do), then it's because of copy_from inheritance and different CPU timings. You can run the game after a few tries, potentially, but if not then please use the new beta branch v2078_last_before_fast_loading on Steam. We should have a fix for this tomorrow, which should also be a big boon for various mods in the future.

• Fixed a bug where not-installed expansions in the game window were showing up with their text on top of other text.
  • Thanks to -NR-SirLimbo for reporting.

• Added a new "XMLMods_NonDistributed" folder and type for internal use, so that as developers we can keep popular mods on hand for testing their savegames without accidentally releasing them to all players if we don't have permission to do so.

Much Faster Game Start Times

• For the sake of simplicity and accuracy (order of operations, mainly), all of our loading of unity objects (textures, materials, meshes, sprites, music, sound effects, etc) from asset bundles has always been synchronous on the main thread.
  • Except in the case of music, where things are streamed in from disk, this actually causes the program to load more slowly than it needs to.
  • There are some bugs in unity in the async versions of their code, so long term we may not be able to use this, but we have now expanded the arcen asset bundle handling to be able to support both synchronous and asynchronous loading of items from asset bundles. We will use the new async version where we can as long as we are not running into engine bugs.

• The bulk of our really central art and sounds and so on now loads asynchronously when the game starts, making the whole load process vastly faster for you.

• Several icons and materials used by ships were previously being loaded inline with the xml reading.
  • This may or may not have caused some actual problems in practice, but best case it was wasting some processing time (making game load slower) when a copy_from or is_partial_record reset the icon.

• The way that sound effect and voice clip playback is scheduled has been rejiggered fairly heavily, to allow for the indirection that is needed for our style of asynchronous loading of these effects.

• We no longer support the side_specific_visual tag, which was used to give different visuals to different factions for the same ship.
  • In basically every case where we do that, we need more specificity than that could provide, and we use copy_from. So in reality this was just wasting a heck of a lot of time with the internal data structures that were there to support it.
  • Literally the only place that used this at all was the metal harvester points, which look different when not owned by anyone.
  • There is now a new visuals_if_natural_object and visuals_bundle_if_natural_object field pair that is being used for literally that one singular case.
  • Removing this changed a lot of code but let us fix a mysterious error that was happening since the switch to async loading, AND it makes the loading process even faster.

• Added a new GetFactionTypeSafe() on squads, which can be used to get the Type on the faction without risk of nullrefs.

• It turns out that one of the big problems with our use of AssetBundle.LoadAssetAsync() was NOT that we were getting inconsistent results back from it (it seemed like a lot of random things were not loading), but rather that things were coming back so fast that before we could assign our event handler to an event, it had already fired the event.
  • So we built in a subscription wrapper method that lets us never miss an event firing complete from these, even if it's already done, so that all code gets executed as requested.
  • The overall end result is a much faster-loading game without any bugs.

And Even Faster Still

• A bunch of extra data, or data that was not being properly cleaned, has been fixed up.
  • It's mostly things like duplicate lists or dictionaries when there are built-ins right there.

• For NodeProcessor on xml reading, we're now using Parallel.ForEach, meaning that each table still gets processed one at a time, but all the rows are parsed in the background all at once on many threads.
  • This cuts down further on loading times, but does mean that activities that are not threadsafe should be in DoPostInitializationLogic() (which we wanted to have happen anyhow), or should use a threading lock() when absolutely needed.

• A new optional ReadXml parameter is on dynamic tables that we are initializing.
  • It defaults to ReadXml.InParallelThreads, but you can change it to ReadXml.InOrderOnMainThread.
  • This is now changed to being in order on the main thread for UIWindow, because that needs to do a LOT of nested prefab stuff and not a whole lot else. It breaks on the background thread.

• deprecated_in_favor_of has been removed, as we didn't use it and it probably was not functional.

• If the game runs into an exception while it is trying to load, it no longer logs errors silently and just seems to hang. Instead there's a solid chance that it will just fill the screen with crazy amounts of errors, but at the very least it is extremely clear that things have gone off the rails.
  • The game generally loads for people unless there is a mod or new code that is misconfigured, or something downloaded wrong from Steam, but in the really rare cases where it just seemed to sit there infinitely, that was really frustrating for folks.
  • And internally, we run into that a lot more as we change a lot of things and test them.

• A bunch of how we handle errors during xml reading during startup is now more efficient and more direct.
  • The order of operations of errors really matters a lot when you've got all this multithreading going on, so our prior level of indirection had become extremely undesirable.
  • And then after a bit more investigating, we found that we needed to decentralize this more in general, and use the general exception-handling tools of C# for logging properly from each of the innumerable threads that might now be doing work. Our old style of logging was incredibly single-threaded-minded.

• Similarly, ArcenXML.ReadingPartialRecord had to be removed, and ReadingPartialRecord is now on each individual ArcenXMLElement that the game reads from.
  • The number of things that were single-threaded-focused in xml reading is really quite typified by this, but now it works great in both styles.

• Previously, the order in which entities were parsed mattered a LOT.
  • If the game had a copy_from entry prior to the thing it was copying being read, or if there was an is_partial_record prior to the original node, then things would error out.
  • In a lot of ways we were able to just work around this, always putting those things lower down in the same files, etc. But the ability to have partial records from another xml file, and not wonder if some OSes might sort the files differently (that does happen at times) was not present.
  • The game now actively imports rows in all kinds of random orders, even within one file, since they are often happening in parallel background threads.
  • To solve this problem, including our older version of the problem, we now do a couple of iterations to catch straggling "that wasn't there yet" nodes, making the order of your entries in files (and the order of files) no longer such a problem.

• The Matches method on rollups really was never supposed to be used directly, because it can be quite slow and there is a perfectly-good precalculated version already in place on each game entity type data.
  • The methods have been renamed around and adjusted so that you can just call GetMatches_SemiSlow( yourrollup ) on the game entity typedata. It's vastly faster than the other kind of check, but still the sort of thing to cache if you're doing it a lot in your mod.

• The way that the game handles checking over the nodes for incorrect xml at the very end now actually gets counted as part of the load timer, and is much faster than it was before.
  • All in all it takes around 4 seconds or so, but even with that added on the time to load the game is dramatically lower than it used to be.

• The unrequested XML attributes file is no longer a thing, and those now get logged to the main thread instead.
  • The game now runs this in approximately half a second instead of 4 seconds, thanks to running 500 parallel items at a go on background threads. Bear in mind that this will vary between machines, and even between runs on the same machine. But it's vastly faster everywhere, regardless.

Final Results

• The overall load process for Chris used to vary from maybe 16 seconds to 90 seconds depending on the cache state of some things and the way the wind was blowing, phase of the moon, etc. Most of the time is was on the lower end, but several times a day more than 60 seconds.
  • It now launches reliably in between 7 seconds and 24 seconds, with almost all launches being less than 12 seconds.

• When it comes to the GameEntityReferenceData.csv file that we write with a lot of the data from ships, they are identical between the prior version of the game and this new faster-loading one.
  • (Also we now write that from a background thread even though it only takes maybe a quarter second on an average HDD).

• There is a new v2078_last_before_fast_loading beta branch on Steam that you can use to load the prior version of the game (version 2.078) if the new one gives you problems, or you just want to see the difference on your machine.

Beta 2.078 Mod and Expansion Prepping

(Released June 22nd, 2020)

Note: at the moment in this beta, to enable or disable a mod or expansion, you have to do that in the settings screen and then restart the program. The restart of the program should become unneeded soon.

• Fixed a kind of funky error that was newly showing up in the factions tab in the settings screen, and in the view/edit factions window in the main game, if you clicked between the AI and human factions.
  • Basically it was not taking a full frame to stop drawing the old fields and populate the new ones, so the old fields from one tab were trying to get (very invalid) data from the new data points that would be shown the next frame.
  • In the past this was still happening but did not throw an error. Was this ever causing any problems? It's hard to be sure, but probably not. Either way, having this error out now and then us able to put in a fix is some peace of mind that it can't bother things.
  • It's possible that we have a similar issue with the galaxy options tab, but that those just don't hit the point of throwing an error. If we ever run into trouble with multiplayer inconsistencies on the galaxy tab, we'll just need to remember to come in and implement this kind of fix there, too.
  • Thanks to Ovalcircle and Lord Of Nothing for reporting.

• Fixed a bug in the last few beta versions where the high importance intel items were sorted above low, but below normal.
  • Thanks to GophTheGreat for reporting.

Improving Expansion and Mod Handling

• Savegames have kept track of which expansions are enabled in them for a long time now, but until now they did not stop you from trying to load a savegame that used an expansion you did not have.
  • Older quickstarts that had expansions on (usually just with beacons) but no expansion content enabled will still be usable by non-expansion players.

• Savegames have never tracked which mods are enabled for them, but now they do.
  • They also now prevent you from loading a savegame with a mod enabled unless you also have that mod enabled.
  • This sort of thing, like the expansions, gets a lot more relevant now that multiplayer is coming up, but it's also useful for our own internal testing.

• In the case of a savegame that you are trying to load that you are missing some expansion or mods with, it gives you a complete list of what you are missing, now -- assuming that the save was created in version 2.078 or later.

• The in-game escape menu now shows what expansions or mods are enabled in the savegame you are playing, if any are.

• Under the hood, GetExpansionStatus_SemiExpensive() has been removed, as it was slow and also kind of confusing.

• Added ConvertToCondensedFormat() as an extension method on strings, and it makes a unicode-style string into a condensed-format-viable string.
  • This is useful for mods that might have names that extend out of the ASCII name range, particularly for international modders.

• Added SplitIntoWordsByCapitalization() as an extension on strings.
  • We are now using this to generate automatic "display names" for xml mods.
  • So for instance "BobsFancyMod" becomes "Bobs Fancy Mod" and "UTTTransport8" would become "UTT Transport8"

• When you are loading the game, it now shows any expansions or mods you have installed at the top of the list of things it is loading.
  • Additionally, on the main menu if you hover over the "load time" entry at the bottom right, it shows those again.
  • We definitely predict that there would have been confusion among people with mods and expansions that don't match who are trying to play multiplayer.

• Expansions now actually have a description field, which lets players see what they are from in-game in a brief tooltip if the expansion is discussed.

• A variety of pieces of code for the initialization of dynamic data tables now takes in more explicit enum-based parameters to determine how they should act. Do they reload themselves if we try an reload of the xml (impossible until now), are they primary key strict, do they support indexing, etc.

• The game now properly is able to read these settings and actually disable mods or expansions if you don't want them on for now.

Improved Modder Features

• All of the xml FillList() methods and variants now include the following two parameters: IfPresent IfP, Uniqueness Unique
  • The former can be either ReplaceExistingList or AddToExistingList.
    • It turns out that, until now, it was always doing ReplaceExistingList. Which we had honestly kind of forgotten about.
  • The latter can either be Unenforced (always has been until now) or Required (no exceptions are thrown, it just will skip adding duplicates so there's not extra junk in there).
  • This lets us have a lot more control over exactly how xml is read in, which is mostly useful for mods, but somewhat also for variants of ships within the main game.

• There's also a new ClearBeforeReading, which can be either Never, Always, or IfNotPartialRecord.
  • This lets us inline some of our logic for these, which previously were always either Never or IfNotPartialRecord, but you had to hunt through the code to find it before.

• The game now supports a few new general-purpose attributes, for the first time being attribute variants.
  • Essentially, if there is an attribute that is a list, and it is called bacon for whatever reason (this could be a mod or a main game attribute, after all), you can append these things after bacon to change how the xml is read in for that specific xml node.
  • So if you have bacon="5,6,6", and normally that is read in as Uniqueness Required in the code, then you're powerless as a modder to get 5,6,6 as your results, instead only getting 5,6.
  • But NOW you can add in bacon_uniqueness in order to override that to the value you want, as one example. This works with any attribute that is a list type, whether that is defined in the core game or in code you added for a mod.
  • The valid new attribute variants are:
    • [attribute]_uniqueness, with the values: Unenforced and Required. These overwrite whatever was passed into the FillList method in code.
    • [attribute]_if_present, with the values: ReplaceExistingList and AddToExistingList. These overwrite whatever was passed into the FillList method in code.
    • [attribute]_clear_before_reading, with the values: Never, Always, and IfNotPartialRecord. These overwrite whatever was passed into the FillList method in code.

• Previously there was no way to add to the list of tags, but now you can use the generalized feature tags_clear_before_reading="Never" and tags_if_present="AddToExistingList" combo to change that to be additive instead.
  • Thanks to StarKelp for suggesting this, which led to the rest of these generalized additions.

• Added a new FillEnumIfPresent method, which basically only reads an enum if it is present, but otherwise leaves the original alone.
  • Previously FillEnum was always overwriting the prior value if it was present, which was unlike our other Fill methods.
  • There is a bool on here that lets us complain if it is missing (aka no new assignment, default or otherwise).

• Added a complementary FillEnumIfAndComplainIfDefault method, which basically will give distinct complaints if it is missing, invalid, or default as a value.
  • Why we would ever really want to use this is kind of suspect, but it's actually along the lines of what FillEnum used to be doing, which was confusing and inconsistent with other Fill methods.

• FillEnum has been entirely removed from the codebase, now instead expecting one of the two above to be used.
  • The idea here is to force us as developers plus anyone who is a modder to be explicit in what they actually want to have happen.
  • There were some frustrating oddities that were bothering modders that were probably happening because of the way FillEnum used to work.
  • Almost all cases that were previously FillEnum are now FillEnumIfPresent, which now means that they keep their value if missing a field rather than resetting to default.
    • This has huge implications, mostly positive, for any is_partial records and copy_from records. Having a record of that sort that doesn't duplicate an enum attribute will no longer reset the enum attribute to its default (probably None).
    • Places this could bite us: special_entity_type in particular. But assumedly if you are not reassigning the special_entity_type of a descendent entity, it is the same type as the original -- but you will need to actually check for that now in your xml data if you're a modder.

• Fixed a bug where galaxy settings could not have their type properly inherited on partial records.
  • Same for core settings.
  • Same with setting_type on custom_field for factions.
  • And type on factions.
  • Several fields on damage modifiers.
  • category and firing_timing on systems.
  • design_logic on fleet design templates, and ship_cap_groups on sub-entries of fleet design templates.
  • a bunch of fields on ships.
  • type on resource production and multiplier
  • ship_cap_group on fleet items

• The max field can now be zero on fleet_membership entries under ships.
  • This is something that is on those entries BEFORE it tries to ever add the ship to the fleet membership in general.
  • Fleet items with a maxCap <= 0 are now universally not actually added to the fleet.
  • This lets modders remove entries from fleets via mods.
  • You can also set the weight to 0 or less, and that will even more aggressively remove it.
  • The same has also now been done for ai_ship_group_membership entries.
  • Thanks to -NR-SirLimbo for suggesting.

Ability To Ship Off-By-Default Mods

• Mods can now have a ModDescription.txt file in their folder, which lets them have a description that players can read when seeing them listed in the Game tab of the settings menu.

• Mods can now have a ModIsOffByDefault.txt file in their folder.
  • If this file is present, then the mod won't load until the player goes into the Game tab of the settings window and turns it on.
  • This is really useful for any mods that we might want to ship with the game by default (for ease of people finding them), but make something that people have to opt into to turn on.
  • If ModIsDisabled.txt is present, then that overrides things to the point that the mod is invisible, and this file is ignored.
  • If you are a modder and you'd like to have us distribute your mod with the game, but off by default,

• The text in ModIsDisabled.txt has been updated, to make the function of that more clear to modders.

• The Game tab of the settings menu has been updated so that you can see all the expansions and their status (installed or not, enabled or disabled), and all of the mods that you have not hard-disabled.
  • Now that mods can be disabled more softly, we're able to do a whole lot of things that are a lot more like what a proper modloader can do.
  • We also reorganized several of the startup sequence items to properly handle this new information, now that we need to have GameSettings data loaded sooner than we previously did.

First Off-By-Default Included Mod: The Spire Railgun Shop by Lord of Nothing

• The Spire Railgun Shop mod by Lord of Nothing is now included with the game by default (with permission), but disabled.
  • This lets people simply enable the mod by clicking a button in the settings menu, rather than having to find and download the mod from elsewhere.
  • Full description of the mod is included in the tooltips in the game
  • Description here:
    • This is intended to be balanced without the need to run different settings to a normal fallen spire game.
    • It contains:
      • Spire RailFrigates.
      • Spire Raildestroyers.
      • Spire Riot Control Cruisers.
      • Spire Interdictors. (Light and Regular.)
      • And their neural nets, of course. (Which does start to clutter the city build menu a bit, unfortunately.)
    • To elaborate: RailFrigates and Raildestroyers are what their names suggest. I’ve tried to retain the what I feel is the original flavour of the two types – Railfrigates are a bit of a crowd brawler, while Raildestroyers are more focused on killing big stuff, but also have a utility role, since their railguns are somewhat… Arachnid.
    • The Riot control cruiser is named after the starship from the original game, but is more of a fusion between the original game's spire tractor platform and the spire gravity drain. It's designed to keep things at range to give your railguns time to use their range.
    • The two Interdictor variants are designed to further enable a slightly more control-based, long range playstyle, through the use of very powerful gravity effects, since I felt this went well with railguns. They are big. They are not cheap.
    • Further balance notes:
      • Railfrigates and destroyers give up some DPS for range, since the spire already push glass cannon status and I felt pushing them further that way was unwise. They also innately apply their DPS slightly less efficiently than the coilbeam.
      • The Interdictors: They’re seriously expensive and fairly deep into the techtree, but mobile planetwide gravity has a lot of potential, so… Enjoy.

Beta 2.077 No More Shot Errors

(Released June 19th, 2020)

• Player-allied scourge aren't allowed to suicide at targets. It makes players nervous that the scourge might just be inept.
  • Thanks to tadrinth for the bug report

• Some minor UI tweaks; "listen" becomes "listening", remove an unwanted space in a log message
  • Thanks to Lord of Nothing for reporting

• Split the hunter fleet stage 3 logic into several separate sub-methods so that we can more easily see where something is going wrong if there are exceptions that pop up.

• Fixed a bug in the hunter fleet where if there were no allied AI factions it would get a divide by zero exception every 10 seconds or so.
  • This should never be a case that actually happens, but that's a separate matter.
  • Thanks to Lord Of Nothing for reporting.

• Fixed a bug that is probably as old as this game itself, where shots would sometimes go zipping off to the bottom left of the gravity well and disappear offscreen. It turns out that this was not just a display artifact, but actually shots that had no target to find (probably their target died around the time they came into existence, so they were trying to quickly make their way to the "no location" point.
  • Most recently in the betas, this has been causing exceptions when trying to save such shots, as they were out of bounds for the gravity well and thus causing visible exceptions during autosave or intentional save (which were harmless but annoying).
  • These shots no longer do any movement if they have nowhere to go, and thus won't go streaking off into the night.
  • Thanks to Badger, Ovalcircle, marty651, and others for reporting.

• There were some random cases where it was possible that trying to add another ship visually to draw would not work for various reasons. It was rare, but should no longer be possible for that to throw an exception.
  • Thanks to marcaunon for reporting.

Improving Identification of Expansion and Mod Content

• The "ForExpansion" field on the dynamic table rows has been removed. It was not being properly set in a lot of cases, anyhow.
  • Now dynamic table rows have DataSource RowFrom, Expansion RowFromExpansion, and XmlMod RowFromXmlMod.
  • These let you see if the row was originally created from an expansion, the base game, programmatically, or from an xml mod. If an expansion or an xml mod, then it links to the mod.
  • Note that variants of these things (copy_from, etc) would wind up with a value that is whatever the variant was created in. If a mod makes a variant, then the original would still be from an expansion or the base game, but the new variant would show up as being from the specific mod. Basically what you would expect.
  • Note that partial records do NOT overwrite the value. So if there's something from the base game, and then a mod uses a partial record to change data, it would still show up as being a base game item.

• THAT said, there is a new List<RefThreeTuple<DataSource, Expansion, XmlMod>> RowAlteredByPartialRecords on dynamic table rows, which lets you track exactly how many partial records have contributed to it, and if they were from the base game, expansions, or mods.
  • Note that these row alterations are not copied over to copy_from entries. So if one mod alters an entry a bunch via partial records, and then another mod makes a variant, the variant would not have a record of the partial records that went into what it was copied from.
  • If you needed to find that for whatever reason, you could check the existing CopiedFrom attribute on the variant row. That has a chain all the way back to the original row, if you need it.

• GetRowByName() on dynamic tables used to require three parameters -- name (stil there), create-if-missing (gone now), and expansion (gone now and was never used).
  • Mods will have to be updated to account for the new method style and behavior, and hopefully the core game itself is also handling this properly now (but there may be bugs in the short term).
  • This was a VERY old way of handling this, before we even had proper formal xml mod support, and there were some implicit behaviors that were problematic.
  • There were various overloads of this method that would complain if it could not find a row by name, but the default was for it to just silently return null. This could then lead to unexpected invisible problems.
    • This now throws a MissingArcenDynamicTableRowException instead of returning null if the name can't be found.

• The game now uses the MissingArcenDynamicTableRowException in several key points during deserialization (loading a savegame, in other words) to let us show a message stating "hey, this savegame is talking about something called X, but that doesn't exist, so you may not have the same mods and/or expansions that the savegame was using).
  • Previously if it couldn't find things, it would either just spam errors silently and then show one final parsing error that made no sense whatsoever to anyone, or it would try to patch over things by loading a "dummy ship" in without the actual stats on it. The latter is completely gone, because that was going to absolutely break multiplayer but in very inconvenient-to-figure-out ways.

• For the sake of flexibility and retaining prior functionality, there are a couple of new methods on arcen dynamic tables beyond just GetRowByName().
  • CreateNewRow() has been added for if you want to create a row programmatically from whatever source.
    • Previously the only way to explicitly programmatically create a row was to call GetRowByName() with an invalid name and create-if-missing true. That felt very hacky, and it made code that used it (maybe two lines in the whole codebase) very hard to read.
    • Additionally, this was kind of the only real reason for ever passing in the Expansion field, since it would then set the expansion on the created field. That didn't justify us passing in lots of mysterious nulls all around the codebase.
    • This method now, naturally, takes in string Name, DataSource Source, Expansion ForExpansion, XmlMod ForXmlMod, so that you can fully document the row's origins as you wish, just like any other row.
    • This is still a kind of method that we almost never should need to use, because the whole purpose is to let these dynamic tables be initialized and filled by xml itself.
  • GetRowByNameOrNullIfNotFound() works like GetRowByName() used to, in that it will return null if it can't find the row, rather than throwing an exception.
    • There are actually a LOT of places in the codebase, particularly in factions and commands, where we use this. Probably a lot of mods will want to use this, also.
    • The purpose of this is basically when you want to "find it if it's there, or else handle the fact that it isn't there in an intelligent way." When you have that sort of scenario, using exception handling to control program flow would be both crude and slow and hard to read.
    • There are a number of places where this method is now used in the external part of the codebase, and if the result is null it generates some other sort of error or just does something else and isn't even an error case.
    • There are some other places in the external part of the codebase where we still use GetRowByName() instead, but we took out the follow-up exceptions where it used to say "if the result is null, then throw an exception." The new exceptions automatically happen, saving code, and are more detailed.
  • GetRowByNameOrCreateIfNotFound() is really not used frequently, but is nice to have just in case. We use it a couple of places. It does what you would expect.

• Overall we may have some short-term bugs from these changes, but they should also help us -- and modders -- avoid "invisible errors" that are way harder to track down.
  • These changes also make the code more readable, and give equal precedence to mods and expansions in terms of figuring out "what came from where."

• Speaking of invisible errors, we found some places where dynamic tables were trying to reference other dynamic table rows in the wrong order of operations for initialization, and that was potentially causing invisible errors before (or certainly wrong data). These now errored during game startup, so we were able to fix them:
  • MapTypeDataTable was being referenced by ScenarioDataTable, so had to move to come before it.
  • GameEntityTypeDataTable was being referenced by TestChamberTable, so TestChamberTable had to move below that.
  • SpecialFactionDataTable was referencing TeamColorDefinitionTable, so that had to move up (and we took TeamColorPrefabsTable with it).
  • These somewhat fall under the category of "how did these ever work?"

• We then had some other problems relating to entities and the order in which they were present in xml, and they were working fine but it was kind of luck that they did.
  • Something like the WarspiteArtillery, for instance, would be defined prior to the Warspite, which is something it would spawn on death.
  • The reason that this has historically worked is that it would create a stub entry for Warspite as soon as it was entered, and then later that would get filled in by the actual entity.
  • But... what if the actual entity was never filled in? We'd have zero way of knowing, and the game would just proceed with this practically-null ship. So it was time to change how these are read in, so that we actually get proper errors for the latter case, but can still order our xml how we want.
  • We now load three fields where entity type datas referenced other entity type datas in a delayed fashion, so that none of this happens. If there's an error in xml, it will no longer be invisible.

• Added a new debug setting to personal settings: Write Savegame Serialization Logs In Realtime
  • This is used only on conjunction with the main 'Write Savegame Serialization Logs', and should almost never be used.
  • This causes the log process to write to disk immediately as it does every log action, rather that writing to a buffer and then writing the results of that buffer at the end.
  • This is one of those features that you should use if you're needing to test a deserialization that is crashing the program, which should basically never happen if you're not a core developer of the game.
  • As you may surmise, our first result after getting the game functional again after all the above changes was that any savegame was crashing the program with no error messages, so hence this.

• The ArcenSerializationTester.Writer object has been made private, and now you just call Append or AppendLine on ArcenSerializationTester directly, so it can work in the prior style or the new realtime format.

• The exceptions you get now when you try to load a savegame that had a mod or expansion in it that you are missing now gives an error message that is brief and actually makes it pretty clear why this is happening. It doesn't say what mods or expansions you are missing yet, but that's another next step.
  • This particular fix catches even "mods that are just changed files with added stuff" and not just properly set up mods that someone put in their own folder.
  • Thanks to Badger for suggesting.

Beta 2.076 Hotfix

(Released June 18th, 2020)

• Hunter ships against a specific target (for example, MDCs) are now allowed to attack other things if there are no MDCs they can get to effectively.
  • Hunter Ships against a specific faction are still required to go after that faction.
  • Thanks to tadrinth for the report

• Bonus income for AIs against specific targets is now split evenly between all AI factions in the game. This may change a bit, who knows.
• Some nerfs to anti-player-allied-marauder hunter income
  • Thanks to ANGRYABOUTELVIS for mentioning

• Fixed a bug from the last couple of beta versions (maybe somehow just the most recent one, but that seems unlikely) where any savegame that had shots in flight would fail to load properly.
  • All of the saves that were broken and unable to load should now be able to load fine, although we've not tested them exhaustively. Please let us know if you find any more.
  • Huge thanks to ANGRYABOUTELVES for a savegame that let us reproduce this within a few seconds. Finding the bug was then a matter of turning on savegame logging, triggering a save that would be broken, trying to load it, and then comparing the differing outputs. Without that we would have had to go with examination of the failed load log alone.
  • Thanks to ANGRYABOUTELVES, zeusalmighty, Lord Of Nothing, Bluebizzer, and others for reporting.

• There's a handy "combat attack and speed multiplier" that you can set by holding Ctrl and clicking the faster and slower buttons at the bottom of the screen, and that's not new.
  • However, there were some problems with us never really setting that back to the proper defaults, and in the case of our switch of FInt formatting that caused a lot of things to wind up at speed 4.1 rather than 1.
  • This is now automatically and retroactively fixed for all savegames, and will work properly in the future if you have those speeds up or down in one game and then start a new one, too.
  • Thanks to NRSirLimbo for figuring out the source of the problem, and Lord Of Nothing, ANGRYABOUTELVES, and ynof for reporting.

Beta 2.074 FInt Hotfix

(Released June 16th, 2020)

• Warning: any savegames loaded into 2.073 and then re-saved now have all of their FInt values multiplied by about 4.1.
  • The harmless end of this was that your speed was 4.1x, but on the nastier end it meant your AIP was 4.1x higher. Or on the plus side, your science...
  • This ALSO would affect any new games that were created in 2.073, since those load an old savegame no matter what (to get your last settings).
    • If your old settings are off because you loaded a save in 2.073, then please make sure that you hit Reset To Defaults in the lobby, or load a different save or quickstart into the lobby for editing.
  • Why this was affecting damage output is not entirely clear, but in some cases it seems to have been. Please let us know if you continue to observe that for some reason. We thought that was based on the galaxy options, but it seems like not.
  • Thanks to NRSirLimbo and Lord Of Nothing for catching this.

• During gameplay, when you click into the Galaxy-Wide Options screen it now shows you ALL of the values, not just the ones you can edit.
  • As with the factions screen, it just has the ones that are non-editable as text values, but you can examine your settings this way and also see their tooltips.

Beta 2.073 Filesize Finalization

(Released June 16th, 2020)

• Make the Marauder less likely to suicide themselves against Praetorian Dragons
  • Thanks to ynof for reporting

• Fix another problem with waves going to an illegal location
  • Thanks to ussdefiant for the bug report.

• Added the following methods into GameEntity_Base, to make for exception-proof thread-safe ways of finding out certain information:
  • GetTypeInternalNameSafe
  • GetTypeDisplayNameSafe
  • GetPlanetNameSafe
  • GetFactionDisplayNameSafe

• Some various overloads have been added to the UltraEfficient methods, allowing us to pass in an optional parameter to get UEErrorStyle.ThrowExceptionIfProblem where we can add in extra context about what is wrong with data if the data is out of range.
  • Applied this to the WorldLocation serialization of all entities, so that if we have an entity that is being saved from out of range, we find out what it is, what planet, who owns it, and what the value was. This also uses the new safe getname methods.
  • Thanks to Badger for discovering that apparently in one of his savegames a shot was being saved out of the gravity well.

• Fixed a very unusual bug where the AI was basically able to get frigates "for free" as part of their reinforcements in the rare cases that they actually used them.
  • Normally the AI does not particularly use frigates, but a few types like the special forces master or turtle do so.
  • This bug was allowing for unlimited growth of frigate strength on even planets that had been neutered, but now the strength of frigates are counting against the "strikecraft" AI reinforcement budget.
  • This won't fix old saves with too many frigates, but those should be fairly few and far between in general. And it will prevent those games from getting yet more frigates stacked up on the planets in question.
  • Thanks to DEMOCRACY_DEMOCRACY for reporting.

• The original game version is now saved into savegames, and you can see it on the escape menu. If it was started on any version other than the most recent one, it will say "Unknown."
  • You can also now see the campaign start date/time in the escape menu. We've been storing that forever, but never showed it for some reason.

Fix To Beta Branch Inactive Wardens and Hunters

• Nota Bene These two fixes may cause a giant flood of enemies in games that have been played for some time on the beta

• Fix a bug (beta only) where the Hunter Fleet was just chillin
  • Thanks to a bunch of people for reporting, and to ynof for having an unrelated save game that reproduced the problem readily

• Fix a bug (beta only) where the Warden Fleet was just chillin
  • Thanks to ussdefiant and Tadrinth for the bug report

Savegame Filesize Reduction Continues

• Converted the various combat-space positions in shot and wormhole objects to use the new UE format.
  • Test savagame dropped from 2.3449 MB to 2.3444 MB. But hey, it took like 5 minutes. This is a good example of why we should mainly focus on heavily-used places for shifting to UE format, though.

• Added a new GetUltraEfficientStyleThatFits() method, which lets us find the smallest UE data format of a given group that will fit a specific number.
  • Note that the actual UltraEfficientStyle enums should NEVER be serialized directly, as they are not considered value-safe as integers.
  • So in order to use this, it needs to be based off of some number that is calculatable on the serialization and deserialization process, independently.
  • Essentially, for things like time or primary key IDs, we can look at the central values that have been issued and find the appropriate wrapper on both the serialization and deserialization process.

• We are now using the gated-smaller serialization method for GameSeconds, Primary Key IDs for all entity types, fleet PKIDs, speed group PKIDs, frame numbers, planet indices, and faction indices.
  • Any part of the game, mods or otherwise, that want to use the now-old-but-recently-new "modern" style of formatting to store or transmit these, instead of the new UltraEfficient methods, are free to do so. It won't affect anything correctness-wise, but is less efficient if it's a field that is written a whole lot of times.
  • A lot of the data in external classes just isn't written frequently enough for this to be warranted, unless it's something that is on a vast number of entities. This is really more focused on GameCommands (later) and squads (now).

• TimeAlive has been removed from shots, as it was never used and could be inferred from info on GameEntity_Base.

• RemainingDelayUntilEntersSim has been moved to just be on shots, and thus not take up extra space on squads, since it is only used by shots.

• Switched over entity orders to use the new more-efficient style of reads, and that's dropped the savegame file size down to 2.2886 MB from that original 2.3449 MB we started with in this build.
  • That's another 2.4% savings compared to what we had in last night's build.

• Now switched over Entity Systems to use the new format.

• All of the "lookup name by index" type things for dynamic data tables like GameEntityTypeData and such have all been transitioned over to to using adaptive UE values from GetUltraEfficientStyleThatFits()
  • This plus the entity systems change drops us down to 2.2642 MB from 2.3449 MB, another 3.5% improvement in file size.

• A bunch of extra info has been added to the world serialization and deserialization logs.
  • These are absolutely HUGE, even larger than ever before (from 40mb to 80mb in one test case for a 2mb save, for instance), if you have these turned on. However, you can also often see individual field names now, so the added clarity is enormously high.
  • A lot of other methods now contain the ability to send a field name for errors as an optional parameter, so that it can be added to this display. If you have a mod that you don't recompile against this, even if it has no errors before you recompile, you'll get method missing exceptions.

• The main primary keys on entities of all sorts are now in the more efficient format, taking us from 2.2642 MB to 2.2464 MB.

• We are no longer bothering to save/transmit the strength counting data in savegames, because that was utterly pointless.
  • In a midsize game it was some 100 KB, and it gets recalculated on ever sim frame prior to the sim frame running, even while paused, on the clients and the host. So within 1/10th of 1 second of connecting, the data was completely discarded anyhow.
  • This brings our test savegame down to 2.1107 MB.

• The game will now only serialize a maximum of 63 orders per ship at once (good grief that would still be massively more than we can imagine needing). This saves a "whopping" 2.2KB.

• The way that OrderSource is stored is now just using 1 bit, which saves 25 KB. Wow.

• The max cloaking points that ships can have is now 32767. The max that any in the main game actually DO have is 20k, and most of them have far less than that.
  • This lets us store the cloaking points lost in the UE format "M_0_To_32ˌ767_Def0"

• Added some new 26-bit integer formats that let us track numbers up to 67 million. Turns out the very largest health ships in the game would fall under this.

• The maximum number of seconds that a ship can be forced to wait for repair is now 511 seconds instead of 600, letting us store this in a much smaller-format number.

• The maximum number of ships of a single type that can be stored in an AI reinforcement point is now 4095. The maximum number of types of ships one can contain is now 511.

• Converted a number of things on units/squads/ships to use the new more efficient methods that are adaptive for the primary key IDs, or to time for gameseconds.

• Total drop in filesize from the above changes is down to 2.0933 MB from the prior 2.1107. A 0.8% drop is not exactly what we had in mind with this, but then again it will scale better with different savegames depending on what their data is. This is just using a single benchmark savegame right now.

• Fixed up a few things with the damage amplification to be a little simpler and slightly more efficient, but it's so infrequently used that it's not really the problem.
  • At this point we are conscious of the fact that what was precise has turned into shooting in the dark; we need more detailed logging to identify the remaining problem cases other than strings (which will be big and easy when we get to them).

• All of the read and write serialization methods now take a string fieldname for errors.
  • This is also now used for the deserialization logging, optionally, if the programmer passes in an extra TrackerStyle.ByFieldNameAsWell. TrackerStyle is now required when starting a tracker.
  • The idea is that now we can drill down to individual fields, where appropriate, and find out what the heck is exactly using so much space -- not just by type, but by specific field.
  • In short, this gives us the power to no longer be shooting in the dark when there is a section of code where the variables types are too dense or vague for us to understand why they are having issues.

• Squads now export their information per field, which lets us target the fields that are actually taking up the most space.
  • Turns out that world location for squads is STILL taking up a whopping 113.7617 KB, which is more than any other field. There's an average of 32 bits per X,Y coordinate pair, and there's just no way to make that smaller without also making it less precise.
  • Less sensibly, ActiveHack_Target is taking 81.7662 KB, with an average bits of 23.00??
    • It turns out that this was because of our new UE formats with Neg1ToPos not having a special two-bit case for the value of 0.
    • This was in an attempt to save 1 bit on such fields, but ultimately this requires too much of the end programmer and is likely to lead to filesize bloat in other ways.
    • Making the shift to have that extra bit in there, but handle the two-bit case efficiently, lets us shave this down to 2.0031 MB for our test savegame from 2.0933.
    • But honestly... this feels kind of gross, because we're using an extra bit on a LOT of numbers to compensate for programmers not being precise enough. That's okay with the normal modern format of buzzsaw binary, but with the ultraeffecient version that seems antithetical.
    • Instead we adjusted the methods in ArcenSerializationAIWar2SpecificExtensions to auto-correct data that is being sent in, and if it would be a 0 in the case where that means nothing, but would take a ton of bits to store, then it puts it to -1 instead. This is a healthy middle-ground between auto-correction and data efficiency.
    • The end result of THIS method was instead 2.0169 MB, which is slightly more wasteful, but clearly that means we have some lingering bad data in there somewhere... or it's just the absence of savings from so many things that have a 0 as part of their valid indices. Long-term, despite slightly worse compression, this seems like the way to go because it would likely lead to good results eventually.

• Added a new AddIntUltraEfficient_Clamped, which lets us automatically clamp rogue data when we are writing a savegame, versus having some sort of error pop up or having to handle data ranges that are wider than they need to be.
  • • There's one version called AddIntUltraEfficient_ClampedDoNotAlterOriginal, which is not really ideal for network play as it doesn't affect the original value. The other version alters the original value to match.
    • In most cases it probably doesn't REALLY matter which one is used, since they probably don't affect sim or things that would trigger a desync, but it's good to have the flexibility to adjust as needed.
  • The items that were using the most data were things that surprised us, so all the extra instrumentation saved us a ton of time and trial and error as these would not have been high up our list.
    • CurrentParalysisSeconds converted over, was using 56.8809 KB. Now uses 3.5551 KB.
    • CurrentWeaponAddedReloadSeconds converted over, was using 56.8809 KB. Now uses 3.5551 KB.
    • CurrentEngineStunSeconds converted over, was using 56.8040 KB. Now uses 3.5558 KB.
    • NumExtrasToCreateWhenTransforming converted over, was using 56.8040 KB. Now uses 3.5551 KB
    • NextGuardingOffsetIndex converted over, was using 56.8809 KB. Now uses 3.5587 KB.
      • Also adjusted the number of GuardingOffsets to have a maximum of 63 instead of 200, since that's just ludicrous amounts in general.
  • Converted Guarding.PrimaryKeyID to the general new AddSquadPrimaryKeyID_Neg1ToPos. Was 9.9150 KB, now 7.6639 KB.
    • SecondsTillTransformation converted over, was using 56.8809 KB. Now 3.5551 KB.
    • FleetID switched to use the central AddFleetPrimaryKeyID_Neg1ToPos(), was using 33.2352 KB, now 42.6606 KB (huh!?).
    • Metal flows statuses were using 35.5505 KB for ActiveOutflowAmount (291,230 reads!!) and 28.4404 KB for MetalFlowPurpose.Length, when these almost never need to be used. This has been switched over to an intelligently-gated format.
      • ActiveOutflowAmount now uses only 11.2500 bytes (90 reads), and there are 4.5000 bytes used for MetalFlowPurpose.Length (9 reads).
  • After all these changes, we're down to a filesize of 1.6519 MB where it was 2.0933 at the start of the day. That's a 21% reduction! Now we're talkin'...

• For really long games, we need some better ways of storing some of the information about time intervals in them.
  • We've added a new GameSecond_Neg1ToPos_Differential_Clamped method group to handle this.
  • LastGameSecondOfDecollision has been renamed to LastGameSecondOfDecollision_OrZeroIfMoreThan15sAgo, because we really don't care about how long it's been if it's been more than 15 seconds.
    • This was previously taking up 57.6484 KB of space in our savegame test case that we 10 hours in. It now takes up 5.5648 KB.
  • There's really nothing else in the game aside from strings that is common enough in our main test case games to be worth shrinking down. We're at 1.5698 MB, 2,812,809 reads, and everything is so hyper efficient except for strings that that is the last avenue open for us. We'll sort that out and then move on from this filesize-chasing stuff.

• Split the way that strings and chars are serialized int two tyoes: FullUnicode and Condensed.
  • The only reason to use FullUnicode is if it's something that we're relating to the file system in some way (reading, not writing). But even that is a stretch.
  • There are only about 102 characters that the game actually uses, and frankly it doesn't really use all of those. The fonts supported by the game can't even display quite all of those.
  • We thus are able to map what is normally a 16-bit character encoding into our own custom 7-bit super-lightweight encoding that includes all of ASCII that is relevant to us, plus some bits beyond standard ASCII that were in our InternalNames for various ships or whatnot.
  • Anything that is outside of this range will be converted to a ? character.
  • If a modder uses unsupported characters in the "name" field of a ship or system, then that ship or system will not be loadable properly when they save and then load their game.
    • It's worth noting that there are no such restrictions on the display_name, but those already would just show up as blank characters anyhow.
  • This set of changes drops the total size of our test savegame to its final size of 1.3444 MB. We don't think we can get it any smaller than that.

FInt Data Type Internals

• FInt has been converted over to using a "multiplication of 1000" method instead of a "bitshift by 12" method.
  • This lets us keep doing fixed-integer math where we need to, and we've tested it in unity's implementation of mono and we see no measurable practical difference in the speeds of either approach.
  • The accuracy of this new method is much improved, however, as we can appropriately represent numbers like 0.9. Previously, based on how integer math works, we could not represent that number except as 0.899.
  • It turns out that, ever since we've been using FInts (2010 or so??), we've had this problem with representing post-decimal digits in FInt. So a multiplication by what we thought of us 0.9 was actually a multiplication by 0.899, which caused various confusion in the interface when it was large numbers being multiplied-by (where did that extra 5 damage come from, etc).
  • The accuracy is now perfect in terms of storing numbers, pre and post digit, and the accuracy of operations was already perfectly accurate in an integer-math sense given the limitations of how many significant digits we allow for (three). The existing pieces of accuracy have not lowered.
  • All in all, this is a minor improvement, but something we wanted to go ahead and get in during our data transformation work. We had worried performance would be worse, but for various reasons to do with how we structured our FInt class, and in how modern processors (those from the 2012 and on) handle division -- and the fact that everything is 64bit processors now -- it's functionally identical in speed. Back in 2010, this would not have been the case.
  • Curiously, in our test savegame, this did actually shrink the size of floats and ints very slightly.
    • Floats (stored as FInts) dropped from 26.61 avg bits (798.2500 bytes total) to 24.56 avg bits (736.7500 bytes total).
    • Direct FInts dropped from 4.73 avg bits to 4.68 avg bits, for a total drop of less then 0.2 KB. Why not; that wasn't the goal here, anyway.
  • Thanks to -NR-SirLimbo for discovering the issue, suggesting a solution, and investigating this with us.

Beta 2.065 To Infinity And... No, Knock It Off

(Released June 11th, 2020)

• If seeding multiple copies of the same faction type (for example, 3 Nanocausts), make sure we don't put those copies next to eachother. It tends to lead to boring gameplay where those factions just try to kill eachother.
  • Thanks to ArnaudB for the bug report

Solving An Odd Case Of Infinite Waves Caused By Games Started In A Recent Beta

• Split out a number of methods into sub-methods so that our stack traces will actually be useful when a thread gets hung. We have one savegame where background threads are running infinitely and stopping play, but there's a bit more than that going on.

• Fixed what was ultimately a harmless exception that could happen when you exit to the main menu if the game is running extremely slowly.

• Fixed a bug where the AI could add null waves into the wave list, causing all sorts of problems.

• Put in a limiter where if a single faction tries to add more than 200 waves at once, it will throw an exception.
  • In one savegame, we were having linear growth of waves, causing the entire simulation to stop.
  • Most of the waves were null, but there are still north of 200 waves generated, so that's a piece we still need to fix separately from that.

• PlanWave() has been renamed to PlanWave_OrGetNull(), since it's possible that these don't actually plan a wave if they are for instance trying to do a type of wave that there is no valid target for, or there's no budget for it, or whatever.
  • This is a reminder to calling code to check for nullness and not do things like increment counters for successful waves or whatever if it is null.
  • Adjusted all the surrounding code to account for this. The only one that was notably affected was wormhole invasions.

• Essentially, some saves started in 2.062 might have their allegiances so messed up that AIs are generating waves against "natural objects," aka metal deposits and similar. If you started your game fresh in 2.062 beta, you really might want to start a new one...
  • Players with save games from 2.062 will receive a warning message saying "You probably should start a new game" when they load that game.

• In the escape menu, you can now see how many waves the AIs currently are launching at the moment. They might be null or not actually "waves" in the classic sense, and they might not all be against you. But if the number is super high, it's at least not invisible and we'll know something is up.
  • With waves, if there are more than 10, it now shows up as a red number in the escape menu. Same for the "last cycle time" entries if those are more than 5 seconds.
  • This way it's even more obvious when something is amiss.
  • Fun fact, now that we have this display, we can see that even in the broken savegame where it still generates 200+ waves at the moment, it then quickly collapses back down to 2 and cancels the rest.

First Usage Of The New UltraEfficient Serialization Variants

• Implemented a new ArcenAIWar2Core class called ArcenSerializationAIWar2SpecificExtensions, to let us efficiently and consistently serialize and deserialize certain data types.
  • The first of these are "combat space ArcenPoints," aka those that are coordinates within a planet's gravity well.
  • We're able to do several things to make these a lot more efficient. Previously we had to store these as signed int32s.
  • Now we're able to first subtract 200,000 from each of their coordinates (to make them centered around zero), and then store them in a signed 15-bit integer instead (that in total takes 16-17 bits to store the way we do it).
  • Because there are sometimes values set to "literal 0,0", we also added a bool check in front of them for that state, which lets all those defaults be stored in one bit for the two numbers.
  • We then used this for simply two positions the ships have, WorldLocation and DecollisionMoveTarget.

• Added and tested another 14 variants of UltraEfficient number types, these being signed but with no default value.
  • These are less efficient in almost all cases, except if there's something unusual like the above ArcenPoints where the zero case is already gated in an external fashion.
  • All in all this lets us shave an additional two bits off each non-0,0 point pair.
    • Given that we're talking about 2 fields on 29,123 ships in the test savegame we use for a lot of these cases, that adds up to just over 7KB. For savings from just two fields, that's nontrivial!
    • Hey, wait a second -- why isn't it more than that? Because only 29,123‬ out of the 58,246 fields were non-0,0. Those get gated behind a simple central bit, as noted above.

• In our go-to savegame for testing serialization and deserialization with a ton of entities, there are 29,123 ships/buildings/squads/units in there.
  • When last we looked at this, before our UltraEfficient additions were put to use at all, the savegame was down to 2.4009 MB with 3,453,855 data points read.
  • We've now switched over WorldLocation and DecollisionMoveTarget, just two fields on the GameEntity_Squad class, and that has reduced the savegame file size to 2.3449 MB, with 3,508,900 data points.
    • Prior to the additional data formats noted in the above section, it was 2.3518 MB
  • The number of bools grew from 941,752 at 114.9600 KB to 999,998 at 122.0701 KB.
  • The number of Int32s fell from 174,250 at 205.7513 KB, to 57,758 at 27.9987 KB.
  • Then we also now have a new group of 58,246 UE15Bits at 113.7617.
    • Prior to the additional data formats noted in the above section, it was 120.8230 KB
  • In other words, from just changing two fields on the GameEntity_Squad class, we've reduced the savegame filesize by 2.4%. Good grief!
    • Processing-wise, this also means it did 873,690 few CPU operations to save these new numbers, as well. These are very small operations, measured in individual nanoseconds, but still. When it comes to passing a lot of data around across the network in multiplayer, it's not something to sneeze at.

XML Reading Improvements For DLC and Mods

• Added a few dozen new "FillArrayIndex" methods into our ArcenXML class.
  • Normally we use a method called Fill, which takes in a variable by ref and assigns to it only if there is actual new data. This keeps things so that they work with is_partial records which are missing lots of values (by design) since they only want to change some values. This would primarily apply to mods, or to some extent DLC. Very occasionally to variant units.
  • At any rate, for things that are in an array or a list, you can't pass them in by ref -- so the natural instinct of the end programmer/modder is to make a temporary variable and pass that in. BUT, if they do that without initializing the temproary variable to the existing value of the array at a given index, they'll wind up getting a default value (zero, null, whatever) back in places where a partial record is used. Aka, mods and DLC won't be able to use partial records without breaking something else.
  • This new set of methods let you pass in the array or list itself (not by ref since that is implicit on arrays or lists), and the index in it that you want to assign to. This keeps the general pattern of Fill, and gets rid of the need to use a temporary variable, and thus makes the code shorter on the end coder location as well as making it work with is_partial records for DLC or mods.
  • Thanks to Badger and zeusalmighty for discovering this problem.

• Fixed three methods in the Mercenary class that were using Fill() with a temporary variable and a string, and then assigning it in. This would not work with partial records. Now it uses the proper FillEnum method, and thus will work with partial records. Since these are not in an array, it doesn't need to use the sub-methods.

• There are a a dozen or so places in GameEntityTypeData where we are using working variables to read in an attribute via the fill method, but these are all already okay for one of two reasons:
  • Either they check to make sure that the attribute exists in the current xml file before doing any parsing at all, so they never overwrite with default values.
  • Or they read no matter what, but then check to make sure their value is not the default value before overwriting the existing value.
  • These are all funky pieces of data, usually lookups into another table or something that requires a modifier before being assigned to the direct value, hence why they deviate from the usual patterns established elsewhere. They should not be emulated unless absolutely required.

• On the AI difficulty table, the following xml is now read in via a loop and using the new fillarrayindex methods, making it safe for is_partial_record.
  • income_for_extragalactic_war_tier
  • hacking_wave_multiplier_
  • The ArcenSparseLookup<> for HackingDifficultyMultiplier was also changed to a List<> to be compatible with this methodology, as well as more efficient in general.
  • hacking_level_
    • The ArcenSparseLookup<> for HackingDifficultyLevel was also changed to a List<> to be compatible with this methodology, as well as more efficient in general.
  • aip_for_mark_
    • The markless version of this was also renamed to mark_0 in the xml so it could be read linearly.
  • These do need to be tested to make sure they're what we want, but they should be good.

• Added a new FillDictionaryKey<> set of methods for most of the data types (strings and numbers) in ArcenXML, so that we can fill dictionary entries safely with these directly, while maintaining is_partial_record compatibility.
  • The dark zenith costs and outputs (DLC2) have been converted to use this.
  • This also needs to be tested to make sure it works, but it should.

Beta 2.064 Fixes to Allegiances

(Released June 11th, 2020)

• Shark Plots now trigger off of AI sub-fleets like Instigators or Hunters as well as the AI. Shark Plots will continue to not trigger for AI-allied minor factions like the scourge or marauders.
  • Thanks to Lord of Nothing for reporting. This is a bug that's been in the code since Badger first added Shark Plots.

• Scrapping a player command station is no longer allowed to cause reconquest warp gates to die
  • Reported by ANGRYABOUTELVES

• Fix a number of problems related to faction allegiances; this includes the AI killing neutral structures and a number of Scourge related issues.
  • Note that if you started a save game on the previous beta you may still see a variety of problems.
  • Thanks to a large number of people for reporting
    • Note: players will see odd issues in save games from 2.063 beta patch

• Fix a null reference exception you could hit by leaving the ARS hacking popup open when leaving one tutorial and entering another tutorial
  • Thanks to EtoileLion for reporting

• Take another pass at making the Teach A Man To Fish achievement work
  • Prompted by a report from CRCGamer on steam.

• The "Win on 200 planet map" achievement has been changed to "Win on 160 planets" instead, since we reduced the maximum map sizes to 160 across all maps.
  • Thanks to ArnaudB for reporting.

• Created a new extension to our deserialization and serialization logic, named "UltraEfficient," which lets us specify extremely granular information about how we want to save and load data.
  • Basically exactly how many bits to store is built in. Thanks to UncleYar for mentioning he'd seen something like this in another game engine.
  • We kept our other innovations with things like default values using only 1 bit, and the result is something that blends the best of the method of others, and what we had in our buzzsaw binary. This is a lot more efficient to write than buzzsaw binary, too.
  • This is something we have implemented and tested, and it works, but we have not actually integrated it into the game itself yet, so there are no savings to report just yet. We'll be spot-implementing this in fields that are commonly used and which can benefit from this sort of compaction. For the rest, we'll continue using our prior approach, which is slightly more adaptive than what we have heard about in other game engines or network libraries.

Beta 2.063 Fixes and Tweaks

(Released June 5th, 2020)

Note: we discovered an issue in the 2.062 version that will cause strange issues with campaigns you started in that patch. Things like the AI hating neutral factions, or unusual factions being allied. Apologies for this! Games started in a prior version to 2.062 but played through this one should be fine. It's just new games started in Beta 2.062. This patch fixes that problem

• The Parasitic Starting Fleet now includes 25 Parasite Hydras in addition to the existing 40 Parasites and 2 Muggers.
  • This was a bit of an underpowered starting fleet after having some changes to make it less overpowered a while back.
  • Thanks to tadrinth and Lord Of Nothing for suggesting.

• The "strength counting data" experiment we tried last version was a disaster on pretty much every level, causing lots of confusion in the code and definitely showing that that was not the way to go with GameCommand. The strength counting data has been returned to how it used to be, which fixes bugs like the galaxy map not showing your enemy strengths properly.
  • Thanks to Badger, Metrekec, and others for reporting.

• Discovered that some of our changes when we added fleet-wide effects for player ships caused AI ships to show their own healths in a strange way, or outright have the wrong healths. We fixed that partially last release, but overall there were a lot of things that just needed to be separated out for humans and for npcs (AI or otherwise). Those things are now separated out properly and no longer cause any strangeness that we're aware of, like AI ships having "more than 100%" health (when really they were just at full health).
  • Thanks to Ovalcircle for the report.

• Fix a bug where the AI reserves were winding up neutral towards the player
  • Thanks to x4000 for the bug report

• Fix a bug where player-allied scourge requested in the game lobby weren't spawning at the beginning of the game
  • Thanks to ynof for reporting

• The AI now responds to players hacking a minor faction beacon to get a new faction; this is the Scourge, Devourer or HRF. The unlock Fallen Spire campaign hack does not provoke an AI response and explicitly says so in the hack description.
  • Thanks to Tadrinth for the suggestion

• Some tweaks to the anti-structure Hunter ships; in general fewer are spawned because they felt OP. Also improve their ability to find targets; if you have 2 MDCs then all ships against MDCs will be able to try to make an optimal choice of which to target using generic Fireteam code during the game, instead of being locked in at spawn time.
  • Thanks to Democracy for some balance concerns

• Fixed a bug in the beta branch with risk analyzers throwing an exception every time they tried to fire. This was related to their data being stored as an Int16 now, but some extension methods still trying to cast them from an object to an Int32. For some reason this was causing an invalid cast exception, which surprises us since the data is a valid Int32 and can be directly cast from an Int16. But apparently it being stored in the super-old format where it's an internal object class invalidates a direct cast. Most likely this is due to boxing of a value type in and object type causing the normal casts defined in the core language libraries to not kick in. It's yet another good reason to avoid the super-old style of external data storage we used for Risk Analyzers.
  • Thanks to ynof and BadgerBadger for reporting.

Beta 2.062 Savegame/Networking Data Compaction

(Released June 5th, 2020)

To play this on Steam, please go under betas and choose current_beta from the drop-down list. We would really appreciate some testers on this so that we can get back out of beta status as quickly as possible! There is not currently a way to get the beta versions on GOG, but we won't be in that status for more than a week, knock on wood. We want to make sure we didn't break anything with all the substantial changes in here, but we fully expect some savegames to throw cosmetic errors at the very minimum. Please report and upload those here: https://bugtracker.arcengames.com/

• Add some new Planet Names based on ancient Mesopotamia

• Allow the Outguard to kill Dyson Antagonizers
  • Thanks to GreatYng for suggesting

• Move the "Scout the planets" objective into Beginner Objectives

• Imperial Spire Ships should now grant vision so you can watch the Fireworks
• When the AI spawns a ship from Spire Debris, it takes the general mark level of the AI, and is no longer just mark 1
  • Thanks to ANGRYABOUTELVES for the suggestions

• Improve the Brownout notification hovertext
  • Thanks to Lord of Nothing for reporting

• Added a new PKID class, and a PKIDGenerator.
  • These are not actually used yet, as they will complicate things and make save files larger again, probably.
  • This will be really important for multiplayer sync, particularly not in having rolling cascades of incorrect data that requires corrections. But as Chris started to get this implemented, he decided to wait at least one version for getting this all the way in there.

• Added a PlanetImportanceUIOnly, which is something we can later use to let players do AIWC-style P1-P9 markings.
  • It's just the data storage right now.

General Fixes

• Fix a null reference in the experimental astro trains code
  • Thanks to NRSirLimbo for the bug report

• Fixed a rare but longstanding but where the message "Potential bug: planet [name] has no metal generators. This is unexpected" was showing up several times a second in some late-game macrophage games.
  • Basically this would be triggered at times where the metal harvester points were all too close together for the macrophage to be able to patrol between. Now they just hang out near the cluster of them when there's a planet with that sort of situation going on.
  • Thanks to GreatYng for reporting.

• Fixed another instance of "DoShotMovement: Error at debug stage 3200" that could happen if your background threads raced just wrong.
  • Thanks to NRSirLimbo for the reporting.

• Fixed an issue that may or may not have come up before, but where fire teams could not be found, even if they existed, if at least one game second of the long range planning thread had not been run already.

• Fixed a couple of places in the macrophage code where it was making decisions based on what the humans are able to see of threat, not based on what actually exists for it to see of threat. Totally an accidental discovery, we'll see how that affects macrophage behavior when you don't have vision on its part of the map. It should be much smarter in those cases, now.

• Improve the Major Data Center objective text.

• Fixed a bug that could result in Spores being unable to move off planet when using smart expansion logic in a small section of the galaxy.
  • They were basically trying to go to the midway point between the two planets constantly, which doesn't exist if they're adjacent.
    • Thanks to GreatYng on Mantis for the bug report and saves -- and StarKelp for fixing.

• Fix several null references in Notifications
  • Thanks to GreatYng for reporting

• Fix a bug where AI Spire Research Labs and Citadels could spawn on neutral planets; they are now required to spawn on AI planets.
  • Thanks to GreatYng for reporting

• Fix a bug in the brownout notification hovertext
  • Thanks to fwiffoforce for reporting

• Partially constructed Logistical Command Stations no longer grant vision
  • Thanks to GreatYng for reporting

• Improve the lobby text for Risk Analyzers to clarify things
  • Thanks to GreatYng for reporting.

• Fixed a very old and very annoying bug where any strike or officer fleets you tried to assign to the None hotkey would reassign themselves a second later.
  • Thanks to deo, Ecthelon, and Admiral for reporting.

Achievement Fixes

• The "Teach a Man to Fish" achievement should now work
  • Thanks to Lord of Nothing, fwiffoforce and Thrankos on steam for reporting. Probably others as well.

• The "This Will Definitely Make Things Better And Not Worse" should now work
  • Thanks to Lord of Nothing for reporting

• The So We're the Bad Guys Now? achievement should now work
  • Thanks to Lord of Nothing for reporting

• The achievements for beating the Bar Fighter AI Type should now work
  • Thanks to ArnaudB for reporting

• The achievements for beating the Kite Master hunter fleet should now work
  • Thanks to Lord of Nothing for reporting

Major Fireteams Upgrade For Some Factions (Backported From Upcoming DLC2)

Please bear in mind that none of the work that is happening on DLC2 is slowing down multiplayer at all. These are two independent projects by Badger and Chris at the moment.

• Fireteams have been reworked to allow for factions that opt in to support requirements for "Which faction(s) or units am I required to attack?"
  • Currently only used for the Hunter Fleet in the base game.

• The goal is to allow the AI have very strong responses against secondary factions that won't also kill you
  • For example, lets say you've requested a pretty OP secondary faction team of Nanocaust, Scourge and Marauders.
    • The AI can now spawn Extragalactic War Units that are only allowed to go after that minor faction team (which allows the AI to defend itself much more effectively).
    • When these special Extragalactic War Units defeats its enemies, it will warp out of the galaxy.

• This lets us make more terrifying minor factions without them just ending the game (by attacking you, when you didn't do anything), and is a required item for DLC2 (because that's going to have some terrifying factions) that is backported to the base game.

• Somewhat related: allow the Nanocaust and Marauders to have an Overall Power Level.
  • This plays into the way that the AI is able to respond to things that are "OP", in the way that we've got them reacting to the Fallen Spire in DLC1. But this lets that sort of response be mounted in purely the base game.

• Also add a mechanism for hunter fleet fireteams to be created to target specific units in the game.
  • We can use this mechanism going forward to try to let the AI have a bonus response if you take certain very powerful objectives, which allows us to create stronger capturable structures (since we can invisibly balance them by just giving the AI a bit more oomph that will force you to defend those structures very carefully.

• Also non-scourge minor factions can now have explicitly Defensive Fleets. Currently not used for any base game factions.

Balance

• Fix a bug for the last two weeks where all AI units had the hull/shield values for Mark 1 units. This was a significant unintentional nerf to the AI
  • Thanks to a number of people for reporting, including Starkelp and Democracy

• Player-allied minor factions no longer kill warp gates by default. Instead the Hunter Fleet gets lots of bonus ships intended to go after your allies.
  • Needs lots of balance testing

• When Marauders attack with their ships, those ships can now attack other planets through wormholes. Previously those ships were required to be defensive, and the Marauders would need to build Raiders to spread through wormholes.
  • This is a potentially significant buff to allowing Marauder Empires to get off the ground quickly in unguarded areas of the galaxy.

• When you take a Major Data Center the hunter fleet will just get a bit stronger with hunter fleet ships that will be focused exclusively on killing your MDCs.
  • This removes the anti-MDC exo waves, which were a hacky/kludgey response at best. This is probably a nerf to the AI right now and will need some tuning

Savegame File Size Reduction

• Arcen's xml reader now supports directly reading 16bit integers and individual bytes from xml fields, which fits with the new variance in types that we have coming up in the data stored throughout the game.

• The game now explicitly mentions that Immediate Invasion Nanocaust is very hard

• Put in a couple of improvements so that the game is now able once again to load a good savegame after failing to load a bad savegame.

• Adjusted the serialization class to let us specify larger buffers, and buffer growth rates, to prevent cases where incremental small additions cause repeat buffer resizes. This was slowing down even loopback network transport at the moment (aka single player), and slowing down game saves.
  • This is a performance improvement, not a data filesize improvement, hence it not being in the section below.

• Removed all of the types of serialization code that we have except for Keith's BuzzsawBinaryArray.
  • We were wasting some cycles -- not a lot, but some -- on checking what mode we were in, and if there were delimiters, and if there were type labels, etc. Since we use none of those features, stripping it down to just the highly-performant core is a good idea for performance of network message encoding and decoding in particular.
  • This also does help make the code a bit easier for us to follow, since it's all one mode now.
  • This is a performance improvement, not a data filesize improvement, hence it not being in the section below.

• Added a new "Write Deserialization Data Sizes" debug option in personal settings.
  • loading a game, write WorldDeserializationDataSizes.txt in the PlayerData folder. When a savegame is really large, turning this on lets us figure out what is taking up so much space so that we can potentially make some optimizations.

• Added some complicated new classes and write structures to actually pull out data being read by type, and aggregate it as we wish, so that we can write the above sort of information.

• Improved the way that the "external data patterns" are saved in savegames.
  • On a test case 8mb savegame, this saves half a mb.

• A bunch of extra instrumentation has been added into the game for savegame deserialization and serialization, both, to let us see what is happening and why.

• Found a trio of places directly in fleets, and two places in "minor faction common external" data internal dictionaries that were serializing ship names by name instead of index.
  • Converting this to use indices instead saves another 0.3 of a MB off of the large test savegame case we found.

• New debug settings option: Write Deserialization Fireteam Data
  • When loading a game, write information about all the fireteams you're loading to disk. When a savegame has a ton of fireteams and you don't know why, or who owns them or what they are doing, this helps debug.

• Got rid of the ExtensionMethodsFor_SpecialFactionData class, which was something that required editing a central file in order to get certain faction features. Those factions features are now implemented as virtual methods that can be overridden by other classes with us blind.
  • This did require us to add a FireteamBase class in the main ArcenCore code, but that was a solid tradeoff for making fireteams automatically work with any potential mods in the future without them having to use any central methods to do so.

• Refactored various fireteams code to get less repetition and more centralized code for a few basic things.

• Fixed a few places with scourge and nanocaust fireteams to clear our disbanded fire teams each long range planning pass if they were somehow missed before.
  • All of the other factions did this already, but adding this to these two shrunk the test savegame we had from 7mb to 4mb on one save and load cycle, then to 2.8mb on the next cycle.
  • This tells us that, somehow, duplicate copies of fireteams were probably being logged into at least one of the central lists, which would have this effect as well as make processing a lot slower since it would process a few things multiple times.

• ArcenLinkedList has been split into two.
  • There is now ArcenOverLinkedList, which is still used for the three main GameEntity classes.
    • This one has a couple of funky hooks in for when an entity dies, which is appropriate for them but for nothing else. They're connected on purpose for benefits like performance and cleanup when something dies.
  • There is also now ArcenLessLinkedList, which is used for all of them, planets, fire teams, frenzy fleets, and more.
    • These basically treat each list as an independent entity, and there's no concept of an entity "dying and being removed from everything." They're independent on purpose.

• Converting over so many things to use the ArcenLessLinkedList was a huge number of code shifts, so hopefully we didn't break anything with that.
  • We did fix some minor inefficiencies as we made these changes, aside from the efficiency boosts of this being a thing in general.
  • Now that this is done, however, we immediately get the benefit of on the first save-load cycle it takes the 7mb save down to 2.8mb. In other words, duplicate fire teams are no longer possible to accidentally have, because their lists reject that.

• The number of fireteams has been cut down dramatically by the fix to deprecated fireteams hanging around in general, but fireteam history items are still very long compared to what they could be in terms of data size.
  • In the example savegame that was originally 8mb, which is now down to 2.8mb, we are left with 790 fireteam history strings, which total 219.6729 KB in savegame space. This is an average of 284 bytes per history item.
  • We have implemented a new history item style which stores between 2 and 5 integers, with the most common case being 4 integers.
  • Normally a 32bit integer requires 4 bytes, but thanks to Keith's buzzsaw binary array, our average storage space required is 1.79 bytes in practice.
  • We can't actually measure the effect on this exact savegame, because old history items will be kept in the less-efficient string format. But it will only keep the most recent 2 history events that are string-style, cutting it down some (often they had 3 or so).
  • However, if we take a general average of assuming that we had 790 events that were in the new format, and on average have 4 integers with, let's say, 3 bytes required for each (to be even more pessimistic), then we would go from 219.6729 KB in savegame space for those fireteam histories down to 9.2578 KB. That's a 96% compression!
  • Bear in mind that since this formmat is new, we might have introduced some bugs in the display of fire team histories, but hopefully not. It also lets us make the formatting of them be more consistent, since it's all centrally defined now. Testing thus far has been promising on the bugs front.
  • Interestingly, after letting the savegame in question run for about 14 minutes (gametime), it did increase in size by about 100 KB (huh?). But upon loading the new 14-minutes-later savegame, the fire team data had dropped from 237.6553 KB down to 21.6162 KB total for the fire teams and their histories. That is with 572 fireteams, and it looks like 10 old-style text histories and all the other histories being in the new style.

• Mostly as a test case for something we want to do later with GameCommand, but also as a way to improve the data storage requirements (trivial) and calculation requirements (less so) for "strength counting data" for each faction, it has been split into several subordinate data structures which are only calculated when actually relevant.
  • In a game with 22 factions on 80 planets, this amount of data drops from 129.9697 KB to 94.9980 KB (see? trivial).
  • However, this does translate into only 54,651 integers being calculated 10 times per second, down from 75,238 integers 10 times per second. And that is less trivial, as those integers are not all trivial to calculate.
  • We may be able to cut this down even further, but on maps of reasonable sizes it doesn't look like this is a real bottleneck, so this remains for now just a useful test case (the test shows that this sort of thing can work quite elegantly, which is great news).
  • In savegames from before this game version, loaded into this game version, there may be strange decision-making for 1/10th of one second. But actually, probably not even that because it recalculates this even while you are paused, so by the time you could possibly unpause the data is there again. This game is kind of a technological beast.

• Also moved the strength counting data code into its own file for easier editing in the future.
  • Also also, this provides the template for us to have more specialized strength counting things added in the future for just some factions without it being something that loads down every faction's processing time. Which could be useful or may not be.

• The way that entity systems are serialized is now far more granular, using some serialization flags that are written at the time of saving a game (or transmitting across the network) to say what kind of data to expect.
  • This lets us omit a lot of data in cases where it is a bunch of blanks or otherwise not needed (becuse of the type of system not needing that type of data, etc).
  • The cost of this is one extra 32bit integer, in an enum flag format, and the savings is pretty substantial. This is primarily a test case for what we will do with GameEntity itself, although it's also useful inherently on systems -- to some extent we were avoiding adding too much data onto systems, because of risk of bloating save files. Now we have the mechanisms by which we can do it.
  • In our existing large-savegame test case, this moves the system data from being 166.4932 KB (not huge, we know) for 29,123 ship systems, down to 118.4746 KB for those same number of ships. That's not exactly a giant compression of data, but it was the test case after all.

• There were some very old player profiles stored in a different serialziation format which we simply no longer support. It will probably affect almost nobody, since it's been prior to version 2.012 that profiles were in that format, and if it fails to load the old profile they just create a new one and nothing is really lost.

• Fixed an internal issue where we were calculating the amount of bytes being deserialized slightly wrong; individual pieces of data are stored in a bitstream that can span a couple of bytes, and so typically this meant that we were overestimating how many bytes of data was actually being read.
  • This gives us a more accurate view of how large our data actually is, particularly as we drill down more.
  • The overall data size was corrected from being 2.5705 MB to 2.5702 MB, so as you can see at a macro scale it is hardly any difference at all.
  • Oddly, we discovered that a lot of bools were not properly being logged, although their macro file size was. Not sure what was up with that, but it gives us a more correct picture now. That does change our total reads of data points in the example file from 1,329,568 to 3,420,471, though -- now sub-tracked all properly. 859,317 bools read out at a size of 104.8971 KB, versus the wrong statement of 107,830 bools read out at 105.3027 KB.
  • 32bit integers were in the same boat, more or less, with 1.9432 MB being the agreed-upon correct amount for both new and old formats. But in the new format we can see it was actually 2,362,745 ints making that up, not just 1,143,492 of them. That makes a big difference to how we evaluate the cost of things in our saves.

• Adjusted how we are saving the data for entity systems, based on how we've learned about things from the deserialization log accuracy improving.
  • Essentially, a blank value (empty string, 0 on an int or similar) is just one bit. Using a large int32 bitflag array, we don't save enough space to warrant using them. We should instead be using bool gates.
  • BUT, because of the tiny size of blank values, we need to be using them ONLY for places where we would be saving at least a couple of real values, or 2+ blanks, in order to break even on space or save space.
  • This is harder to do in small data classes, but an easier rule of thumb and actually a lot easier to read in code.
  • In this particular case, the data stored is actually NOT smaller, but this is because of some of the data types we're using, not the methodology of actually doing the saving. So we'll be improving those data types, along with a number of other things.

Revised Serialization Format

• As part of our changes to our serialization buffer writing numbers, we now have it do a few quick comparison checks to do far fewer searches in a loop.
  • This is a major CPU timesaver during serializing a number for either disk or the network. Literally in an average savegame it may save as many as 40 million loops, 40 million bitshifts, and 40 million bitwise operators.

• For our serialization process, we've been extremely efficient at serialiizing things via "buzzsaw binary," which has a lot in common with "variable length quantity" encoding, but is even more aggressive in size reduction. This was something that Keith LaMothe originally designed, and he documented the vast savings at the time that we had in terms of file size compared to other formats (straight out binary, or character strings), and the processing power compared to these other formats if they are then compressed for transport (as they would have to be) is really notable.
  • However, we're now taking this further, by quite a bit, and have implemented a new version of the serializer and deserializer that supports smaller data types, tighter packing of certain data types, fewer loops in all cases (as noted above), and a few other features.
  • The savings that you get out of the new type is something that you have to look at a savegame in aggregate in order to tell how much space you're saving.
  • We've verified that we can now accurately store in the new format:
  • strings (more compact) and chars (could not directly do before)
    • We may later add in an Latin-letters-centric version that has heavily stripped-down data. Right now this takes 16 bits per character, and is 15% of the savegame filesize that we're experiencing in our test save.
  • bools (literally 1 bit, same as before)
  • bytes (now stored in native size, whereas before each byte took an average of 2 bytes to store -- but we also were not using this format).
  • 16 bit ints, signed and unsigned.
    • For numbers, we also have the ability for either 0 to take 1 bit, or -1 to be the sole negative number and take 1 bit as the "null case."
      • Previously, a -1 in these circumstances would take a whopping 10 bits, so any code that had -1 as the default instead of 0 was a bloat if it was used commonly.
    • In these cases, because of some "inferred bit" logic that we've also now added (shaving another bit off most numbers), we can't tell the difference between a 0 and 1 in an unsigned zero-not-default format. So there's some extra minor code in there that actually turns 0s into just two bits of storage, while -1s are 1 bit in those formats. Kind of a double bonus, in the main!
  • 32 bit ints, signed and unsigned once again, and with the alternative defaults and so on.
    • Not counting the new defaults keeping things so much smaller as described above (and that will make the biggest difference), we're saving something like 3 bits per integer compared to the old version.
    • By the time you get to 32bit or 64bit integers, Keith had squeezed pretty much all out of it that could be squeezed, a few edge cases in our style of data aside. It was mostly in smaller number formats (supporting them directly at and below 16 bits, and using smaller bitlength number formats for 32bit and down).
  • 64 bit ints, which are used as the foundation for floats and FInts to be passed around.

• So, after all that conversion to a new data type work, and drastic code changes, and some 30+ hours of work, we get an improvement in data sizes of...about 10%.
  • That's disheartening in the extreme (though we already got huge gains elsewhere on savegames), but one thing that is positive is that the new serialization does far less work to actually do the saving of the data, and that's a notable win for multiplayer.
  • We do also have other things we can do to get the data sizes even lower, but mostly those are strategies we could have used without having gone through all of this giant effort to get the data formats themselves 10% more efficient.
  • On the other other hand, had we not gone through all this work to get the 10% gains, we would have always wondered if it would have been more gains than that, and by going through the process of getting the 10% gains we gained a better understanding of some aspects of the codebase and fixed a number of other small bugs.
  • In the end it wasn't a wasted venture, but the payoff is disappointingly low in terms of performance/savegame gains compared to what we'd hoped it would be.

• A variety of places in the new ArcenSerializationBuffer were throwing exceptions when data that was out of range was passed in, but now those just show an error message and write to you log, while correcting the data as best it can so that it can save.
  • This isn't for overflows or underflows, but is for when a negative number was passed or less than -1 was passed in other cases.

Data Cleanup And Fixes We Found On The Way

• Got rid of the FPoint class, as it's not something we ever use, and we're going to be adjusting how we handle a number of our internal classes.
  • We're preparing to implement a number of new custom data types, some of which are targeted at making multiplayer performance possible without worrying about cascading desyncs or other issues of that sort.

• Got rid of the old solar system planet info, which has been completely unused since who knows when.
• The data it was storing should not have been serialized anyhow, but was.

• Fixing something that has been bothering me for years, the PlanetTypeData stuff has been moved out into its own file.
  • We've always kept these type definitions in their own files, separate from the actual object definition, and why this one was an exception is unknown.

• Basically all the enums in the game now specify their underlying data type directly, instead of just defaulting to int32.

• A ton of data types have become more specific, as in hundreds of fields, and this will make everything serialize more compactly as well as initialize new objects slightly faster.
  • This is going to require basically every mod to be recompiled at bare minimum, along with the new requirements on how data is deserialized.

• The old serialization gates have been removed, as we only had one in the first place, and have been using something that is xml-driven for the last few years.
  • It's apparently spring cleaning time in the code, since we're converting so many things over.

• The science and hacking points gathered on a planet are no longer stored by faction index, but instead are stored globally for all humans.
  • This is how we made the code work anyway, so now the data storage matches. It's a major QOL improvement for players in multiplayer to not have to each gather science on each other's planets, and something we discussed publicly a while ago with a lot of people being happy.

• There was a whole bunch of code that was using the mark level by RowIndex, instead of by Ordinal.
  • This may or may not have been a problem, it's hard to say, but if there were later modded-in mark levels it definitely would have been.
  • As it stands, it's possible that this fix might break something else, so if you see any odd mark levels, please let us know.
  • Honestly, as the code reads, it's kind of surprising that there weren't problems from this already.

• Got rid of our "ContainsValueType" list extensions, as that seemed more potentially problematic than anything else. It was also very rarely used.

• Got rid of the "RelatedEnumValue" field on GameCommands, as we only used that for one thing with AI scrapping units for budget refunds.
  • That scrap code for refunds was no longer in use, so has also been removed.

• Preliminary adjustments made to our network library to handle the better data formatting that we are able to send now.
  • There's a bunch more to test with that later, but we now get a very nice binary-to-binary style of transfer, super slim, with no conversions to char arrays in the middle, or compression of parts, etc.

• Renamed a whole lot of variables that were too vague, so that now we know if an "index" means that of a planet or faction or whatever.
  • Just general code sanitation.

• Fixed a bug with effective full membership strengths in fleets being able to be a negative number in some cases if there was not an explicit squad cap set.
  • These now correct themselves to 0, but it doesn't seem to have been breaking anything.

• Turns out there was all sorts of funky data in some older savegames, and we now have optional overrides on the deserializers that let us pass in a field name so that we can easily see what the field that is out of range was.
  • This is not something that you need to generally do, if you're a modder. It's just there for if you're having problems.

• Also added in a new SetIfNextIntErrorsThenCorrect method on deserializers, which lets us correct old data rather than complain about it, for cases where we find it.
  • This is something that we should only be using where it actually comes up that there is bad data in some saves, but it's a tool to make those saves load in properly and have correct data, versus failing to load or continuing to have bad data.

• In general there is more error-checking and self-checking now as we read in and out data, but in a way that doesn't slow the actual read and write process down.

• Fireteams were variously setting their invalid state to be -1 through -5. By convention we use just -1 normally, and we can read/write that as two bits instead of 10 bits, so we've converted all of it to be -1 now.
  • As part of that, places we were serializing the fireteam IDs now auto-correct old data that was less than -1 to be -1.

• Apparently it was sometimes possible to have negative hull points lost, aka more health than baseline. This is now self-correcting.

• Fixed a transient exception that could happen in PrivateExoNotifier if you were loading a lot of savegames back to back.

• More detailed serialization logging is now in place for when savegame logging is enabled, which lets us get a lot more info on things that are different between the versions if they are.

• Fix to loading and then resaving certain very old savegames from around the time extra hacking event data was added (August 2019 saves were where we saw the problem in our testing). This is unrelated to any of our recent changes, but just something we caught during regression testing.

Benchmark Results

• Test case improvements in savegame filesize:
  • SPIRER322.save: 8.3507 MB (3,858,950 data points) to 2.4009 MB (3,453,855 data points). 28.7% size
  • Last_Hope_16.save: 1.4435 MB (1,699,258 data points) to 1.0071 MB (1,570,970 data points). 69.7% size
  • decloakingTest.save: 1.4546 MB (1,752,227 data points) to 1.0690 MB (1,630,603 data points). 73.4% size
  • tech history.save: 594.1520 KB (696,137 data points) to 483.9174 KB (629,171 data points). 81.4% size
  • golem insane.save: 588.5422 KB (696,573 data points) to 483.7046 KB (628,923 data points). 82.1% size
  • fail to deser.save: 541.8727 KB (653,298 data points) to 441.2745 KB (588,517 data points). 81.4% size
  • performance_concern.save: 2.2050 MB (2,170,709 data points) to 1.3692 MB (2,127,092 data points). 62.1% size
  • LateGameZeusSpire.save: 3.5690 MB (3,073,064 data points) to 1.9468 MB (2,904,438 data points). 54.5% size
  • FreshSpireCity.save: 614.7300 KB (678,371 data points) to 500.2831 KB (642,760 data points). 81.3% size
  • assist menu.save: 412.8888 KB (545,698 data points) to 385.0891 KB (503,506 data points). 93.2% size
  • manychanges.save: 415.7845 KB (487,742 data points) to 356.0349 KB (430,439 data points). 85.6% size
  • extragalactic slow.save: 1.1419 MB‬ (787,800 data points) to 615.9395 KB (730,003 data points) 52.6% size
  • TurretTest Dec7 2019.save: 388.5265 KB (500,837 data points) to 365.4358 KB (445,445 data points) 94.0% size
  • Releasing Nanocost Aug 29 2019.save: 1.7157 MB (2,142,369 data points) to 1.1955 MB (1,992,190 data points) 69.6% size
  • Aug 5 2019.save: 1.5154 MB (1,615,230 data points) to 1.0923 MB (1,599,636 data points) 72.1% size
  • 000 bug Nov 8 2019.save: 598.0068 KB (789,163 data points) to 447.1401 KB (613,200 data points) 74.8% size
  • spire save scourge 3.save: 851.2179 KB (962,252 data points) to 643.9442 KB (881,762 data points) 75.6% size
  • Please note that all of these savegames (among others) were used as regression tests, and we verified that the data went in and came out identically. A number of these failed regression tests at various points, which we fixed before moving on. They all pass now.

• Here are some specific subsets of data sizes based on data type, in real-world average case savegames:
  • Bools take exactly 1 bit to store. 100% size
  • Bytes (normally 8 bits) take 4.74 bits to store on average, depending on the save. 59.2% size
  • 16-bit integers that are not allowed to be null take an average of 1.78 bits instead of 16. 11.1% size
    • Largely this is probably because a very small minority of them have data, or they have very low numbers in them that can be represented in a small size.
  • 16-bit integers that cannot be less than -1 take an average of 11.38 bits instead of 16. 71.1% size
    • Largely this is because a ton more of them have data, but not all of them. And we have a special case for the -1 value where it only costs two bits. 0 costs 1 bit.
    • These are used heavily to identify fireteams, planets, and factions by index.
  • 16-bit signed integers are barely used in the game, to the point we don't have enough data to give any sort of average.
  • 32-bit integers that are signed are used almost exclusively for coordinates, and take an average of 12.71 bits (instead of 32) 39.7% size
  • 32-bit integers that are not allowed to be null take an average of 5.27 bits instead of 32. 16.4% size
    • Largely this is probably because a very small minority of them have data, or they have very low numbers in them that can be represented in a small size.
  • 32-bit integers that cannot be less than -1 take an average of 2.71 bits instead of 32. 8.46% size
    • Largely this is because a ton more of them have data, but not all of them. And we have a special case for the -1 value where it only costs two bits. 0 costs 1 bit.
    • These are used heavily to identify ships/squads, shots, and wormholes by ID.
  • 64bit signed integers are not used enough for us to have valid averages for them. But the worst case is pretty much used for the single instance of one, and it winds up using 70.00 bits instead of 64 bits. 109.0% size
  • Strings are highly variable in size, but they require one Int16 non-negative for the length, and then the equivalent of one signed Int16 for each character. 100.0% size
    • We can cut this down in a future update, but rarely do strings use more than 14% of our total savegame size on a late-game savegame.
    • There IS at the moment a floor of about 100KB of data that is being used by strings in the saves, and cutting that in half (ish) with future improvements would be nice.
  • Floats are not used for any sim-related since they are not reliably deterministic between machines.
    • But we do store some of them for purposes of some purely visual aspects, and internally we convert them to an FInt first. Under the hood, THAT is an Int64, and at any rate they wind up taking 26.54 bits instead of 64bits in our case. 41.4% size
    • Basically none of them are zero, but they are small enough numbers that we save a lot of space. Yes, smaller numbers take fewer bits in this crazy format.
  • FInts, or fixed-int numbers, which have three significant digits of precision at the moment (we are thinking of changing the format up soon, but must investigate first) take an average of 5.48 bits instead of 64. These are Int64s internally. 8.56% size

Version 2.048 Bugfixes

(Released May 22nd, 2020)

• The text of forcefield frigates no longer mentions (erroneously) that ships under them deal half damage.
  • Additionally, they now benefit from the Heavy hull type, letting them get above mark 4 if you invest in enough upgrades.
  • Thanks to Nyteblade for reporting both.

• Fixed a typo in the hack for claiming the new Fleet Research Station ships that was not charging you their AIP properly.
  • Thanks to kasnavada and Spook for reporting.

• Added a new cheat code:
  • heal me
    • The issuing human player gets all of their deployed ships on the planet they are currently view completely healed in terms of shields, hull, and cloaking points.
    • This is incredibly useful for testing, if you're trying to accomplish something but your test ships are getting shot to bits.

• Added a new "LocalMultiplierWhenNoDeltaTime" on planets, which factors in two things:
  • Firstly, if game speed is running faster. Secondly, if there is coarse background processing going on.
  • For things that normally just happen "every sim step" at a flat rate, which are few and far between, they should now be multiplied by this.
  • The lack of this is why it was taking 16 realtime seconds to decloak ships when at 4x sim speed when normally it would take 4 seconds in 1x sim speed.
  • Thanks to wm46, DEMOCRACY_DEMOCRACY, and ArnaudB for reporting.

• Put in some code that makes decloaking far more efficient, and far more clear.
  • Essentially, as it tries to find tachyon targets it only looks at ships that can possibly be cloaked.
  • But secondarily, it then only issues orders to decloak ships that are currently cloaked.
  • This makes it so that you can tell something is busy decloaking because it haas a beam out to decloak an enemy, NOT because it has a beam out to all the things it has decloaked or could decloak on a planet (that got intense).
  • The end result is functionally the same but visually more clear and computationally vastly more efficient.

• Found an incredibly old style of profiling logging data that we have not used for years, but which was in there slowing things down on random occasions, writing to disk in things we did not even need.
  • We've implemented several new kinds of profiling since this, and this was supposed to be toggle-on in the first place, but somehow it was getting toggled on some of the time. It has now be stripped out to be sure it is never accidentally on and wasting your CPU and disk cycles.

• For reasons we can't at all trace, it was possible to sometimes get extra flagships of both the correct and incorrect types in a fallen spire city.
  • It now detects and removes these extras, and logs a "soft warning" invisibly to your local log. If you see these sorts of soft warnings, it won't hurt anything, but it does let you know it fixed the error.
  • This was reasonably rare, and we're taking care to make sure and add in missing flagships if there ever are any, too.
  • Thanks to Lord Of Nothing for the sole two repo cases and reports we had of this.

• Fixed a very annoying issue where if you had ships selected directly (as in not by fleet), then when they went through a wormhole they would become unselected. This was basically working as-coded, but why we had it that way is a big question mark. When we reworked the selection code, we had thought we'd removed all things of that sort. And with the ability to give cross-planet commands (move to a point on a planet you are not on, or attack a unit on a planet you are not on), the desire to keep selections consistent regardless of where you go is definitely longstanding.
  • Thanks to FDru and Benkyo for reporting.

Version 2.047 Perks By Another Mother

(Released May 21st, 2020)

• Fixed up the tips to no longer mention EXP, and to mention the direct science upgrades.

• Fixed a bug where if you were C-clicking a fleet's upgrade without having enough science to do the upgrade, it wouldn't show the popup with info about what the upgrade would be like.
  • Thanks to Mitsuha for reporting.

• Fix a divide by 0 in the scourge code
  • Thanks to Lord Of Nothing for reporting

• Tweak the rules for when nanocaust spawners are allowed to spawn
  • Thanks to Histidine for reporting

• Make sure AI Reserves are friendly to All AIs and hostile to all Players
  • Thanks to Lord Of Nothing for reporting some oddities

• Added a new cheat code:
  • my potluck OR use the short form mp
    • The issuing human player gets a random grab bag of ships spawned on the center of the planet they are viewing in a new fleet named "Potluck [Something]." Many of the ships will not be ships that a player could normally get.
    • This is incredibly useful for testing hacks, for instance.

Balance Changes

• Agile Transports now have half the health and shields of other transports, to match their description.
  • Thanks to Waladil for noticing the inconsistency.

• Raiders, Daggers, and so on now have basically double the hull health that they did before, except vicious raiders have a bit more than double the prior.
  • Thanks to DEMOCRACY? DEMOCRACY! for suggesting.

• All of the player and AI mobile forcefield generators, which already no longer shrink when taking damage, also no longer cause a firing penalty for ships firing from under them (aka they are now great-forcefields). This is consistent with AIWC, and makes them a lot more useful.
  • Thanks to DEMOCRACY? DEMOCRACY! and StarKelp for suggesting.

• The AI guard post with a shield, and shield generator in general, also no longer have firing penalties for out from under them (though they do shrink). This is also now consistent with AIWC, and is basically because they are more "finely tuned."
  • Thanks to DEMOCRACY? DEMOCRACY! for suggesting.

Fireteam Changes

• When a bunch of fireteams are deciding on a new target, the code correctly now figures out when a target is being significantly overkilled, and stop assigning fireteams to that target.
  • The practical upshot of this is that fireteam-based factions (Scourge, Hunter Fleet, etc) are now much better at dividing their attention and striking at multiple points simultaneously
  • This code has been in for some time, but there was a bug that was preventing it from working right
  • A faction can choose to keep the previous behaviour by setting NoDeathballing = false on all its fireteams.
  • For right now, the Nanocaust is going to keep its old deathball behaviour, pending discussion/polling at https://steamcommunity.com/app/573410/discussions/0/2246679252927977012/

• Reduce a fireteam's speed bonus when attacking a planet. They were going a bit too quick, and this nerf should help compensate for the intelligence buff

Fleet Changes

• Previously, you could only have 5 ship lines per fleet. This led to a lot of juggling of ship lines, and other unpleasant things. This limit has been increased to 40 ship lines.
  • One reason among many that this was a restriction was because of how the EXP system was set up previously. At this point there is no good reason to have things be so limited... with the possible exception of some units that give fleet-wide bonuses, like Raiders. It may be that we need to rethink Raiders having this advantage, as it may simply be too powerful.
  • At any rate, fleets can basically be whatever size organizational unit you need them to be, now, which lets them function a bit more like control groups once did.
  • Thanks to SplitterWind and others for inspiring this change.

• Any ships in fleet lines that you swap between fleets no longer are destroyed if the member is on the same planet as its new flagship.
  • Aka, if 20 of your ships from that line are on a different planet, they would still be destroyed, but if 40 from that same line were with their new flagship, they would be preserved.
  • Thanks to Badger for suggesting.

New Hackable: Fleet Research Stations

• Technical details: ARSes and similar can now be modded via xml to give different numbers of options of strikecraft and frigates with these options:
  • grants_stuff_to_be_added_to_player_fleets_strikecraft_options and grants_stuff_to_be_added_to_player_fleets_frigate_options.
  • The defaults remain 4 and 1, respectively.

• Technical details: Added a new "Fleet Research Station," or FRS, which is a lot more rare than the Advanced Research Station (ARS). These are also only on the other side of the galaxy, mainly.
  • These are the first of a new group of structures to use a "grants_stuff_to_be_added_to_player_fleets_required_tag" xml attribute, which lets us limit what sort of specific ships they can grant. Modders can make us of this to do more things, if they desire.
  • This is paired with a new fleet design "OtherHackables," which can keep these new ship types out of the main rotations but still able to be gotten by special buildings.
  • Since it has to match by tag, this is really great for mods to be able to do all sorts of specific ship-granting sub buildings. And we can do that in the game if we so wish, as well.

• Technical details: There is a new "aip_when_granted_by_hack", which is on the new FRS ship types. These are things that cost you more than mere hacking points.
  • Here again, mods can do some very intersting things!

• The idea behind the FRSes is that they grant you costly, but very powerful and rare ship types that you can't get any other way, which give you fleet-wide bonuses.
  • The actual ships from the FRS are barely the point -- yeah, they can be cool, but the main point is the sort of bonus that they give to the entire fleet as a whole.
  • These are basically a new way of handling the Perks system, letting you get Perks from FRSes instead of what we had thought about via EXP gains in the past.
  • AND these perks are transferable between fleets, since the perks are in the form of a specialized ship.
  • It's worth noting that your fleet gets the perk even if none of the ships of the line are actually built at the moment (or at all).

6 New FRS Ships To Capture (Fleet-Wide Perks)

• The ability that Raiders had that supercharges the speed of the rest of the fleet was indeed too powerful to just have as a random one-off ability on a variety of ships. That has been removed.

• Added three new supercharge abilities, which caused a LOT of code changes.
  • These let a ship buff the hulls, shields, or attack power of their entire fleet.
  • This is ONLY meant to be used for player factions.
  • The strength of fleets and the ships within them get recalculated properly to account for these changes, which is a bigger deal than you might think.
  • We can also build upon this framework in a lot of various ways in order to allow for ambient effects at a planet to affect player ship stats, as another example.
    • Trying to use this for ambient effects on any form of NPC ships would absolutely not work, and also this would assume that an entire fleet is on one planet, so there are some pitfalls even there.
  • Trying to keep these things computationally-brief is tough if we're not careful, but we've been careful.

• There is a new Turbo Raider ship, which you can only get via hacking an FRS, and which costs you 40 AIP to get, but which gives you 90 raider-like ships (wow powerful), and also makes the entire rest of the fleet (except flagship) super fast.

• There is a new Tutelar Pulsar Tank that you can get from an FRS for 40 AIP, giving you 50 pulsar-tank-like ships (holy smokes) that ALSO buff the health of all non-flagship members of the fleet by 1.5x.

• There is a new Inciting Parasite that you can get from an FRS for 60 AIP, giving you 90 parasite-like ships (holy smokes) that ALSO buff the attack power of all non-flagship members of the fleet by 1.5x.

• There is a new Encircling Spider that you can get from an FRS for 50 AIP, giving you 120 spider-like ships (wow) that ALSO buff the shields power of all non-flagship members of the fleet by 1.5x (including bubble shields, if there are any, which is why some of the cost here in AIP).

• There is a new Ireful MLRS Corvette that you can get from an FRS for 30 AIP, giving you 70 mlrs-like ships (wow) that ALSO buff the attack power of all non-flagship members of the fleet by 1.25x.

• There is a new Ultima Fusion Bomber that you can get from an FRS for 80 AIP, giving you 80 heavy-bomber-like ships (holy smokes) that ALSO buff the attack power AND hulls of all non-flagship members of the fleet by 1.5x.

Version 2.046 Hotfix

(Released May 20th, 2020)

• Fixed a bug where any savegame that you loaded that had drones in a fleet with a dead centerpiece would cause an exception in PerFrame_CalculateEffectiveFleetData, debugStage 784600.
  • Thanks to choam, DEMOCRACY? DEMOCRACY!, and Lord Of Nothing for reporting.

• Fixed a bug where C-clicking a fleet's direct science upgrade button was not properly showing the past science upgrades to a fleet in terms of the comparison of mark levels.
  • Thanks to Mitsuha for reporting.

Version 2.045 Scientific Emergence

(Released May 20th, 2020)

• Fixed a bug where if you started a new game and opened the factions menu and clicked back and forth between the AI and then the human faction, you'd get an exception.
  • Thanks to Mitsuha for reporting.

• The fallen spire (expansion 1) code that allows for neural nets to grant new types of frigates and destroyers has been made a lot more general-purpose so that people can mod in their own neutral nets with specific frigates and destroyers (or whatever else, frankly) and have that function properly. Previously it was not possible to add custom neutral nets, but now you can do it just via xml alone.
  • Thanks to Lord Of Nothing for the request.

• Starting from this version, the game now also tracks direct science upgrades to fleets, and not just the central tech upgrades. Any past upgrades you spent science on prior to this version will not be reflected, but future ones are.
  • Thanks to Mitsuha for suggesting.

• In the fleet management screen where you can directly upgrade either flagships (plus any drones they have) or the entire fleet, it now notes in its tooltip that you can C-click to see what all will be upgraded.
  • Once in the C-click screen, it shows you all the benefits in great detail, just like it does on the tech upgrade screen for non-direct techs. This way you have so much more information about what the benefit of an actual tech is. This is pretty important now that there's so much more that we can upgrade by science!
  • Thanks to Mitsuha for suggesting.

• Hopefully add support to some new voice lines related to the Dyson Sphere
  • Not very well tested.

• Drones of all sorts, be they AI-based or part of a combat factory, or off a hive golem, now all inherit the mark level of their spawner. If their spawner dies (in the case of AI stuff, mainly), then they will stick at the mark level they were.
  • Thanks to NRSirLimbo, Poppy, Mitsuha, and Badger for reporting.

• There's an experimental change where any drones that happen to have had techs that benefit them as a type applied, making them centrally higher than mark 1, now get those added marks on top of whatever their parent spawner's level is. This isn't used in the main game, but should help some mods.
  • Thanks to -NR-SirLimbo for suggesting.

Easier To Mod Direct Science Costs

• Fixed up several places where the game would let you get infinite upgrades after the 6th for a given fleet, or things like that. These would not do anything, but seemed very strange.

• The direct science upgrade costs (and number of upgrades possible) are now handled via data in the usual tech format, and assigned using a new tech_upgrade_that_is_used_for_my_direct_science field on ships.
  • This makes it so that people can mod in their own direct science costs for things, or differentiate further between specific ship types, as needed.
  • It also lets us adjust the data so that we can differentiate things for certain units ourselves, as well.
  • Please note that for anything where the flagship/centerpiece can change, like command stations, you really don't want to differentiate since this data is on the fleet itself. So the direct upgrade cost will always be the same for all command stations in general, is the main case of this right now. All other fleets tend not to have flagships that change (fallen spire aside).
  • Big thanks to -NR-SirLimbo for suggesting this refactor. It makes things way more flexible, without really changing anything you see on the interface.

• A form of self-checking has been added for anything that it thinks should be directly-upgradeable by science.
  • This winds up catching some non-player ships, and potentially some ships from mods or expansions doing unusual things. If you want to suppress the exception in your mod, just add tech_upgrade_that_is_used_for_my_direct_science="Direct_ActuallyUnused"

• Basic battlestations no longer benefit from the help of the central Citadel tech. They are only upgradeable by direct science, now.
  • They were far too similar to citadels in terms of their ability to tech up, otherwise, which kind of devalued citadels. It doesn't seem like they warrant their own full separate tech, though.
  • The direct science upgrades are now a bit cheaper than last build for battlestations, and a bit more expensive for citadels. Nothing giant in either case, but it skews them both a bit more, which is good.
  • Thanks to -NR-SirLimbo for inspiring this change.

• The regenerator golem now costs 1.5x as much as the other golems to upgrade via science, except for the top two marks which are not that steep.
  • Same with the botnet golem, except it is 2x as much, top two marks aside.
  • And same with the hive golem, which is about halfway between those two, on average.
  • Thanks to -NR-SirLimbo and StarKelp for suggesting these values.

Exotic Transports

• Cloaked transports can now be found throughout the galaxy, not just as part of the cloaked starter fleet. They are not super common, but they are not as rare as the regenrator golem.
  • They also now have their own tech line, separate from the regular transports, and their techs are 3x as expensive as normal. Except for the last mark, which is toned down because that would be crazy.

• Agile transports can now be found throughout the galaxy, not just as part of the one of the starter fleets. They are the same rarity as the cloaker transports.
  • They also now have their own tech line, and their techs are 2x as expensive as normal. Except for the last mark, which is toned down because that would be crazy.
  • Thanks to Endovior, Fluffiest, Strategic Sage, and others for suggesting that these two above be something you can find, as well as noting that the agile ones have no downsides. If these seem too hard or easy to find in practice, please let us know.

Beta 2.044 Scientific Equality

(Released May 19th, 2020)

• Experimental: Add a new Debug Setting, 'Enable Astro Train Power Increase'
  • When enabled, the Astro Trains will start spawning stronger trains once the player has killed enough trains. For testing/feedback purposes.

• In order to make entering commands more rapid, particularly if you're doing many at once -- or have a small typo you need to fix -- it now remembers the last 50 commands you've entered in the chat window.
  • To cycle through past commands you've entered, hit the up arrow on your keyboard. If you passed what you wanted, hit the down arrow.
  • This only works with commands, not with actual general text chat, which it excludes on purpose.
  • So if you want to spawn some ships (or do whatever else), you can now enter the command for that once, then just hit up arrow and enter repeatedly to keep repeating what you just said.
  • Thanks to Badger for inspiring this addition.

• There is now an alias for "player potluck" which is just "pp"
  • Thanks to Badger for suggesting.

• Fixed a one-line error in the code in the last beta version, which led to any clicks to choose your home planet making it give you an error and then seed you on an enemy planet.
  • Thanks to Endovior for reporting.

• Fixed a rare exception that could happen in shot visualizer if ships were being fired on at just the wrong millisecond as the ship was visually destroyed.
  • Thanks to NRSirLimbo for reporting.

Direct Science Upgrade Improvements

• Added a new is_upgradeable_by_direct_science xml tag, which lets certain centerpieces allow for upgrades like golems or arks do.
  • This is assigned to combat factories and transports.

• Also added is_spire_lone_wolf, which does the same thing, but uses its own cost schedule that is separate from those of arks, golems, and "other."
  • This is applied to the lone spire frigates.

• Added a new this_centerpiece_grants_its_direct_science_upgrades_to_rest_of_fleet, which now must be true in order for anything beyond the centerpiece in a fleet to get upgrades from the centerpiece's science upgrades.
  • This was basically true for all the command stations, previously, which were all that used direct science upgrades. And it needs to be used for fallen spire cities, which use science upgrade data under the hood (though not in the interface). But for things like golem fleets, they REALLY need to not apply to their ships.

• Added a new this_unit_gets_direct_science_upgrades_from_central_fleet, which now must ALSO be true on specific ship lines in order for the this_centerpiece_grants_its_direct_science_upgrades_to_rest_of_fleet to pass to themm.
  • This simply defaults to true. In cases where for some reason we want to turn it off, this exists. This is probably overkill and can be ignored, but is useful to have just in case.

• Various parts of the interface have been improved to further explain the mechanics as needed, and clarify if something upgrades just the golem/ark itself, or if it upgrades the rest of the fleet along with it.

• TLDR of most of the above:
  • Combat Factories and Transports can now be upgraded by spending direct science. But it is configured to not benefit their fleets.
  • Unlike the last two beta versions, upgrading a golem or ark now won't upgrade its fleet as well. That was crazy OP, and different from how things worked when EXP was a thing.
  • Lost Spire Frigates now work like the golems.
  • Fallen Spire cities should continue to work like they always have.
  • Arks, Golems, Lost Spire Frigates, Battlestations, Command Stations, and "anything else" all now can have their own upgrade costs that are separate from one another.
  • Thanks to Pireciter and Poppy for reporting.

Related Science Cost Adjustments

• After a lot of discussion from people looking at the beta and going "wow, golems are just not attractive anymore without EXP because of their absurd science costs," we thought about it some and decided that the simplest thing would be to adjust some science costs a bit more for now.
  • Command stations (which also benefit their entire fleet) were: 1000, 3000, 6000, 9000, 18000, and 36000.
    • This is just patently absurd past a point, and very few people even give one upgrade to them. Granted, we didn't want them to be used to over-upgrade them, but the later costs might as well be "infinity" for how costly they are.
    • We're shifting this to 500, 1500, 3000, 4000, 5000, and 7000.
  • Battlestations were the same as command stations, cost wise, and they do also involve upgrading all the ships in their fleet, but that's still too little benefit for it to be commonly worth it. They don't give economic upgrades, for instance.
    • We're shifting them to 300, 900, 2000, 3000, 4000, and 6000.
  • Golems and arks in the last few beta versions were 500/600, 900/1000, and then 2000, 3000, 4000, 6000.
    • They WERE upgrading their entire fleets, which was a Bad Idea (tm), and that doesn't happen anymore. But even for just the golems, this was too expensive.
    • For the golems, arks, and lone spire frigates, we're switching them all for now to be 100, 300, 500, 2000, 3000, 5000
    • Investing a little bit into these is SO cheap and tempting now, and the payoff is great. Investing a lot into one really costs a lot, but not so much that it is infeasible.
  • "Other" stuff, meaning transports and combat factories, upgrade only themselves (and their drones), and in the recent beta versions could not be upgraded at all.
    • Now they cost 50, 150, 300, 1000, 1500, 3000.
    • This may be TOO cheap, but probably not. The idea is that these are definitely tempting and cheap for just a couple of marks at least. Upgrading them all the way is kind on the expensive side and doesn't seem worth it.

• While we're at it, regular tech costs:
  • Light, medium, and heavy all previously cost 10k and then 10k for their two upgrades. Now it's 7k and then 10k.
  • Turrets previously cost 7.5k and then 7.5k for their two upgrades. Now it's 3.5k and then 7.5k.
    • Turrets should be a lot more viable in various interesting ways with the battlestation and citadel changes, and the changes here and to the command station costs. Being able to have fixed defenses that are able to keep the peace without getting your fleets involved as much is good.
  • Sentries and minefields are both a bit cheaper now.
  • Citadels are very slightly cheaper now.

Major New "Hooks" Framework For Modding

• Added a major new framework to the game to support modders in particular being able to subscribe to various events that might happen.
  • Each hook is defined in GameData\Configuration\ExternalCodeHook, or optionally in equivalent folders for expansions or mods themselves.
    • As we did with the expansion external data, however, we might wind up keeping the expansion ones with the base game for simplicity. That may keep mods in particular as compatible as possible.
  • Each hook has a name, which is what event handlers reference.
  • Each hook has a description which explains what the heck it is for, in terms that a modder or another developer will hopefully understand. These are not ever seen by players.
  • Each hook has a mainobject_is, which defines for the modder what type the "MainObject" parameter will be (so they can cast to that from type object).
  • Each hook has a secondaryobject_is, which defines for the modder what type the "SecondaryObject" parameter will be (so they can cast to that from type object).
  • Each hook has a additionalobjects_are, which defines for the modder what the heck is in the "AdditionObjects" array parameter (this can be an array of unlike type objects, or a list of multiple of the same type of objects, or more commonly just a null array).
  • Each hook has a context_is, which defines for the modder if there is anything unusual to know about the ArcenSimContextBase that is being passed in.
    • Generally speaking there is nothing to say, but sometimes it will always be null, for instance. It will also generally be something you want to cast to ArcenSimContext from ArcenSimContextBase, but we define it as ArcenSimContextBase so that we can call this even from the ArcenUniversal dll.

• Event handlers are defined in GameData\Configuration\ExternalCodeHookHandler, or the equivalent folder for expansions or mods.
  • These would typically be what people are actually modding in, rather than adding actual hooks (unless their mod is that huge).
  • These just define the dll and class names of the handler, and then the hook that they are attached to, and a name that needs to be unique but doesn't get used for anything.
  • It's worth noting that every time a hook handler object is defined, it instantiates a new copy of it that object. So if four different handler entries in xml all reference the same class, there would be four objects of the type of that class. You CAN store permanent data on those, but it's not recommended in most cases. You also can do something like define a static Instance variable to try to have a singleton pattern, but if more than one xml entry calls the same class, you'd have three classes not properly referenced, and then the static Instance pointing to the last-instantiated version. So a static List<> or similar would be better.

• Full documentation now provided for hooks in general: https://wiki.arcengames.com/index.php?title=AI_War_2:_Code_Hooks

• Four hooks have been added to start:
  • OnJournalFirstQueued
    • Executed the first time a specific final journal entry is queued. Will only ever execute once, even if the journal can occur multiple times.
  • OnJournalQueued
    • Executed whenever a journal entry is queued. Can occur multiple times per campaign, but only if the journal can occur multiple times.
  • OnJournalOpened
    • Executed whenever the journal is opened to be read by the player, just before the text is displayed.
  • OnJournalClosed
    • Executed whenever the journal entry is closed.
  • Al of these are related to journals, as you may notice, but overall the functionality of hooks is unrelated to journals. It can be used for many different things in the future, but this is just what it happens to be first used for.
  • Thanks to NRSirLimbo for suggesting these four events and thus inspiring the whole new hooks system.

• There is documentation for the journal hooks here: https://wiki.arcengames.com/index.php?title=AI_War_2:_Journal_Entries#Code_Hooks_For_Journals

Beta 2.043 The Missing Human King

(Released May 18th, 2020)

• Map planet size ranges changed for the following:
  • Concentric from 40-250 to 40-160.
  • X, Grid, Maze, Clusters, Wheel, Honeycomb, Encapsulated, Octopus, and Simple from 40-200 to 40-160.
  • Realistic from 40-100 to 40-160.
  • Linked Rings, Swirl, and Bubbles from 10-300 to 40-160.
  • Below about 40 planets, it's rare to be able to fully generate a map properly.

• Fixed several bugs that could happen if you had no planet or were not viewing a planet at the moment.
  • Those states should not have been happening in the first place, but if they do we want to not flood the error log with that sort of downstream error.

• Fixed a couple of funky bugs that could lead to the player or an AI not seeding properly on their homeworlds on excessively small maps.
  • Thanks to NRSirLimbo and Ymir for reporting.

Beta 2.042 Journals, Cheats, And EXP Retirement

(Released May 13th, 2020)

To play this on Steam, please go under betas and choose current_beta from the drop-down list. We would really appreciate some testers on this so that we can get back out of beta status as quickly as possible! There is not currently a way to get the beta versions on GOG, but we won't be in that status for more than a few days, knock on wood. We want to make sure we didn't break anything with all the substantial changes in here, but also with the EXP stuff we want to make sure that good alternatives exist to something that was a bit... hacky.... before.

• Added a new (optional to override) method DoOnFactionDismantle_ThisIsAGameBeingExited( Faction ) that provides a hook for cleaning up whatever faction data someone has amassed prior to the game exiting. This lets you clean up prior to a new game being started or loaded, and most factions don't need to do this. But some mods definitely could make use of this, we've been informed.
  • Thanks to StarKelp for requesting this feature.

• Fixed a typo that said "prompty" instead of "prompt" in one screen.
  • Thanks to ParadoxSong for reporting.

• A variety of citybuilding structures in The Fallen Spire (The Spire Rises Expansion) stated that they required local shard reactors and engineering centers, but none of them really did. We were going to add that sort of extra requirement a while back, but in the end the city level limitations and socket limitations (plus energy costs in general) were the right mix of limiting and flexible.
  • We've fixed it so that it no longer refers to some requirements that were never really enforced.

• Fix a typo in the CPA notification hovertext
  • Thanks to Joey on steam for reporting.

• Ships that have a tech assigned to them (usually from inheritance) to be able to be upgraded, but which are markless and thus non-upgradeable, should no longer appear in the "things that benefit from this" lists on the tech window. This would most notably at the moment include the Zenith Forcefield Generator.
  • Thanks to ParadoxSong for reporting.

• The list of ships that benefit from a tech was sometimes cut off at the bottom a bit if the list was too long, but now should never be.
  • Thanks to info on Steam for reporting.

• Hopefully make sure that the AI Reserves can no longer be allied to the player
  • Thanks to info on steam for reporting

• Minor code cleanup for Exo generation code. No functional change

• Slight performance enhancement for scourge code

• The game should no longer allow you to center your view on enemy units that are not visible to you because they are cloaked or otherwise invisible to you.
  • Thanks to rkfg and Stormking2010 for reporting. This has not been tested, but should work.

• Fixed a mis-transcription of one of the "funny" dialogue lines from the AI, and one not-funny one.
  • Thanks to Ovalcircle for reporting.

• Fixed a couple of obscure bugs where if the game had multiple factions of the same type, then looking for the "first entity on a planet belonging to a faction of that type" would only check the first faction of that type.
  • So far as we know, this has not actually caused any issues ever, but we found it in a code review while adding some new bits.

• Fixed a bug where if you clicked a sidbar tab while the chat window was open, it would instead open to the tab you last had open.

• Fixed a bug that could happen when an exception popup appeared with the chat log window open, where it would try to start a coroutine on an inactive game object.

• Fixed an error that would happen if you tried to click "Change Music" in the debug tools when music was disabled. Now it just gives you a popup saying that it's not playing any music right now.

No More Fleet EXP

• All of the stuff relating to ships granting EXP, and fleets gaining EXP, and the (planned for sometime in the future) fleet perks from EXP has all been removed.
  • There was just absolutely no way to do this that did not encourage grinding. Plain and simple. This one aspect of the game had to be scaled back over and over again to the point where it was barely present since back in the 0.800ish days of the game. But even where it was still present, its been something people find annoying at best.
  • Additionally, when it comes to too many fleets on a planet having a penalty for being there in terms of EXP gains, that was sending a rather punitive message that people didn't appreciate. Overall this whole thing wasn't working well, and the addition of exciting perks would have just made it worse, not better.

Direct Science Upgrades For Golems And Arks!

• There are now explicit flags for units that say is_ark and is_golem, for letting them behave certain ways.
  • Arks and Golems are now able to be directly upgraded by science like command stations are.
  • Golems are more expensive than Arks, but both are a lot cheaper than command stations. It needs to be tempting enough to use science on something that is single-fleet only but not holding your position.
  • Thanks to Democracy for suggesting that we move to science here.

• Any fleets in old savegames that were upgraded by EXP now have those levels transitioned over to be science upgrades instead.
  • For some things like combat factories, you will thus see "fleet levels from science (unexpected)" on them.

Journals, Chat, And Cheats

• void QueueLogMessageCommand( string Message, JournalEntryImportance Importance ) has been removed, since the old-style "journal entries are just retained chat messages" is going away.
  • Essentially chat messages can still be logged, of course, but they won't stay any longer than any other chat message does.
  • The new way to log a chat message is void QueueChatMessageOrCommand( string Message ), which also allows for scheduling cheat codes.
  • Additionally, there is now void QueueLogJournalEntryToSidebar( string UniqueID_ToUseIfGroupIDNotPresent, string OptionalGroupID ), which allows for writing the new-type journal entries.

• Rather than having a journal entry importance (which has gone away), voice groups now have the potential to be linked to journal entries if you want to have them appear that way.

• The old "InGameStatistics" object is no longer saved into savegames, as it was a waste of space and not really used like we thought it was going to be.

• Holy cow! There was a huge and wasteful "Local Message Log" buried in the savegames, which was really old code that was saving every last message that ever appeared on the right-hand side of your campaign.
  • This was not ever showing those messages again, and had a lot of bloated metadata on the subject, and was in general just an awful thing to have hanging around because it made savegames way larger than need be.
  • We were being very cautious with how we saved chat messages, not saving too many, but all along we were saving a SECOND copy of EVERYTHING, which partly unravels the mystery of why some savegames were so large despite having relatively few raw ships.
  • This also was something that would cause processing slowdown to at least a minor degree, as the GUI for the right-hand sidebar had to iterate over every last message ever shown to decide if it should still be showing it or if it had expired.
  • Now it has a non-serialized version of this that it keeps for just the messages that should be showing RIGHT NOW. When it's time to no longer display them, they are removed.
  • Update: in the end it was not as inefficient as we had hoped. In an 8mb savegame, this only was half of a mb. Where the heck is that other 7.5mb of data?? We will be investigating that this week.

Journals

• The journal sidebar is now fully functional, based off of the specifications you can read here: https://wiki.arcengames.com/index.php?title=AI_War_2:_Journal_Entries
  • There is even some text replacement capability in there, and a lot of other goodies.
  • Right now it is only possible to programmatically create and log journal entries, but we will soon be adding some triggers from when you encounter ships for the first time, destroy them for the first time, etc. Basically things that a modder or other designer could include via xml only, without having to touch code.

Chat Logging From Voice Lines

• A whole bunch of data relating to the text and "is funny" status of voice commands, as well as how frequently they play individually or as a group, has been moved over to our core xml and is no longer part of the embedded resource packs.
  • This lets us make tweaks and changes far more easily, as well as allowing for modding of them.

• As a part of this, a number of things the watch commander says which are important to not miss are now written out as well as having the voice clip play.

• Also, even if you have audible voice stuff turned off, the watch commander and AI items vocal items which would normally log text messages will continue to do so.
  • These usually accompany something important, and it's nice to see in the chat log so that you don't miss something that is happening. A lot of what the watch commander says is not written to the log on purpose, though.

Cheats / Debugging Commands

• Cheats have been added to the game in a format you can type into your text box (like the first game): https://wiki.arcengames.com/index.php?title=AI_War_2:_Cheats
  • The following old cheats from the debug menu have been removed:
    • Decrease AIP, Increase AIP.
  • Thanks to Badger for suggesting!

• The following debug menu buttons have been removed, as they are now cheat codes / commands you can type with more control:
  • give metal.
  • give hacking.
  • give science.
  • give wave and CPA points.
  • give energy
  • spawn wave
  • spawn player ships
  • kill enemies on planet
  • finish construction on planet
  • spawn relic
  • clear local achievements
  • give metal
  • increase or decrease AI difficulty
  • test local achievement
  • lose instantly

• The following debug menu buttons have simply been removed with nothing added back:
  • grant exp to fleet.
    • EXP for fleets are going away.

• In the debug menu, there's now a "Wiki: Cheats And Command List" button that links to the wiki for quick access if someone wants to discover cheat codes.
  • The buttons for exploring or watching all planets are still in there, as those were not cheats and are things that people use quite a lot as a lifestyle choice.

• The new "scrub foes" cheat now works properly with stacks of units, which the old button in the debug menu did not.

• The new cheat command "player potluck" is basically the old "spawn player ships" button from the debug menu, but improved to actually create a new fleet that has the ships in it. It gives the fleet the name "Potluck [Something]," and is a whole lot more useful and interesting this way.
  • It also randomly gives you AI golems in small numbers, too, with these.

• A new cheat/debugging command has been added that logs journal entries with some parameters of your choosing. The cheats page has example usages which all work.
  • https://wiki.arcengames.com/index.php?title=AI_War_2:_Cheats#Debugging

• There are a variety of "test case" journal entries included with the game now, which are used for testing individual calls to one journal entry infinitely, calls to a group that has three entries, and calls to a cascade of two individual entries. All of these use cases have been tested via the new journal debugging command, and do work.

Version 2.032 Savegame Hotfix

(Released April 28th, 2020)

• When a savegame fails to deserialize, it now mentions what version it was serialized as, and what it's trying to deserialize into. This takes some guesswork out on our part.

• Fixed a bug where savegames made in 2.026 could not be loaded in 2.031. A few weeks ago we had some internal confusion over version numbering that led to this. The fix will make all mainline 2.026 savegames function again, and anything saved in 2.025 or prior was already fine.
  • It's possible that a very few savegames from the beta branch that were saved in an "alternate timeline" sort of 2.026 will now fail to load, but there should be a handful of those at most, and mostly in the hands of our developers or most-core testers. And for most of them, their saves were already on 2.029, which again would not have been affected previously or now.
  • Thanks to Arc-3N-4B, Lystraeus, Aedris, Mada, and Wowzer for reporting.

Version 2.031 Faction Color Choice

(Released April 27th, 2020)

• The rule that drones never cost energy is now more strictly enforced.

• Previously ships that were not yet fully claimed could still produce and consume energy, both of which were wrong.
  • This led to things like them not being claimable without having double the energy they consume, and also the potentially that claiming something would cause a brownout.
  • They now work consistently with remains, not costing energy until they are fully built/rebuilt.
  • This is different from things that are self-constructing, where you need the energy at the time of construction. You can fully control when you place new structures, and it won't ever brown you out because you can't overspend your energy that way. But things that get claimed or auto-rebuilt are just a different category, really.
  • Do please note that if you don't have enough energy to finish claiming a unit, like the 50k you need for a hive golem, engineers and such still won't work on repairing it. But in the case of existing saves with an unclaimed golem that belonged to you, you now have 50k more energy compared to the prior build, so that works out just fine.
  • Thanks to NR-SirLimbo for reporting, and providing a save where the hive golem was consuming 50k energy too soon and thus causing these problems. That save now works as expected.

Viewing / Editing Factions Once The Game Has Started

• Added a new should_not_be_shown_even_as_separate_line_items_in_game_factions_menu attribute for faction entries, which lets us control which ones should not even show up in the in-game editing of factions (which shows more factions than the lobby, so you can adjust their colors at minimum -- for things like outguard and zombies, etc).
  • All of the various faction entries in xml have comments on why they are or are not visible in-game, for purposes of us understanding this. We may make some changes in the future to which ones should be shown.
  • These ones now specifically show up in an "other factions" section of the in-game view/edit factions window, where it just lets you adjust their colors.

• The view/edit factions window in the game now properly shows all the factions, including beacon factions, and gets at the root of their in-game data rather than just their more loose config data.

• Added show_only_if_faction_with_this_name_is_present, so that the tamed and enraged macrophages will only appear in the factions edit window if there is actually a general macrophage infestation.

• On the window where you see the colors for all the various other factions, it now shows their description in the tooltips for their colors.

• A new gamecommand has been added, along with some numerous changes to allow for post-lobby editing of certain pieces of certain factions.
  • It will complain if you try to make changes that are not allowed, such as basically any non-color-info about factions that are already in the game (aka not beacons).

• The game now allows you to change the colors of any faction in the game at any time after you've started. Any player can do this in multiplayer, and it will be multiplayer-safe synced for everyone.

• For factions that are still in beacon form, you can now change any of their settings in that same window after the game has started (but before you hack the beacon), and the faction will then have those properties when they join the game, if they ever do.
  • This is really useful for late-addition factions, where you might want to have a stronger nanocaust or something along those lines but you were not originally thinking of having them in your game.

• For all the fields that you are not allowed to edit once the game has already started, it just shows those in a textual format, but still with the various tooltips and so on.

• The discoverable factions are no longer directly shown in the escape (system) menu, but instead are shown only in the view/edit factions window.

• The formatting of the starting fleet display in the player tab in the lobby is now a lot nicer, using the actual display name rather than the underlying lookup key.
  • In the during-game display version of this, it now instead shows which player is controlling that faction, or if there are multiple or no players controlling it.
    • It's entirely possible to have a faction named "Adam" which is controlled by "Dave" and "Adam" if you wanted to do that. That could be because you are in multiplayer sharing control of a faction called "The Davadium Empire," perhaps, or because you've loaded someone else's savegame.
    • It's ALSO possible that in multiplayer, for whatever reason, Dave and Adam decided to switch which factions they were controlling, but not to rename what their factions were. We'll get to odd cases like that later, but for now it shows things properly so you can understand things. This will certainly get more complicated as multiplayer is built out, and we aim to make it a bit more flexible and simple than in the first game.

Beta 2.029 Journal And Faction Edit Framework

(Released April 21st, 2020)

• Fixed a rather inexplicable "bug" (really a design choice in code that made no sense in retrospect but has been there since the fleet redesign well over a year ago) where if a unit was put into a different fleet than initially expected, it would wipe out all levels from EXP or science.
  • The main practical effect of this was that anytime a command station died, then any levels that you got from science upgrades to its fleet were reset to zero, which was a huge negative of course. This is now fixed.
    • This did lead to some ships or turrets being a higher level (based on the old science upgrade levels) compared to newer ones. Basically items that are a certain level won't level-down, so even though the science was taken away, they wound up still being at the higher level. But new items built would be at the lower level. This isn't something that we can fix in existing savegames (same thing with the lost levels from science in existing saves in general. But for all future upgrades, including in existing savegames if the upgrades happen in this or future versions, they will now be retained.
  • Thanks to DEMOCRACY_DEMOCRACY for reporting.

• Fixed a bug from the last few weeks where if you changed your player color in the lobby, it would quickly reset to default. We couldn't duplicate this with any other factions at this time, so if you see any other factions getting set back to defaults please do let us know with steps for reproducing it.
  • Thanks to GreatYng for reporting.

• The mugger frigates have zombification, making them extra powerful. Normally you only get one of them where you would get two of many other frigate types. However, the parasitic starting fleet was giving you FOUR, which is crazy powerful. It now gives you two.
  • Thanks to ParadoxSong for reporting.

• Put in some rather substantial changes to things to now have a DamageSource Damage instead of a bool IsSelfDamage/WasFromSelfDamage.
  • This now lets us differentiate the following:
    • SomeSortOfEnemy: used for any sort of external hostile weapon, attrition, or other bad thing someone else does to this unit.
      • All the normal death effects should happen, and things like being able to be regenerated by regenerators to prevent death, etc. This was already working fine.
    • SelfDamageFromMyOwnWeapons: used for any sort of self-inflicted damage, such as attrition to myself over time, or using my own health to regenerate someone else or power an explosion (minefields).
      • Some death effects should happen, but very few. Basically if I die to remains, I should go ahead and still do that (aka minefields). But beyond that, regenerators can't save me, etc.
  • BeingScrapped: used for something that needs to die for some reason, and it really just needs to go. Often this is literally a player scrapping something, either directly or by swapping lines between fleets, or the AI scrapping one set of units to move them into a new stack, etc.
    • • The absolute minimum case. We're supposed to DIE, so we don't even go to remains if we normally would. This last case was being used for minefields self-damaging themselves in the last few weeks for the game, and is why we had to make this a tri-state thing instead of a boolean self/other damage source. If you scrap a minefield it should not go to remains, but if it damages itself to death it should go to remains.
  • Basically minefields were never going to remains recently unless enemies were decloaking them and shooting them in the final shots, or some edge case like that.
  • Thanks to Valkyr, Strategic Sage, and Mischief Maker for reporting.

• If the AI has too few planets they won't spawn Relic Trains anymore
  • Thanks to Scout1Treia, who clearly has too much time on their hands, for reporting

• A new "Journal" sidebar tab has been added, although it is not really functional yet. But the framework is there.
  • We'll be filling this in this week before we come back out of beta, but meanwhile we wanted other fixes to get to you.
  • The hotkey "J" was amazingly available, and is now used for switching to this tab.

• We also made both Intel and Journal show up higher than Outguard, with Outguard now being the bottommost tab.

• In the system menu (hit escape), there is now a "View/Edit Factions" button that currently does nothing, but soon will let you pre-edit beacon factions, as well as see other details about existing factions and change faction colors. It will also let you see "hidden" factions like the zombies, to adjust their colors if you so desire.

• If some UI element has an exception in its tooltip generation (or mouseover code in general), it now reports that instead of breaking the whole UI.

• If a text element errors out while trying to init, it now gives you useful info rather than breaking the entire load process of the game, too.

Beta 2.028 Eat Bad Lobby Saves

(Released April 20th, 2020)

• The fleet management sidebar popout now uses the display name for sidebar (which is shorter) instead of the full display name, to keep things from wrapping.
  • Thanks to -NR-SirLimbo for reporting.

• If a savegame would be loading in and fails for some reason, then it now shows what the savegame name is rather than writing ???.
  • In the event that for some reason your game version's name can't be found, it now shows you what game version it was trying to find. Sometimes this can be as simple as an artifact of someone doing custom development.
  • Thanks to Badger and mofowe4644 for inspiring this change.

• Previously if you tried to load a savegame that was broken for some reason, then it would give you a warning about that but then keep on trying to load stuff, cascading errors upon errors. Now it just stops and doesn't keep letting things get worse!
  • This also makes it so that loading a broken save doesn't mean you have to restart the entire program or else things are funky and broken until you do, too!
  • Thanks to ParadoxSong and Badger for reporting.

• Additionally, previously if you had an exception while trying to load the "last settings" savegame for the lobby when you do a new custom start, it would just be all kinds of broken, making custom starts not possible. Now it instead notices the failure and just does a fresh "reset to defaults" type of start for you.
  • In this path, to keep things smooth it doesn't even bother warning you now.
  • Thanks to ParadoxSong and Badger for reporting.

• Added three new settings to the Performance section of the personal settings:
  • Disable Tachyon Beam Visuals
    • If you have absolutely giant battles beyond the norm, and there are too many tachyon emitters in play, it can cause serious performance degradation. For the people who run into that, or are on lower-end hardware, this is a great way to get more performance at basically very litte drawback.
  • Disable Tractor Beam Visuals
    • If you have absolutely giant battles beyond the norm, and there are too many tractor beams grabbing ships, it can slow things down. This is a lot less likely to be a problem than tachyon beams, but can be helpful in some cases.
  • Disable Beam Weapon Visuals
    • If you have absolutely giant battles that we can't fathom, all with beam weapons, then the load of that is higher than that of other shots. This should basically never happen, but this lets you turn them off if it's ever a problem for you for some problem. This would be missing a lot of info during play, though.
  • Thanks to Emeritus Puffin for suggesting.

Bugs From The Beta

• Fixed a bug where any scouting was scouting all planets in the prior beta version. This was one of those sorts of things that happens when you make widespread code changes and sometimes mangle a bit of code that doesn't directly translate.
  • Thanks to ParadoxSong for reporting.

• For some reason, there was a version issue or something along those lines with certain lobby settings in the beta version for some (most?) players. The game now recovers from that either way, but we're not sure what was up with that. Possibly there was an intermediate version that was needed in the xml, and we added that, which will fix things either way. It wasn't hurting quick starts or savegames, just the custom start lobby, so the existing larger fixes above will prevent this beta issue from happening anyhow.

Beta 2.027 Planetary Mathematics

(Released April 16th, 2020)

To play this on Steam, please go under betas and choose current_beta from the drop-down list. We would really appreciate some testers on this so that we can get back out of beta status as quickly as possible! There is not currently a way to get the beta versions on GOG, but we won't be in that status for more than a few days, knock on wood.

• Code has been added in (but is untested) to remove destroyed planets from the galaxy map.

• Fixed an exception that could happen in RemoveIncomingShot when multiple threads were racing.
  • Thanks to ParadoxSong for reporting.

• Techs that have already been fully unlocked now continue to show up on the tech sidebar, since having those disappear was freaking confusing.
  • They now show up with a line item saying "COMPLETE" instead of some sort of science cost.

• When viewing the tooltip for a unit, it now shows how many upgrades have happened out of how many upgrades for each tech line that benefits the unit.
  • This way you can tell WHY it is the mark level it is, aside from things at the fleet itself that are leveling them up.

Dangerous Changes Useful For Future Features For Both Free Multiplayer And Paid DLC

• The game used to allow for a list of galaxies (more than one at once), but we only ever used one at a time. We've now restructured it to just be a CurrentGalaxy object rather than a List<Galaxy>. This is very slightly more efficient for whenever we need to loop over planets, etc, since there's no array lookup that needs to happen to Galaxies[0] and instead we just call a direct object reference. This is a minor improvement, but we do it so very much that it's good to have every little bit of improvement.

• There is a new DoForPlanets on the World_AIW2 and the Galaxy object that allows you to iterate over all the planets, with a boolean flag to let you skip destroyed planets if you want.
  • The existing Planets list on galaxies has been renamed to the long-and-discouraging name AllPlanetsIncludingDeadOnes_GenerallyDoNotReferenceDirectly, so hopefully people really only use that when there is a true need to do so.
  • There is also a version of DoForPlanets on World_AIW2 that lets you pass in a faction that must be controlling or influencing the planet. We can add other overloads as need be. In the case of this one we assume that the planet isn't destroyed, since it's hard to own something that doesn't exist.
  • It IS worth noting that for anything that is in mapgen, the galaxy object is being passed around and so using the methods on the world are NOT a good thing to be using.
    • With that in mind, AllPlanetsIncludingDeadOnes_GenerallyDoNotReferenceDirectly is still used quite a lot when we're talking about things that are in mapgen. Technically those could use the DoForPlanets on the galaxy they are passing around, but since there are never any dead planets during mapgen that does seem a bit pointless.

• Also added in some new methods for things like ensuring all the planets are in a galaxy are linked, to specifically AVOID ones that are destroyed, since those should probably not be linked and we don't care.
  • In a bunch of semi-related methods, looking at things like neighbor planets and such, there is now a bool saying if you want to include destroyed ones, etc. Hopefully the destroyed ones are unlinked already, but in case not this lets us ignore them here.

• Added in various other methods, like GetRandomPlanet() on the galaxy object to get a random planet that can be limited down to just the ones that are non-destroyed.

• Fixed a bug that we found purely by accident in code review where threatToLeaveBehind was being calculated extremely incorrectly -- using the indexer i instead of the indexer pairIndex. This could have led to anything from stuff not staying behind properly to a hard-lock of the AI thread (where the AI stops responding at all). It would have been incredibly rare, and basically only happened when the AI was fighting on a non-king planet and wanted to leave some threat behind but take some away because they were over-running the planet.

• It's worth noting that there are a TON of changes in here that may cause some other bugs that are unexpected, so we're going to go to the beta branch on this after all (which is funny because just yesterday Chris was talking about not doing that for a while).
  • There are also probably some further uses of AllPlanetsIncludingDeadOnes_GenerallyDoNotReferenceDirectly that we have right now that we can clean up, so we may go ahead and do that, too.

• A day passes.

• Added a funky new data structure called ArcenTwoDimensionalFlexibleLists<T>, which is basically a generic and flexible replacemenet for things like bool[][].
  • This lets us have things like two-dimensional arrays for pathfinding that get larger as the number of planets increase.
  • Previously if we added planets to the game late, without a save and load it was going to have some major issues with pathfinding in particular.
  • ProjectedMultiPathData has been updated to use this new data structure.

• After some thinking, AllPlanetsIncludingDeadOnes_GenerallyDoNotReferenceDirectly has been made private to Galaxy, and there are instead nicer wrapper methods for getting at the data in that.
  • This reduces the duplication of code that we otherwise have, and most importantly helps to make the intent of how each method is used more clear by their names.

• Wow this is a lot of changes, all throughout. We're totally doing a beta for a few days at least to see if things break or what happens. So far everything works perfectly, which always makes us suspicious.
  • But now we have a lot more flexibility for adding or removing planets during gameplay, which is a pretty neat new thing to be able to do in general, for a variety of reasons.
  • Adding extra planets for players who join late in multiplayer is an example of one thing that we want to be able to eventually do, so that if player 2 joins in 2 hours into a campaign they still get a homeworld of their own, but it's a new one. One thing at a time, but that's something for during the later beta stages of multiplayer implementation, probably.

Version 2.026 The Faction Duality Problem

(Released April 14th, 2020)

• The linux version of the game now has code in place to not even try to launch anything GOG-related, since GOG Galaxy does not exist on linux.
  • For most people it was failing silently (as it should), and basically causing no harm. But for at least one person, it was causing a crash on startup. This should work around that. When GOG galaxy support is added on linux at some point, then we'll revisit.
  • Thanks to shampoocat for reporting.

• Fixed a regression where if you were logged into GOG Galaxy and not Steam, it wasn't shown your username properly on the main menu.

• Fixed a couple of expansion spire hybrids from getting invulnerability from thoraxian hybrids when they should not.
  • Thanks to DHR Valkyr and Puffin Emeritus for reporting.

• Put in some fixes to planet tooltip generation in the event that the player faction is null for some reason.
  • Thanks to Valkyr for reporting.

• Made the exports for data about factions in a game a lot more detailed.

• Made it so that if you get a "no human king found" error, it also writes a report about what factions are there.

• Fixed a bug during deserialization of savegames/quickstarts that made it so that you had duplicate copies of the WorldSetup used for the lobby and the long-term storage, and that in turn led to chaos like deleting players or other factions or creating many duplicate factions when starting a new quickstart.
  • This bug has been in place since April 1st (go figure), but it wasn't really fully rearing its head until yesterday when some unrelated changes made it suddenly more apparent.
  • In fact we even had a second copy of this error from the same day in a second spot for loading quickstarts, so it was really going crazy.
  • This is less a case of "how did we break this yesterday" and more of a case of "how has this been working at all prior to yesterday for the two weeks prior?"
  • Thanks to AreU4Cereal, esassaman, GalateanGemmate, Strategic Sage, and Mischief Maker for reporting.

• If you have an old savegame with duplicate factions in them (multiple outguard or something), it will warn you about that when you load it, so that you know it's there, but it can't fix it. If you load that as a template for something else, that should fix it.
  • There is a vague possibility that you might be able to create a new game with duplicates again in the new version, simply by having old bad settings in your PlayerData/Save/_Internal/LastLobbySettings.save file.
    • Simply hitting "Reset To Defaults" in the lobby, or loading a quick start or savegame into the lobby would remove that problem. But even with a known-bad save in LastLobbySettings.save, we can't duplciate having a new custom game with the problem now, so maybe it's not possible. If you're able to make it happen we'd be grateful for having your LastLobbySettings.save if it's something that you can make happen over and over again.
  • None of this would affect any savegames except for those that were created in 2.025 (yesterday's build), or MAYBE via a LastLobbySettings.save from 2.025.
  • Thanks to Strategic Sage for reporting.

Version 2.025 The Twenty Million Problem

(Released April 13th, 2020)

• Fix a potential bug with macrophage notifications after the player is dead
  • Thanks to Ovalcircle for reporting

• Fixed a bug in the most recent two versions of the game where the various zombie factions and the outguard faction were not seeding UNLESS you happened to go into the lobby and hit Set to Defaults and then start the game. It's possible that quick starts were working properly, but not definite.
  • This won't affect existing savegames regarding outguard being summonable, but will affect any new ones created. For zombies, it will fix any problematic existing saves.
  • Thanks to Valkyr, Strategic Sage, Lord Of Nothing, DEMOCRACY_DEMOCRACY, Scout1Treia, and Poppy for reporting.

• Fixed a bug where certain savegames could have not all the expansions in use actually noted as in use. This was going to be a problem for multiplayer, but we will probably have some further changes to make to that in the future.
  • In the short term, this could also cause some problems with detecting if relic trains were supposed to be enabled if you used certain quick starts or older savegames.
  • Thanks to Lord Of Nothing for reporting.

• Somehow or other, Ovalcircle managed to get the game to remember his last settings as being those from the test chamber. That shouldn't have been possible, and we can't replicate it, but we've put in extra fixes to hopefully keep it from happening to anyone again.
  • Thanks to Ovalcircle for reporting.

• Put in a fix that makes it so that if there are more factions than planet factions, it won't give an index out of range exception.

• Put back in some code that will now add AI Reserves (and the zombies and outguard) to existing savegames that don't have them.
  • It won't actually seed any outguard beacons if those are missing, but it will make all the various sorts of zombies function properly if they were missing, which is a pretty big deal.
  • Previously we had taken this out, and only had it run when loading as a template of some fashion or from the lobby, because it caused a mismatch in factions and planet factions.

• Put in some code that automatically fixes any missing planet factions for any galaxy-wide factions that are missing in a savegame that is loaded or other things that are created.
  • This fixes some serious issues in the test chamber, for one, but also makes it so that the other fixes above don't break the world.

• Put in some more defensive code for a rare threading race condition exception that could happen in TryUpdatingTargetPoint of ShotVisualizer.
  • Thanks to Endovior for reporting.

• Improved the wording in the quick start menu to say "no scenario selected" and similar, instead of "no save selected," so it's more clear what to do if you are having an error.
  • Thanks to UFO for reporting the confusion.

• Fixed an issue in the quick start window and in the load savegame window where your old settings would be kept, but invisibly, when you exited the screen but went back in. You'd still have a scenario/save selected, but not visible.
  • Thanks to UFO for reporting.

• Put in a fix that ignores an exception in GenerateCaret() for text mesh pro when we disabled a canvas. This has only been popping up recently for whatever reason, but should now just ignore the problem and keep moving when the canvas is next showed.
  • Thanks to Endovior and Badger for reporting.

• Fixed an issue where, inexplicably, if we had a primary key for ships or shots that was more than 20 million it would start just giving them all the same primarykeyid which would then lead to all manner of problems.
  • This is not new, but it's only being encountered now, which is probably due to some particular factions in the specific savegame setup which cause a high ship PKID turnover. It still took 13 hours for this to come about.
  • The new cap is now properly a bit over 2 billion, which is 100x as high (and nearing the limit of 32bit integers). In the same sort of extreme scenario, with the same amount of churn, we can then assume it would take 1300 hours in one campaign to hit that new limit, so that's... certainly okay.
  • Huge thanks to choam and DarkWing for reporting and providing a save where we could reproduce this.

• Because of the prior bug, when a savegame is loaded that has an entity with id 20000001, it now reassigns it on load. This will definitely cause some problems in those saves, but definitely fewer problems than them all sharing a primary key ID.
  • The PKIDs are supposed to be unique, but in the case of one particular savegame we had 9895 ships with all the ID 20000001, which meant all sorts of problems like ships not being selectable, not able to decollide, not able to get movement orders in general, and not able to get orders from the player or other factions. Instead, they could kind of sit in place and autotarget and that was it.
  • Reassigning PKIDs is itself one of those "never do this" things, so probably any save affected by this (very few, if any beyond the single one we have on hand) may find some inexplicable bugs like fire teams missing entries or pointing to the wrong thing, etc. Things seem to load and run fine, but if there are boogeymen in a save of this sort then you know it's probably related to this.
  • Huge thanks to choam and DarkWing for reporting and providing a save where we could reproduce this.

Version 2.024 UI Focus Hotfix

(Released April 10th, 2020)

• Fixed a complicated bug where the game was not properly able to detect when an input field lost focus by virtue of its whole ui canvas (window) being disabled. This led to large parts of the ui being disabled at times after showing an input field in the prior version of the game.
  • Two logic inversions in semi-related code also made that even worse, although one of them made it "better" enough that the ui worked at all. Nonetheless, this was a fix that needed to happen quickly.
  • Thanks to Valkyr for reporting.

• Fixed a rare bug that could happen in fire teams updating and getting an index out of range exception at debug code 3100.

Version 2.023 Chat Hotfix

(Released April 10th, 2020)

• Allow non-scourge factions that use fireteams to stack when/where appropriate
  • Thanks to Swizzlewizzle for the bug report and save.

• Fixed several nullref bugs relating to regenerator golems.
  • Thanks to Badger for reporting.

• Added in new targeting logic for melee units that should allow them to better attack moving targets.
  • If issues crop up, or you prefer the old style, this can be enabled and disabled under the Units tab in Galaxy Settings.

UI / Chat Improvements

• The hotkey for opening the chat window is now Return/Enter rather than C. This actually lets you do C-clicking still and C-hovering, etc, which has been impossible since adding the chat.
  • Thanks to UFO and Strategic Sage for reporting.

• All of the various modifier keys for doing things like hiding tooltips no longer function while you are entering text into a textbox.

• Simply having the chat window open no longer causes all other input to be blocked. Now you actually have to have the cursor in the chat textbox for that to be the case.

• Hitting the escape key while you are in the chat window now properly closes that window, even if you are in text entry at the moment.

• Hitting the escape key to close whatever window or popup no longer comes with a risk of opening the in-game escape/system menu. That has been annoying for a long time.
  • Thanks to a variety of people for reporting over the last few years.

• When you first go into that chat window, or the modal textbox window, it now focuses the textbox there for immediate typing.
  • Additionally, after you send a chat message it immediately re-focuses the chat textbox.

• The chat log now actually uses gametime timestamps, rather than "how many gametime seconds ago was it" to show things. This is a lot easier to read, since every second every line isn't changing width because of all the lines changing numbers every second.

Version 2.021 Hotfix

(Released April 8th, 2020)

• Fix to a deserialization error with AI Reserves in the prior build which was affecting quick starts but not existing savegames. Sorry about that!

Version 2.019 Regenerator Sanity

(Released April 7th, 2020)

• When the AI Reserves decide to attack, they now give you a Notification saying "AI Reserves arriving in X seconds" instead of just appearing instantly. This gives you the chance to scrap units, retreat, snipe the command station, etc...
  • Thanks to a number of people for requesting including Iocaine, swizzlewizzle, Lord of Nothing and his Inimitable Puffinness

• Fix a null reference exception in in the Spire Debris notification; could only be hit if you were hovering the notification when the debris vanished
  • Thanks to Endovior for reporting

• Alarm Posts now also free ships in Guard Posts
  • Thanks to GreatYng for reporting

• Tidied up some names.
  • Extragalactic Thunderbird is now Extragalactic War Thunderchild.
  • Extragalactic Chimera is now Extragalactic War Chimera.
  • Extragalactic AI Hunter / Seeker is now Extragalactic War Hunter / Seeker.
  • Extragalactic AI Hunter / Annihilator is now Extragalactic War Hunter / Annihilator.
  • Extragalactic Jackalope is now Extragalactic War Jackalope.
  • Extragalactic Mothership is now Extragalactic War Mothership.
  • Extragalactic Planet Cracker is now Extragalactic War Planetcracker.
  • Thanks to Fluffiest for noting some of the inconsistencies.

• For at least a month or so now, ever since we switched in the linked lists for storing entity lists instead of List<> and ArcenMob<>, and possibly slightly before that, if you were to do a swap between ship lines on a planet with a regenerator golem on it, you'd get a crash to desktop and a ton of extra ships in general.
  • In general, for some time now it has been allowing for various things to happen when you self-kill units (by scrapping or otherwise) that should not have been happening. This includes things like zombification copies being made, entities spawning on death in terms of things like hydra heads out of hydras, dying to remains, and regenerator golems spawning replacement ships. In this particular instance it could cause an infinite loop, but it was problematic in general.
  • There are still various other things that DO need to happen on-death even for scrapped units, like various loss conditions, AIP gains, and so on. And in the past, prior to us making some changes here that have led to this crash, we weren't running those things, which is problematic.
  • The game now does a much better job of telling all the sub-factions and so on what is happening and letting it react properly in the event of death from self-damage. So that solves the most recent problems, but keeps the longer-term benefits we've had for a while now.
  • Thanks to Bob for reporting.

• Tweaked a couple of things in the "regenerator golem" code so that they should take the proper amount of damage now, rather than too much or too little. They also shouldn't be able to kill themselves anymore by taking too much damage.
  • Additionally, crippled regenerators stop helping at all.

• Regenerators can't regenerate one another anymore (the AI could have multiple of them regenerating one another).
  • Thanks to Puffin for reporting.

• The game now has 3 Pathing Modes to let ships figure out which planets to move through. Shortest (always use shortest path), Safest (always take the safest path) and Default (take the safest path unless it's significantly longer than the shortest path, in which case use the shortest path).
  • By default all player commands use the Default mode, but the player can hold a modifier key for the Safest or the Shortest path
  • Note to modders: FindPath() calls now take an extra argument.
  • Resolves pathing problems reported by a number of people, including UFO and Sigma7

Version 2.018 Hotfix

(Released April 6th, 2019)

• Fix to a bug that was causing some-but-not-all savegames to fail to load in the previous version of the game. Mainly saves that were games started in 2.016 that were being loaded in 2.017. This was to do with fixing the AI reserves missing. Turns out those should not be added in mid-save. Now that no longer happens, but in the other cases where AI reserves were missing, it adds them properly.
  • Thanks to Badger for reporting.

• Also added in a helpful method on the World_AIW2 singleton object which lets us just quickly write all the factions (invisible and otherwise) to the log.

Version 2.017 GOG Achievements

(Released April 7th, 2020)

• The Scourge now have some power scalings based on the Overall Power Level of enemy factions. It maxes out at a 10% increase in Neophyte spawning rates if the scourge's enemy factions have an overall power level of >= 5.
  • Thanks to swizzlewizzle among others for requesting

• Fixed a rare exception that can happen in the fallen spire code.

• Fixed an issue that could have AI Reserves not appearing in the prior version of the game sometimes.

Achievement Fixes And Improvements

• The Steam and GOG Galaxy client integrations have been moved out into their own sections of code, and a bunch more of GOG Galaxy integration has been included now.
  • As networking is added via Steam, its code will grow enough that it really needs to be on its own.

• The GOG Galaxy client now logs you in (if you are logged into that program on your system), if you are on Windows or OSX.
  • This is a prerequisite for both GOG Achievements, as well as for networking through GOG.

• GOG Achievements now log in the same manner that has worked for Steam -- it's about time we got this in!
  • It took us longer to get into their API than it really should have, but we need this sort of thing for multiplayer anyhow, as well as in general this being great for GOG players in the first place.

• Steam and GOG now properly only get sent achievements that it doesn't think you've already unlocked.

• Apparently we were saving the "all time statistics" but never actually loading them. Fixed to just not have them, now, as they contained very little aside from local achievements and that was not working.

• Added local achievements tracking that actually remembers what the heck you have previously unlocked.
  • This also prevents your achievements from popping up every time you restart the game program and enter a savegame with the conditions for the achievements met.
  • Thanks to GreatYng for reporting.

• The "resubmit offline achievements to Steam and GOG" code now actually does what you'd expect, since it now logs things offline.
  • We tightened this up a bit, anyway, to make it run a little more efficiently on the main menu.

Version 2.016 Thunderchild

(Released April 2nd, 2020)

• HRF has received many buffs to their budget, and how they use it
  • Maximum budget increased by 500%
  • Budget per second increased by 450%
  • Minimum budget required to attack increased by 3000%
    • Thanks to the a large portion of the community for their notes on HRF

• Make the "Enable HRF" hack cost fewer hacking points
  • From a discussion on discord

Extragalactic Surprises

• The AI will spawn more extragalactic ships at AI difficulty >= 7
  • There's been a good amount of feedback that the spire is too easy.

• Jackalope and Thunderchild extragalactic war units now make their debut. Expect to run into them when using the Fallen Spire, and potentially with other factions in the future. These are SCARY...

Lobby Fixes And Improvements

• We're now using the ability to load savegames into the lobby to make it so that every time you open the lobby, it now has the selected options that you last used -- factions and otherwise.
  • This fixes several bugs with things showing values that were not a match to the underlying data, as well as making it a lot more convenient in general.
  • Thanks to ptarth, Histidine, NRSirLimbo, Fluffiest, Ecthelon, and deo for reporting.

• Added a new Reset To Defaults button in the lobby, which lets you change the lobby back to the default settings and build up fresh again if you want to (now that it instead remembers your settings from last time).

• The "this is experimental" warning on loading savegames and quickstarts into the lobby is now removed.
  • We're using this pipeline for very central things in a lot of places now, and have beat the heck out of it testing permutations.
  • If you find any problems with it, please let us know and we'll fix that as soon as we can, rather than treating it like an optional side feature.

• When you load any old savegame or quickstart or whatever into the lobby, it now uses your profile's player color properly.

• If you try to load an old savegame into the lobby and it has a fleet that is no longer valid (or was from a mod or something that you don't have), it now loads properly and just randomizes the fleet in question.

• When loading a quick start in general, or a savegame/quickstart into the lobby, it was previously possible to have the underlying map seed be different from the one on the UI, which was confusing for sure.

• When you are in the lobby and the map regenerates, it now refreshes all the faction data rather than potentially keeping some stale data in there. This solves several problems with it showing out of data info for starting fleets in particular.

Other Fixes And Improvements

• Fix an exception with the Extragalactic War: Ignore Player-Allied Minor Factions option
  • Thanks to Greatmar2 for reporting

• Fix a bug where the player overall power level was being miscalculated

• Fix a bug where the Hacking Menu looked incorrect at game load time
  • Thanks to Arc-3N-4B for reporting

• Fixed a bug with setting the game mode on background threads where it would crash the game because it was trying to instantiate some planet visual objects on those background threads; that's a main-thread-only thing, and was quite the cascade of things to chase down.
  • In general it now protects it against itself, and waits to do it later on the main thread as needed.
  • We also tidied up some code into separate methods to make it easier to find things that are crashing in this area if it ever happens again, but it should not.
  • Also made it so that it can center on a planet from a background thread if it has to.

• The game had very old multiplayer sync code that has been partially running in place the entire time (even in single player), and which is very outdated compared to what we want to do in the near future. We've commented out the old in preparation for the new, which will be adaptive and scan some entities/planets at a time; the old method was about detecting wholesale problems as soon as possible and halting the simulation, which is not the approach we're taking with this game.

• If the game is missing a surrogate table for some reason, it now gives an appropriate visible message rather than just hanging game load.

• Fixed an exception that could happen when loading certain savegames or quick starts into the lobby for further editing. As part of this, was able to verify that all the custom settings are properly being loaded in properly, too, when loading highly-customized saves.

• A bunch of added robustness and fixes to some strange edge cases when loading worlds either from disk or across the network, or from savegames or directly.
  • Also some things to robustness in the display of galaxies when things go wrong during this load and you get in afterward.

• Fixed it so that if you are trying to generate a galaxy with a null map type (possibly by loading a quick start or savegame that is using a map type that no longer exists), then it will choose a random map type and use that instead, rather than erroring and dying.

• Really fixed a ton of different things that would give funky errors that were hard to interpret if the data was messed up enough.
  • These were all cases where we never have had data that messed up "in the wild," but as we get into multiplayer's alpha we may run into that from time to time and we want to make sure that we are able to have it recover more gracefully.

• The way that savegames and quick starts and the lobby are loaded under the hood has been streamlined and simplified a lot, and made much more clear.
  • This is good news for them functioning really well with the new ability to load savegames and quickstarts into the lobby, and also very relevant for loading multiplayer savegames and into multiplayer lobbies. This was quite a thing to untangle, but now it makes our lives much easier in the future as we get into multiplayer alpha soon.

• Fixed a harmless but annoying bug that could randomly pop up when trying to load some savegames and would give the message "ArcenDynamicTable<MercenaryGroupData>.GetRowByName() called with null/empty name"
  • Thanks to Malformata for reporting.

• Fixed a bug that could happen when trying to load a faction that was referencing a named color that is not installed on the local machine for whatever reason. Whether that's for an expansion or mod, or the color was removed for some reason. It now assigns a random color instead, rather than failing to generate the game.
  • This was interfering with loading the Helping Hands scenarios for some reason in the most recent version of the game if you didn't have the expansion installed.
  • Thanks to ArkTheSpark, minnow990, Vola, and Roadrunner for reporting.

• Fixed a variety of issues where the game could not load quick starts that were created with expansions enabled if you didn't have those same expansions and there were beacon-enable-able factions in that quick start.
  • The game now is able to deserialize expansion content no matter what by moving the "external data pattern" definitions into the main game, and also by making it so that missing ship entities get a dummy stub created for purposes of deserialization (and then being tossed away).
  • Thanks to Thanks to ArkTheSpark, minnow990, Vola, and Roadrunner for reporting.

Version 2.012 Populous

(Released March 26th, 2020)

• Fix a bug where allied nanocaust could be hostile for a second or two after loading a save game
  • Thanks to GreatYng for the bug report

• Fix a bug where Marauder Outpost strength wasn't appearing in the galaxy map
  • Thanks to Parch for reporting

• Fix a bug where Spire Debris intended for a recently killed AI faction could cause a null reference
  • Reported by Cadbury and Admiral

• Quiet some dev-only debugging code visible in the Threat window
  • Thanks to the crew of the USS Defiant for the report

• Fix a crash in the BuildingDronesInternally metal flow planning code
  • Thanks to LCEnzo for reporting

• Fix a bug where Dyson Spheres antagonized by hacking weren't actually doing anything
  • Thanks to Ovalcircle for a nice bug report with a save game

• AI Mines on a given planet now decloak once the AI command station on that planet is killed
  • Should be a nice little QoL improvement. Suggested by a number of people, including corfe83

• Fixed centerpieces occasionally reporting being under self construction, thus breaking them entirely.
  • Thanks to Lord of Nothing and Mayheim for reporting.

• Exogalactic Attacks are not eligible to come through the Exogalactic Wormhole from Wormhole Invasions until the first other ships have come through the wormhole. It's still very hard, but you can see it coming so there's less RNG.
  • This is primarily for the DLC (since there are a lot more Exos there), but applies to the base game as well
  • Suggestion from Lord of Nothing, with additional bug reports by Strategic Sage and The Puffinster

• Risk Analyzers now show their intensity-analogue in the Escape menu
  • Suggested by Ovalcircle

• Remove some deprecated settings
  • Thanks to Sombre and ovalcircle for reporting

• Fix a bug where Exos could sometimes wind up idle after killing their target
  • Thanks to Lord Of Nothing for reporting

• Fixed the issue with loading Quickstarts/saves into the lobby, but not playing at all
  • Thanks Fluffiest for reporting, Democracy for testing, and Puffin for helping figure out the issue

• Fixed a few more achievements that weren't triggering correctly
  • Thanks Lord Of Nothing for reporting

• Added menu to the debug menu that allows players to select the currently playing music track.
  • Thanks DEMOCRACY_DEMOCRACY for suggesting, as well as Steam user "Micah" and "NB_FlankStrike" for reporting similar requests.

• Fix a bug where usurpers would sometimes leave planets instead of capturing them
  • Thanks to Ovalcircle for reporting

• Fixed up the Montserrat-Thin font, which would get very distorted when shown in small pixel space on the game. Now it has enough data in there to render precisely even when very small.
  • This should help in a variety of parts of the interface, but most notably in the chat window when your screen resolution is very small.

Lobby Fixes / Improvements

• The "can edit presets" debug setting has been removed, and the editing of savegames and presets into the lobby is now supported for everyone.
  • There may still be problems with this that we need to iron out, but we're getting there and everyone should at least be aware of the option.

• When loading a savegame or quick start into the lobby, it no longer has any problems with "factions that must be discovered" showing up inappropriately.
  • Thanks to a lot of players for reporting.

• Loading a quick start into the lobby no longer requires you have a campaign name first.

• The factions that are discoverable in a game, but not yet discovered, are now shown at the bottom of the list of factions as (Discoverable) in the escape menu.

• Fixed a bug that was allowing you to add more than one Zenith Trader or Devourer Golem or other "must only be one" factions.
  • This had then led to a variety of other bugs after that, but we're less concerned about those since the core thing can no longer happen.
  • Thanks to DEMOCRACY_DEMOCRACY and StarKeep for reporting.

Forward-Compatible Settings and Profiles

• Made it so that settings files and input bindings files -- from this version onward -- are now forwards-compatible.
  • Aka, if you load a version 2.016 settings file in version 2.012, then it would let you do that. In the past, loading a version 2.012 file in 2.011 would cause it to show an error message and toss out all your settings.

• The player profile is now stored in an xml format rather than the char field format that it was using before, which allows for hand-editing if we ever need to and is more flexible in general.
  • The player profile also now is forward-compatible from version 2.012 onwards, like the settings and player inputs.
  • Thanks to a variety of players, but including Shnatsel, for suggesting.

Changes for The Spire Rises DLC

• Shifted the values for Fallen Spire exogalactic-strikes around, resulting in low AIP being stronger, and higher AIP not increasing as much.
  • Big thanks to Astilious for reporting and assisting in sorting out the problem.

• Fix a bug with the hacking sidebar that was causing numbers not to display properly
  • Thanks to Dominus for doing some useful debugging. Thanks to ParadoxSong and Ranakastrasz for the bug reports

• Fix a bug where mark 7 scourge would try to refuse to fight by constantly thinking they could upgrade
  • Thanks to Ecthelon for the bug report

• Fix a bug where player or minor faction allied scourge wouldn't realize that the AI Overlord was a legit target
  • Thanks to ArnaudB for the bug report

• Fix a bug where the Fallen Spire beacon was available even if the Fallen Spire faction was enabled in the game lobby
  • Thanks to Strategic Sage for the bug report

• Fix a bug where relic trains could go to the AI overlord early
  • Thanks to Lord of Nothing for reporting

• Fix a bug where the Imperial Spire Fleet was attacking an invulnerable Overlord instead of destroying the guard posts
  • Thanks to Oryutzen for reporting

• Exos that kill a Spire City no longer get stuck afterwards
  • Thanks to Lord Of Nothing for reporting

• The scourge was getting stuck if the only path to any target was extremely well defended
  • This was only happening in very late game scenarios with the fallen spire on for certain map types
  • Thanks to Lord of Nothing for reporting

Macrophage Phase 3 - Multi Phage Takeover

• The Macrophage are now a credible threat. You can have as many of them as you want, and they are fully capable of eating entire chunks of the galaxy on higher intensities. Be warned.

Base Game Changes

• You can now have multiple Macrophage per game, and they can have seperate allegiences set.
  • If Spores from different Macrophage factions meet; and there are enough unique Spores on the planet to spawn a Telium, one of the spores at random gets priority for deciding whom the new Telium belongs to.
    • Tamed still has absolute priority.
  • There is still only a single Tamed faction.
    • You are still unable to tame any Macrophage subfactions that are friendly to players.
  • There is still only a single Enraged faction.
    • They will still eat everything.

• Added in two new spawn options for Macrophage: Lone Telium and Clustered Telia.
  • Lone Telium spawns a single Telium for the Subfaction.
    • This Lone Telium is always berserk, never produces spores, its harvesters are much more defensive, and its harvesters only enrage when it dies.
  • Clustered Telia spawns Telia on half as many planets, but spawns 2 Telia per planet.
    • Added in a new quickstart, the Buggalactic War, which makes use of all the spawning styles.

• Spawn event costs now scale with the number of harvesters that a Telium owns.
  • Base event cost reduced to 125k Metal. (175k for hostile-to-player)
  • On Intensity 5, the cost is increased by 35% for each (non enraged) harvester owned.
    • This value scales with intensity, being 20% at Intensity 10, and 47% at Intensity 1.
    • Tamed is considered Intensity 10 for this purpose.

• Spore event chance now decreases by 2% for every Telia in the galaxy.
• Spore event chance is now clamped between 25% and 75%.
• Higher Mark Macrophage break and snack for longer.
• Enraged Macrophage snack on homeworlds for thrice as long as they do other planets.
• Updated the Macrophage Attacking message to no longer spam the log when you kill a large Telium.
• Tame Telium hack reduced to 25 HaP cost.
• Chance for Enraged Harvesters to attack the ai starts at 50%, and increases as your Tamed Telia start to outnumber Wild, up until they reach 90%.
• Gave a facelift to various descriptions such as hacking and entity appenders.

The Spire Rises DLC Changes

• When a Macrophage faction gets debris, it will upgrade one of its Telia into a powerful Spire Infested Telia.
  • A Spire Infested Telia is equipped with a multitude of short ranged Reaper beams, and is capable of (slow) movement. Be wary.
    • If you manage to kill this Telia, the Macrophage that its part of will be unable to build new Spire Infested Harvesters until they acquire new Debris.
    • You can optionally attempt to hack it to steal the debris, gaining double the science you would from a normal debris.
      • Perhaps its not the best idea to enrage the already powerful bugs that are more powerful than normal.

Balance Tweaks

• Halved durability of AI Tachyon Sentinels.

• Black Hole Machines affect units of engine gx 20 and lower, rather than 14. Crippled units can now leave regardless, preventing possible softlocks. Flagship engine gx reduced slightly to be affected.

• Black Hole Machine, Raid Engine and Magnifier now die with the Command Station.

• AI Turrets are now at least 75% of the range of player ones.

• Added numerous AI Ship Groups.
  • Some things of note:
  • There are 50ish Turret Groups, 37 Strikecraft/Frigate groups, and 14 Dire Guard Post groups all added.
  • Swarmer AI has more variety/difference in what it likes to bring.
  • Fortress Baron is the only AI type to have Mini Forts, as a unique thing for it.
  • Turtle AI has 6x the reinforcement group count as before.
    • These vary as to which AIs they are assigned. More specific AI types don't have all of them, while Full Ensemble has most.
    • Many Turret groups require the first DLC to ever show up, and some groups use both base game and DLC Turrets. If you don't have the DLC, those'll just look like normal mono-Turret groups.

• Some range nerfs to some Extragalactic units. Range and speed nerf to AI Dragon.

• Dire Plasma Guardian no longer has Overdrive effect, instead is just a larger Plasma Guardian.
  • Nasty interactions with Fallen Spire, particularly the Great Shield.
  • Thanks to Lord of Nothing for reporting.

• Upgraded AI Fortress power. Large upgrade to AI Super Fortress power.

• Increased Stingray damage and durability by 40%, metal cost by 60%.

• Tripled Ion Disruptor damage.

• Sentinel Gunboat damage increased 60%, metal cost doubled.

• Ranger damage increased 60%, metal cost increased 50%.

• Medic Gunboat now has vampirism, metal cost doubled.

• Agravic Pod and Aggressor damage increased 30%.

• Raider and Dagger damage increased 50%.

• Grenade Launcher Corvette and Molotov reload time 12s -> 9s.

• Guard Posts and Dire Guard Posts now use the same armour/albedo stats as their relevant Guardian, if they have one.

• Put a small bit of Dire Guard Post shield into hull, going from a 1:5 ratio to a 1:3.

Balance Tweaks (The Spire Rises DLC)

• Both Spy Cradles vision range increased by 1, both have very powerful planet wide tachyon systems.

• Buttress has same durability as normal structures rather than less, requires MK4 city rather than 5, uses 2 sockets rather than 4, has the same effects as a Black Hole Machine, but the gravity part is planet wide.
  • Made the lower tier Extra-Galactic War units susceptible to this.

• Small tweak to Relic chase force sizes as you progress through Fallen Spire.

Version 2.009 Plenty Of Tuning

(Released March 9th, 2020)

• Fix several bugs related to forcefields not handling norris effect stuff properly
  • reported by Fluffiest, Xenorius, ussdefiant and gigastar

• Centerpieces are now always visible even when crippled on both planets and the galaxy map, though still no longer grant vision of anything else when crippled
  • reported most recently by Chthon

• Add some defensive code to the Overall Power Level code paths
  • Thanks to ArnaudB for reporting

• When trying to find a random AI faction for things, try to pick a living faction
  • Thanks to GreatYng for reporting

• Fix a bug with tooltips for units in Fireteams
  • Thanks to a number of people including ParadoxSong for reporting.

• The icons in the bottom right corner, under the selected ships, no longer try to draw text. That was way too small and thus always hard to read and looked blurry. That happens in any text editor, too. Instead, it just relies on the icons, which are now larger and thus also themselves draw more clearly. There are tooltips for anyone forgetting what they do. The hotkeys for these functions are still shown via text, no problem, though.

• Fix a bug in the Save Presets code
  • Thanks to ParadoxSong for the bug report and Dominus for the fix

Dyson

• New Dyson Antagonizer rules
  • Dyson Antagonizers must spawn on Explored planets
  • Dyson Antagonizers are Always Visible, even if you have out of date information about the planet
    • Requested by a number of people

• Fix the Dyson Antagonizer not respecting the Dyson Intensity for spawning, and spawning much more often than intended.

Marauders

• Fix a bug where Marauders weren't spawning turrets for their outposts.
• Hopefully fix a bug where hostile-to-all marauders weren't killing AI command stations
• Fix a bug where marauder outposts were spawning much too quickly.
  • Net nerf to the marauders; they require much more time to scale up after capturing a planet. I've heard some complaints that the marauders are really getting out of control quickly, and that should no longer be possible.
  • Thanks to the crew of the USS Defiant for reminding me about this

Spire Rises changes

• Further clarify the description text on the AI Spire Research Lab
  • Thanks to Lord Of Nothing for reporting

• Spire Debris is no longer allowed to join fireteams
  • Thanks to Smaug for reporting

• Prevent multiple Spire Cities from being built on the same planet.
  • Thanks to Ovalcircle for reporting

• AI-allied scourge at intensity > 5 have a chance of building an additional guard post next to any spawner or armory they create. Should make trying to attack the scourge infrastructure a bit more challenging. This code is readily expandable to spawning other AI structures as well if desired
  • From a conversation with a particularly depraved Peltian sympathizer

• Make doubly sure that spire relics can't spawn on AI homeworlds
  • pointed out by XTRMNTR2K on steam

• Improve player-allied scourge target selection in the mid/late game
  • Thanks to Ecthelon for a useful save game for debugging

• If the Imperial Spire Fleet detects your home planet is under attack, they will prioritize defending it
  • Thanks to ArnaudB for the bug report

• Add additional debugging code to the Spire Sim code
  • Thanks to Admiral for the bug report

Version 2.008 Transport Loading Hotfix

(Released March 4th, 2020)

• Fixed a bug where ships would unload properly from transports, but never load into them again, in the prior version.
  • This was a casualty of the many changes we made in order to facilitate the performance improvements, but was quick to fix once people reported it. Apologies for that and anything else that crops up!
  • Thanks to Strategic Sage and Daniexpert for reporting.

• Imperial Spire vessels are now MK7.

• There is now extra debugging information shown visibly if the fallen spire actually has an issue.
  • Before it would just silently spam the log.

• When a unit is trying to transform and it is in an invalid state, it now gives a more clear error message.

• When centerpieces die, they are now blanked out properly; previously it held onto them in a dead format but mostly-removed, and this could cause some funky errors in the fallen spire in particular, but possibly other places eventually.
  • This was something that the squad wrappers we used to use automatically filtered for, so that's why it pops up now but not previously. It's a lot more efficient the new way, but we needed this extra check for this strange edge case.
  • Thanks to Puffin for reporting.

Version 2.007 Speed Boost

(Released March 4th, 2020)

• When hovering a tech line in the science sidebar, it now lists the ships lines to be upgraded first by mobile ships, then by turrets, instead of always in alphabetical order

• If remains cannot be rebuilt for some reason, it now shows the reason in the hovertext for those remains. This fixes a lot of ambiguity!
  • It also includes the countdown timers for things that can't be rebuilt for a certain amount of time with/without enemies.

• Improved the priorities of repairs and other metal expenditures so that it makes more sense. Command stations with next to no health will now be prioritized extra for repairs even if there's a big ship to complain.
  • In cases where there was a giant golem to expensively complain, it often would not let you even choose to override that and spend metal on repairing the command station. Now that works properly, and at least some engineers will break off to repair things even with a giant capturable there.
  • Thanks to Smaug for a savegame where the command station was just impossible to repair while the unclaimed golem was still present.

• The 'cancel hack' popup now reminds the player that you still pay the full hacking points value

Bugfixes

• Remove a debugging line related to Dyson Spheres that was left enabled by accident
  • Thanks to a number of people for reporting

• Make it much harder for structures to start to rebuild themselves mid battle.
  • Thanks to Ubifan for reporting

• Fix a very long-standing bug where the Threat Count in the resource bar was sometimes counting Warden or Praetorian units

• Fixed Fleet Centerpieces vanishing from the galaxy map if they were crippled, then moved to a planet that was only Explored.
  • Thanks to Ubifan, Daniexpert and Histidine for reporting.

• Fixed a bug where command stations still under construction could still help with claiming other ships.

• Fixed a really longstanding bug where ships that were overkilled that were in a stack were not rolling over their damage properly.
  • They actually were winding up over-healing the health of the next ship in the stack, not causing over-damage.

• It was somehow possible to have ships with zero health that did not die, seemingly only since the other performance improvements below but frankly the code was pretty unrelated to the area we can think of.
  • These now automatically re-check for dying within the next sim step, and do so properly. This is a nice safety net in general to have, we suppose.

• Fixed an issue where it would usually return a spurious "." at the start of type names it could not find.

• When PerFrame_CalculateEffectiveFleetData on fleets has an exception, it now gives more info on where that happened.
  • Fixed an exception that could happen due to centerpieces not fully deserializing into their fleet lines properly. That only seemed to matter for drone producers, but it was a thing.
    • This fix also fixed a common thing that could happen when exiting out of really large savegames to the main menu.

Performance Improvements

• Improved performance slightly on player planetary fleets without a centerpiece (aka command station), to basically not do all the checking and counts for them. Every planet has a fleet like that for every human player, so doing some checking like that on all of them was a mild expense that was not worth it.

• Put in some more debugging variables that let us turn off more parts of the simulation to find performance hotspots.

• Moved some logic for entity removal to be slightly more centralized, which makes it less common and also thus something that doesn't ever accidentally add the same entity to a big list more than once. There is some performance gain from this, but it's not huge in the current scheme of things.

• SquadCountsByType is now an int array rather than an ArcenSparseLookup, which makes it much much faster.

• MetalIncomePerPlanet_ForUI and EnergyProducedPerPlanet_ForUIOnly have become simple integers on the actual planet, and are handled in a much more efficient fashion since they are local-player-UI-only.

• Added a new ArcenLinkedList<> class, which is a new specialist collection that is thread-safe, multiplayer-safe, and extremely high performance to iterate over, add to, and remove from.
  • We're replacing all instances of ArcenMob<>, and the rollup uses of List<>, with this new class.
  • This sees a massive improvement in speed in a lot of savegames, and the removal of micro-stutters relating to entities being removed from lists in general. This also sees a big improvement in how quickly you can get back out to the main menu.
  • It's a lot of code changes, though, so it's always possible we made some sort of mistake in this and are causing a new bug somewhere. Fingers crossed that's not the case, but the places it would be a problem are usually central enough that it should be wildly obvious. So far we're not seeing that.
  • Thanks to zeusalmighty for the savegame that helped us pinpoint the fact that ship/shot removal was being the biggest performance drain at the moment. His savegame went from a choppy 45-55% performance to smooth 100% performance.

• Revised the Fleets class to no longer use Squad Wrappers, but instead to directly use squads.
  • This causes a slight performance boost in general, and will allow us to use performance-boosting collections that don't work on structs like SquadWrapper.
  • There's also the possibility for some errant bugs here and there, but we think we caught all of those, as they followed a specific pattern.

• The list of ships inside each fleet membership are now handled via the new linked lists rather than normal lists, extending the speed benefits to them.
  • Most of the "performance trouble" saves that we've collected over the last months now run at full speed, where some used to run at 5%. A couple still only run at about 80%, but those are by far the minority.
    • It is worth noting that just starting a new game at 300 planets will put you immediately at 70-80% sim speed right from the start. Apparently there is something we're doing that is still inescapably planet-bound in terms of calculation cost, even though the number of ships is 1/3 or less of that of much beefier later-game saves that now run at 100% speed. We may need to lower the max number of planets in any map to being more like 140 or so.

DLC changes

• AI extragalactic response no longer counts its own scourge allies when deciding whether to spawn extragalactic war ships
  • Thanks to Strategic Sage for the bug report

• Make the Scourge Beacon Hack more costly and harder
  • Thanks to Strategic Sage for the suggestion

• Awakened Scourge allies should show up in the escape menu now
  • Thanks to vinco for the bug report

• Fix a bug where the Imperial Spire wasn't pressing home its attacks
  • Thanks to Astillious

• Fixed the new Arks being unable to rebuild remains.
  • Thanks to Ryulong for reporting.

• The Objectives for killing the AI Spire Citadel/Research Lab only appear once you have Explored those planets

• There is now a Spire Beacon, and hacking it activates the Spire Quest.
  • This is only available for new games, it's not retroactive
  • Thanks to SCUD on steam for reminding us about this.

• The Scourge's Overall Power level can now go up to 1.5 if it has a Nemesis
  • Only relevant for player or minor faction allied scourge

Balance Tweaks (The Spire Rises DLC)

• Increased durability of Scourge Spawners and Armouries by 50%. Added Forcefields to both.

Balance Tweaks (Base Game)

• Raid Frigate, Bounty Hunter and Apparition mass 4 -> 5.

• Raid Frigate damage 5,000 -> 3,500.

• Dyson and Dark Spire units are immune to being captured.
  • This solves some nasty faction interactions, such as an AI Wendigo finding a nice tasty group of Dark Spire to eat, or the Nanocaust gaining enormous hordes of units.
  • Also fixes some possible player shenanigans using Hacks.
    • Thanks to ArnaudB for reporting a Dark Spire and Nanocaust incident, Histidine for reporting a Wendigo problem, and Fluffiest for suggesting (all separately).

Version 2.006 Hotfix

(Released February 29th, 2020)

• Viral Shredders now inherit pursuit mode
  • Thanks to Ecthelon for the bug report

• Quiet some text in the galaxy view hover-planet view. We don't want planets being suddenly Nomadic or anything...
  • Thanks to Fluffiest for reporting

• Improve the hovertext for AI Spire Research Labs to clarify that they violate other city placement rules
  • As a result of some discussions on discord

• Fix a bug where you couldn't build Spire structures if the Spire Flagship (but not the city) was crippled
  • Thanks to WolfWhiteFire for reporting

• Tweaks to the Spire Debris; Debris can no longer be captured by another faction while you are hacking it. The Spire Debris spawn message now shows the correct time that the debris will last for the first debris.
  • Thanks to Vinco for reporting

• Add some additional debugging code to PreFillFactionBasedData.
  • Thanks to Binary Blitz for the bug report

• Fix a bug where Outguard hacks were causing the hacking history menu to crash. Add some defensive code there too
  • Thanks to Strategic Sage's thot bug reporting ways

• Improve the 'ai stops doing anything once it's killed' code to make sure that Anti-MDC exos are no longer spawned after the Overlord is killed
  • Thanks to GreatYng for the bug report

• You now know which faction will capture Spire Debris if you don't get to it in time. This lets you make informed decisions about which debris you want to prioritize.
  • You also get 3 debris per City instead of 2.
  • Suggested by Starkelp

• Fixed Viral Shredder copies being swappable, allowing for infinite lines in a single Fleet.
  • Thanks to Lampshade for reporting.

• Put in defensive coding against some rare nullref exceptions that could happen during some combination of scourge and fallen spire, but which we weren't able to directly duplicate.
  • Thanks to ArnaudB for reporting.

• Removed the maximum Mark restriction set on Battlestations, Citadels, and Home Forcefields.
  • They now go to Mark 7 like normal.

Version 2.005 Hotfix

(Released February 28th, 2020)

• Fixed an invalid cast exception that could happen in GetScourgeGlobalDataExt for unexpected reasons if the scourge had wrong metadata for some reason.

• Definitely fixed the minor capturables exception in mapgen, so that even if it ever happens in the future (even mods could validly cause this), it will just silently log a note in case developers need it, but keep everything on the interface running smoothly. The actual error in this case was related to an expansion, but a mod could easily cause the same thing, and in neither case is it actually a bug or something to notify the user about.
  • Thanks to Jodan008 for letting us know this was still happening.

• Fixed an issue in the most recent hotfix related to dyson antagonizers not being found.
  • Thanks to ArnaudB and Yog-Sothoth for reporting.

Version 2.004 Hotfix

(Released February 28th, 2020)

• Fixed units with ShouldNeverBeCapturedByAnotherFaction set to true, such as Tesla Torpedo Frigates, from being loaded into Transports.
  • Thanks to Ubifan for reporting.

• Clarify the Scourge Neinzul Fortress text to mention that it produces hostile to all zombies
  • Thanks to vinco for reporting

• Fixed a mapgen error that could happen for people without the new expansion installed. Apologies for that!

Version 2.002 Turret Icons

(Released February 27th, 2020)

• All 30 of the new turrets now have updated flair denoting what they are. Definitely important for seeing from a distance and in the sidebar what they are!

Version 2.001 Expansion 1: The Spire Rises!

(Released February 27th, 2020)

• The new expansion is out now!
  • More info is here on Steam: https://store.steampowered.com/app/1196420/AI_War_2_The_Spire_Rises/
  • Or on GOG: https://www.gog.com/game/ai_war_2_the_spire_rises
  • Or on Humble: https://www.humblebundle.com/store/ai-war-2-the-spire-rises

Prior Release Notes

AI War 2: The Grand New AI