HotM:Personal Essay About Testing

Intro

Hi, I'm Chris McElligott Park. I've been making mods and games for 26 years now, largely under a pseudonym for the first 11 years. And then I've been doing this in a public and professional capacity for the last 15 years.

Here is some perspective I have come to have over the last decades, particularly since I've been doing this professionally. I get a lot of questions from people about why I don't do things one way versus another when it comes to testers, or why I try to keep at arm's length from certain kinds of narrative freeform feedback. So here's the answer to that.

Fans Come And Go - Sadly

In general, every time I've released any new game after the first one, I've had loads of people who profess to be fans of my prior work, or who I know are fans of my prior work, and they really feel beholden to let me know how much the latest game has let them down, or just isn't for them, or whatever else. This is just an incredibly taxing emotionally-fraught situation, it's my least favorite thing about the job, and it's not something I can control.

So in a lot of respects, I have to just not care. Someone doesn't like the game? I can't care. I do, I shouldn't, but I really do, but I can't if I'm to function.

In general there are also huge numbers of new people with every game, and so each game winds up with its own audience.

People Who Sign Up To Test Are A Different Breed

The people who are signed up for testing any given game are primed for extreme reactions in general, which I have to really take with a grain of salt.

Overall, most of you are very primed to love this game more than the average person, because you've built it up, and you're in a special relationship with the game by having it early, and so on.

But by the same token, a lot of you are very primed for extreme disappointment, as you're getting the earliest version of what is close-to-public-ready (but NOT public-ready), and a lot of you have been projecting all sorts of hopes and magic onto the game that probably no game could deliver.

For someone who is just in the market to buy a new game, browsing storefronts, it's a totally different prospect. For one thing, they are playing a game that is fully ready for the public, or the demo for said game, and/or they read reviews for it that prime their expectations up or down in a different fashion, etc. They just experience the game in a completely different way.

Testers are in an odd state where they can actually dislike a game, but love the testing of the game, and I can't tell the difference until they tell me. Strange stuff like that is actually kind of... normal, when you do a lot of test groups for a long time.

So Who's Judgement To Use?

Hooded Horse tells me this game has legs and should do well, with refinement from testers. This is also my own assessment, after about two years of being very frustrated with the current state of the game but feeling on the cusp of being happy with it.

So for me, that has to be good enough. Testers are here to help me find issues with the game that exists, not a hypothetical game-that-it-could-become, and make it a better version of what it is through any things I missed. The fundamental nature of the game is set, which keeps this simpler for everyone.

Avoiding The Emotional Morass

Just think about all the people who are so so mad about AI War 2, and love to tell me off about it, even to this day. If I let myself get distracted by stuff like that, then I just wouldn't be able to function.

And it definitely does have an effect on me, slowing down my whole day, so for the most part I'm just trying not to read stuff like that, as it's not actionable and causes me more slowdowns than anything useful I get out of it.

That's part of what the folks doing triage are going to be for, is to pick the useful bits out of feedback like that, and give it to me in a non-emotional context I can just plow through.

Final Thoughts

I dunno. It weighs on me. But ultimately, I need testers, and I need the wider net of feedback. But I can't allow myself to get sucked into the emotional morass of it, and I can't start designing by committee or trying to please everyone (thereby pleasing no-one), and at the same time I have to be duly respectful to the people donating their time and mental energy to giving considered feedback. It's exhausting.

It's part of why I put off the testing until it was to the point where it was so abundantly clear what the game is that people couldn't misunderstand that part. So if they don't like the game, that's fine, there's not much more to say. But if they do like the game, but there are parts that bug them, now we have something interesting.

I am deeply appreciative of everyone who signs up to be a tester, but I also recognize that I can't please all of them. We mutually have to be okay with that.

Cheers, Chris

P.S. About The Prior Testing Group

When it came to the tester group that played this game a year ago (April-May 2023), it was an interesting experience. The game was really not that fun for most of them, I think, but there was absolutely no feedback of that sort. I had to read between the lines, and also the publisher was very blunt with me that things were not in good shape (they were 100% right in every note they gave).

So what was going on? Some of the testers were having fun, and will state to this day that they got 5-10 hours out of what was supposed to be 15 minutes of content. But they were exploring and tinkering, and having fun with THAT. That's different from actually enjoying the game as a game you play. Others were just enjoying the atmosphere, or the intellectual elements of it. Others were just being polite and biting their tongue (in a few cases, I could tell), or did not even test the game at all.

The Game Has Been 100% Overhauled Since Then

Since then, the game has been complete revamped -- literally no gameplay and almost no UI are in common between the two versions. It's now at a point where the publisher and I both agree it rocks. I think it's the best thing Arcen's ever done, to be frank. But that doesn't mean it's going to land with everyone.

What Didn't Work, Last Time

The main negative things that happened during the prior round of testing were the sheer volume of walls of text of thoughts from testers on discord. To be fair I did invite that, and at the time it was a team of us working on the game rather than me being solo. But even still, it was so many hours a day to parse through that feedback in that format. To be clear, I asked for things in that fashion, and it's nothing bad about the testers in that group. But it's one of the things that has adjusted how I'm requesting feedback now.