Difference between revisions of "HotM:Instructions For Testers"

From Arcen Wiki
Jump to navigation Jump to search
 
(28 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
== What Are You Expected To Do? ==
 
== What Are You Expected To Do? ==
  
* Play the game for at least 2-4 hours within the first 3 days of getting access to it.
+
* Play the game for at least 2-4 hours within the first 3 days of getting access.
 +
** Provide a naive player experience, in general, the more blind you can go into the game, the better.
 
** But life happens, you know?  If something comes up, this isn't a giant crisis and you don't need to feel bad at all.
 
** But life happens, you know?  If something comes up, this isn't a giant crisis and you don't need to feel bad at all.
* There are specific questions below; please leave those data points in the specific linked butracker tickets.
+
* There are specific questions below; please leave those data points in the specific linked bugtracker tickets.
* If you have some constructive feedback, feel free to leave it in the bugtracker.
+
* If you have some constructive feedback, please add it in the bugtracker as a suggestion ticket.
* If you have some half-formed thoughts that are not actionable items, and you want to discuss it with other testers and what they think, then there's a private spot on discord for doing that.
+
* If you have some half-formed thoughts, that are not actionable items, and you want to discuss it with other testers, use the private discord channels.
 
** This is a great way for you and others to talk about your thoughts, and then for someone to eventually bubble up an actual actionable item into the bugtracker.
 
** This is a great way for you and others to talk about your thoughts, and then for someone to eventually bubble up an actual actionable item into the bugtracker.
 
* Beyond that, play more or don't play more as you wish.  This is not a job.
 
* Beyond that, play more or don't play more as you wish.  This is not a job.
 +
 +
== Information For You To Keep Private (Important!) ==
 +
 +
* If the information not on the Steam Store page, then please don't discuss it outside of the Arcen discord prior to June 4.
 +
* If I have not discussed the information in the public part of the Discord, then please don't discuss it in the public part of the Discord.
 +
* You can discuss anything you like on the bugtracker or in the private tester section of the Discord.
 +
* After June 4, you can discuss anything you like up through chapter one.  Please do NOT share information about chapter two until a date yet to be decided.
 +
 +
=== Why That Date? ===
 +
 +
It's a great question, but I can't share the details on that yet.  You'll see around the time of that date!
  
 
== What To Expect From Me ==
 
== What To Expect From Me ==
  
I am a solo dev at this point, which means my time is incredibly limited.
+
I am a solo dev at this point, which means my time is incredibly limited.  I have written a [[HotM:Personal Essay About Testing|personal essay about testing]] if you want more context, but don't feel like you need to read it.
  
=== I Can't Engage In Long Conversations Or Read Long Writeups ===
+
==== I Can't Engage In Long Conversations Or Read Long Write-ups ====
  
 
* Any time I spend reading feedback or conversing about things is time that is directly taken out of development.
 
* Any time I spend reading feedback or conversing about things is time that is directly taken out of development.
** In past games, just reading and responding to format has taken me 4-6 hours per day.
+
** In past games, just reading and responding to feedback has taken me 4-6 hours per day.
 
* I have a lot of stuff to finish in the coming months, so I can't afford to do that, or even close.
 
* I have a lot of stuff to finish in the coming months, so I can't afford to do that, or even close.
 
* Thank you for understanding.
 
* Thank you for understanding.
  
=== Others Will Help Triage For Me ===
+
==== Others Will Help Triage For Me ====
  
* Hooded Horse is providing some professional QA assistance to help read the longer unformatted feedback and get that into the bugtracker for me.
+
* Hooded Horse is providing some professional QA assistance to help read the longer feedback and get that into the bugtracker.
 
* My dad is also going to be helping with that.
 
* My dad is also going to be helping with that.
 
* Based on this, I'll spend more time working on the key things people bring up which are surfaced to me, and then content development.  This is where you want my time to go.
 
* Based on this, I'll spend more time working on the key things people bring up which are surfaced to me, and then content development.  This is where you want my time to go.
  
== Your First Play Session ==
+
== For Your First Play Session ==
  
* Please don't read anything from the wiki other than this page.
+
* I'm looking for naive player experiences.  
* Please don't read any of the other tester's thoughts or comments.
 
 
* In general, the more blind you can go in, the better.
 
* In general, the more blind you can go in, the better.
** I get more of '''your''' thoughts, then, rather than those of the crowd.
+
* Avoid learning anything about the game, from other testers, the wiki, Discord, BugTracker, Steam, etc.
 +
* The more I get of '''your''' thoughts, rather than those of the crowd, the better.
  
 
== Sessions Beyond Your First ==
 
== Sessions Beyond Your First ==
  
Please do as you like.  This isn't your job.   
+
* Please do as you like.  This isn't your job.   
 +
* If you don't enjoy the game and want to bow out of testing, that's absolutely fine. 
 +
* If you just can't get enough of it, also fine.
 +
 
 +
== What Is The Current State Of The Test Build? ==
 +
 
 +
* In general, it is highly polished the game is what it is meant to be.
 +
** I am not looking for feedback on alternative ways for the game to work in a broad sense; it's very much there already.
 +
* However, it has a vastly lower amount of content than it will eventually (half a dozen hours of content at most, compared to hundreds of hours of content).
 +
** There are some ragged edges where more content is going to come, or where more options will be provided.
 +
** At some point during your first play session, in chapter one and after at least 2 hours of play, but possibly closer to 5 depending on your speed, you will run into some areas where you feel like things just kind of... stop.  At that point, feel free to check out this link and see if you're at the end of the current content: [[HotM:Current State For Testing|Current State For Testing]].
 +
*** Please don't read that link prior to your first blind testing session, but after that please feel free to refer back repeatedly as updates go out.
 +
 
 +
=== Link For The Current State ===
 +
 
 +
Again, reminder, please only refer to this after your first blind playthrough: [[HotM:Current State For Testing|Current State For Testing]]
 +
 
 +
== Your Mission AKA What I'm Looking For ==
  
If you don't enjoy the game and want to bow out of testing, that's absolutely fine.  If you just can't get enough of it, also fine.
+
==== Use The Bugtracker ====
  
== Please Take Notes On These Specific Things ==
+
[[File:MantisSignIn.png|right|400px|Mantis Sign In]]
  
=== Use The Bugtracker ===
+
The majority of your feedback will go in the [https://bugtracker.arcengames.com/view_all_bug_page.php Arcen bugtracker].
  
The majority of your feeback will go in the bugtracker, here: https://bugtracker.arcengames.com/view_all_bug_page.php
+
* You will need an account.  If you don't have one already, you can either sign up directly or through your Steam login.
 +
** '''To use the Steam login click "Login"''', and then click the Steam image that says "click here to log in through Steam".
 +
** '''To sign up directly, click "Signup for a new account."''' You'll need an email address and is slightly more steps than just using Steam.
 +
*** Either method works fine.
  
* You will need to sign up for an account. However, you can use your steam login rather than your email address to make it very quick to do.
+
[[File:MantisSignInViaSteam.png|right|400px|Signing In Via Steam]]
** '''To use the steam login, just click Log In''', and then click the image that says "click here to log in through steam.
 
** If you instead click "signup for a new account," that's fine, but it's going to use your email address and is slightly more steps.  Ultimately both methods are great.
 
  
=== Actual feedback requested ===
+
* When using the bugtracker, be sure to look in the upper right corner of [https://bugtracker.arcengames.com/view_all_bug_page.php the page] and '''change your project to just Heart of the Machine'''.
 +
** This will allow you to see just things related to this project, not other games Arcen has done in the past.
  
* '''If you are confused by something, please note that down.'''
+
[[File:MantisProjectSelect2.png|right|400px|Project Select]]
** What did you expect to happen, and what happened instead?
 
** This info goes on the bugtracker, ideally not in a duplicate ticket of someone else with the same idea: https://bugtracker.arcengames.com/view_all_bug_page.php
 
* '''If you experience actual bugs or glitches, please upload those to the bugtracker.'''
 
** Ideally with a savegame that can duplicate it quickly: https://bugtracker.arcengames.com/view_all_bug_page.php
 
** Again please search for duplicates first.
 
* '''If there was a clever idea you had, but the game wouldn't let you execute on that.'''
 
** Please log that as a suggestion on the bugtracker, ideally with a savegame where you were all set to do the thing, but were unable to.
 
* '''Please note how long it takes you to reach the following milestones'''
 
** Specified in game time (hover the turn-change button in the center of the radial menu in the bottom right of the game:
 
** How long to complete the prologue, please specifically note that here: https://bugtracker.arcengames.com/view.php?id=28176
 
** How long to get to the part of chapter one where it says "goodbye solitude." That goes here: https://bugtracker.arcengames.com/view.php?id=28177
 
*** If you can say both the total time it took you to get here, and the difference from finishing the prologue to here, that would be super ideal.
 
** How long it takes you to get to the part of chapter one where the Contemplate ability becomes available.  That goes here: https://bugtracker.arcengames.com/view.php?id=28178
 
*** If you can say both the total time it took you to get here, and the difference from the prior milestone, that would be super ideal.
 
  
== Information For You To Keep Private (Important!) ==
+
* On that same page, there are also handy filters in the center of the screen.  You can bulk set those via the dropdown in the upper left.  '''The Complete List Except For Closed''' can be useful if you're browsing what other people have said that is currently still outstanding.
 +
 
 +
[[File:MantisBestFilter.png|right|400px|Project Select]]
 +
 
 +
* But if you want to search for duplicate issues, it's best to do it with no filters at all (hit the '''Reset''' button next to the dropdown, if it is there).
 +
 
 +
[[File:MantisResetButton.png|right|400px|Project Select]]
  
* If the information not on the steam store page, then please don't discuss it outside of the Arcen discord prior to June 4.
+
==== General Feedback ====
* If I have not discussed the information in the public part of the discord, then please don't discuss it in the public part of the discord.
+
These are the primary issues I'm looking for feedback for. Create new bugtracker issues for these.
* Discuss anything you like on the bugtracker, or in the private tester section of the discord.
+
* If you are confused by something.
* After June 4, discuss anything you like up through chapter one.  Please do NOT share information about chapter two until a date yet to be decided.
+
* If something unexpected happened.
 +
* If you experience a bug or glitches.'''
 +
* If there was a clever idea you had, but the game wouldn't let you execute on that.
 +
** This includes potential future content (e.g. "I started trying to see if I could do X but it turns out I can't, this feels like something other people might want to do and is something you could add").
 +
** This is probably the most valuable kind of feedback long-term for this game, and frankly probably also the most fun for you to give.
  
== What A Good Bugtracker Report Looks Like ==
+
==== Progression Feedback ====
 +
* Please note how long it takes you to reach each of following milestones and add your time as a comment to the appropriate bugtracker ticket.
 +
* Record your time as game time (hover the turn-change button in the center of the radial menu in the bottom right of the game).
 +
* [https://bugtracker.arcengames.com/view.php?id=28176 Completing the prologue ]
 +
* [https://bugtracker.arcengames.com/view.php?id=28177 Getting to the part of chapter one where it says "goodbye solitude."]
 +
** Missed this?  On your inventory screen, under the Inventions section, it would fall directly between "Microbuilder Programing" and the direct next tech unlock (probably SecForce special, but there are four options).
 +
** If you can also add how long it took to get to this milestone from the prior prologue milestone, that would be super ideal.
 +
* [https://bugtracker.arcengames.com/view.php?id=28178 Getting to the part of chapter one where the '''Computronium''' unlock happens, related to a second discussion with some researchers.]
 +
** If you can also add how long it took to get to this point from the prior 'goodbye solitude' milestone, that would be super ideal.
  
 +
== What A Good Bugtracker Ticket Looks Like ==
 +
* '''It's a new issue (AKA Not a duplicate).'''
 +
** Search for your issue before reporting it. If it's a duplicate and it looks like more data is needed, you can add your details to the existing ticket.
 
* '''It has a concise title.'''
 
* '''It has a concise title.'''
** Thus anyone else searching for duplicates of what they want to report can find it easily.
+
** It makes it easier for others to identify duplicate issues.  
 
* '''It includes a savegame that shows the issue''', or where you'd like some new feature, or whatever.
 
* '''It includes a savegame that shows the issue''', or where you'd like some new feature, or whatever.
 
** Those are located in your install folder of the game, in '''/PlayerData/Saves/[your profile name]'''.
 
** Those are located in your install folder of the game, in '''/PlayerData/Saves/[your profile name]'''.
Line 86: Line 124:
 
* '''It includes your logs if you got an error.'''
 
* '''It includes your logs if you got an error.'''
 
** Those are located in your install folder of the game, in '''/PlayerData/Logs/HeartOfTheMachineLog.txt'''
 
** Those are located in your install folder of the game, in '''/PlayerData/Logs/HeartOfTheMachineLog.txt'''
* '''If it is a laundry-list of items, they are all DEEPLY related.'''
+
* '''If it is a laundry-list of items, that are all DEEPLY related.'''
 
** For example, if you have a big list of typos, feel free to just dump that in one ticket.
 
** For example, if you have a big list of typos, feel free to just dump that in one ticket.
 
** Or for another example, if there are a bunch of locations that all have a similar sort of inconsistency with how clicking works or something, then all those go in one ticket.
 
** Or for another example, if there are a bunch of locations that all have a similar sort of inconsistency with how clicking works or something, then all those go in one ticket.
Line 97: Line 135:
 
** If I need to be involved in the discussion for some reason, then I can be pinged about it.
 
** If I need to be involved in the discussion for some reason, then I can be pinged about it.
 
** Discord is great for discussions; the bugtracker is not.
 
** Discord is great for discussions; the bugtracker is not.
 +
 +
=== Ad Blockers May Mess Up Ticket Submission ===
 +
 +
I'm not sure if this is a generalized thing, but there are no ads on the arcen website at all, anyway.  I have [https://bugtracker.arcengames.com/view.php?id=28184 one report] of getting api errors on the bugtracker when trying to submit tickets until the adblocker was off.
 +
 +
If I had to guess, possibly certain adblockers are stripping out some data that is supposed to be sent with the form (like the savegame or log you're uploading, I dunno).
 +
 +
But in general, this bugtracker has been in continuous use for us since 2011 and this is the sole report of it, so maybe it was just gremlins on that one day in 2024.
  
 
== Other Ways You Can Help ==
 
== Other Ways You Can Help ==
Line 102: Line 148:
 
* If there are reports on the bugtracker that are marked as "feedback requested," then I'm looking for either more information, or a savegame that reproduces (repros) a problem.  If you can provide that, awesome!
 
* If there are reports on the bugtracker that are marked as "feedback requested," then I'm looking for either more information, or a savegame that reproduces (repros) a problem.  If you can provide that, awesome!
  
* If there's a discussion on the private part of discord where other testers are grappling with some issue, trying to decide how best to ask for something, or if to ask for something at all, please feel free to get involved there!
+
* If there's a discussion on the private part of Discord where other testers are grappling with some issue, trying to decide how best to ask for something, or if to ask for something at all, please feel free to get involved there!

Latest revision as of 21:34, 26 May 2024

So, you're private testing the game. Please read the following instructions before anything else.

What Are You Expected To Do?

  • Play the game for at least 2-4 hours within the first 3 days of getting access.
    • Provide a naive player experience, in general, the more blind you can go into the game, the better.
    • But life happens, you know? If something comes up, this isn't a giant crisis and you don't need to feel bad at all.
  • There are specific questions below; please leave those data points in the specific linked bugtracker tickets.
  • If you have some constructive feedback, please add it in the bugtracker as a suggestion ticket.
  • If you have some half-formed thoughts, that are not actionable items, and you want to discuss it with other testers, use the private discord channels.
    • This is a great way for you and others to talk about your thoughts, and then for someone to eventually bubble up an actual actionable item into the bugtracker.
  • Beyond that, play more or don't play more as you wish. This is not a job.

Information For You To Keep Private (Important!)

  • If the information not on the Steam Store page, then please don't discuss it outside of the Arcen discord prior to June 4.
  • If I have not discussed the information in the public part of the Discord, then please don't discuss it in the public part of the Discord.
  • You can discuss anything you like on the bugtracker or in the private tester section of the Discord.
  • After June 4, you can discuss anything you like up through chapter one. Please do NOT share information about chapter two until a date yet to be decided.

Why That Date?

It's a great question, but I can't share the details on that yet. You'll see around the time of that date!

What To Expect From Me

I am a solo dev at this point, which means my time is incredibly limited. I have written a personal essay about testing if you want more context, but don't feel like you need to read it.

I Can't Engage In Long Conversations Or Read Long Write-ups

  • Any time I spend reading feedback or conversing about things is time that is directly taken out of development.
    • In past games, just reading and responding to feedback has taken me 4-6 hours per day.
  • I have a lot of stuff to finish in the coming months, so I can't afford to do that, or even close.
  • Thank you for understanding.

Others Will Help Triage For Me

  • Hooded Horse is providing some professional QA assistance to help read the longer feedback and get that into the bugtracker.
  • My dad is also going to be helping with that.
  • Based on this, I'll spend more time working on the key things people bring up which are surfaced to me, and then content development. This is where you want my time to go.

For Your First Play Session

  • I'm looking for naive player experiences.
  • In general, the more blind you can go in, the better.
  • Avoid learning anything about the game, from other testers, the wiki, Discord, BugTracker, Steam, etc.
  • The more I get of your thoughts, rather than those of the crowd, the better.

Sessions Beyond Your First

  • Please do as you like. This isn't your job.
  • If you don't enjoy the game and want to bow out of testing, that's absolutely fine.
  • If you just can't get enough of it, also fine.

What Is The Current State Of The Test Build?

  • In general, it is highly polished the game is what it is meant to be.
    • I am not looking for feedback on alternative ways for the game to work in a broad sense; it's very much there already.
  • However, it has a vastly lower amount of content than it will eventually (half a dozen hours of content at most, compared to hundreds of hours of content).
    • There are some ragged edges where more content is going to come, or where more options will be provided.
    • At some point during your first play session, in chapter one and after at least 2 hours of play, but possibly closer to 5 depending on your speed, you will run into some areas where you feel like things just kind of... stop. At that point, feel free to check out this link and see if you're at the end of the current content: Current State For Testing.
      • Please don't read that link prior to your first blind testing session, but after that please feel free to refer back repeatedly as updates go out.

Link For The Current State

Again, reminder, please only refer to this after your first blind playthrough: Current State For Testing

Your Mission AKA What I'm Looking For

Use The Bugtracker

Mantis Sign In

The majority of your feedback will go in the Arcen bugtracker.

  • You will need an account. If you don't have one already, you can either sign up directly or through your Steam login.
    • To use the Steam login click "Login", and then click the Steam image that says "click here to log in through Steam".
    • To sign up directly, click "Signup for a new account." You'll need an email address and is slightly more steps than just using Steam.
      • Either method works fine.
Signing In Via Steam
  • When using the bugtracker, be sure to look in the upper right corner of the page and change your project to just Heart of the Machine.
    • This will allow you to see just things related to this project, not other games Arcen has done in the past.
Project Select
  • On that same page, there are also handy filters in the center of the screen. You can bulk set those via the dropdown in the upper left. The Complete List Except For Closed can be useful if you're browsing what other people have said that is currently still outstanding.
Project Select
  • But if you want to search for duplicate issues, it's best to do it with no filters at all (hit the Reset button next to the dropdown, if it is there).
Project Select

General Feedback

These are the primary issues I'm looking for feedback for. Create new bugtracker issues for these.

  • If you are confused by something.
  • If something unexpected happened.
  • If you experience a bug or glitches.
  • If there was a clever idea you had, but the game wouldn't let you execute on that.
    • This includes potential future content (e.g. "I started trying to see if I could do X but it turns out I can't, this feels like something other people might want to do and is something you could add").
    • This is probably the most valuable kind of feedback long-term for this game, and frankly probably also the most fun for you to give.

Progression Feedback

What A Good Bugtracker Ticket Looks Like

  • It's a new issue (AKA Not a duplicate).
    • Search for your issue before reporting it. If it's a duplicate and it looks like more data is needed, you can add your details to the existing ticket.
  • It has a concise title.
    • It makes it easier for others to identify duplicate issues.
  • It includes a savegame that shows the issue, or where you'd like some new feature, or whatever.
    • Those are located in your install folder of the game, in /PlayerData/Saves/[your profile name].
    • If I have to spend hours to recreate the conditions of your report, that's a giant waste of resources.
    • For things that are just text-based, I don't need a savegame. Just make sure you have the exact text, so I can search for it.
  • It includes your logs if you got an error.
    • Those are located in your install folder of the game, in /PlayerData/Logs/HeartOfTheMachineLog.txt
  • If it is a laundry-list of items, that are all DEEPLY related.
    • For example, if you have a big list of typos, feel free to just dump that in one ticket.
    • Or for another example, if there are a bunch of locations that all have a similar sort of inconsistency with how clicking works or something, then all those go in one ticket.
  • Otherwise, it's one ticket per actual item that needs changing, please.
    • When I am going through and (closing/fixing/resolving) issues, I can't do that by partial-ticket.
    • So in other words, if you have a 5 minute fix, one thing that is already working as intended, and one thing that will take me 4 hours but is super low priority, I can't remotely deal with that as one ticket. Normally I would fix the first fast, close the second with a note, and then mark the third one with the appropriate status to get to later. But when they're all in one ticket, it doesn't work.
  • It is actionable in some way. It's not discussion-oriented.
    • If you have something you want to discuss, then please do that with other testers in the tester section of the discord.
    • Where actionable items result out of some sort of consensus, one person can then put that on the bugtracker.
    • If I need to be involved in the discussion for some reason, then I can be pinged about it.
    • Discord is great for discussions; the bugtracker is not.

Ad Blockers May Mess Up Ticket Submission

I'm not sure if this is a generalized thing, but there are no ads on the arcen website at all, anyway. I have one report of getting api errors on the bugtracker when trying to submit tickets until the adblocker was off.

If I had to guess, possibly certain adblockers are stripping out some data that is supposed to be sent with the form (like the savegame or log you're uploading, I dunno).

But in general, this bugtracker has been in continuous use for us since 2011 and this is the sole report of it, so maybe it was just gremlins on that one day in 2024.

Other Ways You Can Help

  • If there are reports on the bugtracker that are marked as "feedback requested," then I'm looking for either more information, or a savegame that reproduces (repros) a problem. If you can provide that, awesome!
  • If there's a discussion on the private part of Discord where other testers are grappling with some issue, trying to decide how best to ask for something, or if to ask for something at all, please feel free to get involved there!