About the eventual testing (Patreon)

Although, there is Still A Lot To Do™ ,
The main content of the quests is done
strong exhale
tear up a little
jesus fuckityfucking fuck

Okay, so let's talk about testing phase. It's still gonna be a bit before we get there, but I'm getting more and more questions about it, and especially for y'alls part of things that's fair because there's a few changes I want to make. 

First, let's talk about our usual testing processes. In past recent episodes, our testing phases have gone like this:

Supervised > Internal > Alpha (Patreon) > Beta > Community  > Public Release

Supervised Testing: I'm watching someone play through the new content of the game. Usually it's Ikaru or someone living with me IRL. I like doing this because in addition to getting bugfixes, I can immediately help them if they get stuck, I can see if there are small details I'd like to improve, and I can be seeing what things are unclear and what I need to improve on without instruction. ....also live reactions are always fun <3

Internal Testing: The dev team plays through! They've already been doing a little bit of this here and there, but this time for Realsies. I expect this phase to go smoother now that we're actually on git. God fucking bless git tbh. 

Alpha Testing: Historically, that's patrons!  Hi! Thanks for being here!

Beta Testing: Usually a group of forum veterans or hand-picked applicants doubling the alpha size. Trying to get stress test with reliable people.

Community Release: What would usually be called an 'open beta'; available to anyone with a forum account. I feel strongly about doing these because it means people who care more can sign in and get it, but it's not the publicly available download that a rando playing the game for the first time would see. That way if there's a crash or something the people on the unstable version are at least more likely to come back for the patch and try again.
Also, community release is always always always the most chaotic hellfuck nightmare rush time for me. Through sheer volume, I don't think there's a way to get around that but it's yikesville every time.
I'm hoping the new verison not being public right away also softens the blow-up/risk of it being noticed by disapproving parties.

Public Release: The game is Done and Out and I am Free and the bugs are Fixed and-- lol jk somehow despite all the testing, someone reported a fatal crash on day 1 fml
(every time.)

--

...So that's how it has been. But we're going to try approaching things a little differently this time.

Supervised > Internal > Multi-supervised > Alpha > Beta (Patreon) > Community > Public

I am pushing you guys from Alpha to Beta, with respect and concern. I know having to wait a little longer sucks, and if everyone is like, deeply unhappy with this, we can talk about it. But I feel pretty strongly about it. Some reasons:

• It would not be appropriate for the beta group to be smaller than the alpha, and there's like twice of many of you compared to last time. Trying to be selective about several hundred people is not a good use of time and energy.
• Rejuv's patreon test discord was of concerning chaos and caliber. To be fair I haven't done testing discords in the past; forums only-- but I plan to have one this time. However: yikes.
• For the support you all have given me, I want to give you a (at least slightly) more polished experience. 
• Some of you are more eager and able to help than others. Pretty much everyone is very busy always. There will certainly be room to help, but picking a smaller and more active group for Alpha will help get certain things smoothed out sooner too.

As a side note, while we're talking about patreon reward changes, I plan to expand the VIP status on the discord down to the Level 25 tier in hopes anyone is actually there to use the channel. Anyway, testing stuff--

Instead, Alpha will be a smaller, selected group of people who can help keep everything manageable. Right now I imagine that's going to look like, the server staff so they've already played and can focus on helping people out, community modders so they can start making updates if they want, other people who've involved with helping fix bugs and code in the past, etc. Basically whoever can stack the deck in our favor so that everyone has the smoothest experience possible.

Additionally, for similar reasons, I want to have a more fluid rollout this time. Normally when we go into Alpha or Beta, it's like, okay everyone in the test phase now has access to the entire new game, go ham and we will deal with the consequences. And then everything is stressful and we're slammed with bug reports and we cry. Rejuv's new bug form was great (and mind you, I'm speaking as an outsider to their team here), but it seemed like the new reports came in so fast they didn't even have time to read them all. And that's not really helpful to anyone. Rather than that, I want to just hand this out to a few more people at a time, every few days or so or whenever we feel like we're abreast of the incoming reports. Basically, instead of the whole tier testing at once, we will let our pace of fixing dictate cadence of the test.

This is also important because it guarantees at least a few people will be playing on the most recent version of the game. The reason so many bugs make it through to Community and Public Release, aside from the sheer volume of players finding esoteric things at that point, is because a lot of them are like "oops this fix from beta didn't work" or "oops this fix from alpha broke something else in beta" or "oops this fix from alpha broke something in beta which we then fixed but it created the original problem again haha :)"

frankly we had a case like that already-- we the handler for postgame quest tiers had an issue that stopped someone from progressing, so we fixed it. but we didn't realize the Genesect quest had been completely skipped over and forgotten, lol... Perry got to the end of the current quests and was like "what happened to genesect?" and I was like "??????" so we slotted Genesect back in, which in combination with the previous fix, broke it for everyone else again.

The answer to that is just to have more waves. Or perhaps, as we'll be trying, to stop having distinct waves altogether.

There's a second reason for the above in this case in particular: There is just a lot of fucking content in the coming update. I plan on posting some Stats soon to make my point, but like... e18 supervised testing already took two or three evenings. e19 was more than that. and that's before any postgame.
...and that's just one person's test. Getting stuff out there bit by bit means that we can still have other people start testing while we're waiting for other people to finish.

By the way, that Multi-supervised phase up there is a bit of an anomaly (heh). It's kind of happened during the past few betas anyway; a group of friends of mine has a habit of playing through the game all screen-sharing together, usually like ~10 of us doing various challenge runs. And it's been pretty helpful to watch everyone do it, and frankly just a lot of fun.  That happened by chance the last couple releases; this time I'm just planning on it.

So anyway, fluid rollout. Especially when it comes to beta, I'm sure there are some questions about who gets it first. I've been thinking about this a lot and I think the fairest thing to do is to give priority to those with the most lifetime support. What I do not want to do, especially given the precarious nature of this project, is create a situation where people are upping pledges just to get the test sooner. In addition to that, going by lifetime support seems like a great way to appreciate those who've stuck around for so long. This makes sense to me, but again, if many people are very unhappy with this, we can talk.

Okay, all that business aside, let's answer some basic questions about testing:

How long will testing take?

We will let it take as long as it takes, or until I go crazy. Mentally, I am planning on about three months. That is not a promise, or even an official estimate. That is a leaf on the wind, and no one knows where it will go.

How long until we start testing??

don't do this to me

How should we report bugs?

The forum is outdated. Discord is too chaotic. Rejuv's bug form was neat, but overwhelming and unstructured. I'll have to get back to you once I've actually set it up, but as we've discussed on the server I'll be looking at setting up a pre-existing bug tracking software for us. I'll make sure this is broken in and communicated by the time we start testing.

How will testing be structured?

Subject to change based on how things go; I imagine a separate testing discord that we gradually invite people too as the game is made available to them. It will mainly be for discussion. Bugs will be reported elsewhere, possibly feedback too. 

Are we looking for feedback and critique or just bug testing?

I am very interested in feedback and reactions. I definitely want to know how the game has made you feel at various points-- what worked well, what was frustrating? What was unclear? What did you like, what was underwhelming? What were your difficulty spikes?  However, I am much more cautious of critique. I would like to be told the problems; not the solutions. It's our job to figure out the solutions. I am all for ideas and talking out problems, but I do not want to invite critique because it brings expectations and entitlement with it*.

(*I expect that is moreso the case with the general fanbase rather than you all in particular, but that will be my stance for testing overall)

If you guys have other questions about testing, I will be happy to continue answering them to the best of my ability. Otherwise, I will once again thank everyone for being patient enough to get to this point with us. Just a while longer while we polish everything up...

Expect another post on our next immediate project soon <3