Internal Testing, Day 14 (Patreon)

right after i posted yesterday i set off to work ordering up our bug list with all the things that need to be tested and such. it was very neat. very tidy. "great," i thought to myself with pronounced satisfaction, "this will be perfect to expound on for tomorrow's patron post."

unfortunately that list has since been actively deleted. mostly.

you know i was just happily testing along, oh, that's not an issue, oh, that one is, here's what to do about it
and anyone who knows me well knows i love deleting things. it's cleansing. enriching. i am a forest fire. destruction is growth.
finish a bug? delete it! that's one less space we have to worry about. that's what progress feels like!

and then only after a fifteenth ctrl+s did i realize i had deleted most of what i intended to post today without backing any of it up.
ka-whoops.

this means i have to go by my memory.
this is a mistake. 

one thing i do remember though is a certain issue i found and fixed last night and was very upset about. okay. so. in a past post when talking about AI y'all remember how i was like "we used to use the standard deviation for a range of the best moves but since that doesn't count outliers it's garbage so we're not gonna do that anymore"

right so we changed that. the old system before that asks for the moves that are above 80% of the best move's score and only looks at those. this is essentials' default handling. i brought this back. it was removed before because it wasn't really an appropriate way to handle things when moves were on the scale they were before, so we used the standard deviation to make it scale with the.. uh, scale. this was an improvement. 

however, now that we have a very set scale based on %HP damage this kind of treatment is much more appropriate. i brought back this flat threshold although i made it 95% instead of 80 to be more choosy. this was an improvement as well.

so the bug was that it wasn't doing that.

i noticed when we were testing something else. the AI had scored some moves (in this case, blizzard vs flamethrower) correctly (vs K-Sandslash-- blizzard is obviously the better choice. and it knew it). but it used the wrong move, flamethrower. i checked the values. it was under 95%. flamethrower should not have been considered. so why. i had given it clear instructions and it had disobeyed me. how dare it.

it didn't take me long to notice the problem. when we're getting the best moves, we have a loop. the loop says something like, for each move, see if the score is higher than (95% of highestscore) and if so include it, then go to the next move

in order to get 95% of highestscore we do something like "highestscore * 0.95". simple, right?
but wait. unfortunately, the code had that in as "highestscore *= 0.95". computer science folks already see the problem now but i'll explain for the rest. 

in ruby (and many other languages) you have various operators for math, addition subtraction etc, we know this, and * is for multiplication. also, all of those operators have a variant where you can do like +=, -=, etc to do that math and set the original variable equal to the result. it's a handy shortcut. but you see the problem? *= updates highestscore, even if it's in the middle of a conditional statement. 

so for the first move, we're asking for 95% of highestscore.
but for the second move we're asking for 95% of 95% of the highest score....
and so on.
by the fourth move slot, we're down to generously considering as low as 85% of the original highest score. dameyo! the order of moves was affecting their selection

the reason i am so upset about this is because the only thing i changed from the 80% code is the number..... meaning this issue exists in base essentials. or existed-- since we're on an older version. that also means that in the old AI before we switched to Std Dev (around E13) our fourth move was getting to considering as little as 50% of the highest score. even if it knew what a good move was, it wouldn't often be picking it!!
but there i was thinking "all right well that doesn't make sense to me, to have the *= there, but it's native code so i'll trust that it's there for a reason."

foolish fool.