Fixed it (Patreon)

https://www.youtube.com/watch?v=DlgWsFJScU8

I was able to rescue it entirely with editing! Rejoice.

This is the final version which I'll be releasing tomorrow. The two edits begin at about 1:16:25 and 1:28:00; I essentially just cut out the conjecture about color increasing the complexity/bandwidth of sensors, because that appears to be completely false, and moved the rest of the CCD talk into the explanation of 3CCD systems.

This error happened because I've had a misunderstanding of bayer sensors for over twenty years. Everyone I've described this mistake to has replied "...that's how I thought it worked too," so I figured I'd explain what I got wrong.

I was under the impression that they had implicit pixel binning; that if you had a 640x480 camera, that meant it had four times as many photosites, and that each group of RGGB got blended together to produce a finished pixel. This seemed reasonable enough, and it also seemed necessary in order to get accurate luma for each pixel - you can't do that without looking at all three color channels.

This is completely false. A 640x480 sensor has 640x480 photosites (ignoring the actual practice of pixel binning, which I believe is relatively recent?) and simply does not capture full chroma or luma for any given output pixel. If you look at (5,5) in a digital camera image, you are not seeing what was captured at the fifth row and column of the sensor; if you did, it would just be stark red, green or blue, as you can see in various "debayered" sample images online.

To produce a full color pixel in the finished image, the processor has to infer from the surrounding photosites. By all reports this is "incredibly good" technology and has been for a long time. I do not pretend to understand how that's possible - if you didn't capture the red channel at (5,5) then you can never know how much red was at (5,5). You can guess, based on the pixel next to it, but if there was a red dot that fit entirely within a single pixel, that information is lost forever. And that's not just a chroma thing - that dot contributed to luma. If it didn't spill over to the adjoining sites, then you can't guess that it was there, so the final luma will be wrong.

Perhaps this doesn't matter; perhaps there's no functional difference between the way this actually works, and the way I was imagining it with the microlenses etc. maybe because of the realities of optics and maybe because it Involves A Lot Of Math. I am not good at intuiting what is and isn't possible with A Lot Of Math.

What I'm confident about however is that bayer sensors do not capture full data for every single pixel, and this is why they are subject to moire etc. And I'm also confident that 3-sensor systems do - the output from a three-CCD array with a prism system is 4:4:4, every pixel is a stack of red, green and blue values from the exact same physical spot on the image circle cast by the lens. The luma and chroma of the finished pixel are (within limits of optics and ADCs) completely accurate.

The specific description that will be present in the finished video is probably still nitpickable; I expect to get some comments about it, but I'm satisfied with it for the purpose of a summary in passing. In the future I will probably just avoid the hell out of discussing the details of image sensors, I am starting to think that I will just never really fully get it.