Looks like Lance was right!
I was able to get around this issue by closing all other windows while watching something fullscreened.
Turns out, I didn't even need to close out of the other windows- even just minimizing them worked.
In short, the current workaround is to minimize all other windows you have open.

Still happening, on 118.0.5461.50.
Is anyone submitting bug reports?
I'd think that such an annoying issue would be looked into a bit quicker.