Don't know if it'll help, but you can try all or some of following:
Make sure you're not using any limiters in the GX Control panel.
Make sure the battery saver is off at the URL opera://settings/batterySaver.
Make sure RGX is off at the URL opera://settings/lucidMode.
Make sure Opera's adblocking and tracking protection is completely disabled at the URL opera://settings/privacyProtection. It can cause Youtube to purposely make things slow. Use uBlock Origin instead and read the Youtube sticky thread at https://reddit.com/r/uBlockOrigin/ if you have any issues with Youtube.
Make sure "Snooze inactive tabs to save memory" is disabled at the URL opera://settings/userInterface, install https://chromewebstore.google.com/detail/disable-automatic-tab-dis/dnhngfnfolbmhgealdpolmhimnoliiok and disable the "Calculate window occlusion on Windows" flag at the URL opera://flags/#calculate-native-win-occlusion.
Make sure "Use graphics acceleration when available" is on at the URL opera://settings/system and then adjust the Angle flag for your GPU.
If all that doesn't help, you can try disabling the "Hardware-accelerated video decode" flag at the URL opera://flags/#disable-accelerated-video-decode while watching Youtube in Opera while gaming to see how things go. If that doesn't help, you can enable the flag again and test with "Use graphics acceleration when available" disabled at the URL opera://settings/system to see how things work with Youtube while gaming.
If you have more than one GPU on your system, you can set Opera to use the slower GPU that you don't use for gaming. In settings in Windows, goto System -> Display -> Graphics, add Opera GX to the list and change its options to use the GPU you want. Or, the other way around, if Opera is using the slower GPU by default, you can make it use the better one to give Opera more performance so that Youtube plays better. You'll have to test how that interferes with your game performance if any.
If you normally use Opera GX's CPU limiter while gaming, consider doing it on the "Details" tab in the Windows task manager instead. You can right-click on an opera.exe process, choose "set affinity" and select only a single core like "core 5" for example that the game you're using might not make much use of. You can do that for all opera.exe processes if you start Opera GX like this.
You can also disable any extensions and mods in Opera while you're gaming.
You could have a standalone installation of Opera GX to use to play Youtube just while gaming and do all the tweaks to just that standalone installation.
