Hmm... it solved the exact same problem for me in Opera 12.14. After making the "masking" change, did you reload the site page or restart the browser to clear any old settings/site-cookies? Otherwise, I'm not sure what is happening for you... I suppose it's always possible something in the 12.15-12.17 series of updates boggled things such that 12.17 doesn't work with changes VirusTotal may have made to their site code - but I wasn't aware of any version change-notes that should have impacted things to this extent.
If you want to play H.264 videos, you have to install HTML5 Extension for Windows Media Player Firefox Plug-in
Looks like an extension that uses an the Firefox Windows Media NPAPI plug-in to support HTML5 video. But, I see the extension is in Firefox's XPI format. Is there an Opera version of the extension too?
Sadly this is the reality of a browser that is old by web standards and is no longer being developed. You would have no issues opening FB with the new Opera browser. Why not install it alongside the older 12.16 and use it specifically for sites like FB that won't open properly in the older version of Opera?