VideoEncoder API with H.264 high profile config produces choppy video
-
jozefchutka last edited by
I have observed there is something wrong with VideoEncoder API esp. in combination with H.264 high profile producing choppy video when played in browser.
I am using opera
Version: 93.0.4585.11
Update stream: Stable
System: Windows 10 64-bit
Chromium version: 107.0.5304.88Steps to reproduce:
- run attached HTML code (creates h264 stream from simple animation on canvas)
- save generated blob.h264
- run generated ffmpeg command to mux into blob.h264.mp4 (without re-encoding)
- play blob.h264.mp4 in browser to observe choppy video
<body> <style> canvas, a, code {display:block} </style> <script> (async () => { // avc1.640016 (H.264 high, 2.2) - choppy // avc1.420028 (H.264 baseline, 4.0) - smooth const codec = "avc1.640016"; const width = 640; const height = 360; const framerate = 20; const config = {codec, width, height, framerate, avc:{format:"annexb"}, hardwareAcceleration:"prefer-software"}; console.log(await VideoEncoder.isConfigSupported(config)); const chunks = []; const encoder = new VideoEncoder({ output:chunk => { const buffer = new ArrayBuffer(chunk.byteLength); chunk.copyTo(buffer); chunks.push(buffer); }, error:error => console.log(error) }); encoder.configure(config); const canvas = document.createElement("canvas"); canvas.width = width; canvas.height = height; document.body.append(canvas); const frames = framerate * 5; const ctx = canvas.getContext("2d"); for(let i = 0; i < frames; i++) { ctx.clearRect(0, 0, width, height); ctx.beginPath(); ctx.rect(i / frames * (width * .8), height / 2.5, width / 5, height / 5); ctx.fillStyle = "red"; ctx.fill(); const frame = new VideoFrame(canvas, { format:"RGBA", codedWidth:width, codedHeight:height, timestamp:Math.round(i / framerate * 1000000), duration:Math.round(1 / framerate * 1000000)}); const keyFrame = !(i % framerate); encoder.encode(frame, {keyFrame}); frame.close(); } await encoder.flush(); encoder.close(); const blob = new Blob(chunks); const name = "blob.h264"; const link = document.createElement("a"); link.href = URL.createObjectURL(blob); link.download = name; link.innerHTML = `download ${name} (${blob.size} bytes)`; const code = document.createElement("code"); code.textContent = `ffmpeg -r ${framerate} -i ${name} -vcodec copy ${name}.mp4`; document.body.append(link, code); })() </script>
Observations:
- h264 generated in Opera using avc1.640016 encoder plays choppy in Opera
- h264 generated in Chrome using avc1.640016 encoder plays smooth in Opera
- h264 generated in Opera using avc1.640016 encoder plays smooth in VLC
- h264 generated in Opera using avc1.420028 encoder (baseline profile) plays smooth in Opera and VLC
-
burnout426 Volunteer last edited by burnout426
If you mess with the Angle flag, does the resulting mp4 still play choppy?
-
burnout426 Volunteer last edited by
A bug (DNA-103558) was filed internally for this to investigate, when there's time. I'll post if there are any updates.
-
burnout426 Volunteer last edited by
See https://bugs.chromium.org/p/chromium/issues/detail?id=1383643#c5 and https://bugs.chromium.org/p/chromium/issues/detail?id=1383643#c7, which might have something to do with this.
-
jozefchutka last edited by
Thanks for keeping an eye on this. Potentially related, but hard to say for sure.
My issue is only reproducible on opera using
hardwareAcceleration:"prefer-software"
, while such configuration it not supported in my chrome or edge. -
leocg