Now that the GPAC / Osmo4 player supports HEVC decoding and playback, it's becoming easier to compare the quality of the new High Efficiency Video Coding (HEVC) standard with its predecessor, H.264/AVC. I've uploaded three versions of the same sequence:
a. Encoded using HEVC / HM10
b. Encoded using H.264 / x264 at twice the bitrate of a.
c. Encoded using H.264 / x264 at the same bitrate as a.
To compare H.264 and HEVC / H.265:
1. Download and install the latest version of GPAC, which includes the Osmo4 player:
http://gpac.wp.mines-telecom.fr/downloads/gpac-nightly-builds/
(Choose Latest Available Binaries)
(Choose Latest Available Binaries)
2. Download the following test sequences:
a. Kristen and Sara, 720p, 10 seconds, 60 frames per second, HEVC / HM10 Anchor (Random Access, QP=32), approx 420 kbps.
b. Kristen and Sara, H.264 / x264 / veryslow preset, QP=31, approx 810 kbps. https://www.box.com/s/is2sdqh7tqdiuc1i76g6
c. Kristen and Sara, H.264 / x264 / veryslow preset, QP=37, approx 440 kbps.
3. Play back each test sequence using the Osmo4 player.
Things to look out for:
- At the same bitrate (~420kbps), is the quality of the HEVC sequence (a) significantly better than the H.264 sequence (c)? In my view, yes.
- At half the bitrate (420kbps vs. 810kbps), is the quality of the HEVC sequence (a) as good as the H.264 sequence (b)? In my view, the HEVC sequence displays smoother motion, but loses some fine detail.
To experiment with other test sequences:
1. Download + install GPAC, as described above:
This package includes the Osmo4 player and the MP4Box utility.
2. Download an encoded HEVC anchor bitstream (.bin) from:
ftp://ftp.kw.bbc.co.uk/hevc/hm-10.0-anchors/bitstreams/
(I chose "Kristen and Sara, QP=32" from the Random Access folder for the above examples).
(I chose "Kristen and Sara, QP=32" from the Random Access folder for the above examples).
3. Rename the downloaded bitstream: from (name).bin to (name).265
4. Package as an .mp4 file. Open a command prompt and type:
mp4box -fps (fps) -add (name.265) (name).mp4
- where (name.265) is the renamed bitstream file and (fps) is the number of frames per second of the original sequence.
(Here's an example of the command line for the Mac, your path may be different: /Applications/Osmo4.app/Contents/MacOS/MP4Box -fps 24 -add Kimono1_qp32.265 Kimono1_qp32.mp4)
(Here's an example of the command line for the Mac, your path may be different: /Applications/Osmo4.app/Contents/MacOS/MP4Box -fps 24 -add Kimono1_qp32.265 Kimono1_qp32.mp4)
5. Open with Osmo4 and play back.
Notes
Windows: Playback of 720p HEVC sequences is a little jerky but works reasonably well. Playback of higher-resolution sequences is very jerky on my PC.
Mac : Playback of 720p and 1080p HEVC clips on my Mac (10.8.3, 2.9GHz) is reasonably smooth.
Mac : Playback of 720p and 1080p HEVC clips on my Mac (10.8.3, 2.9GHz) is reasonably smooth.
Quality is subjective
Video quality is, of course, highly subjective. I'm interested in your feedback. Do you think the 420kbps HEVC sequence (a) is as good as, or better than, the 810kbps H.264 sequence (b)?
- Iain Richardson
17 comments:
Hi,
Some notes from my experiments:
HEVC video is not playing on Mac OS X 10.8.3 (black screen, no progress). H.264 - no problems.
On Windows 7 (x86) - both H.264 and HEVC play correctly but the frame rate is dropped for HEVC.
Obviously quality is better in HEVC, especially skin tones of the lady on the right hand side.
Regards,
Maciej Pedzisz
Thanks Maciej. I had the same problem with MacOS, I think I will contact the GPAC team about this.
I found that at higher resolutions (e.g. 1080p), HEVC video did not play smoothly on Windows.
Iain, can you also upload the HM 10.0-encoded bitstreams?
Raul,
I used one of the HM10 Anchor sequences, Kristen and Sara / Random Access / QP=32, see the link in the article.
All the best
Iain
Hi, Iain,
From our experiments HEVC provides twice bitrate saving starting from 1080. For SD it saves around 15-25%, for 720p - 30-40%.
In fact, it hardly depends on the content. To be more precise, on number of PU's bigger than 16x16 and TU's bigger than 8x8.
If we'll limit size of CU to 16x16 and TU 8x8, HEVC will give around 10-15% gain. Mainly due to Intra Prediction and better MV's encoding.
Another story is filtering. HEVC encoding looks better than AVC even at lower PSNR. Thats what you see "HEVC sequence displays smoother motion, but loses some fine detail"
For windows playback you may use free Elecard HEVC demo player
http://www.elecard.com/en/technology/researchlab/hevc-player.html
It easily run 1080p on average PC.
Hi, Iain,
From our experiments HEVC provides twice bitrate saving starting from 1080. For SD it saves around 15-25%, for 720p - 30-40%.
In fact, it hardly depends on the content. To be more precise, on number of PU's bigger than 16x16 and TU's bigger than 8x8.
If we'll limit size of CU to 16x16 and TU 8x8, HEVC will give around 10-15% gain. Mainly due to Intra Prediction and better MV's encoding.
Another story is filtering. HEVC encoding looks better than AVC even at lower PSNR. Thats what you see "HEVC sequence displays smoother motion, but loses some fine detail"
For windows playback you may use free Elecard HEVC demo player
http://www.elecard.com/en/technology/researchlab/hevc-player.html
It easily run 1080p on average PC.
Dear Iain, to the best of my knowledge, there were a lot of subjective tests that compared HEVC and AVC in terms of efficiency within JTC-VC meetings. But I would like to propose comparison between HEVC, AVC and probably google video codec (as it might be one of strong rivals) based on different video sequencies, several subjectve criteria clearly defined before the evaluation and probably complexity. Last one is not very simple to be implemented - nevertheless there are several papers devoted to this issue. In my understanding, it would be practically good to have a chart with 3 axises: quality, bitrate, complexty.
BTW: considering focusing of HEVC on UD and higher resolutions, should we consider "wide-angle" sequencies that have the same number of lines as HD but significantly larger number of pixles? Total number of pixels will be the same as in UD, but correlation details and motion structure will be different. I think such type of content might be useful in future as well. Good luck.
This is cool!
Hi Iain, the Osmo4 cannot run any of the streams (both HEVC and H.264) and the message was
The document "kr_x264_qp31.mp4" could not be opened. Osmo4 cannot open files in the "MPEG-4 File" format.
I downloaded the GPAC-0.5.1-DEV-r4690.dmg onto my Mac running version 10.6.8 in 32-bit mode.
Hi, not sure what the problem is (1-Aug-13). I wonder if it's because you have a fairly old version of MacOS? If you email me directly we can discuss further (iain AT vcodex.com).
- Iain
Hi,
- Will the new h265 codec improve all the nasty video artefacts commonly found with previous encoders (that would degrade the user experience looking for an overall better image quality, not for better compressions due to wrong commercial reasons)?
- DivX/MP4: pixelisation in dark areas (blinking black square effect)
- MPEG2: end of lines interlacing in snapshots or scrollings + jerky wide areas scrollings (don't-want-to-let-move-this-block effect)
- AVC/h264: dizzy scrollings and waving distorded face on a close-up (tile effect)
- Will h265 codec improve compression at higher bit rates, let say, at 150 fps -- more frames, less differences, less encoding?
- What kind of size ratio shall we expect between a 4k 24 ips movie and the same in 100 ips for instance (based on your h264 800kb/s equals h265 400kb/s quality) ?
- Will it help in encoding 3D videos -- the same way mp3 had special features to better encode stereo channels ?
- For benches, following Михаил's advice and based on what I exposed above, I propose to also add framerate, 3D, wide areas slow scrollings and still close-ups inputs on top of his unique scene complexity parameter (linked only to the number and moving ability of mobile objects I guess) leading to his dual quality / bitrate outputs.
- I remember my first experience when decoding a x264 video that would freeze my PC. What kind of power would be required to decode realtime a 4K 100 ips video?
Note: my 'old' WinXPSP3 Intel Core Duo @ 2,4Ghz 3GB is down on its knees and can't process more than the 2 first seconds of the h265 file you provide ;-) -- no particular issue with the h264 files, though...
Of course I understand that the now wide spread multicore GPUs would help in that a lot, but is your GPAC/Osmo4 h265 decoder enhanced for these yet?
Btw, Andrey's Elecard video player doesn't work for me and pops up a 'connection error' while opening the file ('source filter can't connect to demultiplexer').
- Is the h265 codec also aiming at preserving the Bluerays media while soon encoding 4K on it or shall we expect yet another home decoding device with new laser beam technology and new media, as we already had been subjected to between CD, DVD and BD?
- What can we expect from a hypothetical future h266 format? Didn't we already have reached the limits of all kind of video compression?
In term of differences between videos, I am still alarmed by the vertical bars / square effects of both h264/h265 formats, e.g. at the right boundaries of the left handside girl's hair on her face/forehead.
Can't we get rid of those?
OR is it possible to check videos with a better antialiasing filter applied on?
Thanks a lot,
BR/Olivier
Hi Ian,
I encoded raw video sample to .hevc and to .264 using HM11 and x264 reference encoders. Also, I muxed encoded videos to mp4 container. Now, I need player that supports both hevc and x264 videos and which has option to capture certain frame, since I want to compare frame quality.
Hi Ian,
What software should I use to be able open both .265 and .264 video samples to compare frame guality?
I just want to open certain frames of both codecs like:
http://3.bp.blogspot.com/-Q9GvLnoaKh0/UXeR1t7Q6zI/AAAAAAAAAJI/jl6yK6xLawI/s1600/kristen_800_420_comparison.png
Hi,
To open sample frames and compare quality: I would suggest using the JM and HM software respectively to decode to a .YUV file, i.e. a raw output video file. Then you can view frame by frame using a YUV viewer. I use GLYUVPlay on the Mac.
- Iain
Thanks for reply.
Why you prefer comparing them after decoding them back to YUV format?
I have two video samples (.265 and .264) muxed in .mp4 container. Why you don't prefer comparing them in encoded format?
Hi Iain,
I found somewhere tutorial for testinfg hevc against 264. Firstly, raw YUV video was encoded by reference encoders, THEN it was decoded back to YUV and THEN comparison of frames was realized. My question is, why it is better to compare subjective quality after decoding back to YUV? Why comparison wasn't made directly from encoded sequences?
Should you compare quality by (a) playing back the encoded material using a player with integrated decoder, or (b) playing back fully decoded YUV / "raw" files?
I would say both have their pros and cons.
With (a), you have to ensure that the playback environment is the same for every clip under test, which may be difficult unless your player supports every tested codec in the same way. Plus you can't play back the original (uncompressed) YUV file this way.
With (b), you may have problems playing back YUV files smoothly, especially at higher resolutions (1080p+). This is because of the very large amount of data that must be read from the disk, rendered + displayed in real time.
- Iain
Post a Comment