Here's an example. This is a close-up of a frame from the sequence "Kristen and Sara", 720p resolution. On the left, the sequence has been encoded at 800kbps using the x264 video encoder, a popular H.264 implementation. On the right is the same frame from the sequence encoded at 420kbps using the HEVC Test Model encoder, HM10.0. The frame on the right was decoded from one of the HEVC Anchor Sequences (Kristen and Sara, QP=32) which you can download here: ftp://ftp.kw.bbc.co.uk/hevc/hm-10.0-anchors/bitstreams/ra_main/. The original YUV test sequences are available here: ftp://hvc:US88Hula@ftp.tnt.uni-hannover.de/testsequences.
 |
| Left: x264 at 800kbps. Right: HM10 at 420kbps. |
 |
| x264 at 800kbps: extreme close-up |
 |
| HM10 at 420kbps: extreme close-up |
Here's another example, this time a close-up from a frame of the Racehorses sequence, originally 480p resolution:
 |
| x264 at 1.8Mbps : extreme close-up |
 |
| HM10 at 960kbps: extreme close-up |
HEVC will give a clearer, higher quality image than H.264 at the same bitrate. But is it really twice as good as H.264, i.e. does it give the same quality at half the bitrate? These still frames imply that HM10 does not perform as well as H.264 at half the bitrate. However, recent research by Wang et al indicates that HEVC has bigger performance gains for moving image sequences than for still images taken out of a sequence.
It's also worth noting that x264 is a highly efficient implementation of H.264 that has been optimized over a period of years. As HEVC and its implementations mature, we can expect improvements in performance and image quality.
I'll be posting more results soon and explaining exactly how to test this for yourself. To download the HM reference software, visit this page.
20 comments:
Thank you for an interesting comparison. With reference to Sara, I agree that the HEVC frame looks smoother in comparison to the H.264 frame, but it's also lacking some of the artifacts that the H.264 frame suffers from. For example, the blue background to the left of the face looks a bit "noisy", and ringing around the edges appears stronger for H.264.
I'm not sure what's worse: more blur or stronger artifacts. It'd be interesting to see how they stand out in an actual moving picture. I'm looking forward to seeing more results.
Thanks Misha, good point. I agree about the visible artefacts in the H.264 frame. However, what concerns me about the HEVC frame is the loss of information. Comparison of moving images requires a real time decoder or playing back the full raw decoded files. I'll be posting more results and examples soon.
- Iain
Could you please add the uncompressed input images a reference to compare against? How can you compare the output of two lossy codecs without looking at the input?
Hello Markus, thanks for the comment. If you follow the link you can download the original YUV sequences yourself - you will need software such as GLYUVPlay to play them.
It's common practice to compare the output of different processes without the original (reference). For example, Absolute Category Rating and Pair Comparison (ITU-T P.910) and Single Stimulus Continuous Quality Evaluation (ITU-T BT.500) are all methods of no-reference subjective quality testing.
- Iain.
Zhou Wang had a paper on this: https://ece.uwaterloo.ca/~z70wang/publications/vpqm13.pdf
Long story short, the magic of HEVC is in its massive reduction of temporal artifacts, not necessarily spatial ones.
In fact, the paper continues to detail tests showing how subjective analysis of stillframes will significantly underestimate the quality of a clip as a whole.
Very interesting paper by Zhou Wang, I'll update the blog with a link. Thanks for the comment.
I'm going to suggest that you're looking at the issue from the wrong perspective, Ian. The codec isn't about using half the resources to be as good, it's about a fixed pipe capacity delivering more channels. Whether its fibre and the 'conversations' or its digital transmission, broadcasters have the key issue of more channels of content in a fixed volume of delivery channel. Now, add 4K, or yet another must have home device and the new iteration of the codec is essential. I'm guessing that even 6months of scrutiny and shaping will improve it greatly. Thanks for the great work, by the way.
Very interesting results!
I have also read the following papers, which compares HEVC and H264, without however noticing the details of your listing.
However, they report some interesting findings on the performance of the two codecs.
http://aias.iit.demokritos.gr/~koumaras/EuroITV%20KOUMARAS-%20Camera%20ready.pdf
and
http://aias.iit.demokritos.gr/~koumaras/KOUMARAS_ICT_CAMERA%20READY.pdf
It seems that the performance of HEVC is doubled in comparison to AVC, but especially in the temporal domain. In the spatial domain, needs more investigation (in cases of half bit rate).
Please keep posting more results.
I think taking a still image out of a moving video which it has been optimized to is not a good way of judging stuff.
Most importantly dancing noise is impossible to see on a still but obvious on a video. I think I will trust the pros and vector encoding with more info in less space until I see it at NAB. Also, how about an example with modern 1080p instead of 720 and who shoots 480i?
Nice to see the debate around this topic. I will look into generating some sample clips that can be downloaded and compared.
Great post, Iain.
Any chance of some downloadable short clips being forthcoming?
Great blog post, Iain,
Any movement on generating some short downloadable clips for evaluation/comparisons?
Yes I will put together some short YUV clips...
Do you mind posting the .cfg file you used with the reference HEVC encoder...or did you just use one of the default .cfg files that come with HM10? (However, I'd venture a guess that you probably tweaked at least some of those numerous parameters) :) Also, Do you mind posting what x264 parameter values you used to create your h264 comparison?
Sean - I used an HEVC anchor sequence, please see the link to these clips. The configuration settings are listed in a JCT-VC document, if I remember correctly. I'll add the x264 command line parameters shortly.
- Iain
This is cool!
Hi Iain,
I downloaded hm10 bitstream from site you gave in post, and tried to decode it with tappdecoder (from HM11) and got this erro:
http://postimg.org/image/un16auskd/
Is that because newer HM can't decode older HM bitstreams?
If you have tappdecoder from HM10, can you upload it?
Hi
Instead of trying to decode the HM10 bitstream, I would suggest downloading HM11 anchor bitstreams instead: ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/
- Iain
Hi All,
Where can i get test bit-streams for main-still picture profile? And yuv for test images.
Very Informative, Thanks a Lot Ian ..
Post a Comment