Vcodex

High Efficiency Video Coding (HEVC) : Test model HM3

2011-07-05T11:16:00.003Z

The Joint Collaborative Team on Video Coding (JCT-VC), a joint effort of the MPEG and VCEG standardisation committees, is developing a new video coding standard known as High Efficiency Video Coding (HEVC). HEVC is expected to be published as an International Standard in 2013.

The HEVC "Test Model" is a working prototype that includes the tools or algorithms that are under consideration for the proposed HEVC standard. Version HM3 of the test model (March 2011) defines two groups of tools or "configurations", High Efficiency and Low Complexity.

Both High Efficiency and Low Complexity configurations include the following coding tools:

Coding Unit tree structure (8x8 up to 64x64 luma samples)
Prediction Units
Transform unit tree structure (3 level max.)
Transform block size of 4x4 to 32x32 samples
Mode-dependent Transform for 4x4 block
Spatial Intra Prediction (34 angular directions and Planar)
Adaptive Intra Smoothing
Intra Chroma Prediction using Luma samples
DCT-based interpolation filter for luma samples (1/4-sample, 8-tap)
DCT-based interpolation filter for luma samples (1/8-sample, 4-tap)
Coding Unit based Skip & Prediction Unit based merging
Advanced motion vector prediction
Deblocking filter

The Low Complexity configuration includes Context Adaptive Variable Length Coding (CAVLC).

The High Efficiency configuration uses Context Adaptive Binary Arithmetic Coding (CABAC) and supports increased bit precision for internal operations and an adaptive loop filter.

Download the HM3 Test Model description document.

Keep up to date with the HEVC contributions and documents via the JCT-VC working site:
http://phenix.int-evry.fr/jct/index.php

MPEG Royalty Free video coding standard

2011-05-17T09:04:00.002Z

From MPEG's March 2011 press release:
'As announced in January, MPEG will develop a new video compression standard in line with the expected usage models of the Internet. The new standard is intended to achieve substantially better compression performance than that offered by MPEG-2 and possibly comparable to that offered by the AVC Baseline Profile. MPEG issued a Draft Call for Proposals (CfP) for Internet Video Coding Technologies that is expected to lead to a standard falling under ISO/IEC “Type-1 licensing”, i.e. intended to be “royalty free”. Proposals are due in October 2011 and will be evaluated at the 98th MPEG meeting in Geneva. It is expected that this standard will become the default video codec for internet applications.

The text of the Call for Proposals is available at http://mpeg.chiariglione.org/working_documents/explorations/opt1-licence/opt1-lic-cfp.zip. Responses to this call are due in October 2011 and will be evaluated at the 98th MPEG meeting in Geneva.'

At CES, January 2011

2010-12-10T10:57:00.002Z

I'll be at the Consumer Electronics Show in Las Vegas on 6th and 7th January 2011, with a demo of the new OneCodec concept (onecodec.com). If you would like to meet me, let me know (iain@vcodex.com).

High Efficiency Video Coding: HEVC Test Model

2010-11-30T10:20:00.003Z

The JCT-VC have specified the first version of the Test Model for the new HEVC standard. The Test Model, HM1, will gradually evolve into the final version of the new standard as coding tools are evaluated + added to it. I've posted a summary here together with a link to the JCT-VC document site.

So far, indications are that HEVC will deliver around 2x better compression compared with H.264/AVC, at the expense of a big increase in computational complexity. By the time the new standard is published (around 2013), the hope is that this increased complexity will be well within the capabilities of consumer devices.

There will still be many people who question the need for a new video coding standard. That's why we are developing the OneCodec concept, so that you can deliver improved video coding performance as and when it's needed, without the high cost of switching over to a completely new standard.

Getting the best out of H.264 / talk tomorrow

2010-11-16T10:33:00.002Z

A free web seminar on H.264 & how to get the best performance. I'll look at coding options, the effect they have on compression and computation and how an H.264 codec can make "good" coding choices.

Register here to join the session:
http://www.brighttalk.com/webcast/23533
Wednesday 17th November 2010, 4.30pm GMT / 11.30am EST / 8.30am PST

Webinar, 17th November: Getting the best out of H.264

2010-10-28T08:10:00.002Z

Sign up for my next webinar, "getting the best out of H.264", a one-hour presentation on 17th Nov 2010 at 4.30pm GMT (that's 8.30am Pacific time and 11.30am East Coast time in the US). It's free to attend and you get the chance to ask questions during the talk.

Register here:
http://www.brighttalk.com/webcast/23533

- Iain

Internet Enabled Television

2010-10-01T10:18:00.002Z

What will be the next significant development in TV technology? Electronics manufacturers are investing heavily in 3DTV, hoping to capitalise on the success of 3D movies such as Avatar. However, there are indications that consumers are not in a hurry to buy 3D television sets. According to some reports, Internet Enabled Television, IETV, shows signs of being much more popular with consumers.

Internet Enabled TV makes it possible to access "watch again" services such as BBC iPlayer, as well as video sharing services such as YouTube and Vimeo, through your TV set. Also described as "Hybrid IPTV", IETV uses a home broadband connection to provide internet video services on your television.

Interesting links:
HbbTV consortium
YouView platform
"Sales of IETV to out-perform 3DTV", Digitimes, August 2010

I just want one video codec...

2010-09-30T13:31:00.000Z

New book mentioned on ITU-T web site

2010-09-29T13:58:00.002Z

A mention for my new book on H.264 / AVC on the ITU-T website:

http://www.itu.int/ITU-T/newslog/New+Book+On+Emmy+Award+Winning+ITU+Video+Codec.aspx

- Iain

In Boston for the Embedded Systems Convention

2010-09-17T17:17:00.003Z

I'm in Boston, giving a talk at the Embedded Systems Convention on Wednesday 22nd September at 4.30pm, with my colleague Maja Bystrom:

Our talk at ESC

We'll be talking about our work on Configurable Video Coding. By exploiting the increased flexibility of programmable processors, a configurable video codec can handle multiple video formats using a single coding “engine”, making efficient use of system resources and reducing the time-to-market for new systems.

I'll be at ESC on Tuesday 21st and Wednesday 22nd September.

- Iain

View my slides...

2010-08-11T17:48:00.003Z

You can view the presentation slides for two recent talks here:
An overview of H.264 video compression
Book launch: H.264 Advanced Video Compression

- Iain

Book launch tomorrow - tune in for free...

2010-06-15T19:52:00.003Z

Still time to register for tomorrow's book launch (16 June, 4pm UK time). I'll be giving a 30-minute presentation and answering any questions from viewers. Click to register:
http://www.brighttalk.com/webcast/21438

After the talk is over, you will be able to re-play it using the link.
- Iain Richardson.

Book Launch: The H.264 Advanced Video Compression Standard

2010-06-03T11:24:00.002Z

The new book has been available in the UK for a few weeks and should be available in the US and the rest of the world in June. I will launch the book via a free webcast on Wednesday 16th June 2010 at 4pm BST / 8am Pacific time / 11am East Coast US time. During the half-hour webcast I will give a brief overview of the main chapters. Participants will have the opportunity to post questions which I will answer during the talk. Click here to register for free:
http://www.brighttalk.com/webcast/21438

- Iain Richardson

Comparison of VP8 and H.264

2010-05-24T08:52:00.002Z

Google released the open-source WebM multimedia framework last week:
http://www.webmproject.org/
WebM includes the VP8 video codec.

Streamingmedia.com has published a comparison between VP8 and H.264:
http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/First-Look-H.264-and-VP8-Compared-67266.aspx

- Iain

High Efficiency Video Coding Standard

2010-05-03T16:41:00.003Z

Work is continuing on the new video coding standard, currently known as "High Efficiency Video Coding" (HEVC). A Joint Collaborative Team on Video Coding (JCT-VC) has been set up by ISO/IEC MPEG and ITU-T VCEG. Following a Call for Proposals in January, 27 proposals were submitted to the first meeting of the JCT-VC in April. Elements of some of these proposals have been combined to develop an initial Test Model, a starting point for development of the new standard. The initial Test Model has similarities to earlier standards such as H.264/AVC, including block-based intra/inter prediction, block transform and entropy coding. New features include increased prediction flexibility, more sophisticated interpolation filters, a wider range of block sizes and new entropy coding schemes. Coding performance varies across the different proposals. It looks like we might expect to see a 2x compression improvement compared with H.264/AVC (i.e. half the bitrate at the same visual quality), at the expense of a significant increase in computational complexity (perhaps 3x or more). You can find the technical proposals here.

The new book...

2010-04-22T18:31:00.001Z

Just got an advance copy...

New book...

2010-04-21T09:38:00.004Z

Just heard that I will receive some advance copies of my new book in a few days. It should be available in the UK very soon; I'll post a note here once I have a confirmed date for US and the rest of the world.
http://www.vcodex.com/h264book/index.html

- Iain

SKIP macroblocks

2010-04-07T10:04:00.004Z

Praveen asked the following question:
"I have a question regarding 'SKIP' block coding. When is a block decided to be "SKIPPED". What are the criteria and how is the SKIP macroblock coded?"

A SKIP macroblock is a macroblock (16x16 displayed pixels) for which no information is sent to the decoder - i.e. no coded coefficients, no header and no prediction information. In H.264, an encoder can choose SKIP mode for a macroblock in a P- or B-slice. The decoder estimates a motion vector for the skipped macroblock from neighbouring coded macroblocks and uses this to calculate a motion compensated prediction for the skipped MB. Since there is no residual, the motion compensated prediction is directly inserted into the decoded frame or field.

It's up to the encoder to choose whether to code or skip a MB. Typically the encoder might choose SKIP when the rate-distortion cost of SKIP mode is lower than any coded mode, i.e. when a weighted combination of bitrate and distortion (the quality loss of the decoded MB) is lower. The encoder may also "guess" that SKIP mode is suitable using other criteria such as local scene statistics, in order to save computation.

For a more detailed discussion, see e.g. http://www4.rgu.ac.uk/files/image584_november07.pdf or our other papers on mode selection.

- Iain

In the Bay area, 26-29 April

2010-03-29T12:04:00.003Z

I'll be in the SF Bay / San Jose area from 26th-29th April 2010, visiting the Embedded Systems Conference. Please get in touch if you are interested in meeting me.
- Iain Richardson

The proofs are corrected...

2010-03-24T17:46:00.002Z

I've made the final corrections to the proofs of my new book, "The H.264 Advanced Video Compression Standard". Hopefully it'll appear in May 2010. You can find some more information here. If you have any questions about the new book, please let me know.
- Iain Richardson.

This blog has moved

2010-03-23T08:59:00.001Z

This blog is now located at http://vcodex.blogspot.com/.
You will be automatically redirected in 30 seconds, or you may click here.

For feed subscribers, please update your feed subscriptions to
http://vcodex.blogspot.com/feeds/posts/default.

Yesterday's webcast

2010-03-23T08:54:00.002Z

Hundreds of viewers and tens of questions.. Thanks for watching the webcast yesterday, "An Introduction to H.264". If you missed it, you can watch the recording here.

I hope to do more webcasts, explaining H.264/AVC in more depth, perhaps starting with a "book launch" webcast.

If you liked the webcast or any of the other resources on this site, please tell me about it and/or click the "Share This" link on any of the pages on the website.

- Iain Richardson

Webcast today

2010-03-22T09:08:00.002Z

Today's webcast is an introduction to H.264/AVC. You can register for free at:
http://tinyurl.com/h264talk1 . It starts at 4pm GMT. Note that the UK is NOT yet on Summer Time, so that's 12noon on the East Coast (US), 11am in Texas, 9am on the West Coast. Hope you can join the webcast...
Iain.

Webcast on H.264/AVC

2010-03-04T11:09:00.004Z

I'll be presenting a free webcast on H.264/AVC in association with DCKTN on 22nd March 2010. I will cover the basics of H.264 technology and I'll take questions from participants.

The talk will be presented live on 22nd March at 4pm GMT, that's 11am EST / 10am CST / 8am PST.

You can register here:
http://tinyurl.com/h264talk1

Hope you can tune in....
Iain.

New book - ordering info.

2010-02-22T14:05:00.002Z

I've put up a page with some information about my new book, H.264 Advanced Video Compression:
http://www.vcodex.com/h264book/

Apart from chapters 2 and 3 (video quality and video compression concepts), the material has been completely rewritten. Chapters 4-7 cover the H.264 standard in detail and chapters 8-10 cover performance, implementation and extensions. Please get in touch if you have any questions about the book.
- Iain.

CES, Las Vegas, January 2010

2009-12-13T12:10:00.002Z

I'll be at the Consumer Electronics Show (CES) in Las Vegas on 9 and 10 January 2010, then at the ICCE conference to present a paper on our configurable video coding work. If you're at CES and would like to meet to discuss video coding technology, let me know.
- Iain Richardson

A new video coding standard by 2013?

2009-11-19T10:23:00.002Z

MPEG have set out a timetable for development of a new video coding standard, currently known as High Performance Video Coding (HVC). A formal Call for Proposals will be issued in January 2010 and registration, submission and evaluation of proposals will take place from February to April 2010. The aim will be to show significantly better coding performance than H.264/AVC. The best technical features of the proposals will be incorporated into a Test Model, a working reference codec, in late 2010. A new standard could be published in 2012/2013. The new standard is likely to deliver better compression performance for current and emerging video applications, probably at the expense of increased computational cost.

- Iain.

New book is on its way...

2009-11-14T09:28:00.003Z

I've finally finished the manuscript for my new book. It's with the publishers now (November 2009) and should be published in a few months. It's a completely revised and updated follow up to my last book, titled "H.264 Advanced Video Compression". It took longer than I planned to complete, but hopefully the wait will be worth it...
- Iain.

What's next after H.264 ?

2009-09-10T09:43:00.002Z

H.264 Advanced Video Compression (also known as MPEG-4 Part 10) is widely accepted as a leading video compression standard. Other formats such as VC-1 and AVS can (arguably) offer similar performance but H.264 is definitely a front runner in terms of coding efficiency. First standardized in 2003, H.264/AVC is now a mature technology. So what is next ?

The Moving Picture Experts Group (MPEG) and Video Coding Experts Group (VCEG) are examining the need for a new video compression standard. At the recent MPEG meeting (June 2009), several proposals for improved video compression were presented. The consensus was that (a) there is likely to be a need for a new compression format, as consumers demand higher-quality video and as processing capacity improves and (b) there is potential to deliver better performance than the current state-of-the art.

The current plan is to set up a Joint Collaborative Team (JCT) of MPEG and VCEG representatives to work on a new video coding standard. Proposals for the new standard will be reviewed in early 2010 and a new standard could be finalized around 2012/2013. It will aim to deliver significantly better compression performance than H.264/AVC, probably at a higher computational cost. Its working title ? High Performance Video Coding or Enhanced Performance Video Coding.

- Iain.

Video coding and cloud computing ?

2009-09-03T10:03:00.004Z

The announcement of OnLive's "on-demand gaming" platform has generated a lot of interest. The basic idea is that a video game runs on a central server, not on your PC or games platform. You interact with the game via a broadband connection. The key to making this work is response time - the time lag between your input reaching the server and the updated game screen reaching your display. The gameplay video is rendered on the server and so a critical question is whether a high-definition, high framerate game display can be streamed to your display in real time.

OnLive claim that this can be done using video compression; i.e. compression of the rendered scene, streaming, decompression and display in a fraction of a second. If the platform matches up to the claims, then this might be the start of a trend towards cloud computing, in which high-performance computing is done on a remote server and the results (a rendered screen in this case) are sent to your display using video compression. Are current video compression algorithms good enough to support the cloud computing model ? We'll soon find out...

The ideal video codec ?

2009-08-19T13:42:00.002Z

What would the idea video codec look like ? Here's a list of suggestions:

1. Just one video codec (instead of the ever-increasing number of competing video coding formats).

2. Delivers the best performance, i.e. a combination of:
a. maximum image quality and frame rate
b. minimum bit rate
c. minimum computational complexity or power consumption,
- the best combination of these factors for the platform (e.g. mobile, hardware, software...)

3. Easy upgrades (preferably completely transparent to the user) to take advantage of new ideas and techniques.

4. No license fees (or at least a license-free option).

5. No inter-operability problems.

6. Easy scalability (the same content on multiple platforms at multiple resolutions).

Impossible ? Maybe...

Comments very welcome !

- Iain.

Quantization - "flat" scaling factor.

2009-08-19T13:28:00.002Z

JianFeng Zheng sent me a good question about H.264 inverse transform and quantization. In the H.264 standard, there is an extra division by 16 that occurs in section 8.5.8 (in the 2007 version of the document) that I don't mention in my tutorial on the inverse transform. It's not easy to see where this factor comes from.

H.264 includes the option to send a non-uniform quantization scaling matrix ("scaling list"), which makes it possible to quantize coefficients differently depending on their position in the block. The default scaling matrix, Flat_4x4_16 (or 8x8 equivalent) is a constant value for every coefficient position. The constant value is 16 (rather than 1 as might be expected). This is cancelled out by the division by 16 (implemented as a right-shift by 4 binary places) in section 8.5.8. So it all works out neatly.

Thanks to JianFeng for the question.

- Iain.

In the Bay area

2009-07-16T10:24:00.002Z

I will be in the SF Bay area in August (10-14 Aug 09). If your company is working in the video coding space and you're keen to talk to me, let me know and I'll see if I can arrange a visit. - Iain.

H.264 quantization parameter

2009-04-04T18:23:00.002Z

Quantization in an H.264 encoder is controlled by a quantization parameter, QP, that ranges from 0 to 51. QP is an index used to derive a scaling matrix. It is possible to calculate the equivalent quantizer step size (Qstep) for each value of QP (see my tutorial on H.264 4x4 transform). As QP increases, Qstep increases; in fact, Qstep doubles for every increase of 6 in QP. The logarithmic relationship can be seen in this graph of QP (x-axis, linear) vs. Qstep (y-axis, logarithmic).

Guest lectures

2009-03-31T08:30:00.003Z

I will be giving a number of guest lectures on video coding over the next few months, in Rochester (NY), Aberdeen (UK), France and Spain. You can find out about arranging a lecture on H.264/AVC, video codec design and optimization, or emerging video coding topics, by visiting this page.

Modified transform tutorial

2009-03-04T08:22:00.001Z

I have modified the tutorial on the H.264 4x4 transform and quantization processes and corrected a minor error. The new tutorial is available here.

H.264 vs MJPEG2000 vs MJPEG

2009-01-27T12:23:00.003Z

Here's an interesting article:

http://www.isc365.com/Why_We_Like_MJPEG_Compression.aspx

It's interesting because it claims that MJPEG is better than H.264. This is a strange claim. If you want to do Intra-only coding (i.e. no motion compensation), you can use H.264 in Intra mode. Detlev Marpe demonstrated that H.264 in Intra mode has similar performance to Motion JPEG2000:
http://www.f4.fhtw-berlin.de/~barthel/paper/spie03_marpe_et_al.pdf

Note that Marpe includes actual results, unlike the article above. So, even in Intra mode, H.264 will significantly out-perform the older MJPEG standard. Using Intra and Inter coding (i.e. adding motion compensation) will increase the performance dramatically, even if there is a lot of camera movement and detail in the scene.

If in doubt about competing claims, ask for rate-distortion results and independent subjective test results. For example, see this site for some third-party codec comparisons:
http://compression.ru/video/codec_comparison/mpeg-4_avc_h264_2007_en.html

Or send me an email - I will try and answer it and give you an impartial opinion.

- Iain.

Analyzing H.264 clips

2009-01-20T13:01:00.003Z

I have put two examples of H.264 coded sequences on the downloads page. Both are short segments of the "Foreman" CIF video sequence, coded using the Baseline and Main profiles of H.264.

Use a viewer such as Elecard's Streameye to analyze the clips:

http://www.elecard.com/products/products-pc/professional/streameye/
(you can download a time-limited trial version).

Here's an screenshot of the Main Profile clip. You can see the P-slices (blue bars) and B-slices (green bars). Use the Streameye menus to examine frames, look at the distribution of macroblock types, motion vectors, etc.

What does the "H" in H.264 stand for?

2009-01-20T10:52:00.002Z

I was asked this question today and I think it's a really good question: what does the "H" stand for in H.264 ?

H.264 is an ITU standard or "Recommendation". ITU Recommendations are
assigned letters and numbers; so everything that starts with H. is in
a similar area (audiovisual/multimedia) :
http://www.itu.int/rec/T-REC-h

Other groups of standards start with different letters, always a
letter and a number. E.g. P.910 is a standard for subjective quality measurement.

ISO/IEC standards have numbers, so MPEG-2 Video is ISO/IEC 13818-2. "MPEG" is a standardisation group within ISO/IEC.

H.264 is co-published with ISO/IEC MPEG so it also has an MPEG number. Confusingly,
it's given the same basic number as MPEG-4:

MPEG-4 Visual (older standard) is ISO/IEC 14496-2 ("MPEG-4 Part 2")
H.264/Advanced Video Coding is co-published as ISO/IEC 14496-10 ("MPEG-4 Part 10")

A list of a few current standards:
http://www.citeulike.org/user/xistenshal/tag/standard

Finally, a note about the publication date of standards. The ISO/IEC or ITU website lists the date of the most recent published version of a standard. However, this can be confusing. For example, ITU-T H.263 was first published in 1995, but the most recent updated version was published in 2005 (after the first publication of H.264). Generally, updated versions will correct errors in the text and (in some cases) add extensions. For a realistic timeline of standards development, look at the original publication date. In chronological order of original publication date:

H.261
MPEG-1
MPEG-2 / H.262 (co-published)
H.263
MPEG-4 Visual
H.264 / MPEG-4 Part 10 (co-published)
VC-1

New tutorial on transform and quantization

2009-01-08T09:41:00.001Z

New white paper / tutorial posted on the derivation of the 4x4 forward + inverse transforms. Comments welcome. - Iain.

Fully Configurable Video Coding

2008-09-25T07:32:00.003Z

I gave a plenary talk at IEEE System on Chip Conference last week and used the opportunity to launch our new research topic, Fully Configurable Video Coding. The idea is to enable complete, dynamic reconfiguration of a video codec. The new concept is attracting a lot of interest from industry and researchers. Here's some more information.

Article on video compression patents

2008-03-27T14:52:00.002Z

I've posted a new article on video compression patents and IPR. The article gives an overview of the IPR position related to video coding and video compression, with particular reference to video coding standards and patent licenses.

New version of H.264/AVC overview

2007-12-03T14:52:00.001Z

I have posted a new version of my introductory overview, "What is H.264?". Visit the page and please contact me with any comments.

Computer-drawn or hand-drawn diagrams ?

2007-11-01T10:33:00.000Z

I'm developing diagrams for a new edition of my book on H.264/AVC. I've been experimenting with hand-drawn vs. computer-drawn diagrams. I quite like the immediacy of hand-drawn diagrams. Here are a few examples - I'd like to have your opinions on these. Please leave a comment (or send me an email) with your views - do you prefer hand-drawn or computer-drawn?

The examples are in this file:
Figures_H264book_poll.pdf

New overview of H.264/AVC

2007-03-29T10:27:00.000Z

I have written a new white paper that gives an overview of the H.264/AVC standard (here). It's intended to be a (relatively) non-technical introduction to the standard. The new paper describes what H.264 is, how an H.264 video codec works and why H.264 is an important industry standard. I'd be happy to receive any comments.

Topics for new tutorials

2007-03-06T10:47:00.000Z

I will be posting a number of new H.264 tutorials over the next few months. I plan to update the existing tutorials (here) and add further topics. The first will be an "easy" introduction to the H.264/AVC standard. I'd welcome requests for new tutorial topics, so please add a comment if you'd like to suggest a topic for a tutorial paper.
- Iain

Welcome

2007-02-27T09:07:00.000Z

Welcome to the newly-published Vcodex site. This site contains resources, links and information about video compression and about my work. I have kept the existing resources (including the popular H.264 tutorials) and added new material. I will continue to add resources to the site. Contact me if you'd like to suggest or request further resources, tutorial topics, etc, or to find out more about what I do.