CES, Las Vegas, January 2010

I'll be at the Consumer Electronics Show (CES) in Las Vegas on 9 and 10 January 2010, then at the ICCE conference to present a paper on our configurable video coding work. If you're at CES and would like to meet to discuss video coding technology, let me know.
- Iain Richardson

A new video coding standard by 2013?

MPEG have set out a timetable for development of a new video coding standard, currently known as High Performance Video Coding (HVC). A formal Call for Proposals will be issued in January 2010 and registration, submission and evaluation of proposals will take place from February to April 2010. The aim will be to show significantly better coding performance than H.264/AVC. The best technical features of the proposals will be incorporated into a Test Model, a working reference codec, in late 2010. A new standard could be published in 2012/2013. The new standard is likely to deliver better compression performance for current and emerging video applications, probably at the expense of increased computational cost.

- Iain.

New book is on its way...

I've finally finished the manuscript for my new book. It's with the publishers now (November 2009) and should be published in a few months. It's a completely revised and updated follow up to my last book, titled "H.264 Advanced Video Compression". It took longer than I planned to complete, but hopefully the wait will be worth it...
- Iain.

What's next after H.264 ?

H.264 Advanced Video Compression (also known as MPEG-4 Part 10) is widely accepted as a leading video compression standard. Other formats such as VC-1 and AVS can (arguably) offer similar performance but H.264 is definitely a front runner in terms of coding efficiency. First standardized in 2003, H.264/AVC is now a mature technology. So what is next ?

The Moving Picture Experts Group (MPEG) and Video Coding Experts Group (VCEG) are examining the need for a new video compression standard. At the recent MPEG meeting (June 2009), several proposals for improved video compression were presented. The consensus was that (a) there is likely to be a need for a new compression format, as consumers demand higher-quality video and as processing capacity improves and (b) there is potential to deliver better performance than the current state-of-the art.

The current plan is to set up a Joint Collaborative Team (JCT) of MPEG and VCEG representatives to work on a new video coding standard. Proposals for the new standard will be reviewed in early 2010 and a new standard could be finalized around 2012/2013. It will aim to deliver significantly better compression performance than H.264/AVC, probably at a higher computational cost. Its working title ? High Performance Video Coding or Enhanced Performance Video Coding.

- Iain.

Video coding and cloud computing ?

The announcement of OnLive's "on-demand gaming" platform has generated a lot of interest. The basic idea is that a video game runs on a central server, not on your PC or games platform. You interact with the game via a broadband connection. The key to making this work is response time - the time lag between your input reaching the server and the updated game screen reaching your display. The gameplay video is rendered on the server and so a critical question is whether a high-definition, high framerate game display can be streamed to your display in real time.

OnLive claim that this can be done using video compression; i.e. compression of the rendered scene, streaming, decompression and display in a fraction of a second. If the platform matches up to the claims, then this might be the start of a trend towards cloud computing, in which high-performance computing is done on a remote server and the results (a rendered screen in this case) are sent to your display using video compression. Are current video compression algorithms good enough to support the cloud computing model ? We'll soon find out...

The ideal video codec ?

What would the idea video codec look like ? Here's a list of suggestions:

1. Just one video codec (instead of the ever-increasing number of competing video coding formats).

2. Delivers the best performance, i.e. a combination of:
a. maximum image quality and frame rate
b. minimum bit rate
c. minimum computational complexity or power consumption,
- the best combination of these factors for the platform (e.g. mobile, hardware, software...)

3. Easy upgrades (preferably completely transparent to the user) to take advantage of new ideas and techniques.

4. No license fees (or at least a license-free option).

5. No inter-operability problems.

6. Easy scalability (the same content on multiple platforms at multiple resolutions).

Impossible ? Maybe...

Comments very welcome !

- Iain.

Quantization - "flat" scaling factor.

JianFeng Zheng sent me a good question about H.264 inverse transform and quantization. In the H.264 standard, there is an extra division by 16 that occurs in section 8.5.8 (in the 2007 version of the document) that I don't mention in my tutorial on the inverse transform. It's not easy to see where this factor comes from.

H.264 includes the option to send a non-uniform quantization scaling matrix ("scaling list"), which makes it possible to quantize coefficients differently depending on their position in the block. The default scaling matrix, Flat_4x4_16 (or 8x8 equivalent) is a constant value for every coefficient position. The constant value is 16 (rather than 1 as might be expected). This is cancelled out by the division by 16 (implemented as a right-shift by 4 binary places) in section 8.5.8. So it all works out neatly.

Thanks to JianFeng for the question.

- Iain.

In the Bay area

I will be in the SF Bay area in August (10-14 Aug 09). If your company is working in the video coding space and you're keen to talk to me, let me know and I'll see if I can arrange a visit. - Iain.

H.264 quantization parameter

Quantization in an H.264 encoder is controlled by a quantization parameter, QP, that ranges from 0 to 51. QP is an index used to derive a scaling matrix. It is possible to calculate the equivalent quantizer step size (Qstep) for each value of QP (see my tutorial on H.264 4x4 transform). As QP increases, Qstep increases; in fact, Qstep doubles for every increase of 6 in QP. The logarithmic relationship can be seen in this graph of QP (x-axis, linear) vs. Qstep (y-axis, logarithmic).

Guest lectures

I will be giving a number of guest lectures on video coding over the next few months, in Rochester (NY), Aberdeen (UK), France and Spain. You can find out about arranging a lecture on H.264/AVC, video codec design and optimization, or emerging video coding topics, by visiting this page.

Modified transform tutorial

I have modified the tutorial on the H.264 4x4 transform and quantization processes and corrected a minor error. The new tutorial is available here.

H.264 vs MJPEG2000 vs MJPEG

Here's an interesting article:

http://www.isc365.com/Why_We_Like_MJPEG_Compression.aspx

It's interesting because it claims that MJPEG is better than H.264. This is a strange claim. If you want to do Intra-only coding (i.e. no motion compensation), you can use H.264 in Intra mode. Detlev Marpe demonstrated that H.264 in Intra mode has similar performance to Motion JPEG2000:
http://www.f4.fhtw-berlin.de/~barthel/paper/spie03_marpe_et_al.pdf

Note that Marpe includes actual results, unlike the article above. So, even in Intra mode, H.264 will significantly out-perform the older MJPEG standard. Using Intra and Inter coding (i.e. adding motion compensation) will increase the performance dramatically, even if there is a lot of camera movement and detail in the scene.

If in doubt about competing claims, ask for rate-distortion results and independent subjective test results. For example, see this site for some third-party codec comparisons:
http://compression.ru/video/codec_comparison/mpeg-4_avc_h264_2007_en.html

Or send me an email - I will try and answer it and give you an impartial opinion.

- Iain.

Analyzing H.264 clips

I have put two examples of H.264 coded sequences on the downloads page. Both are short segments of the "Foreman" CIF video sequence, coded using the Baseline and Main profiles of H.264.

Use a viewer such as Elecard's Streameye to analyze the clips:

http://www.elecard.com/products/products-pc/professional/streameye/
(you can download a time-limited trial version).

Here's an screenshot of the Main Profile clip. You can see the P-slices (blue bars) and B-slices (green bars). Use the Streameye menus to examine frames, look at the distribution of macroblock types, motion vectors, etc.

What does the "H" in H.264 stand for?

I was asked this question today and I think it's a really good question: what does the "H" stand for in H.264 ?

H.264 is an ITU standard or "Recommendation". ITU Recommendations are
assigned letters and numbers; so everything that starts with H. is in
a similar area (audiovisual/multimedia) :
http://www.itu.int/rec/T-REC-h

Other groups of standards start with different letters, always a
letter and a number. E.g. P.910 is a standard for subjective quality measurement.

ISO/IEC standards have numbers, so MPEG-2 Video is ISO/IEC 13818-2. "MPEG" is a standardisation group within ISO/IEC.

H.264 is co-published with ISO/IEC MPEG so it also has an MPEG number. Confusingly,
it's given the same basic number as MPEG-4:

• MPEG-4 Visual (older standard) is ISO/IEC 14496-2 ("MPEG-4 Part 2")
• H.264/Advanced Video Coding is co-published as ISO/IEC 14496-10 ("MPEG-4 Part 10")

A list of a few current standards:
http://www.citeulike.org/user/xistenshal/tag/standard

Finally, a note about the publication date of standards. The ISO/IEC or ITU website lists the date of the most recent published version of a standard. However, this can be confusing. For example, ITU-T H.263 was first published in 1995, but the most recent updated version was published in 2005 (after the first publication of H.264). Generally, updated versions will correct errors in the text and (in some cases) add extensions. For a realistic timeline of standards development, look at the original publication date. In chronological order of original publication date:

• H.261
• MPEG-1
• MPEG-2 / H.262 (co-published)
• H.263
• MPEG-4 Visual
• H.264 / MPEG-4 Part 10 (co-published)
• VC-1

New tutorial on transform and quantization

New white paper / tutorial posted on the derivation of the 4x4 forward + inverse transforms. Comments welcome. - Iain.