x264 Settings of the mp4 movies in the Session Gallery

Message boards : Server backend and mirrors : x264 Settings of the mp4 movies in the Session Gallery
Message board moderation

To post messages, you must log in.

AuthorMessage
Professor Desty Nova
Avatar

Send message
Joined: 21 Mar 05
Posts: 97
Credit: 294,357
RAC: 3
Message 10426 - Posted: 6 Apr 2010, 10:25:30 UTC

I don't know how the x264 settings were chosen, but I think some settings could be improved.

1. Use at maximum Level 4.1 settings for good playability/compatibility in hardware:
This means --ref 4 for 1080p, --ref 9 for 720p. You can also calculate the maximum --ref allowed for a given resolution in Level 4.1 using this formula adapted from the h264 specification --ref = Truncate(8388608/(width * height)) [PS: Truncate is there to make the following examples true: division gave 4.165 -> 4 reference frames; division gave 9.856 -> 9 reference frames)

2. Motion estimation method:
Even the x264 developers think using more than --me umh is overkill unless you have time to waste. That's why --me tesa is only used in the "placebo" --preset. And if using more than umh they recommned using tesa over esa.

3.Subpixel motion estimation:
--subme 9 at least. You could also try --subme 10, that is 3-5% more efficient than --subme 9. It requires Trellis 2, so it's slower, but you could try using --subme 10 / --trellis 2 with --me umh instead of --me esa.

4. Bframes:
Since you are not using an insane number of Bframes, you could try using --b-adapt 2 to increase the quality even more.

5. --tune film
For CGI/3D animation, x264 developers recommend trying --tune film.



Professor Desty Nova
Researching Karma the Hard way
ID: 10426 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Fischer-Kerli
Project donor

Send message
Joined: 24 Mar 05
Posts: 70
Credit: 78,553
RAC: 0
Message 10429 - Posted: 6 Apr 2010, 11:38:23 UTC

I don't know if this is related to the improvable settings the Prof. mentioned.

I downloaded quite a few session videos from the old Alpha site. When I watch one of the old files that were encoded with FMP4, for instance s000919_movie_mpeg4_q96.avi (HQ), using VLC, everything is fine. Playback of s919hi.mp4 with the same resolution (but encoded with avc1 according to VLC) from the new site, however, stutters markedly on my 2006 system (machine specs here), whether BOINC is paused or not. As the size of the new file is only half as big as the one of the old file, I suppose either the CPU or the GPU has more work to do with the new codec. I should add that my graphics chip is far from being state-of-the-art (SiS761GX onboard with 64MB shared memory).

Have I become a victim of technological evolution, or is something else not right? Would increasing the shared graphics memory help? (Can't test this right now because BURP is running so I can't reboot the machine without losing work.)
ID: 10429 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Professor Desty Nova
Avatar

Send message
Joined: 21 Mar 05
Posts: 97
Credit: 294,357
RAC: 3
Message 10432 - Posted: 6 Apr 2010, 13:32:00 UTC - in response to Message 10429.  
Last modified: 6 Apr 2010, 13:36:44 UTC

I don't know if this is related to the improvable settings the Prof. mentioned.

I downloaded quite a few session videos from the old Alpha site. When I watch one of the old files that were encoded with FMP4, for instance s000919_movie_mpeg4_q96.avi (HQ), using VLC, everything is fine. Playback of s919hi.mp4 with the same resolution (but encoded with avc1 according to VLC) from the new site, however, stutters markedly on my 2006 system (machine specs here), whether BOINC is paused or not. As the size of the new file is only half as big as the one of the old file, I suppose either the CPU or the GPU has more work to do with the new codec. I should add that my graphics chip is far from being state-of-the-art (SiS761GX onboard with 64MB shared memory).

Have I become a victim of technological evolution, or is something else not right? Would increasing the shared graphics memory help? (Can't test this right now because BURP is running so I can't reboot the machine without losing work.)


The cause is the 1080p resolution video with 15 Reference Frames (AVC High Profile@Level 5.1). MPEG4-AVC/H264 needs more CPU power than the older MPEG4-ASP/XVID/etc., and one of the features that is more CPU hungry is Reference Frames. If the video had only 4 reference frames (Level 4.1) it would probably play fine on your dual core.
Up to 1080p, AVC Level 4.1 is enough. Only if you go beyond 1080p resolution do you need AVC at Level 5.1.
Actually, the developers of x264 say that for most material, 4 reference frames is enough.


Professor Desty Nova
Researching Karma the Hard way
ID: 10432 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Fischer-Kerli
Project donor

Send message
Joined: 24 Mar 05
Posts: 70
Credit: 78,553
RAC: 0
Message 10433 - Posted: 6 Apr 2010, 15:12:11 UTC - in response to Message 10432.  

The cause is the 1080p resolution video with 15 Reference Frames (AVC High Profile@Level 5.1). MPEG4-AVC/H264 needs more CPU power than the older MPEG4-ASP/XVID/etc., and one of the features that is more CPU hungry is Reference Frames. If the video had only 4 reference frames (Level 4.1) it would probably play fine on your dual core.
Up to 1080p, AVC Level 4.1 is enough. Only if you go beyond 1080p resolution do you need AVC at Level 5.1.
Actually, the developers of x264 say that for most material, 4 reference frames is enough.


Thanks for your reply! With that explanation, I definitely second the proposal of going down to 4 reference frames from the original post. My computer isn't new, but it's not ancient - it has an Athlon 3800+ dual core CPU, not a Pentium III, and it runs BURP reasonably fast, so it should be enabled to play the movies it has helped to render ...
ID: 10433 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 10440 - Posted: 6 Apr 2010, 16:53:34 UTC - in response to Message 10426.  

I don't know how the x264 settings were chosen

Based on the idea that "We have all the time in the world and these files are going to stay here forever".

but I think some settings could be improved.

Improvements are always welcome.

1. Use at maximum Level 4.1 settings for good playability/compatibility in hardware

Even up to 16 can be helpful for animated content, video game capture, CGI, and other similar content, however, I agree with you that it probably makes more sense to stay just within the level 4.1 specifications (Playstation, XBox360 etc. guarantee this for Blu-Ray anyways) for content playable on HD TVs.

Thanks for the formula for computing the maximal ref based on framesize. I guess the final formula could be something like:
ref = max(4, floor(8388608/(width * height)))
This way we stay within the specs up to 1080p and go above that for higher resolution material.

2. Motion estimation method

We have time to waste. The movies are encoded on a small expandable cluster with plenty of CPU power to spare. We are trying to get as much quality per bit as possible - regardless of the cost.

Do you have any links to the difference between the exhaustive search and the transformed exhaustive search? Why do x264 devs favour tesa over esa when going above umh?

3.Subpixel motion estimation

subme=10 was a relatively recent addition when I went through the options the first time. I guess it has become stable by now. I wonder what the performance hit will be with the trellis quantizer on everything (trellis=2) and exhaustive search... probably enough to switch to umh for 1080p+ content at least.

4. Bframes:

Absolutely

5. --tune film

Good idea, but I think we'll let it stay un-tuned for now. The reason is that there is no knowing what the output will really be like. Some of the sessions here have grain, some don't, some have cartoon-like features with rapid movement where others have gradients and slow movement, some have static backgrounds, some don't etc. etc.

I've added your suggestion to the todo-list. It won't make it into this week's changes but it will certainly be included soon.
ID: 10440 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Professor Desty Nova
Avatar

Send message
Joined: 21 Mar 05
Posts: 97
Credit: 294,357
RAC: 3
Message 10449 - Posted: 7 Apr 2010, 8:01:48 UTC - in response to Message 10433.  

The cause is the 1080p resolution video with 15 Reference Frames (AVC High Profile@Level 5.1). MPEG4-AVC/H264 needs more CPU power than the older MPEG4-ASP/XVID/etc., and one of the features that is more CPU hungry is Reference Frames. If the video had only 4 reference frames (Level 4.1) it would probably play fine on your dual core.
Up to 1080p, AVC Level 4.1 is enough. Only if you go beyond 1080p resolution do you need AVC at Level 5.1.
Actually, the developers of x264 say that for most material, 4 reference frames is enough.


Thanks for your reply! With that explanation, I definitely second the proposal of going down to 4 reference frames from the original post. My computer isn't new, but it's not ancient - it has an Athlon 3800+ dual core CPU, not a Pentium III, and it runs BURP reasonably fast, so it should be enabled to play the movies it has helped to render ...


I have to retract a bit of what I said. I should have gone and reread some of the stuff before posting here. After that, from what I reread in threads with input from x264 developers in the doom9 forums, it seems increasing reference frames doesn't cost much CPU power, it only really matters for compatibility with hardware players. The most CPU hungry is deblocking, CABAC, weighted prediction. But any of these are important for visual quality or efficiency, so no point in disabling them. Bitrate/Peak bitrate can also matter. As the resolution of a video goes up, you need more bitrate, and more bitrate needs more CPU power.
So in your case, your computer might not be ably to decode a higher bitrate AVC file. I don't know if in this case increasing the memory of your video card might help, but you can try it. Also I just tried running the file from session 919 you talked about in VLC, and it's unplayable in my computer (an Athlon XP 3000), but it plays (slow) in MPC-HC (media player classic-home cinema). So you could try downloading the stable version of MPC-HC and give it a try. It's decoder is multithreaded, so your your computer should play it better than in mine.


Professor Desty Nova
Researching Karma the Hard way
ID: 10449 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Professor Desty Nova
Avatar

Send message
Joined: 21 Mar 05
Posts: 97
Credit: 294,357
RAC: 3
Message 10450 - Posted: 7 Apr 2010, 9:50:29 UTC - in response to Message 10440.  
Last modified: 7 Apr 2010, 9:53:01 UTC


1. Use at maximum Level 4.1 settings for good playability/compatibility in hardware

Even up to 16 can be helpful for animated content, video game capture, CGI, and other similar content, however, I agree with you that it probably makes more sense to stay just within the level 4.1 specifications (Playstation, XBox360 etc. guarantee this for Blu-Ray anyways) for content playable on HD TVs.

Thanks for the formula for computing the maximal ref based on framesize. I guess the final formula could be something like:
ref = max(4, floor(8388608/(width * height)))
This way we stay within the specs up to 1080p and go above that for higher resolution material.

I know about the animation/etc. benefiting more from higher ref frames, that's why I said "most material" later. It's really balancing cost/benefit. If, for example, 90% of reference frames used are up to 4, and you set 16, do you really need the rest? I guess it will depend on what you want to do with the video. (x264 outputs ref frames statistics in "Ref P" and following lines)

2. Motion estimation method

We have time to waste. The movies are encoded on a small expandable cluster with plenty of CPU power to spare. We are trying to get as much quality per bit as possible - regardless of the cost.

Do you have any links to the difference between the exhaustive search and the transformed exhaustive search? Why do x264 devs favour tesa over esa when going above umh?

I tried finding the thread that had more comparison of TESA/ESA in terms of quality/compression versus time, but couldn't find it (it must have been in 2008...). The only one I found was the one talking about the commit of TESA, where there are some graphs of compression versus time (with different --merange) for the different motion estimation methods. The thread is http://forum.doom9.org/showthread.php?t=134179. As with other features of x264, the gain of TESA over ESA depends on the source.

3.Subpixel motion estimation

subme=10 was a relatively recent addition when I went through the options the first time. I guess it has become stable by now. I wonder what the performance hit will be with the trellis quantizer on everything (trellis=2) and exhaustive search... probably enough to switch to umh for 1080p+ content at least.

I have to correct myself here a bit. The gain from subme 9 to 10 is 1-2% depending on the encoded source. I guess I suffered from memory inflation :-P

4. Bframes:

Absolutely

With the new --b-adapt 2 you could also try using more bframes. The only downside is that it's really slooow with 16 bframes (but it should more efficiently use them than the older method in --b-adapt 1).

5. --tune film

Good idea, but I think we'll let it stay un-tuned for now. The reason is that there is no knowing what the output will really be like. Some of the sessions here have grain, some don't, some have cartoon-like features with rapid movement where others have gradients and slow movement, some have static backgrounds, some don't etc. etc.

I've added your suggestion to the todo-list. It won't make it into this week's changes but it will certainly be included soon.


You are right. With so much diversity in content, it's better sticking in this case to the defaults.

There is also one last setting that recently is on by default. Weighted prediction for Pframes (--weightp). It makes x264 now much nicer to fades. But there might be still some bugs on it. There has been some people complaining about it to the developers (even getting them angry by not giving samples). It also seems there are decoders that have bugs in decoding weighted Pframes, most notably pre 2.0 versions of CoreAVC, and some portable hardware players.

Finally, if space was not an issue (because of unpredictable size), everybody had quad cores and compatible decoders (since it seems x264 is the first to implement this feature of the specification), for supreme quality, BURP could use lossless AVC encoding with x264 . ;-)


Professor Desty Nova
Researching Karma the Hard way
ID: 10450 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Fischer-Kerli
Project donor

Send message
Joined: 24 Mar 05
Posts: 70
Credit: 78,553
RAC: 0
Message 10451 - Posted: 7 Apr 2010, 11:39:58 UTC - in response to Message 10449.  

Also I just tried running the file from session 919 you talked about in VLC, and it's unplayable in my computer (an Athlon XP 3000), but it plays (slow) in MPC-HC (media player classic-home cinema). So you could try downloading the stable version of MPC-HC and give it a try. It's decoder is multithreaded, so your your computer should play it better than in mine.


That's a nice tip, thank you! I guess I was relying too much on VLC being the "perfect" media player ... Most of the files I've tried play well in MPC-HC, only session 929 still has problems. I can't compare s929hi.mp4 to the FMP4 avi from the same session because I don't have it (if it ever existed), so maybe the problem is independent from the encoder. Personally, I'd like to see the old FMP4 kept as an additional download option, but I guess that won't happen.

ID: 10451 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Server backend and mirrors : x264 Settings of the mp4 movies in the Session Gallery