[XviD-devel] codec performance
Bobololo
xvid-devel@xvid.org
Mon, 16 Dec 2002 00:49:13 +0100
> > For this purpose I started to write a little encoder bench tool
> > that feeds
> > the different encoders with well know 420 YUV test sequences (like
> > coastguard, foreman, silent, etc) and that measures encoding time,
> > compressed size and PSNR.
>
> sounds good, but iam pretty sure this has been done before... do a
> search on the xvid forums.
I found nothing corresponding to what I wanted.
> > \codec\decoder\decoder.c(970) : fatal error C1600: unsupported
data type
[...]
> it sounds like a bug in the REALmagic codec.
> i suggest you contact sigma designs about it.
You're right, it's a real pain, I gave up the support of this codec. I
finally restricted the codec set to 3ivX, DivX, Xvid and Amp4. You can
find below the first results I got.
Before showing the figures, just a little note about the benchmarking
procedure and other things.
First of all, this benchmark's purpose is to allow an objective
quality and performance comparison of different MPEG-4 Simple Profile
codecs. The second aim is to evaluate the performance of the MPEG-4
encoder we're developing at my company against all others.
Several well known MPEG reference test sequences are encoded with the
different codecs in variable bitrate mode (quantizer) at different
quantisation step.
These sequences are 300 CIF (352x288) frames and formatted in YUV 420
(IYUV) in order to avoid the conversion function overhead to be taken
in the results. They can be found at this URL :
http://meru.cecs.missouri.edu/free_download/videos/
The encoding time, the compressed size and the PSNR for each sequence
for each quantisation step is measured. For the PSNR, the compressed
bitstream is decoded using DivX 5.02 decoder, and the result is used
for the PSNR computation. In the results below, only Luminance (Y)
PSNR is displayed for a layout constraint. The PSNR for chrominance
channel are also computed but they are less significant (always
higher) compared to Y, so they are not shown.
The configurations set for the different codecs try to produce
roughly the same PSNR :
- 3ivx Delta 4 PR 2 : constant quality, Qp = quantisation step
It is not clearly said in 3ivx description, but their encoder is
simple profile only. There is no BVOP, no quarter pel, no
quantisation method 1, etc...
- DivX 5.0.2 : 1-pass quality-based, quantizer = quantisation step,
performance/quality = slowest, GOP = 300.
- Xvid nic's build 251102 : 1-pass quantizer, motion search
precision very high, quantisation h263, GOP = 300
- Amp4 : standard simple profile compliant config (quantisation h263,
GOP = 300 ...). This is the encoder we're currently developing at my
company. I can't disclose much about it. All I can say is that is designed
for speed and low bitrates and it primarily runs on embedded platforms
(guess which ones ;). The version tested here is a x86/win/linux port that
we use as reference encoder for evaluation. And I guess it will never be
publicly available.
As you can see, Xvid is always the best in the ratio psnr/bitrate. This
is great ! Btw DivX's trick to increase the psnr at the expense of
bitrate is pitiful.
Bobololo.
ps: BTW, I can provide the source code of the bench tool if anyone is
interested (I need to clean it up a bit before, it's too ugly for
now ;)
********************
VfWBench 1.0 - Bobololo & Mel - 12/2002
Benchmark of the encoder, Divx 5.02 used as the decoder
Test system :
Windows 2000 - Service Pack 3
AMD Athlon(tm) Processor (x86 Family 6 Model 4 Stepping 2) at 1364MHz
Total physical memory 511Mo - Available physical memory 284Mo
Benchmark made on Mon Dec 16 00:12:06 2002
********************
Sequence : p:/algo/mpeg4/stream/ref/akiyo_cif.yuv (352x288)
Benchmarking 3ivx Delta 4 PR2 Codec (3ivxVfWCodec.dll)
q 3 - size 481 kb - encoding speed 103 fps - psnr Y 43.21 dB
q 5 - size 233 kb - encoding speed 105 fps - psnr Y 40.42 dB
q 9 - size 104 kb - encoding speed 103 fps - psnr Y 37.18 dB
q 15 - size 55 kb - encoding speed 102 fps - psnr Y 34.51 dB
Benchmarking XviD MPEG4 Codec (Nic's build 25/11/02) (xvid.dll)
q 3 - size 457 kb - encoding speed 146 fps - psnr Y 43.60 dB
q 5 - size 226 kb - encoding speed 145 fps - psnr Y 40.81 dB
q 9 - size 104 kb - encoding speed 148 fps - psnr Y 37.51 dB
q 15 - size 58 kb - encoding speed 145 fps - psnr Y 34.64 dB
Benchmarking DivX 5.02 Codec (divx.dll)
q 3 - size 474 kb - encoding speed 158 fps - psnr Y 43.65 dB
q 5 - size 266 kb - encoding speed 160 fps - psnr Y 41.27 dB
q 9 - size 131 kb - encoding speed 159 fps - psnr Y 37.82 dB
q 15 - size 97 kb - encoding speed 160 fps - psnr Y 34.76 dB
Benchmarking AMP4 PC Reference Codec (amp4.dll)
q 3 - size 454 kb - encoding speed 331 fps - psnr Y 43.57 dB
q 5 - size 218 kb - encoding speed 361 fps - psnr Y 40.75 dB
q 9 - size 95 kb - encoding speed 393 fps - psnr Y 37.34 dB
q 15 - size 49 kb - encoding speed 411 fps - psnr Y 34.41 dB
Sequence : p:/algo/mpeg4/stream/ref/coastguard_cif.yuv (352x288)
Benchmarking 3ivx Delta 4 PR2 Codec (3ivxVfWCodec.dll)
q 3 - size 3837 kb - encoding speed 75 fps - psnr Y 39.92 dB
q 5 - size 1973 kb - encoding speed 79 fps - psnr Y 35.92 dB
q 9 - size 852 kb - encoding speed 84 fps - psnr Y 32.02 dB
q 15 - size 409 kb - encoding speed 88 fps - psnr Y 29.21 dB
Benchmarking XviD MPEG4 Codec (Nic's build 25/11/02) (xvid.dll)
q 3 - size 3884 kb - encoding speed 108 fps - psnr Y 39.95 dB
q 5 - size 2044 kb - encoding speed 113 fps - psnr Y 36.02 dB
q 9 - size 920 kb - encoding speed 118 fps - psnr Y 32.16 dB
q 15 - size 462 kb - encoding speed 122 fps - psnr Y 29.22 dB
Benchmarking DivX 5.02 Codec (divx.dll)
q 3 - size 3997 kb - encoding speed 106 fps - psnr Y 39.99 dB
q 5 - size 2293 kb - encoding speed 114 fps - psnr Y 36.61 dB
q 9 - size 991 kb - encoding speed 125 fps - psnr Y 32.51 dB
q 15 - size 482 kb - encoding speed 133 fps - psnr Y 29.66 dB
Benchmarking AMP4 PC Reference Codec (amp4.dll)
q 3 - size 4035 kb - encoding speed 189 fps - psnr Y 39.97 dB
q 5 - size 2093 kb - encoding speed 216 fps - psnr Y 35.96 dB
q 9 - size 904 kb - encoding speed 246 fps - psnr Y 31.98 dB
q 15 - size 405 kb - encoding speed 271 fps - psnr Y 28.98 dB
Sequence : p:/algo/mpeg4/stream/ref/foreman_cif.yuv (352x288)
Benchmarking 3ivx Delta 4 PR2 Codec (3ivxVfWCodec.dll)
q 3 - size 2594 kb - encoding speed 78 fps - psnr Y 40.42 dB
q 5 - size 1223 kb - encoding speed 83 fps - psnr Y 36.89 dB
q 9 - size 516 kb - encoding speed 88 fps - psnr Y 33.45 dB
q 15 - size 278 kb - encoding speed 90 fps - psnr Y 30.99 dB
Benchmarking XviD MPEG4 Codec (Nic's build 25/11/02) (xvid.dll)
q 3 - size 2634 kb - encoding speed 109 fps - psnr Y 40.54 dB
q 5 - size 1268 kb - encoding speed 116 fps - psnr Y 37.02 dB
q 9 - size 549 kb - encoding speed 120 fps - psnr Y 33.55 dB
q 15 - size 292 kb - encoding speed 123 fps - psnr Y 30.81 dB
Benchmarking DivX 5.02 Codec (divx.dll)
q 3 - size 2710 kb - encoding speed 106 fps - psnr Y 40.60 dB
q 5 - size 1459 kb - encoding speed 116 fps - psnr Y 37.57 dB
q 9 - size 606 kb - encoding speed 125 fps - psnr Y 33.84 dB
q 15 - size 316 kb - encoding speed 130 fps - psnr Y 31.18 dB
Benchmarking AMP4 PC Reference Codec (amp4.dll)
q 3 - size 2772 kb - encoding speed 196 fps - psnr Y 40.51 dB
q 5 - size 1325 kb - encoding speed 226 fps - psnr Y 36.93 dB
q 9 - size 551 kb - encoding speed 254 fps - psnr Y 33.37 dB
q 15 - size 262 kb - encoding speed 281 fps - psnr Y 30.63 dB
Sequence : p:/algo/mpeg4/stream/ref/hall_cif.yuv (352x288)
Benchmarking 3ivx Delta 4 PR2 Codec (3ivxVfWCodec.dll)
q 3 - size 2320 kb - encoding speed 76 fps - psnr Y 40.39 dB
q 5 - size 749 kb - encoding speed 88 fps - psnr Y 38.19 dB
q 9 - size 217 kb - encoding speed 98 fps - psnr Y 35.43 dB
q 15 - size 96 kb - encoding speed 99 fps - psnr Y 32.66 dB
Benchmarking XviD MPEG4 Codec (Nic's build 25/11/02) (xvid.dll)
q 3 - size 2136 kb - encoding speed 114 fps - psnr Y 41.07 dB
q 5 - size 748 kb - encoding speed 129 fps - psnr Y 38.49 dB
q 9 - size 234 kb - encoding speed 141 fps - psnr Y 35.61 dB
q 15 - size 108 kb - encoding speed 144 fps - psnr Y 32.69 dB
Benchmarking DivX 5.02 Codec (divx.dll)
q 3 - size 2205 kb - encoding speed 114 fps - psnr Y 41.11 dB
q 5 - size 968 kb - encoding speed 133 fps - psnr Y 38.87 dB
q 9 - size 282 kb - encoding speed 155 fps - psnr Y 35.91 dB
q 15 - size 140 kb - encoding speed 159 fps - psnr Y 33.15 dB
Benchmarking AMP4 PC Reference Codec (amp4.dll)
q 3 - size 2104 kb - encoding speed 198 fps - psnr Y 41.06 dB
q 5 - size 711 kb - encoding speed 241 fps - psnr Y 38.49 dB
q 9 - size 212 kb - encoding speed 336 fps - psnr Y 35.59 dB
q 15 - size 94 kb - encoding speed 390 fps - psnr Y 32.70 dB
---- End of Measures