[XviD-devel] [BUG]

[Michel LESPINASSE]

Hi Skal :)

On Wed, Oct 29, 2003 at 09:09:30PM +0100, skal wrote:
> IEEE-1180 specifies an acceptable range of errors when you follow
> > the following process:
> > 1) take a random 8x8 matrix with coefficients in range [-R,R[
> > 2) perform double-precision "exact" fdct on this matrix
    2b) quantify fdct output to be integers, saturate to -2048;+2047 range
> > 3) Apply the Idct you want to test to the result of 2)
> > 4) Compare result of 3) with the original random matrix of 1)
> > 5) repeat 10000 times
> 
> 	In fact the test is not that one. Instead, you
> 	must compare, at step 3), the result of the reference
> 	idct (double-precision) against your Idct. And not
> 	compare with the original matrix (which is an interesting
> 	test, but not the IEEE one).

Step 2b is important there. Because of this quantize-and-sature step,
the full precision idct result is not identical to the original matrix -
and if saturation occured, the difference could be large too.

In a real mpeg encoder, step 2b is even worse: you quantify not just
to the nearest integer, but to the closest multiple of a given
quantization factor, which might potentially be large.

So even though we have a guarantee that the original fdct input is
within -255;+255 range, we still don't have any guarantee that the
full precision idct output will be in that range too. That's why the
IEEE people specify a test range of +- 300; the mpeg people on the
other hand, do not consider this sufficient and require an additional
test with a range of -384;+383 and one million iterations. There is
also an mpeg1 test stream, known as compcore/ccm1, which has been
purposefully created with checker like patterns that tend to create
internal overflow in IDCT routines, and large quantization factors. In
that stream the fdct input *was* within -255;+255 range, however the
IDCT output goes well out of the -300;+300 range (stays within
-384;+383 though).

Writing an IDCT routine that passes these tests and uses only 16 bits
of internal precision is challenging, but it's doable. Come on skal !
We know you can do it :)

PS skal: hmmm, maybe I should ask if you have an ultrasparc VIS
routine handy ? :)

Cheers,

-- 
Michel "Walken" LESPINASSE
"In this time of war against Osama bin Laden and the oppressive
Taliban regime, we are thankful that OUR leader isn't the spoiled son
of a powerful politician from a wealthy oil family who is supported by
religious fundamentalists, operates through clandestine organizations,
has no respect for the democratic electoral process, bombs innocents,
and uses war to deny people their civil liberties." --The Boondocks