[XviD-devel] big trellis bug

[skal]

	Hi all,

On Mon, 2003-11-24 at 21:10, Edouard Gomez wrote:
> skal (skal at planet-d.net) wrote:
> > 	i've sent GomGom a new code that handles this big
> > 	levels separately... should hopefully be ok to fix 
> > 	the overflow problems...
> 
> Yes i'm testing it right now, two notes tho:
>  - the red block is still here, no panic ! it's part of the source, but
>    it's the first time i noticed it, probably because i was looking for
>    similar errors ;-)
>  - not sure but it seems to me that kind of optimization:
>        if ((uint32_t)signed_value<threshold) which is a trick for
>        if (signed_value>=0 && signed_value<threshold)
>    was miscompiled by msvc for the GMC code (remember ?) perhaps the
>    chroma bug in the Nemo rip was just caused by this type of msvc
>    wrong code generation ?

	Note: i've reversed the test operands, bearing this msvc
	bug in mind. It's now:

	if ( ((uint32_t)(2*T+1)) > (uint32_t)(val+T) ) {
		... /*we're with val in [-T,T] */
        }

	to-be-tested, with an obvious fallback to double-test...

> 
> I think the big level (non) optimization is good anyway, so we'll have
> two (three in fact) zones:
>  - -26<=level<25 (with a specialized loop for {-1, 0, +1})
>  - and all other values

	Note: Unless i made a bug, mid range is [-25,25], because
	level=+-25 is the limit above which levels starts being
	definitively coded esc3 (31b) with not a single chance for
	|level|-1 to be an esc2 candidate (level=24 is the limit).


	bye!
Skal