[XviD-devel] quant functions b0rked

[Edouard Gomez]

Radek Czyz (syskin at ihug.com.au) wrote:
> Hi everyone,
> 
> I have some bad news.
> 
> Two asm functions - quant4_intra_xmm and quant_intra_3dne are broken.
> When quantizing DC coefficient and this coefficient is negative, their
> result is decreased by one.

True, i tested them and that's here since ages...

I found the error in the 3dne function, it's caused by w wrong
optimization in the (DC + sign(DC)*dscaler/2) intermediate result is
computed:

The errors comes from the asumption (sign(DC) = 0 if DC>0, 1 otherwise)
sign(DC)*dcscaler/2 == sign(DC) xor dcscaler/2.

At least that's what i've found.

-- 
Edouard Gomez