[XviD-devel] big trellis bug

[skal]

	Hi all,

On Sat, 2003-11-22 at 08:34, Radek Czyz wrote:
> Hi everyone,
> 
> I spent this day trying to catch an ugly bug. In the end, I got the
> following debug output from mbtransquant, showing the processing of
> one (chroma) block:
> 
> [912] AFTER QUANT
> [912] -181, 0, 1, 0, 0, 0, 0, 0
> [912] -3, 2, 0, 0, 0, 0, 0, 0
> [912] 1, 0, 0, 0, 0, 0, 0, 0
> [912] -1, 0, 0, 0, 0, 0, 0, 0
> [912] 1, 0, 0, 0, 0, 0, 0, 0
> [912] 0, 0, 0, 0, 0, 0, 0, 0
> [912] 0, 0, 0, 0, 0, 0, 0, 0
> [912] 0, 0, 0, 0, 0, 0, 0, 0
> 
> [912] AFER TRELLIS QUANT
> [912] 0, 1, 1, 0, 0, 0, 0, 0
> [912] -3, 2, 0, 0, 0, 0, 0, 0
> [912] 1, -1, 0, 0, 0, 0, 0, 0
> [912] -1, 0, 0, 0, 0, 0, 0, 0
> [912] 1, 0, 0, 0, 0, 0, 0, 0
> [912] 0, 0, 0, 0, 0, 0, 0, 0
> [912] 0, 0, 0, 0, 0, 0, 0, 0
> [912] 0, 0, 0, 0, 0, 0, 0, 0
> 
> See what happened to DC value? Gone...
> I was doing my best to understand what happened to it. Here is the
> reconstruction of the block from all trellis nodes:
> 
> [912] node 9, value -1 run 1
> [912] node 8, value -1 run 3
> [912] node 5, value 1 run 1
> [912] node 4, value 2 run 1
> [912] node 3, value 1 run 1
> [912] node 2, value -3 run 1
> [912] node 1, value 1 run 2
> 
> ... which is consistent with the resulting block.
> 
> Unfortunately I'm completely stucked. The idea that trellis
> *increases* the level of coefficients from 0 to +/-1, thus
> *increasing* the filesize and *increasing* the psnr at fixed quant
> (look at the first AC coeff here) is completely against my
> understanding of trellis quantization, and quantization in general.
> 
> There must be something very smart happening here but unfortunatly I
> was unable to figure out what.
> 

	Let's take an example: suppose all coeffs are small
	(level~=1), and let's look at table Code_Len20[],
	which is the cost of coding a given run 'Run' when level
	is 1. One sees that runs 4,5 and 6 all requires 6 bits.
	Hence the sequence: [1 0 0 0 1 0 0 x...]  (run=4, then run=3)
	costs 6 + 5bits. Now, 'moving' the second level by 1 position:
	[1 0 0 0 0 1 0 x...] gives a cost of 6 + 4bits. And moving
	it further: [1 0 0 0 0 0 1 x...] costs 6+3bits. Hence,
	if the PSNR drop incurred by switching some coeffs from 1 to 0,
	and some other from 0 to 1 is compatible with the bits saved,
	the strangeness you describe occurs...

	Now, for the brutal transformation of DC=-181 to 0, this could
	be explained by the fact that coding the level '-181' costs
	a loooot of bits, compared to the rest. But i admit it's a bit
	too much. In fact, even if Fourier transform preserves L2
	norm, not all squared errors are born equal, in fact. Having
	.01dB of error on DC coeffs is not 'visually' equivalent than
	a 0.01dB error on coeff #31, even if mathematically they are.

	Ok, enough babble... i think we should prevent DC coeff to
	be modified (ie. "optimized") by trellis-quant func, even if
	it's (mathematically) justified... don't you think?
	Another idea: saturating the "PSNR + Lambda x Bitrate" cost
	function, when PSNR is too high...


	later!
Skal



> I hope you can help me.... or at least you can fix the bug. Thanks.
> 
> Radek
> 
> _______________________________________________
> XviD-devel mailing list
> XviD-devel at xvid.org
> http://list.xvid.org/mailman/listinfo/xvid-devel