[XviD-devel] adaptive quant improvement

[Marco "elcabesa" Belli]

 Wednesday 29 January 2003 21:10, Edouard Gomez wrote:
> Hello again,
>
> i worked a bit on the code to understand it from the "math" point of
> view, i cleaned it up a bit making clearer math formulas.
>
> One of the formulas is also an optimization because it avoids some
> substraction and allows variance and mean values computation at the same
> time.
>
> Now i'll perform some tests
>


yeah you are right  variance is e(x^2)-m^2 so the code should be faster

now a reply ofr you and for Gruel

my code, that should work better if joint with your lumi_mask/weber_law, only 
calc for each mbloc variance/mean and the if result is a higher thant a 
threshold value it decrease the quanitzer, if it's lower it raise it

 some test  told me that variance /mean is a simple way to see where are edge 
or high detail area, since flat area have a low variance.

> @marco: where do these magic numbers come from ?
>         the 256 divisor and thresholds ?

256 divisor is to calc average

but 65536 division is wrong..... 

'couse mean is  
e[x]= 1/(16*16) sum         {  lum[x,y] }
                      x:0 to 15
                      y:0 to 15

but 
variance is

1/(16*16) sum         {  (lum[x,y]-e[x])^2  }
              x:0 to 15
              y:0 to 15

so my divsion by 65536 is wrong

BUT this will not make us go to a wrong quantizer setting..couse threshold are 
not calc value.. but experimentally tuned over image=)
so all division inside code can be move inside the treshold value=)

the way i get this threshold value is this: i changed my code  to something 
like this
	if(result < min_level) {
		quant[k*mb_width + l] += 4;
		ptr[7 * stride + 7]=255;
		ptr[7 * stride + 8]=255;
		ptr[8 * stride + 7]=255;
		ptr[8 * stride + 8]=255;
	}

then i do a encode with quant=0 and graphically see which mblock quantizer has  
been changed.

i think i have reply to all your question. if more please tell me=) 
thank gruel for the  paper i'll look at it asap

>Anyway, my answer is more a good way to ask everyone here, why the luminance
>masking is done in YUV while in DCT space the DC component of each block is
>already the average of the block (vertical and horyzontal null frequencies == 
>average). It's just scaled by a N/sqrt(2) or something very close to that.
>
>Would not it be more logical to do luminance masking just after fDCT to avoid
>a "let's do computations in plain C now and then do them again in optimized 
>assembly" ?

i know it and i think that lumi maskk too should be done this way, but i 
wanted not change all codec to do my test , and i don0t know mpeg4 standard.. 
so i don't know when i broke standards changing something in code=)

if you like i could try to move lumi mask after dct, but i don't know how many 
fuction i need to change =)

a quesiton  e[x]= sqrt(2)/16 *dct[0,0] ?