[XviD-devel] better trellis?

Michael Militzer michael at xvid.org
Fri Sep 3 17:01:41 CEST 2004


Hi Radek,

using a bias of zero in quantization before trellis could definitively pay
off as the original paper about trellis quantization did even suggest to
use the unquantized DCT coefficients as input for the trellis iirc. But
imho there are some things that should be checked:

I suspect that trellis quantization will become (significantly) slower when
using quantization without bias. That's simply because we have more non-
zero coefficients among the quantized DCT coefficients and therefore more
nodes in the trellis. I guess even when you only check for lowering levels,
it might still be slower than the current implementation.

As another note, I would expect that higher quality (no bias + high lambda)
might only pay off in certain situations PSNR-wise. I know that you're
aiming to improve perceived quality and not PSNR, but a good quality/size
ratio is required. I'd expect that no bias and high lambda may perform
rather well in scenes with high motion, a lot of texture data and just
few skipped blocks. In this case, we don't have long runs anyway and from
a quality/size perspective it might be beneficial to reduce the 
quantization error as much as possible (bias = 0), since this won't 
increase the size of the coded bitstream that much while the quality
improvement could be large. On the other hand, in 'easy' scenes where we
would normally skip many blocks or have long runs (with bias=Q/2 iirc),
switching to bias=0 may not improve the quality much (because all blocks
can rather well be predicted from the reference frame with just few
stored coefficients) while the coding efficiency might be severely harmed
when runs become significantly shorter and only fewer blocks can be skipped
with bias = 0.

So probably, your lambda should also take motion or current avrg. texture 
size into consideration. On the other hand: from a HVS perspective we may
not like to have best quality in high motion scenes as the human eye may
not be capable at all to perceive the extra quality here - oh well, just
some thoughts...

Michael


Quoting Radek Czyz <syskin at ihug.com.au>:

> Hi everyone.
> 
> I'm starting to think about a backend for HVS modules. One of the 
> possible ways of using HVS data in encoder is to modulate lambda during 
> trellis quantization - more important parts of the picture get higher 
> lambda, less important get lower.
> 
> It's a good method, more precise and less costly than adaptive quantization.
> 
> There is a problem with current trellis though. It is only able to 
> decrease bitrate with low lambda, I can't count on it when I try to 
> increase quality.
> 
> This is because of the way it works: Firstly, normal quantization is 
> performed. Secondly, trellis optimizes the coefficients, but mostly by 
> decreasing levels. It will try to increase some zeroes to ones but only 
> some.
> The most extreme case is when block is zero after "normal" quantization. 
> Trellis will not be able to touch it at all, even if HVS model suggests 
> high lambda.
> 
> What I have done: instead of doing "normal" quantization before trellis, 
> I'm using a "minimum error" quantization. Such quantization has no 
> deadzone and no bias, and if the coefficients were kept this way, it 
> would be horribly inefficient. But this is where trellis kicks in: it 
> lowers (ONLY lowers) the levels to make it efficient. Since it only 
> lowers, it does less work (in many cases) than before, as there is only 
> one "direction" to check.
> 
> Depending on lambda, it is now perfectly possible to lower the 
> coefficients less or more, to achive a per-block quality that we need.
> 
> My current test results are a bit surprising. Bitrate at Q4 goes up by 
> whole 6%, PSNR goes up accordingly. At 2-pass, average PSNR is the same 
> as before (+/- 0.02dB) but the minimum PSNR (worst frame) goes up by as 
> much as 0.5dB. This seems to be the only benefit until we have HVS 
> models to modulate the quality - but this benefit is quite high, as it's 
> exactly the worst-case that is seen.
> 
> I don't have a proper code to show you yet, but I'll create something 
> interesting soon. In particular, I'll try moving trellis to separate 
> file - it's so messy currently :/ There is also a couple of ther fixes I 
> want to try (cbp cost in not 2 bits per coded block, especially not in 
> bframes).
> 
> Comments, suggestions?
> 
> Radek
> 
> PS. perhaps such trellis will like another lambda by default - quality 
> might increase
> _______________________________________________
> XviD-devel mailing list
> XviD-devel at xvid.org
> http://list.xvid.org/mailman/listinfo/xvid-devel
> 





More information about the XviD-devel mailing list