[XviD-devel] TOOSMALL-LIMIT + Trellis

[Michael Militzer]

Hi,

Quoting Edouard Gomez <ed.gomez at free.fr>:

> Christoph Lampert (chl at math.uni-bonn.de) wrote:
> > GomGom removed that (and the source tells to complain), but I'd like to
> > reactive this. 
> 
> No i did nothing to that yet. Michael removed itand thus this results in

yep.

> comparing oranges with apples as i explained in a previous email. But i
> have a patch for that ready. It makes trellis returns sum of absolute
> coeffs. I asked for review, but no one answered[1]...
>  
> > Any objection? Or did anyone find the combination of both useful? At least
> > in theory, it shouldn't.
> 
> Well trellis  can optimize the block in  such a way that  it matches our
> further block skipping rules, so  if trellis rules are "transitive" with
> our skipping rules... this would make sense.

I think what Christoph wanted to express is that trellis quantization is
R-D optimal (well, the simplified speed-up version we use is not...), so
performing coefficient thresholding afterwards will give no improvement
(R-D-wise).

While this is true, I like coefficient thresholding for the cartoon mode.
For cartoons, most of a frame is typically stationary (but often with some
noise) - so coefficient thresholding helps to remove unneeded details and
to achieve more skipped/not coded blocks which is very helpful for cartoons.
Well, of course this is not beneficial PSNR-wise, however the loss in
detail is not visible in cartoons (in fact the picture might even look 
better/cleaner).

> [1] The patch is still available for review:
> http://ed.gomez.free.fr/vrac/xvid-patches/mbtransquant-cleanup.diff

I'm not competent enough. (yeah, I'll say that always from now on when it
saves me from further work ;-)) - Christoph or Skal should have a look...

bye,
Michael