[XviD-devel] [BUG] Last patch in encoder.c breaks encoder

skal skal at planet-d.net
Fri Dec 10 08:54:56 CET 2004


	Howdy,

On Fri, 2004-12-10 at 02:40, Radek Czyz wrote:
> Edouard Gomez wrote:
> > Hey,
> > 
> > seems like the last patch from sysKin breaks the encoder badly.
> > 
> > This patch relies on the fact every block goes through RD optimisation
> > which is obviously bad idea, so encoder ends up trying to encode inter
> > block filled with 0, but these function assume there is at least one non
> > zero value in the 8x8 block and doesn't check for index<64 in order to speed
> > up things a bit.
> > 
> > This results in nice segfaults whatever sequence you encode w/o vhq enabled.
> > Meaning all first passes would fail.
> 
> What did you ever expect from me? ^^

	oh! talking about bugs, here's mine: it's been reported
	in doom9's forum that trellis quant is overflowing for
	very low Q=1~2. Thanks to Koepi, it seems that either
	reducing the precision used for trellis (TL_SHIFT) or
	forbidding usage of trellis for Q<=2 fixes the problem.
	I'd favor the last solution, which seems to have been
	tested ok, without noticeable loss of quality.
	So, if it's ok with you, we could change line 219 of
	mbtransquant.c to something like:

if(sum && pMB->quant>2 && (frame->vop_flags & XVID_VOP_TRELLISQUANT))
{
  ...
}

	to prevent call of trellis quant for low quant.

	bye!
Skal



More information about the XviD-devel mailing list