[XviD-devel] Trellis Quant

[Christoph Lampert]

Hi,

okay, since dev-api-4 seems to be stable enough for committing some new
features (which don't interfere with plugins or encoder.c etc.), here is
the first: 

Trellis based R-D "optimal" quantisation, currently only for H263-quant
and only for INTER-frames, INTRA is going to follow later. It's a little
more difficult because of changing scanorder. 
I'm not sure yet, how I'll do MPEG-quant, maybe I'll change infrastructure
back to the way ffmpeg does by combining quantization itself and the R-D
optimization. At the moment, usual quantization routine is called and R-D
optimization acts on the resulting coefficients.

Trellis quantization is switched on by XVID_VOP_TRELLISQUANT , the main 
routine is in bitstream/mbcoding.c Not that trellis itself has anything to
with mbcoding, but it needs access to the VLC tables and those are only
available from there. 

There is one free parameter in the calculation of "lambda", I did some
quick tests and fixed it to 123, but you can check yourself, maybe a
different value (between 100 and 150?) will prove better. 
Don't make this lambda available in the GUI, it's just for internal
optmization. Trellis quant itself also might need a few more days before
it's GUI ready. 
Also, it's not completely optimized for speed, yet. The lower the quant,
the slower it gets. 

Have fun! 

gruel 

P.S. When Trellis is active, TOOSMALL_LIMIT must not be used anymore,
because the meaning of "sum <= limit" changes, that's why I set limit=1


--------------------------------------------------------------------
My Results with on dual Athlon XP 1600+ (MMX/MMXEXT/3dnow/SSE), 
all clips are CIF (352x288), encoded two-pass with xvid_encraw -stats 

#coastguard.cif 256kbps
#Avg: fps =  94.65, length = 1342, psnr y = 27.87, u = 39.33, v = 41.93
#Avg: fps =  85.28, length = 1335, psnr y = 28.13, u = 39.51, v = 41.88

#coastguard.cif 512kbps
#Avg: fps =  96.07, length = 2619, psnr y = 31.28, u = 41.23, v = 43.10
#Avg: fps =  68.82, length = 2604, psnr y = 31.44, u = 41.26, v = 43.13

# container.cif 256kbps  
#Avg: fps = 101.99, length = 1307, psnr y = 35.88, u = 42.78, v = 42.52
#Avg: fps =  86.57, length = 1308, psnr y = 36.02, u = 43.00, v = 42.63

# container.cif 512kbps  
#Avg: fps = 111.72, length = 2586, psnr y = 38.43, u = 44.45, v = 44.35
#Avg: fps =  73.13, length = 2584, psnr y = 38.75, u = 44.49, v = 44.49

#foreman.cif   256kbps
#Avg: fps =  90.25, length = 1283, psnr y = 30.79, u = 37.77, v = 38.61
#Avg: fps =  84.18, length = 1271, psnr y = 31.07, u = 37.90, v = 38.72

#foreman.cif   512kbps
#Avg: fps =  96.38, length = 2571, psnr y = 34.86, u = 39.97, v = 41.28
#Avg: fps =  63.93, length = 2534, psnr y = 35.01, u = 40.02, v = 41.38

#hall_monitor.cif  256kbps 
#Avg: fps = 104.63, length = 1234, psnr y = 37.55, u = 39.16, v = 41.37
#Avg: fps = 101.29, length = 1226, psnr y = 37.64, u = 39.22, v = 41.45

#hall_monitor.cif  512kbps 
#Avg: fps = 109.92, length = 2501, psnr y = 38.85, u = 39.93, v = 41.94
#Avg: fps =  83.07, length = 2485, psnr y = 39.02, u = 39.94, v = 41.92

#mobile.cif   512kbps
#Avg: fps =  96.81, length = 2699, psnr y = 24.63, u = 29.00, v = 28.38
#Avg: fps =  59.75, length = 2681, psnr y = 24.99, u = 29.31, v = 28.72

#paris.cif 256kbps
#Avg: fps = 103.81, length = 1299, psnr y = 33.10, u = 38.40, v = 38.66
#Avg: fps =  81.58, length = 1299, psnr y = 33.35, u = 38.52, v = 38.78

#paris.cif 512kbps
#Avg: fps = 102.06, length = 2568, psnr y = 36.92, u = 40.96, v = 41.21
#Avg: fps =  76.91, length = 2568, psnr y = 37.17, u = 40.95, v = 41.23