[XviD-devel] [BUG] Segfault with large fincr/fbase values

[Adam Thayer]

libxvidcore is segfaulting in WriteVopHeader() when fincr and fbase are 
set to unreasonably high values. While these values are not realistic, 
the lack of bounds checking in the Bitstream<Op>() functions and the 
lack of setting bounds when entering xvid_encore/encode() causes it to 
write gobs of memory.

The values in question that I have seen cause a segfault are: 
fincr=2000000, fbase=83411. This yields an fps of roughly 23.98 fps, 
and unless some sort of framerate estimation algorithm is used before 
setting these values, 57 seconds into the encode, WriteVopHeader() 
attempts to write about 20+MB of bits and eventually leaves the bounds 
of the buffer (which is 20MB in this case), segfaulting the process.

The exact situation in question is after I wrote some code to allow 
ffmpeg to use libxvidcore as an xvid/mpeg4 encoder on Linux. An anime 
fansub (which was encoded in XviD) got slightly corrupted after 
encoding (one bit was off I believe, not XviD's fault) and ffmpeg 
calculated the framerate as shown above. If I use an estimation 
algorithm, it simplifies to: fincr=24000, fbase=1001, and encodes just 
fine.

Regards,
Adam Thayer