[XviD-devel] multithreading

[Radek Czyz]

Hi,

 > In avidemux I set the xvid threads manually to 12

Ugh, why did you do that, is it any faster than 2 or 3? If my tests are 
any indication it should be significantly slower.

 > only get a cpu usage of around 250% even on the second run of 2-pass 
encoding. The first run is at about 175% cpu usage max.

Yeah that's a limitation of both current threading model as well as 
current implementation.
If someone wants to rewrite implementation with proper mutexes etc (or 
in any other way drop Sleep(); win98 compatibility is no longer the 
issue it used to be) please go ahead.

 > Could you elaborate the threading model for xvid in simple words? Are 
all threads encoding a single frame until it's finished?

Yes all threads are motion-estimating (not even fully encoding) a single 
frame. For the most part this is required if xvid is supposed to stay 
VfW-compatible, because VfW requires that a frame is encoded and written 
before next frame can be delivered.

 > I took some thinking about how I would do it and therefore liked to 
know if the first run on a 2-pass encoding fixes the series of frame types?

Yes, frame types are written to stats file by first pass and reused. In 
case you wonder this doesn't do anything useful for any parallization.

 > And am thinking of changing my encoding scheme to multiple encodings 
in parallel using only 1 thread for every process.

That will be significantly more efficient yes.

Regards
Radek


Lars Täuber wrote:
> Hi there.
> 
> I'm a heavy user of xvid to transcode my private camcorder videos to xvid mostly using avidemux.
> I want to thank you very much for your work! Go ahead!
> 
> Lately I got a 3-core computer and wonder why the threading of xvid is so bad scaling with the available cores.
> In avidemux I set the xvid threads manually to 12 but only get a cpu usage of around 250% even on the second run of 2-pass encoding. The first run is at about 175% cpu usage max.
> 
> Could you elaborate the threading model for xvid in simple words? Are all threads encoding a single frame until it's finished?
> 
> I took some thinking about how I would do it and therefore liked to know if the first run on a 2-pass encoding fixes the series of frame types? In other words does the 2 run of a 2-pass encoding knows the frame type (I, P, B) to encode to of every frame in advance?
> 
> Mostly I do batch encoding all the videos of the last weeks. And am thinking of changing my encoding scheme to multiple encodings in parallel using only 1 thread for every process.
> 
> Thanks
> Lars