[XviD-devel] multithreading

[Lars Täuber]

Hallo Radek,

Radek Czyz <radoslaw at syskin.cjb.net> schrieb:
> Hi,
> 
>  > In avidemux I set the xvid threads manually to 12
> 
> Ugh, why did you do that, is it any faster than 2 or 3? If my tests are 
> any indication it should be significantly slower.

fewer threads give me much lower cpu usage. But I'll try again. You suggest as many threads as cores available?

>  > only get a cpu usage of around 250% even on the second run of 2-pass 
> encoding. The first run is at about 175% cpu usage max.
> 
> Yeah that's a limitation of both current threading model as well as 
> current implementation.
> If someone wants to rewrite implementation with proper mutexes etc (or 
> in any other way drop Sleep(); win98 compatibility is no longer the 
> issue it used to be) please go ahead.

I don't know anything about Win stuff. I'm on a Linux box.

>  > Could you elaborate the threading model for xvid in simple words? Are 
> all threads encoding a single frame until it's finished?
> 
> Yes all threads are motion-estimating (not even fully encoding) a single 
> frame. For the most part this is required if xvid is supposed to stay 
> VfW-compatible, because VfW requires that a frame is encoded and written 
> before next frame can be delivered.

As mentioned before the VfW is non of my business.

>  > I took some thinking about how I would do it and therefore liked to 
> know if the first run on a 2-pass encoding fixes the series of frame types?
> 
> Yes, frame types are written to stats file by first pass and reused. In 
> case you wonder this doesn't do anything useful for any parallization.

I don't agree. Because if you know which frame will be encoded independently (I-Frame) or independently from future frames (P-Frame) you could encode like this:

- initialize two buffers. one for future I-frames one for future P-frames
- a dispatcher thread cares that at min. 2(?) encoded I-frames are in the I-buffer
- same with P-buffer
- next free thread gets to encode a B-frame if a future P or I-frame are available in the buffers
--- otherwise get this thread to encode an P or I-frame
- each thread encodes one frame completely

When I'm right then there is an encoding thread never waiting for something.
This might not be possible with VfW as of your comment above. But I don't care.

How much faster/slower are the B-frames encoded in contrast to P or I-frames? Is there a rule of thumb?

What do you think about this?

Lars
-- 
Lars Täuber <lars.taeuber at web.de>