[XviD-devel] multithreading

[Radek Czyz]

Hi Pete and all,

 > Disagree. Using the first pass data, the encoder could assign a target
 > rate/size for each GOP. I am sure somebody in academia has explored doing
 > this.

Unless you want to go into 3rd pass and maybe even 4th pass, the target 
bitrate for each gop is going to be so unreliable that combined bitrate 
will almost certainly risk being 25% off.

The fact that xvid encodes first pass at constant quant (rather than 
doing what x264 is doing, again) is making precise scaling hard.

 >> Is this possible?

You're dealing with another issue here: you expect encoding application 
to be able to arbitrarily seek in source video. Almost no application 
can do that, and those that can will:
1. Risk not being frame-accurate, which would be a disaster
2. Waste a lot of CPU and memory while doing so. Just like encoding can 
be independent at keyframe boundary, decoding is only independent at 
keyframe boundary too, and source keyframes are not controllable. This 
makes random seeking hard.

 > Of course this is only meant for quantized encoding without any 
bounds for the bitrate.

Which makes first pass a "bit" of a waste...

Radek


Peter Ross wrote:
> On Thu, Jun 25, 2009 at 08:43:35PM +0200, Lars Täuber wrote:
>> Hi Michael,
>>
>> On Thu, 25 Jun 2009 16:37:07 +0200 Michael Militzer <michael at xvid.org> wrote:
>>> Your idea with the 2-pass stats works too: You could split the input stream
>>> in e.g. four parts, run four xvid encoder instances in parallel that do the
>>> first-pass. You then end up with four first-pass stat files. Your app
>>> then joins the stat files and analyzes the relative complexity of each of
>>> the sub-parts. According to that analysis you assign target bitrates or
>>> filesizes to each of the four parts, launch four xvid encoder instances in
>>> second-pass mode and again join everything together in the end.
>>>
>>> I remember that this idea had already been discussed in the past, so there
>>> should be some tool already that can do this for you. In any case, this
>>> method should give good quality (as long as you don't partition the input
>>> into too many small pieces), scales well, will be fast and really easy to
>>> implement (on the application level).
>> this is not exactly my idea but near.
>>
>> I'd like Xvid to go through the video in a 1st run and tell the application the series/order of frames types.
>> Then the application could split the video depending on the now known GOP borders into pieces containing exactly the frames used for the new GOPs. Then it could run as many parallel encoding processes as it likes given enough pieces are available.
>> (or pairs of threads: decoder => encoder)
>>
>> This will only work efficiently when all the new GOPs are independently encodable. As far as I know this is the case with xvid right?
> 
> Yes.
> 
>> Of course this is only meant for quantized encoding without any bounds for the bitrate.
> 
> Disagree. Using the first pass data, the encoder could assign a target
> rate/size for each GOP. I am sure somebody in academia has explored doing
> this.
> 
>> Is this possible?
> 
> Yes.
> 
> -- Peter
> (A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
> _______________________________________________
> Xvid-devel mailing list
> Xvid-devel at xvid.org
> http://list.xvid.org/mailman/listinfo/xvid-devel
>