[Xvid-devel] xvid_encraw: AVI output >2 GB?

[Michael Militzer]

Yes, I think AviMux-GUI does not support raw m4v as input.

Another option would be to use Xvid's "clustering" feature and let
xvid_encraw break up your input video into smaller pieces that get
encoded in parallel in separate threads.

This was so far not supported for AVI output but I just added some code
to make this work. To use this feature you must compile xvid_encraw
with the USE_APP_LEVEL_THREADING define (you can uncomment line 66) and
you need to use the very latest version of xvid_encraw.c from VCS.

With USE_APP_LEVEL_THREADING enabled, xvid_encraw will break up your
input video into as many pieces as you have CPU cores and encode them
in parallel. So if you have four cores, it will output four AVI files.
You can then use AviMux-GUI to concatenate these AVIs and add the audio.

This does not change anything about the 2GB limit per each AVI file but
since xvid_encraw now outputs as many partial AVI files as you have CPU
cores, you can handle larger output sizes in total. This also has the
added benefit that internally we run multiple instances of xvidcore in
parallel which scales better over many cores than just multi-threading
a single xvidcore instance. So encoding will be faster in particular for
CPUs that have many (logical) cores...

One problem that remains is that you currently cannot control the number
of AVI output files that xvid_encraw will produce - it depends on the
number of CPU cores of the host. So when run on a low-end computer with
just 1 or 2 cores, it wouldn't help much to defeat the 2 GB limit. So to
solve your problem, we'd need to introduce a new option to xvid_encraw
that allows forcing a minimum segmentation no matter the number of CPU
cores. This should not be very difficult to do but I didn't want to spend
time on it now. Let me know in case you're interested in this feature...

Best regards,
Michael


Quoting Mario *LigH* Rohkrämer <contact at ligh.de>:

> The problem with AviMux-GUI is that it probably does not support m4v;
> therefore MeGUI had to start using ffmpeg first to multiplex a video-only
> AVI from a raw m4v, then AviMux-GUI to add audio streams.
>
> If AviMux-GUI supports MKV well, then it may be possible to have
> xvid_encraw encode large results to an MKV containing MPEG-4 ASP, then use
> AviMux-GUI to add audio streams and create an AVI. This will have to be
> tested... Or maybe using ffmpeg with libxvid instead of xvid_encraw to
> create the video in the first place.
>
> A VfW/DS/MF encoder (starting with VirtualDub or avs2avi) will write AVIs
> with the FourCCs "xvid"/"XVID" (vids/BITMAPINFOHEADER), but would require
> installing a VfW codec into the user's system. But ffmpeg does not support
> distinct FourCCs while multiplexing a raw m4v into an AVI container:
> "-vtag xvid" will create AVIs with the FourCCs "xvid"/"xvid", "-vtag XVID"
> will create AVIs with the FourCCs "XVID"/"XVID".
>
>
> Am 07.07.2015, 15:32 Uhr, schrieb Michael Militzer <michael at xvid.org>:
>
>> Hi,
>>
>> the xvid_encraw example does not have its own AVI muxer but it uses the
>> AVIFile API on windows. And I think you are correct: AVIFile has a size
>> limitation of 2 GB.
>>
>> We have no plans creating an own AVI muxer for xvid_encraw to circumvent
>> this size limitation. What you could do instead is to use the VfW encoder
>> via DirectShow or Media Foundation (like the Xvid MiniConvert example
>> does) - the DirectShow AVI muxer does not have the 2 GB size limitation.
>>
>> Other options would be to encode to a raw .m4v and then mux to AVI with
>> some muxing tool. ffmpeg would be one option and setting the FourCC
>> to "xvid"/"XVID" afterwards should be rather easy. A bigger issue might
>> be the "packed b-frames" feature of xvid_encraw in case you need it - I'd
>> guess this won't come out properly when you first encode to .m4v and then
>> mux with ffmpeg. Another option over ffmpeg might be MP4Box from the
>> gpac project - iirc, it can import raw m4v streams and output video muxed
>> as AVI. Also it's supposed to handle packed b-frames correctly. Finally,
>> there's also still AVIMux-GUI, which I think can also be scripted...
>>
>> Best regards,
>> Michael
>>
>>
>> Quoting Mario *LigH* Rohkrämer <contact at ligh.de>:
>>
>>> Greetings.
>>>
>>> Is it correct that xvid_encraw does not support OpenDML AVI output,
>>> hence
>>> limits created AVIs to 2 GB?
>>>
>>> And because it is an "example project", it is probably never planned to
>>> be
>>> supported?
>>>
>>> Would it be necessary to either prefer ffmpeg/mencoder with linked
>>> libxvid
>>> to encode directly to a large AVI, or let xvid_encraw create a raw m4v
>>> file and use a supporting multiplexer (probably only ffmpeg) later?
>>>
>>> The latter will have the disadvantage that a following multiplexer needs
>>> to get told the FourCCs to be filled in the AVI header (except it would
>>> derive them from the m4v input format). I guess that the second FourCC
>>> (in
>>> the compression field of the BITMAPINFOHEADER structure) will be the
>>> only
>>> relevant for most players; but I can't be certain that no software ever
>>> will check the first FourCC too (the "used encoder" in the "vids"
>>> chunk).
>>> I found that ffmpeg as AVI multiplexer will use a forced FourCC via
>>> "-vtag" parameter, but apply both the same, whereas the VfW uses
>>> distinct
>>> FourCCs (lowercase "xvid" in "vids" chunk and uppercase "XVID" in
>>> compression field).
>>>
>>> --
>>>
>>> Fun and success!
>>> Mario *LigH* Rohkrämer
>>> mailto:contact at ligh.de
>>>
>>> _______________________________________________
>>> Xvid-devel mailing list
>>> Xvid-devel at xvid.org
>>> http://list.xvid.org/mailman/listinfo/xvid-devel
>>>
>>
>>
>>
>>
>> _______________________________________________
>> Xvid-devel mailing list
>> Xvid-devel at xvid.org
>> http://list.xvid.org/mailman/listinfo/xvid-devel
>
>
> --
>
> Fun and success!
> Mario *LigH* Rohkrämer
> mailto:contact at ligh.de
>
> _______________________________________________
> Xvid-devel mailing list
> Xvid-devel at xvid.org
> http://list.xvid.org/mailman/listinfo/xvid-devel
>