[XviD-devel] Byte swap bug on IA64 (including fix)

[Thomas Köckerbauer]

Hi

I've found another bug in the IA64 version of XviD. I'm working on a IA64 
machine with Linux which uses little endian. After patching the file quant.h 
(as described in my previous mail) everything compiled fine and the decoder 
worked without any problems, but the encoder produces invalid bitstreams.

Here is the start of a hexdump of the produced bitstream:
00000000  b0 01 00 00 20 00 00 00  20 00 00 00 20 00 00 00  |.... ... ... ...|
00000010  20 00 00 00 20 00 00 00  82 1e 0a 31 b2 01 00 00  | ... ......1....|
00000020  58 76 69 44 30 30 34 31  b6 01 00 00 20 00 00 00  |XviD0041.... ...|
00000030  20 00 00 00 df c6 db 7f  20 00 00 00 20 00 00 00  | ....... ... ...|
00000040  c6 db 7f 1b 20 00 00 00  20 00 00 00 20 00 00 00  |.... ... ... ...|
00000050  20 00 00 00 b6 df c6 db  20 00 00 00 20 00 00 00  | ....... ... ...|
00000060  df c6 db 7f 20 00 00 00  20 00 00 00 c6 db 7f 1b  |.... ... .......|
00000070  20 00 00 00 20 00 00 00  20 00 00 00 20 00 00 00  | ... ... ... ...|
00000080  b6 df c6 db 20 00 00 00  20 00 00 00 df c6 db 7f  |.... ... .......|
00000090  20 00 00 00 20 00 00 00  c6 db 7f 1b 20 00 00 00  | ... ....... ...|

As you can see the byte order is wrong. The file starts with b0 01 00 00 
instead of 00 00 01 b0 (= VISOBJSEQ_START_CODE).
The problem is in the BSWAP macro for IA64 (in portab.h):

#        define BSWAP(a)  __asm__ __volatile__ \
        ("mux1 %1 = %0, @rev" ";;" \
         "shr.u %1 = %1, 32" : "=r" (a) : "r" (a));

which should be:

#        define BSWAP(a)  __asm__ __volatile__ \
        ("mux1 %0 = %1, @rev" ";;" \
         "shr.u %0 = %0, 32" : "=r" (a) : "r" (a));

A diff file is at: http://www.gup.uni-linz.ac.at/~tkoeck/xvid_ia64_fix2.diff


Regards,
Thomas Köckerbauer