[XviD-devel] [BUG] Small LUT implementation - Previous patchwasnot the right one

[Marco Al]

From: "Marco Al" <marco at simplex.nl>

> If you absolutely cannot abide the test relying on underflow and the
> sequence of evaluation just keep my code and add an extra

Also level_esc has the same issue ... so you'd have more code to change.

I propose the following patch.

Marco-------------- next part --------------
97c97
< 	ptr_t i, j, intra, last, run,  run_esc, level, level_esc, escape, escape_len, offset;
---
> 	uint32_t i, j, intra, last, run,  run_esc, level, level_esc, escape, escape_len, offset;
132c132
< 			for (j = 0; j < 1 << (12 - coeff_tab[intra][i].vlc.len); j++)
---
> 			for (j = 0; j < (uint32_t)(1 << (12 - coeff_tab[intra][i].vlc.len)); j++)
157c157
< 				for (level = 1; level < 32 << intra; level++)
---
> 				for (level = 1; level < (uint32_t)(32 << intra); level++)
169,172d168
< 					/*use this test to use shorter esc2 codes when possible
< 					if (level_esc <= max_level[intra][last][run] && run <= max_run[intra][last][level_esc]
< 						&& !(coeff_VLC[intra][last][level_esc + offset][run].len + 7 + 1
< 							 > coeff_VLC[intra][last][level + offset][run_esc].code + 7 + 2))*/
182c178
< 						if (level <= max_level[intra][last][run_esc] && run_esc <= max_run[intra][last][level])
---
> 						if (run_esc <= max_run[intra][last][level] && level <= max_level[intra][last][run_esc])
223c219
< 				for (level = 32 << intra; level < 2048; level++)
---
> 				for (level = (uint32_t)(32 << intra); level < 2048; level++)