From: Paul Eggert Date: Thu, 14 Jul 2011 20:40:35 +0000 (-0700) Subject: Merge from trunk. X-Git-Tag: emacs-pretest-24.0.90~104^2~152^2~149 X-Git-Url: http://git.eshelyaron.com/gitweb/?a=commitdiff_plain;h=1fd800e53d98ac4ea8ed560db81bd36545788b56;p=emacs.git Merge from trunk. --- 1fd800e53d98ac4ea8ed560db81bd36545788b56 diff --cc src/ChangeLog index 0977b12ab38,50a2d82fc7e..ce080f6b239 --- a/src/ChangeLog +++ b/src/ChangeLog @@@ -1,185 -1,223 +1,405 @@@ +2011-07-14 Paul Eggert + + Integer signedness and overflow and related fixes. (Bug#9079) + + * alloc.c (__malloc_size_t): Remove. + All uses replaced by size_t. See Andreas Schwab's note + . + + * image.c: Improve checking for integer overflow. + (check_image_size): Assume that f is nonnull, since + it is always nonnull in practice. This is one less thing to + worry about when checking for integer overflow later. + (x_check_image_size): New function, which checks for integer + overflow issues inside X. + (x_create_x_image_and_pixmap, xbm_read_bitmap_data): Use it. + This removes the need for a memory_full check. + (xbm_image_p): Rewrite to avoid integer multiplication overflow. + (Create_Pixmap_From_Bitmap_Data, xbm_load): Use x_check_image_size. + (xbm_read_bitmap_data): Change locals back to 'int', since + their values must fit in 'int'. + (xpm_load_image, png_load, tiff_load): + Invoke x_create_x_image_and_pixmap earlier, + to avoid much needless work if the image is too large. + (tiff_load): Treat overly large images as if + x_create_x_image_and_pixmap failed, not as malloc failures. + (gs_load): Use x_check_image_size. + + * gtkutil.c: Omit integer casts. + (xg_get_pixbuf_from_pixmap): Remove unnecessary cast. + (xg_set_toolkit_scroll_bar_thumb): Rewrite to avoid need for cast. + + * image.c (png_load): Don't assume height * row_bytes fits in 'int'. + + * xfaces.c (Fbitmap_spec_p): Fix integer overflow bug. + Without this fix, (bitmap-spec-p '(34359738368 1 "x")) + would wrongly return t on a 64-bit host. + + * dispnew.c (init_display): Use *_RANGE_OVERFLOW macros. + The plain *_OVERFLOW macros run afoul of GCC bug 49705 + + and therefore cause GCC to emit a bogus diagnostic in some cases. + + * image.c: Integer signedness and overflow and related fixes. + This is not an exhaustive set of fixes, but it's time to + record what I've got. + (lookup_pixel_color, check_image_size): Remove redundant decls. + (check_image_size): Don't assume that arbitrary EMACS_INT values + fit in 'int', or that arbitrary 'double' values fit in 'int'. + (x_alloc_image_color, x_create_x_image_and_pixmap, png_load) + (tiff_load, imagemagick_load_image): + Check for overflow in size calculations. + (x_create_x_image_and_pixmap): Remove unnecessary test for + xmalloc returning NULL; that can't happen. + (xbm_read_bitmap_data): Don't assume sizes fit into 'int'. + (xpm_color_bucket): Use better integer hashing function. + (xpm_cache_color): Don't possibly over-allocate memory. + (struct png_memory_storage, tiff_memory_source, tiff_seek_in_memory) + (gif_memory_source): + Use ptrdiff_t, not int or size_t, to record sizes. + (png_load): Don't assume values greater than 2**31 fit in 'int'. + (our_stdio_fill_input_buffer): Prefer ptrdiff_t to size_t when + either works, as we prefer signed integers. + (tiff_read_from_memory, tiff_write_from_memory): + Return tsize_t, not size_t, since that's what the TIFF API wants. + (tiff_read_from_memory): Don't fail simply because the read would + go past EOF; instead, return a short read. + (tiff_load): Omit no-longer-needed casts. + (Fimagemagick_types): Don't assume size fits into 'int'. + + Improve hashing quality when configured --with-wide-int. + * fns.c (hash_string): New function, taken from sxhash_string. + Do not discard information about ASCII character case; this + discarding is no longer needed. + (sxhash-string): Use it. Change sig to match it. Caller changed. + * lisp.h: Declare it. + * lread.c (hash_string): Remove, since we now use fns.c's version. + The fns.c version returns a wider integer if --with-wide-int is + specified, so this should help the quality of the hashing a bit. + + * emacs.c: Integer overflow minor fix. + (heap_bss_diff): Now uprintmax_t, not unsigned long. All used changed. + Define only if GNU_LINUX. + (main, Fdump_emacs): Set and use heap_bss_diff only if GNU_LINUX. + + * dispnew.c: Integer signedness and overflow fixes. + Remove unnecessary forward decls, that were a maintenance hassle. + (history_tick): Now uprintmax_t, so it's more likely to avoid overflow. + All uses changed. + (adjust_glyph_matrix, realloc_glyph_pool, adjust_frame_message_buffer) + (scrolling_window): Use ptrdiff_t, not int, for byte count. + (prepare_desired_row, line_draw_cost): + Use int, not unsigned, where either works. + (save_current_matrix, restore_current_matrix): + Use ptrdiff_t, not size_t, where either works. + (init_display): Check for overflow more accurately, and without + relying on undefined behavior. + + * editfns.c (pWIDE, pWIDElen, signed_wide, unsigned_wide): + Remove, replacing with the new symbols in lisp.h. All uses changed. + * fileio.c (make_temp_name): + * filelock.c (lock_file_1, lock_file): + * xdisp.c (message_dolog): + Don't assume PRIdMAX etc. works; this isn't portable to pre-C99 hosts. + Use pMd etc. instead. + * lisp.h (printmax_t, uprintmax_t, pMd, pMu): New types and macros, + replacing the pWIDE etc. symbols removed from editfns.c. + + * keyboard.h (num_input_events): Now uintmax_t. + This is (very slightly) less likely to mess up due to wraparound. + All uses changed. + + * buffer.c: Integer signedness fixes. + (alloc_buffer_text, enlarge_buffer_text): + Use ptrdiff_t rather than size_t when either will do, as we prefer + signed integers. + + * alloc.c: Integer signedness and overflow fixes. + Do not impose an arbitrary 32-bit limit on malloc sizes when debugging. + (__malloc_size_t): Default to size_t, not to int. + (pure_size, pure_bytes_used_before_overflow, stack_copy_size) + (Fgarbage_collect, mark_object_loop_halt, mark_object): + Prefer ptrdiff_t to size_t when either would do, as we prefer + signed integers. + (XMALLOC_OVERRUN_CHECK_OVERHEAD): New macro. + (xmalloc_overrun_check_header, xmalloc_overrun_check_trailer): + Now const. Initialize with values that are in range even if char + is signed. + (XMALLOC_PUT_SIZE, XMALLOC_GET_SIZE): Remove, replacing with ... + (xmalloc_put_size, xmalloc_get_size): New functions. All uses changed. + These functions do the right thing with sizes > 2**32. + (check_depth): Now ptrdiff_t, not int. + (overrun_check_malloc, overrun_check_realloc, overrun_check_free): + Adjust to new way of storing sizes. Check for size overflow bugs + in rest of code. + (STRING_BYTES_MAX): Adjust to new overheads. The old code was + slightly wrong anyway, as it missed one instance of + XMALLOC_OVERRUN_CHECK_OVERHEAD. + (refill_memory_reserve): Omit needless cast to size_t. + (mark_object_loop_halt): Mark as externally visible. + + * xselect.c: Integer signedness and overflow fixes. + (Fx_register_dnd_atom, x_handle_dnd_message): + Use ptrdiff_t, not size_t, since we prefer signed. + (Fx_register_dnd_atom): Check for ptrdiff_t (and size_t) overflow. + * xterm.h (struct x_display_info): Use ptrdiff_t, not size_t, for + x_dnd_atoms_size and x_dnd_atoms_length. + + * doprnt.c: Prefer signed to unsigned when either works. + * eval.c (verror): + * doprnt.c (doprnt): + * lisp.h (doprnt): + * xdisp.c (vmessage): + Use ptrdiff_t, not size_t, when using or implementing doprnt, + since the sizes cannot exceed ptrdiff_t bounds anyway, and we + prefer signed arithmetic to avoid comparison confusion. + * doprnt.c (doprnt): Avoid a "+ 1" that can't overflow, + but is a bit tricky. + + Assume freestanding C89 headers, string.h, stdlib.h. + * data.c, doprnt.c, floatfns.c, print.c: + Include float.h unconditionally. + * gmalloc.c: Assume C89-at-least behavior for preprocessor, + limits.h, stddef.h, string.h. Use memset instead of 'flood'. + * regex.c: Likewise for stddef.h, string.h. + (ISASCII): Remove; can assume it returns 1 now. All uses removed. + * s/aix4-2.h (HAVE_STRING_H): Remove obsolete undef. + * s/ms-w32.h (HAVE_LIMITS_H, HAVE_STRING_H, HAVE_STDLIB_H) + (STDC_HEADERS): Remove obsolete defines. + * sysdep.c: Include limits.h unconditionally. + + Assume support for memcmp, memcpy, memmove, memset. + * lisp.h, sysdep.c (memcmp, memcpy, memmove, memset): + * regex.c (memcmp, memcpy): + Remove; we assume C89 now. + + * gmalloc.c (memcpy, memset, memmove): Remove; we assume C89 now. + (__malloc_safe_bcopy): Remove; no longer needed. + + * lisp.h (struct vectorlike_header, struct Lisp_Subr): Signed sizes. + Use EMACS_INT, not EMACS_UINT, for sizes. The code works equally + well either way, and we prefer signed to unsigned. + + 2011-07-14 Lars Magne Ingebrigtsen + + * data.c (Fcdr, Fcar): Revert the last change, since it didn't + really clarify much. + + * search.c (Fre_search_backward): Mention `case-fold-search' in + all the re_search_* functions (bug#8138). + + * keyboard.c (Fopen_dribble_file): Document when the file is + closed (bug#8056). + + 2011-07-14 Eli Zaretskii + + * bidi.c (bidi_dump_cached_states): Fix format of displaying + bidi_cache_idx. + + Support bidi reordering of display and overlay strings. + * xdisp.c (compute_display_string_pos) + (compute_display_string_end): Accept additional argument STRING. + (init_iterator, reseat_1): Initialize bidi_it->string.s to NULL. + (reseat_to_string): Initialize bidi_it->string.s and + bidi_it->string.schars. + (Fcurrent_bidi_paragraph_direction): Initialize itb.string.s to + NULL (avoids a crash in bidi_paragraph_init). Initialize + itb.string.lstring. + (init_iterator): Call bidi_init_it only of a valid + buffer position was specified. Initialize paragraph_embedding to + L2R. + (reseat_to_string): Initialize the bidi iterator. + (display_string): If we need to ignore text properties of + LISP_STRING, set IT->stop_charpos to IT->end_charpos. (The + original value of -1 will not work with bidi.) + (compute_display_string_pos): First arg is now struct + `text_pos *'; all callers changed. Support display properties on + Lisp strings. + (compute_display_string_end): Support display properties on Lisp + strings. + (init_iterator, reseat_1, reseat_to_string): Initialize the + string.bufpos member to 0 (zero, for compatibility with IT_CHARPOS + when iterating on a string not from display properties). + (compute_display_string_pos, compute_display_string_end): Fix + calculation of the object to scan. Fixes an error when using + arrow keys. + (next_element_from_buffer): Don't abort when IT_CHARPOS is before + base_level_stop; instead, set base_level_stop to BEGV. Fixes + crashes in vertical-motion. + (next_element_from_buffer): Improve commentary for when + the iterator is before prev_stop. + (init_iterator): Initialize bidi_p from the default value of + bidi-display-reordering, not from buffer-local value. Use the + buffer-local value only if initializing for buffer iteration. + (handle_invisible_prop): Support invisible properties on strings + that are being bidi-reordered. + (set_iterator_to_next): Support bidi reordering of C strings and + Lisp strings. + (next_element_from_string): Support bidi reordering of Lisp + strings. + (handle_stop_backwards): Support Lisp strings as well. + (display_string): Support display of R2L glyph rows. Use + IT_STRING_CHARPOS when displaying from a Lisp string. + (init_iterator): Don't initialize it->bidi_p for strings + here. + (reseat_to_string): Initialize it->bidi_p for strings here. + (next_element_from_string, next_element_from_c_string) + (next_element_from_buffer): Add xassert's for correspondence + between IT's object being iterated and it->bidi_it.string + structure. + (face_before_or_after_it_pos): Support bidi iteration. + (next_element_from_c_string): Handle the case of the first string + character that is not the first one in the visual order. + (get_visually_first_element): New function, refactored from common + parts of next_element_from_buffer, next_element_from_string, and + next_element_from_c_string. + (tool_bar_lines_needed, redisplay_tool_bar) + (display_menu_bar): Force left-to-right direction. Add a FIXME + comment for making that be controlled by a user option. + (push_it, pop_it): Save and restore the state of the + bidi iterator. Save and restore the bidi_p flag. + (pop_it): Iterate out of display property for string iteration as + well. + (iterate_out_of_display_property): Support iteration over strings. + (handle_single_display_spec): Set up it->bidi_it for iteration + over a display string, and call bidi_init_it. + (handle_single_display_spec, next_overlay_string) + (get_overlay_strings_1, push_display_prop): Set up the bidi + iterator for displaying display or overlay strings. + (forward_to_next_line_start): Don't use the shortcut if + bidi-iterating. + (back_to_previous_visible_line_start): If handle_display_prop + pushed the iterator stack, restore the internal state of the bidi + iterator by calling bidi_pop_it same number of times. + (reseat_at_next_visible_line_start): If ON_NEWLINE_P is non-zero, + and we are bidi-iterating, don't decrement the iterator position; + instead, set the first_elt flag in the bidi iterator, to produce + the same effect. + (reseat_1): Remove redundant setting of string_from_display_prop_p. + (push_display_prop): xassert that we are iterating a buffer. + (push_it, pop_it): Save and restore paragraph_embedding member. + (handle_single_display_spec, next_overlay_string) + (get_overlay_strings_1, reseat_1, reseat_to_string) + (push_display_prop): Set up the `unibyte' member of bidi_it.string + correctly. Don't assume unibyte strings are not bidi-reordered. + (compute_display_string_pos) + (compute_display_string_end): Fix handling the case of C string. + (push_it, pop_it): Save and restore from_disp_prop_p. + (handle_single_display_spec, push_display_prop): Set the + from_disp_prop_p flag. + (get_overlay_strings_1): Reset the from_disp_prop_p flag. + (pop_it): Call iterate_out_of_display_property only if we are + popping after iteration over a string that came from a display + property. Fix a typo in popping stretch info. Add an assertion + for verifying that the iterator position is in sync with the bidi + iterator. + (handle_single_display_spec, get_overlay_strings_1) + (push_display_prop): Fix initialization of paragraph direction for + string when that of the parent object is not yet determined. + (reseat_1): Call bidi_init_it to resync the bidi + iterator with IT's position. (Bug#7616) + (find_row_edges): If ROW->start.pos gives position + smaller than min_pos, use it as ROW->minpos. (Bug#7616) + (handle_stop, back_to_previous_visible_line_start, reseat_1): + Reset the from_disp_prop_p flag. + (SAVE_IT, RESTORE_IT): New macros. + (pos_visible_p, face_before_or_after_it_pos) + (back_to_previous_visible_line_start) + (move_it_in_display_line_to, move_it_in_display_line) + (move_it_to, move_it_vertically_backward, move_it_by_lines) + (try_scrolling, redisplay_window, display_line): Use them when + saving a temporary copy of the iterator and restoring it back. + (back_to_previous_visible_line_start, reseat_1) + (init_iterator): Empty the bidi cache "stack". + (move_it_in_display_line_to): If iterator ended up at + EOL, but we never saw any buffer positions smaller than + to_charpos, return MOVE_POS_MATCH_OR_ZV. Fixes vertical cursor + motion in bidi-reordered lines. + (move_it_in_display_line_to): Record prev_method and prev_pos + immediately before the call to set_iterator_to_next. Fixes cursor + motion in bidi-reordered lines with stretch glyphs and strings + displayed in margins. (Bug#8133) (Bug#8867) + Return MOVE_POS_MATCH_OR_ZV only if iterator position is past + TO_CHARPOS. + (pos_visible_p): Support positions in bidi-reordered lines. Save + and restore bidi cache. + + * bidi.c (bidi_level_of_next_char): clen should be EMACS_NT, not int. + (bidi_paragraph_info): Delete unused struct. + (bidi_cache_idx, bidi_cache_last_idx): Declare EMACS_INT. + (bidi_cache_start): New variable. + (bidi_cache_reset): Reset bidi_cache_idx to bidi_cache_start, not + to zero. + (bidi_cache_fetch_state, bidi_cache_search) + (bidi_cache_find_level_change, bidi_cache_iterator_state) + (bidi_cache_find, bidi_peek_at_next_level) + (bidi_level_of_next_char, bidi_find_other_level_edge) + (bidi_move_to_visually_next): Compare cache index with + bidi_cache_start rather than with zero. + (bidi_fetch_char): Accept new argument STRING; all callers + changed. Support iteration over a string. Support strings with + display properties. Support unibyte strings. Fix the type of + `len' according to what STRING_CHAR_AND_LENGTH expects. + (bidi_paragraph_init, bidi_resolve_explicit_1) + (bidi_resolve_explicit, bidi_resolve_weak) + (bidi_level_of_next_char, bidi_move_to_visually_next): Support + iteration over a string. + (bidi_set_sor_type, bidi_resolve_explicit_1) + (bidi_resolve_explicit, bidi_type_of_next_char): ignore_bn_limit + can now be zero (for strings); special values 0 and -1 were + changed to -1 and -2, respectively. + (bidi_char_at_pos): New function. + (bidi_paragraph_init, bidi_resolve_explicit, bidi_resolve_weak): + Call it instead of FETCH_MULTIBYTE_CHAR. + (bidi_move_to_visually_next): Abort if charpos or bytepos were not + initialized to valid values. + (bidi_init_it): Don't initialize charpos and bytepos with invalid + values. + (bidi_level_of_next_char): Allow the sentinel "position" to pass + the test for valid cached positions. Fix the logic for looking up + the sentinel state in the cache. GCPRO the Lisp string we are + iterating. + (bidi_push_it, bidi_pop_it): New functions. + (bidi_initialize): Initialize the bidi cache start stack pointer. + (bidi_cache_ensure_space): New function, refactored from part of + bidi_cache_iterator_state. Don't assume the required size is just + one BIDI_CACHE_CHUNK away. + (bidi_cache_start_stack, bidi_push_it): Use IT_STACK_SIZE. + (bidi_count_bytes, bidi_char_at_pos): New functions. + (bidi_cache_search): Don't assume bidi_cache_last_idx is + always valid if bidi_cache_idx is valid. + (bidi_cache_find_level_change): xassert that bidi_cache_last_idx + is valid if it's going to be used. + (bidi_shelve_cache, bidi_unshelve_cache): New functions. + (bidi_cache_fetch_state, bidi_cache_search) + (bidi_cache_find_level_change, bidi_cache_ensure_space) + (bidi_cache_iterator_state, bidi_cache_find) + (bidi_find_other_level_edge, bidi_cache_start_stack): All + variables related to cache indices are now EMACS_INT. + + * dispextern.h (struct bidi_string_data): New structure. + (struct bidi_it): New member `string'. Make flag members be 1-bit + fields, and put them last in the struct. + (compute_display_string_pos, compute_display_string_end): Update + prototypes. + (bidi_push_it, bidi_pop_it): Add prototypes. + (struct iterator_stack_entry): New members bidi_p, + paragraph_embedding, and from_disp_prop_p. + (struct it): Member bidi_p is now a bit field 1 bit wide. + (bidi_shelve_cache, bidi_unshelve_cache): Declare + prototypes. + + * .gdbinit (xvectype, xvector, xcompiled, xchartable, xboolvector) + (xpr, xfont, xbacktrace): Use "header.size" when accessing vectors + and vector-like objects. + + * dispnew.c (buffer_posn_from_coords): Save and restore the bidi + cache around display iteration. + + * window.c (Fwindow_end, window_scroll_pixel_based) + (displayed_window_lines, Frecenter): Save and restore the bidi + cache around display iteration. + 2011-07-14 Lars Magne Ingebrigtsen * editfns.c (Fdelete_region): Clarify the use of the named