Prefix all itree.h type names with itree_

Rename interval_node -> itree_node, interval_tree -> itree_tree,
interval_tree_order -> itree_order.

* src/alloc.c: Renames.
* src/buffer.c: ditto.
* src/itree.c: ditto.
* src/itree.h: ditto.
* src/lisp.h: ditto.
* src/pdumper.h: ditto.
* src/textprop.h: ditto.
* src/xdisp.h: ditto.

Revert "mark_overlays: Use the normal ITREE_FOREACH"

This reverts commit b8fbd42f0a7caa4cd9e2d50dd4e4b2101ac78acd,
with edits.

* src/alloc.c (mark_overlays): restore function.
(mark_buffer): Call it, not ITREE_FOREACH.
(garbage_collect): eassert (!itree_busy_p ()).
* src/itree.h: Comment tweak: explain why GC is considered risky.  It
isn't that GC itself is risky, it is that GC can call ELisp by way of
a hook, and running ELisp during iteration is risks nested iteration.

Remove the ITREE_NULL macro and use NULL everywhere.

* src/itree.h: Delete the ITREE_NULL macro.
* src/itree.c (check_subtree): Use NULL everywhere.
* src/pdumper.c (dump_buffer): ditto.

; * src/buffer.c (set_overlay_region): remove unused function.

* src/itree.c: Eliminate all prototypes for static functions

Massive code reorganization to move definitions of static functions
before their first use, so as to remove the need for
redundant prototypes.
While at it, fix a few places where the used more than 80 column.

Rename itree iterators with itree_iterator prefix

* src/itree.h: Rename struct interval_generator -> itree_iterator.
Rename functions: itree_busy_p -> itree_iterator_busy_p,
interval_tree_iter_start -> itree_iterator_start,
interval_generator_narrow -> itree_iterator_narrow,
interval_tree_iter_finish -> itree_iterator_finish,
interval_generator_next -> itree_iterator_next.
* src/itree.c: Use new names everywhere.
* src/eval.c: ditto.

; * src/itree.c: remove 'inline' from functions with extern linkage

Delete the itree_null sentinel node, use NULL everywhere.

This effort caught a few (already commited) places that were
dereferencing through ITREE_NULL in a confusing way.  It makes some
functions have to check for NULL in more places, but in my experinece
this is worth it from a code clarity point of view.

In doing this I rewrote `interval_tree_remove` completely.  There
there was one final bug in that function that I simply could not find
when I #define'd ITREE_NULL to NULL.  I couldn't easily understand
that function, so instead I rewrote it completely with a focus on code
clarity.  Bug went away.

I have left the ITREE_NULL #define in code, to reduce code review
noise.  It is easily removed later, mechanically.

* src/itree.h: #define ITREE_NULL to NULL and leave a FIXME.
* src/itree.c (itree_null): Delete the itree_null static variable.
(null_is_sane): Delete (and all callers).
(interval_tree_insert): Insert the first node as black straight away.
(itree_newlimit): Handle NULL children.
(interval_tree_remove_fix): ditto.
(interval_tree_insert_fix): ditto.
(interval_tree_remove): Rewrite for clarity.
(null_safe_is_red): New function.
(null_safe_is_black): New function.
(interval_tree_replace_child): Renamed from interval_tree_transplant.
(interval_tree_transplant): New function that something I think is
more like a full transplantation.  (names are hard)

src/textprop.c (get_char_property_and_overlay): Fix bug#58479

Correct `get-char-property` which failed to ignore empty overlays.

* src/textprop.c (get_char_property_and_overlay): Make sure the
overlay covers the character.
* test/src/buffer-tests.el (buffer-tests--overlay-bug58479): New test.
* src/buffer.h (buffer_has_overlays): Avoid `interval_tree_size`.

; * src/itree.c (check_subtree): fix logical error in eassert

; * src/itree.c (check_tree): assert that the tree root is black

; * .clang-format: Add ITREE_FOREACH.

itree.c (check_tree): Simplify

* src/itree.c (struct check_subtree_result): Remove `complete`.
(check_subtree): Remove `max_depth` arg (and adjust callers).
Use 0 as black-depth of empty tree.
Remove redundant `node->parent` check (already performed by the caller).
(check_tree): Replace with `check_tree_common` (update all callers).
Check the root's `parent` field.
(check_tree_no_rb): Delete function, inlined in its sole caller.
(interval_tree_remove): Add call to `check_tree` (without RB checks)
before `interval_tree_remove_fix`.  Move update of `size`
field accordingly.

Stop reading and writing the itree_null.parent field entirely.

With this change all fields in the itree_null sentinel are read only.
This makes accessing itree_null thread safe in an obvious way.

Because it took two commits from two peole to get this right, I think
we can call this design fragile and difficult to reason about.
Another benefit of this commit is as preparation for removing sentinel
node completely, and just using NULL.

* src/itree.c (itree_null): Statically initialize itree_null.parent to
NULL.  It is never accessed.
(null_is_sane): Assert parent == NULL.
(interval_tree_remove_fix): Remove unecessary assignments to parent
from node->parent.  These were the last places itree_null.parent were
read.
(interval_tree_remove): Avoid an assignment to itree_null.parent
through min->right->parent.
(interval_tree_transplant): Avoid an assignment to itree_null.parent
through source->parent.

Simplify itree_null initialization

* src/itree.c (null_is_sane): call eassert directly, check
REAR_ADVANCE, FRONT_ADVANCE.  Add FIXME that PARENT is still
read/write.
(itree_null): initialize statically
(itree_init): remove initialization code, call eassert(null_is_sane())
(check_tree_common): call eassert (null_is_sane())

Check red-black invariants in most places

Stefan recently disabled this but I happened to want it back soon
after.

* src/itree.c (check_subtree): new arg: allow_red_red
(check_tree_common): renamed from check_tree, pass allow_red_red
through.
(check_tree): new function, pass allow_red_red=false
(interval_tree_insert): check_tree -> check_tree_common with
allow_red_red=true.

Improve check_subtree

* src/itree.c (struct check_subtree_result): new struct returned by
check_subtree.
(check_subtree): new function, renamed from recurse_check_tree.  Add
new black height assertions.
(check_tree): assert that the tree has non-negative size, assert that
limiting to interval_tree_max_height(tree) levels is enough to
traverses the complete tree.

; * test/src/buffer-tests.el: Remove unecessary `message' calls.

Tighten up handling of `otick`

Move args between `build_overlay` and `add_buffer_overlay`,
to try and keep buffer positions together with their buffer.
Be more strict in the `otick` values passed to `interval_tree_insert`.
Move a few things around to try and reduce dependencies through `.h` files.
Fix a thinko bug in `check_tree`.

* src/alloc.c (build_overlay): Remove `begin` and `end` args.

* src/buffer.c (add_buffer_overlay): Move from `buffer.h`.
Add `begin` and `end` args.
(copy_overlays): Adjust accordingly.
(Fmake_overlay): Use BUF_BEG and BUF_Z; adjust call to `build_overlay`
and `add_buffer_overlay`.
(Fmove_overlay): Use BUF_BEG and BUF_Z; Use the new `begin` and `end`
args of `add_buffer_overlay` so we don't need to use
`interval_node_set_region` when moving to a new buffer.
(remove_buffer_overlay, set_overlay_region): Move from `buffer.h`.

* src/buffer.h (set_overlay_region, add_buffer_overlay)
(remove_buffer_overlay): Move to `buffer.c`.
(build_overlay): Move from `lisp.h`.
(maybe_alloc_buffer_overlays): Delete function (inline into its only
caller).

* src/itree.c (interval_tree_insert): Move declaration `from buffer.h`.
(check_tree): Fix initial offset in call to `recurse_check_tree`.
Remove redundant check of the `limit` value.
(interval_node_init): Remove `begin` and `end` args.
(interval_tree_insert): Mark it as static.
Assert that the new node's `otick` should already be uptodate and its
new parent as well.
(itree_insert_node): New function.
(interval_tree_insert_gap): Assert the otick of the removed+added nodes
were uptodate and mark them as uptodate again after adjusting
their positions.
(interval_tree_inherit_offset): Check that the parent is at least as
uptodate as the child.

* src/lisp.h (build_overlay): Move to `buffer.h`.

* src/itree.h (interval_node_init): Adjust accordingly.
(interval_tree_insert): Remove declaration.
(itree_insert_node): New declaration.

itree.c: Use `interval_tree_inherit_offset`

The insertion code tried to manipulate the offset in its own way,
and apparently there was a bug in it.  Replace that with a call to
`interval_tree_inherit_offset`, making the whole logic a bit simpler,
and fixing a bug along the way (not sure where the bug was, to be honest).

* src/itree.c (interval_tree_insert): Use `interval_tree_inherit_offset`.
Check the tree before insert_fix.
(recurse_check_tree): Don't check RB invariants.
(itree_limits_are_stable): Delete function (subsumed by `check_tree`).

Debug check overlay tree invariants

* src/itree.c (check_tree):
(recurse_check_tree): new functions.
(interval_tree_insert): call them.
(interval_tree_remove): ditto.
(interval_tree_insert_fix): ditto.

; * test/src/buffer-tests.el (test-overlay-randomly): new test.

Comment change: explain inheriting "dirty" offsets

; * src/itree.c (interval_generator_next): explain why the code
handles inheriting offsets from dirty nodes.

itree: Try and detect non-local exits during itree iterations

* src/itree.c (itree_busy_p): New function.
* src/eval.c (signal_or_quit): Use it.
* src/itree.h (itree_busy_p): Declare it.

; * src/itree.c: Add comment describing when noverlay is O(N)

Use a bool instead of a bitfield

* src/itree.c (struct interval_generator): use a bool instead of a
bitfield, since space is not an issue.

; * src/itree.c: change comments for clarity.

; * src/itree.h (struct interval_node): document field invariants.

; * src/itree.h: include "lisp.h" for Lisp_Object

itree.c: Get rid of the trick using null->parent

* src/itree.c (interval_tree_remove_fix): Add a `parent` argument.
Change the loop so it always keeps both `node` and `parent` in sync,
thus avoiding the use of `node->parent` on the initial node (since
that one can be null).
(interval_tree_remove): Manually keep track of the `broken` node's
parent to pass it to `interval_tree_remove_fix`.

* src/itree.c (interval_tree_remove_fix): Move before first use

itree.c: Fix corner case errors in offsets

In some cases, `interval_tree_remove` could cause some nodes to
inherit fewer (or additional) offsets than the should because nodes
were transplanted between two parts of the tree where offsets had not
been propagated "equally".  So we remove/apply all offsets along the
path between the two points of a transplant before doing the transplant.

* src/itree.c (interval_tree_subtree_min): Move before first use; delete
the declaration; add an `otick` argument, and use it to update offsets
along the way.
(interval_tree_remove): Update all offsets on the way from `node` to `min`.
Reorder some of the operations so that when we transplant `min` to `node`
those nodes are in the proper state where `interval_tree_transplant`
can do its sanity checks.
(itree_total_offset): New function.
(interval_tree_transplant): Use it to sanity check that improper
offsets aren't accidentally inherited/lost because of the transplant.
(itree_newlimit): New function.
(itree_limit_is_stable, interval_tree_update_limit)
(interval_tree_propagate_limit): Use it.
(null_is_sane): Remove `inline` annotation; it's not needed.
(interval_tree_inherit_offset): Sanity check that `offset` is 0 when
`otick` is uptodate.  Skip the unneeded increments when the offset is 0.
(interval_tree_insert_fix): Add sanity check that we indeed have 2 reds.

itree.c: Fix incomplete update of `limit`s in corner cases

`interval_tree_remove` called `interval_tree_propagate_limit (subst)`
and `interval_tree_propagate_limit (min_right)` but both of those nodes
are moved without touching their subtrees, so their `limit`s are
"stable" causing `interval_tree_propagate_limit` to do nothing.
Indeed we don't need to update those nodes's `limit`s but we *do*
need to update their parents since those nodes have been moved.
Incidentally, this removes some uses of `null->parent` :-)

There are more uses of `null->parent`, tho, so I added more comments
explaining them (with the help of the matching section of the book
from which the algorithm was taken).

* src/itree.c (interval_tree_update_limit): Remove unused arg `tree`.
(interval_tree_rotate_left, interval_tree_rotate_right): Adjust callers.
(interval_tree_contains): Mark as static.
(itree_limit_is_stable, itree_limits_are_stable): New functions.
(interval_tree_remove): Fix incomplete update of `limit`s in corner
cases.
(interval_generator_next): Add sanity check to make sure the `limit`s
were properly updated.

* src/itree.h (interval_tree_contains): Remove declaration.

itree.c: Clarify how the sentinel is used

* src/itree.c (null_is_sane): New function.
(interval_tree_iter_start): Use it to make sure we haven't garbled null.
(itree_init): Fully initialize `itree_null` here...
(interval_tree_clear): ...instead of here.
(interval_tree_propagate_limit): Remove special code that read NULL->parent.
(interval_tree_remove): Do it explicitly before the call in those two
places where it can happen.

itree.c: Remove some unnecessary `tree` arguments

* src/itree.c (interval_tree_propagate_limit)
(interval_tree_subtree_min): Remove `tree` argument.
(interval_node_set_region, interval_tree_remove)
(interval_tree_insert_gap, interval_tree_delete_gap): Adjust callers.

itree: Use a single iterator object

Instead of having one iterator object per buffer, use just a single
global one.  There is virtually no benefit to having per-buffer
iterators anyway: if two iterations can be active at the same time,
then there can be cases where those two iterations happen
to operate on the same buffer :-(

* src/itree.h (struct interval_tree): Remove `iter` field.
* src/itree.c (interval_generator_destroy)
(interval_tree_iter_ensure_space): Delete functions.
(iter): New global variable.
(init_itree_null): Rename to `itree_init` and adjust all callers.
Initialize `iter` as well.
(interval_tree_create, interval_tree_init):
Don't initialize `iter` field any more.
(interval_tree_destroy): Don't destroy `iter` field any more.
(interval_tree_insert): Don't bother growing the iterator (it's grown
in `interval_stack_push_flagged` if needed anyway, and in any case
there's no `iter` here to grow any more).
(interval_tree_remove): Tweak assertion to be more precise and
self-evident.
(interval_tree_iter_start): Use the global `iter`.
(interval_generator_create): Make it work with a NULL argument.

mark_overlays: Use the normal ITREE_FOREACH

This commit basically reverts commit 5b954f8f9.  The problem of nested
iterations hasn't been fixed in the mean time, but since the GC can
run arbitrary ELisp code (via `post-gc-hook`), running the GC from
within an itree iteration is already unsafe anyway :-(

* src/alloc.c (mark_overlays): Delete function.
(mark_buffer): Use ITREE_FOREACH.

itree.c: Remove `tree` field from iterator

* src/itree.c (interval_generator_ensure_space, interval_generator_reset):
Inline and then delete functions.
(interval_tree_inherit_offset): Only take the tree's `otick` as arg.
Update all callers.
(struct interval_generator): Remove `tree` field, replace with a copy
of the tree's `otick`.
(interval_stack_push_flagged): The arg should be a real node.
(interval_tree_insert_gap): Prefer checking root==NULL rather than size==0.
Skip loop when tree is empty to avoid pushing&processing the NULL node.
(interval_tree_inherit_offset): Prefer parent==NULL rather than
node==root to avoid accessing the tree object.

Merge remote-tracking branch 'origin/feature/noverlay' into noverlay

New ITREE_FOREACH macro

* src/itree.h (interval_tree_iter_start): Adjust type.
(interval_tree_nodes): Delete declaration.
(ITREE_FOREACH, ITREE_FOREACH_ABORT, ITREE_FOREACH_NARROW): New macros.

* src/itree.c (interval_tree_contains, interval_tree_insert_gap):
Use the new ITREE_FOREACH macro.
(interval_tree_nodes): Delete function.
(interval_tree_iter_start): Return the iterator.
(interval_generator_next, interval_tree_destroy):
Don't accept a NULL arg any more.

* src/xdisp.c (load_overlay_strings, strings_with_newlines):
* src/textprop.c (get_char_property_and_overlay):
* src/buffer.c (copy_overlays, delete_all_overlays)
(set_overlays_multibyte, swap_buffer_overlays, overlays_in)
(next_overlay_change, previous_overlay_change, overlay_touches_p)
(overlay_strings, Foverlay_lists, report_overlay_modification)
(evaporate_overlays): Use the new ITREE_FOREACH macro.

* src/buffer.h (buffer_overlay_iter_start1)
(buffer_overlay_iter_start, buffer_overlay_iter_next)
(buffer_overlay_iter_finish, buffer_overlay_iter_narrow):
Delete declarations.

Make it compile with ITREE_DEBUG defined

* src/buffer.c (make_lispy_interval_node): Use make_fixnum.
(overlay_tree): Use ITREE_NULL.

itree.c: Improve division between tree and iterator

* src/buffer.c (delete_all_overlays): Add comment.

* src/itree.c (struct interval_generator): New fields `running`,
`file`, and `line` moved from `interval_tree`.
(interval_stack_push_flagged): Adjust comment to resolve a FIXME.
(interval_tree_clear): Replace assignment with an a
(interval_tree_iter_next): Delete function.
(interval_tree_clear): Don't set `iter_running` here any more.
(interval_generator_create): Set it here instead.
(interval_tree_iter_start): Fetch `iter` once and for all.
(interval_generator_narrow): Mark it as non-static.
(interval_tree_iter_next, interval_tree_iter_narrow):
Delete functions.  Inline their old bodies in the callers.
(interval_tree_iter_finish): Take the iter rather than
the whole tree.  Adjust all callers.
(interval_generator_next): Move `running `assertion here from
`interval_tree_iter_next`.

* src/buffer.h: Adjust accordingly.

* src/itree.h (struct interval_tree): Remove fields `iter_running`,
`file`, and `line`, moved to `interval_generator`.
(interval_generator_narrow): Replace `interval_tree_iter_narrow`.

Remove the per-tree null node

"make check" shows 0 unexpcted.

* src/itree.h (itree_null): Declare extern.
(ITREE_NULL): New macro
(struct interval_tree): Remove null member.
* src/alloc.c (mark_overlays): Use ITREE_NULL.
* src/itree.c: Use ITREE_NULL insteads of a tree's null.
* src/pdumper.c (dump_buffer): Use ITREE_NULL.

itree: Remove the `visited` flag from the tree nodes

These bits really belong in the "workstack" used within
`interval_generator_next`, so move them there.

* src/itree.c (nodeptr_and_flag): New type;
(struct interval_stack): Use it.
(make_nav, nav_nodeptr, nav_flag): New functions.
(interval_tree_insert_gap, interval_tree_delete_gap): Adjust accordingly.
(interval_generator_next): Stash the `visited` bit in the work stack
rather than inside the tree nodes.
(interval_stack_create, interval_stack_destroy, interval_stack_clear)
(interval_stack_ensure_space, interval_stack_push_flagged)
(interval_stack_push, interval_stack_pop): Move before first use.

* src/itree.h (struct interval_node): Remove `visited` field.
* src/pdumper.c (dump_interval_node): Adjust accordingly.

free_buffer_overlays: Move nearer to its sole caller

* src/buffer.c (free_buffer_overlays): Move from `buffer.h`.
* src/buffer.h (free_buffer_overlays): Move to `buffer.c`.

* src/itree.c (interval_tree_iter_narrow, interval_tree_iter_finish)
(interval_tree_iter_next): Prefer `eassert`.

* src/itree.c (interval_tree_iter_start): Improve error message

alloc.c: Avoid nested overlay iterations (bug#58158)

* src/alloc.c (mark_overlays): New function.
(mark_buffer): Use it instead of using the overlay iterator.

itree.[ch]: Add sanity checks, comments, and minor tweaks

* src/alloc.c (mark_overlay): Add sanity check.

* src/buffer.c (next_overlay_change, previous_overlay_change):
Tweak code to keep the same vars for the bounds.

* src/itree.c (interval_tree_clear, interval_tree_insert)
(interval_tree_remove, interval_tree_insert_fix, interval_tree_remove_fix):
Adjust to the `color` -> `red` change.
(interval_tree_clear): Prefer `true/false` for booleans.
(interval_generator_create): Use an actual `interval_tree_order` value
rather than 0.
(interval_generator_next): Simplify a tiny bit.  Add comment.
(interval_generator_narrow): Add sanity check.

* src/itree.h (struct interval_node): Replace `color` field with
boolean `red` field.
(enum interval_tree_order): Remove unused `ITREE_DEFLT_ORDER` value.

* src/pdumper.c (dump_interval_node): Adjust to the
`color` -> `red` change.

Merge remote-tracking branch 'origin/scratch/noverlay-wip' into noverlay

Avoid nested iteration over intervals (bug#58144)

* src/xdisp.c (strings_with_newlines): Call
buffer_overlay_iter_finish.

Add debugging help for nested iterators (nug#58144)

When starting an iteration, store __FILE__ and __LINE__ where this
happens in the interval_tree structure.

* src/buffer.h (buffer_overlay_iter_start): New macro adding __FILE
and __LINE__.
(buffer_overlay_iter_start1): Renamed from ..._start.
* src/itree.h (struct interval_tree): Add file and line info.
* src/itree.c: (interval_tree_contains, interval_tree_nodes,
interval_tree_insert_gap): Pass __FILE__ and __LINE__ to iter_start.
(interval_tree_iter_start): Record file and line info in tree.

Fix last change

Fix macOS build (bug#58108)

* src/itree.h (struct interval_tree): Rename member nil to null.
* src/itree.c: Use null instead of nil
* src/pdumper.c (dump_buffer): Use null instead of nil.
* src/itree.c: Fix copyright.
* src/itree.h: Fix copyright.

* src/buffer.c (overlays_in): Fix confusion Z-vs-ZV

This fixes test failures in `test-overlays-in-2` and `test-remove-overlays`.

Merge 'master' into noverlay

Merge remote-tracking branch 'refs/remotes/origin/master'

hideshow.el: Improve hs-toggle-hiding behavior

* lisp/progmodes/hideshow.el
(hs-find-block-beginning-match): New function to be used in
`hs-already-hidden-p'.
(hs-already-hidden-p): Add check if beginning of line is inside a
block.
(hs-toggle-hiding): Don't change to selected-window's buffer when
event arg is absent.

* test/lisp/progmodes/hideshow-tests.el
(hideshow-tests-with-temp-buffer-selected): New helper macro.
(hideshow-tests-make-event-at): New helper function.
(hideshow-already-hidden-p-1): New test.
(hideshow-toggle-hiding-1): New test.
(hideshow-mouse-toggle-hiding-1): New test (bug#52092).

Regenerated ldefs-boot.el

Remove "manual" package--builtin-versions updates

* lisp/emacs-lisp/nadvice.el:
* lisp/emacs-lisp/cl-generic.el: Don't push to
package--builtin-versions "manually", because loaddefs-gen does
this correctly now.

* lisp/net/tramp-docker.el: New file.

(event-start/end): Fix part of bug#52092

When synthesizing a posn for keyboard events, make sure the `posn-point`
i the same as `point`.

* lisp/subr.el (event--posn-at-point): New function.
(event-start, event-end): Use it.

Merge branch 'emacs-28'

* lisp/progmodes/hideshow.el (hs-toggle-hiding): Fix `interactive` form

This fixes the first part of bug#52092, which is a regression
introduced by commit d0e9113de97.

Make dired-do-rename refuse to rename . and ..

* lisp/dired-aux.el (dired-do-rename): Refuse to rename . and ..
(bug#52004).  This used to error out after renaming.

Add a new command vc-pull-and-push

* lisp/vc/vc-svn.el (vc-exec-after):
* lisp/vc/vc-hg.el (vc-exec-after):
* lisp/vc/vc-git.el (vc-exec-after):
* lisp/vc/vc-cvs.el (vc-exec-after):
* lisp/vc/vc-bzr.el (vc-exec-after):
* lisp/org/org-macro.el (vc-exec-after):
* lisp/obsolete/vc-mtn.el (vc-exec-after):
* lisp/obsolete/vc-arch.el (vc-exec-after): Update declaration.
* lisp/vc/vc-dispatcher.el (vc--process-sentinel): Allow running
code only on success.
(vc-exec-after): Ditto.
(vc--inhibit-change-window-start): New variable.
(vc-do-async-command): Use it to allow chaining commands without
moving window point.  Return the process instead of the buffer,
since the process may have exited already, and then we can't get
at the process.

* lisp/vc/vc-git.el (vc-git--pushpull): Return the process object.
(vc-git-pull-and-push): New function.

* lisp/vc/vc.el (vc-pull-and-push): New command (bug#51964).

Tweak binding of `+' in vc-dir

* lisp/vc/vc-dir.el (vc-dir-mode-map): Point to the actual
function instead of the alias.  This makes *Help* more helpful.

CC Mode.  Make AWK Mode work with electric-pair-mode

Also fix some minor anomalies with other CC Mode modes in electric-pair-mode.
This fixes bug #53446.

* lisp/progmodes/cc-mode.el (c-open-string-opener): New buffer local
variable.
(c-after-change-mark-abnormal-strings): Set c-open-string-opener when an
unbalanced string is detected.
(c-before-change): Initilize c-open-string-opener to nil, each buffer change.
(c-electric-pair-inhibit-predicate): Use the value of c-open-string-opener to
flag an unbalaced string rather than trying to calculate it again.

* lisp/progmodes/cc-awk.el (c-awk-syntax-tablify-string): Set
c-open-string-opener when an unbalanced string is detected.

docview: Use svg images when using mupdf for conversion

* lisp/doc-view.el (doc-view-mupdf-use-svg, doc-view-svg-background)
(doc-view-svg-foreground): New user options.
(doc-view-insert-image): Add :background and :foreground image
attributes when display svg images.
(doc-view-set-up-single-converter): Produce svg images when using
mupdf (bug#58041).

image-dired: Prefer command remapping

* lisp/image/image-dired.el (image-dired-thumbnail-mode-map):
Prefer command remapping to binding keys directly.

image-dired: Fix C-a/C-e in thumbnail buffer

* lisp/image/image-dired.el
(image-dired--movement-ensure-point-pos): New function, and...
(image-dired--movement-command): ...new macros broken out from...
(image-dired-next-line, image-dired-previous-line)
(image-dired-beginning-of-buffer, image-dired-end-of-buffer):
...here.
(image-dired-move-beginning-of-line)
(image-dired-move-end-of-line): New commands.
(image-dired-thumbnail-mode-map): Remap 'move-beginning-of-line'
and 'move-beginning-of-line' to above new commands.

Make thumbs.el obsolete

* lisp/obsolete/thumbs.el: Add Obsolete-since.
* etc/NEWS: Announce obsoletion of thumbs.el.  (Bug#57779)

Move thumbs.el to lisp/obsolete

* lisp/thumbs.el:
* test/lisp/thumbs-tests.el: Move from here...
* lisp/obsolete/thumbs.el:
* test/lisp/obsolete/thumbs-tests.el: ...to here.  (Bug#57779)

image-dired: Regenerate old thumbnails when needed

* lisp/image/image-dired.el
(image-dired--get-create-thumbnail-file): Rename from
'image-dired-get-thumbnail-image'.  Update callers and make old
name into an obsolete alias.
(image-dired-display-thumbs): Use above function to simplify and
regenerate the thumbnail if it's outdated.

image-dired: Improve performance of marking commands

* lisp/image/image-dired.el (image-dired-thumbnail-storage): Mark
(image-dired--thumb-update-mark-at-point): New function.
(image-dired--thumb-update-marks):
(image-dired--do-mark-command):  Use it.
(image-dired--thumb-update-marks): Rename from
'image-dired-thumb-update-marks'.  Update all callers and make the
old name into an obsolete alias.

image-dired: Mark 'per-directory' as :safe

* lisp/image/image-dired.el (image-dired-thumbnail-storage): Mark
'per-directory' value as :safe.

Add new macro image-dired--with-dired-buffer

* lisp/image/image-dired-util.el
(image-dired--with-dired-buffer): New macro.
* lisp/image/image-dired.el (image-dired-track-original-file)
(image-dired--on-file-in-dired-buffer)
(image-dired--do-mark-command)
(image-dired--on-file-in-dired-buffer)
(image-dired-jump-original-dired-buffer)
(image-dired-thumb-file-marked-p, image-dired-delete-marked):
Use above new macro to simplify and improve error handling.
(image-dired-show-all-from-dir): Simplify removing Dired marks.
(image-dired-thumb-update-marks): Avoid triggering above new error
handling.

Tweak how :override advice is formatted in *Help*

* lisp/emacs-lisp/nadvice.el (advice--make-single-doc): Format
:overrides specially (bug#57974).
(advice--make-docstring): Put overrides at the front.

Rename file-name-directory

* lisp/emacs-lisp/shortdoc.el (file-name):
* doc/lispref/files.texi (Directory Names): Adjust.
* lisp/files.el (file-name-parent-directory): Rename from
`file-name-directory' (bug#58039).

Ensure that cus-load doesn't add things twice

* lisp/cus-dep.el (custom-make-dependencies): Use it.

* lisp/custom.el (custom--add-custom-loads): New function.

`append' doc string clarification

* src/fns.c (Fappend): Clarify whether arguments are copied.

Clarify message-newline-and-reformat doc string

* lisp/gnus/message.el (message-newline-and-reformat): Clarify the
DWIM-ness in the doc string (bug#57986).

; * lisp/image/image-dired.el: Improve sectioning.

; Make 'M-x image-dired' prompt even clearer

* lisp/image/image-dired.el (image-dired-show-all-from-dir):
Make prompt even clearer.
Suggested by Eli Zaretskii <eliz@gnu.org>.

Improve image-dired-thumbnail-display-external

* lisp/image/image-dired.el: (image-dired-external-viewer): Add more
image viewers and slightly re-arrange.
(image-dired-thumbnail-display-external): Correctly handle
external viewers with spaces or flags.

* lisp/emacs-lisp/shortdoc.el (file-name): Improve examples.

image-dired: Use command substitution for tags bindings

* lisp/image/image-dired-tags.el
(image-dired-dired-edit-comment-and-tags): Use substitute-command-keys

image-dired: Various code clean ups

* lisp/image/image-dired.el
(image-dired--display-thumb-properties-fun): Delete function.
(image-dired-forward-image): Immediately update header line.  Improve
docstring.
(image-dired-backward-image, image-dired-next-line)
(image-dired-previous-line, image-dired-beginning-of-buffer)
(image-dired-end-of-buffer, image-dired-display-image): Improve
docstring.
(image-dired-comment-thumbnail): Fix interactive declaration.

Reduce complexity of scroll bar window protection code

It turns out my previous theories of why ClientMessages were
generated were wrong, and they are just generated so we can set
finish to X_EVENT_GOTO_OUT to have them delivered immediately.

This allows for the code to be simplified greatly, by not
unpacking pointers to the raw window into the client messages.

* src/xterm.c (x_unprotect_window_for_callback): Return the
window removed from the queue, or nil if none.
(x_scroll_bar_to_input_event)
(x_horizontal_scroll_bar_to_input_event): Use window provided by
x_unprotect_window_for_callback.
(handle_one_xevent): Pass dpyinfo to functions that need it.
(x_free_frame_resources): Remove "first scroll bar request"
code.

* src/xterm.h (struct x_display_info): Likewise.

New Flymake backend using the shellcheck program

See bug#57884.

* lisp/progmodes/sh-script.el: Require let-alist and subr-x when
compiling.
(sh--json-read): Helper function to deal with possible absence of
json-parse-buffer.
(sh-shellcheck-program, sh--shellcheck-process,
sh-shellcheck-flymake): Variables and function defining a Flymake
backend.
(sh-mode): Add it to 'flymake-diagnostic-functions'.

; * src/keyboard.c (Fposn_at_point): Doc fix.  (Bug#52092)

vc-git--pushpull: Restore handling of vc-git-program

* lisp/vc/vc-git.el (vc-git--pushpull): Restore handling of
vc-git-program before recent change: respect a buffer-local value of
vc-git-program, and don't ignore user edits to the git program name
when PROMPT.

* lisp/emacs-lisp/eieio.el (defclass): Fix bug#51068

Accept (defclass <class> <superclasses> (.. <slotname> ..)) without
having to wrap the slot name within parentheses.

image-dired: Rewrite and extend slideshow feature

* lisp/image/image-dired.el
(image-dired--slideshow-start-timer)
(image-dired--slideshow-stop-timer)
(image-dired--slideshow-show-message): New functions.
(image-dired--slideshow-current-delay): New variable.
(image-dired--slideshow-initial): Delete variable.
(image-dired-slideshow-start): Simplify and ensure we display the
image at start.
* lisp/image/image-dired.el (image-dired--slideshow-stop): Add support
for pausing, and going backwards and forwards during slideshow.

image-dired: Mark two slideshow defuns as internal

* lisp/image/image-dired.el (image-dired--slideshow-step)
(image-dired--slideshow-stop): Rename from
'image-dired--slideshow-step' and 'image-dired--slideshow-stop'.
Update callers and make old names into obsolete aliases.

cconv.el: Fix interactive closure bug#51695

Make cconv.el detect when a closure's interactive form needs to
capture variables from the context and tweak the code accordingly
if so.

* lisp/emacs-lisp/cconv.el (cconv--interactive-form-funs): New var.
(cconv-convert): Handle the case where the interactive form captures
vars from the surrounding context.  Remove left over handling of
`declare` which was already removed from the cconv-analyze` phase.
(cconv-analyze-form): Adjust analysis of interactive forms accordingly.

* lisp/emacs-lisp/oclosure.el (cconv--interactive-helper): New type and
function.
* lisp/simple.el (function-documentation, oclosure-interactive-form):
Add methods for it.

* test/lisp/emacs-lisp/cconv-tests.el
(cconv-tests-interactive-closure-bug51695): New test.

Make Gnus respect mode-line-buffer-identification-keymap

* lisp/gnus/gnus-group.el (gnus-group-set-mode-line): Use
'propertized-buffer-identification' to buttonise and fontify the
buffer name.
* lisp/gnus/gnus-sum.el (gnus-set-mode-line):
* lisp/gnus/gnus.el (gnus-mode-line-buffer-identification): Adjust
calls to above.
* lisp/gnus/gnus-srvr.el (gnus-browse-foreign-server): Add missing
'gnus-modeline-buffer-identification' call (bug#57977).

Make bounding box of 'image-crop' more noticeable

* lisp/image/image-crop.el (image-crop--crop-image-1): Darken the
selected region to make the bounding-box more noticable in images
which are mostly white (bug#58004).

Don't overwrite cus-load dependencies

* lisp/cus-dep.el (custom-make-dependencies): Don't overwrite
elements added by packages (bug#58015).

Make loaddefs-gen register parent :groups from defcustom

* lisp/emacs-lisp/loaddefs-gen.el
(loaddefs-generate--make-autoload): Also register parent :groups from
`defgroup' entries (bug#58015).

Fix syntax check in python-info-looking-at-beginning-of-defun

* lisp/progmodes/python.el
(python-info-looking-at-beginning-of-defun): Check syntax after moving
to the beginning of line.
* test/lisp/progmodes/python-tests.el (python-nav-beginning-of-defun-6)
(python-end-of-defun-1, python-info-looking-at-beginning-of-defun-3):
New tests (bug#58023).