Optimize UCS normalization tests

Brings the the time for `ucs-normalize-part1' from 200s down to 130s.
* test/lisp/international/ucs-normalize-tests.el
(ucs-normalize-tests--parse-column): Use character instead of string
of length 1 for terminator.  Convert return value into string since
all callers need that form anyway.
(ucs-normalize-tests--normalization-equal-p): Rename from
ucs-normalize-tests--normalize.  Use dedicated buffer instead of
messing with narrowing.  Take string to compare against and insert it
into buffer so that compare-buffer-substrings can be used instead of
allocating a new string from buffer contents.
(ucs-normalize-tests--normalization-chareq-p): New macro, specialized
for comparing single character.
(ucs-normalize-tests--rule1-holds-p)
(ucs-normalize-tests--rule2-holds-p): Turn into defsubst.
(ucs-normalize-tests--rule1-failing-for-partX): Use `eq' instead of
`='.

Update failing lines for UCS normalize tests

* test/lisp/international/ucs-normalize-tests.el
(ucs-normalize-tests--failing-lines-part2): Update for new
admin/unidata/NormalizationTest.txt version.

Semi-automate the procedure for updating UCS normalize test bad lines

* test/lisp/international/ucs-normalize-tests.el: Remove incorrect
commentary describing a manual procedure for producing the updated
failing lines, it did not actually work.  Replace it with pointer to
new function which prints the updated values.
(ucs-normalize-tests--rule1-holds-p): Renamed from
ucs-normalize-tests--invariants-hold-p.
(ucs-normalize-tests--rule2-holds-p): Renamed from
ucs-normalize-tests--invariants-rule2-hold-p.
(ucs-normalize-tests--rule1-failing-for-partX): Renamed from
ucs-normalize-tests--invariants-failing-for-part.
(ucs-normalize-tests--rule1-failing-for-lines): Renamed from
ucs-normalize-tests--invariants-failing-for-lines.
(ucs-normalize-tests--part2-rule1-failed-lines): New variable.
(ucs-normalize-part2): Set it.
(ucs-normalize-part1): Always run through to end of test before
checking for failures.
(ucs-normalize-tests--insert-failing-lines)
(ucs-normalize-check-failing-lines): New functions, used to update
the *--failing-lines-part* variables.

Update Unicode data and files to Unicode 10.0

* admin/notes/unicode:
* admin/unidata/README:
* admin/unidata/BidiBrackets.txt:
* admin/unidata/BidiMirroring.txt:
* admin/unidata/Blocks.txt:
* admin/unidata/IVD_Sequences.txt:
* admin/unidata/NormalizationTest.txt:
* admin/unidata/SpecialCasing.txt:
* admin/unidata/UnicodeData.txt:
* lisp/international/characters.el:
* lisp/international/fontset.el (script-representative-chars):
* lisp/international/mule-cmds.el (ucs-names): Update per Unicode 10.0.

Support '=' in Scheme and Lisp tags in 'etags'

* lib-src/etags.c (get_lispy_tag): New function.
(L_getit, Scheme_functions): Use get_lispy_tag (Bug#5624).
* test/manual/etags/CTAGS.good:
* test/manual/etags/ETAGS.good_1:
* test/manual/etags/ETAGS.good_2:
* test/manual/etags/ETAGS.good_3:
* test/manual/etags/ETAGS.good_4:
* test/manual/etags/ETAGS.good_5:
* test/manual/etags/ETAGS.good_6:
* test/manual/etags/Makefile:
* test/manual/etags/el-src/TAGTEST.EL: Update tests.
* test/manual/etags/scm-src/test.scm: New tests for Scheme.

Avoid compiler warnings in xdisp.c debugging code

* src/xdisp.c (dump_glyph, dump_glyph_row, Fdump_glyph_matrix):
Use pD directives for ptrdiff_t values instead of pI, to avoid
compilation warnings on 64-bit hosts.  (Bug#27597)

Commentary enhancement in frame.el

* lisp/frame.el: Explain how to test whether a variable needs to
be added to the list of those which are watched for buffer
redisplay.

Support display of line numbers natively

This merges branch 'line-numbers'.
* src/buffer.c (disable_line_numbers_overlay_at_eob): New
function.
* src/lisp.h (disable_line_numbers_overlay_at_eob): Add prototype.
* src/dispextern.h (struct it): New members pt_lnum, lnum,
lnum_bytepos, lnum_width, and lnum_pixel_width.
* src/indent.c (line_number_display_width): New function,
refactored from line-number width calculations in vertical-motion.
(Fvertical_motion): Call line_number_display_width when the width
of line-number display is needed.
(Fline_number_display_width): New defun.
(syms_of_indent): Defsubr it.
* src/indent.c (Fvertical_motion): Help C-n/C-p estimate correctly
the width used up by line numbers by looking near the window-start
point.  If window-start is outside of the accessible portion,
temporarily widen the buffer.
* src/term.c (produce_glyphs): Adjust tab stops for the horizontal
space taken by the line-number display.
* src/xdisp.c (display_count_lines_logically)
(display_count_lines_visually, maybe_produce_line_number)
(should_produce_line_number, row_text_area_empty): New functions.
(try_window_reusing_current_matrix): Don't use this method when
display-line-numbers is in effect.
(try_window_id, try_cursor_movement): Disable these optimizations
when the line-number-current-line face is different from
line-number face and for relative line numbers.
(try_window_id, redisplay_window, try_cursor_movement): For
visual line-number display, disable the same redisplay
optimizations as for relative.
(x_produce_glyphs): Adjust tab stops for the horizontal
space taken by the line-number display.
(hscroll_window_tree): Adjust hscroll calculations to line-number
display.
(DISP_INFINITY): Renamed from INFINITY to avoid clashes with
math.h; all users changed.
(set_cursor_from_row): Fix calculation of cursor X coordinate in
R2L rows with display-produced glyphs at the beginning.
(display_line): Use should_produce_line_number to determine
whether a line number should be produced for each glyph row, and
maybe_produce_line_number to produce line numbers.
Don't display line numbers in the minibuffer and in tooltip
frames.
Call row_text_area_empty to verify that a glyph
row's text area is devoid of any glyphs that came from a buffer or
a string.  This fixes a bug with empty-lines indication
disappearing when line numbers or line-prefix are displayed.
(syms_of_xdisp) <display-line-numbers, display-line-numbers-widen>
<display-line-number-width>: New buffer-local variables.
<display-line-numbers-current-absolute>: New variable.

* lisp/cus-start.el (standard): Provide customization forms for
display-line-numbers and its sub-features.
* lisp/faces.el (line-number, line-number-current-line): New faces.
* lisp/frame.el: Add display-line-numbers, display-line-numbers-widen,
display-line-numbers-current-absolute, and
display-line-number-width to the list of variables that should
trigger redisplay of the current buffer.
* lisp/menu-bar.el (menu-bar-showhide-menu): Add menu-bar item to
turn display-line-numbers on and off.
(toggle-display-line-numbers): New function.
* lisp/simple.el (last--line-number-width): New internal variable.
(line-move-visual): Use it to adjust temporary-goal-column when
line-number display changes its width.

* doc/emacs/basic.texi (Position Info): Add cross-reference to
"Display Custom", for line-number display.
* doc/emacs/custom.texi (Init Rebinding):
* doc/emacs/modes.texi (Minor Modes): Remove references to
linum-mode.
* doc/emacs/display.texi (Display Custom): Describe the
line-number display.
* doc/lispref/display.texi (Size of Displayed Text): Document
line-number-display-width.

* etc/NEWS: Document display-line-numbers and its customizations.

Fix more ungetc bugs with encoding errors

* src/lread.c (infile): New variable, replacing ...
(instream): ... this.  All uses changed.
(readbyte_from_stdio): New function, which deals with lookahead.
(readbyte_from_file, Fget_file_char): Use it.
(Fget_file_char): When misused, signal an error instead of
relying on undefined behavior.
(close_infile_unwind): New function.
(Fload): Use it.
(readevalloop): 2nd arg is now struct infile *, not FILE *.
All callers changed.
(read1): Handle lookahead when copying doc strings with
encoding errors.

Avoid ungetc when loading charset maps from files

* src/charset.c (read_hex): New args LOOKAHEAD and TERMINATOR,
replacing the old EOF.  All callers changed.  This avoids the
need to call ungetc.

Fix ungetc bug when reading an encoding error

* src/lread.c (readchar, read_emacs_mule_char): Fix off-by-one
error when reading an encoding error from a file, e.g., a symbol
in an .elc file whose name is "\360\220\200\360".

; Fix extra "," in previous commit

* lisp/wid-edit.el (widget-color--choose-action): Use a closure

* lisp/window.el (display-buffer--special-action): Use a closure.

Add new todo-mode.el tests

* test/lisp/calendar/todo-mode-tests.el (with-todo-test):
Declare an Edebug spec.  Restore pre-test-run state of test files.
(todo-test--show, todo-test--move-item)
(todo-test--insert-item): New functions.
(todo-test-get-archive): Remove, as subsumed by
todo-test--show.  Adjust all callers.
(todo-test--is-current-buffer): Rename from
todo-test-is-current-buffer and adjust uses.
(todo-test-item-highlighting): Use todo-test--show.
(todo-test-revert-buffer01, todo-test-revert-buffer02)
(todo-test-raise-lower-priority)
(todo-test-todo-mark-unmark-category, todo-test-move-item01)
(todo-test-move-item02, todo-test-move-item03)
(todo-test-move-item04, todo-test-move-item05)
(todo-test-toggle-item-header01)
(todo-test-toggle-item-header02)
(todo-test-toggle-item-header03)
(todo-test-toggle-item-header04)
(todo-test-toggle-item-header05)
(todo-test-toggle-item-header06)
(todo-test-toggle-item-header07): New tests.

* test/lisp/calendar/todo-mode-resources/todo-test-1.toda:
* test/lisp/calendar/todo-mode-resources/todo-test-1.todo:
Modify to accommodate new tests.

todo-mode.el: Fix handling of hidden item headers (bug#27609)

* lisp/calendar/todo-mode.el (todo--item-headers-hidden): New variable.
(todo-toggle-item-header): Use it.  Make this command a noop
if the file has no items.
(todo-move-item, todo-item-done): Instead of concatenating the
items to move into one string, make a list of them to
facilitate handling hidden headers.  Adjust insertion accordingly.
(todo-archive-done-item): Handle hidden headers in archive file.
(todo-unarchive-items): Handle hidden headers in todo file.
(todo-backward-item): Use todo--item-headers-hidden and handle
moving backward work when item date-time headers are hidden.
(todo-remove-item): Delete date-time header overlay.
(todo-get-overlay, todo-insert-with-overlays): Make them work
with hidden date-time headers.
(todo-modes-set-2): Make todo--item-headers-hidden buffer local.

Fix several todo-mode bugs found while debugging bug#27609

* lisp/calendar/todo-mode.el (todo-toggle-mark-item): Calculate
current category only once.
(todo-mark-category): Update number of marked items to avoid
spurious duplication in todo-categories-with-marks alist and
corruption of the todo-categories alist.  Handle empty line
when there are no todo items and done items are shown.
(todo-set-item-priority): Make noop if called from
todo-raise-item-priority or todo-lower-item-priority when
point is on a done todo item or an empty line.
(todo-move-item): Use markers instead of integer positions to
correctly handle deleting the now moved items from the source
category (without markers an infinite loop arises when moving
marked item to a preceding category).
(todo-unarchive-items): Put point on the (first) restored done
item, instead of leaving it at the end of the done items
separator string.
(todo-revert-buffer): Ensure buffer remains read-only after
reverting.

Exclude blank columns from value of line-number-display-width

* src/indent.c (Fline_number_display_width): Don't add 2 to the
number of columns we return, to make this consistent with
display-line-number-width.

Fix vertical-motion across the place where line-number width changes

* src/indent.c (line_number_display_width): New function,
refactored from line-number width calculations in vertical-motion.
(Fvertical_motion): Call line_number_display_width when the width
of line-number display is needed.
(Fline_number_display_width): New defun.
(syms_of_indent): Defsubr it.

* doc/lispref/display.texi (Size of Displayed Text): Document
line-number-display-width.

* etc/NEWS: Mention line-number-display-width.

* lisp/simple.el (last--line-number-width): New internal variable.
(line-move-visual): Use it to adjust temporary-goal-column when
line-number display changes its width.

Remove Vwindow_text_change_functions and related code

Vwindow_text_change_functions had been provided for implementing
line numbers but apparently was never functional or in use.

* src/xdisp.c (redisplay_window): Remove handling of
Vwindow_text_change_functions.
(syms_of_xdisp): Remove Qwindow_text_change_functions.
(Vwindow_text_change_functions): Remove variable.
* doc/lispref/hooks.texi (Standard Hooks): Remove entry for
`window-text-change-functions'.

Convert more uses of looking-at to following-char

More followup to Karl Fogel's commit a84da83c1.
* lisp/dired-aux.el (dired-add-entry, dired-subdir-hidden-p):
* lisp/dired-x.el (dired-mark-unmarked-files, dired-mark-sexp):
* lisp/help-fns.el (doc-file-to-man, doc-file-to-info):
* lisp/proced.el (proced-toggle-marks):
* lisp/progmodes/f90.el (f90-indent-line):
* lisp/ses.el (ses-load):
* lisp/tar-mode.el (tar-expunge): Replace instances of looking-at with
char comparisons using following-char.

Don't skip epg tests (Bug#23561)

* test/lisp/epg-tests.el (with-epg-tests): Ignore REQUIRE-PASSPHRASE
parameter, since we supply the passphrase via pinentry-program for all
GPG versions (as of 2017-02-28 "Fix epg-tests with dummy-pinentry
program (Bug#23619)").
(epg-tests-program-alist-for-passphrase-callback): Remove.

Implement line numbers that disregard narrowing

* src/xdisp.c (display_count_lines_logically): New function,
counts line numbers disregarding narrowing.  Suggested by Andy
Moreton <andrewjmoreton@gmail.com>.
(maybe_produce_line_number): Call display_count_lines_logically
instead of display_count_lines.  Adapt BEGV, ZV, etc. to
display-line-numbers-widen.
(syms_of_xdisp) <display-line-numbers-widen>: New buffer-local
variable.

* lisp/cus-start.el (standard): Provide a customization form for
display-line-numbers-widen.
* lisp/frame.el: Add display-line-numbers-widen,
display-line-numbers-current-absolute, and
display-line-number-width to the list of variables that should
trigger redisplay of the current buffer.

* doc/emacs/display.texi (Display Custom): Document
display-line-numbers-widen.

Fix lisp-comment-indent for single-semicolon case

* lisp/emacs-lisp/lisp-mode.el (lisp-comment-indent): Only check for
open paren if we're looking at multiple comment characters.
* test/lisp/emacs-lisp/lisp-mode-tests.el (lisp-comment-indent-1)
(lisp-comment-indent-2): New tests.

Spelling fixes

* lisp/org/org-table.el (org-table-sort-lines):
Fix misspelling in prompt.
* lisp/org/ox-ascii.el (org-ascii--describe-datum):
Fix misspelling in call to org-element-lineage.

Don't put whitespace between open paren and comment in Lisp modes (Bug#19740)

* lisp/emacs-lisp/lisp-mode.el (lisp-comment-indent): If current
line's code ends in open paren, set comment indentation exactly to
column following it.
(lisp-mode-variables): Set `comment-indent-function' to
`lisp-comment-indent'.

Allow comment-indent-functions to specify exact indentation (Bug#385)

* lisp/newcomment.el (comment-choose-indent): Interpret a cons of two
integers as indicating a range of acceptable indentation.
(comment-indent): Don't apply `comment-inline-offset',
`comment-choose-indent' already does that.
(comment-indent-function):
* doc/emacs/programs.texi (Options for Comments): Document new
acceptable return values.
* etc/NEWS: Announce it.

Check for integer overflow in xbm images

* src/image.c (XBM_TK_OVERFLOW): New constant.
(xbm_scan): Check for integer overflow instead of relying on
undefined behavior.  Check that octal digits are actually octal.

Convert hex digits more systematically

This makes the code a bit smaller and presumably faster, as
it substitutes a single lookup for conditional jumps.
* src/character.c (hexdigit): New constant.
(syms_of_character) [HEXDIGIT_IS_CONST]: Initialize it.
* src/character.h (HEXDIGIT_CONST, HEXDIGIT_IS_CONST): New macros.
(hexdigit): New decl.
(char_hexdigit): New inline function.
* src/charset.c: Do not include c-ctype.h.
* src/charset.c (read_hex):
* src/editfns.c (styled_format):
* src/image.c (xbm_scan):
* src/lread.c (read_escape):
* src/regex.c (ISXDIGIT) [emacs]:
Use char_hexdigit insted of doing it by hand.

Don’t use -Woverride-init

I have some further changes in mind that would also need to
disable the -Woverride-init warnings.  In practice these warnings
seem to be more trouble than they’re worth, so disable them in the
cc command line.
* configure.ac: Disable -Woverride-init here ...
* src/bytecode.c: ... rather than here.

* lisp/progmodes/python.el (auto-mode-alist): Add .pyi.  (Bug#27847)

; Standardize some copyright and license notices

* lisp/org/ox-html.el (org-html-infojs-template): Update copyright.

Small fix for bug-reference.el

* lisp/progmodes/bug-reference.el (bug-reference-bug-regexp):
Autoload safety property.  (Bug#27481)

Suppress timers in Tramp

* lisp/net/tramp.el (tramp-file-name-handler): Don't trigger timers.

* test/lisp/net/tramp-tests.el
(tramp-test36-asynchronous-requests): Trigger timers.
(tramp-test37-recursive-load, tramp-test38-remote-load-path):
Set `default-directory' to a trustworthy value.

Add realgud face definitions

Merge branch 'master' of git.sv.gnu.org:/srv/git/emacs

Add realgud faces to tdsh-dark-theme

* lisp/progmodes/cc-awk.el: Mark unused args

* lisp/progmodes/cc-bytecomp.el: Mark unused args

* lisp/progmodes/cc-defs.el (lookup-syntax-properties): Move ...

... before first use

* lisp/progmodes/cc-vars.el: Mark unused args

* lisp/progmodes/cc-langs.el: Mark unused args

(c-primary-expr-regexp): Remove unused vars ambiguous-prefix-ops and
unambiguous-prefix-ops.

* lisp/progmodes/cc-engine.el: Mark unused args

(c-beginning-of-statement-1, c-guess-basic-syntax):
Remove unused var c-in-literal-cache.
(c-debug-sws-msg): Silence byte-compiler, even if we don't use the arg.
(c-append-to-state-cache): Remove unused var `bra+1s'.
(c-remove-stale-state-cache): Remove unused var `pps-point-state'.
(c-invalidate-state-cache-1): Remove unused var `pa'.
(c-forward-decl-or-cast-1): Change comments so they don't look like
outline headers.
(c-restricted-<>-arglists, c-parse-and-markup-<>-arglists):
Declare before first use.
(c-forward-decl-or-cast-1): Remove unused var `backup-kwd-sym'.
(c-backward-over-enum-header): Remove unused var `up-sexp-pos'.

* lisp/progmodes/cc-cmds.el: Remove unused vars

(c-syntactic-context): Declare as dynbound.
(c-beginning-of-defun, c-end-of-defun): Remove unused var `start'.

* lisp/progmodes/cc-guess.el: Remove unused var

(c-guess-view-reorder-offsets-alist-in-style): Remove redundantly bound
and computed variable `guessed-syntactic-symbols'.

* lisp/progmodes/cc-align.el: Mark unused arguments

* lisp/progmodes/cc-mode.el: Mark unused arguments

(c-parse-quotes-before-change, c-parse-quotes-after-change):
Remove unused vars.

Mention `ffap-url-unwrap-local' in find-file-at-point's docstring (Bug#27564)

* lisp/ffap.el (find-file-at-point): Mention `ffap-url-unwrap-local'
and `ffap-url-unwrap-remote'.

Fix infloop in uncomment-region-default (Bug#27112)

When `comment-continue' has only blanks, `comment-padright' produces a
regexp that matches the empty string, so `uncomment-region-default'
will loop infinitely.
* lisp/newcomment.el (comment-padright): Only return a regexp if STR
has nonblank characters.

Fix display of current line number in visual mode

* src/xdisp.c (maybe_produce_line_number): Fix visual-mode display
of current line when line-number-current-line face was customized.
Reported by Filipe Silva <filipe.silva@gmail.com>.

; Remove unused text properties in test

* test/lisp/electric-tests.el (electric-quote-markdown-in-code):
Remove now-unused text properties.

Use hook instead of face list to inhibit electric quoting

This is more flexible and doesn't couple electric quoting to font
locking.
Give that 'electric-quote-code-faces' was just introduced, remove it
without formal deprecation.

* lisp/electric.el (electric-quote-inhibit-functions): New abnormal
hook variable.
(electric-quote-post-self-insert-function): Run the hook.  Remove
use of old 'electric-quote-code-faces' variable.

* test/lisp/electric-tests.el (electric-quote-markdown-in-text)
(electric-quote-markdown-in-code): Adapt unit tests.

Offer non-aligned indentation in lists in js-mode (Bug#27503)

* lisp/progmodes/js.el (js--proper-indentation):
New customization option 'js-indent-align-list-continuation'.
Affects argument lists as well as arrays and object properties.
* test/manual/indent/js-indent-align-list-continuation-nil.js:
Test the change.

Avoid errors in vertical-motion when buffer is narrowed

* src/indent.c (Fvertical_motion): If need to start from
window-start, and it is outside of the accessible portion,
temporarily widen the buffer.  This avoids errors in evil-mode.
Reported by James Nguyen <james@jojojames.com>.

(Re-)activate remote tests of filenotify-tests.el

* test/lisp/filenotify-tests.el
(file-notify-test-remote-temporary-file-directory):
Declare default host for mock method.  Offer home directory
for mock method if it doesn't exist.
(file-notify-test09-watched-file-in-watched-dir-remote):
Remove, it doesn't work reliably.

Reset ansi escape context before printing eshell prompt (Bug#27407)

* lisp/eshell/em-prompt.el (eshell-emit-prompt): Reset
`ansi-color-context-region'.

Let ansi-color overlay hooks work in eshell (Bug#27407)

* lisp/ansi-color.el (ansi-color-make-extent): Add
`ansi-color-freeze-overlay' to `insert-behind-hooks' as well.
* lisp/eshell/esh-mode.el (eshell-output-filter): Let-bind
`inhibit-modification-hooks' to nil while inserting the string.

Fix and simplify ansi escape detection (Bug#21381)

* lisp/ansi-color.el (ansi-color-regexp, ansi-color-drop-regexp):
Remove.
(ansi-color-control-seq-regexp): New constant, matches all escape
sequences.
(ansi-color-filter-apply, ansi-color-apply)
(ansi-color-filter-region, ansi-color-apply-on-region): Use it instead
of matching color sequences separately from ignored sequences.
Differentiate color sequences simply by checking the last character.

Add absolute optional parameter to line-number-at-pos (Bug#26417)

* lisp/simple.el (line-number-at-pos): Add a second optional
  argument 'absolute'.
* test/list/simple-tests.el: Add tests for 'line-number-at-pos'.

Add realgud faces

Fix tramp-tests.el for hydra

* test/Makefile.in: Remove instrumentation for tramp-tests.

* test/lisp/net/tramp-tests.el (tramp-test36-asynchronous-requests):
Remove instrumentation.  Wrap with a timeout.  Give hydra
another timer value.  Set `default-directory' in timer.

; Instrument tramp-tests.el

Merge branch 'master' into scratch/org-mode-merge

Merge branch 'master' into scratch/org-mode-merge

dired-do-shell-command: Fix check for wildcards

* lisp/dired-aux.el (dired-do-shell-command): Replace just '?', '*'
and '`?' i.e., keep the whitespaces.
* test/lisp/dired-aux-tests.el (dired-test-bug27496): Add test.

Split shr-copy-url dwim behavior into separate functions (Bug#26826)

* lisp/net/shr.el (shr-url-at-point, shr-probe-url)
(shr-probe-and-copy-url, shr-maybe-probe-and-copy-url): New functions,
split out from `shr-copy-url'.
(shr-copy-url): Only copy the url, don't fetch it.
(shr-map): Bind 'w' and 'u' to `shr-maybe-probe-and-copy-url', which
has the same behavior as the old `shr-copy-url'.
* etc/NEWS: Announce changes.

Make eww-search-words prompt for query if nothing selected

* lisp/net/eww.el (eww-search-words): Make eww-search-words prompt the
user for a search query if the region is inactive or if the region is
just whitespace.

Copyright-paperwork-exempt: yes

* lisp/emacs-lisp/cl-print.el (cl-print-compiled-button): t by default.

* lisp/emacs-lisp/debug.el (debugger-insert-backtrace):
* lisp/help-fns.el (describe-variable): No need to let-bind
`cl-print-compiled-button' to t anymore.

Electric quotes: Improve support for Markdown mode (Bug#24709)

Introduce a new user option 'electric-quote-context-sensitive'.  If
non-nil, have ' insert an opening quote if sensible.

Also introduce a new variable 'electric-quote-code-faces'.  Major
modes such as 'markdown-mode' can add faces to this list to treat text
as inline code and disable electric quoting.

* lisp/electric.el (electric-quote-context-sensitive): New user
option.
(electric-quote-code-faces): New variable.
(electric-quote-post-self-insert-function): Treat ' as ` if
desired and applicable; disable electric quoting for given faces.

* test/lisp/electric-tests.el (electric-quote-opening-single)
(electric-quote-closing-single, electric-quote-opening-double)
(electric-quote-closing-double)
(electric-quote-context-sensitive-backtick)
(electric-quote-context-sensitive-bob-single)
(electric-quote-context-sensitive-bob-double)
(electric-quote-context-sensitive-bol-single)
(electric-quote-context-sensitive-bol-double)
(electric-quote-context-sensitive-after-space-single)
(electric-quote-context-sensitive-after-space-double)
(electric-quote-context-sensitive-after-letter-single)
(electric-quote-context-sensitive-after-letter-double)
(electric-quote-context-sensitive-after-paren-single)
(electric-quote-context-sensitive-after-paren-double)
(electric-quote-markdown-in-text)
(electric-quote-markdown-in-code): New unit tests.

* doc/misc/tramp.texi: Replace ftp:// and http:// URLs by https://.

Avoid off-by-one errors in column C-n/C-p calculations

* src/indent.c (Fvertical_motion): Help C-n/C-p estimate correctly
the width used up by line numbers by looking near the window-start
point.

Ask confirmation for all suspicious wildcards

* lisp/dired-aux.el (dired-do-shell-command): Check that all
the wildcards are right.  Otherwise, ask for confirmation (Bug#27496).

Extend dired-do-shell-command substitutions

Substitute "`?`" inside command with the current file name.
See details in:
https://lists.gnu.org/archive/html/emacs-devel/2017-06/msg00618.html
* lisp/dired-aux.el (dired-quark-subst-regexp, dired-star-subst-regexp):
Mark as obsolete.
(dired-isolated-string-re): New defun.
(dired--star-or-qmark-p): New predicate.
(dired-do-shell-command): Use dired--star-or-qmark-p.  Substitute "`?`"
with the current file name.
* doc/emacs/dired.texi (Shell Commands in Dired): Update manual.
; * etc/NEWS: Mention it.

Fix bug in yesterday's CC Mode commit.

* lisp/progmodes/cc-mode.el (c-quoted-number-head-before-point): Check a
search has succeded before using the match data.
(c-quoted-number-head-before-point, c-quoted-number-head-after-point):
Specify that the position of the extremity of the head or tail is in the
match data.

Remove FIXME comments about sentinel values

These FIXMEs can't be addressed because they would require breaking
changes to the module API.  Furthermore, other module functions don't
return sentinel values as well, so users generally have to call
non_local_exit_check anyway.

* src/emacs-module.c (module_set_user_ptr)
(module_set_user_finalizer, module_vec_set, module_vec_size): Remove
FIXME comments.

Adapt Lisp reference to reader changes

The reader now warns about some unescaped character literals, but
still allows them for compatibility reasons.  Slightly adapt the
manual to forbid them officially.

* doc/lispref/objects.texi (Basic Char Syntax): Document that
backslashes are now required before some characters.

; Instrument tramp-tests.el

Fix Bug#27502

* lisp/autorevert.el (auto-revert-find-file-function): New defun.
(find-file-hook): Use it.  (Bug#27502)
(auto-revert-remove-current-buffer): New defun.
(auto-revert-mode, auto-revert-buffers): Use it.

; Instrument tramp-tests.el

Let test summary go through even if some logs were not generated

* lisp/emacs-lisp/ert.el (ert-summarize-tests-batch-and-exit): Check
for existence of log files before reading.

* src/module-env-25.h (copy_string_contents): Fix comment.

Also mark module init function as noexcept if possible

* src/emacs-module.h.in (emacs_module_init): Mark as noexcept if
possible.

Improve C++98 compatibility

* src/emacs-module.h.in (emacs_funcall_exit): Lose trailing comma.
C++98 doesn't allow trailing commas in enumerations.

Minor copyedits of manuals regarding bidi conformance

* doc/emacs/mule.texi (Bidirectional Editing):
* doc/lispref/display.texi (Bidirectional Display): Update the
bidi conformance text.

Make C++ digit separators work.  Amend the handling of single quotes generally

Single quotes, even in strings and comments, are now marked with the
"punctuation" syntax-table property, except where they are validly bounding a
character literal.  They are font locked with font-lock-warning-face except
where they are valid.  This is done in C, C++, ObjC, and Java Modes.

* lisp/progmodes/cc-defs.el (c-clear-char-property-with-value-on-char-function)
(c-clear-char-property-with-value-on-char, c-put-char-properties-on-char): New
functions/macros.

* lisp/progmodes/cc-fonts.el (c-font-lock-invalid-single-quotes): New function.
(c-basic-matchers-before): invoke c-font-lock-invalid-single-quotes.

* lisp/progmodes/cc-langs.el (c-get-state-before-change-functions): Remove
c-before-after-change-digit-quote from wherever it occurs.  Insert
c-parse-quotes-before-change into the entries for the languages where it is
needed.
(c-before-font-lock-functions): Remove c-before-after-change-digit-quote from
wherever it occurs.  Insert c-parse-quotes-after-change into the entries for
the languages which need it.
(c-has-quoted-numbers): New lang-defconst/-defvar.

* lisp/progmodes/cc-mode.el (c-before-after-change-digit-quote): Remove.
(c-maybe-quoted-number-head, c-maybe-quoted-number-tail)
(c-maybe-quoted-number): New defconsts.
(c-quoted-number-head-before-point, c-quoted-number-tail-after-point)
(c-quoted-number-straddling-point, c-parse-quotes-before-change)
(c-parse-quotes-after-change): New functions.

Ignore mouse-movement for describe-key-briefly (Bug#12204)

* lisp/help.el (help-read-key-sequence): Add optional argument ot
ignore `mouse-movement' events.
(describe-key-briefly): Use it.
* doc/emacs/help.texi (Key Help):
* etc/NEWS: Mention that mouse movement is ignored.

Refactor key describing commands

* lisp/help.el (help-read-key-sequence, help--analyze-key): New
functions, extracted from `describe-key' and `describe-key-briefly'.
(describe-key, describe-key-briefly): Use them.

Improve display of tabs with line numbers

* src/xdisp.c (x_produce_glyphs): Improve calculation of next tab
stop in hscrolled lines.  Prevent aborts in compute_line_metrics.

Fix threads on NS (bug#25265)

src/nsterm.h (ns_select): Compiler doesn't like sigmask being const.
(ns_run_loop_break) [HAVE_PTHREAD]: New function.
src/nsterm.m (ns_select): Call thread_select from within ns_select.
(ns_run_loop_break) [HAVE_PTHREAD]: New function.
(ns_send_appdefined): Don't wait for main thread when sending app
defined event.
src/process.c (wait_reading_process_output): Call thread_select from
within ns_select.
src/systhread.c (sys_cond_broadcast) [HAVE_NS]: Break ns_select out of
its event loop using ns_run_loop_break.

; Auto-commit of loaddefs files.

; Fix last commit

; Instrument test/Makefile.in for tramp-tests output on hydra

Avoid slow redisplay under 'visual' mode of line numbers

* src/xdisp.c (display_count_lines_visually): Avoid very slow
redisplay when this function is invoked very far from point.
Reported by Alex <agrambot@gmail.com>.

* lisp/help-fns.el (describe-variable): Let-bind cl-print-compiled-button.

* lisp/emacs-lisp/cl-extra.el (cl--random-state): New defstruct

(cl--random-state, cl--random-time): Move from cl-lib.el.
(cl-random): Use struct accessors.
(cl-random-state-p): Remove, provided by the defstruct.
(cl-make-random-state): Rewrite to struct constructor.

* lisp/emacs-lisp/debug.el (debugger-list-functions): Remove obsolete msg

Improve documentation of faces related to display-line-numbers

* lisp/faces.el (line-number, line-number-current-line): Warn
against using non-monospaced fonts.

Fix relative-number display with non-nil display-line-number-width

* src/xdisp.c (maybe_produce_line_number): Don't treat a zero
value of display-line-number-width as acceptable.
Handle the case of 'relative' with display-line-number-width
non-nil and smaller than the absolute line number requires.
Reported by Alex <agrambot@gmail.com>.

Release Tramp 2.3.2

* doc/misc/tramp.texi (Android shell setup): Show default file name.
Structure section.

* doc/misc/trampver.texi:
* lisp/net/trampver.el: Change version to "2.3.2".

* test/lisp/net/tramp-tests.el (tramp-test-temporary-file-directory):
Offer home directory for mock method if it doesn't exist.