Support expiration of metadata by package archives

Expiring package metadata is done by checking the timestamp in package
archive file.  This is intended to limit the effectiveness of a replay
attack.  The onus is on the package archives to implement a secure and
reasonable policy.  (Debian uses 7 days before metadata expires.)

Together with package checksums, this adds sufficient protection
against metadata replay attacks.  (Bug#19479)

* lisp/emacs-lisp/package.el (package-check-timestamp): New defcustom.
(bad-timestamp): New error.
(package--parse-header-from-buffer)
(package--parse-valid-until-from-buffer)
(package--parse-last-updated-from-buffer)
(package--archive-verify-timestamp)
(package--archive-verify-not-expired)
(package--compare-archive-timestamps)
(package--check-archive-timestamp): New defuns.
(package--download-one-archive): Check timestamp of the
'archive-contents' file using above functions.  It is only checked if
it exists, which makes this change backwards compatible.

* lisp/calendar/iso8601.el (iso8601-parse): Add autoload cookie.

* test/lisp/emacs-lisp/package-tests.el
(package-test-parse-valid-until-from-buffer)
(package-test-parse-last-updated-from-buffer)
(package-test-archive-verify-timestamp)
(package-test-check-archive-timestamp)
(package-test-check-archive-timestamp/not-expired)
(package-test-check-archive-timestamp/expired): New tests.

* test/lisp/emacs-lisp/package-resources/archives/older/archive-contents:
* test/lisp/emacs-lisp/package-resources/archives/newer/archive-contents:
New files.

* doc/lispref/package.texi (Package Archives, Archive Web Server):
Document how to increase the security of a package archive using
checksums, signing and timestamps.

Support package checksum verification

Package checksum verification is the first step towards protecting
users of package.el against replay attacks.  Signing individual
packages still allows a hostile actor to distribute an out-of-date
package containing a known security defect.  To avoid that, we need to
distribute signed package metadata (the ELPA "archive-contents" file)
and checksums for the individual packages together.  (Bug#19479)

A subsequent patch will add support for last-update and expiration
timestamps in "archive-contents", without which the protection against
replay attacks will be largely ineffective.

Taken together, this feature will make signing individual packages
obsolete.  We will instead rely on signing the metadata, package
checksums and timestamps.  Note that individual package signatures
should still be distributed for a long time still to support old
versions of Emacs.

For more on replay attacks, see:
https://www2.cs.arizona.edu/stork/packagemanagersecurity/attacks-on-package-managers.html

* lisp/emacs-lisp/package.el (package-verify-checksums): New
defcustom.
(package-desc, package--ac-desc)
(package--add-to-archive-contents, package-install-from-archive): New
fields 'size' and 'checksums'.
(package-desc-filename): New function.

(package-error): New error type.
(bad-signature): Inherit from error type 'package-error'.
(bad-checksum, bad-size): New error types.
(package-insecure-hash-algorithms): New constant.
(package--verify-package-checksum)
(package--verify-package-size): New function to verify that the
checksum and size of a package corresponds to the checksum and size
data in the "archive-contents" file on the package archive.
(package--show-verify-checksum-error): New function to show
details of an error on checksum verification.

* lisp/emacs-lisp/package-x.el (package-upload-buffer-internal):
Update to use above new fields 'size' and 'checksums'.

* test/lisp/emacs-lisp/package-tests.el (package-test-refresh-contents)
(package-test-install-single-from-archive)
(package-test-list-filter-by-archive)
(package-test-list-filter-by-status): Update tests.
(with-install-using-checksum): New macro.
(package-test-install-wrong-size-single)
(package-test-install-wrong-size-tar): New tests for size checking.
(package-test-install-with-checksum/single-valid)
(package-test-install-with-checksum/single-invalid)
(package-test-install-with-checksum/tar-valid)
(package-test-install-with-checksum/tar-invalid): New tests for
installing packages with checksums.
(package-test-verification-text)
(package-tests-valid-md5-checksum)
(package-tests-valid-sha256-checksum)
(package-tests-valid-sha512-checksum): New variables.
(package-tests--run-verify-checksums-test): New macro.
(package-test-verify-package-checksums-nil/ignore-invalid)
(package-test-verify-package-checksums-allow-missing)
(package-test-verify-package-checksums-allow-missing/missing)
(package-test-verify-package-checksums-allow-missing/ignore-unsupported)
(package-test-verify-package-checksums-t)
(package-test-verify-package-checksums-t/invalid-fails)
(package-test-verify-package-checksums-t/missing-fails)
(package-test-verify-package-checksums-all)
(package-test-verify-package-checksums-all/invalid-fails)
(package-test-verify-package-checksums-all/missing-fails)
(package-test-verify-package-checksums-all/no-supported-hash-fails)
(package-test-verify-package-checksums-all/ignore-unsupported)
(package-test-verify-package-size): New tests for the checksum
support.

* test/lisp/emacs-lisp/package-resources/archive-contents:
* test/lisp/emacs-lisp/package-resources/checksum-invalid-1.0.el:
* test/lisp/emacs-lisp/package-resources/checksum-valid-123.el:
* test/lisp/emacs-lisp/package-resources/checksum-valid-tar-0.99.tar:
* test/lisp/emacs-lisp/package-resources/checksum-valid-tar-0.99.tar:
New test data files.

* doc/emacs/package.texi (Package Installation): Document package
checksum checking.
* etc/NEWS: Announce it.

Handle help-form in y-or-n-p and use this in find-file-noselect (bug#5423)

* doc/lispref/help.texi (Help Functions): Mention help-form for
read-char-from-minibuffer and y-or-n-p.

* doc/lispref/minibuf.texi (Yes-or-No Queries): Mention help-form
for y-or-n-p.
(Multiple Queries): Mention help-form for read-char-from-minibuffer.

* lisp/files.el (find-file-noselect): Let-bind multi-line help text
to help-form for y-or-n-p.

* lisp/subr.el (read-char-choice): Mention help-form in docstring.
(read-char-from-minibuffer): Mention help-form in docstring.
(y-or-n-p-map): Remove handling of 'help'.
(y-or-n-p): Mention help-form in docstring.
When help-form is non-nil: add help-char to 'prompt', and bind
help-char to help-form-show in composed-keymap.

Handle connection-local null-device and path-separator variables

* doc/lispref/os.texi (System Environment): Add `path-separator'
function and `null-device' variable and function.

* etc/NEWS: Mention 'null-device' and 'path-separator'.  Fix typos.

* lisp/files-x.el (path-separator, null-device): New defuns.  (Bug#3736)

* lisp/net/tramp-adb.el
(tramp-adb-connection-local-default-shell-variables): Rename from
`tramp-adb-connection-local-default-profile'.

* lisp/net/tramp-integration.el
(tramp-connection-local-default-system-variables): New defvar.
Add it to connection-local profiles.
(tramp-connection-local-default-shell-variables): Rename from
`tramp-connection-local-default-profile'.

* lisp/progmodes/grep.el (grep-hello-file): New defun.
(grep-compute-defaults): Use `null-device' function for remote
case. Handle remote `hello-file'.  Use `process-file-shell-command'.
(grep,grep-expand-keywords, lgrep): Use `null-device' function for
remote case.

Migrate usage of GPM_CLICK_EVENT to MOUSE_CLICK_EVENT.

* src/termhooks.h (enum event_kind):
* src/term.c (term_mouse_click, handle_one_term_event):
* src/keyboard.c (discard_mouse_events, make_lispy_event): Migrate
usage of GPM_CLICK_EVENT to MOUSE_CLICK_EVENT.

* lisp/emacs-lisp/package.el (package-strip-rcs-id): Don't ignore errors

Ignoring errors here just postpones the error and replaces a clear
"invalid version syntax" with a confusing "package lacks a version".

Use cl-letf instead of unwind-protect in a test

* test/lisp/cus-edit-tests.el (cus-edit-tests-customize-saved/show-obsolete):
Good use case for cl-letf, so use it.
Suggested by Stefan Monnier <monnier@iro.umontreal.ca> in:
https://lists.gnu.org/archive/html/emacs-devel/2020-11/msg00914.html

* test/manual/indent/tcl.tcl: Add string interpolation case

Don't optimize away `eval` when its lexical context is different

* lisp/emacs-lisp/bytecomp.el (byte-compile-file-form-eval):
Obey lexical-binding.

Take care of a FIXME in cus-edit-tests.el

* test/lisp/cus-edit-tests.el (cus-edit-tests-customize-saved/show-obsolete):
Add a fake saved-value property, to be able check that the obsolete
option is present in the Customize buffer.  Expect the test to pass
now.

Don't quote lambda in idlwave.el

* lisp/progmodes/idlwave.el (idlwave-keyword-abbrev): Don't quote
lambda.

Remove remaining XEmacs compat code from erc-log.el

* lisp/erc/erc-log.el (erc-save-buffer-in-logs): Remove XEmacs
compat code.

Don't set XEmacs only properties start-open and end-open

* lisp/epa.el (epa-sign-region, epa-encrypt-region):
* lisp/erc/erc.el (erc-display-prompt):
* lisp/gnus/message.el (message-forward-make-body-mime):
* lisp/net/eudc-bob.el (eudc-bob-display-jpeg)
(eudc-bob-display-audio, eudc-bob-display-generic-binary):
* lisp/url/url-http.el
(url-http-chunked-encoding-after-change-function): Don't set XEmacs
only properties start-open and end-open.

Remove XEmacs compat code from idlw-shell.el

* lisp/progmodes/idlw-shell.el:
(idlwave-shell-use-toolbar): Doc fix.
(idlwave-default-mouse-track-event-is-with-button): Declare obsolete.
(idlwave-shell-update-bp-overlays): Remove XEmacs compat code.

Remove some XEmacs compat code from ediff-wind.el

* lisp/vc/ediff-wind.el (ediff-window)
(ediff-compute-toolbar-width): Remove some XEmacs compat code.

Remove Emacs 20 compat code from org.el

* lisp/org/org.el (org-org-menu, org-create-customize-menu):
Remove Emacs 20 compat code.

Remove Emacs 20 compat code from idlwave.el

* lisp/progmodes/idlwave.el (idlwave-indent-line)
(idlwave-toggle-comment-region, idlwave-reset-sintern)
(idlwave-unit-name, idlwave-mode-menu-def)
(idlwave-create-customize-menu): Remove Emacs 20 compat code.

Properly mark obsolete semantic functions as such

* lisp/cedet/semantic/ia.el (semantic-ia-get-completions)
(semantic-ia-get-completions-deprecated): Make obsolete.

Mark compat alias in cus-face.el obsolete

* lisp/cus-face.el (custom-facep): Mark compat alias obsolete.
* lisp/cus-dep.el (custom-make-dependencies):
* lisp/cus-edit.el (customize-changed-options)
(custom-unsaved-options, customize-saved, customize-apropos)
(custom-save-faces): Adjust callers.

Fix compilation on MS-Windows with librsvg > 2.46.0

* src/image.c (rsvg_handle_get_dimensions, init_svg_functions):
Make 'rsvg_handle_get_dimensions' available and defined for all
versions of librsvg.  (Bug#44655)

Hide obsolete options in most customize commands

* lisp/cus-edit.el (custom--filter-obsolete-variables): New defun.
* lisp/cus-edit.el (customize-changed-options)
(customize-apropos, custom-group-value-create): Hide obsolete user
options.  (Bug#44598)
* test/lisp/cus-edit-tests.el: New file.

Revert unintended part of last change to jsonrpc-request

While playing around with the timing in this function, I left
this change that could freeze the function on some platforms.

* lisp/jsonrpc.el (jsonrpc-request): Use accept-process-output.
(Version): Bump to 1.0.14

Move semantic/tag obsolete variables to avoid a warning

* lisp/cedet/semantic/tag.el (semantic-token-version): Move to
avoid a compilation warning.

Fix default timeout handling in jsonrpc-request

* lisp/jsonrpc.el (jsonrpc-request): Use default timeout if not passed.
(Version): Bump to 1.0.13

Avoid a compilation warning about an ambiguous else

* src/data.c (set_internal): Avoid compilation warning.

data.c:1443:9: error: suggest explicit braces to avoid ambiguous ‘else’
 1443 |      if (idx > 0 && bindflag == SET_INTERNAL_SET
      |         ^

* src/data.c (set_internal): Fix bug#44733

Set the default value when `set` encounters a PER_BUFFER variable
which has been let-bound globally, to match the behavior seen with
`make-variable-buffer-local`.

* test/src/data-tests.el (binding-test--let-buffer-local):
Add corresponding test.
(data-tests--set-default-per-buffer): Add tentative test for the
performance problem encountered in bug#41029.

Remove some XEmacs compat code from filesets.el

* lisp/filesets.el (filesets-init): Remove some XEmacs compat code.
(filesets-error): Declare obsolete.
(filesets-directory-files, filesets-get-selection)
(filesets-spawn-external-viewer, filesets-get-filelist)
(filesets-open, filesets-close, filesets-get-menu-epilog)
(filesets-ingroup-collect-files, filesets-build-ingroup-submenu)
(filesets-update-pre010505): Adjust callers.

Remove some compat code from ffap.el

* lisp/ffap.el (ffap-mouse-event, ffap-event-buffer): Make obsolete.
(ffap-menu-ask, ffap-at-mouse): Adjust callers.

Declare some compat aliases obsolete

* lisp/cedet/semantic/tag.el (semantic-token-version)
(semantic-token-incompatible-version):
* lisp/emulation/edt.el (edt-bind-standard-key): Make compat
aliases obsolete.

Remove some compat code from url.el

* lisp/url/url.el (url-warn): Make into obsolete alias for
display-warning.
* lisp/url/url-auth.el (url-register-auth-scheme):
* lisp/url/url-news.el (url-news-open-host):
* lisp/url/url-proxy.el (url-find-proxy-for-url): Adjust callers.

Remove some compat code for old versions and XEmacs

* lisp/ibuf-ext.el (ibuffer-old-saved-filters-warning)
(ibuffer-maybe-save-stuff): Assume customize-save-variable is
bound; it is autoloaded.
* lisp/cedet/semantic/symref/grep.el (semantic-symref-perform-search):
* lisp/password-cache.el (password-cache-remove):
* lisp/cedet/semantic/bovine/el.el (semantic-dependency-tag-file):
Remove Emacs 21 compat code.
* lisp/cedet/semantic/sort.el (semantic-string-lessp-ci):
Remove Emacs 20 compat code.
* test/lisp/cedet/semantic-utest.el (semantic-utest-temp-directory):
* lisp/mail/supercite.el (sc-ask): Remove XEmacs compat code.
* lisp/progmodes/idlw-shell.el (idlwave-shell-mode):
* lisp/progmodes/idlwave.el (idlwave-mode): Remove commented out
compat code.

* src/buffer.h (struct buffer): Remove unused field `minor_modes`

* src/buffer.c (bset_minor_modes): Remove function.
(reset_buffer_local_variables, init_buffer_once): Don't set `minor_modes`.

Use decoding implementation from `insert-directory' in Tramp

* lisp/net/tramp-sh.el (tramp-sh-handle-insert-directory):
Use decoding implementation from `insert-directory', it is more robust.

More string-search optimisations

All-ASCII strings cannot have substrings with non-ASCII characters in
them; use this fact to avoid searching entirely.

* src/fns.c (Fstring_search): For multibyte non-ASCII needle and
unibyte haystack, don't check if the haystack is all-ASCII; it's a
waste of time. For multibyte non-ASCII needle and multibyte
all-ASCII haystack, fail immediately.
* test/src/fns-tests.el (string-search): Add more test cases.

* etc/NEWS: Restore entries accidentally removed by previous commit.

In attempted recursive minibuffer use, display error message in correct frame

This was problematic when minibuffer-follows-selected-frame was non-nil.
Introduce a new parameter DONT-SET-FRAME to set-window-configuration.

* doc/lispref/windows.texi (Window Configurations): Describe the new &optional
parameter to set-window-configuration.

* etc/NEWS (Lisp Changes): Note the new parameter to set-window-configuration.

* src/keyboard.c (read_char_help_form_unwind): Add a new Qnil argument to the
call of Fset_window_configuration.

* src/minibuf.c (read_minibuf): Cons up a Qt with the window configuration in
the argument to record_unwind_protect for the window configuration (twice).

* src/window.c (Fset_window_configuration): Add the new &optional parameter
and document it in the doc string.  At the final do_switch_frame operation,
restore the original frame when DONT-SET-FRAME is non-nil.
(restore_window_configuration): Handle the new parameter when the supplied
argument is a cons.

Remove outdated comment about Emacs 20 from viper.el

* lisp/emulation/viper-util.el (viper-chars-in-region): Remove
outdated comment.

Add new variable cperl-tags-file-name

* lisp/progmodes/cperl-mode.el (cperl-tags-file-name): New variable.
(cperl-write-tags): Use above new variable instead of hardcoding
filename "TAGS".  (Bug#8802)

Declare XEmacs compat function in inversion.el obsolete

* lisp/cedet/inversion.el (inversion-require-emacs): Declare obsolete.

Assume font-lock is provided; it's preloaded since 22.1

* lisp/cedet/semantic/format.el (semantic--format-colorize-text):
* lisp/eshell/em-ls.el (eshell-ls--insert-directory):
* lisp/net/dig.el (dig-mode):
* lisp/progmodes/cperl-mode.el (cperl-pod-here-fontify):
* lisp/progmodes/idlw-help.el (idlwave-help-fontify):
* lisp/progmodes/idlwave.el (idlwave-completion-fontify-classes):
Don't check for feature 'font-lock; it has been preloaded since 22.1.
* lisp/cedet/semantic/format.el (font-lock):
* lisp/epa.el (font-lock):
* lisp/erc/erc.el (font-lock):
* lisp/generic-x.el (font-lock):
* lisp/net/sieve-mode.el (font-lock):
* lisp/progmodes/prolog.el (font-lock):
* lisp/textmodes/rst.el (font-lock): Remove unnecessary require.

* lisp/progmodes/cperl-mode.el: Doc fix.

Remove unnecessary load from idlwave

* lisp/progmodes/idlw-shell.el:
* lisp/progmodes/idlwave.el: Remove unnecessary load; easy-menu-define
is autoloaded.

Make compat alias add-submenu obsolete

* lisp/emacs-lisp/easymenu.el (add-submenu): Make compat alias
obsolete.
* lisp/filesets.el (filesets-build-menu-now): Don't use above
obsolete alias.
(filesets-menu-path, filesets-menu-before)
(filesets-menu-in-menu): Doc fix.

Fix SVG display again (bug#44655)

* src/image.c (svg_load_image): Fall back to
rsvg_handle_get_dimensions if we can't calculate the size of the
image.

Clarify that 'diff-error' is part of Emacs 28.1

* lisp/vc/diff-mode.el (diff-error): Add :version tag (bug#44727).

Turn gdb-wait-for-pending into a plain function

This avoids unnecessary body duplication in expansion and macro
recursion (causing macro-expansions at runtime), making it clearer
what is going on.

* lisp/progmodes/gdb-mi.el (gdb-wait-for-pending): Make it a function,
remove lambda quoting,  η-reduce and simplify.
(gdb-thread-exited, gdb-thread-selected): Adapt callers.

Don't make bibtex-unify-case-convert buffer-local

The :local tag does not currently work as intended (it results in a
default value of bibtex-unify-case-convert of nil rather than
identity), and no other bibtex.el user option is automatically
buffer-local, so revert this recent change.

For discussion, see the following emacs-devel thread:
https://lists.gnu.org/r/emacs-devel/2020-11/msg00734.html

* lisp/textmodes/bibtex.el (bibtex-unify-case-convert): Don't make
automatically buffer-local for consistency with other user options,
and because the :local tag doesn't have the intended results.

Preserve `dired-filename' text properties in Tramp

* lisp/net/tramp-sh.el (tramp-sh-handle-insert-directory):
Restore `dired-filename' text property, which has been destroyed
by `decode-coding-region'.  (Bug#44682)

Remove redundant 'function's around lambdas

* lisp/allout.el (allout-latex-verb-quote):
* lisp/edmacro.el (edmacro-format-keys):
* lisp/ffap.el (ffap-all-subdirs-loop)
(ffap-kpathsea-expand-path, ffap-menu-rescan):
* lisp/files.el (save-buffers-kill-emacs):
* lisp/find-lisp.el (find-lisp-find-dired-internal)
(find-lisp-insert-directory):
* lisp/gnus/gnus-agent.el (gnus-agent-expire-unagentized-dirs):
* lisp/gnus/nnmairix.el (nnmairix-create-message-line-for-search)
(nnmairix-widget-get-values)
(nnmairix-widget-make-query-from-widgets)
(nnmairix-widget-build-editable-fields):
* lisp/international/mule-cmds.el (sort-coding-systems):
* lisp/international/mule-diag.el (list-character-sets-1):
* lisp/international/quail.el (quail-insert-decode-map):
* lisp/mail/reporter.el (reporter-dump-state):
* lisp/mail/supercite.el (sc-attribs-filter-namelist):
* lisp/pcmpl-gnu.el (pcmpl-gnu-zipped-files)
(pcmpl-gnu-bzipped-files):
* lisp/progmodes/cperl-mode.el (cperl-find-tags)
(cperl-write-tags, cperl-tags-hier-init, cperl-tags-treeify)
(cperl-menu-to-keymap, cperl-pod-spell):
* lisp/progmodes/gdb-mi.el (gdb-parent-mode):
* lisp/progmodes/make-mode.el (makefile-browser-fill):
* lisp/simple.el (transpose-lines):
* lisp/term.el:
* lisp/term/w32-win.el (w32-find-non-USB-fonts):
* lisp/textmodes/table.el (table--generate-source-scan-lines): Remove
redundant 'function's around lambdas.

Remove redundant 'function's around lambdas in mh/*.el

* lisp/mh-e/mh-alias.el (mh-alias-tstamp, mh-alias-filenames)
(mh-alias-address-to-alias):
* lisp/mh-e/mh-comp.el (mh-edit-again, mh-redistribute):
* lisp/mh-e/mh-identity.el (mh-identity-make-menu):
* lisp/mh-e/mh-utils.el (mh-help): Remove redundant 'function's around
lambdas.

Enhance syntax-tests.el to test nestable comments (Lisp style)

Also add some tests for braces and parse-partial-sexp amongst Lisp style
comments.

* test/src/syntax-tests.el (\;-in, \;-out): Add syntax for { and }.
(top-level): Add new tests for Lisp style comments.
(\#|-in, \#|-out): New functions.
(top-level): Add new tests for nested Lisp style comments, and mixtures of
nested comments with "ordinary" comments.

* test/src/syntax-resources/syntax-comments.txt (top-level): Add new test
fragments for #|...|#, etc.

Don't quote lambdas with 'function' in calc/*.el

* lisp/calc/calc-aent.el (calc-do-quick-calc)
(calc-do-calc-eval, math-build-parse-table):
* lisp/calc/calc-alg.el (math-polynomial-base):
* lisp/calc/calc-alg.el (math-is-poly-rec):
* lisp/calc/calc-arith.el (calcFunc-scf):
* lisp/calc/calc-arith.el (math-ceiling, math-round):
* lisp/calc/calc-arith.el (math-trunc-fancy, math-floor-fancy):
* lisp/calc/calc-ext.el (calc-init-extensions, calc-reset)
(calc-refresh-top, calc-z-prefix-help, calc-binary-op-fancy)
(calc-unary-op-fancy):
* lisp/calc/calc-forms.el (math-make-mod):
* lisp/calc/calc-frac.el (calcFunc-frac):
* lisp/calc/calc-funcs.el (calcFunc-euler):
* lisp/calc/calc-help.el (calc-full-help):
* lisp/calc/calc-lang.el (c, pascal, fortran, tex, latex, eqn)
(yacas, maxima, giac, math, maple):
* lisp/calc/calc-macs.el (calc-wrapper, calc-slow-wrapper):
* lisp/calc/calc-map.el (calc-get-operator, calcFunc-mapeqr)
(calcFunc-reducea, calcFunc-rreducea, calcFunc-reduced)
(calcFunc-rreduced, calcFunc-outer):
* lisp/calc/calc-misc.el (another-calc, calc-do-handle-whys):
* lisp/calc/calc-mode.el (calc-save-modes):
* lisp/calc/calc-mtx.el (math-col-matrix, math-mul-mat-vec):
* lisp/calc/calc-poly.el (math-sort-terms, math-poly-div-list)
(math-mul-list, math-sort-poly-base-list)
(math-partial-fractions):
* lisp/calc/calc-prog.el (calc-user-define-formula):
* lisp/calc/calc-rewr.el (math-rewrite, math-compile-patterns)
(math-compile-rewrites, math-parse-schedule)
(math-rwcomp-pattern):
* lisp/calc/calc-store.el (calc-var-name-map, calc-let)
(calc-permanent-variable, calc-insert-variables):
* lisp/calc/calc-stuff.el (calc-flush-caches, calcFunc-pclean)
(calcFunc-pfrac):
* lisp/calc/calc-units.el (math-build-units-table)
(math-decompose-units):
* lisp/calc/calc-vec.el (calcFunc-mrow, math-mat-col)
(calcFunc-mcol, math-mat-less-col, math-mimic-ident):
* lisp/calc/calc-yank.el (calc-edit):
* lisp/calc/calc.el
(calc-mode-var-list-restore-default-values)
(calc-mode-var-list-restore-saved-values, calc-mode, calc-quit):
* lisp/calc/calccomp.el (math-compose-expr)
(math-compose-matrix, math-vector-to-string): Don't quote lambdas with
'function'.

Add command to filter package menu by name or description

* lisp/emacs-lisp/package.el (package-menu-filter-by-description):
(package-menu-filter-by-name-or-description): New commands to filter
the package menu.  (Bug#44699)
(package-menu-mode-map): Bind the above new commands.
(package-menu-mode-menu): Add new commands to the menu.
* doc/emacs/package.texi (Package Menu): Document new commands.

Don't quote lambdas with function macro in generic-x.el

* lisp/generic-x.el (apache-conf-generic-mode, ini-generic-mode)
(reg-generic-mode, mailagent-rules-generic-mode)
(vrml-generic-mode, java-properties-generic-mode)
(alias-generic-mode, ansible-inventory-generic-mode)
(inetd-conf-generic-mode, etc-services-generic-mode)
(etc-passwd-generic-mode, etc-fstab-generic-mode)
(spice-generic-mode, astap-generic-mode): Don't quote lambdas with
function macro.

Test that substitute-command-keys preserves text properties

* test/lisp/help-tests.el
(help-substitute-command-keys/preserves-text-properties): New test.
(Bug#17052)

; Fix last change to bibtex.el

* lisp/textmodes/bibtex.el (bibtex-unify-case-convert): Clarify
docstring and remove leading asterisk.  Use defcustom :local tag.

perl-mode and cperl-mode: Recognize regex after "return"

* lisp/progmodes/cperl-mode.el (cperl-find-pods-heres): Add
"return" to the keywords which start a regex.

* lisp/progmodes/perl-mode.el (defconst): Add "return" to
'perl--syntax-exp-intro-keywords' (Bug#26850).

* test/lisp/progmodes/cperl-mode-tests.el (cperl-test-bug-28650):
New test (bug#26850).

Minor Edebug manual keystroke clarifications

* doc/lispref/edebug.texi (Edebug Misc): Also mention the `a'
binding to abort (bug#44697).  Also fix `d' function reference, and
add `P' reference.

Make the handwrite.el PS valid again

* lisp/play/handwrite.el (handwrite): Make the PS valid
(bug#44648).  Suggested by Omar Antolín <omar.antolin@gmail.com>.

Fix gnus-mime-display-alternative button natigation

* lisp/gnus/gnus-art.el (gnus-mime-display-alternative): Mark the
buttons correctly so that TAB can find them (bug#44690).

Use 'eshell-find-alias-function' instead of fboundp

* lisp/eshell/esh-cmd.el (eshell-invoke-directly): Fix problem
with (require 'em-tramp) making password prompting from "sudo
bash" no longer work (bug#43772).

Add new user option bibtex-unify-case-convert

* lisp/textmodes/bibtex.el (bibtex-unify-case-convert): New
variable (bug#44614).
(bibtex-format-entry): Use it (bug#44614).

Use lexical-binding in time-stamp.el

* lisp/time-stamp.el: Use lexical-binding.  Remove redundant :group
args.

Don't quote lambdas in net/*.el

* lisp/net/eudc-export.el (eudc-create-bbdb-record):
* lisp/net/eudc.el (eudc-print-attribute-value)
(eudc-display-records, eudc-process-form)
(eudc-filter-duplicate-attributes, eudc-filter-partial-records)
(eudc-add-field-to-records, eudc-query-with-words)
(eudc-query-form, eudc-menu):
* lisp/net/eudcb-bbdb.el (eudc-bbdb-extract-phones)
(eudc-bbdb-query-internal):
* lisp/net/mairix.el (mairix-widget-make-query-from-widgets)
(mairix-widget-build-editable-fields, mairix-widget-get-values):
Don't quote lambdas.

Don't quote lambdas in emacs-lisp/*.el

* lisp/emacs-lisp/cl-seq.el (cl--parsing-keywords, cl-sort):
* lisp/emacs-lisp/cl-macs.el (cl-typecase):
* lisp/emacs-lisp/cl-extra.el (cl-some, cl-every)
(cl--map-keymap-recursively):
* lisp/emacs-lisp/advice.el (ad-insert-argument-access-forms):
* lisp/emacs-lisp/edebug.el (edebug-sort-alist)
(edebug-set-windows):
* lisp/emacs-lisp/pp.el (pp-display-expression):
* lisp/emacs-lisp/regi.el (regi-interpret): Don't quote lambdas.

Merge from origin/emacs-27

75723ec212 (origin/emacs-27) ; * lisp/emacs-lisp/benchmark.el (benchm...
53e2a612ad ; * lib-src/make-fingerprint.c: Update commentary.
286c632772 Reformat argument commentary in etags.c
4ec740866a Make the invocation of combine-change-calls in comment-reg...
66bcec8838 * lisp/progmodes/cc-langs.el (c-<>-notable-chars-re): Fix ...
03eeab469e ; Update the expected result files in test/manual/etags.
d875a22bc6 Update the various INSTALL files

# Conflicts:
# INSTALL

; Merge from origin/emacs-27

The following commit was skipped:

e2c7b6372d Don't leave lock files after 'replace-buffer-contents'

Merge from origin/emacs-27

19da602991 Fix input method translation near read-only text
5aabf2cc7f Fix display of truncated R2L lines on TTY frames
daff3bda10 Avoid crashes when a reversed glyph row starts with a comp...
b697bb91a1 ; * .gitignore: src/fingerprint.c not generated since 2019...

Don't quote lambdas in calc/calcalg{2,3}.el

* lisp/calc/calcalg2.el (calcFunc-inv\', calcFunc-sqrt\')
(calcFunc-deg\', calcFunc-rad\', calcFunc-ln\')
(calcFunc-log10\', calcFunc-lnp1\', calcFunc-log\')
(calcFunc-log\'2, calcFunc-exp\', calcFunc-expm1\')
(calcFunc-sin\', calcFunc-cos\', calcFunc-tan\', calcFunc-sec\')
(calcFunc-csc\', calcFunc-cot\', calcFunc-arcsin\')
(calcFunc-arccos\', calcFunc-arctan\', calcFunc-sinh\')
(calcFunc-cosh\', calcFunc-tanh\', calcFunc-sech\')
(calcFunc-csch\', calcFunc-coth\', calcFunc-arcsinh\')
(calcFunc-arccosh\', calcFunc-arctanh\', calcFunc-bern\'2)
(calcFunc-euler\'2, calcFunc-gammag\'2, calcFunc-gammaG\'2)
(calcFunc-gammaP\'2, calcFunc-gammaQ\'2, calcFunc-betaB\')
(calcFunc-betaI\', calcFunc-erf\', calcFunc-erfc\')
(calcFunc-besJ\'2, calcFunc-besY\'2, calcFunc-sum)
(calcFunc-prod, calcFunc-integ, calcFunc-if, calcFunc-subscr)
(math-do-integral, calcFunc-integ, math-decompose-poly)
(math-solve-system-rec, math-solve-system-subst, math-solve-for)
(calcFunc-inv, calcFunc-sqrt, calcFunc-conj, calcFunc-abs)
(calcFunc-deg, calcFunc-rad, calcFunc-ln, calcFunc-log10)
(calcFunc-lnp1, calcFunc-exp, calcFunc-expm1, calcFunc-sin)
(calcFunc-cos, calcFunc-tan, calcFunc-arcsin, calcFunc-arccos)
(calcFunc-arctan, calcFunc-sinh, calcFunc-cosh, calcFunc-tanh)
(calcFunc-arcsinh, calcFunc-arccosh, calcFunc-arctanh):
* lisp/calc/calcalg3.el (calc-get-fit-variables)
(calcFunc-polint, calcFunc-ratint, math-all-vars-but): Don't quote
lambdas.

Don't quote lambdas in eshell/*.el

* lisp/eshell/em-basic.el (eshell-echo):
* lisp/eshell/em-cmpl.el (eshell-command-completion-function)
(eshell-default-completion-function, eshell-cmpl-initialize)
(eshell-complete-parse-arguments, eshell-complete-commands-list):
* lisp/eshell/em-dirs.el (eshell-complete-user-reference):
* lisp/eshell/em-hist.el (eshell-hist-unload-hook)
(eshell-hist-initialize):
* lisp/eshell/em-ls.el (eshell-ls-sort-entries):
* lisp/eshell/em-pred.el (eshell-modifier-alist)
(eshell-display-predicate-help, eshell-display-modifier-help)
(eshell-pred-substitute, eshell-split-members):
* lisp/eshell/em-prompt.el (eshell-prompt-function):
* lisp/eshell/em-smart.el (eshell-smart-unload-hook)
(eshell-smart-initialize, eshell-refresh-windows):
* lisp/eshell/em-unix.el (eshell-shuffle-files):
* lisp/eshell/esh-arg.el (eshell-parse-argument-hook):
* lisp/eshell/esh-cmd.el (eshell-cmd-initialize)
(eshell-parse-command):
* lisp/eshell/esh-mode.el (eshell-preinput-scroll-to-bottom)
(eshell-postoutput-scroll-to-bottom):
* lisp/eshell/esh-module.el (eshell-modules-list):
* lisp/eshell/esh-proc.el (eshell-read-process-name)
(eshell-round-robin-kill):
* lisp/eshell/esh-var.el (eshell-envvar-names)
(eshell-variables-list): Don't quote lambdas.

; * lisp/emacs-lisp/benchmark.el (benchmark-run): Fix docstring

; * lib-src/make-fingerprint.c: Update commentary.

New user options 'copy-region-blink-delay' and 'delete-pair-blink-delay'

* lisp/emacs-lisp/lisp.el (delete-pair-blink-delay): New defcustom.
(delete-pair): Use it.  (Bug#4136)

* lisp/simple.el (copy-region-blink-delay): New defcustom.
(indicate-copied-region): Use it.  (Bug#42865)
Thanks to Sean Whitton <spwhitton@spwhitton.name>.
(indicate-copied-region): Use 'query-replace-descr' not to show
newlines literally.  Use "Copied text" instead of misleading
"Saved text" (bug#42865).

Reformat argument commentary in etags.c

* lib-src/etags.c (pfnote, consider_token, C_entries): Resurrect
original format of comments to function arguments.

Reformat comment for 'gnu' compilation rule

* lisp/progmodes/compile.el (compilation-error-regexp-alist-alist):
The comments above the regexp for the 'gnu' rule contained
references to the previous string regexp, which has been difficult
to follow ever since the translation to rx.  Move the comments
to their proper places, and add some guiding notes.

Make the invocation of combine-change-calls in comment-region valid

This fixes bug #44581.  The problem was that whitespace outside of the (BEG
END) region was being deleted, and this made the invocation of
combine-change-calls with (BEG END) invalid.

* lisp/newcomment.el (comment-region-default): Amend the second argument to
combine-change-calls.

* lisp/progmodes/cc-langs.el (c-<>-notable-chars-re): Fix wrong '-' in regexp

Make initial frame match frame-title-format

* src/xterm.c (x_term_init):
* src/w32term.c (w32_initialize_display_info): Sync initial frame
title with new value of Vframe_title_format.
Problem reported by Angelo Graziosi <angelo.g0@libero.it>.

Run menu-item :filter function before showing binding

* lisp/help.el (describe-map): Fix running `menu-item' :filter
functions.  This fixes a mistake in the previous conversion of this
defun from the old C function describe_map.  See the discussion in
Bug#39149.
* test/src/keymap-tests.el
(keymap---get-keyelt/runs-menu-item-filter)
(describe-buffer-bindings/menu-item-filter-show-binding)
(describe-buffer-bindings/menu-item-filter-hide-binding):
New tests.
(keymap-tests--test-menu-item-filter): New defun.

Handle negation of search keys in gnus-search minibuffer completion

* lisp/gnus/gnus-search.el (gnus-search-get-active): Keys might start
with a leading "-": check for that and ignore it.

* lisp/progmodes/xref.el (xref-goto-xref): Prefix arg quits the *xref* buffer.

(bug#44611)

* lisp/org/ob-ruby.el (org-babel-ruby-initiate-session): Use :ruby header arg.

Allow specification of ruby command using the :ruby header arg.
https://lists.gnu.org/archive/html/emacs-orgmode/2020-11/msg00166.html

Fix `speedbar-directory-buttons' when using Tramp

* lisp/speedbar.el (speedbar-directory-buttons): Make speedbar
work with directories accessed via Tramp (bug#44622).

Copyright-paperwork-exempt: yes

Don't quote lambdas in several places

* lisp/align.el (align-highlight-rule):
* lisp/bookmark.el (bookmark-maybe-sort-alist):
* lisp/emacs-lisp/advice.el (ad-read-advice-name)
(ad-retrieve-args-form, ad-make-hook-form, defadvice)
(ad-with-originals):
* lisp/foldout.el (foldout-inhibit-key-bindings):
* lisp/gnus/gnus-bookmark.el (gnus-bookmark-maybe-sort-alist):
* lisp/mail/rfc822.el (rfc822-addresses-1):
* lisp/net/eudcb-ldap.el (eudc-ldap-cleanup-record-simple):
* lisp/net/net-utils.el (network-connection-to-service):
* lisp/net/socks.el (socks-build-auth-list):
* lisp/org/ox-odt.el (org-odt--image-size):
* lisp/pcomplete.el (pcomplete-command-completion-function)
(pcomplete-default-completion-function, pcomplete-opt):
* lisp/progmodes/cperl-mode.el (cperl-highlight-charclass)
(cperl-tags-hier-init, cperl-tags-treeify)
(cperl-next-interpolated-REx, cperl-time-fontification):
* lisp/shadowfile.el (shadow-copy-files, shadow-shadows-of-1)
(shadow-save-buffers-kill-emacs):
* lisp/strokes.el (strokes-renormalize-to-grid):
* lisp/tempo.el (tempo-insert, tempo-forward-mark)
(tempo-backward-mark):
* lisp/textmodes/artist.el (artist-submit-bug-report):
* lisp/textmodes/ispell.el (ispell-complete-word):
* lisp/url/url-auth.el (url-get-authentication):
* lisp/url/url-cache.el (url-cache-create-filename-human-readable):
* lisp/vcursor.el (vcursor-find-window):
* test/lisp/textmodes/reftex-tests.el
(reftex-parse-bibtex-entry-test): Don't quote lambdas.

Work around glib messing with signal handlers more than it should

* src/process.c (init_process_emacs): force glib's g_unix_signal
handler into lib_child_handler where it should belong.

Copyright-paperwork-exempt: yes

Add an option to preserve ANSI sequences

* lisp/ansi-color.el Add an option to preserve the ANSI sequences
* test/lisp/ansi-color-tests.el: Add tests (bug#44589).

project-or-external-find-file doc string fix

* lisp/progmodes/project.el (project-or-external-find-file): Doc
string fix -- it's not "recognizing" file names (bug#44588).

Use lexical-binding in fortune.el and add tests

* lisp/play/fortune.el: Use lexical-binding.  Remove redundant
:group args.
(fortune-in-buffer): Quote function symbol as such.
* test/lisp/play/fortune-resources/fortunes:
* test/lisp/play/fortune-tests.el: New files.
* .gitignore: Ignore generated file fortunes.dat.

; Update the expected result files in test/manual/etags.

Update the various INSTALL files

* nt/INSTALL.W64:
* nt/INSTALL:
* INSTALL: Update the installation information, in particular the
fact that HarfBuzz is now preferred as the shaping library.

Don't leave lock files after 'replace-buffer-contents'

* src/editfns.c (Freplace_buffer_contents): Unlock the buffer's
file if no changes have been made.  (Bug#44303)

(cherry picked from commit a5867ddfbd721568005175bf6c725f7834b21ea4)

Fix input method translation near read-only text

* lisp/international/quail.el (quail-input-method): Don't disable
input method when the character after point has the read-only
property.  Suggested by Evgeny Zajcev <lg.zevlg@gmail.com>
(Bug#44466)

* doc/emacs/mule.texi (Input Methods): Document that input methods
are inhibited in read-only text.

Make Calc windows dedicated by default

* lisp/calc/calc.el (calc-make-windows-dedicated): New defcustom.
(calc, calc-trail-display): Set Calc windows dedicated if
calc-make-windows-dedicated is non-nil.  Patch by Boruch Baum
<boruch_baum@gmx.com>.  (Bug#44108)

* etc/NEWS (Calc): Announce the new behavior.

Make 'prefer-utf-8' heed inhibit-*-detection variables

* lisp/international/mule-conf.el (prefer-utf-8): Inhibit
detection of null bytes and ISO escape sequences if the respective
inhibit-*-detection variables say so.  (Bug#44486)

Merge branch 'master' of git.savannah.gnu.org:/srv/git/emacs

* lisp/faces.el (mode-line-highlight): Use :box only when supported.

Capitalize portable dump messages.

We capitalize all other messages during the dump, so capitalize the
"dump mode" and "dumping fingerprint" ones as well for consistency.

* src/pdumper.c (Fdump_emacs_portable): Capitalize fingerprint message
prefix.

* lisp/loadup.el: Capitalize "dump mode" message.

Face-changing text properties and help-echo now work with xterm-mouse.

* src/dispnew.c (update_mouse_position): New function for mouse
movement logic in 'handle_one_term_event' that can be shared across
different mouse backends.
(display--update-for-mouse-movement): New lisp function, call it.
* lisp/xt-mouse.el (xterm-mouse--handle-mouse-movement): New function
that calls 'display--update-for-mouse-movement'.
(xterm-mouse-translate-1): Call it.
* src/term.c (handle_one_term_event): Inline logic from
'term_mouse_movement' and call 'update_mouse_position'.
(term_mouse_movement): Delete.

Fix display of truncated R2L lines on TTY frames

* src/xdisp.c (extend_face_to_end_of_line): Use a while-loop, not
a do-while loop, to avoid appending an extra glyph at the end of a
line that is one character shorter than the window-width.  This is
needed to fix display of reversed glyph rows that are almost as
wide as the window, because append_space_for_newline already added
one space glyph.

Avoid crashes when a reversed glyph row starts with a composition

* src/dispnew.c (build_frame_matrix_from_leaf_window): Add an
assertion to prevent us from overwriting non-char glyphs with the
vertical border glyph.
* src/xdisp.c (extend_face_to_end_of_line): Account for one glyph
possibly inserted by append_space_for_newline.  (Bug#44506)
Remove a kludgey correction for an off-by-one error in column
counting, which is no longer needed.