Support package checksum verification

Support package checksum verification

Package checksum verification is the first step towards protecting
users of package.el against replay attacks.  Signing individual
packages still allows a hostile actor to distribute an out-of-date
package containing a known security defect.  To avoid that, we need to
distribute signed package metadata (the ELPA "archive-contents" file)
and checksums for the individual packages together.  (Bug#19479)

A subsequent patch will add support for last-update and expiration
timestamps in "archive-contents", without which the protection against
replay attacks will be largely ineffective.

Taken together, this feature will make signing individual packages
obsolete.  We will instead rely on signing the metadata, package
checksums and timestamps.  Note that individual package signatures
should still be distributed for a long time still to support old
versions of Emacs.

For more on replay attacks, see:
https://www2.cs.arizona.edu/stork/packagemanagersecurity/attacks-on-package-managers.html

* lisp/emacs-lisp/package.el (package-verify-checksums): New
defcustom.
(package-desc, package--ac-desc)
(package--add-to-archive-contents, package-install-from-archive): New
fields 'size' and 'checksums'.
(package-desc-filename): New function.

(package-error): New error type.
(bad-signature): Inherit from error type 'package-error'.
(bad-checksum, bad-size): New error types.
(package-insecure-hash-algorithms): New constant.
(package--verify-package-checksum)
(package--verify-package-size): New function to verify that the
checksum and size of a package corresponds to the checksum and size
data in the "archive-contents" file on the package archive.
(package--show-verify-checksum-error): New function to show
details of an error on checksum verification.

* lisp/emacs-lisp/package-x.el (package-upload-buffer-internal):
Update to use above new fields 'size' and 'checksums'.

* test/lisp/emacs-lisp/package-tests.el (package-test-refresh-contents)
(package-test-install-single-from-archive)
(package-test-list-filter-by-archive)
(package-test-list-filter-by-status): Update tests.
(with-install-using-checksum): New macro.
(package-test-install-wrong-size-single)
(package-test-install-wrong-size-tar): New tests for size checking.
(package-test-install-with-checksum/single-valid)
(package-test-install-with-checksum/single-invalid)
(package-test-install-with-checksum/tar-valid)
(package-test-install-with-checksum/tar-invalid): New tests for
installing packages with checksums.
(package-test-verification-text)
(package-tests-valid-md5-checksum)
(package-tests-valid-sha256-checksum)
(package-tests-valid-sha512-checksum): New variables.
(package-tests--run-verify-checksums-test): New macro.
(package-test-verify-package-checksums-nil/ignore-invalid)
(package-test-verify-package-checksums-allow-missing)
(package-test-verify-package-checksums-allow-missing/missing)
(package-test-verify-package-checksums-allow-missing/ignore-unsupported)
(package-test-verify-package-checksums-t)
(package-test-verify-package-checksums-t/invalid-fails)
(package-test-verify-package-checksums-t/missing-fails)
(package-test-verify-package-checksums-all)
(package-test-verify-package-checksums-all/invalid-fails)
(package-test-verify-package-checksums-all/missing-fails)
(package-test-verify-package-checksums-all/no-supported-hash-fails)
(package-test-verify-package-checksums-all/ignore-unsupported)
(package-test-verify-package-size): New tests for the checksum
support.

* test/lisp/emacs-lisp/package-resources/archive-contents:
* test/lisp/emacs-lisp/package-resources/checksum-invalid-1.0.el:
* test/lisp/emacs-lisp/package-resources/checksum-valid-123.el:
* test/lisp/emacs-lisp/package-resources/checksum-valid-tar-0.99.tar:
* test/lisp/emacs-lisp/package-resources/checksum-valid-tar-0.99.tar:
New test data files.

* doc/emacs/package.texi (Package Installation): Document package
checksum checking.
* etc/NEWS: Announce it.