There is now support for three font-locking levels, 'minimal',
'moderate' and 'full'. The richest experience is to be expected from
the 'full', and all levels are enabled by default.
* lisp/progmodes/js.el (js--treesit-font-lock-settings): New defvar
renamed from 'js--treesit-settings'.
(js--treesit-font-lock-settings): New defvar renamed from
'js--json-treesit-settings'.
* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): New defvar
renamed from 'ts-mode--settings'.
Yuan Fu [Tue, 18 Oct 2022 20:06:25 +0000 (13:06 -0700)]
Install new toggle scheme for tree-sitter
Basically we now have treesit-mode and global-treesit-mode. Major
modes set major-mode-backend-function which treesit-mode calls to
activate/deactivate tree-sitter.
js.el needs a bit explanation: I'm so sorry for messing up the
history, but basically now js-mode and js-json-mode should be exactly
the same as before any tree-sitter change was introduced, sans some
initialization code that are moved to js(-json)--backend-toggle.
js-mode and js-json-mode now just sets major-mode-backend-function and
initialize with js(-json)--backend-toggle.
* lisp/treesit.el (treesit-mode-inhibit-message): New option.
(treesit-can-enable-p): Remove function.
(major-mode-backend-function)
(treesit-remapped-major-mode-alist): New variables.
(treesit-mode)
(global-treesit-mode): New minor modes.
(global-treesit-mode--turn-on)
(treesit-ready-p): New functions.
* lisp/progmodes/python.el: Remove option.
(python--backend-toggle): New function.
(python-mode): Remove the if-form, all the initialization code are
moved to python--backend-toggle, python-mode now just sets
major-mode-backend-function and initialize with
python--backend-toggle.
* lisp/progmodes/js.el (js--treesit-can-enable-p)
(js--json-treesit-can-enable-p)
(js--treesit-enable)
(js--json-treesit-enable): Remove functions.
(js--backend-toggle)
(js-json--backend-toggle): New function.
* lisp/progmodes/ts-mode.el (ts-mode): Use treesit-ready-p.
Yuan Fu [Mon, 17 Oct 2022 21:12:24 +0000 (14:12 -0700)]
Simplify error reported by loading tree-sitter language definition
Before:
Error: Cannot load language definition:
"javascript",
("/home/jostein/.emacs.d/tree-sitter/libtree-sitter-javascript:
cannot open shared object file: No such file or directory"
"/home/jostein/.emacs.d/tree-sitter/libtree-sitter-javascript.so:
cannot open shared object file: No such file or directory"
"libtree-sitter-javascript: cannot open shared object file: No such
file or directory" "libtree-sitter-javascript.so: cannot open shared
object file: No such file or directory")
After:
(treesit-load-language-error
not-found
("libtree-sitter-z.so" "libtree-sitter-z.dylib")
"No such file or directory")
* src/treesit.c (treesit_load_language): Add base_candidates, remove
error_list, report base_candidates rather than error_list.
Eli Zaretskii [Sun, 16 Oct 2022 13:22:09 +0000 (16:22 +0300)]
Support MinGW build on MS-Windows
* src/treesit.c [WINDOWSNT]: Add MS-Windows boilerplate for
dynamically-loaded optional libraries.
(init_treesit_functions) [WINDOWSNT]: New function.
(load_tree_sitter_if_necessary): New function.
(ts_initialize): Call 'load_tree_sitter_if_necessary'.
(ts_delete_parser, ts_delete_query, ts_named_node_p): Wrapper
functions for TS calls from outside treesit.c.
(Ftreesit_parser_root_node, Ftreesit_parser_set_included_ranges)
(Ftreesit_parser_included_ranges, Ftreesit_node_type)
(Ftreesit_node_start, Ftreesit_node_end, Ftreesit_node_string)
(Ftreesit_node_parent, Ftreesit_node_child, Ftreesit_node_check)
(Ftreesit_node_field_name_for_child, Ftreesit_node_child_count)
(Ftreesit_node_next_sibling, Ftreesit_node_prev_sibling)
(Ftreesit_node_first_child_for_pos)
(Ftreesit_node_descendant_for_range, Ftreesit_node_eq)
(Ftreesit_query_compile, Ftreesit_query_capture)
(Ftreesit_search_subtree, Ftreesit_search_forward)
(Ftreesit_induce_sparse_tree): Call 'ts_initialize' before any
other TS functions.
(Ftreesit_available_p): 'treesit-available-p' implemented in C, so
that on WINDOWSNT the library could be loaded dynamically.
* src/treesit.h (ts_delete_parser, ts_delete_query)
(ts_named_node_p): Add prototypes.
* src/print.c (print_vectorlike):
* src/alloc.c (cleanup_vector): Call tree-sitter function via
wrappers defined in treesit.c, not directly, because WINDOWSNT
redefines the TS functions to be called via function pointers.
* src/Makefile.in (base_obj): Add treesit.o
(TREE_SITTER_OBJ): Remove.
* lisp/treesit.el (treesit-available-p): Remove: now implemented
in C.
* lisp/term/w32-win.el (dynamic-library-alist): Add libtree-sitter
DLLs.
* configure.ac (TREE_SITTER): Support the MinGW build.
(TREE_SITTER_OBJ): Remove.
Yuan Fu [Thu, 13 Oct 2022 22:08:20 +0000 (15:08 -0700)]
Update tree-sitter manual to reflect the previous commit
* doc/lispref/modes.texi: Update manual to reflect previous change:
remove :toggle and :level, add :feature. Document new variables and
functions.
* doc/lispref/parsing.texi: Add the missing closing bracket in
@code{(t . nil)}.
Yuan Fu [Thu, 13 Oct 2022 21:44:42 +0000 (14:44 -0700)]
Rework tree-sitter font-lock
Remove :toggle and :level, add :feature.
* lisp/progmodes/js.el (js--treesit-settings): Add :feature.
(js--treesit-enable): Set treesit-font-lock-feature-list.
* lisp/progmodes/python.el: Replace :level with :feature.
(python-mode): Set treesit-font-lock-feature-list.
* lisp/treesit.el (treesit-font-lock-feature-list): New variable.
(treesit-font-lock-settings): Change format.
(treesit-font-lock-rules): Remove :toggle and :level, add :feature.
(treesit-font-lock-recompute-features): New function.
(treesit-font-lock-fontify-region): Change to work with the new
format.
(treesit-font-lock-enable): Add call to
treesit-font-lock-recompute-features. And improve the font-lock-mode
workaround.
Yuan Fu [Wed, 12 Oct 2022 17:38:35 +0000 (10:38 -0700)]
Better signal data for treesit-load-language-error
* src/treesit.c (ts_load_language): Better formatted signal data,
(Ftreesit_langauge_available_p): Optionally return the signal data if
language not available.
* doc/lispref/parsing.texi: Document the signal data in detail.
* lisp/progmodes/ts-mode.el (ts-mode): New major mode for TypeScript
with support for tree-sitter. It uses the TSX parser, so that we get
support for TSX as well as TypeScript. If we cannot find tree-sitter,
we default to using js-mode.
Yuan Fu [Tue, 11 Oct 2022 16:30:42 +0000 (09:30 -0700)]
Lazily compile tree-sitter query
See comment on struct Lisp_TS_Query for why. Previous commits adding
python and js support for tree-sitter breaks Emacs build if language
definitions aren't available at build time. This commit fixes that.
Now query object don't compile the query upon creation, but stores the
query source and language symbol, and compiles upon first use (in
treesit-query-capture).
I want ts_ensure_query_compiled to return signal symbol and data
rather than signaling itself, because it's a helper function not lisp
function. But because it calls ts_load_language, I had to update
ts_load_language to also use the signal symbol/data interface.
* src/treesit.h (struct Lisp_TS_Query): Add two new field.
* src/treesit.c (ts_load_language): Return signal symbol and data
rather than signaling itself.
(Ftreesit_langauge_available_p)
(Ftreesit_parser_create): Update usage of ts_load_language
(make_ts_query): Now returns a lisp object.
(ts_query_error_to_string): Moved because it's used by
ts_compose_query_signal_data.
(ts_compose_query_signal_data)
(ts_ensure_query_compiled): New functions.
(Ftreesit_query_compile): Delay compiling the query.
(Ftreesit_query_capture): Instead of creating a query object which
compiles the query, now goes two routes: if QUERY is a query object,
make sure it is compiled and use the TSQuery; if QUERY is a cons or
string, compile directly to TSQuery, and free it after use. Creating
a lisp query requires the language symbol, but in this function we
only have TSLanguage.
Yuan Fu [Mon, 10 Oct 2022 18:56:12 +0000 (11:56 -0700)]
Make treesit-node-at faster
* lisp/treesit.el (treesit-node-at): We only need to use
treesit-node-first-child-for-pos. This is both faster for large
buffers and simpler. Also remove the TODO comment.
Yuan Fu [Sun, 9 Oct 2022 22:17:11 +0000 (15:17 -0700)]
Make tree-sitter font-lock support decoration levels
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Font-Lock.html:
Update.
* admin/notes/tree-sitter/html-manual/Parsing-Program-Source.html:
Update.
* doc/lispref/modes.texi: Mention the new :level option.
* lisp/treesit.el (treesit-font-lock-settings): Update docstring.
(treesit-font-lock-rules): Support :level. Relayout the let form.
(treesit-font-lock-fontify-region): Support :level.
Justus Winter [Wed, 5 Oct 2022 10:36:39 +0000 (12:36 +0200)]
Replace aging OpenPGP artifacts in gnus' mml-sec tests.
This replaces the old OpenPGPv4 keys that are used in the test suite
with more modern OpenPGPv4 keys.
Having old cryptographic artifacts in the test suite presents a
problem once the old algorithms are rejected by contemporary
implementations (bug#58301).
Replace aging OpenPGP artifacts in the package-resources tests.
This replaces the old OpenPGPv4 key that is used in the test suite
with a more modern OpenPGPv4 key. The key is the Alice key from
https://datatracker.ietf.org/doc/draft-bre-openpgp-samples/. All
cryptographic artifacts in the test suite are updated accordingly.
Having old cryptographic artifacts in the test suite presents a
problem once the old algorithms are rejected by contemporary
implementations.
This replaces the old OpenPGPv4 key that is used in the test suite
with a more modern OpenPGPv4 key. The key is the Alice key from
https://datatracker.ietf.org/doc/draft-bre-openpgp-samples/. All
cryptographic artifacts in the test suite are updated accordingly.
Having old cryptographic artifacts in the test suite presents a
problem once the old algorithms are rejected by contemporary
implementations.
Previously, epg invoked GnuPG without an explicit command to verify
signatures. Make that operation explicit. Happily, this aligns how
gpg and gpgsm is invoked, so it actually makes the code simpler.
Manuel Giraud [Thu, 6 Oct 2022 12:46:39 +0000 (14:46 +0200)]
Fix BSD battery mode over TRAMP (bug#58307)
* lisp/battery.el (battery--call-process-to-string): Simple wrapper
to `call-process' that outputs to a string.
(battery-bsd-apm): Use it instead of `shell-command-to-string'
(bug#58307).
Matt Armstrong [Thu, 6 Oct 2022 12:08:20 +0000 (14:08 +0200)]
Fix calling `package-reinstall' just after quick initialization.
* lisp/emacs-lisp/package.el (package-reinstall): call
`package--archives-initialize', just like `package-install' does.
This populates `package-alist', and so fixes calling
`package-reinstall' as the first thing done after package "quick
init" (Bug#53527).
Po Lu [Thu, 6 Oct 2022 02:55:25 +0000 (10:55 +0800)]
Minor fixes to IM locale handling
* lisp/term/x-win.el (x-get-input-coding-system): Translate locales.
* src/xterm.c (x_term_init): If the X library doesn't support
the current locale, don't set up input methods.
Yuan Fu [Wed, 5 Oct 2022 21:11:33 +0000 (14:11 -0700)]
Add tree-sitter admin notes
starter-guide: Guide on writing major mode features.
build-module: Script for building official language definitions.
html-manual: HTML version of the manual for easy access.
* admin/notes/tree-sitter/build-module/README: New file.
* admin/notes/tree-sitter/build-module/batch.sh: New file.
* admin/notes/tree-sitter/build-module/build.sh: New file.
* admin/notes/tree-sitter/starter-guide: New file.
* admin/notes/tree-sitter/html-manual/Accessing-Node.html: New file.
* admin/notes/tree-sitter/html-manual/Language-Definitions.html: New file.
* admin/notes/tree-sitter/html-manual/Multiple-Languages.html: New file.
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Font-Lock.html:
New file.
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Indentation.html:
New file.
* admin/notes/tree-sitter/html-manual/Parsing-Program-Source.html: New
file.
* admin/notes/tree-sitter/html-manual/Pattern-Matching.html: New file.
* admin/notes/tree-sitter/html-manual/Retrieving-Node.html: New file.
* admin/notes/tree-sitter/html-manual/Tree_002dsitter-C-API.html: New
file.
* admin/notes/tree-sitter/html-manual/Using-Parser.html: New file.
* admin/notes/tree-sitter/html-manual/build-manual.sh: New file.
* admin/notes/tree-sitter/html-manual/manual.css: New file.
Stefan Kangas [Wed, 5 Oct 2022 13:23:39 +0000 (15:23 +0200)]
Add face help-key-binding in help-char echo
* src/keyboard.c (echo_add_key): Add help-key-binding face to the
"?" key in 'help-char' echo. (Bug#57906)
(syms_of_keyboard) <Qhelp_key_binding>: New DEFSYM.
Po Lu [Wed, 5 Oct 2022 00:31:44 +0000 (08:31 +0800)]
Fix bug in "macintization" of x_draw_glyph_string
* src/nsterm.m (ns_draw_stretch_glyph_string): Restore text decoration
drawing code ommitted during "macintization" to convert the X function
into NS code. Reported by Qiantan Hong <qthong@stanford.edu>.
Yuan Fu [Tue, 4 Oct 2022 20:28:46 +0000 (13:28 -0700)]
Add :override flag for tree-sitter font-lock
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/treesit.el (treesit-font-lock-settings): Update docstring.
(treesit-font-lock-rules): Handle :override.
(treesit-font-lock-fontify-region): Handle :override. Also set
inhibit-point-motion-hooks to t.
Yuan Fu [Sun, 2 Oct 2022 03:25:25 +0000 (20:25 -0700)]
Ignore some capture name in treesit-font-lock-fontify-region
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/treesit.el: (treesit-font-lock-fontify-region): Ignore names
that are not face nor function.
(treesit-font-lock-rules): Update docstring.
src/comp.c: Use libgccjit's bitcast API for type coercion, when available.
* (type_to_cast_index, define_type_punning, define_cast_from_to,
define_cast_functions): Define functions when
gcc_jit_context_new_bitcast is not available.
* (emit_coerce): Use gcc_jit_context_new_bitcast to coerce types, when
available.
Brian Cully [Tue, 4 Oct 2022 16:56:23 +0000 (18:56 +0200)]
Provide Podman containers on their own method in Tramp
* lisp/net/tramp-docker.el (tramp-podman-program): New defcustom.
(tramp-podman-method): New defconst.
(tramp-docker-program): Remove "podman" from option list.
(top): Update comments to refer to Docker-alike where necessary.
Add description for how to use the podman method.
Alan Mackenzie [Tue, 4 Oct 2022 16:19:33 +0000 (16:19 +0000)]
CC Mode: Optimize c-fontify-new-found-type and amend a debug spec
* lisp/progmodes/cc-fonts.el (c-fontify-new-found-type): Write the `face'
property directly, rather than removing `fontified' properties and letting
font-lock do the work.
* lisp/progmodes/cc-defs.el (cc-eval-when-compile): Amend the debug spec from
t to (&rest body), in line with the fix to bug #16184.
Stefan Kangas [Tue, 4 Oct 2022 15:36:09 +0000 (17:36 +0200)]
image-dired: Shorten several long names
* lisp/image/image-dired.el (image-dired-image-mode-map)
(image-dired-image-mode): Rename from
'image-dired-display-image-mode-map' and
'image-dired-display-image-mode'. Update all uses and make old
names into obsolete aliases.
(image-dired-display-this, image-dired-display-next)
(image-dired-display-previous): Rename from
'image-dired-display-thumbnail-original-image',
'image-dired-display-next-thumbnail-original', and
'image-dired-display-previous-thumbnail-original'. Update all uses
and make old names into obsolete aliases.
* doc/emacs/dired.texi (Image-Dired): Update documentation for the
above changes, and improve indexing.