Yuan Fu [Sun, 6 Nov 2022 01:58:39 +0000 (18:58 -0700)]
Fix incorrect tree-sitter fontification
* lisp/treesit.el (treesit-font-lock-fontify-region): If the captured
node is outside of the region between START and END, don't fontify it.
Wrap fontification code in a when form.
Yuan Fu [Sun, 6 Nov 2022 01:32:01 +0000 (18:32 -0700)]
Allow tree-sitter indent offset to be a variable
* doc/lispref/modes.texi (Parser-based Indentation): Update manual.
* lisp/progmodes/js.el (js--treesit-indent-rules): Change all
occurance of ,js-indent-level to js-indent-level.
* lisp/progmodes/ts-mode.el (ts-mode--indent-rules): Change all
occurance of ,ts-mode-indent-offset to ts-mode-indent-offset.
* lisp/treesit.el (treesit-simple-indent-rules): Change docstring.
(treesit-simple-indent): Allow offset to be a variable.
Yuan Fu [Fri, 4 Nov 2022 08:31:56 +0000 (01:31 -0700)]
Revise tree-sitter facility for multi-language buffers
* doc/lispref/parsing.texi (Multiple Languages): Extend and update
manual.
* lisp/treesit.el (treesit-range-functions): Remove variable.
(treesit-range-settings): New variable.
(treesit-range-rules): New function.
(treesit--merge-ranges): New function.
(treesit-update-ranges): Use treesit-range-settings instead of
treesit-range-functions.
(treesit-font-lock-rules): Fix docstring.
(treesit-indent)
(treesit-indent-region): Only update ranges in a region.
* test/src/treesit-tests.el (treesit-range): New test.
Yuan Fu [Thu, 3 Nov 2022 00:47:14 +0000 (17:47 -0700)]
Check for outdated tree-sitter node when printing
* src/print.c (print_vectorlike): Check for outdated node.
* src/treesit.c (treesit_check_node): Use the new function.
(treesit_node_uptodate_p): New function.
* src/treesit.h: Declare new function.
Yuan Fu [Thu, 3 Nov 2022 00:03:03 +0000 (17:03 -0700)]
Pass region start and end to tree-sitter fontification functions
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Update
function to only fontify within the region.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Pass START and END to
fontification functions.
Yuan Fu [Wed, 2 Nov 2022 23:31:25 +0000 (16:31 -0700)]
Add handling of contextual entities in tree-sitter font-lock
* lisp/progmodes/python.el: Remove function.
(python--treesit-settings): Capture contextual node.
* lisp/treesit.el (treesit--set-nonsticky):
(treesit-font-lock-contextual-post-process): New functions.
(treesit-font-lock-fontify-region): Change local variable START and
END to NODE-START and NODE-END, handle special capture name
"contextual".
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
Yuan Fu [Tue, 1 Nov 2022 20:26:16 +0000 (13:26 -0700)]
Fix string fontification for tree-sitter python-mode
Now when user inserts a ending quote, the whole string is ganranteed
to be refontified and redisplayed.
* lisp/progmodes/python.el (python--treesit-fontify-string): Change
docstring, now it's called on the leading quote.
(python--treesit-fontify-string-end): New function.
(python--treesit-settings): Capture both leading and ending quote in a
string.
Yuan Fu [Mon, 31 Oct 2022 22:27:40 +0000 (15:27 -0700)]
Query on the root node in tree-sitter font-lock
Rather than querying on the smallest node that spans START to END, we
query on the root node between START to END.
* lisp/progmodes/python.el (python--treesit-fontify-string): Accept
the string rather than the quote node.
(python--treesit-settings): Capture the string rather than the quote
node.
* lisp/treesit.el (treesit-font-lock-fontify-region): Query the root
node rather than the smallest node.
Yuan Fu [Mon, 31 Oct 2022 06:59:56 +0000 (23:59 -0700)]
Use tree-sitter for font-lock's syntactic function (optionally)
Before this change, we rebind font-lock-fontify-region-function and
call font-lock's function in tree-sitter's function, and the order of
fontification is tree-sitter, font-lock syntax, font-lock regexp.
Now we make font-lock's syntax function customizable and replace it
with tree-sitter's function, and the order of fontification is
tree-sitter, font-lock regexp.
* doc/lispref/modes.texi (Parser-based Font Lock): Reflect the change
in manual.
* lisp/font-lock.el (font-lock-fontify-syntactically-function): New
varaible.
(font-lock-default-fontify-region): Call
font-lock-fontify-syntactically-function rather.
(font-lock-fontify-syntactically-region): Rename to
font-lock-default-fontify-syntactically
(font-lock-default-fontify-syntactically): Rename to this.
* lisp/treesit.el (treesit-font-lock-fontify-region): Don't call
font-lock functions.
(treesit-font-lock-enable): Remove this function. It is not used even
before this change.
(treesit-major-mode-setup): Instead of binding
font-lock-fontify-region-function, now bind to
font-lock-fontify-syntactically-function. And we can let font-lock do
it's thing.
Yuan Fu [Mon, 31 Oct 2022 03:59:50 +0000 (20:59 -0700)]
Improve python tree-sitter's string fontification
* lisp/progmodes/python.el (python--treesit-fontify-string): Handle
not only f-strings, but also docstrings, and NODE is now the last
quote rather than the whole string.
(python--treesit-settings): Use python--treesit-fontify-string for
every occasion.
Yuan Fu [Mon, 31 Oct 2022 03:56:21 +0000 (20:56 -0700)]
Add tresit--font-lock-verbose
* lisp/treesit.el (tresit--font-lock-verbose): New variable.
(treesit-font-lock-fontify-region): Take tresit--font-lock-verbose
into account. Return jit-lock-bounds.
Yuan Fu [Sat, 29 Oct 2022 16:43:50 +0000 (09:43 -0700)]
Refine handling of node at point in treesit-indent
* doc/lispref/modes.texi (Parser-based Indentation): Document the
semantic clearly.
* lisp/treesit.el (treesit-simple-indent-presets): Replace with
current-indentation.
(treesit--indent-1): Now NODE always starts at BOL, and parent is
always the smallest node above NODE that spans BOL.
(treesit-indent): Don't use save-excurtion, no one else does.
Yuan Fu [Sat, 29 Oct 2022 16:04:35 +0000 (09:04 -0700)]
Fix treesit-indent-region
* lisp/treesit.el (treesit-simple-indent-presets): Remove extra comma.
(treesit--indent-1): Return (ANCHOR . OFFSET) instead of column.
(treesit-indent): Accept (ANCHOR . OFFSET) from treesit--indent-1.
(treesit--indent-region-batch-size): Reduce to 400.
(treesit-indent-region): Put a marker on each line's ANCHOR and
compute the indentation with ANCHOR + OFFSET. Precomputing column
gives wrong indentation when the ANHOR line doesn't have correct
indent yet.
Yuan Fu [Thu, 27 Oct 2022 23:44:33 +0000 (16:44 -0700)]
Remove tree-sitter-indent's integration with cc-mode
Also minor improvement of tree-sitter-indent error messaging.
* lisp/progmodes/js.el (js--treesit-indent-rules): Fix it so it works
with the new presets (which expects regexps).
(js--treesit-cc-indent-rules): Remove variable.
(js-mode): Go back to js--treesit-indent-rules.
* lisp/treesit.el (treesit-indent): Don't report error message when
ANCHOR or OFFSET is nil: let treesit-simple-indent display the
message.
(treesit-simple-indent): Remove cc-mode integration. Display error
message when no rules matches.
Yuan Fu [Thu, 27 Oct 2022 01:31:43 +0000 (18:31 -0700)]
Plug tree-sitter-simple-indent into c-offset-alist
Now tree-sitter indentation can produce a cc-engine syntax symbol and
use c-offset-alist to compute the offset. Catch: line-up functions
don't work with tree-sitter.
* lisp/progmodes/js.el (js--treesit-cc-indent-rules): New variable.
(js-mode): Use cc-indent rules by default.
* lisp/treesit.el (treesit-simple-indent-presets): Consider types as
regexp now. New matchers: n-p-gp, field-is, top-level, catch-all.
New anchors: nth-sibling, grand-parent, and, or, not, list.
first-sibling now returns the actual first sibling rather than the
first named sibling.
Yuan Fu [Wed, 26 Oct 2022 22:34:02 +0000 (15:34 -0700)]
Fix tree-sitter navigation
* lisp/treesit.el (treesit-search-forward-goto): Track current
position with a local variable instead of (point). Clean up the
condition forms.
(treesit-beginning-of-defun): Fix traverse direction.
Yuan Fu [Tue, 25 Oct 2022 21:26:41 +0000 (14:26 -0700)]
Fix treesit-language-at
* lisp/treesit.el (treesit-language-at-point-function): New local
variable.
(treesit-language-at): Change to use
treesit-language-at-point-function rather than trying each parser
one-by-one.
* doc/lispref/parsing.texi (Multiple Languages): Update manual.
Yuan Fu [Tue, 25 Oct 2022 20:54:12 +0000 (13:54 -0700)]
New tree-sitter toggle scheme
This version: central variable, everything controlled by
treesit-settings. Major mode sets up tree-sitter/non-tree-sitter
in a conditional branch, based on the setting.
* lisp/treesit.el (treesit-settings): New option.
(treesit-defun-type-regexp): Change docstring.
(treesit-mode-supported)
(treesit-required-languages)
(treesit--local-variable-backup): Remove variables.
(treesit--backup-local-variable)
(treesit-mode)
(global-treesit-mode--turn-on)
(global-treesit-mode): Remove functions.
(treesit--setting-for-mode): New function.
(treesit-ready-p): New argument MODE, changed REPORT to QUIET, and
LANGUAGEs to LANGUAGE (now it can be a single symbol or a list of
them).
(treesit-major-mode-setup): New function. Mostly comes from
treesit-mode.
* test/src/treesit-tests.el (treesit-misc): New test.
* lisp/progmodes/python.el (python-mode): Move some setup code into
the conditional branch at the end.
* lisp/progmodes/js.el (js-json-mode)
(js-mode): Move some setup code into the conditional branch at the
end.
* lisp/progmodes/ts-mode.el: Move tree-sitter setup into the
conditional branch.
Yuan Fu [Mon, 24 Oct 2022 01:47:02 +0000 (18:47 -0700)]
Fix treesit-beginning/end-of-defun
Now they always move to the top-level defun (i.e., skips nested ones).
* lisp/treesit.el (treesit--find-top-level-match): New function
(treesit-beginning-of-defun): Goes to the top-level match.
(treesit-end-of-defun): Simply move to the end of current match.
Yuan Fu [Mon, 24 Oct 2022 01:29:02 +0000 (18:29 -0700)]
Make treesit-search-forward-goto accept a NODE argument
With NODE argument we can do
(setq node (treesit-search-forward-goto node))
And we can choose what node to pass to it (maybe we want to pass it
the largest node at point, rather than the smallest node, and in case
of multiple parsers, we can choose which parser to use).
Yuan Fu [Mon, 24 Oct 2022 01:23:17 +0000 (18:23 -0700)]
Make treesit-node-at return the last leaf node at EOB
* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Add a (when node) form just to be
explicit, return the last leaf node if there is no node after POS.