Yuan Fu [Mon, 21 Nov 2022 00:56:33 +0000 (16:56 -0800)]
Limit recursion level for tree-sitter imenu functions
Generating imenu index doesn't require going down to the bottom of the
tree (defun's are usually top-level). Add limit so we don't go too
far down on very large buffers.
Yuan Fu [Mon, 21 Nov 2022 00:37:19 +0000 (16:37 -0800)]
Improve tree-sitter fontification on large buffers
* lisp/treesit.el (treesit--children-covering-range)
(treesit--children-covering-range-recurse): New functions. They are
not currently used but could be useful in the future, so I left them
in place.
(treesit-font-lock-fontify-region):
* lisp/treesit.el (treesit-font-lock-fontify-region): Use the result
of treesit-node-on instead of the root node.
Yuan Fu [Sun, 20 Nov 2022 00:56:42 +0000 (16:56 -0800)]
Separate native and tree-sitter variant of js-mode
* lisp/progmodes/js.el (js-base-mode): New mode.
(js-mode): Return to before tree-sitter setup is added, change to
inherit from js-base-mode.
(js-ts-mode): New mode. Now it doesn't use any cc-mode feature,
meaning it looses comment filling.
Yuan Fu [Sun, 20 Nov 2022 00:09:08 +0000 (16:09 -0800)]
Split python-mode into native and tree-sitter variant
* lisp/progmodes/python.el (python-base-mode): New virtual mode that
contains most of the setup.
(python-mode): Change to inherit from python-base-mode.
(python-ts-mode): New mode that sets up tree-sitter.
Yuan Fu [Sat, 19 Nov 2022 23:27:07 +0000 (15:27 -0800)]
Expand css-ts-mode and merge it into css-mode
* lisp/progmodes/css-ts-mode.el: Deleted.
* lisp/textmodes/css-mode.el (css--treesit-indent-rules)
(css--treesit-settings): New variables.
(css--treesit-imenu-1)
(css--treesit-imenu): New functions.
* lisp/textmodes/css-mode.el (css-base-mode): New mode inherited by
both css-mode and css-ts-mode.
(css-ts-mode): New mode.
(css-mode): Inherit from css-base-mode, and move some setup to
css-base-mode.
Yuan Fu [Sat, 19 Nov 2022 08:25:57 +0000 (00:25 -0800)]
Fix python-mode tree-sitter fontification
Forgot to update python-mode's code when I added the tree-sitter
notifier facility. Now it doesn't need any special treatment anymore.
Leaving it as is causes some incorrect fontification.
* lisp/progmodes/python.el (python--treesit-fontify-string): Accept
the string node rather than the quote node.
(python--treesit-settings): Capture string node.
Randy Taylor [Mon, 14 Nov 2022 03:06:33 +0000 (22:06 -0500)]
Utilize new font-lock faces for C/C++ and Python tree-sitter use
* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): Use new
escape, number, property, operator, bracket, and delimiter font-lock
faces.
(c-ts-mode--base-mode): Add them to the feature list.
(c-ts-mode): Fix typo.
* lisp/progmodes/python.el (python--treesit-settings): Use new number,
property, operator, bracket, and delimiter font-lock faces.
(python-mode): Add them to the feature list.
Yuan Fu [Thu, 17 Nov 2022 05:31:04 +0000 (21:31 -0800)]
; Elaborate on tree-sitter fontification features documentation
* doc/lispref/modes.texi (Parser-based Font Lock)
* lisp/treesit.el (treesit-font-lock-feature-list): Explain in more
detail, and change some features.
Yuan Fu [Wed, 16 Nov 2022 22:33:48 +0000 (14:33 -0800)]
Tweaks on c-ts-mode fontification rules
Changes:
1. Remove some :override flags
2. Fontify "!" in font-lock-negation-char-face
3. Use some new faces like font-lock-escape-face and
font-lock-property-face
4. Move all type fontification into 'type' feature. Previously some
are in 'definition' feature.
5. Simplify definition feature, and fontify identifiers in definitions
recursively.
6. Change 'statement' feature to 'label'
7. New feature 'assignment'
8. Remove some rules from 'expression' feature
* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): See
above.
(c-ts-mode--fontify-declarator): New function.
(c-ts-mode): Update feature-list.
Yuan Fu [Wed, 16 Nov 2022 18:00:42 +0000 (10:00 -0800)]
Fix c-ts-mode-imenu
* lisp/progmodes/c-ts-mode.el (c-ts-mode--imenu-1): If cannot find
its name, ignore the node.
(c-ts-mode--imenu): Don't include a category if it's empty.
Yuan Fu [Tue, 15 Nov 2022 10:30:50 +0000 (02:30 -0800)]
Make tree-sitter fontification automatically update
Now nodes that are affected and changed during re-parse will be
correctly refontified. Nodes can change even if it corresponding text
wasn't edited: additional text can complete the parse tree and resolve
error nodes, for example.
* lisp/progmodes/python.el (python-mode): Create Python parser before
calling treesit-major-mode-setup.
* lisp/treesit.el (treesit--font-lock-notifier): New function.
(treesit-major-mode-setup): Register fontifier with every existing
parser.
Yuan Fu [Tue, 15 Nov 2022 10:22:41 +0000 (02:22 -0800)]
Remove the contextual hack in tree-sitter fontification
We now have a better facility that can replace the contextual
hack. The C part is in the previous commit, and the Lisp part work
will be in the next commit.
Yuan Fu [Tue, 15 Nov 2022 10:16:11 +0000 (02:16 -0800)]
Allow tree-sitter to notify parse-tree changes
* src/treesit.c (treesit_call_after_change_functions): New function.
(treesit_ensure_parsed): Call treesit_call_after_change_functions
right after re-parse.
(make_treesit_parser): Initialize after_change_functions.
(Ftreesit_parser_notifiers)
(Ftreesit_parser_add_notifier)
(Ftreesit_parser_remove_notifier): New functions.
* src/treesit.h (Lisp_TS_Parser): New field after_change_functions.
* lisp/progmodes/java-ts-mode.el
(java-ts-mode--imenu): Add categories. Only display categories that
exist in the file.
(java-ts-mode): Add electric-indent.
Yuan Fu [Mon, 14 Nov 2022 08:51:54 +0000 (00:51 -0800)]
Remove feature that checks whether tree-sitter node "has changes"
First of all, we signal error on using an outdated node, so if a node
has changes, calling treesit-node-check would only raise an error.
Besides, in order to properly answer whether a node has changed, we
would have to update the node as the buffer is edited, which we don't
do right now.
Eli Zaretskii [Sun, 13 Nov 2022 08:51:42 +0000 (10:51 +0200)]
Fix crash on MS-Windows due to memory-allocation problem in treesit.c
* src/treesit.c (treesit_load_language): Use 'xstrdup'/'xfree'
instead of 'strdup'/'free', to prevent crashes on MS-Windows,
where we must use our own implementation of 'malloc'/'free',
whereas 'strdup' uses the default implementation in the MS-Windows
C runtime library.
Yuan Fu [Sat, 12 Nov 2022 23:57:02 +0000 (15:57 -0800)]
Improve tree-sitter imenu for c-mode and js-mode
Instead of a flat list, now categorize imenu entries into categories
like "Function", "Variable", "Class", etc.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--imenu-1): Handle more edge
cases.
(c-ts-mode--imenu): Categorize menu entries.
* lisp/progmodes/js.el (js--treesit-imenu-type-alist): Remove variable.
(js--treesit-imenu-label): Remove function.
(js--treesit-imenu-1): Use the name alone for labels.
(js--treesit-imenu): Categorize menu entries.
* etc/NEWS: Mention the new modes
* lisp/progmodes/c-ts-mode.el: New major mode with tree-sitter support
for C and C++.
* lisp/progmodes/java-ts-mode.el: New major mode with tree-sitter support.
* lisp/progmodes/json-ts-mode.el: New major mode with tree-sitter support.
* lisp/progmodes/css-ts-mode.el: New major mode with tree-sitter support.
Yuan Fu [Sat, 12 Nov 2022 04:04:38 +0000 (20:04 -0800)]
Improve treesit-node-at
* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Change semantic. It tries to
return the node that a user would expect in various circumstances.
* test/src/treesit-tests.el (treesit-node-at): New test.
Yuan Fu [Thu, 10 Nov 2022 23:00:29 +0000 (15:00 -0800)]
In end-of-defun, terminate early if no further defun exists
Before this change, end-of-defun calls end-of-defun-function even if
point is not necessarily at the beginning of a defun (contrary to what
end-of-defun-function's docstring claims). Now it terminates early
and doesn't call end-of-defun-function.
* lisp/emacs-lisp/lisp.el (beginning-of-defun-raw): Update docstring
clarifying the return value.
(end-of-defun): Terminate early if beginning-of-defun-raw returns nil.
Yuan Fu [Wed, 9 Nov 2022 20:24:43 +0000 (12:24 -0800)]
New function fast_c_string_match and fast_c_string_match_internal
Generalize fast_c_string_match_ignore_case into
fast_c_string_match_internal. And Make fast_c_string and
fast_c_string_match_ignore_case use fast_c_string_match_internal.
* src/lisp.h (fast_c_string_match_internal)
(fast_c_string_match): New declaration.
(fast_c_string_match_ignore_case): Change to thin wrapper.
* src/search.c (fast_c_string_match_internal): New function.
Yuan Fu [Sun, 6 Nov 2022 01:58:39 +0000 (18:58 -0700)]
Fix incorrect tree-sitter fontification
* lisp/treesit.el (treesit-font-lock-fontify-region): If the captured
node is outside of the region between START and END, don't fontify it.
Wrap fontification code in a when form.
Yuan Fu [Sun, 6 Nov 2022 01:32:01 +0000 (18:32 -0700)]
Allow tree-sitter indent offset to be a variable
* doc/lispref/modes.texi (Parser-based Indentation): Update manual.
* lisp/progmodes/js.el (js--treesit-indent-rules): Change all
occurance of ,js-indent-level to js-indent-level.
* lisp/progmodes/ts-mode.el (ts-mode--indent-rules): Change all
occurance of ,ts-mode-indent-offset to ts-mode-indent-offset.
* lisp/treesit.el (treesit-simple-indent-rules): Change docstring.
(treesit-simple-indent): Allow offset to be a variable.
Yuan Fu [Fri, 4 Nov 2022 08:31:56 +0000 (01:31 -0700)]
Revise tree-sitter facility for multi-language buffers
* doc/lispref/parsing.texi (Multiple Languages): Extend and update
manual.
* lisp/treesit.el (treesit-range-functions): Remove variable.
(treesit-range-settings): New variable.
(treesit-range-rules): New function.
(treesit--merge-ranges): New function.
(treesit-update-ranges): Use treesit-range-settings instead of
treesit-range-functions.
(treesit-font-lock-rules): Fix docstring.
(treesit-indent)
(treesit-indent-region): Only update ranges in a region.
* test/src/treesit-tests.el (treesit-range): New test.
Yuan Fu [Thu, 3 Nov 2022 00:47:14 +0000 (17:47 -0700)]
Check for outdated tree-sitter node when printing
* src/print.c (print_vectorlike): Check for outdated node.
* src/treesit.c (treesit_check_node): Use the new function.
(treesit_node_uptodate_p): New function.
* src/treesit.h: Declare new function.
Yuan Fu [Thu, 3 Nov 2022 00:03:03 +0000 (17:03 -0700)]
Pass region start and end to tree-sitter fontification functions
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Update
function to only fontify within the region.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Pass START and END to
fontification functions.
Yuan Fu [Wed, 2 Nov 2022 23:31:25 +0000 (16:31 -0700)]
Add handling of contextual entities in tree-sitter font-lock
* lisp/progmodes/python.el: Remove function.
(python--treesit-settings): Capture contextual node.
* lisp/treesit.el (treesit--set-nonsticky):
(treesit-font-lock-contextual-post-process): New functions.
(treesit-font-lock-fontify-region): Change local variable START and
END to NODE-START and NODE-END, handle special capture name
"contextual".
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.