]> git.eshelyaron.com Git - emacs.git/log
emacs.git
2 years agoAdd another heuristic to speed up tree-sitter fontification
Yuan Fu [Mon, 21 Nov 2022 01:15:42 +0000 (17:15 -0800)]
Add another heuristic to speed up tree-sitter fontification

* lisp/treesit.el (treesit-font-lock-fontify-region): Use
treesit--children-covering-range to reduce the node size.

2 years agoLimit recursion level for tree-sitter imenu functions
Yuan Fu [Mon, 21 Nov 2022 00:56:33 +0000 (16:56 -0800)]
Limit recursion level for tree-sitter imenu functions

Generating imenu index doesn't require going down to the bottom of the
tree (defun's are usually top-level).  Add limit so we don't go too
far down on very large buffers.

* lisp/progmodes/c-ts-mode.el (c-ts-mode--imenu)
* lisp/progmodes/java-ts-mode.el (java-ts-mode--imenu)
* lisp/progmodes/js.el (js--treesit-imenu)
* lisp/progmodes/json-ts-mode.el (json-ts-mode--imenu)
* lisp/progmodes/python.el (python-imenu-treesit-create-index)
* lisp/textmodes/css-mode.el (css--treesit-imenu): Add limit to
treesit-induce-sparse-tree.

2 years agoImprove tree-sitter fontification on large buffers
Yuan Fu [Mon, 21 Nov 2022 00:37:19 +0000 (16:37 -0800)]
Improve tree-sitter fontification on large buffers

* lisp/treesit.el (treesit--children-covering-range)
(treesit--children-covering-range-recurse): New functions.  They are
not currently used but could be useful in the future, so I left them
in place.
(treesit-font-lock-fontify-region):
* lisp/treesit.el (treesit-font-lock-fontify-region): Use the result
of treesit-node-on instead of the root node.

2 years ago; Improve recently-modified documentation
Eli Zaretskii [Sun, 20 Nov 2022 08:30:38 +0000 (10:30 +0200)]
; Improve recently-modified documentation

* lisp/progmodes/python.el (python-base-mode)
(python--treesit-fontify-string):
* lisp/textmodes/css-mode.el (css-base-mode):
* lisp/treesit.el (treesit-ready-p, treesit-comment-end)
(treesit-simple-indent-presets): Fix typos and wording in doc strings.

* doc/lispref/parsing.texi (Tree-sitter major modes):
* doc/lispref/modes.texi (Parser-based Indentation): Fix wording
and add cross-references.

2 years agoFix uses of treesit-ready-p
Yuan Fu [Sun, 20 Nov 2022 02:58:12 +0000 (18:58 -0800)]
Fix uses of treesit-ready-p

* lisp/progmodes/c-ts-mode.el (c-ts-mode)
(c++-ts-mode)
* lisp/progmodes/java-ts-mode.el (java-ts-mode)
* lisp/progmodes/js.el (js-ts-mode)
* lisp/progmodes/json-ts-mode.el (json-ts-mode)
* lisp/progmodes/python.el (python-ts-mode)
* lisp/progmodes/sh-script.el (sh-mode)
* lisp/progmodes/ts-mode.el (ts-mode): Remove the MODE argument.

2 years agoRemove treesit-settings
Yuan Fu [Sun, 20 Nov 2022 02:53:53 +0000 (18:53 -0800)]
Remove treesit-settings

Remove it since we are using separate major modes for tree-sitter and
native variant now.

* doc/lispref/parsing.texi (Tree-sitter major modes): Update manual.
* lisp/treesit.el (treesit-settings): Remove option.
(treesit--setting-for-mode): Remove function.
(treesit-ready-p): Don't check for user preference in treesit-settings.

2 years agoRemove tree-sitter setup from js-json-mode
Yuan Fu [Sun, 20 Nov 2022 02:45:45 +0000 (18:45 -0800)]
Remove tree-sitter setup from js-json-mode

* lisp/progmodes/js.el (js-json--treesit-font-lock-settings)
(js--json-treesit-indent-rules): Remove variables.
(js-json-mode): Remove tree-sitter setup.

2 years agoFix tree-sitter comment indentation for C-like languages
Yuan Fu [Sun, 20 Nov 2022 01:59:14 +0000 (17:59 -0800)]
Fix tree-sitter comment indentation for C-like languages

The goal is to indent like this:

/* comment
   comment  --> This line aligns with the beginning of the first line
 */         --> This line aligns with the opening comment token

* lisp/treesit.el (treesit-comment-start)
(treesit-comment-end): New variables.
(treesit-simple-indent-presets): New preset comment-end,
comment-start, comment-start-skip

* lisp/progmodes/c-ts-mode.el (c-ts-mode--indent-styles)
(c-ts-mode)
* lisp/progmodes/java-ts-mode.el (java-ts-mode--indent-rules)
(java-ts-mode)
* lisp/progmodes/js.el (js--treesit-indent-rules)
(js-ts-mode)
* lisp/progmodes/ts-mode.el (ts-mode--indent-rules)
(ts-mode): Add identical indent rules to each mode, and set identical
treesit-comment-start/end's.

* doc/lispref/modes.texi (Parser-based Indentation)
* doc/lispref/parsing.texi (Tree-sitter major modes): Update manual.

2 years agoSeparate native and tree-sitter variant of js-mode
Yuan Fu [Sun, 20 Nov 2022 00:56:42 +0000 (16:56 -0800)]
Separate native and tree-sitter variant of js-mode

* lisp/progmodes/js.el (js-base-mode): New mode.
(js-mode): Return to before tree-sitter setup is added, change to
inherit from js-base-mode.
(js-ts-mode): New mode.  Now it doesn't use any cc-mode feature,
meaning it looses comment filling.

2 years agoMerge function/class-name features in python-mode
Yuan Fu [Sun, 20 Nov 2022 00:10:46 +0000 (16:10 -0800)]
Merge function/class-name features in python-mode

To keep consistency with other tree-sitter major modes.

* lisp/progmodes/python.el (python--treesit-settings)
(python-ts-mode): Merge function-name and class-name features into
definition.

2 years agoSplit python-mode into native and tree-sitter variant
Yuan Fu [Sun, 20 Nov 2022 00:09:08 +0000 (16:09 -0800)]
Split python-mode into native and tree-sitter variant

* lisp/progmodes/python.el (python-base-mode): New virtual mode that
contains most of the setup.
(python-mode): Change to inherit from python-base-mode.
(python-ts-mode): New mode that sets up tree-sitter.

2 years agoExpand css-ts-mode and merge it into css-mode
Yuan Fu [Sat, 19 Nov 2022 23:27:07 +0000 (15:27 -0800)]
Expand css-ts-mode and merge it into css-mode

* lisp/progmodes/css-ts-mode.el: Deleted.
* lisp/textmodes/css-mode.el (css--treesit-indent-rules)
(css--treesit-settings): New variables.
(css--treesit-imenu-1)
(css--treesit-imenu): New functions.
* lisp/textmodes/css-mode.el (css-base-mode): New mode inherited by
both css-mode and css-ts-mode.
(css-ts-mode): New mode.
(css-mode): Inherit from css-base-mode, and move some setup to
css-base-mode.

2 years agoChange tree-sitter indent preset 'match' to count on all nodes
Yuan Fu [Sat, 19 Nov 2022 23:25:06 +0000 (15:25 -0800)]
Change tree-sitter indent preset 'match' to count on all nodes

I changed other presets to count all nodes a while ago, but forgot to
change 'match' too.

* lisp/treesit.el (treesit-simple-indent-presets): Change to count on
both named and anonymous nodes.

2 years agoUse keyword-face for declaration commands in tree-sitter sh-mode
Yuan Fu [Sat, 19 Nov 2022 09:33:36 +0000 (01:33 -0800)]
Use keyword-face for declaration commands in tree-sitter sh-mode

* lisp/progmodes/sh-script.el: Change builtin to keyword.

2 years ago; * lisp/treesit.el (treesit--font-lock-notifier): Add messaging.
Yuan Fu [Sat, 19 Nov 2022 08:41:19 +0000 (00:41 -0800)]
; * lisp/treesit.el (treesit--font-lock-notifier): Add messaging.

2 years agoFix python-mode tree-sitter fontification
Yuan Fu [Sat, 19 Nov 2022 08:25:57 +0000 (00:25 -0800)]
Fix python-mode tree-sitter fontification

Forgot to update python-mode's code when I added the tree-sitter
notifier facility.  Now it doesn't need any special treatment anymore.
Leaving it as is causes some incorrect fontification.

* lisp/progmodes/python.el (python--treesit-fontify-string): Accept
the string node rather than the quote node.
(python--treesit-settings): Capture string node.

2 years agoUtilize new font-lock faces for C/C++ and Python tree-sitter use
Randy Taylor [Mon, 14 Nov 2022 03:06:33 +0000 (22:06 -0500)]
Utilize new font-lock faces for C/C++ and Python tree-sitter use

* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): Use new
escape, number, property, operator, bracket, and delimiter font-lock
faces.
(c-ts-mode--base-mode): Add them to the feature list.
(c-ts-mode): Fix typo.
* lisp/progmodes/python.el (python--treesit-settings): Use new number,
property, operator, bracket, and delimiter font-lock faces.
(python-mode): Add them to the feature list.

2 years agoImprovements to the tree-sitter setup in sh-script.el
Yuan Fu [Thu, 17 Nov 2022 18:07:15 +0000 (10:07 -0800)]
Improvements to the tree-sitter setup in sh-script.el

* lisp/progmodes/sh-script.el (sh-mode--treesit-settings): Add defvar.
(sh-mode): Change feature names.
(sh-mode--treesit-fontify-decl-command): Remove function.
(sh-mode--treesit-declaration-commands): New variable.
(sh-mode--treesit-settings): Change feature names.  Simplify
declaration-command feature.

2 years ago; Minor stylistic fix for previous change on sh-script.el
Yuan Fu [Thu, 17 Nov 2022 17:40:37 +0000 (09:40 -0800)]
; Minor stylistic fix for previous change on sh-script.el

* lisp/progmodes/sh-script.el: Stylistic change.

2 years agoInitial fontification in sh-mode with tree-sitter
João P. L. de Carvalho [Thu, 27 Oct 2022 21:45:56 +0000 (15:45 -0600)]
Initial fontification in sh-mode with tree-sitter

* lisp/progmodes/sh-script.el (sh-mode): Add tree-sitter setup.  Move
setup of font-lock-defaults into branch.
(sh-mode--treesit-operators)
(sh-mode--treesit-keywords)
(sh-mode--treesit-settings): New variables.
(sh-mode--treesit-other-keywords)
(sh-mode--treesit-fontify-decl-command): New functions.

2 years ago; Elaborate on tree-sitter fontification features documentation
Yuan Fu [Thu, 17 Nov 2022 05:31:04 +0000 (21:31 -0800)]
; Elaborate on tree-sitter fontification features documentation

* doc/lispref/modes.texi (Parser-based Font Lock)
* lisp/treesit.el (treesit-font-lock-feature-list): Explain in more
detail, and change some features.

2 years agoFix documentation of recent treesit changes
Eli Zaretskii [Thu, 17 Nov 2022 13:03:40 +0000 (15:03 +0200)]
Fix documentation of recent treesit changes

* src/treesit.c (Ftreesit_parser_notifiers)
(Ftreesit_parser_add_notifier, Ftreesit_parser_remove_notifier):
Doc string fixes.

* doc/lispref/parsing.texi (Accessing Node Information): Fix
wording and punctuation.
(Using Parser): Improve indexing and wording.

2 years agoFix treesit-search-forward not matching leaf nodes
Yuan Fu [Wed, 16 Nov 2022 23:13:05 +0000 (15:13 -0800)]
Fix treesit-search-forward not matching leaf nodes

* src/treesit.c: Match against NODE at start of the loop.

2 years agoTweaks on c-ts-mode fontification rules
Yuan Fu [Wed, 16 Nov 2022 22:33:48 +0000 (14:33 -0800)]
Tweaks on c-ts-mode fontification rules

Changes:
1. Remove some :override flags
2. Fontify "!" in font-lock-negation-char-face
3. Use some new faces like font-lock-escape-face and
   font-lock-property-face
4. Move all type fontification into 'type' feature.  Previously some
   are in 'definition' feature.
5. Simplify definition feature, and fontify identifiers in definitions
   recursively.
6. Change 'statement' feature to 'label'
7. New feature 'assignment'
8. Remove some rules from 'expression' feature

* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): See
above.
(c-ts-mode--fontify-declarator): New function.
(c-ts-mode): Update feature-list.

2 years agoFix c-ts-mode-imenu
Yuan Fu [Wed, 16 Nov 2022 18:00:42 +0000 (10:00 -0800)]
Fix c-ts-mode-imenu

* lisp/progmodes/c-ts-mode.el (c-ts-mode--imenu-1): If cannot find
its name, ignore the node.
(c-ts-mode--imenu): Don't include a category if it's empty.

2 years agoAllow checking for outdated nodes in tree-sitter
Yuan Fu [Wed, 16 Nov 2022 05:54:30 +0000 (21:54 -0800)]
Allow checking for outdated nodes in tree-sitter

Now you can run (treesit-node-check node 'outdated).

* doc/lispref/parsing.texi (Accessing Node Information): Update
manual.
* src/treesit.c (Ftreesit_node_check): Add new property 'outdated'.
* test/src/treesit-tests.el (treesit-node-check): Add tests.

2 years ago; Add documentation for tree-sitter parser after-change notifiers
Yuan Fu [Wed, 16 Nov 2022 05:30:13 +0000 (21:30 -0800)]
; Add documentation for tree-sitter parser after-change notifiers

* doc/lispref/parsing.texi (Using Parser): Update manual.
* lisp/treesit.el (treesit): Add shordocs.
* src/treesit.c: Augment docstrings.

2 years agoAdd treesit-explore-mode
Yuan Fu [Wed, 16 Nov 2022 04:29:07 +0000 (20:29 -0800)]
Add treesit-explore-mode

This mode is basically the tree-sitter playground[1] in Emacs:
displays the syntax tree with the source side-by-side, kept in sync in
real-time.

[1] https://tree-sitter.github.io/tree-sitter/playground

* doc/lispref/parsing.texi (Language Definitions): Mention in manual.
* lisp/treesit.el (treesit--explorer-buffer)
(treesit--explorer-source-buffer)
(treesit--explorer-language)
(treesit--explorer-refresh-timer)
(treesit--explorer-highlight-overlay)
(treesit--explorer-last-node): New variables.
* lisp/treesit.el (treesit--explorer--nodes-to-highlight)
(treesit--explorer-refresh)
(treesit--explorer-post-command)
(treesit--explorer-jump)
(treesit--explorer-highlight-node)
(treesit--explorer-draw-node): New functions.
(treesit--explorer-tree-mode)
(treesit-explore-mode): New modes.

2 years agoAdd emacs-devel-specific font-lock rules to c-ts-mode
Yuan Fu [Tue, 15 Nov 2022 18:14:47 +0000 (10:14 -0800)]
Add emacs-devel-specific font-lock rules to c-ts-mode

This rule corrects the fontification for DEFUN macros.

* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings):
Additional font-lock rules.
(c-ts-mode--fontify-defun): New function.

2 years ago; * src/treesit.c: Minor stylistic changes.
Yuan Fu [Tue, 15 Nov 2022 17:24:08 +0000 (09:24 -0800)]
; * src/treesit.c: Minor stylistic changes.

2 years agoFix the MS-Windows build due to use of a new TS function
Eli Zaretskii [Tue, 15 Nov 2022 14:54:41 +0000 (16:54 +0200)]
Fix the MS-Windows build due to use of a new TS function

* src/treesit.c (ts_tree_get_changed_ranges) [WINDOWSNT]: Define,
load from the library, and call through a function pointer.

2 years ago; Enable highlighting error in c-ts-mode--base-mode
Yuan Fu [Tue, 15 Nov 2022 10:39:03 +0000 (02:39 -0800)]
; Enable highlighting error in c-ts-mode--base-mode

I expect this to be turned off by default. but turn this on for
demonstration for now.

* lisp/progmodes/c-ts-mode.el (c-ts-mode--base-mode): Enable error.

2 years agoMake tree-sitter fontification automatically update
Yuan Fu [Tue, 15 Nov 2022 10:30:50 +0000 (02:30 -0800)]
Make tree-sitter fontification automatically update

Now nodes that are affected and changed during re-parse will be
correctly refontified.  Nodes can change even if it corresponding text
wasn't edited: additional text can complete the parse tree and resolve
error nodes, for example.

* lisp/progmodes/python.el (python-mode): Create Python parser before
calling treesit-major-mode-setup.
* lisp/treesit.el (treesit--font-lock-notifier): New function.
(treesit-major-mode-setup): Register fontifier with every existing
parser.

2 years agoRemove the contextual hack in tree-sitter fontification
Yuan Fu [Tue, 15 Nov 2022 10:22:41 +0000 (02:22 -0800)]
Remove the contextual hack in tree-sitter fontification

We now have a better facility that can replace the contextual
hack. The C part is in the previous commit, and the Lisp part work
will be in the next commit.

* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--treesit-font-lock-settings)
* lisp/progmodes/python.el (python--treesit-settings)
* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): Stop marking
contextual nodes.
* lisp/treesit.el (treesit-font-lock-contextual-post-process): Remove
function.
(treesit-font-lock-fontify-region): Remove code processing contextual
nodes.

2 years agoAllow tree-sitter to notify parse-tree changes
Yuan Fu [Tue, 15 Nov 2022 10:16:11 +0000 (02:16 -0800)]
Allow tree-sitter to notify parse-tree changes

* src/treesit.c (treesit_call_after_change_functions): New function.
(treesit_ensure_parsed): Call treesit_call_after_change_functions
right after re-parse.
(make_treesit_parser): Initialize after_change_functions.
(Ftreesit_parser_notifiers)
(Ftreesit_parser_add_notifier)
(Ftreesit_parser_remove_notifier): New functions.
* src/treesit.h (Lisp_TS_Parser): New field after_change_functions.

2 years agoExtract out treesit_make_ranges
Yuan Fu [Tue, 15 Nov 2022 08:31:42 +0000 (00:31 -0800)]
Extract out treesit_make_ranges

* src/treesit.c (treesit_make_ranges): New function.
(Ftreesit_parser_included_ranges): Use treesit_make_ranges.

2 years agoDon't highlight syntax error in c-ts-mode
Yuan Fu [Mon, 14 Nov 2022 18:34:27 +0000 (10:34 -0800)]
Don't highlight syntax error in c-ts-mode

* lisp/progmodes/c-ts-mode.el (c-ts-mode--base-mode): Remove error
feature.

2 years ago; Remove some tree-sitter tests
Yuan Fu [Mon, 14 Nov 2022 09:23:45 +0000 (01:23 -0800)]
; Remove some tree-sitter tests

* test/src/treesit-tests.el (treesit-parser-supplemental): Remove
treesit-set-ranges, and treesit-get-ranges, as these functions are
removed.

2 years agoFix some inconsistencies in *-ts-modes
Theodor Thornhill [Mon, 14 Nov 2022 07:03:11 +0000 (08:03 +0100)]
Fix some inconsistencies in *-ts-modes

(c-ts-mode--base-mode): Extract comment-* so that we can separate
between C and C++.
(c-ts-mode, c++-ts-mode): Set comment-* variables.

* lisp/progmodes/css-ts-mode.el (css-ts-mode): Add electric-indent.

* lisp/progmodes/java-ts-mode.el
(java-ts-mode--imenu): Add categories.  Only display categories that
exist in the file.
(java-ts-mode): Add electric-indent.

* lisp/progmodes/json-ts-mode.el
(json-ts-mode): Add electric-indent.

* lisp/progmodes/ts-mode.el
(ts-mode--font-lock-settings): Whitespace cleanup.
(ts-mode): Add electric-indent.

2 years ago; * test/src/treesit-tests.el (treesit-node-check): New test.
Yuan Fu [Mon, 14 Nov 2022 08:54:54 +0000 (00:54 -0800)]
; * test/src/treesit-tests.el (treesit-node-check): New test.

2 years agoRemove feature that checks whether tree-sitter node "has changes"
Yuan Fu [Mon, 14 Nov 2022 08:51:54 +0000 (00:51 -0800)]
Remove feature that checks whether tree-sitter node "has changes"

First of all, we signal error on using an outdated node, so if a node
has changes, calling treesit-node-check would only raise an error.
Besides, in order to properly answer whether a node has changed, we
would have to update the node as the buffer is edited, which we don't
do right now.

* doc/lispref/parsing.texi (Accessing Node Information): Remove
relevant manual text.
* src/treesit.c (Ftreesit_node_check): Remove docstring mentions,
remove the branch for "has-changes".
(syms_of_treesit): Remove has-changes.

2 years agoFix crash on MS-Windows due to memory-allocation problem in treesit.c
Eli Zaretskii [Sun, 13 Nov 2022 08:51:42 +0000 (10:51 +0200)]
Fix crash on MS-Windows due to memory-allocation problem in treesit.c

* src/treesit.c (treesit_load_language): Use 'xstrdup'/'xfree'
instead of 'strdup'/'free', to prevent crashes on MS-Windows,
where we must use our own implementation of 'malloc'/'free',
whereas 'strdup' uses the default implementation in the MS-Windows
C runtime library.

2 years agoImprove tree-sitter imenu for c-mode and js-mode
Yuan Fu [Sat, 12 Nov 2022 23:57:02 +0000 (15:57 -0800)]
Improve tree-sitter imenu for c-mode and js-mode

Instead of a flat list, now categorize imenu entries into categories
like "Function", "Variable", "Class", etc.

* lisp/progmodes/c-ts-mode.el (c-ts-mode--imenu-1): Handle more edge
cases.
(c-ts-mode--imenu): Categorize menu entries.
* lisp/progmodes/js.el (js--treesit-imenu-type-alist): Remove variable.
(js--treesit-imenu-label): Remove function.
(js--treesit-imenu-1): Use the name alone for labels.
(js--treesit-imenu): Categorize menu entries.

2 years ago; Minor typo fixes for tree-sitter modes
Yuan Fu [Sat, 12 Nov 2022 22:33:14 +0000 (14:33 -0800)]
; Minor typo fixes for tree-sitter modes

* etc/NEWS: Fix typos.  Change Tree-sitter to tree-sitter.
* lisp/progmodes/c-ts-mode.el
* lisp/progmodes/css-ts-mode.el
* lisp/progmodes/java-ts-mode.el
* lisp/progmodes/json-ts-mode.el: Change Tree Sitter to tree-sitter.
Fix typo.

2 years agoAdd tree-sitter modes for C-like languages
Theodor Thornhill [Thu, 10 Nov 2022 16:15:49 +0000 (17:15 +0100)]
Add tree-sitter modes for C-like languages

* etc/NEWS: Mention the new modes
* lisp/progmodes/c-ts-mode.el: New major mode with tree-sitter support
for C and C++.
* lisp/progmodes/java-ts-mode.el: New major mode with tree-sitter support.
* lisp/progmodes/json-ts-mode.el: New major mode with tree-sitter support.
* lisp/progmodes/css-ts-mode.el: New major mode with tree-sitter support.

2 years agoFix recent changes in documentation
Eli Zaretskii [Sat, 12 Nov 2022 08:57:56 +0000 (10:57 +0200)]
Fix recent changes in documentation

* lisp/treesit.el (treesit-node-at): Doc fix.

* doc/lispref/parsing.texi (Language Definitions): Add
cross-reference.
(Retrieving Nodes): Renamed from "Retrieving Node"; all references
changed.
(Retrieving Nodes): Fix wording and markup.

2 years ago; * lisp/treesit.el (treesit--indent-1): Fix treesit-node-at usage.
Yuan Fu [Sat, 12 Nov 2022 05:37:29 +0000 (21:37 -0800)]
; * lisp/treesit.el (treesit--indent-1): Fix treesit-node-at usage.

2 years agoImprove treesit-node-at
Yuan Fu [Sat, 12 Nov 2022 04:04:38 +0000 (20:04 -0800)]
Improve treesit-node-at

* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Change semantic.  It tries to
return the node that a user would expect in various circumstances.
* test/src/treesit-tests.el (treesit-node-at): New test.

2 years agoIn end-of-defun, terminate early if no further defun exists
Yuan Fu [Thu, 10 Nov 2022 23:00:29 +0000 (15:00 -0800)]
In end-of-defun, terminate early if no further defun exists

Before this change, end-of-defun calls end-of-defun-function even if
point is not necessarily at the beginning of a defun (contrary to what
end-of-defun-function's docstring claims).  Now it terminates early
and doesn't call end-of-defun-function.

* lisp/emacs-lisp/lisp.el (beginning-of-defun-raw): Update docstring
clarifying the return value.
(end-of-defun): Terminate early if beginning-of-defun-raw returns nil.

2 years ago; Fix docstring graph alignment in Ftreesit_search_forward
Yuan Fu [Thu, 10 Nov 2022 22:16:25 +0000 (14:16 -0800)]
; Fix docstring graph alignment in Ftreesit_search_forward

* src/treesit.c (Ftreesit_search_forward): Replace tabs with spaces.

2 years agoUtilize new face font-lock-escape-face in python-mode
Yuan Fu [Thu, 10 Nov 2022 18:34:57 +0000 (10:34 -0800)]
Utilize new face font-lock-escape-face in python-mode

* lisp/progmodes/python.el (python--treesit-settings): Use
font-lock-escape-face rather than font-lock-constant-face.

2 years agoAccept older versions of tree-sitter library
Yuan Fu [Thu, 10 Nov 2022 18:00:28 +0000 (10:00 -0800)]
Accept older versions of tree-sitter library

* configure.ac: Accept a tree-sitter library as long as it provides
ts_set_allocator.

2 years ago; * lisp/font-lock.el (font-lock-bracket-face): Improve docstring
Robert Pluim [Thu, 10 Nov 2022 11:46:03 +0000 (12:46 +0100)]
; * lisp/font-lock.el (font-lock-bracket-face): Improve docstring

2 years ago; * lisp/treesit.el (treesit-font-lock-rules): Improve docstring
Robert Pluim [Thu, 10 Nov 2022 11:45:14 +0000 (12:45 +0100)]
; * lisp/treesit.el (treesit-font-lock-rules): Improve docstring

2 years ago; * etc/NEWS: Fix last change.
Eli Zaretskii [Thu, 10 Nov 2022 11:04:42 +0000 (13:04 +0200)]
; * etc/NEWS: Fix last change.

2 years agoAdd more font-lock faces (Bug#58940)
Randy Taylor [Sat, 29 Oct 2022 02:39:50 +0000 (22:39 -0400)]
Add more font-lock faces (Bug#58940)

* lisp/font-lock.el (font-lock-bracket-face, font-lock-delimiter-face,
font-lock-escape-face, font-lock-number-face,
font-lock-misc-punctuation-face, font-lock-operator-face,
font-lock-property-face, font-lock-punctuation-face): Define new
faces.
* lisp/cus-theme.el (custom-theme--listed-faces): Add them to the
list.
* doc/lispref/modes.texi (Faces for Font Lock): Document them.
* etc/NEWS: Mention them.

2 years ago; Revert back to requiring tree-sitter v0.0 for now
Yuan Fu [Thu, 10 Nov 2022 08:31:10 +0000 (00:31 -0800)]
; Revert back to requiring tree-sitter v0.0 for now

2 years ago; Update tree-sitter HTML manuals in admin/notes
Yuan Fu [Wed, 9 Nov 2022 22:50:39 +0000 (14:50 -0800)]
; Update tree-sitter HTML manuals in admin/notes

* admin/notes/tree-sitter/html-manual/Language-Definitions.html
* admin/notes/tree-sitter/html-manual/Multiple-Languages.html
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Font-Lock.html
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Indentation.html
* admin/notes/tree-sitter/html-manual/Retrieving-Node.html: Update.

2 years ago* configure.ac: Change required tree-sitter version to 0.20.2.
Yuan Fu [Wed, 9 Nov 2022 22:48:07 +0000 (14:48 -0800)]
* configure.ac: Change required tree-sitter version to 0.20.2.

2 years agoMake tree-sitter node type match case-sensitive
Yuan Fu [Wed, 9 Nov 2022 20:29:50 +0000 (12:29 -0800)]
Make tree-sitter node type match case-sensitive

* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* src/treesit.c (treesit_traverse_match_predicate): Change to
fast_c_string_match.
(Ftreesit_search_subtree)
(Ftreesit_search_forward)
(Ftreesit_induce_sparse_tree): Update docstring.

2 years agoNew function fast_c_string_match and fast_c_string_match_internal
Yuan Fu [Wed, 9 Nov 2022 20:24:43 +0000 (12:24 -0800)]
New function fast_c_string_match and fast_c_string_match_internal

Generalize fast_c_string_match_ignore_case into
fast_c_string_match_internal.  And Make fast_c_string and
fast_c_string_match_ignore_case use fast_c_string_match_internal.

* src/lisp.h (fast_c_string_match_internal)
(fast_c_string_match): New declaration.
(fast_c_string_match_ignore_case): Change to thin wrapper.
* src/search.c (fast_c_string_match_internal): New function.

2 years agoAdd tree-sitter shortdoc
Yuan Fu [Wed, 9 Nov 2022 04:44:54 +0000 (20:44 -0800)]
Add tree-sitter shortdoc

* lisp/treesit.el (treesit--generate-shortdoc-examples): New function.
(treesit): New shortdoc group.

2 years ago; Fix treesit-query-range docstring
Yuan Fu [Wed, 9 Nov 2022 04:43:24 +0000 (20:43 -0800)]
; Fix treesit-query-range docstring

* lisp/treesit.el (treesit-query-range): Change signature, change
reference of treesit-query-in to that of treesit-query-capture.

2 years agoMimic existing python-mode beg/end-of-defun behavior better
Yuan Fu [Wed, 9 Nov 2022 04:41:58 +0000 (20:41 -0800)]
Mimic existing python-mode beg/end-of-defun behavior better

* lisp/progmodes/python.el (python-treesit-beginning-of-defun)
(python-treesit-end-of-defun): New functions.
* lisp/progmodes/python.el (python-mode): Use custom beg/end-of-defun
functions.

2 years ago; Minor improvements in wording of tree-sitter docs
Eli Zaretskii [Wed, 9 Nov 2022 13:35:51 +0000 (15:35 +0200)]
; Minor improvements in wording of tree-sitter docs

* doc/lispref/parsing.texi (Multiple Languages):
* doc/lispref/modes.texi (Parser-based Font Lock): Improve
wording.

2 years ago; Improve tree-sitter documentation
Yuan Fu [Tue, 8 Nov 2022 18:39:53 +0000 (10:39 -0800)]
; Improve tree-sitter documentation

* doc/lispref/modes.texi (Parser-based Font Lock)
* doc/lispref/parsing.texi (Multiple Languages)
* lisp/treesit.el (treesit-range-rules)
(treesit-font-lock-rules): Improve phrasing around queries and keyword
and values.

2 years ago; Improve docstring of treesit-simple-indent-presets
Yuan Fu [Mon, 7 Nov 2022 10:27:59 +0000 (02:27 -0800)]
; Improve docstring of treesit-simple-indent-presets

* lisp/treesit.el (treesit-simple-indent-presets): Improve docstring.

2 years agoAdd tree-sitter indent preset "point-min"
Yuan Fu [Mon, 7 Nov 2022 10:00:04 +0000 (02:00 -0800)]
Add tree-sitter indent preset "point-min"

* doc/lispref/modes.texi (Parser-based Indentation): Update manual.
* lisp/treesit.el (treesit-simple-indent-presets): Add preset.  Update
docstring.

2 years agoAdd treesit--indent-rules-optimize
Yuan Fu [Mon, 7 Nov 2022 09:59:21 +0000 (01:59 -0800)]
Add treesit--indent-rules-optimize

* lisp/treesit.el (treesit--indent-rules-optimize): New function.
(treesit-major-mode-setup): Optimize indent rules when setting up for
major mode.

2 years ago; Documentation fixes for tree-sitter
Yuan Fu [Mon, 7 Nov 2022 09:27:49 +0000 (01:27 -0800)]
; Documentation fixes for tree-sitter

* doc/lispref/parsing.texi (Multiple Languages): Resolve the FIXME.

2 years ago; More documentation improvements for treesit
Eli Zaretskii [Sun, 6 Nov 2022 09:59:27 +0000 (11:59 +0200)]
; More documentation improvements for treesit

* lisp/treesit.el (treesit-language-at, treesit-range-rules)
(treesit-font-lock-rules): Doc fixes.

* doc/lispref/parsing.texi (Multiple Languages):
* doc/lispref/modes.texi (Parser-based Font Lock): Fix wording.

2 years agoFix incorrect tree-sitter fontification
Yuan Fu [Sun, 6 Nov 2022 01:58:39 +0000 (18:58 -0700)]
Fix incorrect tree-sitter fontification

* lisp/treesit.el (treesit-font-lock-fontify-region): If the captured
node is outside of the region between START and END, don't fontify it.
Wrap fontification code in a when form.

2 years agoAllow tree-sitter indent offset to be a variable
Yuan Fu [Sun, 6 Nov 2022 01:32:01 +0000 (18:32 -0700)]
Allow tree-sitter indent offset to be a variable

* doc/lispref/modes.texi (Parser-based Indentation): Update manual.
* lisp/progmodes/js.el (js--treesit-indent-rules): Change all
occurance of ,js-indent-level to js-indent-level.

* lisp/progmodes/ts-mode.el (ts-mode--indent-rules): Change all
occurance of ,ts-mode-indent-offset to ts-mode-indent-offset.
* lisp/treesit.el (treesit-simple-indent-rules): Change docstring.
(treesit-simple-indent): Allow offset to be a variable.

2 years agoDon't indent empty lines in treesit-indent-region
Yuan Fu [Sun, 6 Nov 2022 00:09:51 +0000 (17:09 -0700)]
Don't indent empty lines in treesit-indent-region

* lisp/treesit.el (treesit-indent-region): Wrap the old code in an if
form that checks whether we are at an empty line.  Add comments.

2 years agoMake treesit-language-at work in more situations
Yuan Fu [Sat, 5 Nov 2022 23:55:17 +0000 (16:55 -0700)]
Make treesit-language-at work in more situations

; * lisp/treesit.el (treesit-language-at): Return sometime sensible
even if treesit-language-at-point-function is nil.

2 years ago; Fix documentation for tree-sitter
Yuan Fu [Sat, 5 Nov 2022 23:52:38 +0000 (16:52 -0700)]
; Fix documentation for tree-sitter

* doc/lispref/modes.texi (Parser-based Font Lock): Fix the signature
style.
* doc/lispref/parsing.texi (Language Definitions): Rephrase the
section, remove documentation for deleted functions, fix signature
style.
* lisp/treesit.el (treesit-range-rules)
(treesit-font-lock-rules): Change ARGS to QUERY-SPECS, fix docstring.
* lisp/treesit.el (treesit-inspect-mode): Fix docstring.

2 years ago; Minor fixes to documentation
Eli Zaretskii [Sat, 5 Nov 2022 08:49:46 +0000 (10:49 +0200)]
; Minor fixes to documentation

* lisp/treesit.el (treesit-ready-p, treesit-inspect-mode)
(treesit-range-settings, treesit-range-rules)
(treesit--merge-ranges, treesit-update-ranges)
(treesit-font-lock-rules, treesit-font-lock-feature-list)
(treesit-font-lock-contextual-post-process)
(treesit-font-lock-recompute-features): Doc fixes.
* lisp/progmodes/js.el (js--fontify-template-string): Doc fix.

* doc/lispref/modes.texi (Parser-based Indentation):
* doc/lispref/positions.texi (List Motion):
* doc/lispref/parsing.texi (Tree-sitter major modes):
* doc/lispref/parsing.texi (Multiple Languages): Fix wording and
markup.

2 years ago; Fix minor issues with Tree-sitter documentation in ELisp manual
Eli Zaretskii [Sat, 5 Nov 2022 07:14:25 +0000 (09:14 +0200)]
; Fix minor issues with Tree-sitter documentation in ELisp manual

* doc/lispref/parsing.texi (Multiple Languages): Add @end example.

* doc/lispref/Makefile.in (srcs): Add $(srcdir)/parsing.texi.

2 years ago; * lisp/treesit.el (treesit-inspect-node-at-point): Minor fix.
Yuan Fu [Fri, 4 Nov 2022 17:50:55 +0000 (10:50 -0700)]
; * lisp/treesit.el (treesit-inspect-node-at-point): Minor fix.

2 years ago; * lisp/treesit.el (treesit-ready-p): Fix typo.
Yuan Fu [Fri, 4 Nov 2022 17:16:33 +0000 (10:16 -0700)]
; * lisp/treesit.el (treesit-ready-p): Fix typo.

2 years ago; Minor improvement in treesit-inspect-mode
Yuan Fu [Fri, 4 Nov 2022 17:15:42 +0000 (10:15 -0700)]
; Minor improvement in treesit-inspect-mode

* lisp/treesit.el (treesit-inspect-node-at-point): Show the current
node in bold.
(treesit-inspect-mode): Update docstring.

2 years ago; * lisp/treesit.el (treesit-update-ranges): Update all parsers.
Yuan Fu [Fri, 4 Nov 2022 17:03:17 +0000 (10:03 -0700)]
; * lisp/treesit.el (treesit-update-ranges): Update all parsers.

2 years ago; Replace error with signal in treesit.el
Yuan Fu [Fri, 4 Nov 2022 16:58:08 +0000 (09:58 -0700)]
; Replace error with signal in treesit.el

(or (treesit-parser-create language)
    (error "Cannot find a parser for %s" language))

is turned into

(treesit-parser-create language)

because treesit-parser-create never returns nil.  (We used to use
treesit-get-parser which don't create parser automatically.)

* lisp/treesit.el (treesit-buffer-root-node): Replace error with signal.

2 years ago; Remove unused tree-sitter functions
Yuan Fu [Fri, 4 Nov 2022 16:57:32 +0000 (09:57 -0700)]
; Remove unused tree-sitter functions

They didn't turn out to be very useful.

* lisp/treesit.el (treesit-set-ranges)
(treesit-get-ranges): Remove functions.

2 years agoAdd treesit-indent-error
Yuan Fu [Fri, 4 Nov 2022 16:56:38 +0000 (09:56 -0700)]
Add treesit-indent-error

* lisp/treesit.el (treesit-indent-error): New error.
(treesit--simple-indent-eval): Signal treesit-indent-error.

2 years agoRevise tree-sitter facility for multi-language buffers
Yuan Fu [Fri, 4 Nov 2022 08:31:56 +0000 (01:31 -0700)]
Revise tree-sitter facility for multi-language buffers

* doc/lispref/parsing.texi (Multiple Languages): Extend and update
manual.
* lisp/treesit.el (treesit-range-functions): Remove variable.
(treesit-range-settings): New variable.
(treesit-range-rules): New function.
(treesit--merge-ranges): New function.
(treesit-update-ranges): Use treesit-range-settings instead of
treesit-range-functions.
(treesit-font-lock-rules): Fix docstring.
(treesit-indent)
(treesit-indent-region): Only update ranges in a region.
* test/src/treesit-tests.el (treesit-range): New test.

2 years ago; * lisp/progmodes/js.el (js--fontify-template-string): Fix typo.
Yuan Fu [Fri, 4 Nov 2022 08:18:59 +0000 (01:18 -0700)]
; * lisp/progmodes/js.el (js--fontify-template-string): Fix typo.

2 years agoFix treesit-parser-set-included-ranges
Yuan Fu [Fri, 4 Nov 2022 05:14:50 +0000 (22:14 -0700)]
Fix treesit-parser-set-included-ranges

Track whether a parser has ranges set ourselves.  See comment in
Ftreesit_parser_included_ranges for detail.

* src/treesit.c (make_treesit_parser): Initialize has_range.
(Ftreesit_parser_set_included_ranges): Set has_range.  Fill docstring.
(Ftreesit_parser_included_ranges): Check has_range.
* src/treesit.h (Lisp_TS_Parser): Add has_range.

2 years ago; * src/treesit.c: Minor comment change.
Yuan Fu [Fri, 4 Nov 2022 02:39:43 +0000 (19:39 -0700)]
; * src/treesit.c: Minor comment change.

2 years ago; Tree-sitter: Don't refontify if entity contained in the region
Yuan Fu [Fri, 4 Nov 2022 02:37:55 +0000 (19:37 -0700)]
; Tree-sitter: Don't refontify if entity contained in the region

* lisp/treesit.el (treesit-font-lock-contextual-post-process): Only
refontify when NODE-START < region START.

2 years agoAdd contextual tree-sitter font-lock to ts-mode and js-mode
Yuan Fu [Thu, 3 Nov 2022 18:53:49 +0000 (11:53 -0700)]
Add contextual tree-sitter font-lock to ts-mode and js-mode

* lisp/progmodes/js.el (js--treesit-font-lock-settings)
* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): Capture
commend and strings.  Add empty lines.

2 years agoMinor change to tree-sitter imenu function of js-mode
Yuan Fu [Thu, 3 Nov 2022 18:44:35 +0000 (11:44 -0700)]
Minor change to tree-sitter imenu function of js-mode

If someone don’t like the labels, they can set them to empty strings.

* lisp/progmodes/js.el (js--treesit-imenu-type-alist): Add space.
(js--treesit-imenu-label): Remove space.

2 years ago; Update guides in /admin/notes/tree-sitter
Yuan Fu [Thu, 3 Nov 2022 18:41:42 +0000 (11:41 -0700)]
; Update guides in /admin/notes/tree-sitter

* admin/notes/tree-sitter/html-manual/Language-Definitions.html
* admin/notes/tree-sitter/html-manual/Multiple-Languages.html
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Font-Lock.html
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Indentation.html
* admin/notes/tree-sitter/html-manual/Parsing-Program-Source.html
* admin/notes/tree-sitter/html-manual/Pattern-Matching.html
* admin/notes/tree-sitter/html-manual/Retrieving-Node.html
* admin/notes/tree-sitter/html-manual/Tree_002dsitter-C-API.html
* admin/notes/tree-sitter/html-manual/Using-Parser.html
* admin/notes/tree-sitter/starter-guide: Update to reflect changes
made recently.

2 years agoMore style fixes to tree-sitter code
Po Lu [Thu, 3 Nov 2022 11:14:16 +0000 (19:14 +0800)]
More style fixes to tree-sitter code

* src/treesit.c (treesit_node_uptodate_p):
* src/treesit.h: Fix coding style.

2 years agoMinor fixes to last tree-sitter change
Po Lu [Thu, 3 Nov 2022 11:13:18 +0000 (19:13 +0800)]
Minor fixes to last tree-sitter change

* src/treesit.c (treesit_find_override_name): Initialize
variable.
(Ftreesit_query_expand): Revert change to behavior.

2 years agoStylistic changes to tree-sitter code
Po Lu [Thu, 27 Oct 2022 13:30:23 +0000 (21:30 +0800)]
Stylistic changes to tree-sitter code

* src/treesit.c (treesit_find_override_name)
(treesit_load_language_push_for_each_suffix, treesit_load_language)
(Ftreesit_langauge_available_p, treesit_record_change)
(make_treesit_parser, make_treesit_node, make_treesit_query)
(Ftreesit_parser_create, Ftreesit_parser_delete, Ftreesit_parser_list)
(treesit_check_range_argument, Ftreesit_parser_included_ranges)
(Ftreesit_node_start, Ftreesit_node_end, Ftreesit_node_check)
(Ftreesit_node_child_by_field_name, Ftreesit_pattern_expand)
(Ftreesit_query_expand, treesit_predicates_for_pattern)
(treesit_predicate_capture_name_to_text, treesit_predicate_equal)
(treesit_predicate_match, treesit_eval_predicates)
(Ftreesit_query_capture, treesit_search_dfs, treesit_build_sparse_tree)
(syms_of_treesit): Use FOR_EACH_TAIL (or FOR_EACH_TAIL_SAFE where not
obviously safe), and check list heads and tails correctly; fix coding
style of various constructs, especially:

  variable =
    mumble (frob (bar), (foo () + bar ()) + (baz () + quux ()))

which should actually be

  variable
    = mumble (frob (bar), (foo () + bar ()) + (baz () + quux ()))

and

  foo =
    mumble (frob (bar), 0)
      + (foo () + bar ())
      + (baz () + quux ())

which should actually be

  foo = (mumble (frob (bar), 0)
 + (foo () + bar ())
 + (baz () + quux ()))

* src/treesit.h: Make declaration coding style consistent.

2 years agoCheck for outdated tree-sitter node when printing
Yuan Fu [Thu, 3 Nov 2022 00:47:14 +0000 (17:47 -0700)]
Check for outdated tree-sitter node when printing

* src/print.c (print_vectorlike): Check for outdated node.
* src/treesit.c (treesit_check_node): Use the new function.
(treesit_node_uptodate_p): New function.
* src/treesit.h: Declare new function.

2 years ago; * src/print.c (print_vectorlike): Fix comment for tree-sitter.
Yuan Fu [Thu, 3 Nov 2022 00:23:29 +0000 (17:23 -0700)]
; * src/print.c (print_vectorlike): Fix comment for tree-sitter.

2 years agoPass region start and end to tree-sitter fontification functions
Yuan Fu [Thu, 3 Nov 2022 00:03:03 +0000 (17:03 -0700)]
Pass region start and end to tree-sitter fontification functions

* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Update
function to only fontify within the region.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Pass START and END to
fontification functions.

2 years agoAdd handling of contextual entities in tree-sitter font-lock
Yuan Fu [Wed, 2 Nov 2022 23:31:25 +0000 (16:31 -0700)]
Add handling of contextual entities in tree-sitter font-lock

* lisp/progmodes/python.el: Remove function.
(python--treesit-settings): Capture contextual node.
* lisp/treesit.el (treesit--set-nonsticky):
(treesit-font-lock-contextual-post-process): New functions.
(treesit-font-lock-fontify-region): Change local variable START and
END to NODE-START and NODE-END, handle special capture name
"contextual".
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.