]> git.eshelyaron.com Git - emacs.git/log
emacs.git
2 years agoRemove feature that checks whether tree-sitter node "has changes"
Yuan Fu [Mon, 14 Nov 2022 08:51:54 +0000 (00:51 -0800)]
Remove feature that checks whether tree-sitter node "has changes"

First of all, we signal error on using an outdated node, so if a node
has changes, calling treesit-node-check would only raise an error.
Besides, in order to properly answer whether a node has changed, we
would have to update the node as the buffer is edited, which we don't
do right now.

* doc/lispref/parsing.texi (Accessing Node Information): Remove
relevant manual text.
* src/treesit.c (Ftreesit_node_check): Remove docstring mentions,
remove the branch for "has-changes".
(syms_of_treesit): Remove has-changes.

2 years agoFix crash on MS-Windows due to memory-allocation problem in treesit.c
Eli Zaretskii [Sun, 13 Nov 2022 08:51:42 +0000 (10:51 +0200)]
Fix crash on MS-Windows due to memory-allocation problem in treesit.c

* src/treesit.c (treesit_load_language): Use 'xstrdup'/'xfree'
instead of 'strdup'/'free', to prevent crashes on MS-Windows,
where we must use our own implementation of 'malloc'/'free',
whereas 'strdup' uses the default implementation in the MS-Windows
C runtime library.

2 years agoImprove tree-sitter imenu for c-mode and js-mode
Yuan Fu [Sat, 12 Nov 2022 23:57:02 +0000 (15:57 -0800)]
Improve tree-sitter imenu for c-mode and js-mode

Instead of a flat list, now categorize imenu entries into categories
like "Function", "Variable", "Class", etc.

* lisp/progmodes/c-ts-mode.el (c-ts-mode--imenu-1): Handle more edge
cases.
(c-ts-mode--imenu): Categorize menu entries.
* lisp/progmodes/js.el (js--treesit-imenu-type-alist): Remove variable.
(js--treesit-imenu-label): Remove function.
(js--treesit-imenu-1): Use the name alone for labels.
(js--treesit-imenu): Categorize menu entries.

2 years ago; Minor typo fixes for tree-sitter modes
Yuan Fu [Sat, 12 Nov 2022 22:33:14 +0000 (14:33 -0800)]
; Minor typo fixes for tree-sitter modes

* etc/NEWS: Fix typos.  Change Tree-sitter to tree-sitter.
* lisp/progmodes/c-ts-mode.el
* lisp/progmodes/css-ts-mode.el
* lisp/progmodes/java-ts-mode.el
* lisp/progmodes/json-ts-mode.el: Change Tree Sitter to tree-sitter.
Fix typo.

2 years agoAdd tree-sitter modes for C-like languages
Theodor Thornhill [Thu, 10 Nov 2022 16:15:49 +0000 (17:15 +0100)]
Add tree-sitter modes for C-like languages

* etc/NEWS: Mention the new modes
* lisp/progmodes/c-ts-mode.el: New major mode with tree-sitter support
for C and C++.
* lisp/progmodes/java-ts-mode.el: New major mode with tree-sitter support.
* lisp/progmodes/json-ts-mode.el: New major mode with tree-sitter support.
* lisp/progmodes/css-ts-mode.el: New major mode with tree-sitter support.

2 years agoFix recent changes in documentation
Eli Zaretskii [Sat, 12 Nov 2022 08:57:56 +0000 (10:57 +0200)]
Fix recent changes in documentation

* lisp/treesit.el (treesit-node-at): Doc fix.

* doc/lispref/parsing.texi (Language Definitions): Add
cross-reference.
(Retrieving Nodes): Renamed from "Retrieving Node"; all references
changed.
(Retrieving Nodes): Fix wording and markup.

2 years ago; * lisp/treesit.el (treesit--indent-1): Fix treesit-node-at usage.
Yuan Fu [Sat, 12 Nov 2022 05:37:29 +0000 (21:37 -0800)]
; * lisp/treesit.el (treesit--indent-1): Fix treesit-node-at usage.

2 years agoImprove treesit-node-at
Yuan Fu [Sat, 12 Nov 2022 04:04:38 +0000 (20:04 -0800)]
Improve treesit-node-at

* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Change semantic.  It tries to
return the node that a user would expect in various circumstances.
* test/src/treesit-tests.el (treesit-node-at): New test.

2 years agoIn end-of-defun, terminate early if no further defun exists
Yuan Fu [Thu, 10 Nov 2022 23:00:29 +0000 (15:00 -0800)]
In end-of-defun, terminate early if no further defun exists

Before this change, end-of-defun calls end-of-defun-function even if
point is not necessarily at the beginning of a defun (contrary to what
end-of-defun-function's docstring claims).  Now it terminates early
and doesn't call end-of-defun-function.

* lisp/emacs-lisp/lisp.el (beginning-of-defun-raw): Update docstring
clarifying the return value.
(end-of-defun): Terminate early if beginning-of-defun-raw returns nil.

2 years ago; Fix docstring graph alignment in Ftreesit_search_forward
Yuan Fu [Thu, 10 Nov 2022 22:16:25 +0000 (14:16 -0800)]
; Fix docstring graph alignment in Ftreesit_search_forward

* src/treesit.c (Ftreesit_search_forward): Replace tabs with spaces.

2 years agoUtilize new face font-lock-escape-face in python-mode
Yuan Fu [Thu, 10 Nov 2022 18:34:57 +0000 (10:34 -0800)]
Utilize new face font-lock-escape-face in python-mode

* lisp/progmodes/python.el (python--treesit-settings): Use
font-lock-escape-face rather than font-lock-constant-face.

2 years agoAccept older versions of tree-sitter library
Yuan Fu [Thu, 10 Nov 2022 18:00:28 +0000 (10:00 -0800)]
Accept older versions of tree-sitter library

* configure.ac: Accept a tree-sitter library as long as it provides
ts_set_allocator.

2 years ago; * lisp/font-lock.el (font-lock-bracket-face): Improve docstring
Robert Pluim [Thu, 10 Nov 2022 11:46:03 +0000 (12:46 +0100)]
; * lisp/font-lock.el (font-lock-bracket-face): Improve docstring

2 years ago; * lisp/treesit.el (treesit-font-lock-rules): Improve docstring
Robert Pluim [Thu, 10 Nov 2022 11:45:14 +0000 (12:45 +0100)]
; * lisp/treesit.el (treesit-font-lock-rules): Improve docstring

2 years ago; * etc/NEWS: Fix last change.
Eli Zaretskii [Thu, 10 Nov 2022 11:04:42 +0000 (13:04 +0200)]
; * etc/NEWS: Fix last change.

2 years agoAdd more font-lock faces (Bug#58940)
Randy Taylor [Sat, 29 Oct 2022 02:39:50 +0000 (22:39 -0400)]
Add more font-lock faces (Bug#58940)

* lisp/font-lock.el (font-lock-bracket-face, font-lock-delimiter-face,
font-lock-escape-face, font-lock-number-face,
font-lock-misc-punctuation-face, font-lock-operator-face,
font-lock-property-face, font-lock-punctuation-face): Define new
faces.
* lisp/cus-theme.el (custom-theme--listed-faces): Add them to the
list.
* doc/lispref/modes.texi (Faces for Font Lock): Document them.
* etc/NEWS: Mention them.

2 years ago; Revert back to requiring tree-sitter v0.0 for now
Yuan Fu [Thu, 10 Nov 2022 08:31:10 +0000 (00:31 -0800)]
; Revert back to requiring tree-sitter v0.0 for now

2 years ago; Update tree-sitter HTML manuals in admin/notes
Yuan Fu [Wed, 9 Nov 2022 22:50:39 +0000 (14:50 -0800)]
; Update tree-sitter HTML manuals in admin/notes

* admin/notes/tree-sitter/html-manual/Language-Definitions.html
* admin/notes/tree-sitter/html-manual/Multiple-Languages.html
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Font-Lock.html
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Indentation.html
* admin/notes/tree-sitter/html-manual/Retrieving-Node.html: Update.

2 years ago* configure.ac: Change required tree-sitter version to 0.20.2.
Yuan Fu [Wed, 9 Nov 2022 22:48:07 +0000 (14:48 -0800)]
* configure.ac: Change required tree-sitter version to 0.20.2.

2 years agoMake tree-sitter node type match case-sensitive
Yuan Fu [Wed, 9 Nov 2022 20:29:50 +0000 (12:29 -0800)]
Make tree-sitter node type match case-sensitive

* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* src/treesit.c (treesit_traverse_match_predicate): Change to
fast_c_string_match.
(Ftreesit_search_subtree)
(Ftreesit_search_forward)
(Ftreesit_induce_sparse_tree): Update docstring.

2 years agoNew function fast_c_string_match and fast_c_string_match_internal
Yuan Fu [Wed, 9 Nov 2022 20:24:43 +0000 (12:24 -0800)]
New function fast_c_string_match and fast_c_string_match_internal

Generalize fast_c_string_match_ignore_case into
fast_c_string_match_internal.  And Make fast_c_string and
fast_c_string_match_ignore_case use fast_c_string_match_internal.

* src/lisp.h (fast_c_string_match_internal)
(fast_c_string_match): New declaration.
(fast_c_string_match_ignore_case): Change to thin wrapper.
* src/search.c (fast_c_string_match_internal): New function.

2 years agoAdd tree-sitter shortdoc
Yuan Fu [Wed, 9 Nov 2022 04:44:54 +0000 (20:44 -0800)]
Add tree-sitter shortdoc

* lisp/treesit.el (treesit--generate-shortdoc-examples): New function.
(treesit): New shortdoc group.

2 years ago; Fix treesit-query-range docstring
Yuan Fu [Wed, 9 Nov 2022 04:43:24 +0000 (20:43 -0800)]
; Fix treesit-query-range docstring

* lisp/treesit.el (treesit-query-range): Change signature, change
reference of treesit-query-in to that of treesit-query-capture.

2 years agoMimic existing python-mode beg/end-of-defun behavior better
Yuan Fu [Wed, 9 Nov 2022 04:41:58 +0000 (20:41 -0800)]
Mimic existing python-mode beg/end-of-defun behavior better

* lisp/progmodes/python.el (python-treesit-beginning-of-defun)
(python-treesit-end-of-defun): New functions.
* lisp/progmodes/python.el (python-mode): Use custom beg/end-of-defun
functions.

2 years ago; Minor improvements in wording of tree-sitter docs
Eli Zaretskii [Wed, 9 Nov 2022 13:35:51 +0000 (15:35 +0200)]
; Minor improvements in wording of tree-sitter docs

* doc/lispref/parsing.texi (Multiple Languages):
* doc/lispref/modes.texi (Parser-based Font Lock): Improve
wording.

2 years ago; Improve tree-sitter documentation
Yuan Fu [Tue, 8 Nov 2022 18:39:53 +0000 (10:39 -0800)]
; Improve tree-sitter documentation

* doc/lispref/modes.texi (Parser-based Font Lock)
* doc/lispref/parsing.texi (Multiple Languages)
* lisp/treesit.el (treesit-range-rules)
(treesit-font-lock-rules): Improve phrasing around queries and keyword
and values.

2 years ago; Improve docstring of treesit-simple-indent-presets
Yuan Fu [Mon, 7 Nov 2022 10:27:59 +0000 (02:27 -0800)]
; Improve docstring of treesit-simple-indent-presets

* lisp/treesit.el (treesit-simple-indent-presets): Improve docstring.

2 years agoAdd tree-sitter indent preset "point-min"
Yuan Fu [Mon, 7 Nov 2022 10:00:04 +0000 (02:00 -0800)]
Add tree-sitter indent preset "point-min"

* doc/lispref/modes.texi (Parser-based Indentation): Update manual.
* lisp/treesit.el (treesit-simple-indent-presets): Add preset.  Update
docstring.

2 years agoAdd treesit--indent-rules-optimize
Yuan Fu [Mon, 7 Nov 2022 09:59:21 +0000 (01:59 -0800)]
Add treesit--indent-rules-optimize

* lisp/treesit.el (treesit--indent-rules-optimize): New function.
(treesit-major-mode-setup): Optimize indent rules when setting up for
major mode.

2 years ago; Documentation fixes for tree-sitter
Yuan Fu [Mon, 7 Nov 2022 09:27:49 +0000 (01:27 -0800)]
; Documentation fixes for tree-sitter

* doc/lispref/parsing.texi (Multiple Languages): Resolve the FIXME.

2 years ago; More documentation improvements for treesit
Eli Zaretskii [Sun, 6 Nov 2022 09:59:27 +0000 (11:59 +0200)]
; More documentation improvements for treesit

* lisp/treesit.el (treesit-language-at, treesit-range-rules)
(treesit-font-lock-rules): Doc fixes.

* doc/lispref/parsing.texi (Multiple Languages):
* doc/lispref/modes.texi (Parser-based Font Lock): Fix wording.

2 years agoFix incorrect tree-sitter fontification
Yuan Fu [Sun, 6 Nov 2022 01:58:39 +0000 (18:58 -0700)]
Fix incorrect tree-sitter fontification

* lisp/treesit.el (treesit-font-lock-fontify-region): If the captured
node is outside of the region between START and END, don't fontify it.
Wrap fontification code in a when form.

2 years agoAllow tree-sitter indent offset to be a variable
Yuan Fu [Sun, 6 Nov 2022 01:32:01 +0000 (18:32 -0700)]
Allow tree-sitter indent offset to be a variable

* doc/lispref/modes.texi (Parser-based Indentation): Update manual.
* lisp/progmodes/js.el (js--treesit-indent-rules): Change all
occurance of ,js-indent-level to js-indent-level.

* lisp/progmodes/ts-mode.el (ts-mode--indent-rules): Change all
occurance of ,ts-mode-indent-offset to ts-mode-indent-offset.
* lisp/treesit.el (treesit-simple-indent-rules): Change docstring.
(treesit-simple-indent): Allow offset to be a variable.

2 years agoDon't indent empty lines in treesit-indent-region
Yuan Fu [Sun, 6 Nov 2022 00:09:51 +0000 (17:09 -0700)]
Don't indent empty lines in treesit-indent-region

* lisp/treesit.el (treesit-indent-region): Wrap the old code in an if
form that checks whether we are at an empty line.  Add comments.

2 years agoMake treesit-language-at work in more situations
Yuan Fu [Sat, 5 Nov 2022 23:55:17 +0000 (16:55 -0700)]
Make treesit-language-at work in more situations

; * lisp/treesit.el (treesit-language-at): Return sometime sensible
even if treesit-language-at-point-function is nil.

2 years ago; Fix documentation for tree-sitter
Yuan Fu [Sat, 5 Nov 2022 23:52:38 +0000 (16:52 -0700)]
; Fix documentation for tree-sitter

* doc/lispref/modes.texi (Parser-based Font Lock): Fix the signature
style.
* doc/lispref/parsing.texi (Language Definitions): Rephrase the
section, remove documentation for deleted functions, fix signature
style.
* lisp/treesit.el (treesit-range-rules)
(treesit-font-lock-rules): Change ARGS to QUERY-SPECS, fix docstring.
* lisp/treesit.el (treesit-inspect-mode): Fix docstring.

2 years ago; Minor fixes to documentation
Eli Zaretskii [Sat, 5 Nov 2022 08:49:46 +0000 (10:49 +0200)]
; Minor fixes to documentation

* lisp/treesit.el (treesit-ready-p, treesit-inspect-mode)
(treesit-range-settings, treesit-range-rules)
(treesit--merge-ranges, treesit-update-ranges)
(treesit-font-lock-rules, treesit-font-lock-feature-list)
(treesit-font-lock-contextual-post-process)
(treesit-font-lock-recompute-features): Doc fixes.
* lisp/progmodes/js.el (js--fontify-template-string): Doc fix.

* doc/lispref/modes.texi (Parser-based Indentation):
* doc/lispref/positions.texi (List Motion):
* doc/lispref/parsing.texi (Tree-sitter major modes):
* doc/lispref/parsing.texi (Multiple Languages): Fix wording and
markup.

2 years ago; Fix minor issues with Tree-sitter documentation in ELisp manual
Eli Zaretskii [Sat, 5 Nov 2022 07:14:25 +0000 (09:14 +0200)]
; Fix minor issues with Tree-sitter documentation in ELisp manual

* doc/lispref/parsing.texi (Multiple Languages): Add @end example.

* doc/lispref/Makefile.in (srcs): Add $(srcdir)/parsing.texi.

2 years ago; * lisp/treesit.el (treesit-inspect-node-at-point): Minor fix.
Yuan Fu [Fri, 4 Nov 2022 17:50:55 +0000 (10:50 -0700)]
; * lisp/treesit.el (treesit-inspect-node-at-point): Minor fix.

2 years ago; * lisp/treesit.el (treesit-ready-p): Fix typo.
Yuan Fu [Fri, 4 Nov 2022 17:16:33 +0000 (10:16 -0700)]
; * lisp/treesit.el (treesit-ready-p): Fix typo.

2 years ago; Minor improvement in treesit-inspect-mode
Yuan Fu [Fri, 4 Nov 2022 17:15:42 +0000 (10:15 -0700)]
; Minor improvement in treesit-inspect-mode

* lisp/treesit.el (treesit-inspect-node-at-point): Show the current
node in bold.
(treesit-inspect-mode): Update docstring.

2 years ago; * lisp/treesit.el (treesit-update-ranges): Update all parsers.
Yuan Fu [Fri, 4 Nov 2022 17:03:17 +0000 (10:03 -0700)]
; * lisp/treesit.el (treesit-update-ranges): Update all parsers.

2 years ago; Replace error with signal in treesit.el
Yuan Fu [Fri, 4 Nov 2022 16:58:08 +0000 (09:58 -0700)]
; Replace error with signal in treesit.el

(or (treesit-parser-create language)
    (error "Cannot find a parser for %s" language))

is turned into

(treesit-parser-create language)

because treesit-parser-create never returns nil.  (We used to use
treesit-get-parser which don't create parser automatically.)

* lisp/treesit.el (treesit-buffer-root-node): Replace error with signal.

2 years ago; Remove unused tree-sitter functions
Yuan Fu [Fri, 4 Nov 2022 16:57:32 +0000 (09:57 -0700)]
; Remove unused tree-sitter functions

They didn't turn out to be very useful.

* lisp/treesit.el (treesit-set-ranges)
(treesit-get-ranges): Remove functions.

2 years agoAdd treesit-indent-error
Yuan Fu [Fri, 4 Nov 2022 16:56:38 +0000 (09:56 -0700)]
Add treesit-indent-error

* lisp/treesit.el (treesit-indent-error): New error.
(treesit--simple-indent-eval): Signal treesit-indent-error.

2 years agoRevise tree-sitter facility for multi-language buffers
Yuan Fu [Fri, 4 Nov 2022 08:31:56 +0000 (01:31 -0700)]
Revise tree-sitter facility for multi-language buffers

* doc/lispref/parsing.texi (Multiple Languages): Extend and update
manual.
* lisp/treesit.el (treesit-range-functions): Remove variable.
(treesit-range-settings): New variable.
(treesit-range-rules): New function.
(treesit--merge-ranges): New function.
(treesit-update-ranges): Use treesit-range-settings instead of
treesit-range-functions.
(treesit-font-lock-rules): Fix docstring.
(treesit-indent)
(treesit-indent-region): Only update ranges in a region.
* test/src/treesit-tests.el (treesit-range): New test.

2 years ago; * lisp/progmodes/js.el (js--fontify-template-string): Fix typo.
Yuan Fu [Fri, 4 Nov 2022 08:18:59 +0000 (01:18 -0700)]
; * lisp/progmodes/js.el (js--fontify-template-string): Fix typo.

2 years agoFix treesit-parser-set-included-ranges
Yuan Fu [Fri, 4 Nov 2022 05:14:50 +0000 (22:14 -0700)]
Fix treesit-parser-set-included-ranges

Track whether a parser has ranges set ourselves.  See comment in
Ftreesit_parser_included_ranges for detail.

* src/treesit.c (make_treesit_parser): Initialize has_range.
(Ftreesit_parser_set_included_ranges): Set has_range.  Fill docstring.
(Ftreesit_parser_included_ranges): Check has_range.
* src/treesit.h (Lisp_TS_Parser): Add has_range.

2 years ago; * src/treesit.c: Minor comment change.
Yuan Fu [Fri, 4 Nov 2022 02:39:43 +0000 (19:39 -0700)]
; * src/treesit.c: Minor comment change.

2 years ago; Tree-sitter: Don't refontify if entity contained in the region
Yuan Fu [Fri, 4 Nov 2022 02:37:55 +0000 (19:37 -0700)]
; Tree-sitter: Don't refontify if entity contained in the region

* lisp/treesit.el (treesit-font-lock-contextual-post-process): Only
refontify when NODE-START < region START.

2 years agoAdd contextual tree-sitter font-lock to ts-mode and js-mode
Yuan Fu [Thu, 3 Nov 2022 18:53:49 +0000 (11:53 -0700)]
Add contextual tree-sitter font-lock to ts-mode and js-mode

* lisp/progmodes/js.el (js--treesit-font-lock-settings)
* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): Capture
commend and strings.  Add empty lines.

2 years agoMinor change to tree-sitter imenu function of js-mode
Yuan Fu [Thu, 3 Nov 2022 18:44:35 +0000 (11:44 -0700)]
Minor change to tree-sitter imenu function of js-mode

If someone don’t like the labels, they can set them to empty strings.

* lisp/progmodes/js.el (js--treesit-imenu-type-alist): Add space.
(js--treesit-imenu-label): Remove space.

2 years ago; Update guides in /admin/notes/tree-sitter
Yuan Fu [Thu, 3 Nov 2022 18:41:42 +0000 (11:41 -0700)]
; Update guides in /admin/notes/tree-sitter

* admin/notes/tree-sitter/html-manual/Language-Definitions.html
* admin/notes/tree-sitter/html-manual/Multiple-Languages.html
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Font-Lock.html
* admin/notes/tree-sitter/html-manual/Parser_002dbased-Indentation.html
* admin/notes/tree-sitter/html-manual/Parsing-Program-Source.html
* admin/notes/tree-sitter/html-manual/Pattern-Matching.html
* admin/notes/tree-sitter/html-manual/Retrieving-Node.html
* admin/notes/tree-sitter/html-manual/Tree_002dsitter-C-API.html
* admin/notes/tree-sitter/html-manual/Using-Parser.html
* admin/notes/tree-sitter/starter-guide: Update to reflect changes
made recently.

2 years agoMore style fixes to tree-sitter code
Po Lu [Thu, 3 Nov 2022 11:14:16 +0000 (19:14 +0800)]
More style fixes to tree-sitter code

* src/treesit.c (treesit_node_uptodate_p):
* src/treesit.h: Fix coding style.

2 years agoMinor fixes to last tree-sitter change
Po Lu [Thu, 3 Nov 2022 11:13:18 +0000 (19:13 +0800)]
Minor fixes to last tree-sitter change

* src/treesit.c (treesit_find_override_name): Initialize
variable.
(Ftreesit_query_expand): Revert change to behavior.

2 years agoStylistic changes to tree-sitter code
Po Lu [Thu, 27 Oct 2022 13:30:23 +0000 (21:30 +0800)]
Stylistic changes to tree-sitter code

* src/treesit.c (treesit_find_override_name)
(treesit_load_language_push_for_each_suffix, treesit_load_language)
(Ftreesit_langauge_available_p, treesit_record_change)
(make_treesit_parser, make_treesit_node, make_treesit_query)
(Ftreesit_parser_create, Ftreesit_parser_delete, Ftreesit_parser_list)
(treesit_check_range_argument, Ftreesit_parser_included_ranges)
(Ftreesit_node_start, Ftreesit_node_end, Ftreesit_node_check)
(Ftreesit_node_child_by_field_name, Ftreesit_pattern_expand)
(Ftreesit_query_expand, treesit_predicates_for_pattern)
(treesit_predicate_capture_name_to_text, treesit_predicate_equal)
(treesit_predicate_match, treesit_eval_predicates)
(Ftreesit_query_capture, treesit_search_dfs, treesit_build_sparse_tree)
(syms_of_treesit): Use FOR_EACH_TAIL (or FOR_EACH_TAIL_SAFE where not
obviously safe), and check list heads and tails correctly; fix coding
style of various constructs, especially:

  variable =
    mumble (frob (bar), (foo () + bar ()) + (baz () + quux ()))

which should actually be

  variable
    = mumble (frob (bar), (foo () + bar ()) + (baz () + quux ()))

and

  foo =
    mumble (frob (bar), 0)
      + (foo () + bar ())
      + (baz () + quux ())

which should actually be

  foo = (mumble (frob (bar), 0)
 + (foo () + bar ())
 + (baz () + quux ()))

* src/treesit.h: Make declaration coding style consistent.

2 years agoCheck for outdated tree-sitter node when printing
Yuan Fu [Thu, 3 Nov 2022 00:47:14 +0000 (17:47 -0700)]
Check for outdated tree-sitter node when printing

* src/print.c (print_vectorlike): Check for outdated node.
* src/treesit.c (treesit_check_node): Use the new function.
(treesit_node_uptodate_p): New function.
* src/treesit.h: Declare new function.

2 years ago; * src/print.c (print_vectorlike): Fix comment for tree-sitter.
Yuan Fu [Thu, 3 Nov 2022 00:23:29 +0000 (17:23 -0700)]
; * src/print.c (print_vectorlike): Fix comment for tree-sitter.

2 years agoPass region start and end to tree-sitter fontification functions
Yuan Fu [Thu, 3 Nov 2022 00:03:03 +0000 (17:03 -0700)]
Pass region start and end to tree-sitter fontification functions

* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Update
function to only fontify within the region.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Pass START and END to
fontification functions.

2 years agoAdd handling of contextual entities in tree-sitter font-lock
Yuan Fu [Wed, 2 Nov 2022 23:31:25 +0000 (16:31 -0700)]
Add handling of contextual entities in tree-sitter font-lock

* lisp/progmodes/python.el: Remove function.
(python--treesit-settings): Capture contextual node.
* lisp/treesit.el (treesit--set-nonsticky):
(treesit-font-lock-contextual-post-process): New functions.
(treesit-font-lock-fontify-region): Change local variable START and
END to NODE-START and NODE-END, handle special capture name
"contextual".
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.

2 years agoChange signature of tree-sitter font-lock functions
Yuan Fu [Wed, 2 Nov 2022 18:26:38 +0000 (11:26 -0700)]
Change signature of tree-sitter font-lock functions

Change from

(START END NODE OVERRIDE &rest _)

to

(NODE OVERRIDE &rest _)

START and END aren't used frequently enough to justify.  If a
fontification function needs them, it can get them from NODE.

* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.

* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string)
(python--treesit-fontify-string-end): Change signature.

* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Remove START and END arguments.

2 years agoFix string fontification for tree-sitter python-mode
Yuan Fu [Tue, 1 Nov 2022 20:26:16 +0000 (13:26 -0700)]
Fix string fontification for tree-sitter python-mode

Now when user inserts a ending quote, the whole string is ganranteed
to be refontified and redisplayed.

* lisp/progmodes/python.el (python--treesit-fontify-string): Change
docstring, now it's called on the leading quote.
(python--treesit-fontify-string-end): New function.
(python--treesit-settings): Capture both leading and ending quote in a
string.

2 years agoDon't print buffer when printing tree-sitter node and parser
Yuan Fu [Tue, 1 Nov 2022 07:39:58 +0000 (00:39 -0700)]
Don't print buffer when printing tree-sitter node and parser

Additionally, print node's position in buffer.

* doc/lispref/parsing.texi (Retrieving Node): Reflect this change in
manual.
* src/print.c (print_vectorlike): Remove printing of buffer, add
printing of node position.

2 years ago; Tree-sitter font-lock debugging: print node type not language
Yuan Fu [Tue, 1 Nov 2022 07:26:15 +0000 (00:26 -0700)]
; Tree-sitter font-lock debugging: print node type not language

* lisp/treesit.el (treesit-font-lock-fontify-region): Print node type.

2 years agoAllow user to add/remove tree-sitter font-lock features
Yuan Fu [Tue, 1 Nov 2022 07:25:47 +0000 (00:25 -0700)]
Allow user to add/remove tree-sitter font-lock features

* lisp/treesit.el (treesit-font-lock-recompute-features): Add
ADD-LIST, REMOVE-LIST argument.

2 years agoQuery on the root node in tree-sitter font-lock
Yuan Fu [Mon, 31 Oct 2022 22:27:40 +0000 (15:27 -0700)]
Query on the root node in tree-sitter font-lock

Rather than querying on the smallest node that spans START to END, we
query on the root node between START to END.

* lisp/progmodes/python.el (python--treesit-fontify-string): Accept
the string rather than the quote node.
(python--treesit-settings): Capture the string rather than the quote
node.
* lisp/treesit.el (treesit-font-lock-fontify-region): Query the root
node rather than the smallest node.

2 years ago; Explain the BEG and END argument in treesit-query-capture better
Yuan Fu [Mon, 31 Oct 2022 22:11:44 +0000 (15:11 -0700)]
; Explain the BEG and END argument in treesit-query-capture better

* doc/lispref/parsing.texi (Pattern Matching): Update manual.
* src/treesit.c (Ftreesit_query_capture): Update docstring.

2 years agoUse treesit-fontify-with-override in tree-sitter functions
Yuan Fu [Mon, 31 Oct 2022 22:08:17 +0000 (15:08 -0700)]
Use treesit-fontify-with-override in tree-sitter functions

* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Use
treesit-fontify-with-override.

2 years agoAdd an argument OVERRIDE to tree-sitter font-lock functions
Yuan Fu [Mon, 31 Oct 2022 22:03:49 +0000 (15:03 -0700)]
Add an argument OVERRIDE to tree-sitter font-lock functions

* doc/lispref/modes.texi (Parser-based Font Lock): Reflect this change
in manual.
* lisp/progmodes/js.el (js--fontify-template-string): Add _OVERRIDE
argument.
* lisp/progmodes/python.el (python--treesit-fontify-string): Add
_OVERRIDE argument.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-fontify-with-override): New function.
(treesit-font-lock-fontify-region): Extract out into
treesit-fontify-with-override.

2 years ago; * lisp/treesit.el (tresit--font-lock-verbose): Fix typo.
Yuan Fu [Mon, 31 Oct 2022 21:26:03 +0000 (14:26 -0700)]
; * lisp/treesit.el (tresit--font-lock-verbose): Fix typo.

2 years agoUse tree-sitter for font-lock's syntactic function (optionally)
Yuan Fu [Mon, 31 Oct 2022 06:59:56 +0000 (23:59 -0700)]
Use tree-sitter for font-lock's syntactic function (optionally)

Before this change, we rebind font-lock-fontify-region-function and
call font-lock's function in tree-sitter's function, and the order of
fontification is tree-sitter, font-lock syntax, font-lock regexp.

Now we make font-lock's syntax function customizable and replace it
with tree-sitter's function, and the order of fontification is
tree-sitter, font-lock regexp.

* doc/lispref/modes.texi (Parser-based Font Lock): Reflect the change
in manual.
* lisp/font-lock.el (font-lock-fontify-syntactically-function): New
varaible.
(font-lock-default-fontify-region): Call
font-lock-fontify-syntactically-function rather.
(font-lock-fontify-syntactically-region): Rename to
font-lock-default-fontify-syntactically
(font-lock-default-fontify-syntactically): Rename to this.

* lisp/treesit.el (treesit-font-lock-fontify-region): Don't call
font-lock functions.
(treesit-font-lock-enable): Remove this function.  It is not used even
before this change.
(treesit-major-mode-setup): Instead of binding
font-lock-fontify-region-function, now bind to
font-lock-fontify-syntactically-function.  And we can let font-lock do
it's thing.

2 years agoImprove python tree-sitter's string fontification
Yuan Fu [Mon, 31 Oct 2022 03:59:50 +0000 (20:59 -0700)]
Improve python tree-sitter's string fontification

* lisp/progmodes/python.el (python--treesit-fontify-string): Handle
not only f-strings, but also docstrings, and NODE is now the last
quote rather than the whole string.
(python--treesit-settings): Use python--treesit-fontify-string for
every occasion.

2 years agoAdd tresit--font-lock-verbose
Yuan Fu [Mon, 31 Oct 2022 03:56:21 +0000 (20:56 -0700)]
Add tresit--font-lock-verbose

* lisp/treesit.el (tresit--font-lock-verbose): New variable.
(treesit-font-lock-fontify-region): Take tresit--font-lock-verbose
into account.  Return jit-lock-bounds.

2 years agoSet jit-lock-contextually in treesit-major-mode-setup
Yuan Fu [Mon, 31 Oct 2022 03:53:44 +0000 (20:53 -0700)]
Set jit-lock-contextually in treesit-major-mode-setup

* lisp/treesit.el (treesit-major-mode-setup): Set jit-lock-contextually.

2 years agoSeparate python tree-sitter fontification into features
Yuan Fu [Sun, 30 Oct 2022 02:59:50 +0000 (19:59 -0700)]
Separate python tree-sitter fontification into features

Level 1: comment string function-name class-name
Level 2: keyword builtin constant type
Level 3: assignment decorator escape-sequence string-interpolation

* lisp/progmodes/python.el (python--treesit-settings): Separate.
(python-mode): Set features.

2 years agoFix byte-compile warning in treesit.el
Yuan Fu [Sat, 29 Oct 2022 21:50:25 +0000 (14:50 -0700)]
Fix byte-compile warning in treesit.el

Originally treesit-simple-indent-presets was

'((parent-is . (lambda (type)
                 `(lambda (n p b)
                    (equal (treesit-node-type p)
                           ,type)))))

Then I changed it to

`((parent-is . ,(byte-compile
                 (lambda (type)
                   (lambda (n p b)
                     (equal (treesit-node-type p)
                            type))))))

but that generates "function already compiled" warning when
byte-compiling treesit.el.  The latest form shouldn't have any problem
anymore.

* lisp/treesit.el (treesit-simple-indent-presets): Change byte-compile
to plain evaluation.

2 years agoNot return noindent in treesit-indent
Yuan Fu [Sat, 29 Oct 2022 21:49:31 +0000 (14:49 -0700)]
Not return noindent in treesit-indent

* lisp/treesit.el (treesit-indent): Don't return noindent.
(treesit-defun-type-regexp): Return (nil . nil) rather than nil.

2 years agoAdd treesit-node-top-level
Yuan Fu [Sat, 29 Oct 2022 21:11:52 +0000 (14:11 -0700)]
Add treesit-node-top-level

Merge treesit-node-top-level-p and treesit--find-top-level-match into
treesit-node-top-level.

* doc/lispref/parsing.texi (Retrieving Node): Add manual entry for it.
* lisp/progmodes/js.el (js--treesit-imenu-1): Replace
treesit-node-top-level-p with treesit-node-top-level.
* lisp/treesit.el (treesit-node-top-level-p)
(treesit--find-top-level-match): Remove functions.
(treesit-node-top-level): New function.
* lisp/treesit.el (treesit-beginning-of-defun)
(treesit-end-of-defun): Replace treesit--find-top-level-match with
treesit-node-top-level.

2 years agoReflect recent change in tree-sitter manual sections
Yuan Fu [Sat, 29 Oct 2022 20:51:45 +0000 (13:51 -0700)]
Reflect recent change in tree-sitter manual sections

* doc/lispref/modes.texi (Parser-based Font Lock):
treesit-font-lock-enable is replaced by treesit-major-mode-setup.
* doc/lispref/modes.texi (Parser-based Indentation): Mention
treesit-major-mode-setup,
* doc/lispref/parsing.texi (Tree-sitter major modes): Remove
treesit-mode and global-treesit-mode, explain setup guideline.

2 years agoFix parent-bol preset for tree-sitter indent
Yuan Fu [Sat, 29 Oct 2022 20:05:52 +0000 (13:05 -0700)]
Fix parent-bol preset for tree-sitter indent

* lisp/treesit.el (treesit-simple-indent-presets): Return point rather
than column.

2 years agoKeep cursor position in treesit-indent
Yuan Fu [Sat, 29 Oct 2022 18:50:15 +0000 (11:50 -0700)]
Keep cursor position in treesit-indent

* lisp/treesit.el (treesit-indent): Create a marker and jump back to
that marker after indentation.

2 years agoRemove hack in js-mode's tree-sitter indentation rules
Yuan Fu [Sat, 29 Oct 2022 16:50:46 +0000 (09:50 -0700)]
Remove hack in js-mode's tree-sitter indentation rules

* lisp/progmodes/js.el (js--treesit-backward-up-list): Remove
function.
(js--treesit-indent-rules): Replace with new rule.

2 years agoRefine handling of node at point in treesit-indent
Yuan Fu [Sat, 29 Oct 2022 16:43:50 +0000 (09:43 -0700)]
Refine handling of node at point in treesit-indent

* doc/lispref/modes.texi (Parser-based Indentation): Document the
semantic clearly.
* lisp/treesit.el (treesit-simple-indent-presets): Replace with
current-indentation.
(treesit--indent-1): Now NODE always starts at BOL, and parent is
always the smallest node above NODE that spans BOL.
(treesit-indent): Don't use save-excurtion, no one else does.

2 years agoFix treesit-indent-region
Yuan Fu [Sat, 29 Oct 2022 16:04:35 +0000 (09:04 -0700)]
Fix treesit-indent-region

* lisp/treesit.el (treesit-simple-indent-presets): Remove extra comma.
(treesit--indent-1): Return (ANCHOR . OFFSET) instead of column.
(treesit-indent): Accept (ANCHOR . OFFSET) from treesit--indent-1.
(treesit--indent-region-batch-size): Reduce to 400.
(treesit-indent-region): Put a marker on each line's ANCHOR and
compute the indentation with ANCHOR + OFFSET.  Precomputing column
gives wrong indentation when the ANHOR line doesn't have correct
indent yet.

2 years agoRemove test of treesit-language-at
Yuan Fu [Sat, 29 Oct 2022 03:17:43 +0000 (20:17 -0700)]
Remove test of treesit-language-at

Because now it just calls a user-supplied function rather than doing
any work by itself.

* test/src/treesit-tests.el (treesit-parser-supplemental): Remove the
test.

2 years agoByte-compile treesit-simple-indent-presets
Yuan Fu [Sat, 29 Oct 2022 00:56:05 +0000 (17:56 -0700)]
Byte-compile treesit-simple-indent-presets

* lisp/treesit.el (treesit-simple-indent-presets): Byte-compile these
functions.

2 years agoAdd treesit-indent-region
Yuan Fu [Sat, 29 Oct 2022 00:54:43 +0000 (17:54 -0700)]
Add treesit-indent-region

Now indenting a large buffer is reasonably fast.

* lisp/treesit.el (treesit-indent): Move meat to treesit--indent-1.
(treesit--indent-1)
(treesit-indent-region): New functions.
(treesit-major-mode-setup): Setup indent-region-function.

2 years agoMake Emacs compile without tree-sitter library
Yuan Fu [Fri, 28 Oct 2022 23:38:37 +0000 (16:38 -0700)]
Make Emacs compile without tree-sitter library

* lisp/treesit.el: Add function declaration forms.
* src/emacs.c: Always include syms_of_treesit.

2 years ago; Make ts-mode indent rules more compact (whitespace change)
Yuan Fu [Fri, 28 Oct 2022 20:36:42 +0000 (13:36 -0700)]
; Make ts-mode indent rules more compact (whitespace change)

* lisp/progmodes/ts-mode.el (ts-mode--indent-rules): fold two-line
rules into one line.

2 years agoAdd in new font lock features in js/ts-mode
Theodor Thornhill [Fri, 28 Oct 2022 19:23:19 +0000 (21:23 +0200)]
Add in new font lock features in js/ts-mode

We want to support font lock features on the syntactic level, not only
the decoration-level level.

New supported features are:
  - comment
  - constant
  - keyword
  - string
  - declaration
  - identifier
  - expression
  - property
  - pattern
  - jsx

* lisp/progmodes/js.el (js--treesit-font-lock-settings, js-mode): Add
in the new features.

* lisp/progmodes/ts-mode.el (ts-mode--indent-rules): Remove faulty '.'
anchor.

(ts-mode--font-lock-settings, ts-mode): Add in the new features.

2 years ago* configure.ac (emacs_config_features): Change to TREE_SITTER
Yuan Fu [Fri, 28 Oct 2022 18:06:56 +0000 (11:06 -0700)]
* configure.ac (emacs_config_features): Change to TREE_SITTER

Last commit on this line has a wrong changelog, the subject should be
emacs_config_features, not WINDOW_SYSTEM_OBJ.

2 years agoMinor doc fixes in treesit.el
Stefan Kangas [Fri, 28 Oct 2022 04:38:00 +0000 (06:38 +0200)]
Minor doc fixes in treesit.el

* lisp/treesit.el (treesit-font-lock-rules)
(treesit-font-lock-settings): Minor doc fixes.

2 years agoReflect tree-sitter indent rule presets change in manual
Yuan Fu [Fri, 28 Oct 2022 04:21:45 +0000 (21:21 -0700)]
Reflect tree-sitter indent rule presets change in manual

* doc/lispref/modes.texi (Parser-based Indentation): String
equivalence checks are changed to regexp matching.

2 years agoImprove tree-sitter indent rules in ts-mode and js-mode
Theodor Thornhill [Fri, 28 Oct 2022 04:05:02 +0000 (21:05 -0700)]
Improve tree-sitter indent rules in ts-mode and js-mode

* lisp/progmodes/js.el (js--treesit-indent-rules)
* lisp/progmodes/ts-mode.el (ts-mode--indent-rules): Add more rules.
(ts-mode): Create parser.

2 years agoAdd some js-mode tree-sitter indent rules
Yuan Fu [Fri, 28 Oct 2022 00:07:12 +0000 (17:07 -0700)]
Add some js-mode tree-sitter indent rules

* lisp/progmodes/js.el (js--treesit-indent-rules): Add rules and a let
form.

2 years ago; * lisp/treesit.el (treesit-simple-indent-presets): Add TODO.
Yuan Fu [Thu, 27 Oct 2022 23:50:20 +0000 (16:50 -0700)]
; * lisp/treesit.el (treesit-simple-indent-presets): Add TODO.

2 years agoRemove tree-sitter-indent's integration with cc-mode
Yuan Fu [Thu, 27 Oct 2022 23:44:33 +0000 (16:44 -0700)]
Remove tree-sitter-indent's integration with cc-mode

Also minor improvement of tree-sitter-indent error messaging.

* lisp/progmodes/js.el (js--treesit-indent-rules): Fix it so it works
with the new presets (which expects regexps).
(js--treesit-cc-indent-rules): Remove variable.
(js-mode): Go back to js--treesit-indent-rules.
* lisp/treesit.el (treesit-indent): Don't report error message when
ANCHOR or OFFSET is nil: let treesit-simple-indent display the
message.
(treesit-simple-indent): Remove cc-mode integration.  Display error
message when no rules matches.

2 years ago* configure.ac (WINDOW_SYSTEM_OBJ): Move tree-sitter placement.
Yuan Fu [Thu, 27 Oct 2022 22:54:04 +0000 (15:54 -0700)]
* configure.ac (WINDOW_SYSTEM_OBJ): Move tree-sitter placement.

2 years agoMinor copy-edits in tree-sitter docs
Stefan Kangas [Thu, 27 Oct 2022 12:01:49 +0000 (14:01 +0200)]
Minor copy-edits in tree-sitter docs

* doc/lispref/parsing.texi (Parsing Program Source): Improve wording.

2 years ago; Fix typos and formatting in tree-sitter docs
Stefan Kangas [Thu, 27 Oct 2022 11:03:17 +0000 (13:03 +0200)]
; Fix typos and formatting in tree-sitter docs

* doc/lispref/parsing.texi (Parsing Program Source): Fix typos and
formatting.