How can I add keyword and namespace highlighting?

[Invertisment]

What would be a way to add the highlighting that is described in this github issue: clojure-vim/clojure.vim#24?

kwd_lit already captures whole keyword but I can't make my own literals in my config (I think).
So should I make a PR into this repository and then make a PR into the nvim repository of treesitter so that it could be enabled in my nvim?

[sogaiu]

@Invertisment For various reasons, at the moment I'm not inclined to change the grammar for this type of thing.

The following is for another lispish language that is somewhat along the lines of what you described:

; XXX: hack for things that might be from "elsewhere"
(
 (sym_lit) @property
 (.match? @property "^.*/.*$")
)

That ends up highlighting the text of the entire symbol though.

I don't know if nvim-treesitter supports highlighting a portion of something that was matched, but perhaps nvim-treesitter folks would know better or may be changes could be made there or customizations provided that could make this kind of thing possible.

May be the following sort of thing might have some hints about what could be done: https://github.com/theHamsta/nvim-treesitter-commonlisp

[Invertisment]

Nah, I'll probably stick to using the regex-based one as I already have my highlighting set up there. I was mostly wondering if it's better.

I was asking about it to understand whether it's as easy to define this kind of highlighting. But the author didn't think about it and it's probably hard to add it.

And looks like it doesn't offer more than the default one. Maybe performance but I'm on nvim and probably it runs in background anyway.

[sogaiu]

My impression has been that tree-sitter doesn't really improve things overall for highlighting for lispish languages -- at least not currently.

I'm going to close this issue for now -- it can be reopened if necessary.

[dannyfreeman]

I'm working on adding treesitter support for clojure-mode in emacs using this grammer. As far as I can tell, there is no way to capture just the namespace portion of a node like sym_lit or kwd_lit using something like

(
 (sym_lit) @property
 (:match @property "^.*/") @namespace
 (:match @property "/.*$") @name
)

in Emacs. (that syntax is wrong because nothing like it exists as far as I know, its only to illustrate)

Having the a namespace and name field on the sym_lit and kwd_lit nodes would make this possible though.

For various reasons, at the moment I'm not inclined to change the grammar for this type of thing

I respect your decision, but thought I would mention it since I've also run into this problem. If you do change your mind I would be happy to try to make a PR. If not, it's just some fancier highlighting that is lost.

[sogaiu]

@dannyfreeman

I agree it would be nice to be able to make such distinctions, but I'm unclear on whether it can be done well enough in the grammar without adversely impacting correctness -- difficulties I've encountered along these lines could often be traced to current tree-sitter capabilities as well as the lack of an appropriate specification for Clojure.

Verifying and supporting sufficient correctness has been one of the more time-consuming aspects of tree-sitter-clojure. Most recently I've been using the method described here.

There are also some users of tree-sitter-clojure so considering potential consequences for them (may be at least nvim-treesitter, difftastic, and Cursorless) seems important.

For at least these reasons, I'm not keen on making changes that aren't fixes at this stage.

However, I'm not adverse to someone else trying :)

Would you like to take over tree-sitter-clojure?

[dannyfreeman]

@sogaiu thanks for the reply. I can appreciate the difficulty of the problem here, and won't pretend I can do better than previous attempts to solve this. That said, I will take some time to learn the grammar language and see if I can make sense of what exists in this repo now.

Would you like to take over tree-sitter-clojure?

I would need to give this some thought. I'm only about two days into working with tree sitter lol

There are also some users of tree-sitter-clojure so considering potential consequences for them (may be at least nvim-treesitter, difftastic, and Cursorless) seems important.

Given this information, maybe it would make sense to transfer ownership of this repo to the tree sitter org: https://github.com/tree-sitter. I had a little trouble figuring out that this was the most up to date grammar. If it lives there I would still think about taking point on it. Again, I need to build some tree sitter chops first and think about it before saying yes.

[dannyfreeman]

Reading through more docs I can really see how a lack of a language spec really makes a project like this difficult. I think the closest thing we have is the informal edn spec https://github.com/edn-format/edn

[sogaiu]

@dannyfreeman

I will take some time to learn the grammar language and see if I can make sense of what exists in this repo now.

My tree-sitter and Clojure knowledge has been suffering from bit-rot but if you think there are questions I might be able to answer, I can try :)

Would you like to take over tree-sitter-clojure?

I would need to give this some thought. I'm only about two days into working with tree sitter lol

Sounds good, no hurry on this end.

maybe it would make sense to transfer ownership of this repo to the tree sitter org: https://github.com/tree-sitter. I had a little trouble figuring out that this was the most up to date grammar.

I can appreciate the point about discoverability. However, I don't think the upstream tree-sitter folks are into taking over grammars -- IIUC, they are ok to list things that other folks are maintaining though (e.g. under Available Parsers). I did consider this, but having observed this comment by one of the authors of a previous Clojure grammar attempt, I was reluctant to step on any toes so didn't pursue the matter. (It may be a non-issue now, but I wasn't successful in contacting him earlier.)

Regarding language spec issues, there is a notes.txt file with some gathered bits along these lines (e.g. mention of The Reader page at clojure.org as well as some EDN bits). Some portions may be out-of-date and it's a bit of a jumble, but FWIW.

On a side note, perhaps you are aware already, but FWIW, various nvim-treesitter contributors have done some neovim-specific work toward highlighting of Clojure code using the current grammar here. I believe there are neovim-specific bits, but perhaps there are some tricks / techniques they've come up with that might be relevant for your Emacs-related activities.

[dannyfreeman]

I can appreciate the point about discoverability. However, I don't think the upstream tree-sitter folks are into taking over grammars -- IIUC, they are ok to list things that other folks are maintaining though (e.g. under Available Parsers). I did consider this, but having observed tree-sitter/tree-sitter#618 (comment) by one of the authors of a previous Clojure grammar attempt, I was reluctant to step on any toes so didn't pursue the matter. (It may be a non-issue now, but I wasn't successful in contacting him earlier.)

Fwiw, I think this repo would be the appropriate one to list since other tools are depending on it now.

On a side note, perhaps you are aware already, but FWIW, various nvim-treesitter contributors have done some neovim-specific work toward highlighting of Clojure code using the current grammar here. I believe there are neovim-specific bits, but perhaps there are some tricks / techniques they've come up with that might be relevant for your Emacs-related activities.

That is helpful, and I'll probably use that as a reference as I work. I've found a work around for the namespaced keyword highlighting as well, someone on the mailing list helped: https://lists.gnu.org/archive/html/emacs-devel/2022-12/msg00122.html

It turns out to not be too complicated to do https://github.com/dannyfreeman/clojure-mode/blob/513403c5e97330b65d024e2e904c165021540480/clojure-ts-mode.el#L162-L177

[sogaiu]

| Fwiw, I think this repo would be the appropriate one to list since other tools are depending on it now.

I don't have a problem with having it listed -- though I don't feel comfortable being the one to make the request given the history.

| I've found a work around for the namespaced keyword highlighting

Glad to hear it :)

[sogaiu]

@dannyfreeman

BTW, I don't know if clojure-ts-mode would be affected, but there is an outstanding issue which hasn't been taken care of because I haven't succeeded in contacting relevant folks at nvim-treesitter to ascertain what consequences it might have for them.

If it ends up being a problem for your case, there is a branch with a fix that might be relevant.

[dannyfreeman]

Fwiw, I think this repo would be the appropriate one to list since other tools are depending on it now.

I don't have a problem with having it listed -- though I don't feel comfortable being the one to make the request given the history.

I would be happy to put in a request then. I can identify the other Clojure grammars and tag their maintainers in an issue somewhere in the tree sitter org. If they'd have objections then we can talk about it over there.

BTW, I don't know if clojure-ts-mode would be affected, but there is #21 which hasn't been taken care of because I haven't succeeded in contacting relevant folks at nvim-treesitter to ascertain what consequences it might have for them.

I think I see that issue happening in emacs 29 as well

A buffer with this

(.get ^ByteBuffer b)

produces this parse tree

(list_lit close: ( (sym_lit)
 (sym_lit
  meta: (meta_lit marker: ^ value: (sym_lit)))
 ))

Trying the updated branch identifies the b

(list_lit close: (
 (sym_lit (sym_name))
 (sym_lit
  meta: 
   (meta_lit marker: ^
    value: (sym_lit (sym_name)))
  (sym_name)) ;; `b`
 ))

I would have to update most of code to work with this sym_name though. Most of the time I think I would want to capture that instead of sym_lit so highlighting the symbol doesn't override the highlighting on the metadata. Overall this is a good change I think that would simplify a lot of my code for emacs.

Edit: I've updated my clojure-mode usage to work of that branch. It also fixed some issues I was having with metadata highlighting

[NoahTheDuke]

I would like to voice my support for this change, as an nvim-treesitter user.

[sogaiu]

@dannyfreeman

| I can identify the other Clojure grammars and tag their maintainers in an issue somewhere in the tree sitter org.

Ok, I see you did that in tree-sitter/tree-sitter#1980

| I've updated my clojure-mode usage to work of that branch. It also fixed some issues I was having with metadata highlighting

Thanks for the report.

I'd like to apply some appropriate fix if practical, but I'll continue this line of thought at #21

[sogaiu]

For the record, thanks to dannyfreeman's efforts in #31, it should be possible to accomplish what was described in the original description.

[oryband]

Can somebody please provide an example on how to chagne symbol and namespace highlighting in nvim so they will not be the same color as plain text?

[sogaiu]

We don't maintain the highlighting queries for Neovim, but can try to help a bit. I'm not primarily a Neovim user, but I think @NoahTheDuke may be.

I don't understand what you mean by "plain text" though.

Would you mind elaborating with an example or two?

For reference, the following (on the right) is the sort of thing I see at the moment:

[sogaiu]

I took a look at nvim-treesitter's README and found this:

All queries found in the runtime directories will be combined. By convention, if you want to write a query, use the queries/ directory, but if you want to extend a query use the after/queries/ directory.

That info seemed to be incomplete though. I found a bit in a discussion that seemed to provide sufficient info: nvim-treesitter/nvim-treesitter#3729 (comment) [1].

I made a file at after/queries/clojure/highlights.scm with the following content:

;; extends
(sym_ns) @namespace

(sym_name) @symbol

That resulted in the namespace portion being highlighted differently from the name portion.

I found @namespace mentioned in this section. It seems like there are a variety of things that might be tried.

If that's not what you are after, it might still be that tweaking the content of a file in that location can lead to success.

---

[1] The missing piece was the ;; extends comment.

[NoahTheDuke]

You have it right, @sogaiu. The only thing I can think is that the @oryband's colorscheme doesn't support treesitter highlighting groups.

[oryband]

I was using https://github.com/dracula/vim and have migrated to https://github.com/Mofiqul/dracula.nvim and the issue has been resolved. dracula.vim supports treesitter but looks unmaintained.

[sogaiu]

@NoahTheDuke Thanks for taking a look and commenting.

@oryband Glad to hear the issue has been resolved :)