Feature Request: Auto-Detection of Terminal-Based Web Apps (e.g., xterm.js) for Default Vimium Disablement

[gwpl]

Feature Request: Auto-Detection of Terminal-Based Web Apps (e.g., xterm.js) for Default Vimium Disablement

Problem:
Vimium currently relies on URL matching to determine where it should be enabled or disabled. This approach is insufficient for dynamic environments where:

• Users access terminal interfaces via tools like [ttyd](https://github.com/tsl0922/ttyd), which run on different IPs/ports.
• These tools utilize [xterm.js](https://xtermjs.org/) under the hood.
• Other apps like [webvm.io](https://webvm.io/) also embed xterm.js or similar terminal emulators.

Request:
Implement a mechanism for Vimium to detect terminal-based web apps dynamically by:

1. Detecting the presence of xterm.js: Check for global objects, DOM elements, or other indicators unique to xterm.js.
2. Auto-disable Vimium by default: When such terminal environments are detected, prevent keybinding conflicts without relying on static URL rules.
3. Optional Setting: Provide a toggle in Vimium settings to enable/disable this behavior for users who may prefer manual control.

Why It Matters:

• IPs and ports change frequently in dynamic setups, making URL-based management impractical.
• Terminal apps often require precise keyboard input, and Vimium keybindings can interfere with terminal workflows.
• A detection-based approach improves ergonomics and user experience without additional configuration overhead.

Potential Indicators for Detection:

• DOM elements with classes like .xterm, .xterm-viewport, .xterm-screen.
• Global JavaScript objects or functions such as Terminal from xterm.js.
• Mutation observers for dynamic terminal instantiation.

Impact:
This change would significantly improve usability for users working with web-based terminals, especially in development, IoT (e.g., Raspberry Pi), and remote administration contexts.

---

Related Projects Using xterm.js:

• [ttyd](https://github.com/tsl0922/ttyd)
• [webvm.io](https://webvm.io/)
• Numerous custom self-hosted terminal UIs

Thank you for considering this enhancement!

[philg-dev]

I'm not entirely sure what your issue is with Vimium on those sites. Could you explain what the problem is?

I just went to webvm.io and it's just a <textarea> so once you set the cursor in the textarea Vimium doesn't do anything. Can you describe a use case where Vimium actually interferes with handling the page?

The only thing I can think of right now is, when you hit ESC, it will release focus from the textarea and then Vimium will handle keystrokes again.

[gwpl]

Thank you for looking into issue, and "escape" key, which is very common when using vim, however does not exactly work in web browser xterm.js , however still I was pressing it with muscle memory and then Vimium bindings were triggered. Therefore issue maybe closed! Before that, let me share canvas and webgl renderings to test if you like.

Now, understanding what triggered it, I see that indeed, [Escape] key was trigger.

Thank you for hint about textarea, as I am more backend programmer so picking up which pieces matter frontend wise.

Would it also be safe in case of webgl and canvas renderings? ttyd may use those two alternatively.

As you looked into my issue, I spin up two instances for you , if you like to test with expert frontend eye if assumptions hold also for canvas and webgl2 renderings:

Below I spin up for few days vm with some examples using ttyd with webgl2 and canvas renderdings if you would like to double check yourself - when I tried, works (unless I press Escape of course) running bash select over https://github.com/ratatui/ratatui/tree/main/examples (btw. "canvas", "chart" are great!):

canvas:
~/ratatui# ttyd -t rendererType=canvas -t disableLeaveAlert=true -t disableResizeOverlay=true -t enableSixel=true -W -p 2047 ~/cargo-example-select --loop
http://188.166.37.88:2047/

webgl2:
~/ratatui# ttyd -t disableLeaveAlert=true -t disableResizeOverlay=true -t enableSixel=true -W -p 2048 ~/cargo-example-select --loop
http://188.166.37.88:2048/

( for curious ones, I placed cargo-example-select => https://gist.github.com/gwpl/f78af717785959ac81ec1fc21fb6056f )

Once issue is closed I assume you finished testing / playing with it, and I will delete VM.

Thank you!

[philg-dev]

Hey, I'm sorry - I lost track of this issue for some days and then I had a medical emergency situation... (All good now though, so don't worry.)

I'm not a super frontend expert either, but I've taken a brief look at your two VMs and it seems like under the hood there's always the textarea element which is used to handle the inputs and I assume the canvas / webgl2 then just renders the contents of the textarea in a pretty way.

You can check it yourself as well by checking in the browser console with document.activeElement as long as the cursor is in the virtual terminal. Hitting Escape will trigger Vimium to remove the focus from the textarea to the body element again.

Since your problem overall seems to be very specific, I can't really think of any good way to solve this issue... The only idea that comes to my mind is to offer an option to disable Escape from switching away focus from any text input fields. I'm not super familiar with the Vimium code yet, but I think that might be fairly easy.

What do you think about this solution? Do you think it might work for you? I'm not sure if this would make it into the mainline Vimium release, but at the very least it could be implemented in a fork for the time being.

[philg-dev]

I just found out that Vimium C seems to support unmapping the Escape key. I've done a very basic test on webvm.io including some vim editing and it seemed to help. Maybe you can test the Vimium C addon instead - from what I've seen it's an actively maintained fork of Vimium and the author of the fork is also active in issue discussions here on the upstream repository.

I've tested with this custom mapping: (Edit: just to avoid confusion - this is currently Vimium C exclusive functionality and does not work in normal Vimium.)

unmap <esc>

It might be possible that this has some unwanted side-effects,... I have tested some use cases in handling Vimium C that involves Escape (e.g. exiting visual / caret / search / link hint mode, closing vomnibar) and everything seemed to work just fine even with Escape unmapped. The only thing that doesn't seem to work anymore is leaving Insert Mode... So once you enter Insert Mode you're stuck with it forever, it seems. I personally almost never use Insert Mode, so for me this wouldn't be a big deal. Maybe this would be considered a bug by the author and could be fixed.

Hope this helps!