I’m using a French dvorak-style keyboard layout (bepo) and I recently noticed that my neovim binding on ; (which I enter with Shift+,, , being on the physical key g on a qwerty keyboard) stopped working.
This behavior is only exhibited by neovim on some yet unreleased master commits. Compiling neovim and runnig it like so (i.e. without any custom configuration and defining the mappings on the fly):
allows to check this behavior on a locally built neovim from the git repository. On the latest neovim stable (v0.6.1), entering ; results in ; being displayed, but not on those latest master commits. Using git bisect, I’ve found this small commit to be the first with the ; mapping not working, while , works.
Meanwhile, the ; mapping still works with Alacritty, for all neovim commits I tested. That suggests that foot might be sending the wrong sequence here.
foot version:
```
foot --version
foot version: 1.11.0 +pgo +ime +graphemes -assertions
```
or
```
./foot --version
foot version: 1.11.0-97-gf2870bf5 (Apr 05 2022, branch 'master') -pgo +ime +graphemes +assertions
```
**************************************
I’m using a French dvorak-style keyboard layout ([bepo][]) and I recently noticed that my [neovim][] binding on `;` (which I enter with `Shift+,`, `,` being on the physical key `g` on a qwerty keyboard) stopped working.
This behavior is only exhibited by neovim on some yet unreleased `master` commits. Compiling neovim and runnig it like so (i.e. without any custom configuration and defining the mappings on the fly):
```
./build/bin/nvim -u NONE +'nmap ; :echo ";"<CR>' +'nmap , :echo ","<CR>'
```
allows to check this behavior on a locally built neovim from the git repository. On the latest neovim stable (`v0.6.1`), entering `;` results in `;` being displayed, but not on those latest `master` commits. Using `git bisect`, I’ve found [this small commit](https://github.com/neovim/neovim/commit/b81c310d4) to be the first with the `;` mapping not working, while `,` works.
Meanwhile, the `;` mapping still works with Alacritty, for all neovim commits I tested. That suggests that *foot* might be sending the wrong sequence here.
[bepo]: https://bepo.fr/wiki/Accueil
[neovim]: https://neovim.io/
Just tested with Kitty, couldn’t reproduce the bug either.
Just tested with Kitty, couldn’t reproduce the bug either.
cljoly
changed title from Some Shift+key not detected in neovim (non-stable) with non-qwerty keyboard to Some Shift+key not detected in neovim (non-stable) with non-qwerty keyboard layout2 months ago
Alacritty does not recognize the modifyOtherKeys sequence (the one Neovim is setting in the commit you linked to), and we explicitly do not send the sequence for kitty since it uses its own encoding. Do you have another terminal emulator you can try to reproduce on?
I am not familiar with foot's codebase, but I believe what's happening is:
Neovim sends CSI > 4 ; 2 m which enables foot's modifyOtherKeys handling
When you press Shift + , this is no longer encoded as ASCII ; but instead CSI 27 ; 2 ; 44 ~ (where 2 is the modifier for Shift and 44 is the decimal representation of ,). Neovim is then not interpreting this sequence properly.
Can you also try reproducing in regular Vim by adding the following to your vimrc?
let &t_TI="\<Esc>[>4;2m"let &t_TE="\<Esc>[>4;0m"
Alacritty does not recognize the `modifyOtherKeys` sequence (the one Neovim is setting in the commit you linked to), and we explicitly do not send the sequence for kitty since it uses its own encoding. Do you have another terminal emulator you can try to reproduce on?
I am not familiar with foot's codebase, but I believe what's happening is:
1. Neovim sends `CSI > 4 ; 2 m` which enables foot's `modifyOtherKeys` handling
2. When you press `Shift + ,` this is no longer encoded as ASCII `;` but instead `CSI 27 ; 2 ; 44 ~` (where `2` is the modifier for Shift and `44` is the decimal representation of `,`). Neovim is then not interpreting this sequence properly.
Can you also try reproducing in regular Vim by adding the following to your vimrc?
```vim
let &t_TI = "\<Esc>[>4;2m"
let &t_TE = "\<Esc>[>4;0m"
```
Could also be worth testing/comparing with xterm, which does support (obviously) modifyOtherKeys.
Also note that foot supports kitty's keyboard protocol as well, so if neovim works better with that, another option is to change neovim to use the kitty kbd protocol (instead of modifyOtherKeys) on foot as well.
I also agree with @gpanders analysis, though I haven't verified it.
Could also be worth testing/comparing with xterm, which does support (obviously) `modifyOtherKeys`.
Also note that foot supports kitty's keyboard protocol as well, so if neovim works better with that, another option is to change neovim to use the kitty kbd protocol (instead of modifyOtherKeys) on foot as well.
I also agree with @gpanders analysis, though I haven't verified it.
Do you have another terminal emulator you can try to reproduce on?
I should be able to test with any terminal emulator installable from Arch’s repositories.
Thanks for your quick feedback on this!
Tomorrow, I will:
- [x] try reproducing in regular Vim
- [x] try reproducing on xterm
- [x] open an issue on neovim issue tracker
> Do you have another terminal emulator you can try to reproduce on?
I should be able to test with any terminal emulator installable from Arch’s repositories.
It appears XTerm is somewhat inconsistent here. For regular (a-z) keys, it produces a CSI 27 ; 2 ; N~ escape for shift combinations. But for some, it does not, but instead produces a plain (but shifted) symbol. , vs. ; is one such example. Most (all?) shift+number keys also behave like this (on my Swedish layout).
However, it's not as simple as letter keys vs. non-letter keys; e.g. - produces a CSI 27 escape.
This can be considered to be an issue with foot, in the sense that we're not producing output identical to XTerm.
But it could also be unintended behavior in XTerm.
It appears XTerm is somewhat inconsistent here. For regular (a-z) keys, it produces a `CSI 27 ; 2 ; N~` escape for <kbd>shift</kbd> combinations. But for some, it does not, but instead produces a plain (but shifted) symbol. `,` vs. `;` is one such example. Most (all?) <kbd>shift</kbd>+<kbd>number</kbd> keys also behave like this (on my Swedish layout).
However, it's not as simple as letter keys vs. non-letter keys; e.g. `-` produces a `CSI 27` escape.
This can be considered to be an issue with foot, in the sense that we're not producing output identical to XTerm.
But it _could_ also be unintended behavior in XTerm.
vim gets the mapping for both ; and ,. (athough it does not understand the <CR> somehow, not a big deal, I get the partial :echo command and that’s enough to test).
EDIT: Tested with vim -u DEFAULTS +'let &t_TE = "\<Esc>[>4;0m"' +'let &t_TI = "\<Esc>[>4;2m"' +'nmap ; :echo ";"<CR>' +'nmap , :echo ","<CR>' as well, and both mappings work.
Trying to reproduce with vim:
```
~> vim --version --help
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Mar 30 2022 23:54:07)
Included patches: 1-4651
Compiled by Arch Linux
Huge version without GUI. Features included (+) or not (-):
+acl +file_in_path +mouse_urxvt -tag_any_white
+arabic +find_in_path +mouse_xterm +tcl/dyn
+autocmd +float +multi_byte +termguicolors
+autochdir +folding +multi_lang +terminal
-autoservername -footer -mzscheme +terminfo
-balloon_eval +fork() +netbeans_intg +termresponse
+balloon_eval_term +gettext +num64 +textobjects
-browse -hangul_input +packages +textprop
++builtin_terms +iconv +path_extra +timers
+byte_offset +insert_expand +perl/dyn +title
+channel +ipv6 +persistent_undo -toolbar
+cindent +job +popupwin +user_commands
-clientserver +jumplist +postscript +vartabs
-clipboard +keymap +printer +vertsplit
+cmdline_compl +lambda +profile +vim9script
+cmdline_hist +langmap -python +viminfo
+cmdline_info +libcall +python3/dyn +virtualedit
+comments +linebreak +quickfix +visual
+conceal +lispindent +reltime +visualextra
+cryptv +listcmds +rightleft +vreplace
+cscope +localmap +ruby/dyn +wildignore
+cursorbind +lua/dyn +scrollbind +wildmenu
+cursorshape +menu +signs +windows
+dialog_con +mksession +smartindent +writebackup
+diff +modify_fname -sodium -X11
+digraphs +mouse -sound -xfontset
-dnd -mouseshape +spell -xim
-ebcdic +mouse_dec +startuptime -xpm
+emacs_tags +mouse_gpm +statusline -xsmp
+eval -mouse_jsbterm -sun_workshop -xterm_clipboard
+ex_extra +mouse_netterm +syntax -xterm_save
+extra_search +mouse_sgr +tag_binary
-farsi -mouse_sysmouse -tag_old_static
system vimrc file: "/etc/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
defaults file: "$VIMRUNTIME/defaults.vim"
fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc -L. -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,-E -Wl,-rpath,/usr/lib/perl5/5.34/core_perl/CORE -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto -L/usr/local/lib -o vim -lm -ltinfo -lelf -lacl -lattr -lgpm -Wl,-E -Wl,-rpath,/usr/lib/perl5/5.34/core_perl/CORE -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -fstack-protector-strong -L/usr/local/lib -L/usr/lib/perl5/5.34/core_perl/CORE -lperl -lpthread -ldl -lm -lcrypt -lutil -lc -L/usr/lib -ltclstub8.6 -ldl -lz -lpthread -lm
```
Using foot 1.11.0, with this command:
```
vim -u NONE +'let &t_TE = "\<Esc>[>4;0m"' +'let &t_TI = "\<Esc>[>4;2m"' +'nmap ; :echo ";"<CR>' +'nmap , :echo ","<CR>'
```
vim gets the mapping for both `;` and `,`. (athough it does not understand the `<CR>` somehow, not a big deal, I get the partial `:echo` command and that’s enough to test).
**EDIT**: Tested with `vim -u DEFAULTS +'let &t_TE = "\<Esc>[>4;0m"' +'let &t_TI = "\<Esc>[>4;2m"' +'nmap ; :echo ";"<CR>' +'nmap , :echo ","<CR>'` as well, and both mappings work.
However, it's not as simple as non-letter keys; e.g. - produces a CSI 27 escape.
I’ve tried to Google how to get the escape sequence produced as one enters the the keys in the terminal, without success. Would you have any pointers on how you do that please?
> However, it's not as simple as non-letter keys; e.g. `-` produces a `CSI 27` escape.
I’ve tried to Google how to get the escape sequence produced as one enters the the keys in the terminal, without success. Would you have any pointers on how you do that please?
But note that you'll probably find the terminal unusable after this...
For "normal" escapes, where you don't have to enable a specific terminal mode, just pressing ctrl+v before your key combo is usually good enough (you'll have to press ctrl+v before each combo).
In this case, you can do
```sh
echo -e '\e[>4;2m' && cat -
```
But note that you'll probably find the terminal unusable after this...
For "normal" escapes, where you don't have to enable a specific terminal mode, just pressing <kbd>ctrl</kbd>+<kbd>v</kbd> before your key combo is usually good enough (you'll have to press <kbd>ctrl</kbd>+<kbd>v</kbd> before **each** combo).
So in a newly opened terminal, I’m switching to the CSI > 4 ; 2 m mode and I enter , then ;
echo -e '\e[>4;2m' && cat -
,^[[27;2;59~
so ; is sent as CSI 27 ; 2 ; 59 ~, if I’m not mistaken.
When you press Shift + , this is no longer encoded as ASCII ; but instead CSI 27 ; 2 ; 44 ~ (where 2 is the modifier for Shift and 44 is the decimal representation of ,). Neovim is then not interpreting this sequence properly.
So in a newly opened terminal, I’m switching to the `CSI > 4 ; 2 m` mode and I enter `,` then `;`
```
echo -e '\e[>4;2m' && cat -
,^[[27;2;59~
```
so `;` is sent as `CSI 27 ; 2 ; 59 ~`, if I’m not mistaken.
> When you press Shift + , this is no longer encoded as ASCII ; but instead CSI 27 ; 2 ; 44 ~ (where 2 is the modifier for Shift and 44 is the decimal representation of ,). Neovim is then not interpreting this sequence properly.
That seems inconsistent with our [initial hypothesis](https://codeberg.org/dnkl/foot/issues/1009#issuecomment-417523)
Correct. 59 is the key code for ;. So it's basically saying, you pressed ;, and shift is the (only) active modifier.
If you do the same experiment in XTerm, you'll get a plain ; instead. But for e.g. shift+a you'll not get a plain A, but a CSI 27 escape.
Correct. 59 is the key code for `;`. So it's basically saying, you pressed `;`, and `shift` is the (only) active modifier.
If you do the same experiment in XTerm, you'll get a plain `;` instead. But for e.g. <kbd>shift</kbd>+<kbd>a</kbd> you'll **not** get a plain `A`, but a `CSI 27` escape.
If you do the same experiment in XTerm, you'll get a plain ; instead. But for e.g. shift+a you'll not get a plain A, but a CSI 27 escape.
That’s right.
> If you do the same experiment in XTerm, you'll get a plain `;` instead. But for e.g. <kbd>shift</kbd>+<kbd>a</kbd> you'll **not** get a plain `A`, but a `CSI 27` escape.
That’s right.
For example, shift+a results in CSI 27 ; 2 ; 65~, where 65 is the decimal representation of upper case A.
It's no different from XTerm's CSI sequences.
For example, shift+a results in `CSI 27 ; 2 ; 65~`, where 65 is the decimal representation of upper case `A`.
What I think XTerm is doing, is checking if the produced symbol (A) is the upper case version of the key's base symbol (a). And if it is, it emits a CSI sequence.
If it's not (for example, ; is not the upper case version of ,), then it emits a plain symbol instead of a CSI symbol.
I'll try to put together a POC for foot, to see if that improves our compatibility with XTerm.
What I think XTerm is doing, is checking if the produced symbol (`A`) is the upper case version of the key's base symbol (`a`). And if it is, it emits a CSI sequence.
If it's not (for example, `;` is not the upper case version of `,`), then it emits a plain symbol instead of a CSI symbol.
I'll try to put together a POC for foot, to see if that improves our compatibility with XTerm.
You said that to enter ; you press Shift + ,, right? But the escape sequence is reporting Shift + ;. I guess that might be the problem.
Yes, that’s right.
> You said that to enter `;` you press `Shift + ,`, right? But the escape sequence is reporting `Shift + ;`. I guess that might be the problem.
Yes, that’s right.
foot version:
or
I’m using a French dvorak-style keyboard layout (bepo) and I recently noticed that my neovim binding on
;
(which I enter withShift+,
,,
being on the physical keyg
on a qwerty keyboard) stopped working.This behavior is only exhibited by neovim on some yet unreleased
master
commits. Compiling neovim and runnig it like so (i.e. without any custom configuration and defining the mappings on the fly):allows to check this behavior on a locally built neovim from the git repository. On the latest neovim stable (
v0.6.1
), entering;
results in;
being displayed, but not on those latestmaster
commits. Usinggit bisect
, I’ve found this small commit to be the first with the;
mapping not working, while,
works.Meanwhile, the
;
mapping still works with Alacritty, for all neovim commits I tested. That suggests that foot might be sending the wrong sequence here.Just tested with Kitty, couldn’t reproduce the bug either.
Some Shift+key not detected in neovim (non-stable) with non-qwerty keyboardto Some Shift+key not detected in neovim (non-stable) with non-qwerty keyboard layout 2 months agoAlacritty does not recognize the
modifyOtherKeys
sequence (the one Neovim is setting in the commit you linked to), and we explicitly do not send the sequence for kitty since it uses its own encoding. Do you have another terminal emulator you can try to reproduce on?I am not familiar with foot's codebase, but I believe what's happening is:
CSI > 4 ; 2 m
which enables foot'smodifyOtherKeys
handlingShift + ,
this is no longer encoded as ASCII;
but insteadCSI 27 ; 2 ; 44 ~
(where2
is the modifier for Shift and44
is the decimal representation of,
). Neovim is then not interpreting this sequence properly.Can you also try reproducing in regular Vim by adding the following to your vimrc?
Could also be worth testing/comparing with xterm, which does support (obviously)
modifyOtherKeys
.Also note that foot supports kitty's keyboard protocol as well, so if neovim works better with that, another option is to change neovim to use the kitty kbd protocol (instead of modifyOtherKeys) on foot as well.
I also agree with @gpanders analysis, though I haven't verified it.
@cljoly I am not convinced that this is not a bug in Neovim, so could you also open an issue in our issue tracker?
Thanks for your quick feedback on this!
Tomorrow, I will:
I should be able to test with any terminal emulator installable from Arch’s repositories.
It appears XTerm is somewhat inconsistent here. For regular (a-z) keys, it produces a
CSI 27 ; 2 ; N~
escape for shift combinations. But for some, it does not, but instead produces a plain (but shifted) symbol.,
vs.;
is one such example. Most (all?) shift+number keys also behave like this (on my Swedish layout).However, it's not as simple as letter keys vs. non-letter keys; e.g.
-
produces aCSI 27
escape.This can be considered to be an issue with foot, in the sense that we're not producing output identical to XTerm.
But it could also be unintended behavior in XTerm.
Just tried reproducing, XTerm does not have the bug.
Trying to reproduce with vim:
Using foot 1.11.0, with this command:
vim gets the mapping for both
;
and,
. (athough it does not understand the<CR>
somehow, not a big deal, I get the partial:echo
command and that’s enough to test).EDIT: Tested with
vim -u DEFAULTS +'let &t_TE = "\<Esc>[>4;0m"' +'let &t_TI = "\<Esc>[>4;2m"' +'nmap ; :echo ";"<CR>' +'nmap , :echo ","<CR>'
as well, and both mappings work.For the corresponding GitHub issue: https://github.com/neovim/neovim/issues/18010 @gpanders
I’ve tried to Google how to get the escape sequence produced as one enters the the keys in the terminal, without success. Would you have any pointers on how you do that please?
In this case, you can do
But note that you'll probably find the terminal unusable after this...
For "normal" escapes, where you don't have to enable a specific terminal mode, just pressing ctrl+v before your key combo is usually good enough (you'll have to press ctrl+v before each combo).
Another option is
showkey -a
Thanks!
So in a newly opened terminal, I’m switching to the
CSI > 4 ; 2 m
mode and I enter,
then;
so
;
is sent asCSI 27 ; 2 ; 59 ~
, if I’m not mistaken.That seems inconsistent with our initial hypothesis
Correct. 59 is the key code for
;
. So it's basically saying, you pressed;
, andshift
is the (only) active modifier.If you do the same experiment in XTerm, you'll get a plain
;
instead. But for e.g. shift+a you'll not get a plainA
, but aCSI 27
escape.That’s right.
You said that to enter
;
you pressShift + ,
, right? But the escape sequence is reportingShift + ;
. I guess that might be the problem.It's no different from XTerm's CSI sequences.
For example, shift+a results in
CSI 27 ; 2 ; 65~
, where 65 is the decimal representation of upper caseA
.What I think XTerm is doing, is checking if the produced symbol (
A
) is the upper case version of the key's base symbol (a
). And if it is, it emits a CSI sequence.If it's not (for example,
;
is not the upper case version of,
), then it emits a plain symbol instead of a CSI symbol.I'll try to put together a POC for foot, to see if that improves our compatibility with XTerm.
Yes, that’s right.