raw escape output on certain key combinations #628

Open
opened 7 months ago by rightnow · 12 comments

Certain key combinations, such as SHIFT+ENTER and CTRL+ENTER result in garbage output printed to the screen. Other terminals usually print out a carriage return, which is what I expected. Can foot behave similarly? These aren't useful key combinations, but I happen to hit them frequently while typing fast.

My setup: sway and foot 1.8.1

Certain key combinations, such as SHIFT+ENTER and CTRL+ENTER result in garbage output printed to the screen. Other terminals usually print out a carriage return, which is what I expected. Can foot behave similarly? These aren't useful key combinations, but I happen to hit them frequently while typing fast. My setup: sway and foot 1.8.1
Owner

It's not garbage, but an escape sequence that contains modifier (i.e. control, shift, etc) information.

What other terminals do is simply ignore modifiers for Return (meaning, it is impossible for applications to tell the difference between Return and Shift+Return).

Coming back to foot; the problem is that your shell doesn't recognize and parse these sequences, and instead just dumps them on stdout.

Ignoring modifier+Return in foot too has been discussed before. The problem that I see with that is, if we do remove them, we'd make it completely impossible to use these combos, since foot also does not support binding keys to escape sequences in foot.ini.

Thus, I really don't want to remove these escapes until #325 has been fixed. Then we'd at least have the option to remove them.

Until then, your best option is to manually map these escapes in your shell. In zsh, for example, you can do:

bindkey "\e[27;2;13~" accept-line  # shift+return
bindkey "\e[27;5;13~" accept-line  # ctrl+return

If there are more combos, use Control+v, then type your combo, to see the full escape sequence. For Control+Return, you'll see:

^[[27;5;13~

where ^[ is ESC, i.e. \e in the bindkey examples above.

It's not garbage, but an escape sequence that contains modifier (i.e. control, shift, etc) information. What other terminals do is simply ignore modifiers for <kbd>Return</kbd> (meaning, it is **impossible** for applications to tell the difference between <kbd>Return</kbd> and <kbd>Shift</kbd>+<kbd>Return</kbd>). Coming back to foot; the problem is that your shell doesn't recognize and parse these sequences, and instead just dumps them on stdout. Ignoring <kbd>modifier</kbd>+<kbd>Return</kbd> in foot too _has_ been discussed before. The problem that I see with that is, if we _do_ remove them, we'd make it completely impossible to use these combos, since foot also does not support binding keys to escape sequences in `foot.ini`. Thus, I really don't want to remove these escapes until https://codeberg.org/dnkl/foot/issues/325 has been fixed. Then we'd at least have the _option_ to remove them. Until then, your best option is to manually map these escapes in your shell. In zsh, for example, you can do: ```sh bindkey "\e[27;2;13~" accept-line # shift+return bindkey "\e[27;5;13~" accept-line # ctrl+return ``` If there are more combos, use <kbd>Control</kbd>+<kbd>v</kbd>, then type your combo, to see the full escape sequence. For <kbd>Control</kbd>+<kbd>Return</kbd>, you'll see: ```sh ^[[27;5;13~ ``` where `^[` is ESC, i.e. `\e` in the `bindkey` examples above.
Poster

Thanks, I will use the workaround for now. For those using bash, this worked for mapping SHIFT+ENTER and CTRL+ENTER to a newline:

$ bind '"\e[27;2;13~":"\n"'
$ bind '"\e[27;5;13~":"\n"'
Thanks, I will use the workaround for now. For those using bash, this worked for mapping SHIFT+ENTER and CTRL+ENTER to a newline: ```bash $ bind '"\e[27;2;13~":"\n"' $ bind '"\e[27;5;13~":"\n"' ```
rightnow changed title from garbage output on certain key combinations to raw escape output on certain key combinations 7 months ago

It's not garbage, but an escape sequence that contains modifier (i.e. control, shift, etc) information.

Can you share a reference that documents these encodings? Are they canonical escape sequences standardized somewhere or arbirary ones chosen by foot?

> It's not garbage, but an escape sequence that contains modifier (i.e. control, shift, etc) information. Can you share a reference that documents these encodings? Are they canonical escape sequences standardized somewhere or arbirary ones chosen by foot?
Owner

Foot doesn't emit any custom escapes by default (we do have custom escapes for Escape that can be enabled with a custom Private Mode sequence).

All sequences are XTerm compatible, and are documented throughout https://invisible-island.net/xterm/ctlseqs/ctlseqs.html. A good starting point is https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h2-Special-Keyboard-Keys.

Note that foot by default emits escapes that require modifyOtherKeys to be enabled in XTerm. I.e. foot's default doesn't match XTerm's default.

Foot doesn't emit any custom escapes by default (we do have custom escapes for <kbd>Escape</kbd> that can be enabled with a custom _Private Mode_ sequence). All sequences are XTerm compatible, and are documented throughout https://invisible-island.net/xterm/ctlseqs/ctlseqs.html. A good starting point is https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h2-Special-Keyboard-Keys. Note that foot by default emits escapes that require `modifyOtherKeys` to be enabled in XTerm. I.e. foot's default doesn't match XTerm's default.
Owner

@craigbarnes are you aware of any better documentation?

@craigbarnes are you aware of any better documentation?

Ok, thanks. Maybe I can convince python-prompt-toolkit to support these codes.

Ok, thanks. Maybe I can convince python-prompt-toolkit to support these codes.
Collaborator

@dnkl I can't think of any better docs for key encodings, although I haven't looked for any recently.

I think certain apps/shells are mostly to blame for this issue though. If they started parsing input sequences instead of string matching them, they could easily discard unbound/unknown sequences, with no extra configuration required from users.

The status quo in things like readline probably won't change any time soon though and I think this issue will probably keep popping up. I'm wondering if it'd make sense to go back to the legacy xterm behavior for common problem cases like Shift+Enter and then offer #319 (and/or an opt-in modifyOtherKeys mode) as a solution for those who need it to be bindable. I vaguely remember kitty having taken a similar course of action on the Shift+Enter issue, after originally doing something similar to foot.

@dnkl I can't think of any better docs for key encodings, although I haven't looked for any recently. I think certain apps/shells are mostly to blame for this issue though. If they started *parsing* input sequences instead of string matching them, they could easily discard unbound/unknown sequences, with no extra configuration required from users. The status quo in things like readline probably won't change any time soon though and I think this issue will probably keep popping up. I'm wondering if it'd make sense to go back to the legacy xterm behavior for common problem cases like Shift+Enter and then offer #319 (and/or an opt-in `modifyOtherKeys` mode) as a solution for those who need it to be bindable. I vaguely remember kitty having taken a similar course of action on the Shift+Enter issue, after originally doing something similar to foot.

Okay that was a good read. If you don't mind, I'll summarize what I read here.

I understand that foot today is closest to xterm's modifyOtherKeys = 1 mode [1]. In this mode xterm also emits \e[27;2;13~ on Shift+Return. xterm also understands a control sequence CSI > "mode" ; "value" m which changes the disambiguated key mode [2].

I also understand that there are (at least) two competing disambiguated key formats. First, the original FK27 format implemented by xterm and now (partially) by foot, and another CSI-u format, also originally implemented by xterm, then later described in [3], implemented in libtermkey and used by neovim for input disambiguation. Xterm switches between the two using the formatOtherKeys resource.

Some useful history is given here [5] (link from the xterm man page). This page seems to excoriate the champion of CSI-u format; the author of this article has a clear preference for the FK27 format.

Finally, more recently, the author of kitty has tried to expand on the panned fixterm proposal in his own proposal [6] which attempts to address the relevant shortcomings and errata.

The only client adoptions I know of are those mentioned by [5], which suggests emacs understands the FK27 format and neovim understands the CSI-u format.

After reading this I'm thinking we actually will need a way to disable the disambiguated key mode. To me it sounds like [4] suggests modifyOtherKeys should always be set to 0 outside of a client application. It would also be worth considering replacing the implementation with [6].

[1] https://invisible-island.net/xterm/manpage/xterm.html#VT100-Widget-Resources:modifyOtherKeys
[2] https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h3-Functions-using-CSI-_-ordered-by-the-final-character_s_
[3] http://www.leonerd.org.uk/hacks/fixterms/
[4] https://invisible-island.net/xterm/xterm.faq.html#xterm_modother
[5] https://invisible-island.net/xterm/modified-keys.html
[6] https://sw.kovidgoyal.net/kitty/keyboard-protocol/

Okay that was a good read. If you don't mind, I'll summarize what I read here. I understand that foot today is closest to xterm's modifyOtherKeys = 1 mode [1]. In this mode xterm also emits `\e[27;2;13~` on Shift+Return. xterm also understands a control sequence `CSI > "mode" ; "value" m` which changes the disambiguated key mode [2]. I also understand that there are (at least) two competing disambiguated key formats. First, the original FK27 format implemented by xterm and now (partially) by foot, and another CSI-u format, also originally implemented by xterm, then later described in [3], implemented in libtermkey and used by neovim for input disambiguation. Xterm switches between the two using the formatOtherKeys resource. Some useful history is given here [5] (link from the xterm man page). This page seems to excoriate the champion of CSI-u format; the author of this article has a clear preference for the FK27 format. Finally, more recently, the author of kitty has tried to expand on the panned fixterm proposal in his own proposal [6] which attempts to address the relevant shortcomings and errata. The only client adoptions I know of are those mentioned by [5], which suggests emacs understands the FK27 format and neovim understands the CSI-u format. After reading this I'm thinking we actually will need a way to disable the disambiguated key mode. To me it sounds like [4] suggests modifyOtherKeys should always be set to 0 outside of a client application. It would also be worth considering replacing the implementation with [6]. [1] https://invisible-island.net/xterm/manpage/xterm.html#VT100-Widget-Resources:modifyOtherKeys [2] https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h3-Functions-using-CSI-\_-ordered-by-the-final-character_s_ [3] http://www.leonerd.org.uk/hacks/fixterms/ [4] https://invisible-island.net/xterm/xterm.faq.html#xterm_modother [5] https://invisible-island.net/xterm/modified-keys.html [6] https://sw.kovidgoyal.net/kitty/keyboard-protocol/
Collaborator

If you don't mind, I'll summarize what I read here.

Seems like a pretty good summary to me.

I understand that foot today is closest to xterm's modifyOtherKeys = 1 mode

I guess that's somewhat true, although I think, for practical reasons, it makes sense to see the modifyOtherKeys modes and the CSI 27 encoding as 2 different (albeit related) things.

After reading this I'm thinking we actually will need a way to disable the disambiguated key mode. To me it sounds like [4] suggests modifyOtherKeys should always be set to 0 outside of a client application.

foot only uses the CSI 27 encoding for keys that previously had no distinct encoding in the xterm protocol, so in most cases they shouldn't present a problem. Enter is the only one that seems to be an issue for some people (myself included) and only due to accidentally holding modifier keys.

There are sequences that xterm emits, even with modifyOtherKeys=0, that can still cause "garbage" to be printed (Ctrl+arrows for example). That's just a general flaw with the way some apps handle input.

It would also be worth considering replacing the implementation with [6].

There's an open issue on the subject of supporting the kitty protocol, although so far I've managed to find less time to spend on it than I anticipated.

> If you don't mind, I'll summarize what I read here. Seems like a pretty good summary to me. > I understand that foot today is closest to xterm's modifyOtherKeys = 1 mode I guess that's somewhat true, although I think, for practical reasons, it makes sense to see the `modifyOtherKeys` modes and the CSI 27 encoding as 2 different (albeit related) things. > After reading this I'm thinking we actually will need a way to disable the disambiguated key mode. To me it sounds like [4] suggests modifyOtherKeys should always be set to 0 outside of a client application. foot only uses the CSI 27 encoding for keys that previously had no distinct encoding in the xterm protocol, so in most cases they shouldn't present a problem. <kbd>Enter</kbd> is the only one that seems to be an issue for some people (myself included) and only due to accidentally holding modifier keys. There are sequences that xterm emits, even with `modifyOtherKeys=0`, that can still cause "garbage" to be printed (Ctrl+arrows for example). That's just a general flaw with the way some apps handle input. > It would also be worth considering replacing the implementation with [6]. There's an open [issue](https://codeberg.org/dnkl/foot/issues/319) on the subject of supporting the kitty protocol, although so far I've managed to find less time to spend on it than I anticipated.
Owner

I'm somewhat hesistant to removing escapes, without having anything that replaces them. I also don't think that adding an option similar to modifyOtherKeys is the right way to go; like @craigbarnes says, it's only a few keys that are causing issues (Enter, and maybe Backspace and/or Delete?).

Once the kitty protocol has been implemented, it will be somewhat easier to remove some of the legacy escapes.

FWIW, I (too) accidentally hit Enter while holding modifiers (typically Shift).

I'm not aware of any applications that make use of Mods+Enter. I guess if we're all in agreement, we can remove the CSI 27 escapes for Enter. Should we remove all modifiers, or limit ourselves to Shift and Control?

I'm somewhat hesistant to removing escapes, without having anything that replaces them. I also don't think that adding an option similar to `modifyOtherKeys` is the right way to go; like @craigbarnes says, it's only a few keys that are causing issues (<kbd>Enter</kbd>, and maybe <kbd>Backspace</kbd> and/or <kbd>Delete</kbd>?). Once the kitty protocol has been implemented, it will be somewhat easier to remove some of the legacy escapes. FWIW, I (too) accidentally hit <kbd>Enter</kbd> while holding modifiers (typically <kbd>Shift</kbd>). I'm not aware of any applications that make use of <kbd>Mods+Enter</kbd>. I guess if we're all in agreement, we can remove the `CSI 27` escapes for <kbd>Enter</kbd>. Should we remove all modifiers, or limit ourselves to <kbd>Shift</kbd> and <kbd>Control</kbd>?

For those using fish, this might help:

File: ~/.config/fish/conf.d/foot.fish

# Shift-Return
bind \e\[27\;2\;13\~ accept-autosuggestion

# Control-Return
bind \e\[27\;5\;13\~ accept-autosuggestion execute

# Control-Shift-Return
bind \e\[27\;6\;13\~ execute

# Super-Return
bind \e\[27\;9\;13\~ execute

# Super-Shift-Return
bind \e\[27\;10\;13\~ execute

I've taken the liberty to illustrate how some of those combos can be used creatively, for instance, to accept the suggested completion and executing it immediately.

This is my first interaction on codeberg ever (which I found because of foot), and I wanted to say that I'm deeply impressed with foot. You have earned my respect @dnkl. Thanks for unleashing this fantastic piece of engineering on the world. Kudos, and thank you.

For those using `fish`, this might help: File: `~/.config/fish/conf.d/foot.fish` ``` console # Shift-Return bind \e\[27\;2\;13\~ accept-autosuggestion # Control-Return bind \e\[27\;5\;13\~ accept-autosuggestion execute # Control-Shift-Return bind \e\[27\;6\;13\~ execute # Super-Return bind \e\[27\;9\;13\~ execute # Super-Shift-Return bind \e\[27\;10\;13\~ execute ``` I've taken the liberty to illustrate how some of those combos can be used creatively, for instance, to accept the suggested completion and executing it immediately. This is my first interaction on codeberg ever (which I found because of *foot*), and I wanted to say that I'm deeply impressed with *foot*. You have earned my respect @dnkl. Thanks for unleashing this fantastic piece of engineering on the world. Kudos, and thank you.
Owner

@pancho thanks for sharing! Awesome to see someone making use of the extended escape codes :)

@pancho thanks for sharing! Awesome to see someone making use of the extended escape codes :)
Sign in to join this conversation.
No Milestone
No Assignees
5 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.