#1 Consult async closure

Closed
opened 1 month ago by minad · 19 comments
minad commented 1 month ago

Hi!

I just read your blog post consulting spotify! I didn't expect that people already start using the internal API for their purposes, but it is good to see! One small correction though - instead of collecting the candidates in the espotify--async-search closure you can pass them through to the async functions sitting below. consult--async-sink will then take care of collecting all candidates. The only action which should be handled by the espotify--async-search function is the input string action, which should then send its candidate below. The other actions should be passed through. Please let me know if you have any suggestions regarding the API or additional commands!

Daniel

Hi! I just read your blog post consulting spotify! I didn't expect that people already start using the internal API for their purposes, but it is good to see! One small correction though - instead of collecting the candidates in the espotify--async-search closure you can pass them through to the async functions sitting below. consult--async-sink will then take care of collecting all candidates. The only action which should be handled by the espotify--async-search function is the input string action, which should then send its candidate below. The other actions should be passed through. Please let me know if you have any suggestions regarding the API or additional commands! Daniel
jao commented 1 month ago
Poster
Owner

hi minad! i'm not sure i follow what you mean. collecting results from spotify is in itself an asynchronous operation: you pass a callback to url-get and process the results there... so i am not sure how i would "pass them through the async calls below". if getting the list of results were a synchronous operation, i think i'd know what you mean, but here i'm missing something. sorry for being thick :)

many thanks for reading and comenting, and thanks for consult. i think eventually something like consult--read would be nice to have as a public api, and the same for consult-read-async (in some form, i'm making the name up) somehow encapsulting the chain of functions (i saw you started doing something like that for shell commands, and was wondering how something similar for (asynchronous) elisp calls would look). i at least find it very natural to want to write my own consult-* functions :)

hi minad! i'm not sure i follow what you mean. collecting results from spotify is in itself an asynchronous operation: you pass a callback to url-get and process the results there... so i am not sure how i would "pass them through the async calls below". if getting the list of results were a synchronous operation, i think i'd know what you mean, but here i'm missing something. sorry for being thick :) many thanks for reading and comenting, and thanks for consult. i think eventually something like consult--read would be nice to have as a public api, and the same for consult-read-async (in some form, i'm making the name up) somehow encapsulting the chain of functions (i saw you started doing something like that for shell commands, and was wondering how something similar for (asynchronous) elisp calls would look). i at least find it very natural to want to write my own `consult-*` functions :)
minad commented 1 month ago
Poster

I mean the following:

  (defun espotify--async-search (next type filter)
    (lambda (action)
        (pcase action
          ((pred stringp)
           (espotify-search-all
            (lambda (x)
              (funcall next 'flush)
              (funcall next (mapcar 'espotify--format-item x)))
            action ;; when we receive a string as the action,
                   ;; it's the user input
            type
            filter))
          (_ (funcall next action)))))

Another improvement you might want to consider is to factor out the espotify--format-item function and instead use consult--async-map for that. This way you have everything nicely separated.

There are the following async functions in the pipeline:

  • async-sink: collects the candidates and refreshes
  • async-refresh-timer: refreshes, when candidates are pushed, throttles with a timer
  • async-map: transform candidates, for example espotify--format-item
  • async-filter: filter candidates
  • some source: for example the espotify--async-search, or consult--async-process
  • async-throttle: throttles the user input
  • async-split: splits the input string, one part for async, one part for filtering

You could also just write a single monolithic async function which implements the whole protocol. But then you don't get the nice reuse and separation of concerns.

I originally also had a async-indicator function which shows the * and ! symbols in the prompt. But I merged this into consult--async-process since it was too process specific.

For http access I also considered something like this: 7542ae1ac8

But keeping your espotify functions separate is also perfectly fine!

many thanks for reading and comenting, and thanks for consult. i think eventually something like consult--read would be nice to have as a public api, and the same for consult-read-async (in some form, i'm making the name up) somehow encapsulting the chain of functions (i saw you started doing something like that for shell commands, and was wondering how something similar for (asynchronous) elisp calls would look). i at least find it very natural to want to write my own consult-* functions :)

Yes, maybe at some point this could happen but it needs more time to stabilize. However I would also find it okay if people just start using the consult--read private API in dependent packages which would have to be updated in lock-step with consult.
From all the private functions in Consult, the consult--read function is rather stable since it is like an internal API.

I mean the following: ~~~ elisp (defun espotify--async-search (next type filter) (lambda (action) (pcase action ((pred stringp) (espotify-search-all (lambda (x) (funcall next 'flush) (funcall next (mapcar 'espotify--format-item x))) action ;; when we receive a string as the action, ;; it's the user input type filter)) (_ (funcall next action))))) ~~~ Another improvement you might want to consider is to factor out the espotify--format-item function and instead use consult--async-map for that. This way you have everything nicely separated. There are the following async functions in the pipeline: - async-sink: collects the candidates and refreshes - async-refresh-timer: refreshes, when candidates are pushed, throttles with a timer - async-map: transform candidates, for example espotify--format-item - async-filter: filter candidates - some source: for example the espotify--async-search, or consult--async-process - async-throttle: throttles the user input - async-split: splits the input string, one part for async, one part for filtering You could also just write a single monolithic async function which implements the whole protocol. But then you don't get the nice reuse and separation of concerns. I originally also had a async-indicator function which shows the * and ! symbols in the prompt. But I merged this into consult--async-process since it was too process specific. For http access I also considered something like this: https://github.com/minad/consult/commit/7542ae1ac842c3023759de877ec1e5be29972072 But keeping your espotify functions separate is also perfectly fine! > many thanks for reading and comenting, and thanks for consult. i think eventually something like consult--read would be nice to have as a public api, and the same for consult-read-async (in some form, i'm making the name up) somehow encapsulting the chain of functions (i saw you started doing something like that for shell commands, and was wondering how something similar for (asynchronous) elisp calls would look). i at least find it very natural to want to write my own consult-* functions :) Yes, maybe at some point this could happen but it needs more time to stabilize. However I would also find it okay if people just start using the consult--read private API in dependent packages which would have to be updated in lock-step with consult. From all the private functions in Consult, the consult--read function is rather stable since it is like an internal API.
jao commented 1 month ago
Poster
Owner

Ah, thanks! I was trying something similar, but was missing the flush call and seeing spurious entres as a result. Using consult-async--map makes things neater, yes.

I've pushed updates to the library following those suggestions. I agree the layered pipeline is far better than a monolithic solution.

I guess i'll have to write another blog posts using the valuable information you're sharing above...

Ah, thanks! I was trying something similar, but was missing the flush call and seeing spurious entres as a result. Using consult-async--map makes things neater, yes. I've pushed updates to the library following those suggestions. I agree the layered pipeline is far better than a monolithic solution. I guess i'll have to write another blog posts using the valuable information you're sharing above...
minad commented 1 month ago
Poster

Great! I am looking forward to another post!

Great! I am looking forward to another post!
minad commented 1 month ago
Poster

I just saw that you also added Embark actions! Omar will love this. This is a real nice show case of the Consult, Embark Marginalia package suite! Do you intend to package this up somehow?

I just saw that you also added Embark actions! Omar will love this. This is a real nice show case of the Consult, Embark Marginalia package suite! Do you intend to package this up somehow?
jao commented 1 month ago
Poster
Owner

Well, i've just been playing around for a couple of days, so no plans as of yet; but i'm liking what i see and how you guys think (Omar did also kindly comment on my videos post, like you, with very useful insight), so chances are the collection will grow!

Well, i've just been playing around for a couple of days, so no plans as of yet; but i'm liking what i see and how you guys think (Omar did also kindly comment on my videos post, like you, with very useful insight), so chances are the collection will grow!
jao commented 1 month ago
Poster
Owner

@minad i actually have a little doubt arising from those embark actions i just wrote, that i think responds to a pattern i'm likely to encounter again. it arises when we have a new consult function that uses a generator, like the spotify one, which retrieves a rich set of metadata (artist, album, duration, and so on). in that case, we want (iiuc) a consult-async--map function that transforms that metadata into a string that can be shown in the minibuffer (i'm currently using selectrum for that), but want also to keep the metadata around for the use of marginalia and embark (i understand here an alternative, perhaps better, is to perform that transformation in consult--read's :lookup, but the intention of that callback seems other to me). in order to keep that metadata available, my trick has been to store it as a text property of the string displayed in the minibuffer. for marginalia, that works well: it receives the selected candidate, extracts the text property and there we go. so i was going to do exactly the same with embark, but found a problem: iiuc, embark puts the candidate first in a (mini)buffer and then invokes the action's command on that minbuffer contents. the result seems to be that the text properties are lost in the process, so embarks actions don't have access to that metadata. my dirty trick here has been to use an embark transformer, which sees the candidate before it's injected in the minbuffer for the action, to cache the metadata in a global variable... which besides dirty is ugly. am i missing an alternative mechanism here?

@minad i actually have a little doubt arising from those embark actions i just wrote, that i think responds to a pattern i'm likely to encounter again. it arises when we have a new consult function that uses a generator, like the spotify one, which retrieves a rich set of metadata (artist, album, duration, and so on). in that case, we want (iiuc) a consult-async--map function that transforms that metadata into a string that can be shown in the minibuffer (i'm currently using selectrum for that), but want also to keep the metadata around for the use of marginalia and embark (i understand here an alternative, perhaps better, is to perform that transformation in consult--read's `:lookup`, but the intention of that callback seems other to me). in order to keep that metadata available, my trick has been to store it as a text property of the string displayed in the minibuffer. for marginalia, that works well: it receives the selected candidate, extracts the text property and there we go. so i was going to do exactly the same with embark, but found a problem: iiuc, embark puts the candidate first in a (mini)buffer and then invokes the action's command on that minbuffer contents. the result seems to be that the text properties are lost in the process, so embarks actions don't have access to that metadata. my dirty trick here has been to use an embark transformer, which sees the candidate before it's injected in the minbuffer for the action, to cache the metadata in a global variable... which besides dirty is ugly. am i missing an alternative mechanism here?
minad commented 1 month ago
Poster

The lookup function you are using is similar to consult--lookup-elem. You can use that instead, I think.

Unfortunately text properties are not available to Embark actions. They are also lost by completing-read. Therefore you also need this consult--lookup-elem step to recover them. Basically Embark actions should be seen as normal general-purpose commands which read from the minibuffer. Due to that you can reuse arbitrary commands as actions! Unfortunately, the input these commands read from the minibuffer obviously does not have properties! This is a natural limitation of how things are glued together.

If you want a clean design you could make all the actions normal commands which select from the spotify candidates via async completing read. And then my proposal would be to introduce some espotify cache layer which ensures that no unnecessary web access happens for the actions! This is essentially the same what you are doing with the global variable but maybe dressed up in a slightly nicer way!

To summarize, the rules regarding metadata are the following:

  • If you have access to the candidate/candidate list directly, you obviously also have access to the metadata, this applies to async-map, to marginalia and to Embark exporters (see for example the exporter of consult-location category commands to occur-mode buffers!)
  • If you are accessing the return value of completing-read, there is no metadata! Therefore we need :lookup to regain the metadata. Embark actions receive their input as return value of a minibuffer read, therefore they don't have access to the metadata.
The lookup function you are using is similar to consult--lookup-elem. You can use that instead, I think. Unfortunately text properties are not available to Embark actions. They are also lost by completing-read. Therefore you also need this consult--lookup-elem step to recover them. Basically Embark actions should be seen as normal general-purpose commands which read from the minibuffer. Due to that you can reuse arbitrary commands as actions! Unfortunately, the input these commands read from the minibuffer obviously does not have properties! This is a natural limitation of how things are glued together. If you want a clean design you could make all the actions normal commands which select from the spotify candidates via async completing read. And then my proposal would be to introduce some espotify cache layer which ensures that no unnecessary web access happens for the actions! This is essentially the same what you are doing with the global variable but maybe dressed up in a slightly nicer way! To summarize, the rules regarding metadata are the following: * If you have access to the candidate/candidate list directly, you obviously also have access to the metadata, this applies to async-map, to marginalia and to Embark exporters (see for example the exporter of consult-location category commands to occur-mode buffers!) * If you are accessing the return value of completing-read, there is no metadata! Therefore we need :lookup to regain the metadata. Embark actions receive their input as return value of a minibuffer read, therefore they don't have access to the metadata.
oantolin commented 1 month ago
Poster

The current status of using text properties in embark targets is:

  • It works in Selectrum! And you don't need the transformer trick (@jao called this his "dirty trick"). When I insert the target into the minibuffer I do not strip the properties, so if you Selectrum, normal commands prompting in the normal way, say using an interactive declaration, do get access to text properties.

  • It does not work with default completion or with icomplete. The culprit is the function embark-target-top-completion, which is used to get the current candidate. That function does not preserve text properties. Maybe I can fix that function, but I don't know how yet.

The current status of using text properties in embark targets is: - It works in Selectrum! And you don't need the transformer trick (@jao called this his "dirty trick"). When I insert the target into the minibuffer I do *not* strip the properties, so if you Selectrum, normal commands prompting in the normal way, say using an `interactive` declaration, do get access to text properties. - It does not work with default completion or with icomplete. The culprit is the function `embark-target-top-completion`, which is used to get the current candidate. That function does not preserve text properties. Maybe I can fix that function, but I don't know how yet.
minad commented 1 month ago
Poster

It works in Selectrum! And you don't need the transformer trick (@jao called this his "dirty trick"). When I insert the target into the minibuffer I do not strip the properties, so if you Selectrum, normal commands prompting in the normal way, say using an interactive declaration, do get access to text properties.

This sounds like a hack to me. It won't work if the command does a completing-read in the interactive prologue, since completing-read strips the properties. I would rather strip the properties too in any case before insertion. I think it is better to not rely on the availability of properties.

> It works in Selectrum! And you don't need the transformer trick (@jao called this his "dirty trick"). When I insert the target into the minibuffer I do not strip the properties, so if you Selectrum, normal commands prompting in the normal way, say using an interactive declaration, do get access to text properties. This sounds like a hack to me. It won't work if the command does a completing-read in the interactive prologue, since completing-read strips the properties. I would rather strip the properties too in any case before insertion. I think it is better to not rely on the availability of properties.
oantolin commented 1 month ago
Poster

@minad is right, completing-read is the problem. I used to think that completing-read and embark-target-top-minibuffer completions were separate obstacles to using text properties on candidates, but no, completing-read is the only culprit.

I just carefully checked that embark-target-top-completion does preserve the text properties, and that the text properties are still present when the target is injected into the minibuffer. But then when the action reads the target with completing-read (and using interactive with a string argument does use completing-read), that's when the properties are stripped.

All of which means that using text properties on the candidates is definitely out. I'm going to have to recommend either encoding all the information you need in the candidate string and parsing it out again (consult-buffer uses this approach), or what @minad recommended: storing the metadata externally in a hashtable or alist and looking it up whenever you need it.

If you decide to put all the metadata in the string, you can make it invisible or make it display however you want. Probably storing the metadata externally is a little simpler than serializing and parsing over and over (which is the Unix way, I guess: always use text, always reparse).

@minad is right, completing-read is the problem. I used to think that `completing-read` and `embark-target-top-minibuffer` completions were separate obstacles to using text properties on candidates, but no, `completing-read` is the only culprit. I just carefully checked that `embark-target-top-completion` *does* preserve the text properties, and that the text properties are *still present* when the target is injected into the minibuffer. But then when the action reads the target with `completing-read` (and using `interactive` with a string argument does use `completing-read`), that's when the properties are stripped. All of which means that using text properties on the candidates is definitely out. I'm going to have to recommend either encoding all the information you need in the candidate string and parsing it out again (`consult-buffer` uses this approach), or what @minad recommended: storing the metadata externally in a hashtable or alist and looking it up whenever you need it. If you decide to put all the metadata in the string, you can make it invisible or make it display however you want. Probably storing the metadata externally is a little simpler than serializing and parsing over and over (which is the Unix way, I guess: always use text, always reparse).
minad commented 1 month ago
Poster

All of which means that using text properties on the candidates is definitely out. I'm going to have to recommend either encoding all the information you need in the candidate string and parsing it out again (consult-buffer uses this approach), or what @minad recommended: storing the metadata externally in a hashtable or alist and looking it up whenever you need it.

It would probably be good to document this somewhere, also the other things that have been written in this thread here regarding the async API usage!

If you decide to put all the metadata in the string, you can make it invisible or make it display however you want. Probably storing the metadata externally is a little simpler than serializing and parsing over and over (which is the Unix way, I guess: always use text, always reparse).

Generally I advise against serializing everything inside the string. I only recommend doing this when you necessarily need some hidden string prefix in order to make the candidates unique with respect to string=. Note that properties cannot be used to disambiguate candidates, since two different candidates with the same string representation but different properties are treated as equal by string=.

> All of which means that using text properties on the candidates is definitely out. I'm going to have to recommend either encoding all the information you need in the candidate string and parsing it out again (consult-buffer uses this approach), or what @minad recommended: storing the metadata externally in a hashtable or alist and looking it up whenever you need it. It would probably be good to document this somewhere, also the other things that have been written in this thread here regarding the async API usage! > If you decide to put all the metadata in the string, you can make it invisible or make it display however you want. Probably storing the metadata externally is a little simpler than serializing and parsing over and over (which is the Unix way, I guess: always use text, always reparse). Generally I advise against serializing everything inside the string. I only recommend doing this when you necessarily need some hidden string prefix in order to make the candidates unique with respect to string=. Note that properties cannot be used to disambiguate candidates, since two different candidates with the same string representation but different properties are treated as equal by string=.
jao commented 1 month ago
Poster
Owner

If you decide to put all the metadata in the string, you can make it invisible or make it display however you want. Probably storing the metadata externally is a little simpler than serializing and parsing over and over (which is the Unix way, I guess: always use text, always reparse).

Generally I advise against serializing everything inside the string. I only recommend doing this when you necessarily need some hidden string prefix in order to make the candidates unique with respect to string=. Note that properties cannot be used to disambiguate candidates, since two different candidates with the same string representation but different properties are treated as equal by string=.

I agree. Unix philosophy or not, transforming a rich set of metadata we've already parsed (in the spotify example, from JSON obtained from a remote source) to a string and then reparse it every time, in different contexts, is something that i, personally, wouldn't like to do. Specially with the example of Marginalia at hand, where that is not needed at all. I have already in mind other scenarios where i'll need this dance (e.g., a consult-notmuch, where the generator of results is a query to notmuch, whose result needs to be parsed for presentation purposes, for adding metadata in marginalia and also for defining different embark actions (open single message, open thread, etc.)), so it seems i'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark.

> > If you decide to put all the metadata in the string, you can make it invisible or make it display however you want. Probably storing the metadata externally is a little simpler than serializing and parsing over and over (which is the Unix way, I guess: always use text, always reparse). > > Generally I advise against serializing everything inside the string. I only recommend doing this when you necessarily need some hidden string prefix in order to make the candidates unique with respect to string=. Note that properties cannot be used to disambiguate candidates, since two different candidates with the same string representation but different properties are treated as equal by string=. I agree. Unix philosophy or not, transforming a rich set of metadata we've already parsed (in the spotify example, from JSON obtained from a remote source) to a string and then reparse it every time, in different contexts, is something that i, personally, wouldn't like to do. Specially with the example of Marginalia at hand, where that is not needed at all. I have already in mind other scenarios where i'll need this dance (e.g., a consult-notmuch, where the generator of results is a query to notmuch, whose result needs to be parsed for presentation purposes, for adding metadata in marginalia and also for defining different embark actions (open single message, open thread, etc.)), so it seems i'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark.
oantolin commented 1 month ago
Poster

Just to be clear, I was making fun of Unix for its constant reparsing. In a richer environment like Emacs I don't recommend it.

Just to be clear, I was making fun of Unix for its constant reparsing. In a richer environment like Emacs I don't recommend it.
minad commented 1 month ago
Poster

so it seems i'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark.

@jao What is wrong in regenerating the candidates every time? And then putting a caching layer into the candidate generation step. This is what my preferred approach would be. Then the commands would stay reusable and you could start them independently of Embark.

Just to be clear, I was making fun of Unix for its constant reparsing. In a richer environment like Emacs I don't recommend it.

@oantolin Haha, I am also not a fan of doing the reparsing every time. But on the shell level I am not sure if any of the "better" alternatives which operate on richer data will take over at some point. I am aware of some rust shell which attempts this, powershell exchanges some objects and probably eshell also allows passing richer data. It all starts with the problem that everyone has to agree on a protocol. The simpler the better it seems.

> so it seems i'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark. @jao What is wrong in regenerating the candidates every time? And then putting a caching layer into the candidate generation step. This is what my preferred approach would be. Then the commands would stay reusable and you could start them independently of Embark. > Just to be clear, I was making fun of Unix for its constant reparsing. In a richer environment like Emacs I don't recommend it. @oantolin Haha, I am also not a fan of doing the reparsing every time. But on the shell level I am not sure if any of the "better" alternatives which operate on richer data will take over at some point. I am aware of some rust shell which attempts this, powershell exchanges some objects and probably eshell also allows passing richer data. It all starts with the problem that everyone has to agree on a protocol. The simpler the better it seems.
jao commented 1 month ago
Poster
Owner

so it seems i'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark.

@jao What is wrong in regenerating the candidates every time? And then putting a caching layer into the candidate generation step. This is what my preferred approach would be. Then the commands would stay reusable and you could start them independently of Embark.

i am not sure i understand what you mean. in the scenario i have in mind, i've created the list of candidates and their metadata via an async, the minibuffer has displayed it, marginalia and selectrum/icomplete/embark-collect have done their job (all of them seing the rich metadata), and the user selects a candidate and calls embark-act on it, still in the minibuffer. my problem is that what i receive in the action command is the candidate's string stripped of its metadata. at that point, what would "regenerating the candidates" mean? i am sure you don't mean to go back to the async, re-query, re-select my candidate and get its metadata (even if i've cached the query results in the async, not its job), so i was imagining just caching the metadata a moment before the embark action is called (which is doable for instance using an embark transformer), and no re-parsing is need at all. and since that is an action fully under embark's control, i was also thinking that that mechanism to preserve metadata could be built-in embark (alternatively, i can of course just pre-advice embark-act to do it). either way, it seems better to me to cache the already parsed metadata, and a bit unsatisfactory that marginalia, selectrum and consult don't need that cache at all but their friend embark does :)

is that also what you mean?

> > so it seems i'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark. > > @jao What is wrong in regenerating the candidates every time? And then putting a caching layer into the candidate generation step. This is what my preferred approach would be. Then the commands would stay reusable and you could start them independently of Embark. i am not sure i understand what you mean. in the scenario i have in mind, i've created the list of candidates and their metadata via an async, the minibuffer has displayed it, marginalia and selectrum/icomplete/embark-collect have done their job (all of them seing the rich metadata), and the user selects a candidate and calls embark-act on it, still in the minibuffer. my problem is that what i receive in the action command is the candidate's string stripped of its metadata. at that point, what would "regenerating the candidates" mean? i am sure you don't mean to go back to the async, re-query, re-select my candidate and get its metadata (even if i've cached the query results in the async, not its job), so i was imagining just caching the metadata a moment before the embark action is called (which is doable for instance using an embark transformer), and no re-parsing is need at all. and since that is an action fully under embark's control, i was also thinking that that mechanism to preserve metadata could be built-in embark (alternatively, i can of course just pre-advice embark-act to do it). either way, it seems better to me to cache the already parsed metadata, and a bit unsatisfactory that marginalia, selectrum and consult don't need that cache at all but their friend embark does :) is that also what you mean?
oantolin commented 1 month ago
Poster

I'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark.

@jao Given that Embark parameter passing mechanism is injection at the minibuffer prompt, target are forced to be just plain strings. So then the only mechanisms for this I can imagine would involve calling a function that retrieves the metadata, inside your action command. At that point, you might as well have the action command call a function to regenerate the metadata (possibly with caching) or do a hashtable lookup, etc.

But on the shell level I am not sure if any of the "better" alternatives which operate on richer data will take over at some point.

@minad Hasn't powershell already taken over on Windows?

> I'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark. @jao Given that Embark parameter passing mechanism is injection at the minibuffer prompt, target are forced to be just plain strings. So then the only mechanisms for this I can imagine would involve calling a function that retrieves the metadata, inside your action command. At that point, you might as well have the action command call a function to regenerate the metadata (possibly with caching) or do a hashtable lookup, etc. > But on the shell level I am not sure if any of the "better" alternatives which operate on richer data will take over at some point. @minad Hasn't powershell already taken over on Windows?
jao commented 1 month ago
Poster
Owner

I'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark.

@jao Given that Embark parameter passing mechanism is injection at the minibuffer prompt, target are forced to be just plain strings. So then the only mechanisms for this I can imagine would involve calling a function that retrieves the metadata, inside your action command. At that point, you might as well have the action command call a function to regenerate the metadata (possibly with caching) or do a hashtable lookup, etc.

i see. for the spotify example, regenerating the metadata from the string alone is cumbersome (the latter is going to be just, say, a track's title, and one wants in the action to play its album; or simply to display all available info on that track, which was parsed from json when the candidates were generated), so the natural way is then to cache the metadata itself. and, in fact, for the action at hand, one doesn't even need a hashtable, it's just a variable holding the current selection's metadata.

this is exactly what i'm doing right now in the espotify mini-lib here, so my original question seems answered (thanks for being patient, both @minad and @oantolin !) in that it is a reasonable approach.

i still think that adding to embark a simple API (as you already suggest) with just two functions (say, embark-set-current-selection-metadata and embark-get-current-selection-metadata, implemented in terms of an internal "cache" global embark--current-selection-metadata) is granted (based on the fact that 2 out of the 3 first examples of actions i've wanted to implement would use it), but i'll stop nagging about it now :) (it's very easy for me to have it).

> > I'll end up coding a generic "embark metadata store" of some sort. I think it'd be nice to provide such a mechanism out of the box in embark. > > @jao Given that Embark parameter passing mechanism is injection at the minibuffer prompt, target are forced to be just plain strings. So then the only mechanisms for this I can imagine would involve calling a function that retrieves the metadata, inside your action command. At that point, you might as well have the action command call a function to regenerate the metadata (possibly with caching) or do a hashtable lookup, etc. i see. for the spotify example, regenerating the metadata from the string alone is cumbersome (the latter is going to be just, say, a track's title, and one wants in the action to play its album; or simply to display all available info on that track, which was parsed from json when the candidates were generated), so the natural way is then to cache the metadata itself. and, in fact, for the action at hand, one doesn't even need a hashtable, it's just a variable holding the current selection's metadata. this is exactly what i'm doing right now in the espotify mini-lib here, so my original question seems answered (thanks for being patient, both @minad and @oantolin !) in that it is a reasonable approach. i still think that adding to embark a simple API (as you already suggest) with just two functions (say, `embark-set-current-selection-metadata` and `embark-get-current-selection-metadata`, implemented in terms of an internal "cache" global `embark--current-selection-metadata`) is granted (based on the fact that 2 out of the 3 first examples of actions i've wanted to implement would use it), but i'll stop nagging about it now :) (it's very easy for me to have it).
jao commented 2 weeks ago
Poster
Owner

i've put everything together in its own repo (https://codeberg.org/jao/espotify), as a literate doc that generates the different elisp lib. everything is now, i think, quite smoother, thanks for your comments!

i've put everything together in its own repo (https://codeberg.org/jao/espotify), as a literate doc that generates the different elisp lib. everything is now, i think, quite smoother, thanks for your comments!
jao closed this issue 2 weeks ago
Sign in to join this conversation.
No Label
No Milestone
No Assignees
3 Participants
Notifications
Due Date

No due date set.

Loading…
There is no content yet.