How can I stream data with useMutation?

[finom]

I work on a library that handles normal responses as well as streamed ones, represented as async iterable. I've implemented queryOptions with design similar to tRPC:

handler.queryOptions = (input, opts) => {
      const queryKey = handler.queryKey();
      return {
        queryFn: async ({ client }) => {
          const result = await handler(input);
          if (Symbol.asyncIterator in result) {
            void (async () => {
              for await (const chunk of result) {
                client.setQueryData(queryKey, (data: unknown[]) => [...data, chunk]);
              }
            })();

            return [];
          }

          return result;
        },
        queryKey,
        ...opts,
      };
    }

As you can see, queryFn it checks if the response has asyncIterator and performs client.setQueryData for each chunk.

It works great, but in real work, streamed data is going to come out from mutation. For example, if you submit a form that sends request to the server to retrieve streamed LLM completion.

How would I implement mutationFn then? It seems like there is no way to access the client.

handler.mutationOptions = (opts) => {
      return {
        mutationFn: async (input) => {
          const result = await handler(input);
          if (Symbol.asyncIterator in result) {
             // ?????
          }

          return result;
        },
        mutationKey: handler.mutationKey(),
        ...opts,
      };
    }

Thanks!

[bae080311]

Try using optimistic updates🚀

https://tanstack.com/query/v4/docs/framework/react/guides/optimistic-updates
This is the official documentation.

If you’re not sure, feel free to ask any questions.
I can provide more example code if needed.❤️

[finom]

@bae080311 thank you for the response. How can I get the contextual client at onSuccess or onMutate? The documentation shows how queryClient can be used when it's a separate variable.

[bae080311]

You’re right — mutationFn, onMutate, and onSuccess don’t receive the queryClient (or client) as an argument like queryFn does. However, you can still access it using the useQueryClient() hook from React Query.

use closure

[finom]

@bae080311 the idea of these queryOptions etc is to make these custom functions thinner. Currently, in order to implement these features, I need to import types only, but not the library. If I would do useQueryClient inside queryOptions, it completely destroys the idea behind this feature.

[TkDodo]

FYI, for queries, we now have a built-in abstraction:

• https://tanstack.com/query/v5/docs/reference/streamedQuery

if you submit a form that sends request to the server to retrieve streamed LLM completion.

I’m questioning if this needs to be a mutation. retrieving a response to a question isn’t really mutating something?

[finom]

@TkDodo I didn't know about streamedQuery, thank you.

Do you think I'm misinterpreting "query" and "mutation"?

As far as I understand the "query" supposed to request data on component render: you open a page and can see the content after a delay, like a list of users. In case if you want to get other users, you modify the query input and the key, passing new values.

As for "mutation" it performs API calls on user activity, for example, if you clicked something. Once you submit the form, the data comes back and mutates your UI. If I send a prompt to an LLM, it's not always updates the database, but it mutates the current state of the component, and that was initiated by the user, not by the component. Currently, I don't know how would I implement an AI text chat with queries only.

[TkDodo]

mutations don’t necessarily mean “user initiated”, but they often are. The problem with mutations is that they don’t share state like queries do, they aren’t really cached. If you want good state management, you have to use queries. Have a look at how I did the chat example with streamedQuery in the docs: https://tanstack.com/query/v5/docs/framework/react/examples/chat

[finom]

@TkDodo thank you for clarification. But still, for me it's unobvious why user-initiated action isn't a mutation and this part is quite confusing to me: function ChatMessage({ question }: .... At the example the question represents a state, but I think of this task as of an action like handleQuestion(question), that is better representable by mutation. Since react-query is quite an old and very popular library, that has integration at tRPC, there is definitely a strong idea behind this logic, but I'm struggling to catch it.

[chriszs]

I definitely think of it as a mutation, where the pre-existing chat history is the query, and you're mutating it by adding a message, and then receiving additional messages in response.