React Streaming SSR/Server Components

[Ephem]

Improve React Query hydration, Streaming SSR & Server Components

Update 2, TLDR: This post tries to outline a rough early draft API for integrating React Query with all the new React APIs while preserving a familiar API and trying to minimize the amount of new things RQ users needs to learn to leverage them, as well as outlining some todos to get there.

Update: This a pretty complex and advanced subject where a lot of things are still changing/undecided and is as of yet undocumented and I didn't include a lot of "why's" or explanations in this post to keep it manageable. I tried to include resources that provides some context, and please ask and comment on anything and I'll try to expand on it if I can!

For everything below, we also need to figure out a holistic API that makes sense, that we can build on in the future and with a solid migration path. Hopefully(/probably) APIs won't change too much/be incrementally adoptable, I think most work happens in the background, but that there will be new patterns to learn for users that want to leverage new features.

This is not (yet) an API design document, but I've added some examples of my current thinking at the bottom.

Note that I think a big goal of Query is to have a framework agnostic core to be able to support other frameworks than React. It already has a great custom cache, so the plan is to keep building on that and not really rely on the upcoming React cache APIs except where we have to, for specific things like request scoped things, the fetch-deduplication etc. This might be subject to change of course.

The Suspense SSR Architecture post as well as this post (esp the header "Hydration Data") in the React 18 WG has some great context to this subject, as well as this comment.

Note: If you have an itch to play around/experiment with RSCs and RQ early, there is a WIP PR by @efilion that documents what is currently possible.

Note: If you want to read some earlier thinking on the subject, this discussion has some history, but is no longer current/up to date.

General stuff

• Add 'use client' in the correct places (minor?)
• Refactor Suspense to use() instead of throwing Promises (minor/major?)
  • Can we/do we want to keep the old behaviour for older versions of React?
• useSyncExternalStore should have a correct/frozen getServerSnapshot (minor)
• Hydration implementation details (major)
  • Step 1: Should still happen in render for new queries (and queries with no observers?), but be scheduled for a useEffect for queries that someone is actively listening to since this results in re-renders/observable side effects
  • Step 2 (optional): When hydrating in useEffect also add the new data to a context for that subtree, refactor all reads to patch in data from that context. That subtree safely gets the new data during the same render instead of after, "true" cache and thus anything listening outside of that subtree gets updated via the useEffect (This might be overkill, we should evaluate if step 1 is good enough first)
• Suggestion: Rename <Hydrate> to <HydrationBoundary> (? open to other suggestions, and we can deprecate and keep the current one for migration purposes), for three reasons:
  • It's a descriptive rather than imperative name
  • If we implement Step 2 above, the name better explains why there are two different behaviours for in-tree and out-tree
  • If we can support useQuery(..., { suspense: true }) with SSR Streaming (see below), we might need to support some kind of "automatic, in the background de/rehydration" tied to a specific position in the tree. Also, matching a Suspense(Boundary), ErrorBoundary and HydrationBoundary together feels like it makes sense?
  • This said, we might want to hold off on this until we know more
• Figure out what to do with the imperative hydrate() (should stay for other frameworks) and the hook useHydrate (deprecate/remove?)

Server Side Rendering

• Figure out/document if there is a way for useQuery(..., { suspense: true }) to work out of the box as long as you buffer all the content instead of streaming? Pretty sure this is possible in custom setups, but not sure about metaframeworks.
• Figure out if there will be a way for useQuery(..., { suspense: true }) to work with streaming SSR (without await prefetch() that same query before rendering). Missing primitives in React? (Linked question in the RSC WG: #3)
  • If this is not supported by React out of the box, is there a way to implement this at the library level as an out of band channel or something? (Probably not, at least without meta-framework support, maybe this is a (meta-)framework level concern to start with)

Server Components

• Implement support for de/rehydrating Promises. This will mean rendering can start before all data has loaded (minor/major?)
  • Promise de/rehydration is not yet implemented in React, so we are blocked on this until then
  • This will likely be in the core, how does it affect other frameworks? Are there ways for other frameworks to leverage this that we should consider when designing it?
• Figure out what API we want in Server Components. Is a queryClient even necessary here, or should we implement a lightweight prefetch() API instead?
  • If we still want/need a queryClient to avoid duplication etc, we probably want to use the per request cache() API that is for now only documented in the Next beta docs. Not sure that this is necessary though, especially as a first step since it probably adds a bunch of complexity.

API Examples

This is not thought through or fleshed out yet, just early spitballing to start a discussion. It all depends a lot on what final low level APIs will be available from React(/the metaframeworks) and what they will look like, but I consider it a decent guess based on what we know.

Basic hydration

No Suspense, no streaming, no RSCs, just what's always been available.

import { prefetchQuery, HydrationBoundary } from '@tanstack/react-query';

// Will work both for initial SSR and for page navigations
// (page1->page2) where loaders are still executed serverside
// This could be a custom framework, Remix loaders,
// Next getServerSideProps or whatever
export async function loader() {
  // Note that we have no queryClient here, it's still supported, for
  // example for the client side case, but shouldn't be necessary here
  const dehydratedQuery = await prefetchQuery(...);

  // Could support prefetchQueries() too,
  // or HydrationBoundary could support either a single query,
  // or an array of queries, that way you could just do Promise.all()
  // and hydrate the result
  // Key here is that if you await all Promises here, no Promise
  // de/rehydration happens, the dehydrated state is pure JSON
  // We might want some way to guarantee this?
  
  // We are still expecting the framework to handle actual serialization here
  return { props: { dehydratedQuery }};
}

function PageComponent({ dehydratedQuery, children }) {
  return (
    <HydrationBoundary state={dehydratedQuery}>
      {children}
    </HydrationBoundary>
  )
}

// Deeper in the tree
function SomeComponent() {
  const { data } = useQuery(...);
  // or, but guaranteed to never suspend if await prefetchQuery
  // has always been called:
  const { data } = useQuery(..., { suspense: true });

  // ...
}

Streaming SSR without prefetching

This probably needs primitives from React we don't have or know anything about yet, but those should(?) be transparent to the user, so we can still imagine an API.

// Higher in the tree
function Page({ children }) {
  return (
    // Note: No state, in this case it just provides some
    // behind the scenes setup, might not be necessary
    <Suspense fallback="Loading...">
      <HydrationBoundary>{children}</HydrationBoundary>
    </Suspense>
  );
}

// Deeper in the tree
function SomeComponent() {
  // This will de/rehydrate the Promise/data in the background
  const { data } = useQuery(..., { suspense: true });

  // ...
}

Optionally, the Page component above could start a prefetchQuery, but not await it, just to kick off loading earlier. Some details to work out around that case (does it need to be queryClient.prefetchQuery in this case, or can we rely on the fetch cache here as well as in RSCs, should/can that prefetch happen in a loader function before rendering, or in render (since it's an idempotent side effect), etc?)

Server Components

export default async ServerComponent({children}) {
  // Note that in this case, we are not awaiting the
  // prefetch, so the thing that is inside the dehydratedQuery
  // is a Promise that is passed down to the "client", automatically
  // resolved when data comes in etc
  const dehydratedQuery = prefetchQuery(...);

  // Just do illustrate:
  const dehydratedQuery2 = prefetchQuery(...);

  return (
    <Suspense fallback="Loading...">
      <HydrationBoundary state={[dehydratedQuery, dehydratedQuery2]}>
        {children}
      </HydrationBoundary>
    </Suspense>
  );
}

// Deeper client component
function SomeComponent() {
  // Since we've hydrated an active Promise here, it will
  // suspend without a new clientside fetch and the Suspense
  // fallback will render. This will happen both during the
  // server rendering and in the browser.
  // When the Promise on the server resolves, React will take
  // care of resolving and rerendering correctly (hopefully, I think)
  const { data } = useQuery(..., { suspense: true });

  // ...
}

What did I miss?

This is somewhat of a braindump on my current thinking, it's not well researched or thought through in all aspects, so take it as a starting point. Please challenge everything you can think of, tell me where I'm wrong, ask questions on anything that is unclear and join in the discussion!

[TkDodo]

the Server Components example looks pretty cool. If we could make it work that way, it would make passing props down to a client component unnecessary 👍

Add 'use client' in the correct places (minor?)

yes this should be doable as a minor

Refactor Suspense to use() instead of throwing Promises (minor/major?)
Can we/do we want to keep the old behaviour for older versions of React?

we can do this as soon as use becomes non-experimental I guess ? With v5, we will drop support for react < 18, so we should be good once use makes it to 18. This could pan out nicely with our v5 timeline.

Suggestion: Rename to

like it 👍

Figure out what to do with the imperative hydrate() (should stay for other frameworks) and the hook useHydrate (deprecate/remove?)

my first instinct is: hydrate should stay (query-core) because we also use it for other things like persisters. useHydrate can go.

---

useQuery(..., { suspense: true })

one bigger question from my side would be: Do we want a dedicated useSuspenseQuery (and useSuspenseInfiniteQuery hook instead of the suspense:true boolean?

This would be beneficial for TypeScript users because right now, data is always T | undefined (and not T). We could change types for those hooks, and also e.g. disallow the enabled option to streamline the types with what people want mostly when using suspense: not to check for undefined data 😅

[Ephem]

it would make passing props down to a client component unnecessary

Yeah, I guess another way of saying it would be that the HydrationBoundary is the client component so to speak, and then that can render either more server components (works since they are passed in from above), or other client components depending on what is passed in as children.

This also reminds me, one thing I didn't cover in the post above is the case where you want to do something like this in a Server Component, and use the data immediately:

async function ServerComponent() {
  const { data } = useQuery(...);
}

What should the API for that look like? If we support useQuery that relies on a queryClient currently which would be nice to avoid in RSCs if we can (at least to start with), but having another API could feel cumbersome. Something to figure out.

With v5, we will drop support for react < 18, so we should be good once use makes it to 18. This could pan out nicely with our v5 timeline.

Lets hope! 👍 I guess this would mean we would require React 18.X, where X is the version where use was introduced though?

useSuspenseQuery

Agreed. I was considering including that here, but it wasn't critical for any of the other things and I knew there were motivations I had forgotten about for changing the name. I like that this gives us the opportunity to tweak the API for things like enabled, we should think about if there are anything else we want to change at the same time I guess? The possible loading states obviously changes, but do we want to make any changes to error handling for example?

Should we remove the suspense: true-option in v5 in that case? How painful a migration would this be for people that have configured this at a global level for their apps? (Probably not very as long as we don't change too much, codemod perhaps?)

[Ephem]

About the case:

async function ServerComponent() {
  const { data } = useQuery(...);
}

The problem here is that since this is a Server Component, it wouldn't rerender if we refetch fresher data on the client, any markup we rendered in the Server Component would still be stale. As a library, I'm not sure we can revalidate Server Components as it stands now, since this is more a router level concern? If something in the key of that useQuery also revalidated the Server Component though, it would probably work. A revalidates both B (SC) and C (the clientside query cache) so to speak.

But whatever we do at a library level here, useQuery in an SC would never mean/work the same as in a client component, so might make sense to design the API around that and have something else instead. Possibly just const { data } = await prefetchQuery(...) and use the data as is. I think this is the case I've thought least about so far.

[TkDodo]

HydrationBoundary is the client component

yes, absolutely the right way to think about it 👍

where you want to do something like this in a Server Component

in a server component, you would just do:

const data = queryClient.fetchQuery({ queryKey, queryFn })

instead of useQuery

I like that this gives us the opportunity to tweak the API for things like enabled, we should think about if there are anything else we want to change at the same time I guess? The possible loading states obviously changes, but do we want to make any changes to error handling for example?

It would default to using errorBoundaries, but you can still change that per query. I think that's fine.

How painful a migration would this be for people that have configured this at a global level for their apps?

suspense: true was clearly marked as experimental, so I don't think the migration strategy should play too much of a role. What I like about the separation is that we can move a bunch of suspense only code to the separate function, which can be tree-shaked if you don't use suspense.

There is an edge case where if you cancel a query while it is fetching, it goes to loading state with undefined data. I don't know what would happen here with suspense though - it probably would just suspend again?

[Ephem]

const data = queryClient.fetchQuery({ queryKey, queryFn })

Oh yeah, I should have written this instead of the hook form of course. 😅 Maybe I'm fighting too hard against having a queryClient in Server Components, I do have a few arguments, but I think mostly it's just a feeling that avoiding stateful abstractions shared between Server Components might be a good thing if we can, but I might very well be wrong. Definitely something to think more on and figure out.

[TkDodo]

I think you would just create a new QueryClient where you need to - I also wouldn't share a client on the server

[amannn]

This is really interesting, thanks for sharing! I'd be curious if you have ideas how existing APIs that rely on React Context will work on the server side (e.g. QueryClientProvider)? RSC don't support React Context at this point.

I just saw the new SSR guide where you're using the cache() API for this use case. So I guess you could use getQueryClient instead of useQuery in RSC's, but it's a bit unfortunate that you have to use different APIs depending on the environment, no?

[amannn]

FYI, createServerContext is available in the latest version of React (although the TypeScript types are not there yet):

import {createServerContext} from 'react';

const Context = createServerContext(globalName: string, value: any);

The value is transmitted for every request between the server and client, so it needs to be really small and also the value needs to be serializable. So I think putting a queryClient there might not work. From my current understanding, using cache() to create a request-scoped QueryClient might be the better choice.

For reference, I'm working on the same problem currently for next-intl, an i18n library of Next.js I'm maintaining.

[Ephem]

Yeah, createServerContext is interesting, I'm not quite sure how it fits into the big picture yet. Maybe one interesting avenue to explore would be to find a minimal format for keeping track of which queries the client already has fresh data for, so the server can avoid prefetching and de/rehydrating those when not necessary?

[TkDodo]

I think one case would be to provide a queryClient so that you can prefetch wherever you want?

[cyco130]

I think that's exactly what you can't do. The value has to be serializable.

[TkDodo]

Oh 🤦‍♂️ I totally misread that. I thought the context would be request scoped

[cyco130]

Hi,

I'm the author of Rakkas, a Vite-based React metaframework with some unique cutting edge features. It doesn't support RSC yet but it has supported streaming SSR with Suspense for quite sometime now (before Next.js did).

I have a Rakkas-Tanstack Query integration example repo in which I devised a scheme to make Tanstack Query work with streaming SSR. I thought it might be of interest in this discussion. I just added some documentation on how it works in the readme but, basically, instead of relying on Tanstack Query's own deydration/hydration features, I handle the serialization myself: Every time a query is resolved on the server, I inject a script into React's HTML stream that calls queryClient.setQueryData with the serialized query data. I haven't done extensive testing but it seems to work well. The implementation uses some Rakkas specific features but the emitBeforeSsrChunk can be implemented in any framework that supports streaming SSR by wrapping the SSR stream and injecting something before each chunk.

It may also be of interest that Rakkas has its own implementation of useQuery and some other Tanstack Query APIs. This implementation is Suspense-only and supports only a subset of the features. It's used under the hood for Rakkas's unique inline RPC mechanism useServerSideQuery. Rakkas's implementation doesn't have the concept of hydration/dehydration and it uses a system similar to the above.

Lastly, you may want to check out @brillout's react-streaming, a library to simplify the setup for implementing React streaming SSR. It comes with a useAsync hook which is similar to useQuery in many ways. It uses a mechanism very similar to the one I used to send server-side query data to the client.

[Ephem]

Exactly. And this is actually happening, see reactjs/rfcs#219, most notably sebmarkbage's comment Therefore our focus has been on adding higher level replacements for each use case..

Thanks for reminding me about this one, I guess the big questions are0 not if, but when they will get around to the data fetching usecase and what it will look like.

Ideally, there would be a 100% reliable way to detect waterfalls, but I can't think of a way that don't invovle some deep integration with React.

This would be nice! I think heuristics might do fine here though. Something slightly different that would be easier to implement would be to help find queries that were not preloaded. This covers less usecases, but might also be helpful.

[brillout]

when they will get around to the data fetching usecase

If the new use() hook allows to the user to determine the cache lifecycle then it'll do the trick. Although I've doubts that such option will be available.

[brillout]

Actually, it will support cache control: https://github.com/acdlite/rfcs/blob/first-class-promises/text/0000-first-class-support-for-promises.md#caveat-data-requests-must-be-cached-between-replays.

Neat, so I think use() will completely replace react-streaming. Neat neat.

[artem-malko]

Hm, @brillout could you explain, how use() can replace react-streaming?

[Ephem]

I think use() only deals with consuming for example a fetch that was started on the server, since it can take a Promise and read if resolved or suspend if not. It doesn't really deal with how to pass that Promise from the server to the client though, but that is mentioned here: https://github.com/acdlite/rfcs/blob/first-class-promises/text/0000-first-class-support-for-promises.md#passing-a-promise-from-a-server-component-to-a-client-component

That's just for passing a Promise from a Server Component though, the thing I'm still unclear on is passing directly from the "normal" SSR pass to the client, for any query that was not prefetched in an SC, but instead discovered during the rendering pass.

[cadoopmin029]

To whom this may concern, I do not know why I am receiving all of these notifications. I did not subscribe to follow or receive anything. Can someone advise as to how this may have happened? Thanks

…

On Mon, Jan 23, 2023 at 10:04 AM Artem Malko ***@***.***> wrote: Hm, @brillout <https://github.com/brillout> could you explain, how use() can replace react-streaming? — Reply to this email directly, view it on GitHub <#4623 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASZ5GK7EUP4U73E6JWJCL6LWT2MYZANCNFSM6AAAAAAS35WEUA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ephem]

Not sure why you are getting notifications, but it usually says why right next to the Unsubscribe-button that should also fix the problem for you:

[tonyxiao]

Prefetching now can happen in multiple server components, how would you collect all of them together to the right HydrationBoundary just once? Or is there a way for HydrationBoundary to be able to merged internally such that it can be called multiple times?

[kbariotis]

@Ephem thank you for this explanation and the work you've put into it. There's one missing piece that I'm struggling to understand and maybe the implementation has changed since this comment was written. If promises are being passed down to components from the queryClient and use is being used to consume those promises, why do we need to the prefetch? Shouldn't that be handled automatically?

It also confuses me in the docs about the elimination of prefetching which will have the waterfall as an effect but I don't see how all these tie up together.

Thank you

[Ephem]

Hi! This is a common question and it is a bit tricky to wrap ones head around.

Prefetch is needed to start the promise as early as possible. By doing this in a server component, any prefetches in nested server components are guaranteed to either run in parallel, or at least only waterfall on the server, because they are guaranteed to run there.

With the experimental approach without explicit prefetch, the fetch is started in a client component, when React renders useSuspenseQuery. On the first page load, this happens on the server, because client components are server rendered on page load. Client components are not rendered on the server at all on subsequent navigations however, and now if you have useSuspenseQuery in two components that are nested, you now have a clientside waterfall which hurts performance.

This might seem like not a huge deal and of course the tradeoffs depend on your specific situation, but in practice I've found many apps have more waterfalls than they think with the prefetch-less approach, especially when you also take into account that code has to load before the useSuspenseQuery can run.

Did that make things any clearer, or just more confusing? 😄