You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I think we should start instrumenting 2 metrics that map to TTFT and TOK/s, but the trigger when we perform UI transitions on the client.
Submit starts timer
When we receive the first token in the frontend that causes us to transition into a first token UI state we mark: TTFUI
And then whenever we actually change the UI display we tick a unit for TOKUI/s
Such a metric would map to actual UX, it would catch regressions through egregious buffering at any layer
Use Cases
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
Feature Description
I think we should start instrumenting 2 metrics that map to TTFT and TOK/s, but the trigger when we perform UI transitions on the client.
Submit starts timer
When we receive the first token in the frontend that causes us to transition into a first token UI state we mark: TTFUI
And then whenever we actually change the UI display we tick a unit for TOKUI/s
Such a metric would map to actual UX, it would catch regressions through egregious buffering at any layer
Use Cases
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: