Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Encode model in GenerateContentRequest only when needed #174

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Encode model in GenerateContentRequest only when needed #174

wants to merge 2 commits into from

Conversation

andrewheard
Copy link
Collaborator

@andrewheard andrewheard commented May 22, 2024 •
edited
Loading

Updated the GenerateContentRequest encoding implementation to serialize the model field only when needed (currently only needed for countTokens).

@github-actions github-actions bot added component:swift sdk Issue/PR related to Swift SDK status:awaiting review PR awaiting review from a maintainer labels May 22, 2024
@andrewheard andrewheard requested a review from ncooke3 May 22, 2024 20:04
@ncooke3
Copy link
Collaborator

ncooke3 commented May 22, 2024

Updated the countTokens implementation to populate generateContentRequest instead of contents in the request body.

  • This provides a more accurate token count since, e.g., function declarations and system instructions from the request are now counted.

Does it imply that the SDK currently only counts tokens from the prompt?

@andrewheard
Copy link
Collaborator Author

Updated the countTokens implementation to populate generateContentRequest instead of contents in the request body.

  • This provides a more accurate token count since, e.g., function declarations and system instructions from the request are now counted.

Does it imply that the SDK currently only counts tokens from the prompt?

@ncooke3 That's accurate. Prior to this PR the SDK would only count tokens from the prompt itself.

ncooke3
ncooke3 previously approved these changes May 22, 2024
View reviewed changes
Sources/GoogleAI/CountTokensRequest.swift Show resolved Hide resolved
Sources/GoogleAI/GenerativeModel.swift Show resolved Hide resolved
@andrewheard andrewheard mentioned this pull request May 22, 2024
Merged
@andrewheard andrewheard marked this pull request as draft May 28, 2024 22:04
@andrewheard andrewheard force-pushed the ah/count-tokens-generate branch from 324e2f8 to 8a0c9bb Compare May 28, 2024 22:21
@andrewheard andrewheard changed the title Send GenerateContentRequest in CountTokensRequest Encode model in GenerateContentRequest only when needed May 28, 2024
@andrewheard andrewheard changed the base branch from main to ah/count-tokens-generate-alt May 28, 2024 22:25
@andrewheard andrewheard changed the base branch from ah/count-tokens-generate-alt to main May 28, 2024 22:27
@andrewheard andrewheard dismissed ncooke3’s stale review May 28, 2024 22:27

The base branch was changed.

# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@ncooke3 ncooke3 ncooke3 left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
component:swift sdk Issue/PR related to Swift SDK status:awaiting review PR awaiting review from a maintainer
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@andrewheard @ncooke3