Skip to content

chore: memoize detected GPUs #5385

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

chore: memoize detected GPUs #5385

wants to merge 1 commit into from

Conversation

mudler
Copy link
Owner

@mudler mudler commented May 17, 2025

Description

This pull request introduces a caching mechanism to the GPUs function in the pkg/xsysinfo/gpu.go file to improve performance by ensuring GPU information is only fetched once. The most important changes are as follows:

Caching Implementation:

  • Introduced a sync.Once variable (gpuCacheOnce) to ensure the GPU information is fetched only once, even in concurrent scenarios.
  • Added a gpuCache variable to store the cached GPU information and a gpuCacheErr variable to store any errors encountered during the fetch.
  • Modified the GPUs function to use the sync.Once mechanism to populate the cache on the first call and return the cached data or error on subsequent calls.

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Copy link

netlify bot commented May 17, 2025

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 0c859ad
🔍 Latest deploy log https://app.netlify.com/projects/localai/deploys/6828ed4ce566a70008cbeda0
😎 Deploy Preview https://deploy-preview-5385--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant