Skip to content

SWAP info added to README #632

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
wants to merge 1 commit into from
Closed

Conversation

jjrbfi
Copy link

@jjrbfi jjrbfi commented Mar 30, 2023

PR to show that we can use larger models if ( RAM < model ) using SWAP.

I explained this in more detail here: #631

@slaren
Copy link
Member

slaren commented Mar 30, 2023

Using memory mapping for the models (#613) may reduce or even remove the need of using a swap at all. Can you test if that's the case? If so, this advice would be outdated by the time that PR is merged.

@jjrbfi
Copy link
Author

jjrbfi commented Mar 30, 2023

Using memory mapping for the models (#613) may reduce or even remove the need of using a swap at all. Can you test if that's the case? If so, this advice would be outdated by the time that PR is merged.

Thanks for the reply.
That PR is something I have to have a look in detail, thanks.

At least for now (Until that PR is merged and we ensure we can work with < 16GB) using SWAP we ensure that, if we don't have enough RAM we can extend it and get all working.

@prusnak
Copy link
Collaborator

prusnak commented Mar 30, 2023

I don't think we should suggest it, because although it's true that you can run the model with weights stored on swap, it is also quite impractical and almost unusable.

@prusnak prusnak closed this Mar 30, 2023
@trollkotze
Copy link

trollkotze commented Apr 4, 2023

@prusnak

I don't think we should suggest it, because although it's true that you can run the model with weights stored on swap, it is also quite impractical and almost unusable.

Seems to work well for me if swap is only used as needed (using RAM first), which I achieved with this tweak:
#753

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants