Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Preload some vocabs in extra to improve runtimes #47

Closed
nomennescio opened this issue Feb 6, 2024 · 13 comments
Closed

Preload some vocabs in extra to improve runtimes #47

nomennescio opened this issue Feb 6, 2024 · 13 comments
Labels
enhancement New feature or request

Comments

@nomennescio
Copy link
Collaborator

nomennescio commented Feb 6, 2024

See codewars/factor#11

As a suggestion, for 0.99, consider preloading some vocabs as mentioned, but final list to be decided

@nomennescio nomennescio added the enhancement New feature or request label Feb 6, 2024
@nomennescio
Copy link
Collaborator Author

nomennescio commented Feb 6, 2024

NOTE: current Factor docs (for 0.100 development version on master branch) only list the following vocabs in extras:

assocs.extras
combinators.extras
grouping.extras
math.extras
math.matrices.extras
namespaces.extras
sequences.extras
sets.extras
sorting.extras
splitting.extras
stream.extras

Check which vocabs are supported by 0.98, and which are supported by 0.99.

@nomennescio
Copy link
Collaborator Author

Factor 0.99 supports the following extras vocabs:

assocs.extras
combinators.extras
grouping.extras
math.extras
math.matrices.extras
namespaces.extras
sequences.extras
sets.extras
sorting.extras
splitting.extras
stream.extras

@nomennescio
Copy link
Collaborator Author

Factor 0.98 supports the following extras vocabs:

assocs.extras
combinators.extras
grouping.extras
math.extras
sequences.extras
sets.extras
sorting.extras
splitting.extras
stream.extras

@nomennescio
Copy link
Collaborator Author

When looking at the original list as suggested in the issue, some of these are not extras vocabs as indicated, but separate vocabs located in the extra vocab root. Reconsider the list of vocabs to be preloaded.

@nomennescio nomennescio changed the title Preload some vocabs in extras to improve runtimes Preload some vocabs in extra to improve runtimes Feb 6, 2024
@nomennescio
Copy link
Collaborator Author

nomennescio commented Feb 6, 2024

Note that the indicated load time increase to 5-6 seconds because of using libraries not preloaded, needs to be balanced against the size of the Factor image, plus the advantage some of these libraries add in terms of implementation effort. It still leaves a level playing field if a Kata depends on such a library. Most katas are not that time sensitive in the first place.

Vocabs in extra by definition are not stable, so a certain hesitance to guard against future incompatibility is warranted.

@nomennescio
Copy link
Collaborator Author

Need to check if some of the mentioned vocabs have been moved to extra in 0.99. Those are primary candidates to preload.

@nomennescio
Copy link
Collaborator Author

nomennescio commented Feb 6, 2024

Out of the list of suggested vocabs, the following list is actually in extra, hence is not preloaded right now (the others are):

coroutines 
decimals
generators
infix
lru-cache
multisets
pair-rocket
pairs
path-finding
qw
trees
variants

I don't think they can all be considered to be "commonly-used", although they might be useful to preload.

@nomennescio
Copy link
Collaborator Author

Out of the list in the previous comment, the following vocabs are also present in 0.98:

decimals
infix
lru-cache
pair-rocket
pairs
path-finding
qw
trees
variants

So coroutines ,generators, and multisets were newly introduced in 0.99.

@Kacarott
Copy link
Contributor

Kacarott commented Feb 6, 2024

When I have referred to extras I do not mean purely vocabs ending in .extras, but rather than vocab in the extras root.

Also, all the vocab prefixes listed have vocabs in extras. It is not a list of explicit imports, but rather a list of prefixes to load. See load.

I have already done extensive testing with compile time, cold load time, warm load time, and image size. The final list is one which includes almost all vocabs which might be useful for kata, with only neglegible size difference (ie. the total size after increased image and removed extras ends up smaller than the current factor docker image).

@nomennescio
Copy link
Collaborator Author

When I have referred to extras I do not mean purely vocabs ending in .extras, but rather than vocab in the extras root.

Also, all the vocab roots listed have vocabs in extras. It is not a list of explicit imports, but rather a list of prefixes to load. See load.

I have already done extensive testing with compile time, cold load time, warm load time, and image size. The final list is one which includes almost all vocabs which might be useful for kata, with only neglegible size difference (ie. the total size after increased image and removed extras ends up smaller than the current factor docker image).

Thanks for investigating that.
Right, so you want to load all vocabs with these prefixes?
So that would include the ".extras" variants.

Right. If you have already tested that then I'm fine with it. I will make sure it will work for both 0.98 and 0.99.

@nomennescio
Copy link
Collaborator Author

I will introduce specializations for versions 0.98 and 0.99, so it will be easy to maintain.

@Kacarott
Copy link
Contributor

Kacarott commented Feb 6, 2024

So that would include the ".extras" variants.

Yes, but not only those. But the .extras are especially important as they are the worst offenders for long load times, sequences.extras in particular.

@nomennescio
Copy link
Collaborator Author

2058344

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants