We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
I download the files of SOLAR 10.7B (https://huggingface.co/neuralmagic/SOLAR-10.7B-Instruct-v1.0-pruned50-quant-ds) in my MacBook Pro (Ventura 13.4) takes 4-5 minutes to load the model and works fine.
Expected behavior I expected to work like that in another computer with linux, but takes a lot of time to load
Environment Include all relevant environment information:
>>> import deepsparse.cpu >>> print(deepsparse.cpu.cpu_architecture())
'L1_data_cache_size': 32768, 'L1_instruction_cache_size': 32768, 'L2_cache_size': 4194304, 'L3_cache_size': 16777216, 'architecture': 'x86_64', 'available_cores_per_socket': 12, 'available_num_cores': 12, 'available_num_hw_threads': 12, 'available_num_numa': 1, 'available_num_sockets': 1, 'available_sockets': 1, 'available_threads_per_core': 1, 'bf16': False, 'cores_per_socket': 12, 'dotprod': False, 'i8mm': False, 'isa': 'avx512', 'num_cores': 12, 'num_hw_threads': 12, 'num_numa': 1, 'num_sockets': 1, 'threads_per_core': 1, 'vbmi': False, 'vbmi2': False, 'vendor': 'GenuineIntel', 'vendor_id': 'Intel', 'vendor_model': 'Intel Xeon Processor (Skylake, IBRS, no TSX)', 'vnni': False, 'zen1': False To Reproduce from deepsparse import TextGeneration model = TextGeneration(model_path="./SOLAR")
Errors
The text was updated successfully, but these errors were encountered:
No branches or pull requests
I download the files of SOLAR 10.7B (https://huggingface.co/neuralmagic/SOLAR-10.7B-Instruct-v1.0-pruned50-quant-ds) in my MacBook Pro (Ventura 13.4) takes 4-5 minutes to load the model and works fine.
Expected behavior
I expected to work like that in another computer with linux, but takes a lot of time to load
Environment
Include all relevant environment information:
'L1_data_cache_size': 32768, 'L1_instruction_cache_size': 32768, 'L2_cache_size': 4194304, 'L3_cache_size': 16777216, 'architecture': 'x86_64', 'available_cores_per_socket': 12, 'available_num_cores': 12, 'available_num_hw_threads': 12, 'available_num_numa': 1, 'available_num_sockets': 1, 'available_sockets': 1, 'available_threads_per_core': 1, 'bf16': False, 'cores_per_socket': 12, 'dotprod': False, 'i8mm': False, 'isa': 'avx512', 'num_cores': 12, 'num_hw_threads': 12, 'num_numa': 1, 'num_sockets': 1, 'threads_per_core': 1, 'vbmi': False, 'vbmi2': False, 'vendor': 'GenuineIntel', 'vendor_id': 'Intel', 'vendor_model': 'Intel Xeon Processor (Skylake, IBRS, no TSX)', 'vnni': False, 'zen1': False
To Reproduce
from deepsparse import TextGeneration
model = TextGeneration(model_path="./SOLAR")
Errors

The text was updated successfully, but these errors were encountered: