Add Inference support for running the BigScience-BLOOM Architecture#2083

Merged

jeffra merged 27 commits intomasterfrom ds-inference/bloom-support

Jul 18, 2022

+1,199-347

Commits on Jul 9, 2022

Add Inference support for running the BigScience-BLOOM Architecture
Reza Yazdani
committed

Commits on Jul 11, 2022

Commits on Jul 12, 2022

add the checkpoint loading at the same of kernel-injection
Reza Yazdani
committed
Merge branch 'ds-inference/bloom-support' of github.com:microsoft/DeepSpeed into ds-inference/bloom-support
Reza Yazdani
committed
releasing checkpoint CPU-memory after loading it
Reza Yazdani
committed
some fixes plus formatting
Reza Yazdani
committed

Commits on Jul 13, 2022

fix layer_past; this caused issues when running inference on several queries
Reza Yazdani
committed
Add support for multi-batch inference
Reza Yazdani
committed

Commits on Jul 14, 2022

fix the padding issue for large bach inference
Reza Yazdani
committed
fixing some bug in softmax kernel for batch_size>1
Reza Yazdani
committed

Commits on Jul 15, 2022

align alibi&mask addition with HF new changes
Reza Yazdani
committed
revert back some changes and support for very large batch size
Reza Yazdani
committed
reduce the max_token_length for now
Reza Yazdani
committed
fix mask-adding
Reza Yazdani
committed

Commits on Jul 16, 2022

fix the large-batch inference for MP > 1
Reza Yazdani
committed

Commits on Jul 18, 2022

Merge branch 'master' into ds-inference/bloom-support
RezaYazdaniAminabadi
authored
Ds inference/bloom support meta (#2104 )
jeffra
authored
fix the Bert and GPT-J unit tests
Reza Yazdani
committed
fix for OneDevice
jeffra
committed
Merge branch 'master' into ds-inference/bloom-support
jeffra
authored
added bloom inference tests
mrwyattii
committed
fixing the masking stride for the GPT models
Reza Yazdani
committed
Merge branch 'ds-inference/bloom-support' of github.com:microsoft/DeepSpeed into ds-inference/bloom-support
Reza Yazdani
committed
revert back some changes on replace_module
Reza Yazdani
committed
fix fp32 softmax
Reza Yazdani
committed
allocate tensors initially on cpu at inference-api
Reza Yazdani
committed