OPT (Open Pre-trained Transformers) is a family of NLP models trained on billions of tokens of text obtained from the internet.
For notes regarding the development of all these models, please refer to our chronicles.
| Model | Parameters | Pretrained weights |
|---|---|---|
| OPT-125M | 125M | part0 |
| OPT-350M | 350M | part0 |
| OPT-1.3B | 1.3B | part0 |
| OPT-2.7B | 2.7B | part0 |
| OPT-6.7B | 6.7B | part0 |
| OPT-13B | 13B | part0, part1 |
| OPT-30B | 30B | part0, part1, part2, part3 |
| OPT-66B | 66B | part0, part1, part2, part3, part4, part5, part6, part7 |
| OPT-175B | 175B | request access here |
For the 2.7B, 6.7B, and 13B, we also release intermediate checkpoints taken at every 10k steps. The full file list for all of these may be found here.
We are including a model card (Mitchell et al., 2018) and data card (Gebru et al., 2021) to help with transparency and accountability in model development.
The use of OPT model weights is subject to the Model License.