New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

No speed improvements after TF-TRT optimizing on a tensorflow BERT model #330

Open

SohaKhazaeli opened this issue Nov 7, 2022 · 0 comments

SohaKhazaeli commented Nov 7, 2022 •

edited

Loading

After optimizing the model with either FP32 or FP16 I don't get any speed improvements.

The optimization is done on tensorflow/tensorflow:2.10.0-gpu docker image. The model uses tensorflow-text and tf-models-official libraries

This is the log from optimization process:

# for free to join this conversation on GitHub. Already have an account? # to comment