New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

关于fastspeech2流式推理的疑问 #3850

Open

world1tree opened this issue Sep 18, 2024 · 0 comments

Labels

world1tree commented Sep 18, 2024

fastspeech2推理时的batch size设置为1，这是否意味着一个请求处理结束，模型才会处理下一个请求？还是说因为async，模型能够同时对多个请求进行推理？如果是后者，与真正的batch推理在性能上是否仍有一定的差距？
我也查看了其它开源的TTS项目，似乎都不支持按batch进行推理。这是否是因为TTS模型相比于LLM，在batch推理上实现比较困难？还是说batch推理会增大响应时间，导致实时性更差？

world1tree added the Question label

# for free to join this conversation on GitHub. Already have an account? # to comment