Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Is there a way to control the compression ratio of tt_embedding? #12

Open
xpai opened this issue Apr 3, 2021 · 0 comments
Open

Is there a way to control the compression ratio of tt_embedding? #12

xpai opened this issue Apr 3, 2021 · 0 comments

Comments

@xpai
Copy link

xpai commented Apr 3, 2021

TT embedding provides good a embedding compression ratio. However, I don't know how to control the compression rate by setting the shapes and TT-rank. My goal is to apply tt_embedding for CTR tasks. I have the following two featues and each has been embeded by tt_embedding. But the following setting has led to a large accuracy drop (3%). I wonder if I can only use a small compression rate, e.g., 2x~5x. How can best setting the shapes to achieve a target compression rate? Thanks!

voc_size=820508 emb_size=192
A TT-Matrix of size 900000 x 192, underlying tensorshape: [90, 100, 100] x [4, 6, 8], TT-ranks: [1, 160, 160, 1] on device 'cpu' with compression rate 11.12

voc_size=2903321 emb_size=192
A TT-Matrix of size 3000000 x 192, underlying tensorshape: [125, 150, 160] x [4, 6, 8], TT-ranks: [1, 160, 160, 1] on device 'cpu' with compression rate 24.69

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant