[Bug] latents_attention_mask not used

[HaoyiZhu]

Environment

/

Describe the bug

Thanks for the amazing project!

In the data loader, latents are zero-padded and recorded by an attention mask. However, during training and model forward, it seems that the latent attention mask is not used. Is this a bug?

Reproduction

/

[foreverpiano]

Thanks for pointing out this. latents_attention_mask is not used. I will remove it and refactor code soon.

[jzhang38]

We are going to add support for variable-length sequence packing. I intentionally retain the latent mask for this future feature.