Python3.8 Type Error

[wzhao18]

Tested with version 0.1.6

lib/python3.8/site-packages/flashinfer/cascade.py:276: in MultiLevelCascadeAttentionWrapper
int_workspace_buffers: list[torch.Tensor],
E TypeError: 'type' object is not subscriptable

[yzh119]

Hi @wzhao18 , Can you provide my context? How did you call the reset_workspace_buffer function? (Ideally you don't need to call this function if you provide a large enough float_workspace_buffer when initializing a wrapper)?

The docstring of this function is wrong, int_workspace_buffer should have type List[torch.Tensor] instead of torch.Tensor, we will fix the docstring soon.

[wzhao18]

@yzh119 Thanks for getting back. I believe the issue is simply because python's type system (at least in python3.8) does not allow type annotations like list[torch.Tensor], like here. I believe as simple as changing to List[torch.Tensor] should solve the problem.

[yzh119]

I see, this might be an issue only with python 3.8.

[yzh119]

Fixed in #486, thank you again for reporting this issue.

[wzhao18]

@yzh119 Thank you!