Have any plans to optimize the prefill kernel for the Hopper architecture?

[alexngng]

I notice that the Flashinfer prefill kernel is much slower than FA3 and TRT-LLM FMHA on SM90.
Do you have any plans to use some SM90 features for optimization?

Here is some data I tested on an SM90. Single H20 GPU, Llama2 7B.

Tokens Number	TRT-LLM FMHA	FA3	Flashinfer
512 x 1	37638.6	39,334.6	74966.6
512 x 2	54729.9	61,680.4	114800.0
512 x 4	103388.8	113,056.2	190688.4

[yzh119]

Hi @alexngng , yes for sure. I still have some slight bug to fix and it's coming soon :)

[jason-huang03]

Really looking forward to it!

[taegeonum]

@yzh119 Hello, any update?

[yzh119]

@alexngng @taegeonum @jason-huang03
Done in #667 .

[taegeonum]

@yzh119 Do you have a plan for supporting FP8 Q,K,V?