Different dtypes of Q (query) and KV (kv-cache) tensors

[ibsidorenko]

This is just a question of whether there are such plans or not...

Right now, Flashinfer lib requires Q (query) and KV (kv-cache) to have the same dtype.
Just an example from the code, q and paged_kv have the same DTypeIn:

template <bool partition_kv, PosEncodingMode pos_encoding_mode, uint32_t num_stages_smem,
          uint32_t tile_size_per_bdx, uint32_t vec_size, uint32_t bdx, uint32_t bdy, uint32_t bdz,
          PageStorage page_storage, QKVLayout kv_layout, typename DTypeIn, typename DTypeOut,
          typename IdType>
__global__ void BatchDecodeWithPagedKVCacheKernel(
    DTypeIn* __restrict__ q, IdType* __restrict__ q_offset,
    paged_kv_t<page_storage, kv_layout, DTypeIn, IdType> paged_kv,
    kv_partition_info_t<IdType> kv_partition_info, DTypeOut* __restrict__ o,
    DTypeOut* __restrict__ tmp, float* __restrict__ lse, float sm_scale, float rope_rcp_scale,
    float rope_rcp_theta)

Are there any plans to support different dtypes for KV-cache and Q (query)?
My personal interest is fp8 for kv-cache and fp16 for query.

Thank you in advance!
cc @yzh119