[Feature]: LoRA support for Qwen model

[zhangfan-algo]

🚀 The feature, motivation and pitch

[rank0]: ValueError: Model QWenLMHeadModel does not support LoRA, but LoRA is enabled. Support for this model may be added in the future. If this is important to you, please open an issue on github.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[jeejeelee]

Are you using Qwen's LLM or VL (Vision-Language) model?