SUPPORTED_MODELS.md

OpenVINO™ GenAI: Supported Models

Large language models

Architecture	Models	Example HuggingFace Models
ChatGLMModel	ChatGLM	THUDM/chatglm3-6b
GemmaForCausalLM	Gemma	google/gemma-2b-it
GPTNeoXForCausalLM	Dolly	databricks/dolly-v2-3b
	RedPajama	ikala/redpajama-3b-chat
LlamaForCausalLM	Llama 3	meta-llama/Meta-Llama-3-8B meta-llama/Meta-Llama-3-8B-Instruct meta-llama/Meta-Llama-3-70B meta-llama/Meta-Llama-3-70B-Instruct
	Llama 2	meta-llama/Llama-2-13b-chat-hf meta-llama/Llama-2-13b-hf meta-llama/Llama-2-7b-chat-hf meta-llama/Llama-2-7b-hf meta-llama/Llama-2-70b-chat-hf meta-llama/Llama-2-70b-hf microsoft/Llama2-7b-WhoIsHarryPotter
	OpenLLaMA	openlm-research/open_llama_13b openlm-research/open_llama_3b openlm-research/open_llama_3b_v2 openlm-research/open_llama_7b openlm-research/open_llama_7b_v2
	TinyLlama	TinyLlama/TinyLlama-1.1B-Chat-v1.0
MistralForCausalLM	Mistral	mistralai/Mistral-7B-v0.1
	Notus	argilla/notus-7b-v1
	Zephyr	HuggingFaceH4/zephyr-7b-beta
PhiForCausalLM	Phi	microsoft/phi-2 microsoft/phi-1_5
QWenLMHeadModel	Qwen	Qwen/Qwen-7B-Chat Qwen/Qwen-7B-Chat-Int4 Qwen/Qwen1.5-7B-Chat Qwen/Qwen1.5-7B-Chat-GPTQ-Int4

Note

LoRA adapters are supported.

The pipeline can work with other similar topologies produced by optimum-intel with the same model signature. The model is required to have the following inputs after the conversion:

1. input_ids contains the tokens.
2. attention_mask is filled with 1.
3. beam_idx selects beams.
4. position_ids (optional) encodes a position of currently generating token in the sequence and a single logits output.

Note

Models should belong to the same family and have the same tokenizers.

Image generation models

Architecture	Text 2 image	Image 2 image	Inpainting	LoRA support	Example HuggingFace Models
Latent Consistency Model	Supported	Supported	Supported	Supported	SimianLuo/LCM_Dreamshaper_v7
Stable Diffusion	Supported	Supported	Supported	Supported	CompVis/stable-diffusion-v1-1 CompVis/stable-diffusion-v1-2 CompVis/stable-diffusion-v1-3 CompVis/stable-diffusion-v1-4 jcplus/stable-diffusion-v1-5 stable-diffusion-v1-5/stable-diffusion-v1-5 botp/stable-diffusion-v1-5 dreamlike-art/dreamlike-anime-1.0 stabilityai/stable-diffusion-2 stabilityai/stable-diffusion-2-base stabilityai/stable-diffusion-2-1 bguisard/stable-diffusion-nano-2-1 justinpinkney/pokemon-stable-diffusion stablediffusionapi/architecture-tuned-model IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Chinese-EN-v0.1 ZeroCool94/stable-diffusion-v1-5 pcuenq/stable-diffusion-v1-4 rinna/japanese-stable-diffusion benjamin-paine/stable-diffusion-v1-5 philschmid/stable-diffusion-v1-4-endpoints naclbit/trinart_stable_diffusion_v2 Fictiverse/Stable_Diffusion_PaperCut_Model
Stable Diffusion Inpainting	Not applicable	Not applicable	Supported	Supported	stabilityai/stable-diffusion-2-inpainting stable-diffusion-v1-5/stable-diffusion-inpainting botp/stable-diffusion-v1-5-inpainting parlance/dreamlike-diffusion-1.0-inpainting
Stable Diffusion XL	Supported	Supported	Supported	Supported	stabilityai/stable-diffusion-xl-base-0.9 stabilityai/stable-diffusion-xl-base-1.0 stabilityai/sdxl-turbo cagliostrolab/animagine-xl-4.0
Stable Diffusion XL Inpainting	Not applicable	Not applicable	Supported	Supported	diffusers/stable-diffusion-xl-1.0-inpainting-0.1
Stable Diffusion 3	Supported	Not supported	Not supported	Not supported	stabilityai/stable-diffusion-3-medium-diffusers stabilityai/stable-diffusion-3.5-medium stabilityai/stable-diffusion-3.5-large stabilityai/stable-diffusion-3.5-large-turbo
Flux	Supported	Supported	Supported	Partially Supported	black-forest-labs/FLUX.1-schnell Freepik/flux.1-lite-8B-alpha black-forest-labs/FLUX.1-dev shuttleai/shuttle-3-diffusion shuttleai/shuttle-3.1-aesthetic shuttleai/shuttle-jaguar Shakker-Labs/AWPortrait-FL

Visual language models

Architecture	Models	LoRA support	Example HuggingFace Models	Notes
InternVL2	InternVL2	Not supported	OpenGVLab/InternVL2-1B OpenGVLab/InternVL2-2B OpenGVLab/InternVL2-4B OpenGVLab/InternVL2-8B OpenGVLab/InternVL2_5-1B OpenGVLab/InternVL2_5-2B OpenGVLab/InternVL2_5-4B OpenGVLab/InternVL2_5-8B
LLaVA	LLaVA-v1.5	Not supported	llava-hf/llava-1.5-7b-hf
LLaVA-NeXT	LLaVa-v1.6	Not supported	llava-hf/llava-v1.6-mistral-7b-hf llava-hf/llava-v1.6-vicuna-7b-hf llava-hf/llama3-llava-next-8b-hf
MiniCPMV	MiniCPM-V-2_6	Not supported	openbmb/MiniCPM-V-2_6
Phi3VForCausalLM	phi3_v	Not supported	microsoft/Phi-3-vision-128k-instruct microsoft/Phi-3.5-vision-instruct	GPU isn't supported These models' configs aren't consistent. It's required to override the default eos_token_id with the one from a tokenizer: generation_config.set_eos_token_id(pipe.get_tokenizer().get_eos_token_id()).
Qwen2-VL	Qwen2-VL	Not supported	Qwen/Qwen2-VL-2B-Instruct Qwen/Qwen2-VL-7B-Instruct

Whisper models

Architecture	Models	LoRA support	Example HuggingFace Models
WhisperForConditionalGeneration	Whisper	Not supported	openai/whisper-tiny openai/whisper-tiny.en openai/whisper-base openai/whisper-base.en openai/whisper-small openai/whisper-small.en openai/whisper-medium openai/whisper-medium.en openai/whisper-large-v3
	Distil-Whisper	Not supported	distil-whisper/distil-small.en distil-whisper/distil-medium.en distil-whisper/distil-large-v3

Some models may require access request submission on the Hugging Face page to be downloaded.

If https://huggingface.co/ is down, the conversion step won't be able to download the models.