Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

新增图文对话模型X-GPT #3541

Closed
chenjjcccc opened this issue Oct 24, 2023 · 0 comments
Closed

新增图文对话模型X-GPT #3541

chenjjcccc opened this issue Oct 24, 2023 · 0 comments
Assignees

Comments

@chenjjcccc
Copy link
Collaborator

问题描述 Please describe your issue

新增图文对话模型X-GPT

任务描述

任务背景

  • X-Decoder 集成了图像理解的多类任务,结合GPT和SD相关生成模型就可以实现All-in-One的图文对话式agnet。参考代码

完成步骤

  1. 基于Paddle复现X-Decoder,其可以进行检测、分割、VQA、取标题等多种应用。(如果训练对齐遇到不可抗问题,可以仅进行前向对齐。)
  2. 结合PaddleMix中基础模型ppdiffuser,进行生成模型迁移。
  3. 结合开源对话模型例如chatglm v2或者llama v2,来实现XGPT,给出使用示例和文档以及UI。提交至PaddleSeg/contrib/XGPT。
  4. 参考PR提交规范提交代码PR到ppseg中。

提交内容:

  1. 代码提交到PaddleSeg。
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants