Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add InstantID Pipeline #6673

Merged
merged 6 commits into from
Jan 23, 2024
Merged

Add InstantID Pipeline #6673

merged 6 commits into from
Jan 23, 2024

Conversation

haofanwang
Copy link
Contributor

What does this PR do?

This PR adds a community pipeline for our recent InstantID work. The original InstantID pipeline is from here, but we add some dependent functions inside the pipeline. We made minimal addition on the top of existing StableDiffusionXLControlNetPipeline.

@sayakpaul
Copy link
Member

Thank you so much! Could you maybe also add a usage example in the README so that people can use it to the max?

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@haofanwang
Copy link
Contributor Author

Sure, will do

@haofanwang
Copy link
Contributor Author

Done. @sayakpaul

pipe.load_ip_adapter_instantid(face_adapter)

# load an image
image = load_image("./examples/yann-lecun_resize.jpg")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use a URL here?

Comment on lines +3521 to +3522
face_adapter = f'./checkpoints/ip-adapter.bin'
controlnet_path = f'./checkpoints/ControlNetModel'
Copy link
Member

@sayakpaul sayakpaul Jan 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for this too. We don't know where it's coming from.

Copy link
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent! Left some more nits.

Once addressed, will merge!

@haofanwang
Copy link
Contributor Author

Added. For now, InstantID cannot be loaded via from_pretrained, so it is necessary to download beforehand. @sayakpaul

@sayakpaul
Copy link
Member

Added. For now, InstantID cannot be loaded via from_pretrained, so it is necessary to download beforehand. @sayakpaul

Works. Could you ensure the code quality at least passes?

@haofanwang
Copy link
Contributor Author

Quality check passed.

@sayakpaul sayakpaul merged commit 9941b3f into huggingface:main Jan 23, 2024
@sayakpaul
Copy link
Member

Congratulations, @haofanwang! Great job!

AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
* add instantid pipeline

* format

* Update README.md

* Update README.md

* format

---------

Co-authored-by: ResearcherXman <xhs.research@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants