[tests] speed up animatediff tests

[a-r-r-o-w]

What does this PR do?

In the past, we tried speeding up tests with #7707 but that never got merged due to inactivity on my part. Picking it up again to run AnimateDiff tests much much faster than before.

Continuing from #8789 (comment). 8789 can probably be merge after we fix some of the broken tests here due to overlapping changes.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@DN6

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[a-r-r-o-w]

Here's a before and after speed comparison of the SD1.5 AnimateDiff tests. I will improve the SDXL ones later since some errors after my changes were not straightforward to debug. The results below are from an A100 machine. The improvements are even better when running on just my CPU locally.

• test/pipelines/animatediff/test_animatediff.py:

  • Before:

  real    2m41,549s
  user    12m24,760s
  sys     1m39,072s
  

  • After:

  real    1m47,307s
  user    1m40,633s
  sys     0m36,952s
  

• test/pipelines/animatediff/test_animatediff_video2video.py:

  • Before:

  real    2m4,454s
  user    5m5,238s
  sys     0m52,440s
  

  • After:

  real    1m19,092s
  user    0m58,346s
  sys     0m27,152s
  

• tests/pipelines/pia/test_pia.py:

  • Before:

  real    1m57,247s
  user    6m33,482s
  sys     0m49,454s
  

  • After:

  real    1m22,727s
  user    1m9,556s
  sys     0m25,225s
  

[sayakpaul]

Thanks.

I think we should try to implement test_from_pipe_consistent_config() in this PR itself rather than skipping it.

[sayakpaul]

LGTM. For the value slices, could you follow https://www.notion.so/huggingface2/SSH-into-our-runners-d9c24de77d0843b2a308f28171106001?

[a-r-r-o-w]

@sayakpaul Fixed the expected slice values. Should be good to merge now. Failing tests are unrelated

[yiyixuxu]

@a-r-r-o-w
is the only difference the repo? any reason we would not update the test in the tester mixin?

diffusers/tests/pipelines/test_pipelines_common.py

Line 678 in 1fd647f

if self.original_pipeline_class == StableDiffusionPipeline:

[a-r-r-o-w]

We only made the model size smaller here, for AnimateDiff, no? It can definitely be applied across everything though to gain a good speed up. In fact, we can still make the model smaller with 1 layer per block instead of 2, smaller block out channels, smaller cross attn, etc. Will be some effort if we'd like to do it, but always good to have these tests running faster IMO

[yiyixuxu]

I think for now it will be pretty easy and safe to update the from pipe tester mixin here

diffusers/tests/pipelines/test_pipelines_common.py

Line 654 in 1fd647f

class PipelineFromPipeTesterMixin:

[a-r-r-o-w]

Won't updating that cause most, if not all, tests inheriting from PipelineFromPipeTesterMixin to fail?

For example, in the test test_from_pipe_consistent_config we use hf-internal-testing/tiny-stable-diffusion-pipe which has larger model sizes than the one we're using here (hf-internal-testing/tinier-stable-diffusion-pipe).

In order to use the latter, we will have to update all model configurations in all tests as well as expected_slice values.