fix: improve gguf performance with torch.compile

[keturn]

Summary

When using torch.compile with a Flux-type GGUF model, tlparse reports this error:

Unsupported method call
Explanation: Dynamo does not know how to trace method __contains__ of class set
Hint: Avoid calling set.__contains__ in your code.
Hint: Please report an issue to PyTorch.

This is in get_dequantized_tensor, which gets called frequently enough for this to have a significant influence.

Changing the collection from a set to a list is sufficient to make it compatible.

Related Issues / Discussions

See pytorch/pytorch#145761

QA Instructions

Run a GGUF.

Merge Plan

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[psychedelicious]

Looks like all we do w/ this constant is check if something is in it - no functional reason it needs to be in a set.

[StrongerXi]

@keturn hi I'm curious how torch.compile is used within InvokeAI?

[keturn]

@StrongerXi It's not used anywhere in core at the moment—this issue came up when I tried applying it in an extension for the Chroma model.

That did see some appreciable performance gains, so I tried to take advantage of that experience and do the same thing with Invoke's FLUX model, but that's been a fraught experience. I've been unable to explain why that gives me so much more trouble in compilation and so much less to gain for it, considering Chroma and FLUX are very nearly the same model.

(In fact, since I tried that with FLUX, Chroma's compilation only succeeds on its second invocation, failing with something about UserDefinedObjectVariable not having "proxy" the first time around. It was compiling okay without that issue a week ago and I'm at a loss as to what changed.)

[StrongerXi]

failing with something about UserDefinedVariableObject not having "proxy" the first time around. It was compiling okay without that issue a week ago and I'm at a loss as to what changed.

That sounds very similar to what we fixed in PyTorch nightly a few months ago, can you give nightly a try? This post has all the context.

[keturn]

Unfortunately nightly (2.8.0.dev20250605+cu128) does not seem to improve things. That category of error ("proxy") is still there, but unlike stable 2.7.1, running it a second time fails with a different error (in compute_ancestors) that I hadn't seen before.

[StrongerXi]

@keturn thanks, if you can provide a repro and/or a full error message running with TORCHDYNAMO_VERBOSE=1, that'll be awesome. I can help look into this.

[keturn]

I'm a long way from a minimal reproduction but I dumped some logs in pytorch/pytorch#155266