Naive Run Compressed Pt. 2

[Satrat]

SUMMARY:
Follow up PR to neuralmagic/compressed-tensors#109, enables loading compressed models into SparseAutoModel, each quantized layer is decompressed on the forward pass.

• Adds run_compressed argument to SparseAutoModel
• Removes structure initialization in QuantizationModifier, its no longer needed as we do this on load

TEST PLAN:
Manual example, will follow up with integration tests once the compressed-tensor branch merges

from transformers import AutoTokenizer
from llmcompressor.transformers import SparseAutoModelForCausalLM
import torch

model_dir = "nm-testing/Meta-Llama-3-8B-Instruct-fp8-compressed"
model = SparseAutoModelForCausalLM.from_pretrained(model_dir, torch_dtype=torch.float16, device_map="auto", run_compressed=True)

tokenizer = AutoTokenizer.from_pretrained(model_dir)
sample_input = ["I love 8 bit quantization because"]
inputs = tokenizer(sample_input, return_tensors="pt").to(model.device)
generated_ids = model.generate(**inputs, max_length=50)
outputs = tokenizer.batch_decode(generated_ids)
print(outputs)

Output: ["<|begin_of_text|>I love 8 bit quantization because it's a great way to reduce the precision of floating point numbers and make them more compact. It's also a great way to make them more robust against noise and quantization errors.\n\nHere's a simple example of how you can implement 8 bit quantization in Python:\n```\nimport numpy as np\n\ndef quantize(x, bits=8):\n x_min = np.min(x)\n x_max = np.max(x)\n scale = "]

Runtime: ~26sec on two A4000s, ~65sec for the non-compressed version