v0.2.5 issue tracker

[casper-hansen]

Fixes/features to publish before release.

• FIX: Add safe guards for static cache + llama on transformers latest #401
• Fix fused models for tf >= 4.39 #418
• Workaround: illegal memory access #421
• See if we still need Not needed: https://github.com/casper-hansen/AutoAWQ/tree/fix_quant
• Transformers requirement: transformers>=4.35.0
• Introduce apply_clip=True for quantize(), partial fix according for QWEN int4 bad perfomance #360 (comment)
• Pin lm_eval==0.4.1 in eval requirements.
• Add download_kwargs for load model (#302) #399

Potentially (experimenting with this):

• Cohere ⌘R and ⌘R+ support
• Rework get_calib_dataset function
• Block batching for large custom calibration dataset
• PR reviews. See if they work out of the box:
  • Add StableLM support #410
  • add starcoder2 support #406

[Atry]

Looks like everything required is fixed. Do you want to tag a 0.2.5 release?

[thincal]

Looks like everything required is fixed. Do you want to tag a 0.2.5 release?

+1, mainly required to use higher version of transformers.

[Atry]

Ping @casper-hansen

[casper-hansen]

Release will be available shortly. Sorry for the delay.

https://github.com/casper-hansen/AutoAWQ/releases/tag/v0.2.5