2024-05-30 Add FP8 PTQ #1877
Open
2024-05-30 Add FP8 PTQ #1877
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
ceci3
reviewed
Jun 17, 2024
| @@ -21,6 +21,7 @@ | |||
| - `EMDObserver`：收集最大绝对值并通过最小化EMD误差，收集量化scale | |||
| - `HistObserver`：将张量值收集到直方图中，并根据百分比计算量化scale | |||
| - `KLObserver`：以最小化浮点值分布与量化浮点值分布之间的 Kullback-Leibler散度计算量化scale | |||
| - `AbsmaxObserver`：根据目标权重的Tensor维度，收集最大绝对值作为量化scale，可使用quant_bits调整量化的数据类型，支持FP8 | |||
| @@ -60,8 +61,8 @@ model = mobilenet_v1() | |||
| q_config = QuantConfig(activation=None, weight=None) | |||
|
|
|||
| # define act_quanter and weight_quanter | |||
| act_quanter = MSEObserver() | |||
| weight_quanter = MSEObserver() | |||
| activation = AbsmaxObserver(quant_bits = (4,3)) # quant_bits = (4,3) and quant_bits = (5,2) for float8_e4m3 and float8_e5m2 formats quantization. | |||
There was a problem hiding this comment.
不要直接修改已有的示例，新增一个示例表示fp8量化，新增文档说明
|
可以把不同observer的fp8量化实验结果贴上来 |
ceci3
previously approved these changes
Jun 25, 2024
ceci3
approved these changes
Jun 25, 2024
|
|
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
修改Uniform Observer支持FP8量化
