Skip to content

Commit

Permalink
Typo + ocp/fnuz quantizers
Browse files Browse the repository at this point in the history
  • Loading branch information
Giuseppe5 committed Aug 20, 2024
1 parent 6a62d3d commit ab8bf37
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 11 deletions.
2 changes: 1 addition & 1 deletion notebooks/minifloat_mx_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@
"```\n",
"\n",
"With respect to pre-defined quantizers, Brevitas offers several Groupwise and MX options.\n",
"The meain difference between the two is that MX is restricted to group_size=32 and the scale factor must be a power-of-2.\n",
"The main difference between the two is that MX is restricted to group_size=32 and the scale factor must be a power-of-2.\n",
"The user can override these settings but the corresponding output won't be MX compliant.\n",
"\n",
"Another difference is that MXFloat relies on the OCP format as underlying data type, while generic groupwise float relies on the non-standard minifloat representation explained above.\n",
Expand Down
13 changes: 5 additions & 8 deletions src/brevitas_examples/llm/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@
'--weight-quant-format',
type=quant_format_validator,
default='int',
help='Weight quantization type. Either int or eXmY, with X+Y==weight_bit_width-1. Default: int.'
help=
'Weight quantization type. Either int or eXmY, with X+Y==weight_bit_width-1. It\'s possible to add float_ocp_ or float_fnuz_ before the exponent/mantissa bitwidth. Default: int.'
)
parser.add_argument(
'--weight-quant-granularity',
Expand All @@ -105,7 +106,9 @@
'--input-quant-format',
type=quant_format_validator,
default='int',
help='Input quantization type. Either int or eXmY, with X+Y==weight_bit_width-1. Default: int.')
help=
'Input quantization type. Either int or eXmY, with X+Y==weight_bit_width-1. It\'s possible to add float_ocp_ or float_fnuz_ before the exponent/mantissa bitwidth. Default: int.'
)
parser.add_argument(
'--input-param-method',
type=str,
Expand Down Expand Up @@ -187,10 +190,6 @@
type=str,
default=None,
help="Filename to save checkpoint. If `None`, no checkpoint is saved (default: %(default)s)")
add_bool_arg(
parser, 'use-ocp', default=False, help='Use OCP format for float quantization. Default: False')
add_bool_arg(
parser, 'use-fnuz', default=True, help='Use FNUZ format for float quantization. Default: True')


def set_seed(seed):
Expand Down Expand Up @@ -385,8 +384,6 @@ def main():
input_quant_granularity=args.input_quant_granularity,
input_group_size=args.input_group_size,
quantize_input_zero_point=args.quantize_input_zero_point,
use_ocp=args.use_ocp,
use_fnuz=args.use_fnuz,
device=device)
layer_map = generate_quant_maps(
linear_input_quant=linear_input_quant,
Expand Down
6 changes: 4 additions & 2 deletions src/brevitas_examples/stable_diffusion/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -759,13 +759,15 @@ def input_zp_stats_type():
type=quant_format_validator,
default='int',
help=
'Weight quantization type. Either int or eXmY, with X+Y==weight_bit_width-1. Default: int.')
'Weight quantization type. Either int or eXmY, with X+Y==weight_bit_width-1. It\'s possible to add float_ocp_ or float_fnuz_ before the exponent/mantissa bitwidth. Default: int.'
)
parser.add_argument(
'--input-quant-format',
type=quant_format_validator,
default='int',
help=
'Input quantization type. Either int or eXmY, with X+Y==input_bit_width-1. Default: int.')
'Input quantization type. Either int or eXmY, with X+Y==input_bit_width-1. It\'s possible to add float_ocp_ or float_fnuz_ before the exponent/mantissa bitwidth. Default: int.'
)
parser.add_argument(
'--weight-quant-granularity',
type=str,
Expand Down

0 comments on commit ab8bf37

Please # to comment.