Skip to content
This repository has been archived by the owner on Oct 23, 2023. It is now read-only.

Commit

Permalink
QONNX ops and new general transformations (#52)
Browse files Browse the repository at this point in the history
* [Op] add skeleton for Quant custom op

* [Op] more comments on Quant op

* [InferDT] add MaxPool to list of identity ops for datatype inf.

* [Transform] temporarily exclude Quant ops from const folding

* [Analysis] add skeleton for inference_cost, limited conv support

* [Analysis] support inference_cost_matmul

* [Deps] add clize

* [Util] add inference_cost util with entrypoint

* [Docker] add clize

* [Infra] add entrypoint for inference_cost

* Preliminary implementation of quant operation

* Fixed wrong numpy type in quant op

* Made Quant pre-commit compatible

* Changed docker base image to PyTorch and added Brevitas dockerfile and entrypoint

* Implemented preliminary test for Quant operation

* Reworked comments for Quant operation

* Readability improvements for Quant operation

* Created preliminary Quant op documentation

* Move Quant op documentation

* Made documentation of quant attributes more precise.

* Created skeleton for Trunc QNNX operation

* [DataType] add experimental SCALED(U)INT DataTypes

* [Quant] use new scaled int dtypes for dt inference

* [TypInf] use non-integer instead of float check in dt inference

* [ConstFold] make op exclusion optional

* [Analysis] deal with dyn matmul, add some zero-cost op types

* [Analysis] make inference_cost optionally sparse-weights-aware

* [Util] pass sparsity option to analysis pass, kwargs only for clize

* [Analysis] add more free ops to inference_cost

* [Quant] fallback to float32 for failed dtype inference

* Inference cost: JSON export, parameter for output onnx

* Removed FINN dependency from documentation

* Added support for executing qonnx custom_ops.

* Add support for Bipolar and Binary FINN datatype for Quant op. (#41)

* Add support for Bipolar and Binary FINN datatype for Quant op.

* [Quant] custom qnt execution for bipolar until Brevitas bug resolved

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* Added support for cost estimation for upsampling (#37)

* Update AUTHORS.rst

* Create python-publish.yml

* Add ZCU111 board to part map (#32)

* Update AUTHORS.rst

* GHA for PyPI, sdist only (#35)

* finn-base v0.0.2 (#34)

* Modified set_nodeattr to allow using it on repeated fields (#18)

* [base]: changed how the floats, ints, strings, tensors, graphs and sparse_tensors field of AttributeProto is set.

* [Core] restrict attributes to tested types

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* Support for non-square input images and kernels for im2col node (#20)

* [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node

* [base]: changed how the "ints" field of AttributeProto set.
[test_general_transformation]: changed the type of the kernel_size attribute to list of integers

* removed unused import

* [base]: added support for writing repeated fields in AttributeProto

* minor style changes

* [im2col, test_im2col]: added support for non-equal padding

* [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding.
[test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution.

* [test_conv_lowering]: included 1D depthwise and regular convolutions in tests

* Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests"

This reverts commit 3ff449c

* Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding."

This reverts commit 15e34ed.

* Revert "[im2col, test_im2col]: added support for non-equal padding"

This reverts commit c524020.

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: minor change in style.

* [Im2Col] style fixes and comments

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* Update AUTHORS.rst

* Support for non-square input images and kernels in LowerConvsToMatMul transformation (#16)

* [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node

* [base]: changed how the "ints" field of AttributeProto set.
[test_general_transformation]: changed the type of the kernel_size attribute to list of integers

* removed unused import

* [base]: added support for writing repeated fields in AttributeProto

* minor style changes

* [im2col, test_im2col]: added support for non-equal padding

* [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding.
[test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution.

* [test_conv_lowering]: included 1D depthwise and regular convolutions in tests

* Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests"

This reverts commit 3ff449c

* Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding."

This reverts commit 15e34ed.

* Revert "[im2col, test_im2col]: added support for non-equal padding"

This reverts commit c524020.

* [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed.
[test_im2col] changed function call to compute_conv_output_dim.
[test_conv_lowering] changed function call to compute_conv_output_dim.
[lower_convs_to_matmul] removed old assertion.

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: minor change in style.

* [test_conv_lowering]: minor fix for test case depthwise and regular convolutions

* Support for non-square input images and kernels for im2col node (#20)

* [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node

* [base]: changed how the "ints" field of AttributeProto set.
[test_general_transformation]: changed the type of the kernel_size attribute to list of integers

* removed unused import

* [base]: added support for writing repeated fields in AttributeProto

* minor style changes

* [im2col, test_im2col]: added support for non-equal padding

* [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding.
[test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution.

* [test_conv_lowering]: included 1D depthwise and regular convolutions in tests

* Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests"

This reverts commit 3ff449c

* Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding."

This reverts commit 15e34ed.

* Revert "[im2col, test_im2col]: added support for non-equal padding"

This reverts commit c524020.

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: minor change in style.

* [Im2Col] style fixes and comments

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* Update AUTHORS.rst

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* Added support for dilation value = 2 for 1D and 2D images/kernels (#17)

* [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node

* [base]: changed how the "ints" field of AttributeProto set.
[test_general_transformation]: changed the type of the kernel_size attribute to list of integers

* removed unused import

* [base]: added support for writing repeated fields in AttributeProto

* minor style changes

* [im2col, test_im2col]: added support for non-equal padding

* [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding.
[test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution.

* [test_conv_lowering]: included 1D depthwise and regular convolutions in tests

* Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests"

This reverts commit 3ff449c

* Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding."

This reverts commit 15e34ed.

* Revert "[im2col, test_im2col]: added support for non-equal padding"

This reverts commit c524020.

* [im2col] added support for dilations = 2 for 2D and 1D images. Dilation value must be equal along each axis.
[test_im2col] added several test cases in case dilations = 2 (a.o. cases where image and kernel are 2D and 1D, with and without padding, and with stride = 2).
[lower_convs_to_matmul] added support for dilation value. Dilation value must be equal along each axis.
[test_conv_lowering] added test case for dilations = 2 for 2D and 1D images and kernels, with and without padding, and with stride = 2.

* [lower_convs_to_matmul] removed old assertion
[test_conv_lowering] added more dilation values to test cases. Dilation values of {1, 2, 3, 4} are tested.

* [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed.
[test_im2col] changed function call to compute_conv_output_dim
[test_conv_lowering] changed function call to compute_conv_output_dim

* [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed.
[test_im2col] changed function call to compute_conv_output_dim.
[test_conv_lowering] changed function call to compute_conv_output_dim.
[lower_convs_to_matmul] removed old assertion.

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: minor change in style.

* [test_conv_lowering]: minor fix for test case depthwise and regular convolutions

* [im2col]: minor style adjustment.
[test_conv_lowering]: merged test functions into one test function.

* Support for non-square input images and kernels for im2col node (#20)

* [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node

* [base]: changed how the "ints" field of AttributeProto set.
[test_general_transformation]: changed the type of the kernel_size attribute to list of integers

* removed unused import

* [base]: added support for writing repeated fields in AttributeProto

* minor style changes

* [im2col, test_im2col]: added support for non-equal padding

* [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding.
[test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution.

* [test_conv_lowering]: included 1D depthwise and regular convolutions in tests

* Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests"

This reverts commit 3ff449c

* Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding."

This reverts commit 15e34ed.

* Revert "[im2col, test_im2col]: added support for non-equal padding"

This reverts commit c524020.

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: minor change in style.

* [Im2Col] style fixes and comments

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* Update AUTHORS.rst

* Support for non-square input images and kernels in LowerConvsToMatMul transformation (#16)

* [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node

* [base]: changed how the "ints" field of AttributeProto set.
[test_general_transformation]: changed the type of the kernel_size attribute to list of integers

* removed unused import

* [base]: added support for writing repeated fields in AttributeProto

* minor style changes

* [im2col, test_im2col]: added support for non-equal padding

* [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding.
[test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution.

* [test_conv_lowering]: included 1D depthwise and regular convolutions in tests

* Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests"

This reverts commit 3ff449c

* Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding."

This reverts commit 15e34ed.

* Revert "[im2col, test_im2col]: added support for non-equal padding"

This reverts commit c524020.

* [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed.
[test_im2col] changed function call to compute_conv_output_dim.
[test_conv_lowering] changed function call to compute_conv_output_dim.
[lower_convs_to_matmul] removed old assertion.

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: minor change in style.

* [test_conv_lowering]: minor fix for test case depthwise and regular convolutions

* Support for non-square input images and kernels for im2col node (#20)

* [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node

* [base]: changed how the "ints" field of AttributeProto set.
[test_general_transformation]: changed the type of the kernel_size attribute to list of integers

* removed unused import

* [base]: added support for writing repeated fields in AttributeProto

* minor style changes

* [im2col, test_im2col]: added support for non-equal padding

* [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding.
[test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution.

* [test_conv_lowering]: included 1D depthwise and regular convolutions in tests

* Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests"

This reverts commit 3ff449c

* Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding."

This reverts commit 15e34ed.

* Revert "[im2col, test_im2col]: added support for non-equal padding"

This reverts commit c524020.

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: minor change in style.

* [Im2Col] style fixes and comments

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* Update AUTHORS.rst

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* Update quantavgpool2d.py (#22)

Add  "Copyright (c) 2021 Xilinx, Inc" heather

* [batchnorm_to_affine]: epsilon value is now read out from the attributes. (#21)

[test_batchnorm_to_affine]: added a test case for various epsilon values.

* Added 3D to 4D (tensor) transformation (#19)

* [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels

* [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node

* [base]: changed how the "ints" field of AttributeProto set.
[test_general_transformation]: changed the type of the kernel_size attribute to list of integers

* removed unused import

* [base]: added support for writing repeated fields in AttributeProto

* minor style changes

* [im2col, test_im2col]: added support for non-equal padding

* [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding.
[test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution.

* [test_conv_lowering]: included 1D depthwise and regular convolutions in tests

* Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests"

This reverts commit 3ff449c

* Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding."

This reverts commit 15e34ed.

* Revert "[im2col, test_im2col]: added support for non-equal padding"

This reverts commit c524020.

* [im2col] added support for dilations = 2 for 2D and 1D images. Dilation value must be equal along each axis.
[test_im2col] added several test cases in case dilations = 2 (a.o. cases where image and kernel are 2D and 1D, with and without padding, and with stride = 2).
[lower_convs_to_matmul] added support for dilation value. Dilation value must be equal along each axis.
[test_conv_lowering] added test case for dilations = 2 for 2D and 1D images and kernels, with and without padding, and with stride = 2.

* [lower_convs_to_matmul] removed old assertion
[test_conv_lowering] added more dilation values to test cases. Dilation values of {1, 2, 3, 4} are tested.

* [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed.
[test_im2col] changed function call to compute_conv_output_dim
[test_conv_lowering] changed function call to compute_conv_output_dim

* [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed.
[test_im2col] changed function call to compute_conv_output_dim.
[test_conv_lowering] changed function call to compute_conv_output_dim.
[lower_convs_to_matmul] removed old assertion.

* [im2col]: minor fix with assumption on kernel dimension
[lower_convs_to_matmul]: minor fix with assumption on kernel dimension

* [change_3d_tensors_to_4d]: added new transformation that transforms 3D tensors to 4D and changes the nodes accordingly
[test_4d_conversion]: test function for 3D to 4D tensor transformation

* [change_3d_tensors_to_4d]: added new transformation that changes 3D tensors to 4D.
[test_4d_conversion]: added a test case for the 3D to 4D transformation.

* [change_3d_tensors_to_4d]: added 3D to 4D transformation (for QuartzNet).
[test_4d_conversion]: added test case for 3D to 4D transform.

* [change_3d_tensors_to_4d]: changed how an invalid graph is handled.
[test_4d_conversion]: changed the test case for an invalid graph.

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: changed how a square kernel is instantiated.
[lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated).

* [im2col]: minor change in style.

* [im2col]: minor style change and changed the way how a square kernel is instantiated.

* [test_conv_lowering]: merged tests for depthwise and standard convolutions.

* [test_conv_lowering]: minor fix for test case depthwise and regular convolutions

* [im2col]: minor style adjustment.
[test_conv_lowering]: merged test functions into one test function.

* [change_3d_tensors_to_4d]: style fixes and comments.
[test_4d_converions]: rearranged code.

* [Transform] check invalid node list length

* [change_3d_tensors_to_4d]: rearranged the code to make it more readable.

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>

* [Docs] update tutorials

* [Util] experimental: fast_mode data packing for binary (#24)

* Generic partitioning feature (#23)

* Add basic partitioning functionality

* Mount build dir within docker container

* Support for non-linear models and multi in/out partitions

* Remove dataflowpartition custom op from finn-base

* Fix temporary build dir for CI

* Fix docstring

* [create_generic_partitions]: minor modification, removed redundant output value_info entries. (#26)

* [extend_partition]: added a new transformation ExtendPartition. (#27)

[test_extend_partition]: added a test case for the new transformation.

* Added support for non-equal strides along different axes (#25)

* [im2col]: added support for non-equal strides along different axes and cleaned up the code.
[lower_convs_to_matmul]: added support for non-equal strides along different axes and cleaned up the code.
[test_conv_lowering]: added test case for non-equal strides along different axes.

* [im2col]: minor fix.
[test_im2col]: added test case for non-equal strides along different axes.

* Changes for supporting vitis_hls (#28)

* [Refactor] split up RTL/HLS-related utils

* [Util] rename to CallHLS and allow specifying vivado_hls/vitis_hls

* [Util] more flexible stream naming in rtlsim_multi_io

* Changes for supporting non-equal dilation (#29)

* added support for non-equal dilation value along (H, W) dimension

* added test cases for non-equal dilation configurations

* appending dilation value along dummy dimension correctly (i.e. with a '1')

* changed tensor sparsity annotation for consistency

* Support infer_datatype for flatten layer (#30)

* Support infer_datatype for flatten layer

* [InferDT] add more identity op types for datatype inference

* [Lint] fix linting issues

Co-authored-by: Yaman Umuroglu <maltanar@gmail.com>

* Update AUTHORS.rst

* Create python-publish.yml

* Add ZCU111 board to part map (#32)

* Update AUTHORS.rst

Co-authored-by: Mirza Mrahorovic <34712307+mmrahorovic@users.noreply.github.com>
Co-authored-by: jalezeta <51440887+jalezeta@users.noreply.github.com>
Co-authored-by: Felix Jentzsch <45395194+fpjentzsch@users.noreply.github.com>

* Update python-publish.yml

* Update python-publish.yml

* Lint

Co-authored-by: Mirza Mrahorovic <34712307+mmrahorovic@users.noreply.github.com>
Co-authored-by: jalezeta <51440887+jalezeta@users.noreply.github.com>
Co-authored-by: Felix Jentzsch <45395194+fpjentzsch@users.noreply.github.com>

* Inference cost: JSON export, parameter for output onnx

* Added support for cost estimation for upsampling

Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>
Co-authored-by: Felix Jentzsch <45395194+fpjentzsch@users.noreply.github.com>
Co-authored-by: Mirza Mrahorovic <34712307+mmrahorovic@users.noreply.github.com>
Co-authored-by: jalezeta <51440887+jalezeta@users.noreply.github.com>
Co-authored-by: Felix Jentzsch <fepaje@mail.upb.de>

* [Quant] fix edge case for 0d input array execution

* Added trunc support for MobileNet, needs some fixing.

* Made trunc datatype inference pass-through.

* Addressed numpy deprecation warning in Truc op exectution.

* Updated Quant op to support rounding mode.

* Added support for new Trunc op.

* Added support for static datatype inference to the Trunc op.

* Resolved left over ToDo in Trunc op.

* Remove scaledInt from Trunc op and replace with float32.

* Moved RemoveIdentityOps transformation from FINN to finn-base.

* Fixed a bug in RemoveIdentityOps, where an identity op would not get removed if it was preceded by another identity op.

* Moved GemmToMatMul from FINN to finn-base.

* Added support for recognizing MultiThreshold node in sign discovery.

* Moved RemoveEmptyPadding transformation from finn to finn-base.

* Updated Quant op documentation to add the rounding_mode attribute.

* Added documentation for trunc op and small adjustments for Quant op documentation.

* Added preliminary support for BinaryQuant node.

* Updated datatype inference for Quant and BinaryQuant to remove scaledInt types.

* Fixed bug in datatype inference for Quant node.

* Added support for merging into non-linear graphs for _remove_node_and_rewire function.

* Added another op to sign_preserving_ops.

* Updated Trunc op documentation.

* Extended graph based signed inference to more quantization nodes.

* Renamed BinaryQuant to BipolarQuant.

* Renamed BinaryQuant to BipolarQuant for Trunc op Signed inference.

* Updated BipolarQuant op documentation and moved documentation of QONNX ops into their own folder.

* Made remove_node_and_rewire function part of the publicly facing api.

* Adapt test_remove_identity_ops to new datatype system.

* Made padding removal more reliable.

* Moved ExtractBiasFromConv transformation from FINN to finn-base.

* Fixed changed import for test_remove_identity_ops test.

* Extended MultiThreshold data type inference to adjust for float scale/bias.

* Removed test_brevitas_quant_onnx_export_and_exec test, since test_QONNX_to_FINN in FINN already tests for the same.

* Fixed spelling typos in QONNX op documentation.

* Specified rounding operation in Quant and Trunc documentation.

* Updated version description for QONNX ops documentation.

* Removed dependency on onnxoptimizer.

* Refactored check for float.

* Refactored get_internal_dtype to get_integer_datatype.

* Disabled constant folding for Quant and BipolarQuant by default.

* Added a warning if an unsupported ONNX version is encountered during the GemmToMatMul transformation.

* Moved functionality of RemoveEmptyPadding into RemoveIdentityOps.

* Made float32 default output for Trunc node and removed #ference from node.

Co-authored-by: Yaman Umuroglu <maltanar@gmail.com>
Co-authored-by: Yaman Umuroglu <yamanu@xilinx.com>
Co-authored-by: Felix Jentzsch <fepaje@mail.upb.de>
Co-authored-by: Felix Jentzsch <45395194+fpjentzsch@users.noreply.github.com>
Co-authored-by: Mirza Mrahorovic <34712307+mmrahorovic@users.noreply.github.com>
Co-authored-by: jalezeta <51440887+jalezeta@users.noreply.github.com>
  • Loading branch information
7 people authored Nov 1, 2021
1 parent 7c2603a commit 71ed2e0
Show file tree
Hide file tree
Showing 18 changed files with 1,782 additions and 4 deletions.
1 change: 1 addition & 0 deletions docker/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
bitstring>=3.1.7
clize==4.1.1
numpy
onnx==1.7.0
onnxruntime==1.4.0
Expand Down
92 changes: 92 additions & 0 deletions docs/QONNX/bipolar_quant_op.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
### <a name="BipolarQuant"></a><a name="abs">**BipolarQuant**</a>

Calculates the binary quantized values of one input data (Tensor<T>) and produces one output data (Tensor<T>).
Additionally, takes one float as input, which define the scaling.

#### Version

This operator is not part of the ONNX standard and is not currently versioned.

#### Attributes

<dl>
</dl>

#### Inputs

<dl>
<dt><tt>X</tt> (differentiable) : tensor(float32)</dt>
<dd>input tensor to quantize</dd>
<dt><tt>scale</tt> : float32</dt>
<dd>The scale factor</dd>
</dl>


#### Outputs

<dl>
<dt><tt>Y</tt> (differentiable) : tensor(float32)</dt>
<dd>Output tensor</dd>
</dl>


#### Examples
<details>
<summary>BipolarQuant</summary>

```python
from onnx import helper
import numpy as np

# Define node settings and input
x = np.random.randn(100).astype(np.float32)*10.
scale = np.array(1.)

# Create node
node = helper.make_node(
'BipolarQuant',
domain='finn.custom_op.general',
inputs=['x', 'scale'],
outputs=['y'],
)

# Execute the same settings with the reference implementation (quant)
# See the sample implementation for more details on quant.
output_ref = binary_quant(x, scale)

# Execute node and compare
expect(node, inputs=[x, scale], outputs=[output_ref], name='test_binary_quant')

```

</details>


#### Sample Implementation

<details>
<summary>BipolarQuant</summary>

```python
# SPDX-License-Identifier: Apache-2.0

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

import numpy as np

def binary_quant(inp_tensor, scale):
# Quantizing
y_int = inp_tensor
y_ones = np.ones(y_int.shape, dtype=y_int.dtype)
y_int = np.where(y_int >= 0.0, y_ones, -y_ones)
# Scaling
out_tensor = y_int * scale

return out_tensor

```

</details>
193 changes: 193 additions & 0 deletions docs/QONNX/quant_op.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
### <a name="Quant"></a><a name="abs">**Quant**</a>

Calculates the quantized values of one input data (Tensor<T>) and produces one output data (Tensor<T>).
Additionally, takes three floats as input, which define the scale, zero-point and bit-width of the quantization.
The attributes narrow and signed define how the bits of the quantization are interpreted, while the attribute
rounding_mode defines how quantized values are rounded.

Note: This operator does not work for binary or bipolar quantization, for this purpose the simpler BipolarQuant node exists.

#### Version

This operator is not part of the ONNX standard and is not currently versioned.

#### Attributes

<dl>
<dt><tt>signed</tt> : int (default is 1)</dt>
<dd>Defines if the quantization includes a signed bit. E.g. at 8b unsigned=[0, 255] vs signed=[-128, 127].</dd>
<dt><tt>narrow</tt> : int (default is 0)</dt>
<dd>Defines if the value range should be interpreted as narrow, when signed=1. E.g. at 8b regular=[-128, 127] vs narrow=[-127, 127].</dd>
<dt><tt>rounding_mode</tt> : string (default is "ROUND")</dt>
<dd>Defines how rounding should be applied during quantization. Currently available modes are: "ROUND", "CEIL" and "FLOOR". Here "ROUND" implies a round-to-even operation.</dd>
</dl>

#### Inputs

<dl>
<dt><tt>X</tt> (differentiable) : tensor(float32)</dt>
<dd>input tensor to quantize</dd>
<dt><tt>scale</tt> : float32</dt>
<dd>The scale factor</dd>
<dt><tt>zeropt</tt> : float32</dt>
<dd>The zero-point</dd>
<dt><tt>bitwidth</tt> : int32</dt>
<dd>The number of bits used by the quantization</dd>
</dl>


#### Outputs

<dl>
<dt><tt>Y</tt> (differentiable) : tensor(float32)</dt>
<dd>Output tensor</dd>
</dl>


#### Examples
<details>
<summary>Quant</summary>

```python
from onnx import helper
import numpy as np

# Define node settings and input
x = np.random.randn(100).astype(np.float32)*10.
scale = np.array(1.)
zeropt = np.array(0.)
bitwidth = np.array(4)
signed = 1
narrow = 0
rounding_mode = "ROUND"

# Create node
node = helper.make_node(
'Quant',
domain='finn.custom_op.general',
inputs=['x', 'scale', 'zeropt', 'bitwidth'],
outputs=['y'],
narrow=narrow,
signed=signed,
rounding_mode=rounding_mode,
)

# Execute the same settings with the reference implementation (quant)
# See the sample implementation for more details on quant.
output_ref = quant(x, scale, zeropt, bitwidth, signed, narrow, rounding_mode)

# Execute node and compare
expect(node, inputs=[x, scale, zeropt, bitwidth], outputs=[output_ref], name='test_quant')

```

</details>


#### Sample Implementation

<details>
<summary>Quant</summary>

```python
# SPDX-License-Identifier: Apache-2.0

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

import numpy as np

def quant(inp_tensor, scale, zeropt, bitwidth, signed, narrow, rounding_mode):
# Port of IntQuant class from Brevitas: https://bit.ly/2S6qvZJ
# Scaling
y_int = inp_tensor / scale
y_int = y_int + zeropt
# Clamping
min_int_val = min_int(signed, narrow, bitwidth)
max_int_val = max_int(signed, narrow, bitwidth)
y_int = np.where(y_int > max_int_val, max_int_val.astype(y_int.dtype), y_int)
y_int = np.where(y_int < min_int_val, min_int_val.astype(y_int.dtype), y_int)
# Rounding
rounding_fx = resolve_rounding_mode(rounding_mode)
y_int = rounding_fx(y_int)

# Re-scaling
out_tensor = y_int - zeropt
out_tensor = out_tensor * scale

return out_tensor

def min_int(signed: bool, narrow_range: bool, bit_width: int) -> int:
"""Compute the minimum integer representable by a given number of bits.
Args:
signed (bool): Indicates whether the represented integer is signed or not.
narrow_range (bool): Indicates whether to narrow the minimum value
represented by 1.
bit_width (int): Number of bits available for the representation.
Returns:
int: Maximum unsigned integer that can be represented according to
the input arguments.
Examples:
>>> min_int(signed=True, narrow_range=True, bit_width=8)
int(-127)
>>> min_int(signed=False, narrow_range=True, bit_width=8)
int(0)
>>> min_int(signed=True, narrow_range=False, bit_width=8)
int(-128)
>>> min_int(signed=False, narrow_range=False, bit_width=8)
int(0)
"""
if signed and narrow_range:
value = -(2 ** (bit_width - 1)) + 1
elif signed and not narrow_range:
value = -(2 ** (bit_width - 1))
else:
value = 0 * bit_width
return value


def max_int(signed: bool, narrow_range: bool, bit_width: int) -> int:
"""Compute the maximum integer representable by a given number of bits.
Args:
signed (bool): Indicates whether the represented integer is signed or not.
narrow_range (bool): Indicates whether to narrow the maximum unsigned value
represented by 1.
bit_width (int): Number of bits available for the representation.
Returns:
Tensor: Maximum integer that can be represented according to
the input arguments.
Examples:
>>> max_int(signed=True, narrow_range=True, bit_width=8)
int(127)
>>> max_int(signed=False, narrow_range=True, bit_width=8)
int(254)
>>> max_int(signed=True, narrow_range=False, bit_width=8)
int(127)
>>> max_int(signed=False, narrow_range=False, bit_width=8)
int(255)
"""
if not signed and not narrow_range:
value = (2 ** bit_width) - 1
elif not signed and narrow_range:
value = (2 ** bit_width) - 2
else:
value = (2 ** (bit_width - 1)) - 1
return value

def resolve_rounding_mode(mode_string):
"""Resolve the rounding mode string of Quant and Trunc ops
to the corresponding numpy functions."""
if mode_string == "ROUND":
return np.round
elif mode_string == "CEIL":
return np.ceil
elif mode_string == "FLOOR":
return np.floor
else:
raise ValueError(f"Could not resolve rounding mode called: {mode_string}")

```

</details>
Loading

0 comments on commit 71ed2e0

Please # to comment.