Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

DeepDiff raises decimal exception when using significant digits #426

Closed
yawhide opened this issue Oct 17, 2023 · 3 comments
Closed

DeepDiff raises decimal exception when using significant digits #426

yawhide opened this issue Oct 17, 2023 · 3 comments

Comments

@yawhide
Copy link

yawhide commented Oct 17, 2023

Describe the bug
Comparing certain decimals with the significant_digits argument leads to an invalidDecimal exception from being raised within deep diff

To Reproduce

from decimal import Decimal

from deepdiff import DeepDiff


diff = DeepDiff([Decimal("999.99999999")], [Decimal("999.9999999999")], significant_digits=6)
# exception will raise

Expected behavior
There should be no exception and no values_changed

OS, DeepDiff version and Python version (please complete the following information):

  • OS: mac os
  • Version: 13.6
  • Python Version: 3.11.4
  • DeepDiff Version: 6.6.0

Additional context
Stack trace:

Traceback (most recent call last):
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/testdeepdiff.py", line 6, in <module>
    diff = DeepDiff([Decimal("999.99999999")], [Decimal("999.9999999999")], significant_digits=6)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 311, in __init__
    self._diff(root, parents_ids=frozenset({id(t1)}), _original_type=_original_type)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 1569, in _diff
    self._diff_iterable(level, parents_ids, _original_type=_original_type, local_tree=local_tree)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 655, in _diff_iterable
    self._diff_iterable_in_order(level, parents_ids, _original_type=_original_type, local_tree=local_tree)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 761, in _diff_iterable_in_order
    self._diff_ordered_iterable_by_difflib(
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 902, in _diff_ordered_iterable_by_difflib
    self._diff_by_forming_pairs_and_comparing_one_by_one(
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 887, in _diff_by_forming_pairs_and_comparing_one_by_one
    self._diff(next_level, parents_ids_added, local_tree=local_tree)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 1551, in _diff
    self._diff_numbers(level, local_tree=local_tree, report_type_change=report_type_change)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 1360, in _diff_numbers
    t1_s = self.number_to_string(level.t1,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/helper.py", line 397, in number_to_string
    number = number.quantize(Decimal('0.' + '0' * significant_digits))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]

I dont quite understand why your code doesnt work. When i try to reproduce outside of your library, i dont have issues:

from decimal import Decimal
n = Decimal('999.999999999')
n.quantize(Decimal('0.' + '0' * 6))
# => Decimal('1000.000000')
@seperman
Copy link
Owner

Hi @yawhide
It is because the count of the significant digits of the rounded value for 999.9999999 is one digit above precision of the context.

>>> with localcontext() as ctx:
...     ctx.prec = 9
...     Decimal('999.99999999').quantize(Decimal('0.000000'))
...
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
>>>
>>> with localcontext() as ctx:
...     ctx.prec = 10
...     Decimal('999.99999999').quantize(Decimal('0.000000'))
...
Decimal('1000.000000')

@seperman seperman mentioned this issue Oct 17, 2023
@yawhide
Copy link
Author

yawhide commented Oct 18, 2023

oh that makes sense. thanks for the quick fix!

@seperman
Copy link
Owner

Sure!

netbsd-srcmastr referenced this issue in NetBSD/pkgsrc Jan 28, 2024
v6-7-1

-  Support for subtracting delta objects when iterable_compare_func
   is used.
-  Better handling of force adding a delta to an object.
-  Fix for
   ```Can't compare dicts with both single and double quotes in keys`` <https://github.com/seperman/deepdiff/issues/430>`__
-  Updated docs for Inconsistent Behavior with math_epsilon and
   ignore_order = True

v6-7-0

-  Delta can be subtracted from other objects now.
-  verify_symmetry is deprecated. Use bidirectional instead.
-  always_include_values flag in Delta can be enabled to include
   values in the delta for every change.
-  Fix for Delta.\__add\_\_ breaks with esoteric dict keys.

v6-6-1

 -  Fix for `DeepDiff raises decimal exception when using significant
    digits <https://github.com/seperman/deepdiff/issues/426>`__
 -  Introducing group_by_sort_key
 -  Adding group_by 2D. For example
    ``group_by=['last_name', 'zip_code']``

v6-6-0

-  Numpy 2.0 support
-  Adding
   `Delta.to_flat_dicts <https://zepworks.com/deepdiff/current/serialization.html#delta-serialize-to-flat-dictionaries>`__

v6-5-0

-  Adding
   ```parse_path`` <https://github.com/seperman/deepdiff/pull/419>`__

v6-4-1

-  Bugfix: Keep Numpy Optional
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants