Initial attempt at implementing property-based testing using Hypothesis #4724

RohitP2005 · 2024-12-30T19:19:05Z

Description

This change introduces the use of Hypothesis-based testing in two files and is being applied to selective functions as requested by the author and members of the issue #4703 (comment)

Starts #4703

Type of change

Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

New feature (non-breaking change which adds functionality)
Optimization (back-end change that speeds up the code)
Bug fix (non-breaking change which fixes an issue)

Key checklist:

No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
All tests pass: $ python -m pytest (or $ nox -s tests)
The documentation builds: $ python -m pytest --doctest-plus src (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ nox -s quick.

Further checks:

Code is commented, particularly in hard-to-understand areas
Tests added that prove fix is effective or that feature works

prady0t · 2024-12-31T12:10:03Z

@RohitP2005 Add hypothesis as a dependency in pyproject.toml for checks to pass.

Here:

PyBaMM/pyproject.toml

Line 100 in 421784e

dev = [

prady0t

I've added reviews for test_averages.py. I'll add reviews for test_unary_operators.py once the failing tests are fixed.

tests/unit/test_expression_tree/test_averages.py

prady0t · 2024-12-31T12:28:38Z

tests/unit/test_expression_tree/test_averages.py

@@ -267,8 +271,9 @@ def test_r_average(self):
        assert pybamm.r_average(a + b) == pybamm.r_average(a) + pybamm.r_average(b)
        assert pybamm.r_average(a - b) == pybamm.r_average(a) - pybamm.r_average(b)

-    def test_yz_average(self):
-        a = pybamm.Scalar(1)
+    @given(st.integers(min_value=-(2**63), max_value=2**63 - 1))


Same question here.

explained here at #4724 (comment)

tests/unit/test_expression_tree/test_averages.py

prady0t · 2024-12-31T12:35:58Z

Also, fix the PR heading. It's not proper.

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

RohitP2005 · 2024-12-31T23:23:53Z

I've added reviews for test_averages.py. I'll add reviews for test_unary_operators.py once the failing tests are fixed.

@prady0t
I've updated the heading as requested, and committed all your suggested changes except for one.
Regarding the implementation of Hypothesis, I limited its use to specific tests to avoid adding overhead by applying it to too many cases. Do you think the way I implemented Hypothesis is optimized, or are there areas where I can improve further?

codecov · 2025-01-02T22:23:16Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.70%. Comparing base (725a42c) to head (6c8d2fc).
Report is 4 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #4724   +/-   ##
========================================
  Coverage    98.70%   98.70%           
========================================
  Files          304      304           
  Lines        23452    23483   +31     
========================================
+ Hits         23149    23180   +31     
  Misses         303      303

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

.gitignore

pyproject.toml

prady0t · 2025-01-07T17:43:59Z

tests/unit/test_expression_tree/test_averages.py

@@ -174,9 +177,10 @@ def test_x_average(self):
        assert pybamm.x_average(a + b) == pybamm.x_average(a) + pybamm.x_average(b)
        assert pybamm.x_average(a - b) == pybamm.x_average(a) - pybamm.x_average(b)

-    def test_size_average(self):
+    @given(st.integers(min_value=-(2**63), max_value=2**63 - 1))


I see. Having large-sized integers causes buffer overflow (probably because of different ways int is handled by pybamm and numpy ). Yes it's good to restrict the range here.

tests/unit/test_expression_tree/test_unary_operators.py

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

RohitP2005 · 2025-02-10T16:38:04Z

Hey @agriyakhetarpal can u trigger the CI workflows for this branch

kratman · 2025-02-10T16:45:38Z

@RohitP2005 Triggered CI and updated the branch

Saransh-cpp · 2025-02-13T00:28:21Z

Requesting a review from @prady0t on this.

RohitP2005 · 2025-02-13T08:44:52Z

@RohitP2005 Triggered CI and updated the branch

thanks @kratman
i will look into the failing docs build asap

RohitP2005 · 2025-02-22T16:34:22Z

Hey @agriyakhetarpal I need updates on this PR. Its been there for a long time.
Looking forward to complete this and move on to the next one.

agriyakhetarpal

Done. Sorry for the delay in the review, @RohitP2005! Thanks for your work here. I just have one comment – happy to approve after that.

pyproject.toml

agriyakhetarpal · 2025-02-22T18:15:23Z

Hi @prady0t, I plan to merge this early next week so that @RohitP2005 can proceed further – unless you have comments by then.

RohitP2005 · 2025-02-22T18:16:15Z

@agriyakhetarpal I would like a final check by @prady0t
before merging this.
Cause there's lot other tests that need property based testing.
It would helpful if you suggest what to do next on this matter.

kratman · 2025-02-22T18:50:51Z

Cause there's lot other tests that need property based testing. It would helpful if you suggest what to do next on this matter.

I think I would like to see this be limited to only where it is needed and useful rather than all tests. We would not want to cause the complexity and runtime of our tests to explode from excessive testing. The functions you have done so far are fairly straight forward, but tests in other parts of the code can be quite slow and complicated.

My other concern is that the codebase is physics based, so you need to make sure that the value ranges make sense for the codebase as well. Some parameters may only be valid in the 0-1 range or need to be only positive floats.

prady0t · 2025-02-24T19:35:26Z

Other than the above review, everything look good. Thanks @RohitP2005

Added hypothesis test to two unit testing

1c87e48

prady0t reviewed Dec 31, 2024

View reviewed changes

hypothesis added in dependencies

06ec7f8

RohitP2005 requested review from martinjrobins, Saransh-cpp, agriyakhetarpal, kratman and arjxn-py as code owners December 31, 2024 23:00

RohitP2005 changed the title ~~Added hypothesis test to two unit testing~~ Implementing Property Based Tesing using Hypothesis Dec 31, 2024

RohitP2005 and others added 11 commits January 1, 2025 04:32

Update tests/unit/test_expression_tree/test_averages.py

603dfc0

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

7fc931a

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

e0e4b81

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

f959fe6

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

6a894d8

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

cc79028

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

06b8981

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

46b713e

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

e33d3b4

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_averages.py

5526029

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Updated test_average.py

007045e

prady0t suggested changes Jan 7, 2025

View reviewed changes

RohitP2005 and others added 5 commits January 8, 2025 06:51

Update .gitignore

aa68741

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update pyproject.toml

17a716a

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_unary_operators.py

e8b0571

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_unary_operators.py

fdd61ae

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Update tests/unit/test_expression_tree/test_unary_operators.py

d59e382

Co-authored-by: Pradyot Ranjan <99216956+prady0t@users.noreply.github.com>

Merge branch 'develop' into issue#4703

c0f636d

Merge branch 'develop' into issue#4703

f191876

Saransh-cpp requested a review from prady0t February 13, 2025 00:28

RohitP2005 requested review from a team, rtimms, valentinsulzer, brosaplanella, DrSOKane and TomTranter as code owners February 13, 2025 09:23

RohitP2005 force-pushed the issue#4703 branch from de2699b to f191876 Compare February 13, 2025 09:26

RohitP2005 added 2 commits February 13, 2025 14:56

Merge branch 'develop' into issue#4703

9519505

Merge branch 'pybamm-team:develop' into issue#4703

792b9c4

agriyakhetarpal reviewed Feb 22, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

RohitP2005 added 2 commits February 22, 2025 23:29

Merge branch 'develop' into issue#4703

c2255ec

hypothesis documented

6043473

agriyakhetarpal approved these changes Feb 22, 2025

View reviewed changes

agriyakhetarpal enabled auto-merge (squash) February 24, 2025 20:43

Merge branch 'develop' into issue#4703

6c8d2fc

agriyakhetarpal changed the title ~~Implementing Property Based Tesing using Hypothesis~~ Initial attempt at implementing property-based testing using Hypothesis Feb 24, 2025

agriyakhetarpal merged commit e69fc9a into pybamm-team:develop Feb 24, 2025
26 checks passed

RohitP2005 deleted the issue#4703 branch February 25, 2025 03:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial attempt at implementing property-based testing using Hypothesis #4724

Initial attempt at implementing property-based testing using Hypothesis #4724

RohitP2005 commented Dec 30, 2024 •

edited by agriyakhetarpal

Loading

prady0t commented Dec 31, 2024 •

edited

Loading

prady0t left a comment

prady0t Dec 31, 2024

RohitP2005 Dec 31, 2024

prady0t commented Dec 31, 2024

RohitP2005 commented Dec 31, 2024 •

edited

Loading

codecov bot commented Jan 2, 2025 •

edited

Loading

prady0t Jan 7, 2025

RohitP2005 commented Feb 10, 2025

kratman commented Feb 10, 2025

Saransh-cpp commented Feb 13, 2025

RohitP2005 commented Feb 13, 2025

RohitP2005 commented Feb 22, 2025

agriyakhetarpal left a comment

agriyakhetarpal commented Feb 22, 2025

RohitP2005 commented Feb 22, 2025

kratman commented Feb 22, 2025

prady0t commented Feb 24, 2025

Initial attempt at implementing property-based testing using Hypothesis #4724

Initial attempt at implementing property-based testing using Hypothesis #4724

Conversation

RohitP2005 commented Dec 30, 2024 • edited by agriyakhetarpal Loading

Description

Type of change

Key checklist:

Further checks:

prady0t commented Dec 31, 2024 • edited Loading

prady0t left a comment

Choose a reason for hiding this comment

prady0t Dec 31, 2024

Choose a reason for hiding this comment

RohitP2005 Dec 31, 2024

Choose a reason for hiding this comment

prady0t commented Dec 31, 2024

RohitP2005 commented Dec 31, 2024 • edited Loading

codecov bot commented Jan 2, 2025 • edited Loading

Codecov Report

prady0t Jan 7, 2025

Choose a reason for hiding this comment

RohitP2005 commented Feb 10, 2025

kratman commented Feb 10, 2025

Saransh-cpp commented Feb 13, 2025

RohitP2005 commented Feb 13, 2025

RohitP2005 commented Feb 22, 2025

agriyakhetarpal left a comment

Choose a reason for hiding this comment

agriyakhetarpal commented Feb 22, 2025

RohitP2005 commented Feb 22, 2025

kratman commented Feb 22, 2025

prady0t commented Feb 24, 2025

RohitP2005 commented Dec 30, 2024 •

edited by agriyakhetarpal

Loading

prady0t commented Dec 31, 2024 •

edited

Loading

RohitP2005 commented Dec 31, 2024 •

edited

Loading

codecov bot commented Jan 2, 2025 •

edited

Loading