ENH: Implement Multivariate Rejection Sampling (MRS) #738

Lucas-Prates · 2024-11-28T21:36:07Z

Pull request type

Code changes (bugfix, features)

Checklist

Unit tests for the changes have been added
Integration tests for the changes have been added
Docs have been reviewed and added / updated
Lint (black rocketpy/ tests/) has passed locally
All tests (pytest tests -m slow --runslow) have passed locally
CHANGELOG.md has been updated (if relevant)
RST documentation
Monte Carlo comparison Feature

New behavior

This PR implements the MRS requested in #162 and described in RocketPy paper.

Breaking change

Yes (perhaps)

Additional information

codecov · 2024-11-28T21:45:20Z

Codecov Report

Attention: Patch coverage is 81.77778% with 41 lines in your changes missing coverage. Please review.

Project coverage is 79.18%. Comparing base (4df0b38) to head (d335d26).
Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
rocketpy/plots/monte_carlo_plots.py	77.38%	19 Missing ⚠️
rocketpy/prints/monte_carlo_prints.py	57.69%	11 Missing ⚠️
...ketpy/simulation/multivariate_rejection_sampler.py	89.81%	11 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #738      +/-   ##
===========================================
+ Coverage    79.11%   79.18%   +0.06%     
===========================================
  Files           96       97       +1     
  Lines        11575    11798     +223     
===========================================
+ Hits          9158     9342     +184     
- Misses        2417     2456      +39

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Lucas-Prates · 2024-11-28T22:20:32Z

This PR is ready for a first "Design Review." I would like to get your opinion if this implementation provides what you think on how the user should use the MRS.

Albeit a implementation as a function seems natural, I implemented as a class because:

the function would be humongous;
if the user wants to resample several times, the data monte carlo data is only read once from the harddrive.

It currently works as follows:

Input: monte carlo filepath prefix, mrs filepath prefix, distribution dictionary;
Load input and output data from a monte carlo simulation into memory (python objects - lists of jsons);
To avoid having to read data twice, while loading, precompute some important properties required in the sampler algorithm;
Select and save iteratively accepted samples;
Output: files are saved in the same "scheme" as the MonteCarlo simulation.

I provided a quick and dirty notebook, which will be removed, just to show how the class is being used at the moment.

Gui-FernandesBR

I don't have much time so I'll be short

The class dies after you sample the data once, this makes it pointless to have a class
Instead I'd remove the sample dictionary from arguments.
First read the data during initialization. Then you use a function to set the variables you are going to allow be varied. This way u can already anticipate which variables may be varied.
We want almost instant results for a MonteCarlo simulation after using MRS.
Other thing is that the user must supply the original pdf. Could we possile estimate this from data? (imagine 70k)
Finally, plotting is crucial for MRS, or even tables. We'd love to see more on that later.

rocketpy/simulation/multivariate_rejection_sampler.py

… (MRS).

…lly checks in MRS

Lucas-Prates · 2025-02-24T19:45:43Z

This PR is ready for review again. Changes from last time:

addressed some of the previous suggestions of @phmbressan;
added a comparison_info method that provides a statistical summary print comparison of the results of two monte carlo simulations, similar to the method used in the regular monte carlo class;
added a comparison_plots method that plots boxplots and histograms comparison of the results of two monte carlo simulations;
added a compare_ellipses method that plots the ellipses for the apogee and landing point comparison of the results of two monte carlo simulations;
added the mrs.rst file to the documentation explaining what is the MRS and how to use it;
added the MultivariateRejectionSampler class .rst documentation;
removed the test_mrs.ipynb notebook since the .rst file does the same in a much clearer way;
on my pc, the MRS is 500x faster than running a monte carlo simulation of the equivalent sample size.

For review, I recommend mostly checking the .rst documentation. Here are some previews of the comparisons if you do not have the time to compile the html:

Print comparison:

Plot comparison:

Ellipses comparison:

Lucas-Prates · 2025-02-24T19:51:52Z

A second comment: I made the design choice to implement the comparison methods in the MonteCarlo class instead of the MultivariateRejectionSampler. Here are the most relevant arguments:

It makes sense to compare monte carlo simulations even outside of MRS, i.e. even one is not a sub-sample of the other. Hence, implementing this inside the MRS would make this usage extremely odd;
It keeps the MultivariateRejectionSampler class implementation really simple and with only one job: to sample.

One option is, of course, to create another class for that job, but it did not seem to be the best option.

Gui-FernandesBR

LGTM.

Great implementation, @Lucas-Prates !

I would kindly recommend adding unit tests to some of the methods you've develop, just to make sure we cover some of the new code lines.

rocketpy/simulation/monte_carlo.py

rocketpy/simulation/multivariate_rejection_sampler.py

docs/user/mrs.rst

docs/reference/classes/MultivariateRejectionSampler.rst

Gui-FernandesBR · 2025-02-25T19:03:56Z

Also, please update the CHANGELOG file!

… python3.9

Lucas-Prates · 2025-02-27T19:12:12Z

I have addressed the suggestions made by @Gui-FernandesBR, implemented unit and integration tests for the MRS, and updated the CHANGELOG.

Gui-FernandesBR

LGTM

Gui-FernandesBR · 2025-02-27T20:52:16Z

@Lucas-Prates good work. Please squash and merge at your earliest convinience

MateusStano

This is really good overall! Great job

docs/user/mrs.rst

rocketpy/plots/monte_carlo_plots.py

MateusStano · 2025-03-02T12:08:44Z

rocketpy/plots/monte_carlo_plots.py

+        plt.scatter(
+            original_apogee_x,
+            original_apogee_y,
+            s=5,
+            marker="^",
+            color="green",
+            label="Original Apogee",
+        )
+        plt.scatter(
+            original_impact_x,
+            original_impact_y,
+            s=5,
+            marker="v",
+            color="blue",
+            label="Original Landing Point",
+        )
+


Improve colors of the points/ellipses. Using oposite colors is probably best

There's no such thing as "opposite colors", but matplotlib does offer a fair good guide to selecting colormaps: https://matplotlib.org/stable/users/explain/colors/colormaps.html

I suggest blue/orange or blue/red.

Ideally the user should be able to select their specific color... But I understand the current limitations.

I guess the right term in complementary not "oposite". Anyway, I meant something like this:

I will use then a "tetradic" color combination I found in this site. Here is the pallete I got, just for reference:

This is the tetradic colors for the ellipses plot:

…n MRS and applying linters

Lucas-Prates · 2025-03-31T13:59:58Z

This PR is ready again for review. Since last time:

1 - addressed the points in Stano's review about documentation and color;
2 - removed some input checks which I think were not that useful but were a bit trick to do correctly;
3 - modified the flatten_dict function to better handle variables in inner levels.

The most important point is 3 since it might introduce a breaking change.

Gui-FernandesBR

All looking good to me, great implementation.
I don't see the changes to flatten_dict as a problem.
Nobody is using that function right now.

Waiting for @MateusStano 's final comments so we can proceed.

rocketpy/simulation/multivariate_rejection_sampler.py

Copilot

Copilot reviewed 14 out of 21 changed files in this pull request and generated 1 comment.

Files not reviewed (7)

.vscode/settings.json: Language not supported
docs/notebooks/monte_carlo_analysis/monte_carlo_analysis_outputs/mrs.outputs.txt: Language not supported
docs/reference/classes/MultivariateRejectionSampler.rst: Language not supported
docs/reference/index.rst: Language not supported
docs/user/index.rst: Language not supported
docs/user/mrs.rst: Language not supported
docs/user/sensitivity.rst: Language not supported

Comments suppressed due to low confidence (1)

rocketpy/tools.py:595

[nitpick] Consider renaming 'flatted_dict' to 'flattened_dict' for clarity and consistency with common terminology.

flatted_dict = {}

rocketpy/plots/monte_carlo_plots.py

Gui-FernandesBR · 2025-04-07T14:01:48Z

rocketpy/plots/monte_carlo_plots.py

+            other_impact_x = np.array([])
+            other_impact_y = np.array([])
+
+        if len(original_apogee_x) == 0 and len(original_impact_x) == 0:


Suggested change

if len(original_apogee_x) == 0 and len(original_impact_x) == 0:

if len(original_apogee_x) == 0 and len(original_impact_x) == 0: # pragma no cover

Gui-FernandesBR · 2025-04-07T14:02:07Z

rocketpy/plots/monte_carlo_plots.py

+        if image is not None:
+            try:
+                img = imageio.imread(image)
+            except FileNotFoundError as e:


Suggested change

except FileNotFoundError as e:

except FileNotFoundError as e: # pragma no cover

Lucas-Prates requested a review from a team as a code owner November 28, 2024 21:36

github-actions bot assigned Lucas-Prates Nov 28, 2024

Lucas-Prates added Enhancement New feature or request, including adjustments in current codes Monte Carlo Monte Carlo and related contents labels Nov 28, 2024

Lucas-Prates linked an issue Nov 28, 2024 that may be closed by this pull request

ENH: Implement MRS method on RocketPy! #162

Open

Lucas-Prates marked this pull request as draft November 28, 2024 21:38

Lucas-Prates requested review from Gui-FernandesBR, MateusStano and phmbressan November 28, 2024 22:20

Gui-FernandesBR force-pushed the enh/MRS branch from 766bd58 to c45fabf Compare November 30, 2024 17:34

Gui-FernandesBR reviewed Dec 6, 2024

View reviewed changes

Lucas-Prates force-pushed the enh/MRS branch from c45fabf to d3ab5f2 Compare December 19, 2024 11:47

phmbressan reviewed Dec 20, 2024

View reviewed changes

rocketpy/simulation/multivariate_rejection_sampler.py Outdated Show resolved Hide resolved

rocketpy/simulation/multivariate_rejection_sampler.py Outdated Show resolved Hide resolved

Gui-FernandesBR force-pushed the enh/MRS branch from 3a3c9e6 to 8ae8826 Compare January 19, 2025 20:41

Gui-FernandesBR force-pushed the develop branch from 2931c30 to 7864590 Compare February 8, 2025 19:00

Gui-FernandesBR force-pushed the enh/MRS branch from 8ae8826 to d5e65a8 Compare February 8, 2025 19:08

Lucas-Prates added 3 commits February 14, 2025 09:49

ENH: implementing a draft version of the Multivarite Rejectio Sampler…

e8d7587

… (MRS).

MNT: quick notebook to test MRS during development

fed170b

MNT: refactoring class to match review suggestions

36ea7f1

Gui-FernandesBR force-pushed the enh/MRS branch from d5e65a8 to 36ea7f1 Compare February 14, 2025 12:49

Lucas-Prates added 4 commits February 21, 2025 19:56

ENH: add comparison prints, plots and ellipses to MonteCarlo and fina…

717b3ae

…lly checks in MRS

MNT: add MultivariateRejectionSampler class to inits and apply format

53b096d

DOC: writting .rst documentation for MRS

9787a17

MNT: adding pylint flags to skip checks

8a5f75e

Lucas-Prates requested review from Gui-FernandesBR and phmbressan February 24, 2025 19:31

DOC: completing missing sections in mrs.rst

232aede

Gui-FernandesBR marked this pull request as ready for review February 25, 2025 18:44

Gui-FernandesBR approved these changes Feb 25, 2025

View reviewed changes

Gui-FernandesBR changed the title ~~ENH: Implementing Multivariate Rejection Sampling (MRS) in RocketPy~~ ENH: Implement Multivariate Rejection Sampling (MRS) Feb 25, 2025

Lucas-Prates added 5 commits February 26, 2025 17:51

DOC: add changelog and apply sugestions in MRS class

b7e6be1

DOC: apply suggestions to the MRS.rst

53e69b4

MNT: use Union instead of | for type hinting since we have to support…

7637c66

… python3.9

TST: adding unit and integration tests to MRS

f565acb

MNT: use pylint flag to fix linter

af3e393

TST: adding tests to MonteCarlo comparison features

cd74e22

Gui-FernandesBR approved these changes Feb 27, 2025

View reviewed changes

MateusStano reviewed Mar 2, 2025

View reviewed changes

Lucas-Prates and others added 3 commits March 28, 2025 16:31

Merge branch 'develop' into enh/MRS

b540ee8

MNT: applying suggestions in .rst, better handling nested variables i…

e2a3ac6

…n MRS and applying linters

MNT: removing TODO comments from monte_carlo_plots

c767a32

Lucas-Prates requested a review from MateusStano March 30, 2025 11:59

Gui-FernandesBR approved these changes Mar 31, 2025

View reviewed changes

rocketpy/simulation/multivariate_rejection_sampler.py Outdated Show resolved Hide resolved

MNT: remove useless TODO

d335d26

Gui-FernandesBR requested a review from Copilot April 7, 2025 13:56

Copilot AI reviewed Apr 7, 2025

View reviewed changes

rocketpy/plots/monte_carlo_plots.py Show resolved Hide resolved

Gui-FernandesBR reviewed Apr 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Implement Multivariate Rejection Sampling (MRS) #738

ENH: Implement Multivariate Rejection Sampling (MRS) #738

Lucas-Prates commented Nov 28, 2024 •

edited

Loading

codecov bot commented Nov 28, 2024 •

edited

Loading

Lucas-Prates commented Nov 28, 2024

Gui-FernandesBR left a comment

Lucas-Prates commented Feb 24, 2025 •

edited

Loading

Lucas-Prates commented Feb 24, 2025

Gui-FernandesBR left a comment

Gui-FernandesBR commented Feb 25, 2025

Lucas-Prates commented Feb 27, 2025

Gui-FernandesBR left a comment

Gui-FernandesBR commented Feb 27, 2025

MateusStano left a comment

MateusStano Mar 2, 2025

Gui-FernandesBR Mar 3, 2025

MateusStano Mar 4, 2025

Lucas-Prates Mar 18, 2025

Lucas-Prates Mar 29, 2025

Lucas-Prates commented Mar 31, 2025

Gui-FernandesBR left a comment

Copilot AI left a comment

Gui-FernandesBR Apr 7, 2025

Gui-FernandesBR Apr 7, 2025

	if len(original_apogee_x) == 0 and len(original_impact_x) == 0:
	if len(original_apogee_x) == 0 and len(original_impact_x) == 0: # pragma no cover

	except FileNotFoundError as e:
	except FileNotFoundError as e: # pragma no cover

ENH: Implement Multivariate Rejection Sampling (MRS) #738

Are you sure you want to change the base?

ENH: Implement Multivariate Rejection Sampling (MRS) #738

Conversation

Lucas-Prates commented Nov 28, 2024 • edited Loading

Pull request type

Checklist

New behavior

Breaking change

Additional information

codecov bot commented Nov 28, 2024 • edited Loading

Codecov Report

Lucas-Prates commented Nov 28, 2024

Gui-FernandesBR left a comment

Choose a reason for hiding this comment

Lucas-Prates commented Feb 24, 2025 • edited Loading

Lucas-Prates commented Feb 24, 2025

Gui-FernandesBR left a comment

Choose a reason for hiding this comment

Gui-FernandesBR commented Feb 25, 2025

Lucas-Prates commented Feb 27, 2025

Gui-FernandesBR left a comment

Choose a reason for hiding this comment

Gui-FernandesBR commented Feb 27, 2025

MateusStano left a comment

Choose a reason for hiding this comment

MateusStano Mar 2, 2025

Choose a reason for hiding this comment

Gui-FernandesBR Mar 3, 2025

Choose a reason for hiding this comment

MateusStano Mar 4, 2025

Choose a reason for hiding this comment

Lucas-Prates Mar 18, 2025

Choose a reason for hiding this comment

Lucas-Prates Mar 29, 2025

Choose a reason for hiding this comment

Lucas-Prates commented Mar 31, 2025

Gui-FernandesBR left a comment

Choose a reason for hiding this comment

Copilot AI left a comment

Choose a reason for hiding this comment

Gui-FernandesBR Apr 7, 2025

Choose a reason for hiding this comment

Gui-FernandesBR Apr 7, 2025

Choose a reason for hiding this comment

Lucas-Prates commented Nov 28, 2024 •

edited

Loading

codecov bot commented Nov 28, 2024 •

edited

Loading

Lucas-Prates commented Feb 24, 2025 •

edited

Loading