datacollector: store separate snapshots of model data per step #2129
Conversation
- Ensure `DataCollector` stores unique copies of the data at each model step. - Use `deepcopy` in `collect` method to avoid storing references that lead to each step holding the same data. - Fixes the issue where PropertyLayer data was stored identically for all steps, preventing accurate visualization and analysis of model progression.
|
Performance benchmarks:
|
|
@dylan-munson this PR should resolve your issue (I tested it locally and for me it does). You can install this branch with: pip install -U -e git+https://github.com/projectmesa/mesa@datacollector_deepcopy#egg=mesaThanks for reporting it and please let us know if this fixed it for you. |
EwoutH
changed the title
|
@EwoutH Thanks for looking into this; I will try this in a bit. Just to check, I will be able to return to the main branch of the mesa repo by just running a usual pip install update right? |
|
I can confirm that after installing this dev branch the issue has been fixed. Thank you for the help! I'll keep my eyes open for when this makes it into the main branch. |
| @@ -197,17 +198,19 @@ def collect(self, model): | |||
| for var, reporter in self.model_reporters.items(): | |||
| # Check if lambda or partial function | |||
| if isinstance(reporter, (types.LambdaType, partial)): | |||
| self.model_vars[var].append(reporter(model)) | |||
| self.model_vars[var].append(deepcopy(reporter(model))) | |||
There was a problem hiding this comment.
Needs a brief explanatory comment before this line. Otherwise someone might mistakenly remove the deepcopy.
|
@dylan-munson Thanks for validating that this fix resolves your issue! @rht Thanks for reviewing. I added the comment and resolved the merge conflict. Please squash when merging. |
|
Can't merge due to conflicts. |
|
Squash and merge worked. |
* datecollector: store separate snapshots of model data per step - Ensure `DataCollector` stores unique copies of the data at each model step. - Use `deepcopy` in `collect` method to avoid storing references that lead to each step holding the same data. - Fixes the issue where PropertyLayer data was stored identically for all steps, preventing accurate visualization and analysis of model progression. * datacollection: Add deepcopy comment
|
@dylan-munson I cherry picked this PR to the 2.3 maintenance branch, and tagged Mesa 2.3.1 which includes this fix. So you can now also use that version. |
…ctmesa#2129) * datecollector: store separate snapshots of model data per step - Ensure `DataCollector` stores unique copies of the data at each model step. - Use `deepcopy` in `collect` method to avoid storing references that lead to each step holding the same data. - Fixes the issue where PropertyLayer data was stored identically for all steps, preventing accurate visualization and analysis of model progression. * datacollection: Add deepcopy comment
This pull request addresses issue #2128, which involves incorrect data collection for
PropertyLayerobjects when using Mesa'sDataCollectormodule. Previously, theDataCollectorwas maintaining references to thePropertyLayerdata array, leading to every step showing identical data due to changes propagating backward.Changes Introduced:
collectmethod of theDataCollectorto usedeepcopyfor model-level data collection.Resolves #2128.