Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat(wallet) save/load collectibles user handled state in DB #13903

Merged
merged 2 commits into from
Mar 27, 2024

Conversation

stefandunca
Copy link
Contributor

@stefandunca stefandunca commented Mar 8, 2024

Closes #13313, #13971 and Updates #13312

  • rebase on top of master
  • fix collectibles type

Add a separation layer for save/load/clear to ManageTokensModel so that we can save/load from external sources.
The separate layer is composed of JSON as protocol, a set of signals and slots for interface. The implementation forwards data to current QSettings for storybook and nim controllers for the app.

Groups are not handled in this PR

Note: I auto formatted some C++ files by mistake using vscode, please use ignore spaces in diff for easier review :(

@stefandunca stefandunca requested review from caybro and dlipicar March 8, 2024 13:52
@status-im-auto
Copy link
Member

status-im-auto commented Mar 8, 2024

Jenkins Builds

Click to see older builds (70)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ b652e71 #1 2024-03-08 13:59:03 ~6 min tests/nim 📄log
✔️ b652e71 #1 2024-03-08 13:59:52 ~6 min macos/aarch64 🍎dmg
✔️ b652e71 #1 2024-03-08 14:02:21 ~9 min macos/x86_64 🍎dmg
b652e71 #1 2024-03-08 14:04:00 ~11 min tests/ui 📄log
✔️ b652e71 #1 2024-03-08 14:07:49 ~15 min linux/x86_64 📦tgz
✔️ b652e71 #1 2024-03-08 14:23:03 ~30 min windows/x86_64 💿exe
✔️ a725576 #2 2024-03-08 19:17:34 ~4 min macos/aarch64 🍎dmg
✔️ a725576 #2 2024-03-08 19:19:18 ~6 min tests/nim 📄log
✔️ a725576 #2 2024-03-08 19:20:41 ~7 min macos/x86_64 🍎dmg
a725576 #2 2024-03-08 19:23:31 ~10 min tests/ui 📄log
✔️ a725576 #2 2024-03-08 19:27:41 ~14 min linux/x86_64 📦tgz
✔️ a725576 #2 2024-03-08 19:47:22 ~34 min windows/x86_64 💿exe
✔️ 19e87ef #3 2024-03-11 12:44:55 ~5 min macos/aarch64 🍎dmg
✔️ 19e87ef #3 2024-03-11 12:45:59 ~6 min tests/nim 📄log
✔️ 19e87ef #3 2024-03-11 12:48:44 ~8 min macos/x86_64 🍎dmg
✔️ 19e87ef #3 2024-03-11 12:51:09 ~11 min tests/ui 📄log
✔️ 19e87ef #3 2024-03-11 12:53:53 ~14 min linux/x86_64 📦tgz
✔️ 19e87ef #3 2024-03-11 13:05:41 ~25 min windows/x86_64 💿exe
✔️ da12104 #4 2024-03-11 19:19:07 ~4 min macos/aarch64 🍎dmg
✔️ da12104 #4 2024-03-11 19:20:46 ~6 min tests/nim 📄log
✔️ da12104 #4 2024-03-11 19:22:11 ~7 min macos/x86_64 🍎dmg
✔️ da12104 #4 2024-03-11 19:25:27 ~11 min tests/ui 📄log
✔️ da12104 #4 2024-03-11 19:29:43 ~15 min linux/x86_64 📦tgz
✔️ da12104 #4 2024-03-11 19:44:07 ~29 min windows/x86_64 💿exe
840351e #5 2024-03-12 20:49:38 ~2 min macos/aarch64 📄log
840351e #5 2024-03-12 20:51:22 ~4 min tests/ui 📄log
840351e #5 2024-03-12 20:51:55 ~4 min macos/x86_64 📄log
✔️ 840351e #5 2024-03-12 20:53:24 ~6 min tests/nim 📄log
840351e #5 2024-03-12 20:54:51 ~7 min linux/x86_64 📄log
840351e #5 2024-03-12 21:06:13 ~19 min windows/x86_64 📄log
✔️ 7b8a7a9 #8 2024-03-13 21:58:11 ~5 min macos/aarch64 🍎dmg
✔️ 7b8a7a9 #8 2024-03-13 21:58:47 ~5 min tests/nim 📄log
✔️ 7b8a7a9 #8 2024-03-13 22:02:21 ~9 min macos/x86_64 🍎dmg
✔️ 7b8a7a9 #8 2024-03-13 22:04:01 ~11 min tests/ui 📄log
✔️ 7b8a7a9 #8 2024-03-13 22:09:14 ~16 min linux/x86_64 📦tgz
✔️ 7b8a7a9 #8 2024-03-13 22:26:02 ~33 min windows/x86_64 💿exe
✔️ 7c2c4fb #9 2024-03-14 12:07:36 ~4 min macos/aarch64 🍎dmg
✔️ 7c2c4fb #9 2024-03-14 12:09:35 ~6 min tests/nim 📄log
✔️ 7c2c4fb #9 2024-03-14 12:11:13 ~8 min macos/x86_64 🍎dmg
✔️ 7c2c4fb #9 2024-03-14 12:13:43 ~10 min tests/ui 📄log
✔️ 73b3cdd #11 2024-03-14 12:20:49 ~5 min tests/nim 📄log
✔️ 73b3cdd #11 2024-03-14 12:21:37 ~6 min macos/aarch64 🍎dmg
✔️ 73b3cdd #11 2024-03-14 12:22:02 ~7 min macos/x86_64 🍎dmg
✔️ 73b3cdd #11 2024-03-14 12:26:51 ~11 min tests/ui 📄log
✔️ 73b3cdd #11 2024-03-14 12:29:26 ~14 min linux/x86_64 📦tgz
✔️ 73b3cdd #11 2024-03-14 12:45:13 ~30 min windows/x86_64 💿exe
✔️ c211c82 #12 2024-03-15 11:00:57 ~4 min macos/aarch64 🍎dmg
✔️ c211c82 #12 2024-03-15 11:04:11 ~7 min tests/nim 📄log
✔️ c211c82 #12 2024-03-15 11:04:22 ~7 min macos/x86_64 🍎dmg
✔️ c211c82 #12 2024-03-15 11:08:07 ~11 min tests/ui 📄log
✔️ c211c82 #12 2024-03-15 11:11:43 ~15 min linux/x86_64 📦tgz
✔️ c211c82 #12 2024-03-15 11:29:35 ~32 min windows/x86_64 💿exe
✔️ 8acdec6 #13 2024-03-18 09:16:08 ~4 min macos/aarch64 🍎dmg
✔️ 8acdec6 #13 2024-03-18 09:17:19 ~6 min tests/nim 📄log
✔️ 8acdec6 #13 2024-03-18 09:20:31 ~9 min macos/x86_64 🍎dmg
✔️ 8acdec6 #13 2024-03-18 09:22:47 ~11 min tests/ui 📄log
✔️ 8acdec6 #13 2024-03-18 09:25:47 ~14 min linux/x86_64 📦tgz
✔️ 8acdec6 #13 2024-03-18 09:43:54 ~32 min windows/x86_64 💿exe
✔️ 6d69658 #14 2024-03-19 10:42:41 ~6 min macos/aarch64 🍎dmg
✔️ 6d69658 #14 2024-03-19 10:45:25 ~9 min macos/x86_64 🍎dmg
✔️ 6d69658 #14 2024-03-19 10:50:36 ~14 min tests/nim 📄log
✔️ 6d69658 #14 2024-03-19 10:55:37 ~19 min tests/ui 📄log
✔️ 6d69658 #14 2024-03-19 10:58:21 ~22 min linux/x86_64 📦tgz
✔️ 6d69658 #14 2024-03-19 11:07:09 ~30 min windows/x86_64 💿exe
✔️ 49a4aeb #15 2024-03-25 21:12:23 ~6 min tests/nim 📄log
✔️ 49a4aeb #15 2024-03-25 21:15:01 ~8 min macos/aarch64 🍎dmg
✔️ 49a4aeb #15 2024-03-25 21:17:21 ~11 min tests/ui 📄log
✔️ 49a4aeb #15 2024-03-25 21:18:05 ~11 min macos/x86_64 🍎dmg
✔️ 49a4aeb #15 2024-03-25 21:20:20 ~14 min linux/x86_64 📦tgz
✔️ 49a4aeb #15 2024-03-25 21:39:39 ~33 min windows/x86_64 💿exe
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 94556db #16 2024-03-26 16:09:54 ~5 min macos/aarch64 🍎dmg
✔️ 94556db #16 2024-03-26 16:11:13 ~6 min tests/nim 📄log
✔️ 94556db #16 2024-03-26 16:12:24 ~8 min macos/x86_64 🍎dmg
✔️ 94556db #16 2024-03-26 16:16:41 ~12 min tests/ui 📄log
✔️ 94556db #16 2024-03-26 16:19:10 ~14 min linux/x86_64 📦tgz
✔️ 94556db #16 2024-03-26 16:37:13 ~32 min windows/x86_64 💿exe
✔️ 2536c66 #17 2024-03-27 10:12:09 ~4 min macos/aarch64 🍎dmg
✔️ 2536c66 #17 2024-03-27 10:13:46 ~6 min tests/nim 📄log
✔️ 2536c66 #17 2024-03-27 10:15:16 ~8 min macos/x86_64 🍎dmg
✔️ 2536c66 #17 2024-03-27 10:18:52 ~11 min tests/ui 📄log
✔️ 2536c66 #17 2024-03-27 10:22:49 ~15 min linux/x86_64 📦tgz
✔️ 2536c66 #17 2024-03-27 10:37:22 ~30 min windows/x86_64 💿exe

@stefandunca
Copy link
Contributor Author

stefandunca commented Mar 8, 2024

@caybro and @dlipicar I'm currently working on this task of redirecting the collectible settings from being stored in QSettings by the ManageTokensController C++ implementation to be stored in the DB (by calling updateCollectiblePreferences).
However, it seems it is becoming quite complicated and error prone and the initial estimation was "easy".

I'm planning to emit signals for save/load events and call the load/save of ManageTokensController with json data. However, I see that the c++ controller is used also for assets which have a different structure and options so the JSON has to be flexible and detect the source based on presence of some properties.

Could you please have a look at the current code and see if there is an easier way to do that that I might have missed?

Thanks

@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch 4 times, most recently from da12104 to 840351e Compare March 12, 2024 20:46
@stefandunca stefandunca marked this pull request as ready for review March 13, 2024 21:42
@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch from 840351e to aaf1e7a Compare March 13, 2024 21:43
@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch 3 times, most recently from 7b8a7a9 to 7c2c4fb Compare March 14, 2024 12:02
@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch 2 times, most recently from cb79f45 to 73b3cdd Compare March 14, 2024 12:14
Copy link
Contributor

@Cuteivist Cuteivist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

The implementation forwards data to current QSettings for storybook and nim controllers for the app.

I would love to have that note in the controller itself to not confuse developers why there are QSettings.

caybro
caybro previously requested changes Mar 15, 2024
Copy link
Member

@caybro caybro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I like the direction where this is going however...

I don't think it's desirable to merge this as-is at this point in the release cycle (during a bugfixing period) when this is clearly:

  • new feature
  • breaking change
  • incomplete (deals with collectibles only)

ui/StatusQ/src/wallet/tokendata.cpp Show resolved Hide resolved
ui/StatusQ/src/wallet/tokendata.cpp Show resolved Hide resolved
@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch from 73b3cdd to c211c82 Compare March 15, 2024 10:56
@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch from c211c82 to 8acdec6 Compare March 18, 2024 09:11
@lukaszso lukaszso self-requested a review March 18, 2024 17:21
Copy link
Contributor

@noeliaSD noeliaSD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice work, code LGTM!
Here my comments, there are just some questions or suggestions!

I cannot test the pr properly to ensure there are no regressions bc right now we have a bug on master that doesn't apply settings to the main wallet view. It should be solved in here: #14018 and it would be nice if you can rebase on top of that to be able to do some proper testing.

Then, related to the topic @caybro exposed, as I explained offline, we need to consider if this refactoring PR should be integrated on this estabilization phase or wait for the next milestone. Considering that the 2.28 rc branch will be create this Wednesday, maybe we can keep this pr open until the branch is created and integrate later, so this code will go to the 2.29 release. WDYT?

I don't really have strong opinion on it if we can do a full and proper test to ensure there are no regressions introduced and we also ensure the bug fixed in here #13984 is also integrated.

CC: @alexjba @micieslak

ui/StatusQ/src/wallet/tokendata.h Outdated Show resolved Hide resolved
ui/StatusQ/src/wallet/tokendata.h Show resolved Hide resolved
ui/StatusQ/src/wallet/tokendata.h Show resolved Hide resolved
Copy link
Contributor Author

@stefandunca stefandunca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot test the pr properly to ensure there are no regressions bc right now we have a bug on master that doesn't apply settings to the main wallet view. It should be solved in here: #14018 and it would be nice if you can rebase on top of that to be able to do some proper testing.

I see a lot of parallel work on token master. I talked with you, @Cuteivist and reach out to @caybro and @dlipicar but still end-up now with parallel refactoring and major changes. Where can I find the work streams on token management (mainly managetokenscontroller.cpp and managetokensmodel.h) so that I could stream the rebase, review, regression test and not drag it forever?

Then, related to the topic @caybro exposed, as I explained offline, we need to consider if this refactoring PR should be integrated on this estabilization phase or wait for the next milestone. Considering that the 2.28 rc branch will be create this Wednesday, maybe we can keep this pr open until the branch is created and integrate later, so this code will go to the 2.29 release. WDYT?

In my testing the functionality was kept and worked fine both in management view and display model. I trust your judgement and will follow with rebase and conflict after the parallel redesign for later on.

I don't really have strong opinion on it if we can do a full and proper test to ensure there are no regressions introduced ...

Testing is waiting for a rebase, however it make no sense to rebase until we merge the all conflicting changes.

and we also ensure the bug fixed in here #13984 is also integrated.

At first glance that change is implementing group ordering and conflicting with my changes. So I expect a rebase to be costly. As discussed privately, I will follow your collective decision and wait for all new features and fixes be in master, then rebase and adapt to the new changes.

@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch from 8acdec6 to 6d69658 Compare March 19, 2024 10:36
@stefandunca
Copy link
Contributor Author

@lukaszso I rebased on top of #14018

Copy link
Member

@micieslak micieslak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though the code is still quite complicated I think it's a move into a right direction. ManageTokensController seems to have too many responsibilities inside and should be split into smaller chunks, excluding persistence layer is really good first step here.

I see some benefits if we could merge it asap - it may safe us from handling migrations from qsettings to db-base storage in the future.

At first glance allowing controller to decide when to load/save makes it more complex than it should be, maybe we could reconsider it in the future.

In the long term I think there is refactor needed, similar as the one done for Profile Showcase Management. CollectiblesView and AssetsView should not depend on the controller at all. The order/visibility info could be attached to the main model using LeftJoinModel making things much easier and cleaner. But I'm aware it requires deeper analysis bc of grouping and additional transformations needed bc of that. Probably GroupingModel should be finished first (#12683) to make it doable.

ui/StatusQ/src/wallet/tokendata.cpp Show resolved Hide resolved

// TODO #13313: call the assets controller for all events
onRequestSaveSettings: (jsonData) => {
// savingStarted()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why in this case this started/finished pair is not used?

Copy link
Contributor Author

@stefandunca stefandunca Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are already done serialy in the internal saveToQSettings (as previously). The marker methods started/finished are just for external storage as discussed above.

@stefandunca
Copy link
Contributor Author

@lukaszso I rebased on top of #14018

@noeliaSD tested and reported that the fix for apply changes doesn't work with my changes. I will investigate this issue next week when I pick it up again.

@alexjba
Copy link
Contributor

alexjba commented Mar 19, 2024

Then, related to the topic @caybro exposed, as I explained offline, we need to consider if this refactoring PR should be integrated on this estabilization phase or wait for the next milestone. Considering that the 2.28 rc branch will be create this Wednesday, maybe we can keep this pr open until the branch is created and integrate later, so this code will go to the 2.29 release. WDYT?

I'd wait for the release branch on Wednesday before merging it, but no longer after that. It could land in 2.28.1 depending on how it will be branched out.

@anastasiyaig
Copy link
Contributor

this task is for 2.28 so we need to merge it before release branch cut. Other words - this change is needed for 2.28

@stefandunca
Copy link
Contributor Author

@micieslak I can't reply to your question so I answer in a comment

#13903 (comment)

Yeah, I also wonder why the emitter cannot call on his own this savingStarted/savingFinished pair?

The main reason is the inability to guarantee that a slot will answer to the save request, hence avoid unnecessary invalidation of the model (this way start and finish notify the controller of storage being active and mark the invalidation).

Another consideration was to be explicit and not to risk an inconstancy in case of connection type between onRequestSaveSettings and start/finish processing. However, now I see that this can still happen and is not guaranteed and your proposal would be as equally working given our simple integration (same thread affinity).

@caybro
Copy link
Member

caybro commented Mar 19, 2024

this task is for 2.28 so we need to merge it before release branch cut. Other words - this change is needed for 2.28

Why is it needed and what added value or fix does it bring to the user?

@anastasiyaig
Copy link
Contributor

@caybro no idea, i just checked the milestone for the task :) @iurimatias @alexandraB99 please clarify

@stefandunca
Copy link
Contributor Author

At first glance allowing controller to decide when to load/save makes it more complex than it should be, maybe we could reconsider it in the future.

I totally agree, the implementation should be refactored with more separation of concerns in mind. I found it outside my task requirements and tried to do an initial quick implementation that escalated in too many long debugging sessions but good learnings.

@lukaszso
Copy link

Tested after rebase and the "Apply to my wallet" worked at first albeit with a delay (it looked like the changes were reflected only after the wallet activity was refreshed?). Then after about 30 minutes of inactivity it stopped working again and i am unable to get it to apply the order again.

Other than this i did not find any other issues here.

@noeliaSD
Copy link
Contributor

I agree with @alexjba and @micieslak on landing this into 2.28.1. One of the reasons to consider that way is (apart from the starting point of reducing complexity on the controller's data management):

  • Michal comment --> it may safe us from handling migrations from qsettings to db-base storage in the future. Noelia comment --> Future that will come soon bc it's a product requirement afaik.

I don't see any risk if we can do a full testing and ensure the existing functionality is still working properly.

@caybro caybro dismissed their stale review March 19, 2024 15:41

Will retest/review

@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch 2 times, most recently from 49a4aeb to 94556db Compare March 26, 2024 16:04
Add a separation layer for save/load/clear to ManageTokensModel
so that we can save/load from external sources.
The separate layer is composed of JSON as protocol, a set of signals
and slots for interface. The implementation forwards data to current
QSettings for storybook and nim controllers for the app.

Updates #13313, #13312
It seems `%*` operator we use in the `rpc` generics doesn't use the
`serializedFieldName` marker and `type` was serialized as `itemType`
and value the enum name instead of `type` and integer value

Updates: #13971
@stefandunca stefandunca force-pushed the collectibles_settings_to_backend-13313 branch from 94556db to 2536c66 Compare March 27, 2024 10:06
@stefandunca
Copy link
Contributor Author

@lukaszso I just rebased on latest master and fixed all found issues. Can you please give it another try?

Copy link

@lukaszso lukaszso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and approved. I did not see any issues with assets/collectibles except for long wait for new ordering to be applied. It's not related to this change so i'll report it separately. LGTM :)

@stefandunca stefandunca merged commit b1f8c8e into master Mar 27, 2024
8 checks passed
@stefandunca stefandunca deleted the collectibles_settings_to_backend-13313 branch March 27, 2024 19:26
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Tokens Management] Missing collectibles settings integration with backend
9 participants