Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Go filesystem implementations refactoring #27

Merged
merged 32 commits into from
Jul 11, 2023

Conversation

djdv
Copy link
Owner

@djdv djdv commented Nov 26, 2022

These are the various changes made to the Go fs.FS implementations we currently have.
As well as a new work in progress for IPFS' MFS API, which will be used to handle writable methods in KeyFS+IPNS later.
Currently being tested against the IPFS Files API, which itself will likely need its own implementation (later).

Edit: removed from this change set. We'll do writable stuff after the staging/refactor branch makes it to master.


PR meta:
Adds on top of #26 originally derived from #20
Initially targeting j/cgofuse-refactoring, which should target staging/refactor after that's merged.

@djdv djdv mentioned this pull request Nov 26, 2022
@djdv djdv force-pushed the j/cgofuse-refactoring branch 2 times, most recently from 115f41b to 5101a8a Compare December 2, 2022 21:16
@djdv djdv force-pushed the j/cgofuse-refactoring branch 2 times, most recently from 4a3a892 to 1dc571e Compare December 20, 2022 02:54
Base automatically changed from j/cgofuse-refactoring to staging/refactor December 21, 2022 02:42
@djdv djdv force-pushed the j/fs-impl-refactoring branch from 74a87e7 to 5c3542e Compare December 21, 2022 04:11
@djdv djdv force-pushed the j/fs-impl-refactoring branch from 957f536 to c8f7bf5 Compare April 3, 2023 06:39
@djdv djdv force-pushed the staging/refactor branch from 0e2c51f to 69e6a04 Compare May 19, 2023 11:33
@djdv djdv force-pushed the j/fs-impl-refactoring branch from c8f7bf5 to bac3e78 Compare May 19, 2023 11:33
Copy link
Owner Author

@djdv djdv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preliminary partial review pass from last week.

@djdv djdv requested a review from aschmahmann May 19, 2023 13:05
djdv added 17 commits June 21, 2023 19:37
This only worked for canonical IPFS paths, not IPNS.
Standard implies we're supposed to return entries or an error, but never
both.
Previously, at the tail of a directory, we'd return ents + EOF.
Now we follow the same convention as standard `os` and `fstest`.
Standard implies we're supposed to return entries or an error, but never
both.
Previously, at the tail of a directory, we'd return ents + EOF.
Now we follow the same convention as standard `os` and `fstest`.
djdv and others added 10 commits June 21, 2023 19:37
- Add caching mechanism to IPFS, IPNS, and PinFS
- Change constructors for all, specifically how options are handled
- Change field parser logic for mount-point file
- Various minor bugfixes
PinFS: 104 pointer bytes -> 80
nodeInfo: 56 pointer bytes -> 32
Previously we were peeking the cache to avoid inflating the hit counter
in commonly seen sequences like `stat;stat;open`. However, it's not
likely to be detrimental / cause unwanted evictions if calls to `stat`
update the hit counter for a file. It may actually allow some paths to
fall out of cache if they're merely being queried/monitored frequently,
which is undesirable.
@djdv
Copy link
Owner Author

djdv commented Jul 11, 2023

This was mainly reviewed offline, and tested during the development of #28
Some issues were found and have been broken out to be resolved later, but the foundation here seems to work well enough to move forward with a merge.

@djdv djdv merged commit 4b264f5 into staging/refactor Jul 11, 2023
@djdv djdv deleted the j/fs-impl-refactoring branch July 11, 2023 14:50
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants