Towards Sustainable Modules 📦

[daviddias]

As it is known by everyone that has been using or contributing to the IPFS project, the number of modules (aka packages) has been increasing significantly and we expect to see it growing given the modular plug and play design that IPFS, libp2p and IPLD have.

Having code organized in modules with clear designed interfaces is one of the key design features that enables IPFS to be so adaptable and it also enables everyone to contribute to different parts of the system without having to understand it all (climb the the whole abstraction mountain).

Now we are starting to see some challenges that result with this approach that are making hard the life of the maintainers and users that want to track some threads of development that span over multiple modules. It is becoming very expensive to maintain a module as it requires of lot of human review and context present and also documentation is scattered and not easy searchable.

Towards solving this, I want to explore, gather and propose ways in which we focus on enabling more developers to make better choices when pushing contributions, that lead maintainers are less required to review PRs until it is absolutely necessary.

The goal is to strive for a golden standard for how modules are set up and maintained, focused on letting more people contribute and take ownership and reducing the the time between initial contribution to shipping. In the end, this would be part of the InterPlanetary Community Contributing Guidelines.

Here are some of the things that I've been thinking (some might be focused on JS land):

• Always at 100% Code Coverage - This way, any future contributor will be confident that their change won't affect other parts of the same module
• Semantic Release through a service such as http://semantic-release.org
• Auto generated API documentation
• Code Linting checks
• Code Quality checks
• OSS Hygiene (make sure that commits are signed)
• Changelogs
• Commit linting
• Parallel CI - Instead of having huge batch of tests testing every runtime and configuration, we should have multiple CI configs to test the runtimes individually so that those tests can run in parallel
• Every module should have a Lead Maintainer and a Co-Lead Maintainer. Any of the Co-Leads should be comfortable to take any patch and minor upgrades, major should require vetting of both. This also facilitates transition.
• Use Github's branch protection to avoid any fast or unchecked contributions
• Reproducible Builds (for JS, reproducible Browser Builds)
• Moaar

Next steps

We need to gather more shortcomings and pain-points that we can provide remedies for, while at the same time learning from other projects and understand what we can incorporate to IPFS. I invite everyone to share their experience and research in this issue.

The next step after that is to start (or reboot) the InterPlanetary Contributing Guidelines by converting the existing ones (https://github.com/ipfs/community/blob/master/contributing.md, https://github.com/ipfs/community/blob/master/contribution-guidelines.md, https://github.com/ipfs/community/blob/master/docs-styleguide.md, https://github.com/ipfs/community/blob/master/go-contribution-guidelines.md and https://github.com/ipfs/community/blob/master/js-project-guidelines.md) into a Gitbook (or alike) format so that all of these can live in one place. Once we have a place to store this information, we can then start applying these best practices across the module base.

References and sources of Inspiration

• Modern Modules - https://medium.com/libraries-io/what-does-a-sustainable-open-source-project-look-like-bf9b8cf824f8
• What does a sustainable open source project look like? https://medium.com/libraries-io/what-does-a-sustainable-open-source-project-look-like-bf9b8cf824f8

[jonnycrunch]

@diasdavid Certainly what would be helpful is to start with a code orientation and documentation of the role that each module does in the bigger picture. It took me a couple of weeks to walk thru the code and understand what certain modules does, such as libp2p, blockservice, blockstore, keystore and what each of them do.

[dryajov]

I would also add a bullet point for a test inventory here. We're getting to a point where we have modules with hundreds (thousands?) of tests. Some of those tests are intentionally skipped, some are skipped depending on environment - this makes it hard to track and detect which tests should be executed and which are intentionally skipped for a particular run.

Having an inventory and automatic enforcement through AEgir should help preventing tests from being skipped inadvertently and also give us a central place to keep track of our test suits.

Ideally, we shouldn't skip any tests, but in practice it's really hard to attain, specially when you have a big and complex codebase with many interdependent modules that might be at different levels of maturity and targeting various environments.

I'm not %100 sure that this is the correct way of tackling this issue, and if there are better ways - ie, bite the bullet and have %100 coverage across the board, I'd love to hear about them.

Anyways, great to see this issue!

Always at 100% Code Coverage - This way, any future contributor will be confident that their change won't affect other parts of the same module

👍 Actually we can shoot for something even better, especially for building blocks libraries: 100% mutation coverage

Semantic Release through a service such as http://semantic-release.org

Mmh skeptical of using an external service. How trustworthy are they? How good is their security game? Who's their hoster? Ideally we can do this through our CI on a trusted worker machine.

Auto generated API documentation

The docs portal will take care of it for both go and js, including search :D ipfs-inactive/docs#43

OSS Hygiene (make sure that commits are signed)

Nah let's not -- we regularly do things that cause commit signatures to get lost: rebase, squash, subtree-merge. We should sign tags though, and make it part of release automation (aegir/gx).

start with a code orientation and documentation of the role that each module does in the bigger picture

This is tracked in ipfs-inactive/docs#26

---

To all other points mentioned I just give a simple thumbs up 👍 :)

[victorb]

Certainly what would be helpful is to start with a code orientation and documentation of the role that each module does in the bigger picture

Introduction material and a developer "quickstart" would be great to have, especially since the ecosystem of different modules are already so vast.

I would also add a bullet point for a test inventory here. We're getting to a point where we have modules with hundreds (thousands?) of tests.

Yes please, testing has to be improved. Not more tests, but better tests and a proper separation between integration and unit tests. Also having standard test utils for reuse between different tests are a need. I'll do a write-up together with this quarters OKR on how we can improve testing in general.

Mmh skeptical of using an external service. How trustworthy are they? How good is their security game? Who's their hoster? Ideally we can do this through our CI on a trusted worker machine.

Semantic-release is not a external service but rather a tool that automates package releasing. From their website:

semantic-release automates the whole package release workflow including: determining the next version number, generating the release notes and publishing the package.

Would remove some maintenance burden from aegir.

[Mr0grog]

Notes/proposed plan from dev conference deep dive on sustainable modules (will repost in nicer written form later):

[Mr0grog]

OK, here’s a hopefully clearer writeup of the above: in yesterday’s deep dive session, @mikeal, @victorbjelkholm, and I walked through and discussed the main points and suggestions from the original post here. The conversation was largely JS-centric, but there are definitely several items that are generalizable across all of IPFS.

These are items that we felt were reasonably actionable:

1. Keep expanding the lead maintainers approach that has started in a lot of the JS projects

2. Every lead maintainer should shoot for a level of coverage they think is “reasonable” in their project (there are strong differences of opinion among people as to whether 100% coverage is a good goal; see The New Internet #297 for example)

   • Make sure all maintainers know how to monitor coverage on codecov (since we have it hooked up for all projects)
   • We need to clearly document how to set up and configure AEgir and the Jenkinsfile in our projects (this needs an owner; possibly @victorbjelkholm)

3. Set up branch protection on all repos so nobody can push directly to master.

4. Clean up contribution guidelines (this is also mentioned above).

   • Concrete and immediately actionable:
     • Rename CONTRIBUTING-2.md to indicate it is the more in-depth doc for repeat contributors who want to be involved, committers, and maintainers.
     • Both contributing and CONTRIBUTING-2.md are long and intimidating for their purpose. Make sure both have a TL;DR section at the top that simply lists specific actions contributors should take without the deeper explanation below.
   • More fuzzy: explore ideas from original post about moving all the contributor guides and styleguides to a gitbook, to the docs repo, or somewhere potentially more organized.

5. We are going to start doing automated releases for some of the smaller modules. If that works well, we’ll start working our way up to bigger ones. There are some concerns about reproducibility and handling NPM’s unreliability for packages with lots of dependencies (e.g. js-ipfs). @victorbjelkholm is going to work on this.

6. We need more automated checking for…

   • Code linting (already done in Jenkins as a PR status)
   • Code quality (@victorbjelkholm is interested in exploring this)
   • Possibly checking that all exported functions have doc comments (this doesn’t necessarily mean generating and publishing docs from them). We aren’t totally sure if this is feasible and need someone to explore it/write a proof-of-concept.

7. Try making our automated checks (above) use GitHub Checks instead of just PR statuses so they are more visible and actionable. (@victorbjelkholm volunteered for this)

8. Automate changelog generation (this is actually done)

9. Explore tooling to help maintainers review commit messages (since automated changelogs depend on commit messages now). Some ideas:

   • Bot/CI script that makes comments in PRs or messages maintainers about things that might signal a need to add a changelog message (like changes to existing tests)? We need help exploring possible heuristics here.
   • Require merge commits to have an explicit indicator about changelog worthiness? (i.e. if we don’t need changelog entries for the PR, the commit message has to explicitly say that somehow.)

10. Write a “project lifecycle” timeline that describes what practices or requirements are critical for projects at different levels of maturity. This will inform what of the above kinds of automated checks we should be doing (@mikeal is already working on this)

    • One of the concerns here is that there could be practices that are really helpful for mature projects but are problematic for experimental or immature projects. (e.g. could published docs could signal that things are more stable than they are?)

11. Have a new policy around what orgs repos live in: we want to try to stop making new repos in the ipfs and ipfs-shipyard orgs. Instead, contributors should create them in their own account and should not move them to “official” orgs until a major IPFS project starts depending on them. We want to lower the burden of “officialness” on experimental work (and the burden of searching through so many repos throughout the IPFS org for consumers of our modules).

12. JS projects need to resolve the bikeshed around how async work is managed internally. All our packages that aren’t top-level (probably everything but js-ipfs and js-ipfs) shouldn’t worry about exposing an API that supports callbacks and promises and streams and pull-streams, etc. We’ll only do that for the interfaces of the top-level projects.