Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

paginate the defintions in the mongo store (fixes #254) #364

Merged
merged 2 commits into from
Jan 30, 2019

Conversation

dabutvin
Copy link
Member

For documents that have more than 1000 files, we will split them across different documents in the store.

This should be transparent to any callers of the store. we will get and set as one logical document.

Things to note:

  • adding a partitionKey set to the canonical coordinates so we can keep multiple pages of a single document on a single partition
  • storing currentPage and totalPages as metadata on the document in a _mongo property specific to this store and not surfaced as part of the definition
  • working with some larger documents surfaced some bugs in our harvested data around the file lists, we are now treating the clearlydefined tool as the source of truth for absolute file list

Copy link
Member

@jeffmcaffer jeffmcaffer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things to check out but otherwise good to go

business/aggregator.js Show resolved Hide resolved
providers/stores/mongo.js Outdated Show resolved Hide resolved
providers/stores/mongo.js Outdated Show resolved Hide resolved
@jeffmcaffer
Copy link
Member

Cool. Ship it when you get those changes in

@dabutvin dabutvin merged commit be97205 into clearlydefined:master Jan 30, 2019
@dabutvin dabutvin deleted the page-files branch January 30, 2019 22:57
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants