Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

(fix) O3-3015: Fetch all the queue entries to support filters on the frontend #1124

Merged
merged 5 commits into from
May 13, 2024

Conversation

vasharma05
Copy link
Member

Requirements

  • This PR has a title that briefly describes the work done including the ticket number. If there is a ticket, make sure your PR title includes a conventional commit label. See existing PR titles for inspiration.
  • My work conforms to the OpenMRS 3.0 Styleguide and design documentation.
  • My work includes tests or is validated by existing tests.

Summary

Currently, the queue entries only fetch the top 50 results and the filters only work on those top 50 results. Now, we will be fetching all the queue entries using useSWRInfinite to fetch all the results and display them accordingly.

This PR also includes some UI improvements on the Table Toolbar.

Screenshots

Before

image image

After

image image

Related Issue

https://issues.openmrs.org/browse/O3-3015

Other

Copy link
Contributor

github-actions bot commented May 7, 2024

Size Change: -1.31 kB (-0.04%)

Total Size: 3.52 MB

ℹ️ View Unchanged
Filename Size Change
packages/esm-active-visits-app/dist/130.js 220 kB 0 B
packages/esm-active-visits-app/dist/255.js 2.21 kB 0 B
packages/esm-active-visits-app/dist/271.js 762 B 0 B
packages/esm-active-visits-app/dist/277.js 13.5 kB 0 B
packages/esm-active-visits-app/dist/316.js 42.9 kB 0 B
packages/esm-active-visits-app/dist/319.js 683 B 0 B
packages/esm-active-visits-app/dist/382.js 1.15 kB 0 B
packages/esm-active-visits-app/dist/410.js 50.1 kB 0 B
packages/esm-active-visits-app/dist/460.js 784 B 0 B
packages/esm-active-visits-app/dist/574.js 588 B 0 B
packages/esm-active-visits-app/dist/588.js 6.66 kB 0 B
packages/esm-active-visits-app/dist/635.js 1.15 kB 0 B
packages/esm-active-visits-app/dist/644.js 762 B 0 B
packages/esm-active-visits-app/dist/729.js 3.1 kB 0 B
packages/esm-active-visits-app/dist/757.js 694 B 0 B
packages/esm-active-visits-app/dist/784.js 2.63 kB 0 B
packages/esm-active-visits-app/dist/788.js 586 B 0 B
packages/esm-active-visits-app/dist/807.js 918 B 0 B
packages/esm-active-visits-app/dist/833.js 732 B 0 B
packages/esm-active-visits-app/dist/879.js 2.94 kB 0 B
packages/esm-active-visits-app/dist/main.js 68 kB 0 B
packages/esm-active-visits-app/dist/openmrs-esm-active-visits-app.js 3.34 kB 0 B
packages/esm-appointments-app/dist/130.js 220 kB 0 B
packages/esm-appointments-app/dist/152.js 257 B 0 B
packages/esm-appointments-app/dist/255.js 2.22 kB 0 B
packages/esm-appointments-app/dist/271.js 1.97 kB 0 B
packages/esm-appointments-app/dist/303.js 258 B 0 B
packages/esm-appointments-app/dist/309.js 868 B 0 B
packages/esm-appointments-app/dist/319.js 1.89 kB 0 B
packages/esm-appointments-app/dist/390.js 26.1 kB 0 B
packages/esm-appointments-app/dist/4.js 1.77 kB 0 B
packages/esm-appointments-app/dist/459.js 42.3 kB 0 B
packages/esm-appointments-app/dist/460.js 2.07 kB 0 B
packages/esm-appointments-app/dist/57.js 252 kB 0 B
packages/esm-appointments-app/dist/574.js 1.86 kB 0 B
packages/esm-appointments-app/dist/588.js 6.65 kB 0 B
packages/esm-appointments-app/dist/591.js 16.9 kB 0 B
packages/esm-appointments-app/dist/592.js 9.5 kB 0 B
packages/esm-appointments-app/dist/644.js 1.97 kB 0 B
packages/esm-appointments-app/dist/729.js 3.1 kB 0 B
packages/esm-appointments-app/dist/757.js 1.74 kB 0 B
packages/esm-appointments-app/dist/784.js 2.63 kB 0 B
packages/esm-appointments-app/dist/788.js 1.68 kB 0 B
packages/esm-appointments-app/dist/807.js 2.31 kB 0 B
packages/esm-appointments-app/dist/833.js 1.97 kB 0 B
packages/esm-appointments-app/dist/main.js 306 kB 0 B
packages/esm-appointments-app/dist/openmrs-esm-appointments-app.js 3.42 kB 0 B
packages/esm-patient-list-management-app/dist/130.js 220 kB 0 B
packages/esm-patient-list-management-app/dist/255.js 2.21 kB 0 B
packages/esm-patient-list-management-app/dist/271.js 1.55 kB 0 B
packages/esm-patient-list-management-app/dist/319.js 1.52 kB 0 B
packages/esm-patient-list-management-app/dist/382.js 1.15 kB 0 B
packages/esm-patient-list-management-app/dist/460.js 1.7 kB 0 B
packages/esm-patient-list-management-app/dist/574.js 1.34 kB 0 B
packages/esm-patient-list-management-app/dist/588.js 6.66 kB 0 B
packages/esm-patient-list-management-app/dist/591.js 16.9 kB 0 B
packages/esm-patient-list-management-app/dist/635.js 1.15 kB 0 B
packages/esm-patient-list-management-app/dist/644.js 1.55 kB 0 B
packages/esm-patient-list-management-app/dist/716.js 4.75 kB 0 B
packages/esm-patient-list-management-app/dist/729.js 3.1 kB 0 B
packages/esm-patient-list-management-app/dist/740.js 97.6 kB 0 B
packages/esm-patient-list-management-app/dist/757.js 1.5 kB 0 B
packages/esm-patient-list-management-app/dist/784.js 2.64 kB 0 B
packages/esm-patient-list-management-app/dist/788.js 1.34 kB 0 B
packages/esm-patient-list-management-app/dist/807.js 1.84 kB 0 B
packages/esm-patient-list-management-app/dist/833.js 1.58 kB 0 B
packages/esm-patient-list-management-app/dist/995.js 22 kB 0 B
packages/esm-patient-list-management-app/dist/main.js 124 kB 0 B
packages/esm-patient-list-management-app/dist/openmrs-esm-patient-list-management-app.js 3.3 kB 0 B
packages/esm-patient-registration-app/dist/130.js 220 kB 0 B
packages/esm-patient-registration-app/dist/152.js 262 B 0 B
packages/esm-patient-registration-app/dist/255.js 2.21 kB 0 B
packages/esm-patient-registration-app/dist/271.js 2.01 kB 0 B
packages/esm-patient-registration-app/dist/303.js 260 B 0 B
packages/esm-patient-registration-app/dist/319.js 1.99 kB 0 B
packages/esm-patient-registration-app/dist/460.js 2.12 kB 0 B
packages/esm-patient-registration-app/dist/481.js 6.75 kB 0 B
packages/esm-patient-registration-app/dist/537.js 2.43 kB 0 B
packages/esm-patient-registration-app/dist/574.js 1.7 kB 0 B
packages/esm-patient-registration-app/dist/591.js 16.9 kB 0 B
packages/esm-patient-registration-app/dist/644.js 2.01 kB 0 B
packages/esm-patient-registration-app/dist/676.js 6.59 kB 0 B
packages/esm-patient-registration-app/dist/686.js 92 kB 0 B
packages/esm-patient-registration-app/dist/729.js 3.1 kB 0 B
packages/esm-patient-registration-app/dist/735.js 464 B 0 B
packages/esm-patient-registration-app/dist/757.js 2.07 kB 0 B
packages/esm-patient-registration-app/dist/783.js 36.4 kB 0 B
packages/esm-patient-registration-app/dist/784.js 2.64 kB 0 B
packages/esm-patient-registration-app/dist/788.js 1.7 kB 0 B
packages/esm-patient-registration-app/dist/807.js 2.43 kB 0 B
packages/esm-patient-registration-app/dist/833.js 1.97 kB 0 B
packages/esm-patient-registration-app/dist/879.js 2.94 kB 0 B
packages/esm-patient-registration-app/dist/main.js 130 kB 0 B
packages/esm-patient-registration-app/dist/openmrs-esm-patient-registration-app.js 3.34 kB 0 B
packages/esm-patient-search-app/dist/130.js 220 kB 0 B
packages/esm-patient-search-app/dist/255.js 2.21 kB 0 B
packages/esm-patient-search-app/dist/271.js 899 B 0 B
packages/esm-patient-search-app/dist/319.js 857 B 0 B
packages/esm-patient-search-app/dist/382.js 1.15 kB 0 B
packages/esm-patient-search-app/dist/460.js 924 B 0 B
packages/esm-patient-search-app/dist/574.js 732 B 0 B
packages/esm-patient-search-app/dist/584.js 21.7 kB 0 B
packages/esm-patient-search-app/dist/588.js 6.66 kB 0 B
packages/esm-patient-search-app/dist/591.js 16.9 kB 0 B
packages/esm-patient-search-app/dist/635.js 1.15 kB 0 B
packages/esm-patient-search-app/dist/644.js 899 B 0 B
packages/esm-patient-search-app/dist/729.js 3.1 kB 0 B
packages/esm-patient-search-app/dist/757.js 864 B 0 B
packages/esm-patient-search-app/dist/778.js 23.3 kB 0 B
packages/esm-patient-search-app/dist/784.js 2.63 kB 0 B
packages/esm-patient-search-app/dist/788.js 726 B 0 B
packages/esm-patient-search-app/dist/807.js 1.03 kB 0 B
packages/esm-patient-search-app/dist/833.js 867 B 0 B
packages/esm-patient-search-app/dist/main.js 48.2 kB 0 B
packages/esm-patient-search-app/dist/openmrs-esm-patient-search-app.js 3.3 kB 0 B
packages/esm-service-queues-app/dist/130.js 220 kB 0 B
packages/esm-service-queues-app/dist/152.js 262 B 0 B
packages/esm-service-queues-app/dist/233.js 1.73 kB 0 B
packages/esm-service-queues-app/dist/255.js 2.22 kB 0 B
packages/esm-service-queues-app/dist/258.js 2.31 kB 0 B
packages/esm-service-queues-app/dist/263.js 3.54 kB 0 B
packages/esm-service-queues-app/dist/271.js 4.12 kB 0 B
packages/esm-service-queues-app/dist/303.js 261 B 0 B
packages/esm-service-queues-app/dist/319.js 3.54 kB 0 B
packages/esm-service-queues-app/dist/328.js 2.99 kB 0 B
packages/esm-service-queues-app/dist/382.js 1.15 kB 0 B
packages/esm-service-queues-app/dist/389.js 0 B -1.89 kB (removed) 🏆
packages/esm-service-queues-app/dist/425.js 2.09 kB 0 B
packages/esm-service-queues-app/dist/460.js 4.4 kB 0 B
packages/esm-service-queues-app/dist/466.js 2.58 kB 0 B
packages/esm-service-queues-app/dist/470.js 52.1 kB +236 B (+0.46%)
packages/esm-service-queues-app/dist/50.js 3.13 kB 0 B
packages/esm-service-queues-app/dist/574.js 3.66 kB 0 B
packages/esm-service-queues-app/dist/588.js 6.66 kB 0 B
packages/esm-service-queues-app/dist/591.js 16.9 kB 0 B
packages/esm-service-queues-app/dist/635.js 1.15 kB 0 B
packages/esm-service-queues-app/dist/644.js 4.12 kB 0 B
packages/esm-service-queues-app/dist/694.js 2.74 kB 0 B
packages/esm-service-queues-app/dist/696.js 570 B 0 B
packages/esm-service-queues-app/dist/729.js 3.1 kB 0 B
packages/esm-service-queues-app/dist/738.js 554 B 0 B
packages/esm-service-queues-app/dist/757.js 3.54 kB 0 B
packages/esm-service-queues-app/dist/784.js 2.63 kB 0 B
packages/esm-service-queues-app/dist/788.js 3.54 kB 0 B
packages/esm-service-queues-app/dist/806.js 1.6 kB 0 B
packages/esm-service-queues-app/dist/807.js 4.83 kB 0 B
packages/esm-service-queues-app/dist/833.js 4.09 kB 0 B
packages/esm-service-queues-app/dist/909.js 1.66 kB 0 B
packages/esm-service-queues-app/dist/94.js 157 kB 0 B
packages/esm-service-queues-app/dist/940.js 21.4 kB 0 B
packages/esm-service-queues-app/dist/981.js 3.03 kB 0 B
packages/esm-service-queues-app/dist/main.js 212 kB +306 B (+0.14%)
packages/esm-service-queues-app/dist/openmrs-esm-service-queues-app.js 3.34 kB +39 B (+1.18%)

compressed-size-action

@chibongho chibongho requested review from mseaton and mogoodrich May 7, 2024 18:42
Copy link
Contributor

@chibongho chibongho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this locally and found a couple issues.

  • The assumption that the chunkSize == 50 is incorrect for PIH distros, and it causes the number of fetched entries to be incorrect. (Some of them get double-fetched)
  • The subsequent fetches to get the next chunk does not seem to get triggered immediately after the first fetch. I was able to trigger it by switching focus out of the page and back in. See video. (Note that the correctly patient count should be 603, as seen in the metrics card)
OpenMRS.10.-.-.Microsoft.Edge.2024-05-07.16-09-23.mp4

if (value != null) {
searchParam.append(key, value?.toString());
const [totalCount, setTotalCount] = useState<number>();
const chunkSize = 50;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we can assume the chunk size to be 50 for everyone. I believe PIH's distro configured the chunk size to be 500. Maybe we can determine the chunk size after the first fetch?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't aware of this scenario, thank you @chibongho for pointing this out!

if (data?.[0]?.data?.totalCount && data[0].data.totalCount !== totalCount) {
setTotalCount(data[0].data.totalCount);
}
}, [data?.[0], totalCount, setTotalCount]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to put the setter function in the dependency array? I've seen this pattern elsewhere in this code base, but I don't understand why.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both the value and the setter are put in the dependency array, AFAIK

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chibongho It's mostly about consistency. We've had issues with deps that should've be included being omitted, leading to inconsistent behavior, so its easiest across a code-base developed by a large number of contributors with varying degrees of familiarity with React, etc., just to include everything in the deps array.

Technically, setters from useState() don't need to be in the dependency array because part of the contract of useState() is that the setter remain consistent across renders. However, that's not necessarily the case for functions returned from other hooks

@@ -29,9 +30,8 @@ const ClearQueueEntries: React.FC<ClearQueueEntriesProps> = ({ queueEntries }) =

return (
<Button
size="sm"
size={isDesktop(layout) ? 'sm' : 'lg'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for this!

@chibongho
Copy link
Contributor

In this PR, the useQueueEntries() hook is modified to make multiple fetches as needed to get all queue entries. I think this is fine for this use case as, typically, the number of active queue entries is in the order of ~10s or ~100s.

It would be nice for use cases like this to have a usePaginate hook where the totalCount and the first batch of results are loaded on first fetch, and it delays fetching subsequent entries until the user paginates to the corresponding page in the DataTable. Thoughts? @ibacher @denniskigen @mseaton

@ibacher
Copy link
Member

ibacher commented May 7, 2024

I agree. That sounds like a useful utility to have. That said it wouldn't fix this part: "the filters only work on those top 50 results".

@vasharma05
Copy link
Member Author

It would be nice for use cases like this to have a usePaginate hook where the totalCount and the first batch of results are loaded on first fetch, and it delays fetching subsequent entries until the user paginates to the corresponding page in the DataTable. Thoughts? @ibacher @denniskigen @mseaton

Yes, this is exactly what we want, but if you see, the main action of the table is to

  1. Sort
  2. Search the rows

Since this is not yet supported by the REST endpoint, we have to fetch all the entries and then handle the sorting and searching the whole results.
If we just fetch the first result, the search will only happen on the first batch, not the whole data.

@vasharma05
Copy link
Member Author

Hi @chibongho !
Can you please provide me the PIH server where I can test my implementation better, since there is not much data in the dev3 right now?
Thanks!

@ibacher
Copy link
Member

ibacher commented May 8, 2024

I will say that making the queue table sortable, in my view, is a mistake. Queues are supposed to be implicitly ordered. I'd rather we applied filtering to allow the users to zoom in on particular statuses, etc.

@chibongho
Copy link
Contributor

Hi @chibongho ! Can you please provide me the PIH server where I can test my implementation better, since there is not much data in the dev3 right now? Thanks!

@vasharma05 just DM'd you the server + credentials

I will say that making the queue table sortable, in my view, is a mistake. Queues are supposed to be implicitly ordered. I'd rather we applied filtering to allow the users to zoom in on particular statuses, etc.

We have no sorting in the queue table (and plan to keep it that way). Vineet's point stands regarding the search filtering though.

@vasharma05
Copy link
Member Author

Hi @chibongho!
I've made the changes, please re-review the changes.
Thanks!

Copy link
Contributor

@chibongho chibongho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works locally. Thanks!

@vasharma05 vasharma05 merged commit 68eacab into main May 13, 2024
6 checks passed
@vasharma05 vasharma05 deleted the fix/O3-3015 branch May 13, 2024 06:15
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants