Chronic insertions view in need of optimization

[k1o0]

The chronic-insertions list endpoint is extremely slow and produces over 9000 queries! The detail view is also inefficient, requiring 111 queries. I attempted to improve the setup_eager_loading method of ChronicProbeInsertionListSerializer with the following:

queryset = queryset.select_related('model', 'session', 'session__subject', 'session__lab')
queryset = queryset.prefetch_related('session__projects')
return queryset.order_by('-session__start_time')

However this does not appear to improve the situation.

What's more, the chronic probe insertion contains subject, lab, and name fields in addition to a probe_insertion field containing the same fields within session_info.

[k1o0]

Commit 6688bd1 provides some improvements to the various serializers. Some of my changes made only a marginal difference, others were substantial. NB: The below times are from single queries so are noisy!

one.alyx.rest('insertions', 'list')
"GET /insertions HTTP/1.1" 200
6 queries, 3.46 seconds -> 5 queries, 3.55 seconds

one.alyx.rest('insertions', 'read', id='cabf05dc-43d2-4ecb-8609-d8ddbd46f3ca')
"GET /insertions/cabf05dc-43d2-4ecb-8609-d8ddbd46f3ca HTTP/1.1" 200
50 queries, 0.51 seconds -> 47 queries, 0.39 seconds

one.alyx.rest('chronic-insertions', 'list')
"GET /chronic-insertions HTTP/1.1" 200
>9000 queries, 19.99 seconds -> 256 queries, 4.18 seconds

one.alyx.rest('chronic-insertions', 'read', id='ea67d80d-a23b-4c86-a2b2-9eddfc101dc3')
"GET /chronic-insertions/ea67d80d-a23b-4c86-a2b2-9eddfc101dc3 HTTP/1.1" 200
90 queries, 1.49 seconds -> 7 queries, 0.25 seconds

one.alyx.rest('imaging-stack', 'list')
"GET /imaging-stack HTTP/1.1" 200
907 queries, 3.47 seconds > 7 queries, 1.28 seconds

one.alyx.rest('fields-of-view', 'list')
"GET /fields-of-view HTTP/1.1" 200
1251 queries, 2.22 seconds > 6 queries, 0.53 seconds

one.alyx.rest('fields-of-view', 'read', id='952f3b4e-e501-4ac1-a3a8-38bbcbfead6d')
"GET /fields-of-view/952f3b4e-e501-4ac1-a3a8-38bbcbfead6d HTTP/1.1" 200
7 queries, 0.27 seconds -> 7 queries, 0.87 seconds

one.alyx.rest('fov-location', 'list')
"GET /fov-location HTTP/1.1" 200
503 queries, 1.15 seconds -> 4 queries, 0.23 seconds

one.alyx.rest('fov-location', 'read', id='8b6d2019-a0a9-47c8-94a2-f2572db28d77')
"GET /fov-location/8b6d2019-a0a9-47c8-94a2-f2572db28d77 HTTP/1.1" 200
N/A -> 3 queries, 0.11 seconds

[k1o0]

The chronic-insertions list endpoint is still slow and comprises hundreds of queries. This is likely because of the subject, lab, and name fields in addition to a probe_insertion field containing the same fields within session_info. This could surely be optimized further...