Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Provide "changes" API endpoint #200

Closed
6 of 7 tasks
dlebrero opened this issue Jan 22, 2020 · 4 comments
Closed
6 of 7 tasks

Provide "changes" API endpoint #200

dlebrero opened this issue Jan 22, 2020 · 4 comments
Assignees
@iperdomo

Comments

@dlebrero
Copy link
Contributor

dlebrero commented Jan 22, 2020 •
edited by iperdomo
Loading

  • Auth
  • Auth Perf
  • Deploy
  • Datapoint changes/deletes
  • "Long pull"
  • Change Flow to push Form.status
  • Survey data
dlebrero added a commit that referenced this issue Jan 22, 2020
@dlebrero
[#200] Typo
0624404
dlebrero added a commit that referenced this issue Jan 22, 2020
@dlebrero
[#200] form instances are loaded
ae6bf60
dlebrero added a commit that referenced this issue Jan 22, 2020
@dlebrero
[#200] Be explicit on what we expose
0ed8bf8
dlebrero added a commit that referenced this issue Jan 22, 2020
@dlebrero
[#200] Mapcat list of form-instances
32d2352
dlebrero added a commit that referenced this issue Jan 22, 2020
@dlebrero
[#200] s/form-changes/form-changed/ for consistency
3556736
dlebrero added a commit that referenced this issue Jan 22, 2020
@dlebrero
[#200] Allow to change the sync API code without (refresh)
a16da36
dlebrero added a commit that referenced this issue Jan 22, 2020
@dlebrero
[#200] Remove table-name and event-log-server config
9bb4f22
@iperdomo iperdomo self-assigned this Jan 22, 2020
dlebrero added a commit that referenced this issue Jan 23, 2020
@dlebrero
[#200] Remove HugSQL and upgrade eftest
1b359e2
@iperdomo iperdomo mentioned this issue Jan 23, 2020
Closed
iperdomo added a commit that referenced this issue Jan 23, 2020
@iperdomo
[#200] Use DefaultJavaSSLFactory for connecting to the database
bfc194a 
Avoid the problem:
/root/.postgresql/root.crt (No such file or directory)
iperdomo added a commit that referenced this issue Jan 23, 2020
@iperdomo
[#200] DataPoint changes
2609f76
iperdomo added a commit that referenced this issue Jan 23, 2020
@iperdomo
[#200] Recover namespace declaration
ccbcfc7
@iperdomo
Copy link
Collaborator

iperdomo commented Jan 24, 2020 •
edited
Loading

"Long pull"

I'm thinking on caching the HTTP 204 request instead of blocking a thread in nginx or the JVM.

It's an easier solution. We can return Cache-Control: max-age=3600 (1h) header from Clojure and nginx will honor it (if proxy_cache is enabled)

@dlebrero any thoughts?

References:

dlebrero added a commit that referenced this issue Jan 24, 2020
@dlebrero
[#200] Remove some duplication
37903df
dlebrero added a commit that referenced this issue Jan 24, 2020
@dlebrero
[#200] Deleted datapoint should not appear in changed
3bcd1bc
@dlebrero
Copy link
Contributor Author

"Long pull"

I'm thinking on caching the HTTP 204 request instead of blocking a thread in nginx or the JVM.

It's an easier solution. We can return Cache-Control: max-age=3600 (1h) header from Clojure and nginx will honor it (if proxy_cache is enabled)

@dlebrero any thoughts?

Seems reasonable and a lot simpler. I assume the cache key will be the user email + url?

@iperdomo
Copy link
Collaborator

Seems reasonable and a lot simpler. I assume the cache key will be the user email + url?

yes, email+request_uri

iperdomo added a commit that referenced this issue Jan 24, 2020
@iperdomo
[#200] Datapoint changes use the same shape as normal API
80fbc24 
* We also introduce the change to convert ids into Strings for delete*
  events.
iperdomo added a commit that referenced this issue Jan 27, 2020
@iperdomo
[#200] Respond HTTP 204 when no more changes are available
bafa6ac 
The proxy server (nginx) will cache this response for 1h. Useful for
_eager_ clients re-requesting the same thing over and over again.
The value of 1h is arbitrary and we can adjust it.

A client can bypass the proxy by using a request header `Pragma: no-cache`
iperdomo added a commit that referenced this issue Jan 28, 2020
@iperdomo
[#200] Reorder and clean up CI build process
c31a538
iperdomo added a commit that referenced this issue Jan 28, 2020
@iperdomo
[#200] Disallow /sync with Keycloak based authentication
bda49a9
iperdomo added a commit that referenced this issue Jan 29, 2020
@iperdomo
[#200] Use Cache-Control header from upstream to control caching
eb71a36 
* Test caching as part of the CI build
iperdomo added a commit that referenced this issue Jan 29, 2020
@iperdomo
[#200] Use a different network name for cache testing
bcfab40
iperdomo added a commit that referenced this issue Jan 29, 2020
@iperdomo
[#200] Decrease cache ttl to 60s
c701ed0
iperdomo added a commit that referenced this issue Jan 29, 2020
@iperdomo
[#200] Use a a single open connection across request
c440b6d
iperdomo added a commit that referenced this issue Jan 29, 2020
@iperdomo
[#200] Add test for X-Cache-Status BYPASS
51bb3eb
iperdomo added a commit that referenced this issue Jan 29, 2020
@iperdomo
[#200] Split changes function
4bcf804
iperdomo added a commit that referenced this issue Jan 29, 2020
@iperdomo
[#200] Increase the values for cache_lock_timeout|cache_lock_age
724a4fb 
* It's quite unlikely that the same user (part of the cache key)
  is requesting the same URL at the same time
iperdomo added a commit that referenced this issue Jan 30, 2020
@iperdomo
[#200] Remove extra empty lines
ab1a29c
iperdomo added a commit that referenced this issue Jan 30, 2020
@iperdomo
[#200] Move test cache to its own script
ebf9416
iperdomo added a commit that referenced this issue Jan 30, 2020
@iperdomo
[#200] Remove Pragma: no-cache capability
7645c5a
iperdomo added a commit that referenced this issue Jan 30, 2020
@iperdomo
[#200] Use with-db-connection macro
b1714c1 
Instead of creating the connection and then creating a `{:connection conn}`
map when needed, we can use the with-db-connection macro
iperdomo added a commit that referenced this issue Jan 30, 2020
@iperdomo
[#200] Refactor nested (if) to (cond)
aa5407e
iperdomo added a commit that referenced this issue Jan 30, 2020
@iperdomo
Merge pull request #203 from akvo/issue/200-caching
e69a16a 
[#200] Respond HTTP 204 when no more changes are available
iperdomo added a commit that referenced this issue Jan 30, 2020
@iperdomo
[#200] Recover cache testing
cae7020
iperdomo added a commit that referenced this issue Jan 30, 2020
@iperdomo
[#200] Adds support for exposing Survey changes
3645daa
dlebrero added a commit that referenced this issue Jan 30, 2020
@dlebrero
Merge pull request #205 from akvo/issue/200-survey
ed16cb0 
[#200] Adds support for exposing Survey changes
iperdomo added a commit that referenced this issue Feb 3, 2020
@iperdomo
[#200] Include surveyId as part of formChanged response
5d549da
iperdomo added a commit that referenced this issue Feb 3, 2020
@iperdomo
[#200] Filter out events related to Folders
c1d4f75 
* Fixes the typo in spec definition
iperdomo added a commit that referenced this issue Feb 3, 2020
@iperdomo
[#200] Add authorization on forms related events
de553ae
iperdomo added a commit that referenced this issue Feb 4, 2020
@iperdomo
[#200] Remove /check_permission endpoint
2486573
iperdomo added a commit that referenced this issue Feb 4, 2020
@iperdomo
[#200] Filter changes by authorized forms
292c7b9 
* formChanged|formDeleted will show only data the user has access to
iperdomo added a commit that referenced this issue Feb 4, 2020
@iperdomo
[#200] Move ds/with-datastore outside list/filter calls
5a43668 
* We expect that RemoteAPI to be installed once per thread (request),
instead of installing/uninstalling several times
iperdomo added a commit that referenced this issue Feb 5, 2020
@iperdomo
[#200] Filter forms/form-instances/surveys/data-points by authorization
155e341
iperdomo added a commit that referenced this issue Feb 6, 2020
@iperdomo
[#200] Refactor the way of handling authorization
6d80792 
- We can't authorize deletes (as the main object on which we authorize may be gone)
- Provide candidates for forms and surveys ids when trying to list-and-authorize
iperdomo added a commit that referenced this issue Feb 6, 2020
@iperdomo
[#200] Fix authorization logic based on last review
81380ff 
- Get rid of using `nil` value when loading forms and surveys. This
  particular one masked a different logic bug. By using explicit ids
  when fetching objects we harden the function "contract".
iperdomo added a commit that referenced this issue Feb 6, 2020
@iperdomo
[#200] Removes the possibility of passing nil as parameter
b78d77a
@dlebrero
Copy link
Contributor Author

Can we close this?

@iperdomo iperdomo closed this as completed Jul 2, 2020
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@iperdomo iperdomo

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@iperdomo @dlebrero