RFC: /config endpoint and filters

[annaCPR]

Description:

/config is an internal endpoint that provides static data to the navigator frontend such as geographies, corpus and organisation info, languages and document variants. However, the way that the json response is currently structured has resulted in a significant amount of traversal and mapping code (including the various theme configs) so that the relevant information can be extracted and used.

For example:

• filter labels can be found under taxonomy for each corpus type and must be individually extracted and potentially combined
• there is a whole function written to extracting geographies, countries and regions from the geographies object
• document variants are never used

Open Questions:

• Are we using the config endpoint in the best way?
• Is there any thing we can do to improve it?
• What is the future of the config endpoint?

Goals:

Make the /config endpoint response easier to use for the frontend.

Out of scope:

• changing the shape of the geographies response (will come back to this when subnational geographies are introduced)

Notes:

• gst1 is not using this endpoint

Proposal:

Separate the /config endpoint into individual endpoints for specific concerns (i.e. geographies, languages, etc.).
Define endpoints as static and dynamic. Static endpoints could be made available from S3 to improve performance.

Pros:

• performance improvement
• simplification of FE logic
• potentially reducing the data transformation happening in the BE as well

Cons:

• not insignificant amount of work required

[henry-cpr]

Commenting here during discussion meeting as too long for meet chat:
Transition Zero described their solution to geographies :
"We have a hierarchical representation for nodes which includes administrative geographical areas and real assets. Nodes have a NodeType that ranges from real assets and groups of real assets (e.g. power generating units, and power plants), to admin-1 geographies and grid-regions. Superior to these we have countries, two levels of supernational regions, and continents according to the UN's M49. Nodes have parents and children, which do not need to be strictly hierarchical (e.g. in the US grid regions are both hierarchically superior and inferior to admin-1 states). So all our nodes form a directed graph. NodeTypes also have an integer rank which allows us to summarise or traverse the node graph to a given level. All our data is indexed to these nodes - so we can roll-up or roll-down all our data along the node graph."

[annaCPR]

We have accepted the proposed solution.

[PatFawbertMills]

For me, representing the front-end, as a matter of priority I order the issues in this order:

1. Simplify and standardise the structure of the values/taxonomies that drive the display of the filters
2. Splitting the concerns into separate endpoints

For geographies I am far less concerned in the short-term because the existing structure is not too bad in my opinion and the code to transform it is because of how it is used, as opposed to transforming because the existing structure is unworkable. If that makes sense?