Implement data validation

[clizbe]

Description

Please edit this description with new bullets as you think of them:

• If GraphAssetData.investable is an array of years, investment_cost should also be an array of equal length. (After multi-year investment is implemented.)
• Consistency checks for connections of assets: We shouldn't allow a transport asset to be the charging flow for a storage asset (but double-check).
• Check that all assets have a type associated (i.e., consumer, producer, conversion, storage, hub)
• Check that the flows that are defined as investable are also defined as transport
• If a storage asset has a missing is_seasonal, the default is false (non-seasonal storage).
• Check that units are consistent - either correct or throw a warning/error (this should include euro value year)
• If storage inflows are missing, then assign the default = 0
• Check that if there is a defined initial_storage_level, it must be between the asset's max/min limits. Also, we need to check that if there are max/min profiles, then you must also check that the initial value is between those values.
• Add validation described in Validate the efficiency for inflows and outflows for conversion assets #437
• Find all cases where the flow resolution can/must be inferred #453
• Check for minimum Tulipa graph criteria TulipaIO.jl#78
• Check if a group in that groups-data file has no members (e.g., assets) in the assets-data file and if an asset in the assets-data file has defined a group that it is not defined in the groups-data file.
• If the parameter unit_commitment of an asset is set to true, then the unit_commitment_method and the units_on_cost parameters cannot be missing
• if the storage asset is investable and it has the parameter use_binary_storage_method different from missing then the asset must have an investment_limit greater than zero
• for multi-year investment investable = false then initial_units > 0 => and only for the ones that year = commision_year
• Check conflicting constraints, such as the sum of minimum constraints is greater than the group maximum constraint
• No duplicate rows in input data #1091

Consider the defaults file Diego has drafted here in function get_default_values:
https://github.com/TulipaEnergy/Tulipa-OBZ-CaseStudy/blob/main/functions.jl

• Implement defaults in pipeline
• Document defaults in How-To (probably after Schema, or maybe within Schema?)

Related Issues

[abelsiqueira]

We just discussed adding arguments or options to define the use case instead of relying on the existence or not of specific files. I'm not sure how to translate that into a bullet point. I think it still needs some definition of the main use cases.

[clizbe]

Yes, depending on how many switches there are, this might be a file. But for now we want to have it as arguments to a function call, with defaults for the "regular" way of running and only specifying to change from the default.

This specification has two advantages:

• Easier programming for us, since we don't have to reverse engineer what the user wants based on their data
  • Not: "IF cost data present, AND bloop > 1, DO this - ELSEIF cost data present AND bloop = 0, DO something else"
  • Instead: "IF invest_flag = true, DO invest" - if missing data, error
• Clearer to the user what they're making Tulipa do
  • Not:
    • User: "I have bloop > 1, but it's not investing"
    • TulipaTeam: "Go look at the cookbook... you can see you need cost data to enable investment"
  • Instead: "ERROR: invest_flag = true, but cost data is missing" or "WARNING: Cost data is present, but invest_flag = false. To activate investment, set invest_flag = true and bloop > 1"

Dunno if that makes it more or less clear.