Scoping doc

[yulric]

No description provided.

[DougManuel]

Re: Import/Export
I wonder if being more opinionated about the metadata structure would orient this section, but I appreciate that this may be the next step.

• We'll store the metadata using a "sidecar" approach with three main tables: catalog, variable, variable_details.
• There is support for variable labels, value labels and tagged_NA and established use. i.e. haven and labelled packages. For import, much metadata for key users comes from SAS or Stata files that are imported using haven. We should allow users to import this metadata and the store it in variables and variable_details.
• There should be future provisions to be interoperable with other common metadata formats. I've added a conceptual diagram.

[DougManuel]

1. Wide versus long may be better situated as a future feature.
2. Do we want to include concepts such as logging and testing? Having clear expectations for the user about what will happen during transformations and the ability to debug, examine transformation effects, etc. Again, this could go in the next document.

[DougManuel]

I've made a few comments and suggestions.

We can close the PR whenever you want and address any outstanding issues with the specifications.

However, feel free to ask for more clarity.

[yulric]

1. Wide versus long may be better situated as a future feature.
2. Do we want to include concepts such as logging and testing? Having clear expectations for the user about what will happen during transformations and the ability to debug, examine transformation effects, etc. Again, this could go in the next document.

1. Agreed
2. Makes sense for those to go into the next document since its hard to provide a high level overview of what should go into logging and testing without getting into the specifications.