Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Rework Dataset File Schema creation functionality #398

Closed
jamesjefferies opened this issue Apr 3, 2017 · 0 comments
Closed

Rework Dataset File Schema creation functionality #398

jamesjefferies opened this issue Apr 3, 2017 · 0 comments

Comments

@jamesjefferies
Copy link
Contributor

Given more time the Dataset File Schema creation functionality would be refactored and tidied up. There are a number of areas where it is messy namely:

  1. Sometimes the DatasetFileSchemaCreation service is used, sometimes it isn't
  2. Schemas get parsed and validated using both CSVLint and the JSON Table Schema gem. There is some tidying up do be done there
  3. The parsed schema is stored in the model as JSON, but isn't always used for validation, as for example, CSVLint wants a URL to parse rather than a string, whereas JSON Table Schema handles any IO object.
  4. The interaction with inferred schemas isn't always as clean as it should be
  5. The forms use url_in_s3 as a reference, whereas actually this field could be a reference to an uploaded file, such as when Javascript is not switched on. It would help for this to be consolidated across all file uploads to be file_reference or similar.
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

No branches or pull requests

3 participants