Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Clean characters in columns #53

Open
sbehrends opened this issue Feb 12, 2019 · 1 comment
Open

Clean characters in columns #53

sbehrends opened this issue Feb 12, 2019 · 1 comment

Comments

@sbehrends
Copy link

While implementing csv-streamify I found out that there was a strange character (or wrongly encoded file) in the CSV that returned on an dirty column name.

example.csv

weird_char,ok_col,some_emoj🤙
1,1,1

The object returning included the exact same string as in the keys.

{
  'weird_char': 1,
  ok_col: 1,
  'some_emoj🤙': 1
}

My request is to add the ability to map or clean this column names to get a cleaner object, like

{
  weird_char: 1,
  ok_col: 1,
  some_emoj: 1
}

One solution would be to add a regex replace in https://github.com/klaemo/csv-stream/blob/master/csv-streamify.js#L56, example

if (state.lineNo === 0) {
  state._columns = state._line.map(col => col.replace(/[^a-zA-Z0-9_]/g,''))
  state.lineNo += 1
  reset()
  return
}

A nicer alternative would be to allow to pass a function as value for columns

const csv = require('csv-streamify')
const parser = csv({
  columns: (cols) => cols.map(col => col.replace(/[^a-zA-Z0-9_]/g,'')),
  objectMode: true,
})
@sbehrends
Copy link
Author

I see that this feature request looks alike to #40

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant