duplicate tokens in user_defined_symbols param cause RuntimeError

[poedator]

when training with user_defined_symbols parameters I ran into this error
RuntimeError: Internal: src/trainer_interface.cc(717) [insert_meta_symbol(w, ModelProto::SentencePiece::USER_DEFINED)]
which was not very informative. Fortunately I soon realized that this was caused by duplicate symbols in the user_defined_symbols list.
You may want to make this error message more explicit and/or add separate check for non-unique symbols.

To reproduce run similar code with duplicate symbols in user_defined_symbols param list:

spm.SentencePieceTrainer.Train(input='manywordswithfreq.tsv',
                               user_defined_symbols=['LOL', 'LOL', 'KEK'],
                               input_format='tsv',
                               hard_vocab_limit=False,
                               model_prefix='m',
                               vocab_size=10000)```

[taku910]

Thank you for the report. We will make the error message more descriptive in the next release.

[taku910]

Fixed in v0.1.98