Case insensitive mapping not working.

[bevankoopman]

I'm getting case sensitive mapping results even though I'm using an index built with --lowercase.

To reproduce.

• index with UMLS 2018AA and -L (lowercase)
• test code:

from quickumls import *

matcher = QuickUMLS('/quickumls-data-2018AA')
text ='Name.'
for p in matcher.match(text, best_match=True, ignore_syntax=False):
    for q in p:
        print(q)

Result is:

{'start': 0, 'end': 4, 'ngram': 'Name', 'term': 'name', 'cui': 'C0027365', 'similarity': 1.0, 'semtypes': {'T170'}, 'preferred': 1}
{'start': 0, 'end': 4, 'ngram': 'Name', 'term': 'name', 'cui': 'C1547383', 'similarity': 1.0, 'semtypes': {'T170'}, 'preferred': 1}
{'start': 0, 'end': 4, 'ngram': 'Name', 'term': 'name', 'cui': 'C1554107', 'similarity': 1.0, 'semtypes': {'T170'}, 'preferred': 1}
{'start': 0, 'end': 4, 'ngram': 'Name', 'term': 'name', 'cui': 'C2599456', 'similarity': 1.0, 'semtypes': {'T201'}, 'preferred': 1}

But if text = 'Name.' is changed to text = 'name.' result is empty.

Similarly, if text = 'Patient Name.' then result is:

{'start': 0, 'end': 12, 'ngram': 'Patient Name', 'term': 'patient name', 'cui': 'C1299487', 'similarity': 1.0, 'semtypes': {'T033'}, 'preferred': 1}

And if `text = 'Patient name.', the result is:

{'start': 0, 'end': 7, 'ngram': 'Patient', 'term': 'patient', 'cui': 'C1550655', 'similarity': 1.0, 'semtypes': {'T031'}, 'preferred': 1}
{'start': 0, 'end': 7, 'ngram': 'Patient', 'term': 'patient', 'cui': 'C1578485', 'similarity': 1.0, 'semtypes': {'T170'}, 'preferred': 1}
{'start': 0, 'end': 7, 'ngram': 'Patient', 'term': 'patient', 'cui': 'C1578486', 'similarity': 1.0, 'semtypes': {'T170'}, 'preferred': 1}

(Note the name part is ignored.)

[soldni]

Hi Bevan,

That's strange. Will take a look and see if I can reproduce. Thanks for the bug report!

-Luca

[soldni]

Turns out SpaCy folks got a bit to aggressive with the definition of stopwords, as "name" is now included in their list. Switched to NLTK in QuickUMLS 1.2.4 to fix this issue. Sorry it took me a while to get to this!