WIP: 1080 enable search metadata

[longshuicy]

Metadata is a dynamic field, making plain text keyword searches not possible. After experimenting with several approaches without success (detailed below), I have added an additional field metadata_stringify to convert the JSON into a string and cast it to text to enable keyword searches. The original metadata field ispreserved for advanced searches.

To Test:

1. Drop Volume

2. Create Metadata Definitions and Metadata:

   • Examples: Abstract, LatLon, DateTime

3. Keyword Search:

   • Search by metadata key, e.g., "abstract", "latitude"
   • Search by metadata value, e.g., "your metadata value"
   • Test both the search bar on the top bar and the search page
   • Make sure public search functionality

4. Advanced Search:

   • Use correct syntax and field names
   • Examples:
     • metadata.abstract:marker
     • metadata.latitude:[-70 TO -60]
     • metadata.longitude:[70 TO 90]

Screenshots:

---

Things I Have Tried That Did Not Work:

1. Updating the Mapping to Cast the Metadata Field to Text

   {
     "mappings": {
       "properties": {
         "metadata": {
           "type": "object",
           "dynamic": true,
           "properties": {
             "*": {
               "type": "text"
             }
           }
         }
       }
     }
   }

2. Wildcard Search

   curl -X GET "localhost:9200/clowder/_search" -H 'Content-Type: application/json' -d'
   {
     "query": {
       "query_string": {
         "query": "test search",
         "fields": ["metadata.*"]
       }
     }
   }
   '

[ddey2]

@longshuicy I looked at your approach. It looks okay.

I was wondering if you've tried putting 'default_field' in field_name and "*" in field name value. (look here)

search_query = {
"query": {
"query_string": {
"query": "John OR Title OR Tech",
"default_field": "*"
}
}
}

[tcnichol]

I did populate fake data. Looks like most of the metadata created is lat and lon, so I can verify that lat and lon is working for both basic and advanced search.

Will be checking other types soon.