Elasticsearch 8.5 Querying with Wildcards in Filter Context Not Performing as Expected
I'm dealing with I'm updating my dependencies and After trying multiple solutions online, I still can't figure this out. I'm currently working with Elasticsearch 8.5 and trying to optimize my search queries that involve filtering using wildcards. I have a requirement to search for documents where a specific field matches a pattern, but when using a wildcard (`*`), the performance is significantly slower than anticipated, and sometimes it doesn't return any results even when I believe there should be matches. For example, I’m using the following query to filter documents based on a `username` field: ```json GET /my_index/_search { "query": { "bool": { "filter": { "wildcard": { "username": "john*" } } } } } ``` I've indexed around 10 million documents, and the `username` field is not analyzed. Specifically, it is set as a keyword type. Given this configuration, I expect the query to match any documents where the `username` starts with 'john'. However, executing this query either takes way too long or yields an empty result set when I know there should be matching documents. I have also tried switching to a `match` query with a prefix query, but the performance doesn't seem satisfactory. The only notable difference is that the `match` returns results faster, but with a less strict matching criterion. I've checked Elasticsearch logs for any signs of errors or issues, and there are none. I’ve run benchmarks using the `_search` endpoint, and the results are inconsistent. Sometimes I get a response in less than a second, while other times it takes over 10 seconds for the same query. I also tried increasing the `index.refresh_interval` to a higher value to allow for better batching, but that didn’t resolve the inconsistency. Could there be something wrong with how I’m using wildcards in the filter context? Are there any best practices or alternative approaches that I should consider in order to improve the performance of these wildcard searches? Is this even possible? My team is using Json for this application. Any help would be greatly appreciated!