Elasticsearch 8.5 Indexing guide with Bulk Requests and Version Conflicts
Does anyone know how to I'm deploying to production and I'm experimenting with I've tried everything I can think of but I'm working with an scenario while trying to use bulk indexing in Elasticsearch 8.5. When I send a bulk request, I sometimes get version conflicts for documents that I am sure are not being modified concurrently. Here's the code snippet I'm using to perform the bulk indexing: ```python from elasticsearch import Elasticsearch, helpers es = Elasticsearch(['http://localhost:9200']) docs = [ {'_index': 'my_index', '_id': '1', '_source': {'field': 'value1'}}, {'_index': 'my_index', '_id': '2', '_source': {'field': 'value2'}}, ] # This is the bulk indexing request response = helpers.bulk(es, docs) print(response) ``` When I run this code, I occasionally see errors like this in the response: ``` {'errors': True, 'items': [{'index': {'_index': 'my_index', '_id': '1', 'status': 409, 'behavior': {'type': 'version_conflict_engine_exception', 'reason': 'Version conflict, document already exists (indexing)', 'index_uuid': 'xyz', 'shard': '0', 'node': 'abc'}}}]} ``` I have tried adding a retry mechanism to handle the version conflict by catching the exception and resending the request, but it still fails with the same behavior after the retry. I've also checked the cluster health and it shows that everything is green. Additionally, I'm using a custom `version_type` of `external` in my mappings, thinking it might help, but it has not resolved the scenario. The mapping for my index is as follows: ```json { "mappings": { "properties": { "field": { "type": "text" } } } } ``` Could this scenario be related to how the bulk request is formed, or could it be a deeper scenario with the Elasticsearch cluster's state? Any advice on how to properly handle or diagnose these version conflicts would be greatly appreciated. What would be the recommended way to handle this? For reference, this is a production microservice. I'd really appreciate any guidance on this. I'm coming from a different tech stack and learning Python. Any ideas how to fix this? Thanks for any help you can provide!