Elasticsearch 8.5 Document Version Conflicts When Using Concurrent Bulk Updates
I'm working on a personal project and Does anyone know how to Hey everyone, I'm running into an issue that's driving me crazy. I'm experiencing issues with document version conflicts when performing concurrent bulk updates in Elasticsearch 8.5. I have a use case where multiple services are trying to update the same documents simultaneously, and I'm getting a lot of `VersionConflictEngineException` errors. My bulk update requests look something like this: ```json { "update": { "_id": "1", "_index": "my_index" } } { "doc": { "field": "value1" } } { "update": { "_id": "1", "_index": "my_index" } } { "doc": { "field": "value2" } } ``` Iām using the official Elasticsearch client for Java, and I send these bulk requests asynchronously. It appears that when one update is in progress, another update interferes, leading to the version conflict. In the client, I'm catching the exception like this: ```java try { BulkResponse bulkResponse = client.bulk(bulkRequest, RequestOptions.DEFAULT); } catch (ElasticsearchException e) { if (e.status() == RestStatus.CONFLICT) { System.out.println("Version conflict occurred: " + e.getMessage()); } } ``` I've tried using the `retry_on_conflict` parameter in the update request, but it doesn't seem to resolve the issue. Here's how I implemented it: ```json { "update": { "_id": "1", "_index": "my_index", "retry_on_conflict": 3 } } ``` Still, the conflicts keep happening, and when I try to log the conflicted documents, I only get the latest error without details on the previous attempts. Is there a better way to handle concurrent updates in Elasticsearch to avoid these version conflicts? Am I missing a best practice in using bulk updates or handling retries effectively? Is there a better approach? I'm using Java 3.9 in this project. What's the best practice here? I recently upgraded to Java LTS. How would you solve this?