Elasticsearch 8.0 scenarios to Aggregate on Timestamp Fields with Timezone Offset Issues
I'm deploying to production and I'm sure I'm missing something obvious here, but I am working with a question with aggregation on timestamp fields in Elasticsearch 8.0... I have a dataset where timestamps are stored in UTC, and I want to perform date histogram aggregations with a specific timezone offset. However, when I apply the timezone in my request, the results seem incorrect and do not align with expected values. Here is the aggregation query I'm using: ```json { "size": 0, "aggs": { "daily_counts": { "date_histogram": { "field": "timestamp", "interval": "1d", "time_zone": "Europe/Berlin" } } } } ``` The `timestamp` field is mapped as follows: ```json { "mappings": { "properties": { "timestamp": { "type": "date", "format": "strict_date_optional_time||epoch_millis" } } } } ``` Despite using the correct timezone offset, I'm seeing that the buckets returned do not reflect the expected date ranges and often miss data because they are shifted incorrectly. For instance, data that should fall on a specific date appears in an adjacent bucket. I've tried different timezone formats and even checked the data itself to ensure that timestamps are correctly formatted in UTC. Additionally, I verified that the aggregation works correctly without the timezone parameter, returning accurate counts across the buckets. I'm not sure if this is a known scenario with the date_histogram aggregation in version 8.0 or if I've configured something incorrectly. Any insights on how to resolve this would be greatly appreciated! I'm working on a service that needs to handle this.