GCP Pub/Sub message order not maintained in Dataflow despite using ordered delivery
I've been struggling with this for a few days now and could really use some help. I'm trying to configure I'm working on a personal project and I'm currently working on a GCP Dataflow pipeline that processes messages from a Pub/Sub topic where message order is crucial... I've enabled ordered delivery in the Pub/Sub subscription and set the appropriate acknowledgment configurations. However, I'm noticing that the output from my Dataflow job does not maintain the order of messages as expected. Here's a snippet of my Dataflow code where I read from the Pub/Sub topic: ```python import apache_beam as beam from apache_beam.options.pipeline_options import PipelineOptions class ProcessMessage(beam.DoFn): def process(self, message): # Simulate some processing logic yield message.decode('utf-8') options = PipelineOptions( runner='DataflowRunner', project='my-gcp-project', temp_location='gs://my-bucket/temp', region='us-central1', ) with beam.Pipeline(options=options) as p: (p | 'ReadFromPubSub' >> beam.io.ReadFromPubSub(subscription='projects/my-gcp-project/subscriptions/my-subscription') | 'ProcessMessages' >> beam.ParDo(ProcessMessage()) | 'WriteToBigQuery' >> beam.io.WriteToBigQuery( 'my-gcp-project:my_dataset.my_table', write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND) ) ``` I have confirmed that ordered delivery is enabled for the subscription settings. Despite this, I notice that the messages are being processed out of order in the BigQuery table. I've also checked the logs, and there don't seem to be any indications of parallel processing that might disrupt the order. I tried using a `GroupByKey` step to enforce order, but that has led to performance issues since my data volume is quite high. Is there a specific configuration I might be missing, or is there a different approach I should consider to maintain the order of messages in Dataflow? Any insights would be greatly appreciated! I recently upgraded to Python latest. What am I doing wrong?