GCP Dataflow job implementing in 'Pending' state for over 24 hours with no clear errors
This might be a silly question, but I'm running a Dataflow job using the Apache Beam SDK (version 2.31.0) that reads from a Pub/Sub subscription and writes to BigQuery. The job seemed to initialize correctly, but it's been exploring in the 'Pending' state for over 24 hours with no behavior logs or clear indications of what might be going wrong. I've checked the Pub/Sub subscription, and there are messages available for processing, but nothing appears to be happening. I've tried the following troubleshooting steps: 1. Confirmed that I have sufficient permissions on both Pub/Sub and BigQuery. 2. Verified that the Dataflow worker service account is correctly set up and has the required IAM roles. 3. Checked the region settings to ensure that the Dataflow job is running in the same region as the Pub/Sub subscription and BigQuery dataset. 4. Looked at the monitoring dashboard in GCP to see if there are any errors or warnings related to the job, but nothing stands out. Here’s the snippet of the code I used to create the pipeline: ```python import apache_beam as beam from apache_beam.options.pipeline_options import PipelineOptions options = PipelineOptions( project='my-gcp-project', runner='DataflowRunner', region='us-central1', job_name='my-dataflow-job' ) with beam.Pipeline(options=options) as p: (p | 'ReadFromPubSub' >> beam.io.ReadFromPubSub(subscription='projects/my-gcp-project/subscriptions/my-subscription') | 'TransformData' >> beam.Map(lambda x: x.upper()) | 'WriteToBigQuery' >> beam.io.WriteToBigQuery( 'my-gcp-project:my_dataset.my_table', write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND) ) ``` I also included logging statements but they don’t seem to be triggered either. Any advice on how to diagnose or resolve this scenario would be greatly appreciated. Is there a way to force a re-deployment or check if the workers are even being allocated? Thanks in advance! Am I missing something obvious?