GCP BigQuery Data Ingestion Failures with JSON Files from Cloud Storage using Python Client Library
I'm relatively new to this, so bear with me. I'm trying to ingest multiple JSON files stored in a GCP Cloud Storage bucket into BigQuery using the Python client library. However, I keep working with an `Invalid JSON` behavior when executing the load job. My JSON files are structured as follows: ```json { "name": "John Doe", "age": 30, "city": "Los Angeles" } ``` I have ensured that each JSON file has only one object per line, as recommended by the BigQuery documentation. The ingestion code I am using is: ```python from google.cloud import bigquery client = bigquery.Client() uri = "gs://my_bucket/my_json_files/*.json" job_config = bigquery.LoadJobConfig( source_format=bigquery.SourceFormat.NEWLINE_DELIMITED_JSON, ) load_job = client.load_table_from_uri( uri, "my_dataset.my_table", job_config=job_config, ) load_job.result() # Waits for the job to complete. print(f'Loaded {load_job.output_rows} rows into my_dataset.my_table.') ``` Despite this, I get the following behavior message when the job fails: ``` Invalid JSON: Unexpected character at line 1 column 1 ``` I have double-checked the file format and tried running the ingestion with a single JSON file, but I still get the same behavior. I also verified that the IAM roles for the service account have the necessary permissions. Is there any specific formatting or configuration I might be overlooking? Also, should I be using a different source format for my JSON files? Any guidance would be greatly appreciated. Thanks in advance! For reference, this is a production desktop app. I appreciate any insights!