CodexBloom - Programming Q&A Platform

AWS S3 Select Not Returning Expected Results When Querying CSV Content

πŸ‘€ Views: 71 πŸ’¬ Answers: 1 πŸ“… Created: 2025-06-09
aws s3 boto3 sql csv Python

I am trying to use AWS S3 Select to query a CSV file stored in an S3 bucket, but I'm not getting the expected results. The CSV file has the following structure: ``` name,age,city Alice,30,New York Bob,25,Los Angeles Charlie,35,Chicago ``` I wrote a query to select names of people older than 28, but it returns an empty result set even though I expect it to return 'Alice' and 'Charlie'. Here’s the code I’m using to execute the S3 Select: ```python import boto3 s3_client = boto3.client('s3') bucket_name = 'my-bucket' object_key = 'path/to/myfile.csv' query = "SELECT s.name FROM S3Object s WHERE s.age > 28" response = s3_client.select_object_content( Bucket=bucket_name, Key=object_key, ExpressionType='SQL', Expression=query, InputSerialization={ 'CSV': { 'FileHeaderInfo': 'Use' # This indicates that the first row is a header } }, OutputSerialization={ 'CSV': {} } ) for event in response['Payload']: if 'Records' in event: print(event['Records']['Payload'].decode('utf-8')) ``` I’ve checked the CSV file format and it is properly formatted in S3. I also ensured that the `FileHeaderInfo` is set to 'Use'. However, I keep getting an empty response. I tried running the same query directly in the AWS console using the 'Query S3 Select' feature, and it returns the expected results. Could there be something wrong with how I'm configuring the request in Boto3? Any insights or suggestions would be greatly appreciated.