OCI Object Storage Performance Issues When Uploading Large Files with Python SDK
I'm working through a tutorial and I've been struggling with this for a few days now and could really use some help. I'm working on a project and hit a roadblock. I'm experiencing significant performance issues when trying to upload large files (around 1GB) to OCI Object Storage using the Python SDK version 3.9.0. The upload time is exceptionally long, and it seems to be affected by the size of the file. For smaller files (under 100MB), the upload is almost instantaneous, but for larger files, it takes several minutes, which is not acceptable for my application. Here's a snippet of my upload code: ```python import oci config = oci.config.from_file() storage_client = oci.object_storage.ObjectStorageClient(config) bucket_name = 'my_bucket' object_name = 'large_file.dat' with open('path/to/large_file.dat', 'rb') as file: response = storage_client.put_object(bucket_name, object_name, file) print(response) ``` I've tried increasing the timeout settings in the client configuration and also using multipart uploads, but I’m still not seeing any improvements. Here’s how I attempted the multipart upload, but it’s giving me the same lengthy upload times: ```python multipart_upload = storage_client.create_multipart_upload(bucket_name, object_name) # Upload parts part_size = 1024 * 1024 * 10 # 10MB parts = [] with open('path/to/large_file.dat', 'rb') as file: part_number = 1 while True: part = file.read(part_size) if not part: break response = storage_client.upload_part(bucket_name, object_name, multipart_upload.id, part_number, part) parts.append(response) part_number += 1 # Commit the multipart upload storage_client.commit_multipart_upload(bucket_name, object_name, multipart_upload.id, parts) ``` I also checked the network performance, and it seems stable. Is there a best practice or configuration that I might be missing to optimize large file uploads to OCI Object Storage? Any insights or recommendations would be greatly appreciated! I'm working on a web app that needs to handle this. Is there a better approach? I'm working on a API that needs to handle this. Thanks in advance! I'm using Python 3.11 in this project. Is there a better approach?