OCI Object Storage: implementing Large File Uploads Using Python SDK
I'm writing unit tests and I'm sure I'm missing something obvious here, but I'm working on a project and hit a roadblock. I tried several approaches but none seem to work... I'm working with an scenario when trying to upload large files (over 5GB) to OCI Object Storage using the Python SDK. The SDK seems to handle smaller files perfectly, but when I attempt to upload a file of this size, I encounter a `ClientError: 413 Request Entity Too Large` behavior. I've already tried breaking the file into smaller chunks and uploading each chunk separately, but I'm unsure about how to implement this correctly with the SDK. Here's the basic code snippet I'm using for the upload: ```python import oci config = oci.config.from_file() # Uses default config file oobject_storage_client = oci.object_storage.ObjectStorageClient(config) bucket_name = 'my_bucket' namespace_name = oobject_storage_client.get_namespace().data file_path = 'path/to/large_file.dat' with open(file_path, 'rb') as f: object_name = 'large_file.dat' response = oobject_storage_client.put_object(namespace_name, bucket_name, object_name, f) print(response) ``` I've attempted to upload the file in chunks by reading the file in binary mode and using the `put_object` method to send each chunk separately, but I need to seem to get it right. I'm not sure how to manage the offsets and the sizes correctly. Any guidance on handling large uploads effectively using the OCI Python SDK would be greatly appreciated! Is there a specific method in the SDK for multipart uploads that I should be using? Any examples or best practices would be helpful. What am I doing wrong? This is part of a larger service I'm building. My development environment is Ubuntu. Is there a better approach? I'm working in a Ubuntu 22.04 environment. What would be the recommended way to handle this? This is for a web app running on Windows 11. Any ideas how to fix this?