Rate Limiting implementing OCI API when Retrieving Large Sets of Resources Using Python SDK
I'm wondering if anyone has experience with I'm performance testing and This might be a silly question, but I'm wondering if anyone has experience with I've been struggling with this for a few days now and could really use some help. I'm currently working with a question when trying to retrieve a large number of resources from OCI using the Python SDK. The code I’m using is supposed to paginate through the results, but I keep hitting a rate limit exception after a few iterations, specifically the behavior message: `TooManyRequests: Rate limit exceeded`. My aim is to fetch all resources without running into this scenario. Here’s a snippet of my code: ```python import oci from oci.config import from_file from time import sleep config = from_file() client = oci.core.ComputeClient(config) compartment_id = 'your_compartment_ocid' def list_instances(compartment_id): instances = [] next_page = None while True: response = client.list_instances(compartment_id, page=next_page) instances.extend(response.data) next_page = response.next_page if next_page is None: break return instances instances = list_instances(compartmentment_id) print(f'Total instances: {len(instances)}') ``` I have tried adding a sleep time of 1 second between API calls to mitigate the rate limiting, but it doesn’t seem to help. I’ve also looked into the configuration settings for the SDK, but I can’t find any specific parameters for adjusting the rate limits. My current OCI Python SDK version is 2.45.0. Is there a best practice for handling rate limits while fetching data from OCI? Should I consider using a different approach or library to manage large sets of resource data? Any insights or suggestions on how to effectively retrieve my resources without hitting those limits would be greatly appreciated! Any help would be greatly appreciated! How would you solve this? For reference, this is a production application. Could this be a known issue? This is for a REST API running on macOS. What are your experiences with this? I'm developing on Linux with Python. Any feedback is welcome!