Azure Cosmos DB SDK v3.0 not retrying scenarios operations as expected
I'm working on a personal project and I'm sure I'm missing something obvious here, but I'm working on a project and hit a roadblock. I'm currently using the Azure Cosmos DB SDK version 3.0 with a .NET Core 3.1 application, and I've run into an scenario where failed operations are not being retried, even though I have configured the retry policy. I expect the SDK to automatically retry transient failures, but it seems to only throw exceptions instead. In my `CosmosClient` initialization, I've set a custom retry options like this: ```csharp var cosmosClientOptions = new CosmosClientOptions { ConnectionMode = ConnectionMode.Gateway, ConsistencyLevel = ConsistencyLevel.Session, ApplicationName = "MyApp", RequestTimeout = TimeSpan.FromSeconds(30), MaxRetryAttemptsOnRateLimitedRequests = 5, MaxRetryDelayOnRateLimitedRequests = TimeSpan.FromSeconds(15) }; var cosmosClient = new CosmosClient("<your-cosmos-db-endpoint>", "<your-cosmos-db-key>", cosmosClientOptions); ``` Despite this setup, when I try to perform an operation that I know will unexpected result (like querying a non-existent container), I get a `CosmosException` with a `StatusCode` of `404`, and it does not retry. Here's the code for the operation: ```csharp var container = cosmosClient.GetContainer("databaseName", "containerName"); try { var response = await container.ReadItemAsync<YourItemType>("nonExistentId", new PartitionKey("nonExistentPartition")); } catch (CosmosException ex) { Console.WriteLine($"behavior: {ex.StatusCode} - {ex.Message}"); } ``` I also checked the logs, and there's no indication that any retries were attempted. I've ensured that my Azure subscription is active and operational, with sufficient throughput to avoid throttling. What am I missing? Are there specific scenarios where the SDK doesn’t trigger retries? Any insights would be greatly appreciated. Has anyone else encountered this? This is part of a larger service I'm building. What would be the recommended way to handle this? I'm on Debian using the latest version of C#. Is there a simpler solution I'm overlooking?