GCP App Engine service working with 'Too Many Requests' scenarios on sudden traffic spikes despite scaling settings
I need some guidance on I've encountered a strange issue with Hey everyone, I'm running into an issue that's driving me crazy. I'm deploying a web application using GCP App Engine Standard (Python 3.8). The application has been configured with automatic scaling, but I'm running into an scenario where, during unexpected traffic spikes, I'm receiving 'Too Many Requests' (HTTP 429) errors. I've set the maximum instances to 10 in my `app.yaml` configuration: ```yaml # app.yaml runtime: python38 automatic_scaling: min_instances: 1 max_instances: 10 target_cpu_utilization: 0.6 target_throughput_utilization: 0.6 ``` Despite having this setup, I still see the service becoming unresponsive and serving the 429 behavior when the incoming request rate exceeds a certain threshold. I've also implemented caching using Memcache to alleviate load, but it doesn't seem to be having the desired effect. To troubleshoot, I checked the App Engine logs and noticed that the behavior occurs after a sustained period of high traffic, suggesting that App Engine is unable to provision new instances quickly enough to handle the load. I’ve tried modifying the `max_concurrent_requests` setting in the `app.yaml` file to limit the number of simultaneous requests handled by a single instance: ```yaml # app.yaml max_concurrent_requests: 5 ``` However, this doesn’t seem to help much, as the requests are still being throttled. I also looked into the Cloud Trace and found some high latency spikes that correspond to the time of the errors. Is there something I'm missing in my configuration, or any best practices for handling sudden traffic bursts in App Engine? Any insights would be appreciated! My development environment is Ubuntu. Thanks in advance! I'd be grateful for any help.