GCP Pub/Sub messages getting lost during high throughput despite ACK configuration
Does anyone know how to I tried several approaches but none seem to work. I'm experiencing an scenario with Google Cloud Pub/Sub where messages seem to be lost when my application is under high throughput. I have a subscriber set up using the Google Cloud Java client library (version 1.113.8), and I'm handling message acknowledgments manually. My subscriber code looks something like this: ```java public void subscribe() { Subscriber subscriber = Subscriber.newBuilder(subscriptionName, this::processMessage).build(); subscriber.startAsync().awaitRunning(); } private void processMessage(PubsubMessage message, AckReplyConsumer consumer) { try { // Simulate message processing Thread.sleep(100); // Processing takes time consumer.ack(); } catch (Exception e) { System.err.println("behavior processing message: " + e.getMessage()); consumer.nack(); // Nack if there's an behavior } } ``` In my testing, when I send messages at a rate of around 100 per second, I notice that after some time, the subscriber stops acknowledging messages, and I see gaps in the message IDs in the Pub/Sub dashboard. I've ensured that my acknowledgment deadlines are appropriately set (10 seconds) and that I'm using the correct subscription configuration. I've also tried increasing the number of concurrent subscriptions, but I still observe lost messages. I checked the logs and found an unusual pattern. When I hit the high throughput, I see logs like: `"Received message but processing is taking too long"`. Is there a recommended pattern for handling this kind of throughput in GCP Pub/Sub? Am I missing something in my acknowledgment or processing logic? Any insights would be greatly appreciated! This is for a service running on Windows 11. Any pointers in the right direction? What am I doing wrong?