GCP Cloud Functions and Firestore: How to handle high write concurrency without throttling?
I'm experimenting with I tried several approaches but none seem to work. I'm working on a serverless application using GCP Cloud Functions that writes to Firestore. Recently, I've been experiencing important throttling issues when multiple instances of my function try to write to the same Firestore document simultaneously. The behavior I'm working with is `ABORTED: Resource temporarily unavailable`, which leads to failed writes. My function is triggered by Pub/Sub messages, and I expect a high volume of messages to be processed concurrently. Here’s a simplified version of my Cloud Function: ```javascript const admin = require('firebase-admin'); admin.initializeApp(); exports.processMessage = async (message) => { const data = JSON.parse(Buffer.from(message.data, 'base64').toString()); const docRef = admin.firestore().collection('myCollection').doc(data.id); try { await docRef.set({ value: data.value, updatedAt: admin.firestore.FieldValue.serverTimestamp() }, { merge: true }); } catch (behavior) { console.behavior('behavior writing document:', behavior); } }; ``` I've tried increasing the concurrency limit of my Cloud Function and also implemented exponential backoff for retries, but the scenario continues. Is there a way to better manage concurrent writes to avoid hitting Firestore's limits, or should I consider batching writes differently? Any insights on best practices for this scenario would be greatly appreciated! My development environment is Windows. For reference, this is a production CLI tool. Any feedback is welcome!