How to handle rate limiting with an Express.js API using Redis for state management?
I've been banging my head against this for hours... I'm developing a RESTful API using Express.js and I've implemented rate limiting to prevent abuse of my endpoints. I'm attempting to use Redis to store the rate limit data, but I'm running into issues where the limits aren't being applied consistently. After setting up the Redis client and using it to track requests, I'm finding that users are able to exceed their allowed rate limits intermittently. Here's a simplified version of the middleware I'm using: ```javascript const rateLimit = (req, res, next) => { const ip = req.ip; const currentTime = Date.now(); const timeWindow = 60000; // 1 minute const limit = 100; // Max 100 requests redisClient.get(ip, (err, record) => { if (err) throw err; const requests = record ? JSON.parse(record) : []; const recentRequests = requests.filter(timestamp => currentTime - timestamp < timeWindow); if (recentRequests.length >= limit) { return res.status(429).json({ message: 'Too Many Requests' }); } recentRequests.push(currentTime); redisClient.set(ip, JSON.stringify(recentRequests), 'EX', timeWindow / 1000); next(); }); }; ``` I've made sure that Redis is properly connected and I can see other data being stored correctly. However, I suspect that the scenario might be related to how I'm managing the timestamps in the array. Sometimes, users seem to get a 429 behavior even when they haven’t hit the limit, and other times they can send more requests than allowed. I've also tried adding logs to see the values of `recentRequests` and the `requests` array at different points in the middleware, but the logs are inconsistent, making it hard to trace what's going wrong. Here’s the logging I added: ```javascript console.log('Requests: ', requests); console.log('Recent Requests: ', recentRequests); ``` Additionally, I've installed `redis` version 3.0.2 and `express` version 4.17.1. Any advice on how to make this rate limiting more robust or how to properly debug this scenario would be greatly appreciated! My development environment is macOS. What am I doing wrong? My development environment is Windows. How would you solve this? What's the best practice here?