advanced patterns with `find_each` in Rails when dealing with large datasets and custom scopes
I'm testing a new approach and I'm integrating two systems and After trying multiple solutions online, I still can't figure this out... I'm working with an scenario where using `find_each` in my Rails application seems to lead to unexpected results when dealing with large datasets in conjunction with custom scopes. I have a model `Order` with a scope `recent` that filters orders created in the last 30 days. When I try to process these records in batches using `find_each`, it seems to skip some records entirely. Here's my code snippet: ```ruby class Order < ApplicationRecord scope :recent, -> { where('created_at >= ?', 30.days.ago) } end # In my service class: Order.recent.find_each(batch_size: 100) do |order| # Processing each order puts order.id end ``` I expected to see all recent orders printed out, but I noticed that some IDs are missing, specifically those that were created exactly 30 days ago. I also checked the SQL generated by Rails and found that the query was correct: ```sql SELECT "orders".* FROM "orders" WHERE (created_at >= '2023-09-12 00:00:00') ``` The records are definitely in the database as I can find them using the Rails console with `Order.recent.count`. I've tried adjusting the `created_at` comparison in the scope to include the edge case: ```ruby scope :recent, -> { where('created_at > ?', 30.days.ago) } ``` However, this doesn't solve the question. I'm using Rails 6.1.4 and PostgreSQL 13. I also tried adding a `debugger` inside the block to see how many times it iterated, and it matched the count from the console, but the printed orders are still missing those edge cases. Is there something I'm overlooking with `find_each` or is there a better way to handle this situation? Any insights would be appreciated! I'd really appreciate any guidance on this. I've been using Ruby for about a year now. Hoping someone can shed some light on this. I'm coming from a different tech stack and learning Ruby. Has anyone dealt with something similar? I'd be grateful for any help.