important latency when retrieving large datasets using Entity Framework Core 6 with SQL Server
I'm trying to configure I've looked through the documentation and I'm still confused about I'm experiencing noticeable latency when fetching large datasets using Entity Framework Core 6 with SQL Server... My application is a web API that needs to return user data, but when the dataset exceeds a certain size (around 10,000 records), the response time increases dramatically, often taking over 5 seconds. I've tried optimizing my queries by using `.AsNoTracking()` to avoid tracking overhead, and I'm also implementing pagination using `.Skip()` and `.Take()`, but the performance is still not acceptable. Here's the code I'm currently using to fetch the data: ```csharp public async Task<List<User>> GetUsers(int pageNumber, int pageSize) { return await _context.Users .AsNoTracking() .Skip((pageNumber - 1) * pageSize) .Take(pageSize) .ToListAsync(); } ``` Additionally, I've ensured that the database indices are set up appropriately on the relevant fields. I also monitored the SQL queries generated by EF Core, and they seem reasonable, but the execution time on the SQL Server side is still high. I receive no specific behavior messages, but I notice that the SQL Server query execution plan shows a lot of table scans. Is there a better way to handle this, or are there specific configurations I might be missing to improve the performance? Any best practices for working with large datasets in EF Core would be appreciated. I'm working on a application that needs to handle this. I recently upgraded to C# stable. Am I missing something obvious? I'd be grateful for any help.