Django Async View Not Returning Response as Expected When Querying Large Dataset
I'm performance testing and I tried several approaches but none seem to work..... I've searched everywhere and can't find a clear answer. I'm working on a Django application (version 4.1) and trying to implement an asynchronous view to fetch a large dataset from the database using Django's ORM. The goal is to return a JSON response containing user data. However, when I hit the endpoint, it seems to hang indefinitely and eventually times out, throwing a `500 Internal Server behavior`. I've confirmed that the database query itself works fine when run synchronously in the Django shell. Here's the view function I've implemented: ```python from django.http import JsonResponse from asgiref.sync import sync_to_async from myapp.models import User async def get_users(request): users = await sync_to_async(User.objects.all)() user_data = list(users.values('id', 'username', 'email')) return JsonResponse(user_data, safe=False) ``` I've also tried utilizing Django channels to handle the asynchronous nature, but I still face the same timeout scenario. I suspect it might be due to the size of the dataset, as I have over 100,000 user records in the database. I tested this endpoint using Postman and it just spins without giving a valid response. I also checked the Django logs and I see this warning: `WARNING: asgiref.local: Trying to release un-acquired lock`. To troubleshoot, I tried breaking down the dataset into smaller batches using pagination, but encountered similar issues. Can anyone guide to understand what might be going wrong here, or if there's a better way to handle large queries asynchronously in Django? This is part of a larger application I'm building. My development environment is Windows. Am I missing something obvious? This issue appeared after updating to Python 3.10. What would be the recommended way to handle this?