How to handle file uploads efficiently in Django with chunked uploads?
I've been banging my head against this for hours. I've searched everywhere and can't find a clear answer..... I'm working on a Django application (version 3.2) where users can upload large CSV files, but the uploads unexpected result or timeout when the file size exceeds 10MB. I've tried utilizing the `FileField` and processing the file in a standard view, but I've run into issues with performance, especially with files around 50MB. I've read about chunked uploads as a solution to mitigate this question, but I'm unsure how to implement it effectively with Django and handle the incoming file streams. Here's the current code snippet I have for the upload view: ```python from django.shortcuts import render from django.http import JsonResponse from .forms import UploadFileForm def upload_file(request): if request.method == 'POST': form = UploadFileForm(request.POST, request.FILES) if form.is_valid(): file = request.FILES['file'] # Save the file or process it # ... return JsonResponse({'status': 'success'}) else: form = UploadFileForm() return render(request, 'upload.html', {'form': form}) ``` I’ve also tried increasing the `DATA_UPLOAD_MAX_MEMORY_SIZE` in the settings, but that didn’t resolve the scenario. Additionally, I'm not sure how to manage the state of uploads since I would also like to provide users with a progress indicator. Are there any libraries or best practices in Django for implementing chunked file uploads? Also, how should I modify my JavaScript to send the file in chunks? Any guidance would be greatly appreciated! This is my first time working with Python 3.9. This is for a application running on CentOS. For reference, this is a production CLI tool. What's the best practice here?