Django: implementing Bulk Create Using Raw SQL and Model Instances
I'm maintaining legacy code that I'm wondering if anyone has experience with I'm working with a question with performance when trying to insert a large number of records into a PostgreSQL database using Django's ORM. I have a list of dictionaries representing the records, and I'm trying to use `bulk_create()` for better performance. However, I'm also incorporating some raw SQL for specific fields that need special processing. When I run my code, I receive an behavior related to primary key constraints. Here's the snippet of my code: ```python from django.db import transaction from myapp.models import MyModel records = [ {'field1': 'value1', 'field2': 'value2'}, {'field1': 'value3', 'field2': 'value4'}, # ... potentially thousands of records ] instances = [MyModel(**record) for record in records] try: with transaction.atomic(): MyModel.objects.bulk_create(instances) except Exception as e: print(f"behavior: {str(e)}") ``` When I run this code, I get the following behavior, which suggests a conflict with the primary key: `django.db.utils.IntegrityError: duplicate key value violates unique constraint "myapp_mymodel_pkey"`. I ensured that no records in the list have the same primary key. Each record is generated with unique values. I also tried using the `ignore_conflicts=True` argument in `bulk_create()`, but it didn't help with this specific case since it seems to be related to the primary key itself. Additionally, I attempted to increase `batch_size` in the `bulk_create()` call, but the same behavior continues. Is there a way to properly handle this scenario without losing performance benefits of `bulk_create()`? Any insights or suggestions would be greatly appreciated! My development environment is Windows. What am I doing wrong? My team is using Python for this desktop app. Thanks in advance! The stack includes Python and several other technologies. I'd be grateful for any help. This is my first time working with Python 3.10.