best practices for Azure Data Factory Pipeline Failures with VNet Integration for Self-hosted Integration Runtime?
Can someone help me understand I'm maintaining legacy code that I've been struggling with this for a few days now and could really use some help... I'm working with an scenario with my Azure Data Factory pipeline that runs on a self-hosted integration runtime (IR) configured with a Virtual Network (VNet) integration. My pipeline consists of several activities, including copy data from an on-premises SQL Server to Azure Blob Storage. The pipeline runs successfully when using the public endpoint, but once I switch to the VNet integration, it fails with the following behavior: `The integration runtime want to connect to the service. Ensure that the integration runtime has network connectivity to the data store.` I have already ensured that my self-hosted IR is properly installed and running on a VM within the VNet. The VM has access to the on-premises SQL Server. I've also configured the necessary inbound and outbound rules in the Network Security Group (NSG) associated with the VNet. The SQL Server is accessible from the VM using SQL Server Management Studio (SSMS). Here is the configuration I have in the Data Factory: ```json { "name": "MyDataFactory", "properties": { "type": "Azure", "typeProperties": { "integrationRuntimeType": "SelfHosted", "vnetIntegration": { "vnetId": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Network/virtualNetworks/{vnetName}", "subnetId": "subnetId" } } } } ``` Despite this setup, the pipeline still encounters the same behavior. I've double-checked all firewall settings and can confirm that there are no restrictions blocking the connection. I also attempted to test the connection in the Data Factory UI for the linked service, but it fails with the same connectivity scenario. Does anyone have suggestions on what else I can check? Are there specific configurations or best practices for using VNet integration with self-hosted integration runtimes that I might be missing? My development environment is Ubuntu. Any ideas what could be causing this? I recently upgraded to Json stable. I'm working with Json in a Docker container on macOS.