An Azure service for ingesting, preparing, and transforming data at scale.
Can the Azure SQL database be scaled up (in the DTU model) automatically before running a pipeline and then scaled down afterwards?
I have several pipelines in ADF that when executed write data to an azure sql database. I am using the Standard DTU model for the pricing tier. Using any available ADF activities, can the Azure SQL database be scaled up (in the DTU model) automatically…
Azure Data Factory
How to Call a ORACLE DATABASE PACKAGE from Azure data factory and receive the output of that package.
Team I need to call a ORACLE DATABASE PACKAGE from azure data factory and use its output in other activities but i am not able to call a PACKAGE. Inside the package there are many stored procedures written so i need to call a specific stored procedure…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
ADF Deployment failure – Global parameter not found under dataFactory
ADF Git Publish/ARM Validation fails when referencing Global Parameter: “Parameter StorageContainerName was not found under dataFactory I’m deploying Azure Data Factory pipelines from a feature branch → main and publishing to our ADF DEV factory. For…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
I have to copy data from share point (Unstructured Data Photo/Video)to Azure blob storage using ADF pipeline
Can anyone tell me which activities I have to use as well which access I have required on share point level as well On Entra ID As well which linked service I have to use
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Adf Pipeline created date
Hi Team, How can we find the excat adf pipeline created date. I tried to get detils from Activity log but i can see ony 3 months data can be fetched. Please suggested how can i get the pipeline creation date. Regards Saim
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Moving SHIR binaries from C: to D: drive
Hi, We have installed Microsoft Selfhosted Integration Runtime(SHIR (both Azure and On‑Premises) on a virtual machine, where the default installation is on the C: drive. We came to know that there is a possibility to move the SHIR binaries from C: to…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Column level type conversion in Mapping section of Copy Activity in Azure Data Factory.
Earlier, I used to get the option to change type conversion settings of a particular datetime column in mapping section of a copy data activity. Here my source is Azure SQL DB and Destination is Azure blob storage. Now, If I am changing my source to…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Table connector not working for incremental logic setup in Databricks
It appears Table Connector to bring the SAP tables data into Blob not working as expected and we have setup the incremental logic on timestamp. However, we observed that a few records in SAP table are being deleted. However, not able to get those deleted…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
ADF - Az function activity 499 error
WE get an error while executing azure function activity inside adf pipeline. Azure function is calling an API and extracts and stores the data returned by api to a storage account. Call to provided Azure function 'api_blob' failed with status-'499'…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Fix format of a datetime column in Copy Activity's Mapping section
We have set format of a Datetime column to "G" and "en-US" in mapping section of copy activity. Here our source is Azure SQL db and Destination is ADLS. The format of the datatype is as expected. Here is the type conversion setting:…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Data Factory Redshift Dataset throws v2 Error when using v1
Our Redshift Copy Pipelines recently stopped working with the error that they cannot locate a V2 ODBC DLL file. We did not change any settings and are still using V1 for Redshift. We attempted to update the Linked Service to use v2 but still receive…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
ADF - postgres: Read method cannot be called when another read operation is pending
I'm using Azure Data Factory copy data activity to move data from Azure PostgreSQL to OneLake. I'm using a self-hosted integration runtime for that (because of how our VNETs are set up). Here is the error I'm getting: Operation on target…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Database for PostgreSQL
An Azure managed PostgreSQL database service for app development and deployment.
What are the supported Open SSH Key Formats when it is using in Azure ADF - Linked services for SFTP access
Hi, I created a ADF Linked services with Open SSH key for SFTP access. It is supports only with RSA format key but not ECDSA format. As per the security requirement, I need to use only ECDSA format. Please help me to provide the suggestions and…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Connection to UiPath with Service Principal in Azure Data Factory
Dear All, I'm very new to Azure Data Factory and recently I got a request to connect to UiPath using Service Principal. I have received Directory (tenant) ID, Application (client) ID, Client secret Value and Secret ID. I could also see that it is already…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Validate Schema option in mapping data flow not failing for invalid type
I have a mapping data flow with 'Validate Schema' ticked and the following projection settings. When I run the mapping data flow in debug mode, it doesn't fail for the following file. Here is the final file. It just skips the erroneous values. How…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
SftpFailedToConnectToSftpServer
I am having an issue with a client supplied SFTP account from within Data Factory, I am able to connect with the credentials via FileZilla locally, and with some C# code when deployed, but Data factory cannot create a valid connection to this AWS server.…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
IBM I Access AS400 connection in ADF
I want to connect IBM I Access AS400 connection for data source in ADF
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
ADF system-assigned managed identity broken after subscription migration
After migrating an Azure Data Factory (ADF) resource from one Azure subscription to another, the system assigned managed identity for the ADF resource appears to be in a broken state. The managed identity is visible in the ADF resource settings, but it…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Failed to resolve the server name "dpsouthafricanorth.svc.datafactory.azure.com", please refer to troubleshooting doc and check network configuration.
Hi Team For some unknown reason we are having this error with our linked services in ADF. All test connection failed with: Failed to resolve the server name "dpsouthafricanorth.svc.datafactory.azure.com", please refer to troubleshooting doc…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Not able to read the Excel file using the download url of the share point file into the Azre ADF http Excel dataset
Hi, I am using the Share point download url in the Azure ADF http Excel dataset, where i have put the download url then also this does not read the sheet name . When i put the sheet name manually then this not show the preview. In that similar way i…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.