When working with historical data, it is often necessary to partition the data into daily files based on a key column for efficient storage and querying. Azure Data Factory (ADF) provides a powerful data integration service, while Azure Notebook offers an interactive environment for data exploration and analysis. In this article, we will guide you through the process of partitioning historical data into daily Parquet files using Azure Data Factory and an Azure Notebook. This approach allows for easy execution and customization of the partitioning functionality.

Prerequisites

Before proceeding, ensure you have the following: