Importing dataset(s) from the local system
Last updated
Last updated
To import a file from your local system to the canvas for performing predictions and generating a modelling pipeline, follow the standard upload procedure. You can upload up to 25 files at a time, with a maximum file size of 5GB each. If you have multiple files grouped in a worksheet, you can upload the worksheet to automatically add each file as a separate dataset on the canvas, making it easier to manage and work with multiple data sources in one go.
You can upload data using the following methods:
Dataset – Import a dataset or file onto the canvas to perform data transformations from the local system or through data connectors. Refer to or through available .
Text File – Import a text file onto the canvas to apply transformations. Refer to .
Artifact – Import artifacts for transformation. You can either use existing artifacts or upload new ones from your local system. Refer to section.
Code Recipe – Fetch dataset by writing code within a code recipe. Refer to .
To import the file from the local system:
Click the project to which you want to upload the file. The Canvas page is displayed.
Do one of the following:
Click the +Dataset option on the canvas. However, this option is displayed only when there are no datasets uploaded onto the canvas.
Click the plus icon and select Dataset to navigate to the Create New Data set window.
By default, the project name is populated in the Project field.
Select the source from where you want to upload. By default, File Import is selected.
After establishing the connection and importing the files, the imported files are populated in this drop-down list.
Select the Mode to upload the file. Possible options:
Single file import - Use this option to import only a single file onto the canvas.
Merge - Use this option to merge multiple files into one file. Ensure that schema in all the files is same.
Segregate - Use this option to upload multiple files together onto the canvas as separate files.
Select Single file upload.
Click Import Files from Local to browse and upload the file from your local system.
Click Import. Once the file is imported, you can view the file name and file size.
You can perform these actions:
If you want to delete the uploaded file, click the delete icon corresponding to this file name.
If you want to rename the file name, click the edit icon in the Dataset name.
Click File Configuration to expand and view the file configuration fields, such as Separator and Encoding.
Separator and Encoding are auto-detected by the platform when you upload the file and this file has a single column containing all column names separated by a specified separator.
Note: The separator option allows you to split all the values separated by a separator into different columns.
Select separator from the drop-down list if the platform failed to auto-detect. Possible values:
Select the encoding option if it is not auto-detected by the platform.
Click Apply to apply the separator and encoding options you have selected. Please note that these options are only available for CSV files.
You can now see the data in the file clicking the Open Dataset option. This takes you to the View data page.
13. View the sample data and the data type of each column in the Sample Data section. To change the data type of a specific column, click the data type drop-down under the column name and select the new data type.
Click Done. Once the dataset is added, you are redirected to the Canvas view page where you can see the uploaded dataset node.
If you have an .xlsx worksheet containing multiple files, you can import all the files within the sheet onto the canvas in one step. Follow the steps below to upload and manage these files:
Navigate to the Project: Click the project to which you want to upload the worksheet. The Canvas page will be displayed.
Start the Dataset Upload:
If no datasets have been uploaded yet, click the +Dataset option on the canvas.
Alternatively, click the plus (+) icon and select Dataset to open the dataset creation window.
Import the Worksheet:
On the Create New Dataset page, click Import Files From Local to upload your worksheet.
The file names of all the sheets contained within the uploaded worksheet will be listed.
You can choose to remove any files you do not want to upload by clicking the delete icon next to each.
Preview the Files: Click Import to proceed. You will be able to view a sample of each file's data. For worksheets with multiple files, use the navigation arrows to scroll through the previews.
Complete the Upload: Click View in Canvas to display all the uploaded datasets on the canvas.
You can append a file to an existing dataset, provided both datasets share the same schema. However, keep in mind that:
When a file is appended, all recipes previously run with the source dataset will become invalid and move to an unbuilt state. You must re-run the flow after appending the dataset.
Any segments created will be deleted, and custom scenarios will use the entire dataset instead of the segmented data.
You can only append a file to the source dataset.
To add a file to the source dataset:
Select a project to open the canvas.
Do one of the following to add a file to the source dataset:
Right-click the source dataset block and select Add File.
Click the dataset block to open the side sheet, then click the plus (+) button and select File.
Click the dataset block, then click Preview to navigate to the View Data page. Click the plus (+) button, then select File.
Review the following details that are preselected and cannot be changed on the Append file page:
Project – The current project is selected by default.
Source – The original source of the dataset is preselected.
Mode – The append mode is selected by default.
Click Import Files From Local to browse and select the file to append. The dataset name is set by default, but you can rename it.
Click Import. Once imported, you can view the file name and file size.
Click Done. You will be redirected to the View Data page, where you can see the total number of rows after the file is appended.
You can either upload the file from the local system or create a new connection using the +New Connection to import files from external Data connectors. For more information, see
