> For the complete documentation index, see [llms.txt](https://cleyrop.gitbook.io/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cleyrop.gitbook.io/docs/documentation-fr-en/data-and-ai-project/dataflow/transformer-sans-code-low-code.md).

# Transform without code (Low Code)

Low Code mode lets you create visual transformations in a Spark Dataflow.

Each block corresponds to a data processing operation (filtering, joining, aggregation, etc.) that you can chain together without writing Python or SQL code.

This mode makes collaboration between technical and business profiles easier, while ensuring the reproducibility and traceability of transformations.

**How it works**

Each Low Code transformation **generates a new output dataset** without modifying the input dataset. This guarantees traceability and makes it possible to chain several transformations safely. The original dataset remains intact and can be reused elsewhere.

***

## Add a Low Code transformation

1. Create a Dataflow of **Spark type**.
2. In the left menu, choose Low Code (1 input) or Low Code (2 inputs) to add a Low Code-type transformation
3. Choose the operation to apply (e.g. filter, aggregate, rename…).

Each Low Code block is visually linked to its input dataset and its output dataset in the Dataflow graph.

{% hint style="success" %}
Each step of a transformation can be validated before full execution: you can **preview the step result**.

The **Check the previous step** lets you check the result of a step while you configure the next one, without interrupting your work.
{% endhint %}

### Combine Low Code and Python / SQL code

Low Code steps can be combined with Python or SQL transformations. This allows a **business user to prepare data in Low Code**, then a technical user to enrich it in code, without breaking the flow. :

* Add a Low Code transformation for simple, visual operations.
* Then add a Python or SQL transformation for more advanced processing (specific cleaning, analytical functions, etc.).
* Cleyrop automatically handles **input and output consistency between steps**.

{% hint style="danger" %}
For Low Code transformations to work, they need the **input Dataset schema :** so it must have been built at least once. For branches, for example, if you chain steps, remember to run them once first
{% endhint %}

### Chain several 1-input transformations

1-input Low Code blocks can be chained together:

1. Each block takes the previous step's output dataset as input
2. You can **add, move, or delete** a block
3. The Dataflow graph updates automatically to reflect the processing logic.

{% hint style="info" %}
The **output schema** is automatically calculated by Cleyrop at each step in the chain. If a schema incompatibility (type, no longer used column) is detected, an **error** will be displayed
{% endhint %}

<figure><img src="/files/722b05ef4c9315bc13329e0dfae7bd731a1f5585" alt="" width="563"><figcaption></figcaption></figure>

## Available operations

**List of 2-input operations**

<table data-full-width="true"><thead><tr><th width="165.25390625">Operation type</th><th width="618.78125">Description</th></tr></thead><tbody><tr><td><strong>Join</strong></td><td>Combine several datasets based on a common key.</td></tr><tr><td><strong>Union</strong></td><td>Stack the rows of 2 datasets</td></tr></tbody></table>

**List of 1-input operations**

<table data-full-width="true"><thead><tr><th width="165.25390625">Operation type</th><th width="618.78125">Description</th></tr></thead><tbody><tr><td><strong>Filter rows</strong></td><td>Keep only certain rows based on a condition.</td></tr><tr><td><strong>Filter columns</strong></td><td>Keep only certain columns</td></tr><tr><td><strong>Rename columns</strong></td><td>Change the names of columns compared to the input dataset</td></tr><tr><td><strong>Group and aggregate</strong></td><td>Aggregate values (sum, average, count…).</td></tr><tr><td><strong>Add a calculated column</strong></td><td>Create a column from an expression: +, -, *, /, modulo (%), parentheses (), power (^).<br><br><span data-gb-custom-inline data-tag="emoji" data-code="26a0">⚠️</span> <em>Only int and double column types can be used</em></td></tr><tr><td><strong>Remove duplicates</strong></td><td>Remove identical rows on one or more fields.</td></tr><tr><td><strong>Sort data</strong></td><td>Order rows by one or more fields.</td></tr><tr><td><strong>Concatenate datasets</strong></td><td>Assemble several compatible sources into a single dataset.</td></tr><tr><td><strong>Create a conditional column</strong></td><td>Creates a column based on conditions applied to other columns. Use AND/OR to add multiple conditions</td></tr><tr><td><strong>Modify Column Types</strong></td><td>Change column types compared to the input dataset<br><br>⚠️ <em>Errors may occur if conversion is impossible (for example, letters to numbers).</em></td></tr></tbody></table>

### Join

**Purpose** : Combine two datasets using common keys.

**Parameters**

* Inputs: you can swap the order of the datasets
  * Left dataset: First dataset (main source)
  * Right dataset: Second dataset to join
* Join type:
  * Inner: Only rows with a match on both sides
  * Left: All rows from the left + matches from the right
  * Right: All rows from the right + matches from the left
  * Outer: All rows from both datasets
* Join keys: 1 to N column pairs (AND condition between pairs)

<figure><img src="/files/43384617d5df8b03f2b2c99df5b9ce9f900bc961" alt=""><figcaption></figcaption></figure>

### Merge (Union)

**Purpose** : Stack the rows of two datasets with a similar structure vertically.

**Parameters**

* Source datasets: Select 2 datasets to union
* Matching mode: By column name or by position
* Non-common columns: Include all (with NULLs) or only common columns
* Deduplication: Union (without duplicates) or Union All (with duplicates)

<figure><img src="/files/996f580bb87d3dc03bc61303fb378d87d6649bb4" alt=""><figcaption></figcaption></figure>

### Filter columns

**Purpose** : Choose the columns to keep.

**Parameters**

* Columns: Select the columns to include in the result
* Keep or delete these columns

<figure><img src="/files/b4ba44a52750ad872919e9ccecba5e61ddeb22dd" alt=""><figcaption></figcaption></figure>

### Filter rows conditionally

**Purpose** : Keep or remove rows based on conditions.

**Parameters**

* Conditions
  * Column: Select the column to filter
  * Operator: Depends in particular on the column type
    * Numeric/Date: =, ≠, <, >, ≤, ≥, between, is null, is not null
    * Text: equal, different, contains, does not contain, starts with, ends with, is null, is not null
  * Value or Column: Comparison with a fixed value or another column
  * Multiple conditions: Combine with AND / OR by clicking on the <i class="fa-plus">:plus:</i> at the end of the line
* Keep or delete matching rows

<figure><img src="/files/551940846d279d047c1ce7d70fb571eb717f41a5" alt=""><figcaption></figcaption></figure>

### Group and aggregate

**Purpose** : Group data and apply aggregation functions.

**Parameters**

* Grouping columns: Define the groups (equivalent to GROUP BY)
* Aggregation functions:
  * SUM, AVG, MIN, MAX, COUNT, COUNT DISTINCT
* Result column names: generated automatically

### Top N

**Purpose** : Extract the first or last N rows according to a ranking criterion, with the option to partition by group.

**Parameters**

* Number of rows (N): Number of rows to keep (e.g. 10, 100, 1000)
* Sort columns: One or more columns used for ranking (configurable order)
* Sort order:
  * Top: The N largest values (DESC)
  * Bottom: The N smallest values (ASC)
* Partition (optional): Grouping columns to get Top N by group
* Rank column (optional): Adds a column indicating the rank of each row

<figure><img src="/files/fe12bcb1de3432f2165221e2d42a61773d3615cd" alt=""><figcaption></figcaption></figure>

### Transform strings

**Purpose** : Apply text transformations to one or more columns.

**Parameters**

* Source columns: Select 1 to N text-type columns
* Destination: Create new columns with a customizable suffix (e.g. \_clean)
* Available operations:

| LEFT(n)       | Extracts the first n characters                |
| ------------- | ---------------------------------------------- |
| RIGHT(n)      | Extracts the last n characters                 |
| TRIM          | Removes spaces (all / left / right / multiple) |
| UPPER         | Converts to uppercase (handles accents: é → É) |
| LOWER         | Converts to lowercase (handles accents: É → é) |
| REPLACE(x, y) | Replaces x with y (case-sensitive)             |

* Chaining: Several operations can be chained in the desired order

<figure><img src="/files/9b5c7c9d54cf3ed68d5551c459f53776933deb8a" alt=""><figcaption></figcaption></figure>

### Search in a string

**Purpose** : Identify the position of text in a column, or check whether it is present.

**Parameters**

* **Source column** : Select a text-type column
* **Pattern** : The string to search for (case-sensitive)
* **Result column name** : Required and unique

**Good to know** : The result is an integer. The returned value is the position of the pattern in the string (starting from 1), or -1 if the pattern is absent.

### Concatenate

**Purpose** : Merge several text columns into a new column.

**Parameters**

* Source columns: Select 2 to N columns to concatenate
* Separator: Character or string between values (default: none)
* Result column name: Required and unique
* NULL handling:
  * Ignore: NULLs are not included in the result
  * Empty string: NULLs are treated as ""
  * Propagate: The result is NULL if at least one value is NULL<br>

<figure><img src="/files/17f0ba1a504608aa346bbb5419f98f1fdbc5bc3f" alt=""><figcaption></figcaption></figure>

### Rename columns

**Purpose** : Give a different name to a column compared to the original dataset

**Parameters**

* Source column: Current name of the column
* New name: Must be unique in the dataset

You can add columns for which you want a different output name or rename all columns by enabling "See all columns"

<figure><img src="/files/7d4e913d1b678e855d88b4039dcacffc266a74a6" alt=""><figcaption></figcaption></figure>

***

## Best practices

* Use Low Code for simple, recurring steps: filtering, joining, aggregating, renaming.
* Keep more technical steps (e.g. API, parsing, complex business logic) in Python or SQL transformations.
* Give each step an explicit name to make the graph easier to read.
* Check the output dataset after each transformation using the built-in preview.
* For production flows, document the calculation logic in the description of each step.