> For the complete documentation index, see [llms.txt](https://cleyrop.gitbook.io/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cleyrop.gitbook.io/docs/documentation-fr-en/data-and-ai-project/dataflow/utiliser-des-modules.md).

# Use Modules

The **Modules** allow you to factor out and share common code across several Dataflows in the same project. They contain **Python functions** (or PySpark) that you can import directly into your transformations. It is a simple way to centralize calculation rules, dictionaries, connectors, or recurring processing.

***

## Available module types

There are two types of modules:

<table><thead><tr><th width="208.859375">Type</th><th width="143.89453125">Language</th><th>Usage</th></tr></thead><tbody><tr><td>Python module</td><td>Python</td><td>For Python-type Dataflows.</td></tr><tr><td>PySpark module</td><td>PySpark</td><td>For Spark-type Dataflows (Spark / pandas-on-Spark / SQL).</td></tr></tbody></table>

{% hint style="warning" %}
A module can only be used in a **Dataflow of the same type**.

A Python module cannot be imported into a Spark Dataflow, and vice versa.
{% endhint %}

## Create a module

1. In the Dataflow menu, tab `Module`
2. Click `Create`.
3. Enter a **clear** that complies with the rules
4. Choose the **type**

## Edit the contents of a module

1. Select your module from the list of modules.
2. Write or paste the Python code in the editor.
3. Click **Save** to save your changes.
4. Click **Deploy** to make the new version of the module available in the project.

### **Python module example**

{% code title="module example" %}

```python
# utils_math

def normalize_column(df, column):
    """
    Normalize a numeric column between 0 and 1
    """
    min_val = df[column].min()
    max_val = df[column].max()
    df[column] = (df[column] - min_val) / (max_val - min_val)
    return df


def compute_ratio(a, b):
    return a / b if b != 0 else None
```

{% endcode %}

You can then use it in your Python transformations:

```python
from utils_math import normalize_column, compute_ratio

df = normalize_column(df, "price")
df = df.with_columns({"ratio": compute_ratio(df["revenue"], df["expenses"])})
return df
```

## Deployment and versioning

Every time a module is **deployed**, all the **subsequent runs** of the Dataflows that use it will be automatically updated with the **new version of the module**.

* This creates a **new version** for each affected Dataflow.
* Existing Dataflows keep their run history, but their subsequent runs will use the updated code.

{% hint style="warning" %}
The update process is **asynchronous** it may take a few seconds for the module to become available to all users in the project.
{% endhint %}

***

## Best practices

* Before deploying a module, it is recommended to test it in a test Dataflow to avoid impacting several production processes.
* Name your modules according to their function (utils\_text, rules\_finance, parser\_api).
* Comment your functions to make them easier to reuse by other team members.
* Avoid external dependencies not installed on the cluster: use only the libraries available by default.
* If a module needs to evolve without impacting production, duplicate it under a new name (e.g. utils\_text\_v2).