> For the complete documentation index, see [llms.txt](https://cleyrop.gitbook.io/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cleyrop.gitbook.io/docs/documentation-fr-en/data-and-ai-project/dataflow/executer-deployer-et-automatiser.md).

# Run, deploy, and automate

Once a Dataflow is ready and validated, you can **run it or schedule it** so that it runs automatically at a given frequency or in a logical sequence with other Dataflows or Datasets.

Scheduling helps keep your processes up to date without manual intervention, and**orchestrate** multiple flows together.

| Mode          | Description                                                    | Example of use                                        |
| ------------- | -------------------------------------------------------------- | ----------------------------------------------------- |
| **Manual**    | One-off launch from the interface (button *Run*).              | Tests, adjustments, validation of a new version.      |
| **Recurring** | Automatic execution at a defined frequency.                    | Daily update of reporting datasets.                   |
| **Event**     | Triggered by an event (Dataset / Dataflow update, file upload) | Refresh of a summary Dataflow after multiple sources. |

***

## View executions

You can view all scheduled or unscheduled executions from the Dataflow list - History tab or from the Dataflow Executions menu

The Executions tab shows the history of all executions, whether manual or scheduled.

For each execution you can find (in the list or detailed record):

* Execution type (manual / scheduled / event),
* Date and time and total duration
* Status (success, error, canceled, pending),
* Partial (only one transformation executed) or Global
* Execution cluster
* Executed Dataflow and branch

You can click on a **transformation** to see its details and consult the **detailed logs** for each transformation.

{% hint style="warning" %}
If your execution remains `Pending`, it is in the [Queue](/docs/documentation-fr-en/data-and-ai-project/dataflow/analyser-les-executions-et-logs.md#position-dans-la-file-dattente) of the Cluster
{% endhint %}

Preview the datasets produced by your transformations directly.

## View or filter execution results

After each execution (manual or scheduled), you can **view the results in a Dataset preview.**

From the Dataflow graph or from the latest execution record:

* Click on a **output dataset** to display its **preview** (up to the first 100 rows) as well as its full schema (columns, types, nullable...).
* To filter or explore the results, go to the **SQL Exploration tab of the preview record** in the dataflow interface

<figure><img src="/files/97e36457215dbec878a19a9adf464f63cf4d3a6f" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
If the Preview does not appear when it should, it is often a [schema issue](/docs/documentation-fr-en/data-and-ai-project/dataflow/creer-un-dataflow/configurer-un-dataset-de-sortie.md#gerer-le-schema-du-dataset-de-sortie)
{% endhint %}

## Deploy a Dataflow <a href="#publier-un-dataflow" id="publier-un-dataflow"></a>

Once your Dataflow is ready, you can switch your Dataflow to **Production** by **deploying it**. Deploying to production will enable:

* Visibility in the output dataset library **Referenced**
* The **reuse of referenced output datasets** from the Dataflow within the project (other Dataflows, Dashboards...)
* The [**publication in the catalog**](/docs/documentation-fr-en/data-and-ai-project/datasets/publier-au-catalogue.md) of datasets so they can be reused in other projects
* The [**collaboration on the Dataflow**](/docs/documentation-fr-en/data-and-ai-project/dataflow/collaborer-et-versionner.md) through versioning with the other members of the project

{% hint style="danger" %}
The **deployment** of the dataflow **will automatically generate a Dataflow execution.**
{% endhint %}

To ensure good data and transformation quality, it is strongly **recommended** before moving to Production that:

* The Dataflow's latest global execution is in **status&#x20;**<mark style="color:green;">**`Success`**</mark>
* That the data be clean; for this you can [**reset the data**](/docs/documentation-fr-en/data-and-ai-project/dataflow/creer-un-dataflow/configurer-un-dataset-de-sortie.md#reinitialiser-les-donnees) of an output Dataset or [**delete columns**](/docs/documentation-fr-en/data-and-ai-project/dataflow/creer-un-dataflow/configurer-un-dataset-de-sortie.md#creer-un-dataset-depuis-un-dataframe-pandas-version-polars) unneeded before moving to Production

To move the Dataflow to Production, click the `Deploy` button at the top right of the page

## Manage your schedules

{% hint style="warning" %}

* Only the **Dataflow Owner** and **Project Manager** can create or modify a schedule
* A Dataflow must be in **Production** order to be automated
  {% endhint %}

1. From the Dataflow list, select the <i class="fa-pen">:pen:</i> `Information` or from the Dataflow click on <i class="fa-gear">:gear:</i> `Information`, then `Edit`
2. In the Refresh section, you can:
   * Add one or more **recurrences** : execution frequency (daily, weekly, hourly, etc.) and start time\
     💡 You can create multiple schedules for the same Dataflow (e.g. daily test version, weekly full version).
   * Add a **event trigger** : choose the type then the trigger name
     * **Dataset** : choose the Dataset for which the trigger will be activated upon **success** of a refresh
     * **Dataflow** : choose the Dataflow for which the trigger will be activated upon **success** of an **execution**
     * **Work Data** : choose the folder for which the trigger will be activated when a file arrives

You can from this panel **modify** or **delete** a schedule.

<figure><img src="/files/85f7e724c695f518d3328225ebb14a91725d63ba" alt=""><figcaption></figcaption></figure>

***

## Best practices

* Use staggered time slots to avoid **load peaks** (e.g. 2 a.m. rather than midnight).
* For critical processing, enable an **alert** in case of failure.
* Always test a schedule on a lightweight cluster before deploying it to production.
* Document your schedules (description, frequency, dependencies) to facilitate maintenance.
