> ## Documentation Index
> Fetch the complete documentation index at: https://docs.corvic.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Engineering

> Pipelines that run themselves, knowledge graphs across sources, and ML-ready datasets.

Data engineering shouldn't take months. Describe what you need, and Corvic builds the pipeline, runs it on a schedule, and keeps the result fresh.

## Pipelines that run every morning

Set it once: pull from your sources, transform, and land a fresh table before anyone logs in.

[Set It Once, It Runs Every Morning: Corvic AI's Data Pipelines](https://www.youtube.com/embed/cEvyINUWOMM)

| | |
| - | - |
| **Bring** | The sources and the table you want at the end |
| **Connect** | [Databases and storage](https://docs.corvic.ai/connector-vault/catalog/databases-data.md) |
| **Start with** | The **New Data Pipeline** playbook, or describe it in [Builder Chat](https://docs.corvic.ai/features/builder-chat.md) |
| **You get** | A scheduled pipeline, versioned in GitHub if you like |

> [!TIP]
>
> **Try asking:** “Every day at 6am, pull yesterday's orders from Postgres, join with customers, and refresh the revenue table.”

## One knowledge graph across every source

Connect APIs, documents, and tables into one graph your agents can reason over, with no keys exposed.

[Every Source, One Knowledge Graph: Unlimited APIs with Zero Exposure](https://www.youtube.com/embed/XqZRBrFnh9M)

| | |
| - | - |
| **Bring** | Documents and tables, plus the services to connect |
| **Connect** | Anything in the [Connector Vault](https://docs.corvic.ai/connector-vault/overview.md) |
| **Start with** | The **Knowledge Graph from Documents** or **Graph over Your Tables** playbook |
| **You get** | A knowledge graph and agents that cite it |

> [!TIP]
>
> **Try asking:** “Which customers mentioned in support tickets also have open opportunities over \$100k?”

Watch the full walkthrough: [Building a Custom Knowledge Graph](https://docs.corvic.ai/tutorials/custom-knowledge-graph.md).

## Dataset cleaning and feature prep

Ingest raw datasets, clean and profile them, and produce ML-ready features.

[Data Engineering Shouldn't Take Months | Meet Corvic AI](https://www.youtube.com/embed/637VO9S5-BI)

| | |
| - | - |
| **Bring** | Raw CSV, Parquet, or JSON, and a description of the ML task |
| **Connect** | [HuggingFace](https://docs.corvic.ai/connector-vault/catalog/databases-data.md), [Kaggle](https://docs.corvic.ai/connector-vault/catalog/databases-data.md) |
| **Start with** | The **Dataset Cleaning & Feature Prep** playbook |
| **You get** | A cleaned, profiled table and a feature set |

> [!TIP]
>
> **Try asking:** “Profile this dataset, fix the obvious quality issues, and build features to predict churn.”

## Enrichment and labeling

Add columns to any table with an LLM, code, or the live web: sentiment, categories, firmographics.

[AI Data Enrichment and Labeling: Text Sentiment Analysis](https://www.youtube.com/embed/8dlF4QCzsRs)

| | |
| - | - |
| **Bring** | The table to enrich |
| **Start with** | The **Augment a Table with New Columns** playbook, or [LLM Augment](https://docs.corvic.ai/features/llm-augment.md) |
| **You get** | New columns on every row, rerun as data changes |

> [!TIP]
>
> **Try asking:** “Add sentiment and a topic label to every review in this table.”

Watch the full walkthrough: [Data Enrichment](https://docs.corvic.ai/tutorials/data-enrichment.md).

## Make it repeatable

When a run works, save it as a [playbook](https://docs.corvic.ai/features/playbooks.md) so anyone on your team can rerun it, or publish the result as a [live app](https://docs.corvic.ai/features/live-apps.md). See [all use cases](https://docs.corvic.ai/use-cases/overview.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.