> For the complete documentation index, see [llms.txt](https://docs.algenta.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.algenta.ai/connectors/files.md).

# Files & uploads

By the end of this page you will have a saved `file` connector that parses your CSV, TSV, JSON, Excel, or Parquet data, a confirmed parse with a record count, that data onboarded as a queryable **dataset**, and a first [governed query](/guides/governed-query.md) returning rows. Once onboarded, the file becomes [governed data](/concepts/governed-data.md): the engine reads it under a typed, validated contract, so results stay deterministic and auditable.

Prerequisites: an API key and a base URL (see [Authentication](/getting-started/authentication.md)), and the data itself — either inline `content` or a `file_path` the engine can read. Use `https://api.algenta.ai` for Algenta cloud, or your own origin such as `http://localhost:8000` for a self-hosted deployment.

{% hint style="info" %}
A `file` connector takes these `config` fields: `file_type` is one of `csv` (default), `tsv`, `json`, `excel`, or `parquet`. Provide the data as `content` — raw text for CSV/TSV/JSON, or base64-encoded bytes for Excel/Parquet — **or** as a `file_path` on disk (used when `content` is empty). `json_path` is a dot-separated path to the records array in JSON (for example `data.items`). `sheet` selects an Excel sheet by name or 0-based index (default `0`). `delimiter` overrides the CSV column delimiter (auto-detected by default). The alias type name `file_upload` resolves to `file`.
{% endhint %}

{% stepper %}
{% step %}

### Set your credentials

Export your API key and base URL so every command below picks them up from the environment.

```bash
export ALGENTA_API_KEY="$ALGENTA_API_KEY"
export ALGENTA_BASE_URL="https://api.algenta.ai"
```

Verify the variables are set:

```bash
echo "${ALGENTA_API_KEY:?set ALGENTA_API_KEY}" >/dev/null && echo "key present"
echo "$ALGENTA_BASE_URL"
```

**Expected result:** two lines print without error — `key present` and your base URL. If either line errors, export the missing variable before continuing.
{% endstep %}

{% step %}

### Confirm the engine accepts `file`

Ask the engine which connector types it supports and confirm `file` is in the list.

```bash
curl -s "$ALGENTA_BASE_URL/v1/connectors/types" \
  -H "Authorization: Bearer $ALGENTA_API_KEY"
```

**Expected result:** a JSON object with a `types` array that includes `"file"` (and its alias `"file_upload"`), for example `{"types": ["file", "file_upload", "s3", ...]}`. If `file` is present, the next step's `config` shape is accepted.
{% endstep %}

{% step %}

### Create the file connector

`POST /v1/connectors` stores the connector and returns it with `status: "untested"`. Pick the tab that matches your format — each sets the right `file_type` and payload shape.

{% tabs %}
{% tab title="CSV / TSV" %}
Pass the rows as raw `content`. Use `file_type` `csv` (default) or `tsv`; override the separator with `delimiter` if auto-detection needs help.

```bash
curl -s -X POST "$ALGENTA_BASE_URL/v1/connectors" \
  -H "Authorization: Bearer $ALGENTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Regional Sales",
    "connector_type": "file",
    "config": {
      "file_type": "csv",
      "content": "region,amount\nEMEA,1200\nAMER,3400\n"
    }
  }'
```

{% endtab %}

{% tab title="JSON" %}
Pass the document as raw `content` and point `json_path` at the records array. When the array is nested, use a dot-path such as `data.items`; when the top level is an object with a well-known wrapper (`data`, `records`, `rows`, `results`, `items`, or `value`), the engine finds the array automatically. Nested objects are flattened with dotted keys.

```bash
curl -s -X POST "$ALGENTA_BASE_URL/v1/connectors" \
  -H "Authorization: Bearer $ALGENTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Regional Sales",
    "connector_type": "file",
    "config": {
      "file_type": "json",
      "json_path": "data.items",
      "content": "{\"data\": {\"items\": [{\"region\": \"EMEA\", \"amount\": 1200}]}}"
    }
  }'
```

{% endtab %}

{% tab title="Excel" %}
Excel workbooks are binary, so pass base64-encoded bytes as `content` and select the worksheet with `sheet` (a name or a 0-based index).

```bash
XLSX_B64="$(base64 < sales.xlsx | tr -d '\n')"
curl -s -X POST "$ALGENTA_BASE_URL/v1/connectors" \
  -H "Authorization: Bearer $ALGENTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"name\": \"Regional Sales\",
    \"connector_type\": \"file\",
    \"config\": {
      \"file_type\": \"excel\",
      \"sheet\": 0,
      \"content\": \"$XLSX_B64\"
    }
  }"
```

{% endtab %}

{% tab title="On disk" %}
When the file lives on a path the engine can read, set `file_path` and leave `content` empty. This works for every `file_type` — text formats are read as UTF-8, Excel and Parquet as bytes.

```bash
curl -s -X POST "$ALGENTA_BASE_URL/v1/connectors" \
  -H "Authorization: Bearer $ALGENTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Regional Sales",
    "connector_type": "file",
    "config": {
      "file_type": "parquet",
      "file_path": "/data/sales.parquet"
    }
  }'
```

{% endtab %}
{% endtabs %}

Capture the returned `id` for the steps that follow:

```bash
export CONNECTOR_ID="$CONNECTOR_ID"
```

**Expected result:** a `201` response with the connector `id`, `"connector_type": "file"`, and `"status": "untested"`.

{% hint style="warning" %}
Provide either `content` or `file_path`. For `excel` and `parquet`, `content` must be base64-encoded bytes — raw text will not parse.
{% endhint %}
{% endstep %}

{% step %}

### Test that it parses

`POST /v1/connectors/{id}/test` parses the configured payload and reports how many records it read, then flips the connector to `live` on success or `error` on failure. Unlike a database connector there is no network round-trip — the test validates the data shape.

```bash
curl -s -X POST "$ALGENTA_BASE_URL/v1/connectors/$CONNECTOR_ID/test" \
  -H "Authorization: Bearer $ALGENTA_API_KEY"
```

**Expected result:** `{"success": true, "status": "live", "latency_ms": 4, "message": "File connector parsed 2 records from inline payload"}` (for a `file_path` connector the source is the path). On failure, `success` is `false`; `error_type` plus `recoverable` tell you whether to fix the payload and retry.
{% endstep %}

{% step %}

### Onboard the file as a dataset

`POST /v1/data/connect` registers the parsed file as a queryable dataset. Reference the connector by `connection_id` and set `connection_type` to `file` — the file's `content` or `file_path` already lives in the connector, so no `selection` is needed.

```bash
curl -s -X POST "$ALGENTA_BASE_URL/v1/data/connect" \
  -H "Authorization: Bearer $ALGENTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"connection_type\": \"file\",
    \"dataset_name\": \"regional_sales\",
    \"connection_id\": \"$CONNECTOR_ID\",
    \"visibility\": \"private\"
  }"
```

**Expected result:** `"status": "ready"` with a `dataset_id`, a `schema_summary` (row and column counts), and the `connection_id`. Save the `dataset_id`:

```bash
export DATASET_ID="$DATASET_ID"
```

{% endstep %}

{% step %}

### Run a governed query

The file is now governed data. `POST /v1/query` runs a deterministic, typed query against the dataset and returns rows under the engine's validated contract.

```bash
curl -s -X POST "$ALGENTA_BASE_URL/v1/query" \
  -H "Authorization: Bearer $ALGENTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"dataset_id\": \"$DATASET_ID\",
    \"select\": [\"region\", \"amount\"],
    \"limit\": 10
  }"
```

**Expected result:** a `200` response with the matching rows and the resolved query plan. Re-running the same request against the same dataset returns the same rows in the same order — governed queries are deterministic by design.
{% endstep %}
{% endstepper %}

## Expected result

{% hint style="success" %}
You have a saved `file` connector with `status: "live"`, a confirmed parse with a record count, a dataset onboarded from it that appears in `GET /v1/data` with a `dataset_id` and a profiled schema, and a first query returning rows. The same intent against the same dataset returns the same deterministic answer, and every access is auditable.
{% endhint %}

## Other ways to connect

The same endpoints back the `de` CLI and the Python SDK — all three hit the identical API.

```bash
de connectors create connector.json     # POST /v1/connectors from a JSON body
de connectors test $CONNECTOR_ID         # parse the configured payload
de data connect connect.json             # POST /v1/data/connect from a JSON body
de data list                             # list registered datasets
```

```python
import os
from decision_engine import AlgentaClient

client = AlgentaClient(
    api_key=os.environ["ALGENTA_API_KEY"],
    base_url="https://api.algenta.ai",
)

connector = client.create_connector(
    name="Regional Sales",
    connector_type="file",
    config={
        "file_type": "csv",
        "content": "region,amount\nEMEA,1200\nAMER,3400\n",
    },
)

test = client.test_connector(connector.id)
assert test.success, test.message

result = client.connect_data(
    connection_type="file",
    dataset_name="regional_sales",
    connection_id=connector.id,
    visibility="private",
)
print(result.status, result.dataset_id)
```

## Troubleshooting

{% hint style="warning" %}
**`Invalid JSON` on test.** The `content` is not valid JSON, or `json_path` points at a value that is not a records array. Confirm the document parses and that `json_path` reaches the array of objects.
{% endhint %}

{% hint style="warning" %}
**Excel or Parquet test fails to decode.** For `excel` and `parquet`, `content` must be base64-encoded bytes. Re-encode the file (for example `base64 < file.xlsx`) or use `file_path` instead.
{% endhint %}

{% hint style="warning" %}
**Test parses `0 records`.** Both `content` and `file_path` were empty, or the file had no data rows. Provide non-empty `content` or a readable `file_path`, and check the header row.
{% endhint %}

{% hint style="info" %}
**A path error for `file_path`.** The path is not readable by the engine. On a self-hosted deployment, mount the file where the engine runs, or send it inline as `content`. See [Troubleshooting](/help/troubleshooting.md) for the full diagnosis flow.
{% endhint %}

## Next

{% content-ref url="/pages/1hWq4uaSOQ9pNlD9Ox0f" %}
[Query governed data](/guides/governed-query.md)
{% endcontent-ref %}

{% content-ref url="/pages/bxnYz4ESKz4Thr7Unlxd" %}
[Make your first query](/guides/first-query.md)
{% endcontent-ref %}

{% content-ref url="/pages/lCqcj2JtRPaotaF5GJvm" %}
[Connectors reference](/connectors/reference.md)
{% endcontent-ref %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.algenta.ai/connectors/files.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.