> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ai-gust.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Upload files through the API

> Step by step: get an access token, create an upload, upload your files and submit the upload for processing.

This walkthrough covers the full upload flow. You need the credentials and URLs from your [onboarding](https://docs.ai-gust.io/integrate/ext-upload-overview.md#what-you-need-to-start). For the exact request and response schemas, see the [API reference](https://api.prod.ai-gust.io/api/redoc#tag/tenant_connector_external_uploads) (ReDoc).

**1. Get an access token**

Standard OAuth2 **client credentials** grant. See [Authentication](https://docs.ai-gust.io/integrate/authentication.md) for how tokens work.

```bash theme={null}
curl -X POST "{TOKEN_URL}" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials" \
  -d "client_id={YOUR_CLIENT_ID}" \
  -d "client_secret={YOUR_CLIENT_SECRET}"
```

Response:

```json theme={null}
{
  "access_token": "eyJhbGciOi...",
  "expires_in": 300,
  "token_type": "Bearer"
}
```

Tokens are short-lived. Cache the token and reuse it until it expires — request a new one when a call returns `401 Unauthorized`.

All timestamps in our API (like `expires_at` above) are **ISO 8601, UTC**.

**2. Create the upload**

Creating the upload reserves a batch and returns its upload URL:

```bash theme={null}
curl -X POST "{API_BASE_URL}/tenants/{TENANT_ID}/connectors/{CONNECTOR_ID}/uploads" \
  -H "Authorization: Bearer {ACCESS_TOKEN}"
```

Optional query parameters:

| Parameter | Default | Effect |
| - | - | - |
| `administration_code` | — | Scope this upload to a single administration |
| `full_upload` | `false` | Send all data from scratch (not just changes) |

Response:

```json theme={null}
{
  "upload_id": "440d08a1-fcba-4627-8e5c-bd6456cdcf57",
  "upload_url": "https://.../uploads/...?sig=...",
  "expires_at": "2026-10-20T12:00:00+00:00"
}
```

The upload URL:

| Property | Value |
| - | - |
| Scope | Your current upload batch only — nothing else is accessible |
| Permissions | Upload only — **no** read, list, or delete |
| Validity | **2 days** (the lifetime of an upload), renewed automatically while it is open |
| Protocol | Azure Data Lake Storage Gen2 (REST API) |

Calling `create` again with the same parameters returns the same URL while your upload is open. Calling it with **different** parameters returns `409 Conflict` — submit or cancel the open upload first.

**3. Upload files**

The upload URL points to a directory. It is a directory URL plus a SAS query string:

```
https://.../uploads/batch-123?sv=...&sig=...
└── directory URL ──────────┘└─ SAS query ──┘
```

Build the file URL as `{DIRECTORY_URL}/{ENDPOINT_TYPE}/{FILENAME}?{SAS_QUERY}&<action parameters>` — the SAS query (everything after the `?` in the upload URL) always stays in the query string. Note the **endpoint-type subfolder** in the path (see [file conventions](https://docs.ai-gust.io/integrate/ext-upload-reference.md#file-conventions)):

```bash theme={null}
# 1. Create the file
curl -X PUT "{DIRECTORY_URL}/{ENDPOINT_TYPE}/{FILENAME}?{SAS_QUERY}&resource=file" \
  -H "Content-Length: 0"

# 2. Append data (repeat in chunks for large files, tracking the offset)
curl -X PATCH "{DIRECTORY_URL}/{ENDPOINT_TYPE}/{FILENAME}?{SAS_QUERY}&action=append&position=0" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @local-file.bin

# 3. Finalize the file
curl -X PATCH "{DIRECTORY_URL}/{ENDPOINT_TYPE}/{FILENAME}?{SAS_QUERY}&action=flush&position={TOTAL_BYTES_WRITTEN}" \
  -H "Content-Length: 0"
```

For large files, append in chunks: a single append call accepts up to **100 MB** (the Azure Data Lake limit). Track the offset yourself — each append continues where the previous one ended.

> [!NOTE]
>
> The last call finalizes an individual **file**. `action=flush` is part of the storage protocol and unrelated to submitting the **upload** (next step).

Or with the official Python SDK (`azure-storage-file-datalake`):

```python theme={null}
from azure.storage.filedatalake import DataLakeDirectoryClient

client = DataLakeDirectoryClient.from_directory_url(upload_url)

file_client = client.get_file_client("invoices/my-data.parquet")
file_client.create_file()
file_client.append_data(data=file_bytes, offset=0)
file_client.flush_data(len(file_bytes))
```

| Language | Library |
| - | - |
| Python | [`azure-storage-file-datalake`](https://pypi.org/project/azure-storage-file-datalake/) |
| .NET | [`Azure.Storage.Files.DataLake`](https://www.nuget.org/packages/Azure.Storage.Files.DataLake) |
| Java | [`azure-storage-file-datalake`](https://central.sonatype.com/artifact/com.azure/azure-storage-file-datalake) |
| JavaScript/TypeScript | [`@azure/storage-file-datalake`](https://www.npmjs.com/package/@azure/storage-file-datalake) |
| Go | [`azdatalake`](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake) |
| Any | Plain HTTPS REST calls as shown above |

**4. Submit the upload**

When your upload set is complete:

```bash theme={null}
curl -X POST "{API_BASE_URL}/tenants/{TENANT_ID}/connectors/{CONNECTOR_ID}/uploads/{UPLOAD_ID}/submit" \
  -H "Authorization: Bearer {ACCESS_TOKEN}"
```

`{UPLOAD_ID}` is the `upload_id` from the create response (or the upload list).

```json theme={null}
{ "run_id": "440d08a1-fcba-4627-8e5c-bd6456cdcf57" }
```

The submit response returns `run_id`, the identifier of the processing run. It has the same value as the submitted `upload_id`: the upload becomes the run. The upload list and get-by-id endpoint continue to use `upload_id`.

Keep the `run_id`. It identifies the processing run when you contact us about it.

> [!WARNING]
>
> Finish uploading and finalizing all files before submitting. The URL may still accept files afterwards, but you must not use it again.
>
> During processing, aiGust takes a separate copy of the batch. Later uploads do not update that copy and are not a way to add data to the submitted run. Use a new upload for additional data.

After submitting, call `create` again (step 2) to open a new batch before uploading again. While your submission is still being processed, a new `create` for an overlapping scope returns `409 Conflict` — retry once the upload disappears from the [upload list](https://docs.ai-gust.io/integrate/ext-upload-reference.md#list-open-uploads).

Next: [manage open uploads, errors and conventions](https://docs.ai-gust.io/integrate/ext-upload-reference.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.