Skip to main content
This walkthrough covers the full upload flow. You need the credentials and URLs from your onboarding. For the exact request and response schemas, see the API reference (ReDoc).
1

Get an access token

Standard OAuth2 client credentials grant. See Authentication for how tokens work.
Response:
Tokens are short-lived. Cache the token and reuse it until it expires — request a new one when a call returns 401 Unauthorized.All timestamps in our API (like expires_at above) are ISO 8601, UTC.
2

Create the upload

Creating the upload reserves a batch and returns its upload URL:
Optional query parameters:Response:
The upload URL:Calling create again with the same parameters returns the same URL while your upload is open. Calling it with different parameters returns 409 Conflict — submit or cancel the open upload first.
3

Upload files

The upload URL points to a directory. It is a directory URL plus a SAS query string:
Build the file URL as {DIRECTORY_URL}/{ENDPOINT_TYPE}/{FILENAME}?{SAS_QUERY}&<action parameters> — the SAS query (everything after the ? in the upload URL) always stays in the query string. Note the endpoint-type subfolder in the path (see file conventions):
For large files, append in chunks: a single append call accepts up to 100 MB (the Azure Data Lake limit). Track the offset yourself — each append continues where the previous one ended.
The last call finalizes an individual file. action=flush is part of the storage protocol and unrelated to submitting the upload (next step).
Or with the official Python SDK (azure-storage-file-datalake):
4

Submit the upload

When your upload set is complete:
{UPLOAD_ID} is the upload_id from the create response (or the upload list).
The submit response returns run_id, the identifier of the processing run. It has the same value as the submitted upload_id: the upload becomes the run. The upload list and get-by-id endpoint continue to use upload_id.Keep the run_id. It identifies the processing run when you contact us about it.
Finish uploading and finalizing all files before submitting. The URL may still accept files afterwards, but you must not use it again.During processing, aiGust takes a separate copy of the batch. Later uploads do not update that copy and are not a way to add data to the submitted run. Use a new upload for additional data.
After submitting, call create again (step 2) to open a new batch before uploading again. While your submission is still being processed, a new create for an overlapping scope returns 409 Conflict — retry once the upload disappears from the upload list.
Next: manage open uploads, errors and conventions.