File Transfer

Copy a single file byte-for-byte from one configured data source to another — SFTP↔S3, S3→S3, or SFTP→SFTP. Precept streams the bytes straight through without re-encoding, so any file type (zip, JSON, Parquet, CSV, or arbitrary binary) is preserved exactly.

File Transfer is useful for:

• Moving export bundles — drop a generated archive from an SFTP drop-zone into an S3 bucket (or the reverse)
• Replication — copy an object from one S3 bucket to another
• Hand-off between systems — relay a file between two SFTP servers without staging it yourself

How It Works

A transfer runs asynchronously. You start it with one request, then poll a status endpoint until it finishes:

1. POST /rest/ingestion/transfer validates the request, confirms both data sources exist, and starts the copy in the background. It responds immediately with a workflowId.
2. GET /rest/ingestion/transfer/{workflowId} reports the current status — RUNNING (with best-effort progress), COMPLETED, or FAILED.

Each request copies one file. To copy many files, call the endpoint once per object.

Prerequisites

• Both endpoints must already exist as Precept data sources. Create them via the data-sources API first, then reference them here by sourceId plus a path within each.
• The destination must support uploads. Only s3 and sftp data sources can receive a transfer; any other type is rejected. The source may be s3 or sftp.

Authentication

Both endpoints require a bearer token:

Authorization: Bearer <token>

The token is either an access token from your configured identity provider (IdP) or a Precept API key (prefixed psk_).

Start a Transfer

POST /rest/ingestion/transfer

Request body:

{
  "from": { "sourceId": "<data-source-uuid>", "path": "/exports/run-42/bundle.zip" },
  "to":   { "sourceId": "<data-source-uuid>", "path": "/incoming/bundle.zip", "overwrite": false }
}

Field	Type	Required	Description
from.sourceId	UUID	yes	Data source to read from
from.path	string	yes	Path of the file within the source
to.sourceId	UUID	yes	Data source to write to (must support uploads)
to.path	string	yes	Destination path within the target
to.overwrite	boolean	no	Replace the destination if it already exists. Defaults to false.

On success, the transfer is started and you receive a 202 Accepted:

{ "workflowId": "transfer-…", "status": "started" }

Keep the workflowId — you need it to check status.

Check Status

GET /rest/ingestion/transfer/{workflowId}

The response always includes workflowId and status. The shape depends on the status:

Running — 200:

{
  "workflowId": "transfer-…",
  "status": "RUNNING",
  "progress": { "bytesTransferred": 131072, "totalBytes": 219000, "percent": 59 }
}

progress is best-effort. It may be absent briefly at the start, and percent is only present once the total size is known.

Completed — 200:

{ "workflowId": "transfer-…", "status": "COMPLETED", "bytesTransferred": 219000 }

Failed — 200:

{ "workflowId": "transfer-…", "status": "FAILED" }

TIP

Treat any status other than RUNNING or COMPLETED as terminal failure. FAILED is the common one, but the field reflects the underlying workflow state, so rare values (for example a timeout or cancellation) are possible.

Behaviour to Know

Overwrite protection

By default a transfer fails if the destination file already exists. Set "overwrite": true on the to endpoint to replace it.

• SFTP destinations create any missing parent directories automatically.
• One file per request — there is no prefix, recursive, or multi-file mode.

Errors

Failures fall into two groups: request-time rejections (returned by the POST) and run-time failures (surfaced on the status endpoint).

Request-time (from POST /rest/ingestion/transfer)

Status	Meaning
400	Invalid request body (missing or malformed fields)
404	Source or destination data source not found
405	Destination type cannot receive uploads (only s3 and sftp are valid destinations)
503	Transfer service temporarily unavailable

The status endpoint also returns 404 ("Transfer not found") if the workflowId is unknown.

Run-time (surfaced as status: "FAILED")

A transfer that starts successfully but cannot finish reports FAILED on the status endpoint. Common causes:

• The destination file already exists and overwrite was false.
• The source file does not exist at the given path.

Example

# Start the transfer
curl -X POST https://<host>/rest/ingestion/transfer \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"from":{"sourceId":"…","path":"/exports/bundle.zip"},
       "to":{"sourceId":"…","path":"/incoming/bundle.zip"}}'
# -> {"workflowId":"transfer-…","status":"started"}

# Poll for status until COMPLETED or FAILED
curl -H "Authorization: Bearer $TOKEN" \
  https://<host>/rest/ingestion/transfer/transfer-…
# -> {"workflowId":"transfer-…","status":"COMPLETED","bytesTransferred":219}

Limitations

The following are not supported:

• Multi-file, prefix, or recursive copies — copy one object per request.
• S3-compatible object stores other than AWS S3.
• A synchronous mode — every transfer is asynchronous and must be polled.