Skip to content

AWS DataSync

Protocol: JSON 1.1 (X-Amz-Target: FmrsService.<Action>) Endpoint: http://localhost:4566/ (SigV4 service datasync)

DataSync's Smithy service shape is FmrsService, so that is the target prefix the SDK and the CLI send, not the service name. Floci emulates the management plane: agents, locations, tasks and tagging. The task-execution data plane is not emulated.

Supported Actions

Action Description
CreateAgent Activate an agent; it reports ONLINE immediately
DescribeAgent Read an agent's status, endpoint type and platform version
UpdateAgent Rename an agent
DeleteAgent Delete an agent
ListAgents Page through agents, ordered by ARN
CreateLocationAzureBlob Create an Azure Blob Storage location
CreateLocationEfs Create an Amazon EFS location
CreateLocationFsxLustre Create an FSx for Lustre location
CreateLocationFsxOntap Create an FSx for NetApp ONTAP location
CreateLocationFsxOpenZfs Create an FSx for OpenZFS location
CreateLocationFsxWindows Create an FSx for Windows File Server location
CreateLocationHdfs Create a Hadoop HDFS location
CreateLocationNfs Create an NFS location
CreateLocationObjectStorage Create a self-managed object storage location
CreateLocationS3 Create an Amazon S3 location
CreateLocationSmb Create an SMB file server location
DescribeLocationAzureBlob Read back an Azure Blob location's configuration
DescribeLocationEfs Read back an EFS location's configuration
DescribeLocationFsxLustre Read back an FSx for Lustre location's configuration
DescribeLocationFsxOntap Read back an FSx for ONTAP location, including the derived FsxFilesystemArn
DescribeLocationFsxOpenZfs Read back an FSx for OpenZFS location's configuration
DescribeLocationFsxWindows Read back an FSx for Windows location's configuration
DescribeLocationHdfs Read back an HDFS location's configuration
DescribeLocationNfs Read back an NFS location's configuration
DescribeLocationObjectStorage Read back an object storage location's configuration
DescribeLocationS3 Read back an S3 location's configuration
DescribeLocationSmb Read back an SMB location's configuration
UpdateLocationAzureBlob Change an Azure Blob location and rebuild its LocationUri
UpdateLocationEfs Change an EFS location and rebuild its LocationUri
UpdateLocationFsxLustre Change an FSx for Lustre location and rebuild its LocationUri
UpdateLocationFsxOntap Change an FSx for ONTAP location and rebuild its LocationUri
UpdateLocationFsxOpenZfs Change an FSx for OpenZFS location and rebuild its LocationUri
UpdateLocationFsxWindows Change an FSx for Windows location and rebuild its LocationUri
UpdateLocationHdfs Change an HDFS location and rebuild its LocationUri
UpdateLocationNfs Change an NFS location and rebuild its LocationUri
UpdateLocationObjectStorage Change an object storage location and rebuild its LocationUri
UpdateLocationS3 Change an S3 location and rebuild its LocationUri
UpdateLocationSmb Change an SMB location and rebuild its LocationUri
ListLocations Page through locations, optionally filtered by LocationUri, LocationType or CreationTime
DeleteLocation Delete a location
CreateTask Create a transfer task; it reports AVAILABLE immediately
DescribeTask Read a task's status, locations, options and filters
UpdateTask Change a task's name, options, filters, schedule or reporting
DeleteTask Delete a task
ListTasks Page through tasks, optionally filtered by LocationId or CreationTime
TagResource Tag an agent, location or task by ARN
UntagResource Remove tags from an agent, location or task by ARN
ListTagsForResource List an agent's, location's or task's tags

Agents report ONLINE and tasks report AVAILABLE on the first read after they are created, so SDK and Terraform waiters, including the one behind aws_datasync_task, complete on their first poll. Tags passed on any create are honoured and come back from ListTagsForResource.

Each DescribeLocation* projects the create request that produced the location, so the configuration you sent is the configuration you read back. LocationUri is derived from the same request the way AWS documents it, per location type:

Location type LocationUri
S3 s3://<bucket>/<subdirectory>
EFS efs://<region>.<file-system-id>/<subdirectory>
FSx for Lustre fsxl://<region>.<file-system-id>/<subdirectory>
FSx for OpenZFS fsxz://<region>.<file-system-id>/<subdirectory>
FSx for Windows fsxw://<region>.<file-system-id>/<subdirectory>
FSx for ONTAP fsxn://<region>.<file-system-id>.<svm-id>/<subdirectory>
HDFS hdfs://<namenode-hostname>:<port>/<subdirectory>
NFS nfs://<server-hostname>/<subdirectory>
SMB smb://<server-hostname>/<subdirectory>
Object storage object-storage://<server-hostname>/<bucket>/<subdirectory>
Azure Blob azure-blob://<account-host>/<container>/<subdirectory>

DataSync's documented server-side defaults are applied on create and returned on describe: S3StorageClass STANDARD, Azure Blob BLOCK and HOT, object storage HTTPS on port 443 (80 for HTTP), SMB NTLM authentication, NFS and SMB mount version AUTOMATIC, HDFS block size 128 MiB with replication factor 3 and PRIVACY QOP, EFS InTransitEncryption NONE, and the full Options block on a task. DescribeLocationFsxOntap derives FsxFilesystemArn from the storage virtual machine ARN, as AWS does.

A task's VerifyMode default follows its TaskMode. A BASIC task, which is what CreateTask assumes when the request omits TaskMode, defaults to POINT_IN_TIME_CONSISTENT. An ENHANCED task defaults to ONLY_FILES_TRANSFERRED and cannot use POINT_IN_TIME_CONSISTENT at all, so asking for that combination fails with InvalidRequestException. UpdateTaskRequest carries no TaskMode, so a task keeps the mode it was created with and UpdateTask applies the same rule against that stored mode.

Not emulated

The task-execution data plane (StartTaskExecution, DescribeTaskExecution, ListTaskExecutions, CancelTaskExecution and UpdateTaskExecution) transfers real bytes between real storage systems, which floci has no way to model. These operations return UnknownOperationException rather than a stub success, so a caller fails fast instead of waiting on a transfer that will never progress.

Credential members are accepted on create but never stored or returned: Password, SecretKey, Azure SasConfiguration, KerberosKeytab and KerberosKrb5Conf, plus the nested Protocol.SMB.Password on the FSx protocol block. AWS also omits them from every describe response. Because no secret is created, ManagedSecretConfig is not returned; CmkSecretConfig and CustomSecretConfig are echoed as sent.

Agent ARNs referenced by a location (AgentArns, OnPremConfig.AgentArns) are not checked against the agents floci knows about, so a location can point at an agent that was provisioned elsewhere. Location ARNs on CreateTask are checked, and an unknown one fails with InvalidRequestException.

DataSync models only InvalidRequestException and InternalException, so a missing resource, a required member that was not sent, and a describe aimed at the wrong location type all return InvalidRequestException with HTTP 400. There is no ResourceNotFoundException in this API.

Configuration

Variable Default Description
FLOCI_SERVICES_DATASYNC_ENABLED true Enable or disable the service

Examples

export AWS_ENDPOINT_URL=http://localhost:4566

aws datasync create-agent --activation-key AAAAA-1AAAA-BB1CC-DDDDD-EEEEE --agent-name my-agent

aws datasync create-location-s3 \
  --s3-bucket-arn arn:aws:s3:::my-bucket \
  --subdirectory /backups \
  --s3-config BucketAccessRoleArn=arn:aws:iam::000000000000:role/datasync

aws datasync create-location-nfs \
  --server-hostname nfs.example.com \
  --subdirectory /export/home \
  --on-prem-config AgentArns=arn:aws:datasync:us-east-1:000000000000:agent/agent-...

aws datasync create-task \
  --source-location-arn arn:aws:datasync:us-east-1:000000000000:location/loc-... \
  --destination-location-arn arn:aws:datasync:us-east-1:000000000000:location/loc-... \
  --name my-task

aws datasync describe-task --task-arn arn:aws:datasync:us-east-1:000000000000:task/task-...

aws datasync list-locations