Skip to content

S3

Protocol: REST XML Endpoint: http://localhost:4566/{bucket}/{key}

Supported Operations

Category Operations
Buckets ListBuckets, CreateBucket, HeadBucket, DeleteBucket, GetBucketLocation
Objects PutObject, GetObject, GetObjectAttributes, HeadObject, DeleteObject, DeleteObjects, CopyObject
Listing ListObjects, ListObjectsV2, ListObjectVersions
Multipart CreateMultipartUpload, UploadPart, CompleteMultipartUpload, AbortMultipartUpload, ListMultipartUploads
Versioning PutBucketVersioning, GetBucketVersioning
Tagging PutBucketTagging, GetBucketTagging, PutObjectTagging, GetObjectTagging, DeleteObjectTagging
Annotations PutObjectAnnotation, GetObjectAnnotation, ListObjectAnnotations, DeleteObjectAnnotation
Policy PutBucketPolicy, GetBucketPolicy, GetBucketPolicyStatus, DeleteBucketPolicy
CORS PutBucketCors, GetBucketCors, DeleteBucketCors
Lifecycle PutBucketLifecycle, GetBucketLifecycle, DeleteBucketLifecycle
ACL PutBucketAcl, GetBucketAcl, PutObjectAcl, GetObjectAcl
Encryption PutBucketEncryption, GetBucketEncryption, DeleteBucketEncryption
Notifications PutBucketNotification, GetBucketNotification
Object Lock PutObjectLockConfiguration, GetObjectLockConfiguration, PutObjectRetention, GetObjectRetention, PutObjectLegalHold, GetObjectLegalHold
Website PutBucketWebsite, GetBucketWebsite, DeleteBucketWebsite
Pre-signed URLs Generates and validates pre-signed GET/PUT URLs
S3 Select SelectObjectContent
Public Access Block PutPublicAccessBlock, GetPublicAccessBlock, DeletePublicAccessBlock
Metrics PutBucketMetricsConfiguration, GetBucketMetricsConfiguration, ListBucketMetricsConfigurations, DeleteBucketMetricsConfiguration
Intelligent-Tiering PutBucketIntelligentTieringConfiguration, GetBucketIntelligentTieringConfiguration, ListBucketIntelligentTieringConfigurations, DeleteBucketIntelligentTieringConfiguration
Analytics PutBucketAnalyticsConfiguration, GetBucketAnalyticsConfiguration, ListBucketAnalyticsConfigurations, DeleteBucketAnalyticsConfiguration
Inventory PutBucketInventoryConfiguration, GetBucketInventoryConfiguration, ListBucketInventoryConfigurations, DeleteBucketInventoryConfiguration
Replication PutBucketReplication, GetBucketReplication, DeleteBucketReplication

RestoreObject tracks temporary restore state for GLACIER and DEEP_ARCHIVE objects per version. HeadObject reports x-amz-restore, and GetObject reports it after restoration completes; ListObjects, ListObjectsV2, and ListObjectVersions report RestoreStatus when requested with x-amz-optional-object-attributes: RestoreStatus. New restores return 202, while extending a completed restore returns 200. Restores complete immediately by default; FLOCI_SERVICES_S3_RESTORE_COMPLETION_DELAY_SECONDS sets a delay for polling workflows. The temporary copy expires at the next UTC midnight after its requested days elapse from completion. GetObject returns InvalidObjectState (HTTP 403) before restoration, while it is in progress, and after the temporary copy expires.

Replication is configuration-only: the replication configuration is stored on the bucket and round-trips through GetBucketReplication, but no objects are actually replicated.

ListBuckets includes each bucket's BucketRegion when bucket-region, continuation-token, max-buckets, or prefix is present. It uses the stored bucket region, falling back to the configured default region. Bare requests omit the field. Buckets are listed in name order and paged by max-buckets (1 to 10000), with a ContinuationToken on every page but the last; a max-buckets out of range or a token that is unreadable or was issued to another account is refused with InvalidArgument. Without max-buckets, every bucket comes back on one page. Filtering by prefix and bucket-region is not implemented; those parameters do not limit the returned bucket list.

Event Notifications

Browser and presigned POST uploads emit s3:ObjectCreated:Post, matching AWS S3. They do not emit s3:ObjectCreated:Put; use s3:ObjectCreated:* to subscribe to objects created by either method. Annotation changes emit s3:ObjectAnnotation:Put and s3:ObjectAnnotation:Delete.

PutBucketNotificationConfiguration checks SQS, SNS, and Lambda destination ARNs and verifies that each destination exists before replacing a bucket's configuration. A failed check leaves the prior configuration unchanged. Accepted configurations send an s3:TestEvent message to SQS and SNS destinations; unlike object notifications, this message has no Records array. x-amz-skip-destination-validation: true skips existence checks and test messages, but still requires well-formed destination ARNs. Destination resource-policy permissions are not checked. Existing SNS topics in another account are checked for existence without receiving a test message, regardless of IAM enforcement. Their object events are not delivered because SNS publish still resolves topics in the caller's account. With IAM enforcement enabled, SQS queues in another account are also checked without a test message. Object-event delivery is not yet subject to a shared service authorizer. AWS's same-Region destination requirement is not enforced. Cross-account Lambda destination routing is not emulated.

Object Annotations

Annotations are named UTF-8 text payloads (up to 1 MiB each, 1,000 per object version) attached to a specific object version through the four ?annotation operations. Notes on the emulation:

  • Annotation names allow letters (any language), digits, _, ., and -; names longer than 512 bytes, empty or whitespace-only names, names with other characters, and names starting with aws or s3 (case-insensitive) are rejected.
  • Payloads must be valid UTF-8 text between 1 byte and 1 MiB; anything else is rejected with 400, and non-UTF-8 payloads return 415 UnsupportedMediaType.
  • x-amz-object-if-match is validated against the parent object's ETag on put and delete.
  • Versioning semantics match AWS: annotations attach to one object version, new versions do not inherit them, overwriting a non-versioned object or deleting it drops its annotations, a delete marker preserves the underlying version's annotations, and deleting a specific version deletes its annotations. Annotation deletion is permanent.
  • CopyObject copies annotations by default; the x-amz-object-annotation-directive header (as the AWS SDK sends it; x-amz-annotation-directive is also accepted) set to EXCLUDE skips them.
  • Checksums are per-annotation and independent of the object checksum. The default algorithm is CRC64NVME. Supported: CRC32, CRC32C, CRC64NVME, SHA1, SHA256. SHA512, XXHASH64, XXHASH3, XXHASH128, and MD5 are rejected as unsupported.
  • Annotations on SSE-C encrypted objects are rejected, as on AWS, and are not copied onto SSE-C copy destinations.
  • Annotation operations serialize against object writes on the same bucket. On Object Lock-protected versions, annotation put and delete follow the same rules as object delete: governance retention requires x-amz-bypass-governance-retention (put never takes the bypass), compliance and legal hold always block.
  • The literal versionId=null names the null version only, as it does on every object operation (see Object Versions); a key without one answers NoSuchVersion, as for any missing version.
  • Annotations are stored per AWS account. With globalBucketNamespace enabled, object reads resolve cross-account but annotation reads, writes, and listings stay in the caller's account.
  • S3 Metadata annotation tables and annotation replication are not implemented.

Object Versions

A request that names versionId=null, the id ListObjectVersions reports for an object written while versioning was off or suspended, acts on that null version only: GetObject, HeadObject, GetObjectAttributes, the object ACL, tagging, retention, legal hold, annotations and a copy source. A key with no null version answers 404 NoSuchVersion, as for any missing version. If the key already has a null version, a later versioned write or delete marker keeps it as a noncurrent version. Deleting newer versions can make that null version current again.

Floci mints every version id as a UUID. On GetObject, HeadObject, DeleteObject, GetObjectAttributes, RestoreObject, the object ACL, tagging, retention, legal hold and annotations, a versionId that is neither a UUID nor null answers 400 InvalidArgument, "Invalid version id specified", as S3 answers an id it did not mint. An empty versionId answers "Version id cannot be the empty string". A UUID that Floci did not mint is handled as the id of a version that does not exist, where S3 answers 400.

GetObject and HeadObject of a key whose current version is a delete marker answer 404 with x-amz-delete-marker: true and the marker's id in x-amz-version-id, as on S3. A versionId that names a delete marker answers 405 MethodNotAllowed with the same headers, the marker's Last-Modified and Allow: DELETE.

DeleteObject without a version id in a versioning-suspended bucket makes a null delete marker the current version, as on S3. It replaces the null version or an earlier null delete marker, and the other versions stay as noncurrent versions. Deleting versionId=null removes that marker and makes the newest remaining version current again. A write in such a bucket, by PutObject, CopyObject or CompleteMultipartUpload, makes its null version current in the same way. The other versions and delete markers stay as noncurrent versions, and deleting versionId=null makes the newest of them current again.

Object Lock works only with versioning enabled. PutBucketVersioning with Suspended on a bucket with Object Lock answers 409 InvalidBucketState, "An Object Lock configuration is present on this bucket, so the versioning state cannot be changed.", and PutObjectLockConfiguration on a bucket whose versioning is unset or suspended answers 409 InvalidBucketState, "Versioning must be 'Enabled' on the bucket to apply a Object Lock configuration". The same refusal fails a CloudFormation update that suspends versioning on such a bucket. A bucket an earlier Floci release stored in either state loads in that state, and its null versions keep their Object Lock protection.

Lifecycle Expiration

A bucket's lifecycle configuration is stored as sent and its expiration actions are applied by a background sweep, every FLOCI_SERVICES_S3_LIFECYCLE_SWEEP_INTERVAL_SECONDS (60 by default). S3 also applies lifecycle asynchronously, so a lower interval only makes Floci act sooner after an object becomes due. Each enabled rule's filter (Prefix, Tag, ObjectSizeGreaterThan, ObjectSizeLessThan, an And of them, or the older rule-level Prefix) selects what it applies to, and disabled rules are ignored.

  • Expiration with Days or Date expires the current version. A day count runs from the object's last-modified time and, as on S3, is rounded up to the next midnight UTC, so an object written at 10:30 UTC under a 3-day rule expires at midnight UTC three days and some hours later. An unversioned bucket deletes the object; a versioning-enabled bucket adds a delete marker, and a versioning-suspended bucket a null delete marker.
  • ExpiredObjectDeleteMarker removes a delete marker that is the only version left of its key. A Days or Date expiration also removes such a marker once the marker itself is old enough.
  • NoncurrentVersionExpiration permanently deletes a noncurrent version, delete markers included, once it has been noncurrent for NoncurrentDays (counted from when the next newer version was written, rounded the same way) and at least NewerNoncurrentVersions newer noncurrent versions are kept ahead of it.
  • AbortIncompleteMultipartUpload aborts an upload DaysAfterInitiation after it was started, rounded the same way. Only a rule whose filter is a plain prefix selects uploads.
  • A version under Object Lock retention or a legal hold is never deleted by lifecycle.
  • Expirations are notified as s3:LifecycleExpiration:Delete and s3:LifecycleExpiration:DeleteMarkerCreated, not as s3:ObjectRemoved:*, with s3.amazonaws.com as the principal and source IP address. Their EventBridge events carry reason Lifecycle Expiration, requester s3.amazonaws.com and no source-ip-address.
  • A rule that S3 would have refused is skipped as a whole, so a bad value never widens what it deletes: Days or NoncurrentDays that is not a positive integer, NewerNoncurrentVersions outside 1 to 100 or without a Filter, a negative DaysAfterInitiation, a size bound that is not a non-negative number, a Tag without both Key and Value or with a key used twice, neither a Filter nor a rule-level Prefix, ExpiredObjectDeleteMarker next to Days or Date, a Date that does not parse, or a Status other than Enabled or Disabled. An empty Filter applies to the whole bucket. A body that is not one well-formed LifecycleConfiguration document has no rules.

PutObject, GetObject and HeadObject on the current version of an object that an Expiration rule matches return x-amz-expiration: expiry-date="Fri, 23 Dec 2012 00:00:00 GMT", rule-id="...", with the earliest matching expiry and that rule's URL-encoded id. Requests for a noncurrent version do not carry it.

Differences from S3:

  • Transition and NoncurrentVersionTransition are stored but not applied: storage classes do not change.
  • CopyObject and CompleteMultipartUpload do not return x-amz-expiration, and CreateMultipartUpload does not return x-amz-abort-date or x-amz-abort-rule-id.
  • The configuration is not validated when it is put: a rule S3 would have refused is stored and returned as sent, and only skipped when rules are applied.
  • A delete marker that lifecycle adds carries the time of the sweep, not midnight UTC.

Website Hosting

Static website requests use a bucket website hostname such as http://my-bucket.s3-website-us-east-1.localhost:4566/. Floci supports:

  • GET and HEAD requests through website hostnames
  • index-document resolution for the site root and slash-terminated prefixes
  • 302 redirects that append a slash when an index document exists below a prefix
  • configured error documents and the default website error response

HEAD returns the same status and object metadata as GET without a response body. Redirect-all and advanced routing-rule configurations are not implemented.

Range Reads

GetObject supports standard Range: bytes= requests. Partial responses return 206, Content-Range, Content-Length, Accept-Ranges: bytes, and object metadata. Whole-object checksum headers are omitted on ranged responses because the stored checksum covers the full object, not the returned byte slice, except for a range covering the whole of an object stored in one request, or completed from one part with a full-object checksum: that object is its one part, and like a range aligned with a single part, the range gets its checksum. A composite checksum is not that of any one part, nor is a checksum of an object of several parts, so their ranges get none. A range that starts past the end of the object, or a suffix of zero bytes (bytes=-0), returns 416 InvalidRange, while a Range that is not one valid range, such as several ranges, a last byte before the first, or a malformed value, is ignored and the whole object is returned with 200, as S3 does. Full-object and ranged reads stream object bytes from storage instead of materializing the response body first.

Suffix ranges against empty objects return an empty 200 response, matching AWS behavior used by clients that issue tail reads.

HeadObject honors Range as the API Reference describes: a satisfiable range changes Content-Length to the length of the range, and one that cannot be satisfied returns 416, while a Range that is not one valid range, such as several ranges or a last byte before the first, is ignored. Like a ranged GetObject, it leaves out the whole-object checksum headers unless the range covers the whole of an object that is one part.

GetObject and HeadObject also read one part of an object with the partNumber query parameter, which works like a range covering that part: 206, its Content-Range and Content-Length, the object's ETag, and, for an object completed from a multipart upload, x-amz-mp-parts-count with the number of its parts. An object stored in one request is its own part 1, and part 1 of an empty object returns an empty 200 with no Content-Range, which no byte range describes. A part streams from storage like a range, so the parts of a large object can be downloaded one at a time. With checksum mode on, a part of an object with a COMPOSITE checksum gets the part's own checksum, of type COMPOSITE, and a part that is all of its object gets the object's checksum; any other part of a FULL_OBJECT multipart object gets none. A part the object does not have, including any part number above 10,000, returns 416 InvalidPartNumber naming PartNumberRequested and ActualPartCount, partNumber together with a Range header returns 400 InvalidRequest, and a part number below 1 returns 400 InvalidArgument.

An If-Range header makes the range conditional, so a resumed download never mixes two versions of an object. The range is served while the validator still describes the object: a quoted ETag equal to the object's (a multipart ETag compared whole, -N suffix included), or an HTTP-date equal to its Last-Modified, an exact match as RFC 9110 section 13.1.5 requires, so a later date does not qualify. Otherwise, including a weak (W/) or malformed validator, GetObject returns the whole object with 200. A presigned URL that signs Range must also sign an If-Range the request carries, or the request fails with 403 AccessDenied naming it in HeadersNotSigned.

S3 Select

SelectObjectContent runs SQL queries directly against S3 objects without downloading the entire file. Floci supports CSV, JSON Lines, JSON arrays, and Parquet inputs. The SQL dialect follows AWS S3 Select SQL reference.

Execution modes

Floci chooses the execution engine automatically based on the input format and whether the floci-duck sidecar is running:

Condition Engine Notes
Input is Parquet floci-duck (required) DuckDB's read_parquet: sidecar must be available
Input is CSV with FileHeaderInfo=USE and floci-duck is running floci-duck Full DuckDB SQL: all operators, LIKE, BETWEEN, IN, IS NULL, AND/OR/NOT
Input is JSON and floci-duck is running floci-duck read_json_auto: supports JSON Lines and JSON arrays
Input is CSV with FileHeaderInfo=NONE or IGNORE, or floci-duck is not running Java evaluator Supports SELECT *, column projection, simple WHERE with =, !=, <, >, <=, >=, LIKE, BETWEEN, IN, IS NULL, AND/OR/NOT, LIMIT

The floci-duck sidecar starts lazily on the first Athena query. Until then, isAvailable() returns false and S3 Select falls back to the Java evaluator for CSV and JSON. Once the sidecar is running, subsequent S3 Select calls route through DuckDB automatically.

If floci-duck is not running and the object is Parquet, S3 Select returns an error: Parquet decoding requires DuckDB.

The Java evaluator reads the object a record at a time and writes the result in Records events as it goes, stopping once a LIMIT is reached, so neither the object nor the result is held in memory. As on S3, a record can be split across Records events, and a record in the input or the result can be at most 1 MB (1,048,576 characters): a longer one fails the query with OverMaxRecordSize, which arrives as an error message at the end of the event stream once the response has begun. BytesScanned and BytesProcessed in the Stats event count the bytes the query read, so one a LIMIT stops early reports less than the object's size. A query floci-duck runs is read by DuckDB, and its result is held in memory before it is returned.

FileHeaderInfo modes (CSV)

Value Behavior
USE First row is the header; column names are available in WHERE and SELECT
IGNORE First row is skipped and not included in output; only positional _N references work
NONE All rows are data; only positional _N references work (e.g. WHERE _1 = 'Alice')

Supported SQL operators

When using the Java evaluator (no floci-duck, or CSV with FileHeaderInfo=NONE/IGNORE):

  • Comparison: =, !=, <>, <, >, <=, >=
  • Pattern matching: LIKE (supports % and _ wildcards)
  • Range: BETWEEN ... AND ...
  • Set membership: IN (...)
  • Null checks: IS NULL, IS NOT NULL
  • Logical: AND, OR, NOT
  • Clauses: SELECT *, column projection, LIMIT

Output formats

S3 Select supports cross-format output: a CSV object can produce JSON output and vice versa.

Input Output Format
CSV CSV Default: comma-separated values
CSV JSON One JSON object per row: {"col1":"val1","col2":"val2"}
JSON JSON Default: one JSON object per line
JSON CSV Comma-separated values, values quoted when they contain commas or newlines

Example

export AWS_ENDPOINT_URL=http://localhost:4566

# Upload a CSV file
printf 'name,age,city\nAlice,30,New York\nBob,25,\nCharlie,35,London\n' \
  | aws s3 cp - s3://my-bucket/people.csv

# Query with WHERE and column projection
aws s3api select-object-content \
  --bucket my-bucket \
  --key people.csv \
  --expression "SELECT name, city FROM S3Object WHERE age >= 30" \
  --expression-type SQL \
  --input-serialization '{"CSV":{"FileHeaderInfo":"USE"}}' \
  --output-serialization '{"CSV":{}}' \
  /dev/stdout

# IS NULL check
aws s3api select-object-content \
  --bucket my-bucket \
  --key people.csv \
  --expression "SELECT name FROM S3Object WHERE city IS NULL" \
  --expression-type SQL \
  --input-serialization '{"CSV":{"FileHeaderInfo":"USE"}}' \
  --output-serialization '{"CSV":{}}' \
  /dev/stdout

# JSON Lines input
printf '{"name":"Alice","score":95}\n{"name":"Bob","score":72}\n' \
  | aws s3 cp - s3://my-bucket/scores.json

aws s3api select-object-content \
  --bucket my-bucket \
  --key scores.json \
  --expression "SELECT * FROM S3Object WHERE score > 80" \
  --expression-type SQL \
  --input-serialization '{"JSON":{"Type":"LINES"}}' \
  --output-serialization '{"JSON":{}}' \
  /dev/stdout

Mock mode note

When FLOCI_SERVICES_ATHENA_MOCK=true is set, Athena queries are stubbed but floci-duck does not start. In that configuration, S3 Select uses the Java evaluator for CSV and JSON. Parquet queries will fail unless FLOCI_SERVICES_DUCK_URL points to an already-running floci-duck instance.

Bucket Names

Floci does not hold bucket names to AWS's full DNS naming rules, so a name AWS would refuse is usually accepted here. The exception is a name that would not stay a single directory under its account's storage root: an empty name, ., .., a name containing a path separator, or one of Floci's reserved storage directories (.accounts, .versions, .annotations) is rejected with InvalidBucketName. On the persistent and hybrid backends such a name either climbs out of the owning account's directory or collides with object version and annotation data.

Object Keys

S3 accepts keys of up to 1024 bytes, and so does Floci in every storage mode. On the disk-backed modes the body file is laid out as <key>.s3data under the bucket's directory, and a single key segment can run past the 255-byte file-name limit the filesystem enforces. Such a segment is stored on disk as a readable prefix followed by the SHA-256 of the whole segment. A segment that fits and can be represented as written keeps its raw path, unless it already looks like an encoded name. Those names are encoded too, so they cannot overwrite an encoded object's body. The mapping is deterministic, so the same key always resolves to the same file, and listing answers with the original key, which the object index holds, never with the encoded on-disk name.

When Floci runs on Windows, a segment Windows cannot hold as a file name is stored the same way: one containing a character Windows refuses (<>:"|?*, \ or a control character), one ending in a dot or a space, which Windows drops, and one naming a device that Microsoft's file naming rules reserve, such as CON, NUL.txt or COM1. For an unversioned body, the last segment is judged with .s3data appended, the name it is stored under, so a key such as logs/report. keeps its earlier path. Versioned bodies use the key as a directory path, so its last segment is judged without that suffix.

Older Windows releases of Floci could fold keys such as dir./file and dir\file onto another key's path. Bodies at those ambiguous paths are not migrated automatically, because their intended key cannot be determined safely. Existing raw filenames that now match the encoded-name pattern are also not migrated automatically. Their metadata can remain while a read of the new encoded path fails; these objects need to be rewritten from a known copy of their contents.

A key keeps the slashes it is sent with: PUT /bucket//file.txt writes /file.txt, and a key made only of slashes is an object too, so PUT /bucket// writes the key / rather than creating the bucket. One trailing slash after the bucket, /bucket/, is still the bucket. The key is read from the path as it arrived wherever it is needed, so the object stored, the object an s3:ExistingObjectTag condition describes and the object ARN the request is authorized against are the same one (see IAM).

Bucket Regions

CreateBucket follows AWS's region rules, in every partition. The endpoint a request is signed for decides what it accepts:

  • The us-east-1 endpoint takes a LocationConstraint naming any other region, and no constraint at all (the bucket then lives in us-east-1). A constraint of us-east-1 itself is InvalidLocationConstraint.
  • Every other regional endpoint requires a constraint naming exactly its own region. No constraint, or a different region (including us-east-1), is IllegalLocationConstraintException.

The AWS SDKs send the constraint for any region but us-east-1, so SDK clients are unaffected. A raw-HTTP client that signs for another region and sends an empty body has to add the CreateBucketConfiguration; the signing region alone does not place a bucket. GetBucketLocation answers an empty constraint only for us-east-1. See AWS Partitions for how this applies in China and GovCloud.

Global Bucket Namespace

Object deletion, including DeleteObjects batches, uses the named bucket's owning account for stored objects, versions, annotations and notifications. Existing authorization checks still use the original caller, and DeleteBucket remains scoped to that caller's account.

By default S3 buckets are isolated per account (see Multi-Account Isolation), so two accounts can each hold a bucket with the same name. Set FLOCI_SERVICES_S3_GLOBAL_BUCKET_NAMESPACE=true to make bucket and object resolution span every account's partition: a bucket created in one account then resolves cross-account, matching real S3 where bucket names are globally unique. Enable this when a workload (for example an LZA log-archive or CDK asset bucket) is created in one account and read from another.

Creating a bucket whose name another account owns returns 409 BucketAlreadyExists. Concurrent creates and deletion share the same bucket-name lock. Recreating an owned bucket succeeds in us-east-1 and returns 409 BucketAlreadyOwnedByYou in other regions. ListObjects and ListObjectsV2 read the named bucket's owner's objects; ListBuckets stays caller-scoped. With IAM enforcement enabled, a foreign bucket's policy neither grants nor denies creation of a bucket with the same name. Caller permission checks, including requested creation settings, still run before the collision check. A bucket owned by the caller retains its existing policy checks. The default per-account namespace and authentication settings are unchanged. Bucket-name reuse after deletion is immediate in Floci; AWS may temporarily return 409 OperationAborted.

Floci-managed staging buckets for Lambda packages, Spectrum, Firehose conversion and CUR remain account scoped even when the global namespace is enabled. Their existing names and object paths are preserved; the internal creation path does not change public CreateBucket collision handling.

Why the flag exists, and what it trades away. Floci partitions every resource by the caller's account, which is the right model for services whose names are account-scoped in AWS. S3 bucket names are not: they are unique across all accounts, a bucket lives in exactly one owning account, and whether another account may touch it is decided by policy rather than by the name being unreachable. A hard per-account partition therefore lets two accounts hold different buckets under one name (a state AWS cannot reach), and makes a legitimate cross-account call fail with NoSuchBucket instead of being authorized or denied on its merits. With the flag on, a caller that names another account's bucket resolves it and only IAM and bucket-policy evaluation stand between the caller and the object: AWS-faithful, but stricter than the implicit isolation Floci gives you elsewhere, provided both FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED (identity-policy checks, applied before the request reaches S3) and FLOCI_SERVICES_S3_ENFORCE_AUTH (the S3-specific flag that gates bucket-policy evaluation itself, in S3Service.authorizeS3Read/authorizeS3Write) are also true. Both default to false, and bucket-policy evaluation in particular is a no-op while FLOCI_SERVICES_S3_ENFORCE_AUTH is off, regardless of the IAM flag, so in the default configuration enabling the global namespace does not trade per-account isolation for a policy-gated boundary, it removes the only isolation Floci gives S3 for that bucket and replaces it with nothing. Mutations are written back to the bucket's owning account rather than forked into the caller's, and ListBuckets stays caller-scoped while ListObjects and ListObjectsV2 list the named bucket's owner-scoped objects, so the flag never reassigns ownership or leaks a bucket inventory. It is off by default, turn it on when emulating a multi-account estate whose real behaviour depends on the global namespace, and leave it off when the per-account partition is the isolation you are relying on. Account-scoped S3 state, notably the account-level Block Public Access configuration, is never widened by this flag. Note also that retrofitting a global namespace onto state that was previously partitioned per account means that if two accounts already hold a bucket under the same name (a state real AWS can't reach, but Floci's prior per-account isolation could), resolution picks whichever account's copy the backend happens to iterate to first; this is unlikely to be reachable on a fresh LZA-driven account structure, but is worth knowing if you enable the flag on an estate with pre-existing same-name buckets.

Signature Verification

AWS treats authentication and authorization as two steps. S3 first verifies the SigV4 signature of a signed request and refuses one that does not verify, whatever the bucket policy says. Only then does it decide, from IAM policies, the bucket policy and ACLs, whether the caller may perform the operation. An unsigned request is anonymous, and is allowed only when a policy or ACL grants access to everyone.

Floci verifies signatures when either of two flags is on:

FLOCI_AUTH_VALIDATE_SIGNATURES FLOCI_SERVICES_S3_ENFORCE_AUTH
Authorization header signatures Verified Verified
Presigned URL signatures (X-Amz-Signature) Verified Verified
Presigned POST signatures (browser form) Verified when the form carries signature fields Verified, and required
Bucket policy and ACL evaluation Not evaluated Evaluated
Unsigned (anonymous) requests Allowed Allowed only when a policy or ACL grants public access

For a signed request, verification rebuilds the canonical request from the wire path, the query string, the headers named in SignedHeaders and the declared payload hash, derives the signing key from the access key's secret, and compares the signatures in constant time. It rejects:

  • a signature that does not verify, or one computed for another key or method, with 403 SignatureDoesNotMatch
  • an access key that is neither test nor registered in Floci IAM (CreateAccessKey keys, and STS keys with their session token), with 403 InvalidAccessKeyId. Under FLOCI_AUTH_VALIDATE_SIGNATURES alone, a presigned URL from such a key falls back to the older FLOCI_AUTH_PRESIGN_SECRET check instead, which an SDK-made URL fails with 403 SignatureDoesNotMatch
  • temporary credentials past their expiry, sent with their own session token, with 400 ExpiredToken, whichever placement carries the signature. Once Floci has removed an expired session it no longer knows the key, and answers as for one that is not registered
  • a header-signed request more than 15 minutes from server time, with 403 RequestTimeTooSkewed
  • a body that does not match a declared x-amz-content-sha256 digest, with 400 XAmzContentSHA256Mismatch
  • an Authorization header whose algorithm is neither AWS4-HMAC-SHA256 nor AWS4-ECDSA-P256-SHA256 (SigV4A), with 400 AuthorizationHeaderMalformed
  • a SigV4A request, which is verified the way AWS does it: Floci derives the ECDSA P-256 key pair from the access key's secret and checks the signature against the public half. The Authorization header and a presigned URL (X-Amz-Algorithm=AWS4-ECDSA-P256-SHA256) are both verified. Its credential scope has no region, so the request must sign the region set: a header request without a signed x-amz-region-set is 400 AuthorizationHeaderMalformed, and a presigned URL without X-Amz-Region-Set is 400 AuthorizationQueryParametersError. A set naming one region serves that region, and a wildcard or a list serves the default region. Floci does not check the set against the bucket's region. A browser POST signed with SigV4A is not accepted
  • a presigned URL that carries any of X-Amz-Algorithm, X-Amz-Credential or X-Amz-Signature but lacks one of X-Amz-Algorithm, X-Amz-Credential, X-Amz-Signature, X-Amz-Date, X-Amz-SignedHeaders or X-Amz-Expires, with 400 AuthorizationQueryParametersError

FLOCI_AUTH_VALIDATE_SIGNATURES is authentication alone: use it when forged or stale credentials must fail but buckets should stay reachable without policies. It is not specific to S3: the same flag verifies the Authorization header signature of every other service (see Signature Validation). FLOCI_SERVICES_S3_ENFORCE_AUTH adds authorization, so it also decides anonymous access the way AWS does: a bucket that anonymous clients read needs a public-read bucket policy or ACL.

A workload must sign with credentials Floci can verify once either flag is on. A container Floci launches without a role, such as a Lambda function with no execution role, is given placeholder credentials that no flag accepts, so give it a role, or sign with test/test.

Presigned uploads ignore checksum values, algorithms, and types supplied only in the query string, matching AWS S3. This includes the empty-body CRC32 (AAAAAA==) that the JavaScript SDK adds when presigning before the file is available. These parameters still participate in signature verification, but are not used to validate the body or stored as the object's checksum. Actual checksum headers and streaming checksum trailers remain validated as usual.

Bucket Policy Enforcement

Floci evaluates S3 bucket policies for authenticated (signed) requests when policy enforcement is enabled. Two flags control this evaluation:

  • FLOCI_SERVICES_S3_ENFORCE_AUTH: gates bucket policy authorization directly within S3Service for bucket read, object write, and bucket write operations. It also turns on signature verification.
  • FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED: activates the global IAM request filter to evaluate identity-based policies and resource policies before requests reach the service layer.

Both flags default to false for backward compatibility.

Evaluated Policies and Operations

When FLOCI_SERVICES_S3_ENFORCE_AUTH or FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED is active: - The bucket's attached policy document (configured via PutBucketPolicy) is retrieved. - The caller's authenticated principal ARN is resolved from the signing credentials (for example, arn:aws:iam::<account>:user/<name>, arn:aws:iam::<account>:role/<name>, or arn:aws:iam::<account>:root). - Policy statements are evaluated for the requested S3 action (e.g. s3:GetObject, s3:PutObject, s3:PutBucketPolicy) and resource ARN (arn:aws:s3:::bucket or arn:aws:s3:::bucket/key). - An object request is checked as the actions IAM checks for it (see IAM): the version action when it names a versionId, and the actions it needs besides its own, such as s3:PutObjectTagging for an object created with tags. The policy is offered the tags and Object Lock settings the request sets (s3:RequestObjectTag/<key>, s3:RequestObjectTagKeys and the s3:object-lock-* keys) and the encryption it asks for (s3:x-amz-server-side-encryption and s3:x-amz-server-side-encryption-aws-kms-key-id, from its headers, or for a part or a completion from what its upload recorded; see IAM) besides the keys that say who is calling, but not the other S3 keys, such as s3:ExistingObjectTag or s3:versionid. An anonymous request's public-policy check is offered none. An anonymous object creation that a public bucket ACL's WRITE grant admits may also apply its ACL, tags and Object Lock settings, since an ACL grant names no actions, unless a public bucket policy decides the setting's action.

Supported Principal Types

Bucket policy statements can specify principals using Principal or NotPrincipal with scalar strings or arrays:

  • Wildcard: "*" or {"AWS": "*"} matches any caller.
  • IAM User: {"AWS": "arn:aws:iam::<account>:user/<name>"}.
  • IAM Role: {"AWS": "arn:aws:iam::<account>:role/<path><name>"}. A session of the role matches it through the role's own ARN, path included.
  • AWS Account: {"AWS": "<12-digit-account-id>"} or {"AWS": "arn:aws:iam::<account>:root"} matches any principal belonging to the specified account.
  • Canonical user: {"CanonicalUser": "<canonical-id>"} names an account by its S3 canonical user ID and matches any principal belonging to that account, as an account principal does. Floci derives the canonical ID deterministically from the account ID as its SHA-256 hex digest, a 64-character lowercase hex string, and reports it wherever S3 identifies an owner: the Owner of ListBuckets, ListParts and both ACLs, the Initiator of ListParts, and every CanonicalUser grantee of an ACL Floci generates. The account ID itself still matches, for policies written when Floci reported the account as the canonical ID.
  • Service Principal: {"Service": "<service>.amazonaws.com"}.
  • NotPrincipal: Inverts matching so the statement applies to any principal not matching the specified patterns. On a Deny, an IAM caller is spared only when the list names its account as well as the caller, and never when it has a permissions boundary attached; see Resource-based policies. Each entry is written as a string, so a NotPrincipal whose entries are no strings, an empty list among them, names nobody: it carves nobody out, so the statement covers nobody rather than everyone, on an anonymous request and a signed one alike. Block Public Access reads it the other way round, and deliberately: S3 "begins by assuming that the policy is public" and calls a policy non-public only when it grants "only to fixed values" of a principal or of one of the condition keys below, so a NotPrincipal leaves the policy public whether it names anybody or not.

Every check against a bucket policy, the anonymous paths and the Block Public Access status included, holds a statement's Effect to Allow or Deny as written, the IAM User Guide saying that "The Effect value is case sensitive", so a statement spelling either word another way states nothing and one bucket policy answers a signed request and an unsigned one the same way.

The same matching applies to every check S3 makes against a bucket policy, including the source of CopyObject and UploadPartCopy.

Evaluation Rules

  1. An explicit Deny always wins and rejects the request.
  2. An explicit Allow grants access.
  3. For bucket owners (principals within the account that owns the bucket), the default decision is Allow unless explicitly denied.
  4. For non-owner principals, an explicit Allow in the bucket policy is required; neutral evaluation results in Deny.
  5. Unsigned (anonymous) requests and ACL-based public access fall back to standard S3 ACL and anonymous authorization paths.
  6. Unknown access key IDs return HTTP 403 with InvalidAccessKeyId.

With FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED also on, the bucket policy and the caller's identity policies are combined the way AWS combines them. For same-account access, an explicit Allow in either policy grants the request, so a principal that only the bucket policy names, by its user, role or session ARN or as "*", needs no identity policy at all. A statement that names only the account, by its id, its root ARN or its canonical user ID, delegates to the account, as on AWS, so within that account the caller's identity policy has to allow the request. For cross-account access, both the bucket policy and the caller's identity policy must allow the action. An explicit Deny in either policy denies the request in both cases.

Wire Response Shape

When a request is denied by bucket policy enforcement, Floci returns HTTP 403 with an S3 XML error body including the <Resource> element:

<Error>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
    <Resource>/my-bucket/my-key</Resource>
    <RequestId>...</RequestId>
</Error>

Object Ownership and ACLs

New buckets start with BucketOwnerEnforced ownership, matching the current S3 default. GetBucketOwnershipControls returns that setting immediately after CreateBucket. Bucket and object ACL writes are disabled, while uploads without an ACL and uploads using bucket-owner-full-control remain supported. To use ACL grants, set x-amz-object-ownership to ObjectWriter or BucketOwnerPreferred during CreateBucket, or change the mode later with PutBucketOwnershipControls. Existing stored buckets keep their previously configured ownership mode.

Canned ACLs and grant headers

A request sets its ACL either with a canned ACL (x-amz-acl) or with grant headers (x-amz-grant-*), not both. As on AWS, CreateBucket, PutBucketAcl, PutObjectAcl, PutObject, CopyObject and CreateMultipartUpload refuse a request carrying both with InvalidRequest (400), Specifying both Canned ACLs and Header Grants is not allowed, and change nothing. A header with a blank value counts as absent.

Block Public Access

Floci enforces all four Block Public Access settings. A bucket's configuration is combined with the bucket owner account's and the most restrictive of the two wins, which for four independent booleans is a per-flag OR: a flag set at either level is in force.

With BucketOwnerEnforced object ownership, ACL writes are rejected and existing object ACL grants no longer provide access. Before enabling it, the bucket ACL must grant access only to the owner; otherwise PutBucketOwnershipControls returns InvalidBucketAclWithObjectOwnership (400).

Setting What it does Needs enforce-auth
BlockPublicPolicy PutBucketPolicy returns AccessDenied (403) when the submitted policy is public No
BlockPublicAcls PutBucketAcl, PutObjectAcl and a PutObject carrying a public ACL return AccessDenied (403) No
RestrictPublicBuckets A bucket whose policy is public serves only the owner account: anonymous and cross-account callers are denied Yes
IgnorePublicAcls A public ACL on the bucket or on an object stops granting access Yes

The split follows AWS. The first two reject the write that would introduce public access, and AWS applies them whoever the caller is, so they need no flag: put-public-access-block followed by a public put-bucket-policy returns 403 in Floci's default configuration. The second two suppress access that an existing policy or ACL would grant, which only has meaning once anonymous authorization runs at all, so they take effect when FLOCI_SERVICES_S3_ENFORCE_AUTH is on. With that flag off, Floci cannot tell an anonymous caller from a signed one and every request is authorized regardless.

A bucket nobody configured enforces the default GetPublicAccessBlock reports for it: all four settings on. DeletePublicAccessBlock or PutPublicAccessBlock replaces that default.

When upgrading, a workflow that already sets BlockPublicPolicy or BlockPublicAcls may now receive 403 on a later public policy or ACL write, even with enforce-auth off. A bucket saved by an earlier version, including one with no configuration, now refuses public policies and ACLs until its configuration is deleted or set. This matches AWS behavior.

As in AWS, enabling a setting never rewrites stored state. An existing public policy or ACL stays exactly as written, and clearing the setting makes the bucket public again.

The meaning of "public"

An ACL is public when it grants any permission to the AllUsers or AuthenticatedUsers predefined groups. This is broader than the grants anonymous authorization consults: a WRITE_ACP grant to AllUsers is public here even though it gives an anonymous caller no read.

A bucket policy is assumed public and then checked for a reason it is not. A statement with a wildcard principal ("Principal": "*", {"AWS": "*"}, or an Allow on NotPrincipal) is public unless a positive condition pins one of aws:PrincipalArn, aws:PrincipalAccount, aws:PrincipalOrgID, aws:PrincipalOrgPaths, aws:SourceArn, aws:SourceVpc, aws:SourceVpce, aws:SourceOwner, aws:SourceAccount, aws:userid, s3:DataAccessPointArn or s3:DataAccessPointAccount to a fixed value, one containing neither a wildcard nor an IAM policy variable. In a bucket policy, s3:DataAccessPointArn may contain a wildcard in the access point name if the account ID is fixed, for example arn:aws:s3:us-west-2:123456789012:accesspoint/*. aws:SourceIp counts too, but only for a range no broader than /8 (IPv4) or /32 (IPv6), matching AWS's treatment of very wide CIDR blocks as public.

For multivalued condition keys such as aws:PrincipalOrgPaths, ForAnyValue: with a positive operator can narrow the grant when each value is fixed. ForAllValues: alone does not narrow it because the condition also matches when the key is missing.

A single public statement makes the whole policy public. As on AWS, RestrictPublicBuckets then withholds even the non-public cross-account delegation another statement grants, until the public statement is removed.

Known gaps

  • Floci does not model access points, so the access-point variants (PutAccessPointPolicy, the VPC-origin rule, the different s3:DataAccessPointArn treatment) do not apply.
  • AWS's organization-level Block Public Access policies are not modelled; only the bucket and account levels combine.
  • CreateBucket applies compatible ACL headers but rejects public ACLs with HTTP 400, including on ObjectWriter buckets, because new buckets start with Block Public Access enabled. After creation, an ObjectWriter bucket without an explicit PutPublicAccessBlock also refuses a public PutBucketAcl with AccessDenied (403), because the default configuration is enforced.
  • GetBucketAcl returns the stored ACL rather than the effective one, so it does not reflect an IgnorePublicAcls that is suppressing a grant.
  • RestrictPublicBuckets exempts AWS service principals on AWS. Floci reaches the signed path only for IAM principals, so there is nothing to exempt.

Account-level operations

The s3control PutPublicAccessBlock, GetPublicAccessBlock and DeletePublicAccessBlock operations are keyed by the x-amz-account-id header, stored per account, and persisted like the rest of S3's state so a restart does not silently drop the control. As in AWS, the header must name the caller's own account; a mismatch returns AccessDenied (403). Floci keeps one deliberate exception: the configured default (management) account may act on any account, because Floci's launched Lambdas run on placeholder credentials that resolve to the management account instead of assuming a role in the target account, which is how LZA's Custom::PutPublicAccessBlock resource reaches this API.

IAM enforcement of S3 Control

With IAM enforcement on, every /v20180820 route is authorized as the s3: action the Service Authorization Reference gives it, against the resource the route names, because S3 Control clients sign with the s3 scope. The s3control model's routes and the actions are vendored from AWS (rest-routes.json and iam-authorized-actions.json, regenerated by make aws-data-sync), and IamRestRouteCoverageTest checks every route Floci serves against them. The account-level Block Public Access is the one exception, written down where that data is checked: the reference files S3's and S3 Control's GetPublicAccessBlock, PutPublicAccessBlock and DeletePublicAccessBlock under the bucket actions, while the S3 User Guide's Block Public Access page gives the account level s3:PutAccountPublicAccessBlock to write and s3:GetAccountPublicAccessBlock to read, and a delete the same action as the write it undoes.

Operation Action Resource
ListTagsForResource, TagResource, UntagResource s3:ListTagsForResource, s3:TagResource, s3:UntagResource The ARN the route names, as written
ListAccessPoints s3:ListAccessPoints All resources (*; AWS publishes no resource type, so a bucket-scoped statement grants nothing)
PutPublicAccessBlock, DeletePublicAccessBlock s3:PutAccountPublicAccessBlock All resources (*)
GetPublicAccessBlock s3:GetAccountPublicAccessBlock All resources (*)

A tagging route carries the bucket's ARN in its path, and both documented forms arrive on the wire: the global form arn:<partition>:s3:::<name> that the Go SDK and Terraform's provider send, and the regional form arn:<partition>:s3:<region>:<account>:bucket/<name> that the operation's own request pattern and the AWS CLI examples use. The ARN as the caller wrote it is the resource the request is checked against, so a policy may name either form, and the ARN is read once, by the IAM check, the bucket-policy lookup and the handler together: the regional form's account is the account whose bucket the request acts on, so a request naming another account's bucket reads and writes that account's bucket, and answers NoSuchBucket when that account holds no bucket of the name, never the caller's same-named one. Query-string parameters the control plane ignores, such as ?tagging, do not re-point the authorization at a data-plane action.

A denial answers in S3 Control's own REST-XML <ErrorResponse> envelope, which follows the route rather than the resolved action, so the s3control SDK deserializers read the message that names the missing action, which Terraform's AWS provider matches to fall back from ListTagsForResource to GetBucketTagging.

Not Implemented

A bucket sub-resource the AWS model defines but Floci does not serve, such as S3 Metadata (?metadataConfiguration, ?metadataTable and the journal, inventory and annotation table updates), attribute-based access control (?abac) or CreateSession (?session), is answered 501 NotImplemented. A sub-resource sent with a method AWS does not define for it, such as DELETE /bucket?versioning, is answered 405 MethodNotAllowed. Neither runs as the bucket's own operation, so neither creates, lists nor deletes the bucket.

Configuration

Variable Default Description
FLOCI_SERVICES_S3_ENABLED true Enable or disable the service
FLOCI_SERVICES_S3_DEFAULT_PRESIGN_EXPIRY_SECONDS 3600 Default pre-signed URL expiry (1 hour)
FLOCI_AUTH_PRESIGN_SECRET local-emulator-secret Secret used to sign pre-signed URLs
FLOCI_SERVICES_S3_ENFORCE_AUTH false Verify SigV4 signatures and evaluate bucket policies and ACLs, including for anonymous callers (see Signature Verification)
FLOCI_AUTH_VALIDATE_SIGNATURES false Verify SigV4 signatures without evaluating bucket policies or ACLs (see Signature Verification)
FLOCI_SERVICES_S3_GLOBAL_BUCKET_NAMESPACE false Enable cross-account bucket and object resolution
FLOCI_SERVICES_S3_LIFECYCLE_SWEEP_ENABLED true Apply bucket lifecycle expiration rules in the background (see Lifecycle Expiration)
FLOCI_SERVICES_S3_LIFECYCLE_SWEEP_INTERVAL_SECONDS 60 How often the lifecycle sweep runs

Storage

Object bodies are files under the S3 data directory; the object index, s3-objects.json, holds the metadata of every object in every bucket. Under persistent mode the index is journaled instead of rewritten on every PutObject, CopyObject or DeleteObject: the change is appended to s3-objects.wal, and s3-objects.json is rewritten from memory on the FLOCI_STORAGE_WAL_COMPACTION_INTERVAL_MS cadence and at shutdown, so the cost of a write no longer grows with the number of stored objects. Bucket definitions and the other S3 stores are still rewritten on every change. See Storage Modes.

In memory mode each object body is a single in-memory byte array, so an object can hold at most about 2 GiB: a PutObject over that, an UploadPart or UploadPartCopy that takes a multipart upload past it, or a CompleteMultipartUpload over it fails with EntityTooLarge. The disk-backed modes write a PutObject or UploadPart body to disk as it arrives, working out its ETag and checksums on the way, and assemble a multipart upload file to file, so no object passes through the heap; set FLOCI_STORAGE_SERVICES_S3_MODE=persistent to keep only S3 on disk. Every request body is still capped by FLOCI_PROTOCOLS_MAX_REQUEST_SIZE, 2048 MB by default: to accept the 5 GiB S3 allows in one PutObject or UploadPart, raise it a little past 5120, since the aws-chunked framing SDKs send makes a request slightly larger than its data. A request whose Content-Length is over the limit gets an empty 413 Request Entity Too Large before S3 sees it. As on S3, more than 5 GiB of data in one PutObject or UploadPart is rejected with EntityTooLarge, whatever the limit is raised to: before any of it is read when the request declares its size, in Content-Length or, for an aws-chunked body, x-amz-decoded-content-length, and otherwise as soon as it passes 5 GiB. As S3's does, the error names the most allowed in MaxSizeAllowed and, when the request declared it, the size proposed in ProposedSize. A request rejected before its body is read that sent Expect: 100-continue is answered with Connection: close and its connection closed, since the client never sends that body. With signature verification on, a request signed over its whole payload, with a hex x-amz-content-sha256 rather than UNSIGNED-PAYLOAD or a STREAMING- value, is still read into memory to check that hash, so it is limited to about 2 GiB. So is a body sent with Content-Encoding: aws-chunked but neither a STREAMING- x-amz-content-sha256 nor an x-amz-trailer, which is read whole to be decoded and fails with EntityTooLarge past that. While it runs, CompleteMultipartUpload needs free disk space for twice the object, its parts and the assembled copy, until the parts are removed. Multipart uploads in progress do not survive a restart or a reset, and their files are removed when either happens. UploadPartCopy reads only the copied range from its source and, in the disk-backed modes, writes it straight into the part, so it can copy a range of up to 5 GiB, S3's limit for one part, from a source of any size. A larger range is rejected with EntityTooLarge. In the disk-backed modes CopyObject stores the source's file for the copy without reading it into memory, and reads it once only when the copy needs an ETag or checksum the source's cannot supply. As on S3, a CopyObject source larger than 5 GiB is rejected with InvalidRequest; copy it in parts with UploadPartCopy instead. S3 Select's Java evaluator reads a CSV or JSON object a record at a time, so it takes a source of any size; see S3 Select.

Examples

export AWS_ENDPOINT_URL=http://localhost:4566

# Create bucket
aws s3 mb s3://my-bucket --endpoint-url $AWS_ENDPOINT_URL

# Upload a file
aws s3 cp ./report.pdf s3://my-bucket/reports/report.pdf --endpoint-url $AWS_ENDPOINT_URL

# Upload inline content
echo '{"hello":"world"}' | aws s3 cp - s3://my-bucket/data.json --endpoint-url $AWS_ENDPOINT_URL

# Download
aws s3 cp s3://my-bucket/data.json ./data.json --endpoint-url $AWS_ENDPOINT_URL

# Inspect object attributes without downloading the body
aws s3api get-object-attributes \
  --bucket my-bucket \
  --key data.json \
  --object-attributes ETag ObjectSize StorageClass \
  --endpoint-url $AWS_ENDPOINT_URL

# List
aws s3 ls s3://my-bucket --endpoint-url $AWS_ENDPOINT_URL

# Delete
aws s3 rm s3://my-bucket/data.json --endpoint-url $AWS_ENDPOINT_URL

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket my-bucket \
  --versioning-configuration Status=Enabled \
  --endpoint-url $AWS_ENDPOINT_URL

# Generate a pre-signed URL (valid for 1 hour)
aws s3 presign s3://my-bucket/report.pdf \
  --expires-in 3600 \
  --endpoint-url $AWS_ENDPOINT_URL

Addressing Styles

Floci supports both path-style and virtual-hosted style S3 addressing.

Path-Style (always works)

Path-style embeds the bucket name in the URL path:

http://localhost:4566/my-bucket/my-key

Enable it in the SDK with forcePathStyle / pathStyleAccessEnabled:

S3Client s3 = S3Client.builder()
    .endpointOverride(URI.create("http://localhost:4566"))
    .forcePathStyle(true)
    .build();
const s3 = new S3Client({
  endpoint: "http://localhost:4566",
  forcePathStyle: true,
});
s3 = boto3.client(
    "s3",
    endpoint_url="http://localhost:4566",
    config=Config(s3={"addressing_style": "path"}),
)

Virtual-Hosted Style

Virtual-hosted style puts the bucket name in the hostname:

http://my-bucket.s3.localhost.floci.io:4566/my-key

An exact configured FLOCI_HOSTNAME or floci.dns.extra-suffixes root is an endpoint without a bucket. For example, with FLOCI_HOSTNAME=edge.localhost.floci.io, ListBuckets at that hostname lists the account's buckets, while my-bucket.edge.localhost.floci.io addresses my-bucket. Exact endpoint roots take precedence over shorter DNS suffixes and existing bucket-name matches, including controller fallback for object PUT and multipart requests.

Floci supports this natively, no forcePathStyle needed. The following wildcard DNS domains resolve to 127.0.0.1 via public DNS, so virtual-hosted requests reach Floci on the host machine automatically:

Domain pattern Resolves to
*.localhost.floci.io 127.0.0.1
*.s3.localhost.floci.io 127.0.0.1
*.localhost.localstack.cloud 127.0.0.1
*.s3.localhost.localstack.cloud 127.0.0.1

Plain http://localhost:4566 also works without forcePathStyle, the SDK sends Host: my-bucket.localhost:4566 and Floci's filter extracts the bucket name from that header.

Configure the SDK endpoint to one of the base domains (no forcePathStyle):

S3Client s3 = S3Client.builder()
    .endpointOverride(URI.create("http://s3.localhost.floci.io:4566"))
    .region(Region.US_EAST_1)
    .credentialsProvider(StaticCredentialsProvider.create(
        AwsBasicCredentials.create("test", "test")))
    .build();
// SDK sends requests to: my-bucket.s3.localhost.floci.io:4566
const s3 = new S3Client({
  endpoint: "http://s3.localhost.floci.io:4566",
  // no forcePathStyle, SDK uses virtual-hosted style by default
});
// SDK sends requests to: my-bucket.s3.localhost.floci.io:4566
s3 = boto3.client("s3", endpoint_url="http://s3.localhost.floci.io:4566")
# SDK sends requests to: my-bucket.s3.localhost.floci.io:4566

Virtual-Hosted Style Inside Docker

Inside Docker containers, 127.0.0.1 resolves to the container itself, not Floci. Floci's embedded DNS server handles this automatically: it resolves *.localhost.floci.io, *.s3.localhost.floci.io, *.localhost.localstack.cloud, and *.s3.localhost.localstack.cloud to Floci's container IP on the Docker network.

To use virtual-hosted style from a test container, point its DNS at Floci and set the endpoint to Floci's service hostname:

FLOCI_IP=$(docker inspect -f '{{.NetworkSettings.Networks.floci_default.IPAddress}}' floci)

docker run --rm \
  --network floci_default \
  --dns "$FLOCI_IP" \
  -e FLOCI_ENDPOINT=http://floci:4566 \
  -e FLOCI_S3_VHOST_ENDPOINT=http://floci:4566 \
  my-test-image

With FLOCI_HOSTNAME=floci set on the Floci container (default in the provided docker-compose.yml), the embedded DNS resolves my-bucket.floci to Floci's IP, and S3VirtualHostFilter extracts the bucket name from the Host header.

Object Attribute Notes

Floci now persists and returns the following object attribute state on S3 object APIs:

  • user metadata from x-amz-meta-*
  • storage class from x-amz-storage-class
  • Content-Encoding (without the aws-chunked token) and Cache-Control from PutObject and multipart initiation, replayed on GetObject and HeadObject
  • checksum metadata for object reads and GetObjectAttributes
  • multipart part manifests for GetObjectAttributes(ObjectParts)
  • CompleteMultipartUpload validates x-amz-mp-object-size against the total size of the selected parts. A mismatch returns InvalidRequest without replacing the object or discarding the upload, so the request can be retried with the correct size. Blank, malformed, overflowing, or negative sizes also return InvalidRequest and preserve the object and upload
  • CompleteMultipartUpload requires every selected part except the last to contain at least 5 MiB (5,242,880 bytes). The last selected part, including a single-part upload, has no minimum size. A smaller non-final part returns 400 EntityTooSmall with its part number, ETag and proposed size. The existing object and uploaded parts are preserved so completion can be retried after replacing the undersized part.
  • multipart checksums follow the AWS checksum type: COMPOSITE (<checksum of the part checksums>-<part count>) for SHA1, SHA256 and, by default, CRC32 and CRC32C; FULL_OBJECT for CRC64NVME or when x-amz-checksum-type: FULL_OBJECT was requested on CreateMultipartUpload. The -N suffix is returned by CompleteMultipartUpload, HeadObject and GetObject, and omitted by GetObjectAttributes, which lists part-level checksums only for COMPOSITE objects (a FULL_OBJECT multipart object reports its PartsCount alone, a single-part object has no ObjectParts), as on AWS. UploadPart echoes the part checksum (x-amz-checksum-<algorithm>) when the upload declared an algorithm, and completing a COMPOSITE upload requires the checksum of every part in the CompleteMultipartUpload body (InvalidRequest otherwise), while FULL_OBJECT uploads accept a body without them
  • canned object ACLs from x-amz-acl on PutObject, CopyObject, and multipart initiation
  • explicit ACL grant headers (x-amz-grant-read, x-amz-grant-write, x-amz-grant-full-control, x-amz-grant-read-acp, x-amz-grant-write-acp) with id and uri grantees on PutObject, CopyObject, and multipart initiation, where they apply when the upload completes
  • Object Lock headers (x-amz-object-lock-mode, x-amz-object-lock-retain-until-date, x-amz-object-lock-legal-hold) on PutObject, CopyObject, and multipart initiation, where they apply when the upload completes. A mode without a date or a date without a mode, an unknown mode, or a date that is not an ISO 8601 timestamp is refused with 400 InvalidArgument. In a bucket without Object Lock, a mode, a date or a legal hold of ON is refused with 400 InvalidRequest, "Bucket is missing Object Lock Configuration", as PutObjectRetention, GetObjectRetention, PutObjectLegalHold and GetObjectLegalHold are there. GetObjectRetention and GetObjectLegalHold of an object without a retention or a legal hold answer 404 NoSuchObjectLockConfiguration. A PutObjectRetention body with a mode but no date, a date but no mode, or an unknown mode is refused with 400 MalformedXML; one naming neither removes the retention
  • explicit object SSE headers from x-amz-server-side-encryption on PutObject, CopyObject, and multipart initiation, and form fields on browser/presigned POST, replayed on GetObject and HeadObject
  • SSE-KMS key IDs from x-amz-server-side-encryption-aws-kms-key-id on PutObject, browser/presigned POST, and multipart initiation, and from CopyObject/UploadPartCopy (inherited from the source object when the request doesn't specify new SSE headers), returned by PutObject, CopyObject, UploadPartCopy, CompleteMultipartUpload, GetObject, and HeadObject. A bucket's default SSE-KMS key is not simulated: aws:kms with no explicit key ID omits the header rather than returning a synthesized aws/s3 key ARN

Current limitations:

  • checksum algorithms: CRC32, CRC32C, CRC64NVME, SHA1 and SHA256 are supported; the other AWS algorithms (SHA512, MD5, XXHASH3, XXHASH64, XXHASH128) are rejected with InvalidRequest
  • copy-based metadata updates support x-amz-metadata-directive: REPLACE for user metadata and content type, but do not yet cover every AWS copy header
  • emailAddress grantees in the x-amz-grant-* headers are refused with 501 NotImplemented
  • cross-account canned ACL variants collapse to the emulator's single synthetic owner where Floci does not model a distinct second principal
  • aws-exec-read is accepted for compatibility, but Floci does not yet model a distinct EC2 bundle-reader grantee in GetObjectAcl
  • log-delivery-write grants the well-known S3 log-delivery group (http://acs.amazonaws.com/groups/s3/LogDelivery) WRITE and READ_ACP, matching AWS on the bucket path. Floci's shared canned-ACL helper also accepts it on PutObjectAcl, where AWS does not: the canned ACL is documented as bucket-only and the SDK's ObjectCannedACL enum omits it. That object path is a Floci leniency, not parity. It is the mirror image of aws-exec-read above, which botocore's BucketCannedACL enum omits, though AWS's canned-ACL reference documents it as valid for both bucket and object; there the narrower party is the SDK model, not AWS

Notification event compatibility

Native queue, topic and Lambda notification records include both response identifiers and the legacy bucket owner identity expected by existing AWS event consumers. Each destination receives its configured rule ID, or emulator if the ID is null, empty or blank. Copies of the same operation retain a common event time, response IDs, object details and sequencer, including version details captured before delivery. Response IDs are synthetic emulator identifiers and do not correlate with the initiating HTTP response headers.

The legacy ownerIdentity.principalId is the owning account's synthetic customer ID, separate from the canonical user ID used by ACL APIs. Notification object keys use form URL encoding: spaces become +, literal plus signs become %2B, and / is preserved. Consumers must ignore additional fields as described in the AWS event message structure.