S3
Protocol: REST XML
Endpoint: http://localhost:4566/{bucket}/{key}
Supported Operations
| Category | Operations |
|---|---|
| Buckets | ListBuckets, CreateBucket, HeadBucket, DeleteBucket, GetBucketLocation |
| Objects | PutObject, GetObject, GetObjectAttributes, HeadObject, DeleteObject, DeleteObjects, CopyObject |
| Listing | ListObjects, ListObjectsV2, ListObjectVersions |
| Multipart | CreateMultipartUpload, UploadPart, CompleteMultipartUpload, AbortMultipartUpload, ListMultipartUploads |
| Versioning | PutBucketVersioning, GetBucketVersioning |
| Tagging | PutBucketTagging, GetBucketTagging, PutObjectTagging, GetObjectTagging, DeleteObjectTagging |
| Annotations | PutObjectAnnotation, GetObjectAnnotation, ListObjectAnnotations, DeleteObjectAnnotation |
| Policy | PutBucketPolicy, GetBucketPolicy, GetBucketPolicyStatus, DeleteBucketPolicy |
| CORS | PutBucketCors, GetBucketCors, DeleteBucketCors |
| Lifecycle | PutBucketLifecycle, GetBucketLifecycle, DeleteBucketLifecycle |
| ACL | PutBucketAcl, GetBucketAcl, PutObjectAcl, GetObjectAcl |
| Encryption | PutBucketEncryption, GetBucketEncryption, DeleteBucketEncryption |
| Notifications | PutBucketNotification, GetBucketNotification |
| Object Lock | PutObjectLockConfiguration, GetObjectLockConfiguration, PutObjectRetention, GetObjectRetention, PutObjectLegalHold, GetObjectLegalHold |
| Website | PutBucketWebsite, GetBucketWebsite, DeleteBucketWebsite |
| Pre-signed URLs | Generates and validates pre-signed GET/PUT URLs |
| S3 Select | SelectObjectContent |
| Public Access Block | PutPublicAccessBlock, GetPublicAccessBlock, DeletePublicAccessBlock |
| Metrics | PutBucketMetricsConfiguration, GetBucketMetricsConfiguration, ListBucketMetricsConfigurations, DeleteBucketMetricsConfiguration |
| Intelligent-Tiering | PutBucketIntelligentTieringConfiguration, GetBucketIntelligentTieringConfiguration, ListBucketIntelligentTieringConfigurations, DeleteBucketIntelligentTieringConfiguration |
| Analytics | PutBucketAnalyticsConfiguration, GetBucketAnalyticsConfiguration, ListBucketAnalyticsConfigurations, DeleteBucketAnalyticsConfiguration |
| Inventory | PutBucketInventoryConfiguration, GetBucketInventoryConfiguration, ListBucketInventoryConfigurations, DeleteBucketInventoryConfiguration |
| Replication | PutBucketReplication, GetBucketReplication, DeleteBucketReplication |
RestoreObject tracks temporary restore state for GLACIER and DEEP_ARCHIVE objects per version. HeadObject reports x-amz-restore, and GetObject reports it after restoration completes; ListObjects, ListObjectsV2, and ListObjectVersions report RestoreStatus when requested with x-amz-optional-object-attributes: RestoreStatus. New restores return 202, while extending a completed restore returns 200. Restores complete immediately by default; FLOCI_SERVICES_S3_RESTORE_COMPLETION_DELAY_SECONDS sets a delay for polling workflows. The temporary copy expires at the next UTC midnight after its requested days elapse from completion. GetObject returns InvalidObjectState (HTTP 403) before restoration, while it is in progress, and after the temporary copy expires.
Replication is configuration-only: the replication configuration is stored on the bucket and round-trips through GetBucketReplication, but no objects are actually replicated.
ListBuckets includes each bucket's BucketRegion when bucket-region, continuation-token,
max-buckets, or prefix is present. It uses the stored bucket region, falling back to the
configured default region. Bare requests omit the field. Buckets are listed in name order and
paged by max-buckets (1 to 10000), with a ContinuationToken on every page but the last; a
max-buckets out of range or a token that is unreadable or was issued to another account is
refused with InvalidArgument. Without max-buckets, every bucket comes back on one page. Filtering by prefix and bucket-region is not implemented;
those parameters do not limit the returned bucket list.
Event Notifications
Browser and presigned POST uploads emit s3:ObjectCreated:Post, matching AWS S3. They do not emit
s3:ObjectCreated:Put; use s3:ObjectCreated:* to subscribe to objects created by either method.
Annotation changes emit s3:ObjectAnnotation:Put and s3:ObjectAnnotation:Delete.
PutBucketNotificationConfiguration checks SQS, SNS, and Lambda destination ARNs and verifies
that each destination exists before replacing a bucket's configuration. A failed check leaves
the prior configuration unchanged. Accepted configurations send an s3:TestEvent message to
SQS and SNS destinations; unlike object notifications, this message has no Records array.
x-amz-skip-destination-validation: true skips existence checks and test messages, but still
requires well-formed destination ARNs. Destination resource-policy permissions are not checked.
Existing SNS topics in another account are checked for existence without receiving a test message,
regardless of IAM enforcement. Their object events are not delivered because SNS publish still
resolves topics in the caller's account. With IAM enforcement enabled, SQS queues in another account
are also checked without a test message. Object-event delivery is not yet subject to a shared service
authorizer. AWS's same-Region destination requirement is not enforced. Cross-account Lambda
destination routing is not emulated.
Object Annotations
Annotations are named UTF-8 text payloads (up to 1 MiB each, 1,000 per object version) attached to a
specific object version through the four ?annotation operations. Notes on the emulation:
- Annotation names allow letters (any language), digits,
_,., and-; names longer than 512 bytes, empty or whitespace-only names, names with other characters, and names starting withawsors3(case-insensitive) are rejected. - Payloads must be valid UTF-8 text between 1 byte and 1 MiB; anything else is rejected with 400,
and non-UTF-8 payloads return 415
UnsupportedMediaType. x-amz-object-if-matchis validated against the parent object's ETag on put and delete.- Versioning semantics match AWS: annotations attach to one object version, new versions do not inherit them, overwriting a non-versioned object or deleting it drops its annotations, a delete marker preserves the underlying version's annotations, and deleting a specific version deletes its annotations. Annotation deletion is permanent.
CopyObjectcopies annotations by default; thex-amz-object-annotation-directiveheader (as the AWS SDK sends it;x-amz-annotation-directiveis also accepted) set toEXCLUDEskips them.- Checksums are per-annotation and independent of the object checksum. The default algorithm is CRC64NVME. Supported: CRC32, CRC32C, CRC64NVME, SHA1, SHA256. SHA512, XXHASH64, XXHASH3, XXHASH128, and MD5 are rejected as unsupported.
- Annotations on SSE-C encrypted objects are rejected, as on AWS, and are not copied onto SSE-C copy destinations.
- Annotation operations serialize against object writes on the same bucket. On Object
Lock-protected versions, annotation put and delete follow the same rules as object delete:
governance retention requires
x-amz-bypass-governance-retention(put never takes the bypass), compliance and legal hold always block. - The literal
versionId=nullnames the null version only, as it does on every object operation (see Object Versions); a key without one answersNoSuchVersion, as for any missing version. - Annotations are stored per AWS account. With
globalBucketNamespaceenabled, object reads resolve cross-account but annotation reads, writes, and listings stay in the caller's account. - S3 Metadata annotation tables and annotation replication are not implemented.
Object Versions
A request that names versionId=null, the id ListObjectVersions reports for an object written
while versioning was off or suspended, acts on that null version only: GetObject, HeadObject,
GetObjectAttributes, the object ACL, tagging, retention, legal hold, annotations and a copy source.
A key with no null version answers 404 NoSuchVersion, as for any missing version. If
the key already has a null version, a later versioned write or delete marker keeps it as a
noncurrent version. Deleting newer versions can make that null version current again.
Floci mints every version id as a UUID. On GetObject, HeadObject, DeleteObject, GetObjectAttributes,
RestoreObject, the object ACL, tagging, retention, legal hold and annotations, a versionId that is
neither a UUID nor null answers 400 InvalidArgument, "Invalid version id specified", as S3
answers an id it did not mint. An empty versionId answers "Version id cannot be
the empty string". A UUID that Floci did not mint is handled as the id of a version that does not
exist, where S3 answers 400.
GetObject and HeadObject of a key whose current version is a delete marker answer 404 with
x-amz-delete-marker: true and the marker's id in x-amz-version-id, as on S3. A versionId that
names a delete marker answers 405 MethodNotAllowed with the same headers, the marker's
Last-Modified and Allow: DELETE.
DeleteObject without a version id in a versioning-suspended bucket makes a null delete marker
the current version, as on S3. It replaces the null version or an earlier null delete marker, and
the other versions stay as noncurrent versions. Deleting versionId=null removes that marker and
makes the newest remaining version current again. A write in such a bucket, by PutObject, CopyObject
or CompleteMultipartUpload, makes its null version current in the same way. The other versions and
delete markers stay as noncurrent versions, and deleting versionId=null makes the newest of them
current again.
Object Lock works only with versioning enabled. PutBucketVersioning with Suspended on a bucket
with Object Lock answers 409 InvalidBucketState, "An Object Lock configuration is present on this
bucket, so the versioning state cannot be changed.", and PutObjectLockConfiguration on a bucket
whose versioning is unset or suspended answers 409 InvalidBucketState, "Versioning must be
'Enabled' on the bucket to apply a Object Lock configuration". The same refusal fails a
CloudFormation update that suspends versioning on such a bucket. A bucket an earlier Floci release
stored in either state loads in that state, and its null versions keep their Object Lock protection.
Lifecycle Expiration
A bucket's lifecycle configuration is stored as sent and its expiration actions are applied by a
background sweep, every FLOCI_SERVICES_S3_LIFECYCLE_SWEEP_INTERVAL_SECONDS (60 by default). S3
also applies lifecycle asynchronously, so a lower interval only makes Floci act sooner after an
object becomes due. Each enabled rule's filter (Prefix, Tag, ObjectSizeGreaterThan,
ObjectSizeLessThan, an And of them, or the older rule-level Prefix) selects what it applies to,
and disabled rules are ignored.
ExpirationwithDaysorDateexpires the current version. A day count runs from the object's last-modified time and, as on S3, is rounded up to the next midnight UTC, so an object written at 10:30 UTC under a 3-day rule expires at midnight UTC three days and some hours later. An unversioned bucket deletes the object; a versioning-enabled bucket adds a delete marker, and a versioning-suspended bucket anulldelete marker.ExpiredObjectDeleteMarkerremoves a delete marker that is the only version left of its key. ADaysorDateexpiration also removes such a marker once the marker itself is old enough.NoncurrentVersionExpirationpermanently deletes a noncurrent version, delete markers included, once it has been noncurrent forNoncurrentDays(counted from when the next newer version was written, rounded the same way) and at leastNewerNoncurrentVersionsnewer noncurrent versions are kept ahead of it.AbortIncompleteMultipartUploadaborts an uploadDaysAfterInitiationafter it was started, rounded the same way. Only a rule whose filter is a plain prefix selects uploads.- A version under Object Lock retention or a legal hold is never deleted by lifecycle.
- Expirations are notified as
s3:LifecycleExpiration:Deleteands3:LifecycleExpiration:DeleteMarkerCreated, not ass3:ObjectRemoved:*, withs3.amazonaws.comas the principal and source IP address. Their EventBridge events carryreasonLifecycle Expiration, requesters3.amazonaws.comand nosource-ip-address. - A rule that S3 would have refused is skipped as a whole, so a bad value never widens what it
deletes:
DaysorNoncurrentDaysthat is not a positive integer,NewerNoncurrentVersionsoutside 1 to 100 or without aFilter, a negativeDaysAfterInitiation, a size bound that is not a non-negative number, aTagwithout bothKeyandValueor with a key used twice, neither aFilternor a rule-levelPrefix,ExpiredObjectDeleteMarkernext toDaysorDate, aDatethat does not parse, or aStatusother thanEnabledorDisabled. An emptyFilterapplies to the whole bucket. A body that is not one well-formedLifecycleConfigurationdocument has no rules.
PutObject, GetObject and HeadObject on the current version of an object that an Expiration
rule matches return x-amz-expiration: expiry-date="Fri, 23 Dec 2012 00:00:00 GMT", rule-id="...",
with the earliest matching expiry and that rule's URL-encoded id. Requests for a noncurrent version
do not carry it.
Differences from S3:
TransitionandNoncurrentVersionTransitionare stored but not applied: storage classes do not change.CopyObjectandCompleteMultipartUploaddo not returnx-amz-expiration, andCreateMultipartUploaddoes not returnx-amz-abort-dateorx-amz-abort-rule-id.- The configuration is not validated when it is put: a rule S3 would have refused is stored and returned as sent, and only skipped when rules are applied.
- A delete marker that lifecycle adds carries the time of the sweep, not midnight UTC.
Website Hosting
Static website requests use a bucket website hostname such as
http://my-bucket.s3-website-us-east-1.localhost:4566/. Floci supports:
- GET and HEAD requests through website hostnames
- index-document resolution for the site root and slash-terminated prefixes
302redirects that append a slash when an index document exists below a prefix- configured error documents and the default website error response
HEAD returns the same status and object metadata as GET without a response body. Redirect-all and advanced routing-rule configurations are not implemented.
Range Reads
GetObject supports standard Range: bytes= requests. Partial responses return 206, Content-Range, Content-Length, Accept-Ranges: bytes, and object metadata. Whole-object checksum headers are omitted on ranged responses because the stored checksum covers the full object, not the returned byte slice, except for a range covering the whole of an object stored in one request, or completed from one part with a full-object checksum: that object is its one part, and like a range aligned with a single part, the range gets its checksum. A composite checksum is not that of any one part, nor is a checksum of an object of several parts, so their ranges get none. A range that starts past the end of the object, or a suffix of zero bytes (bytes=-0), returns 416 InvalidRange, while a Range that is not one valid range, such as several ranges, a last byte before the first, or a malformed value, is ignored and the whole object is returned with 200, as S3 does. Full-object and ranged reads stream object bytes from storage instead of materializing the response body first.
Suffix ranges against empty objects return an empty 200 response, matching AWS behavior used by clients that issue tail reads.
HeadObject honors Range as the API Reference describes: a satisfiable range changes Content-Length to the length of the range, and one that cannot be satisfied returns 416, while a Range that is not one valid range, such as several ranges or a last byte before the first, is ignored. Like a ranged GetObject, it leaves out the whole-object checksum headers unless the range covers the whole of an object that is one part.
GetObject and HeadObject also read one part of an object with the partNumber query parameter, which works like a range covering that part: 206, its Content-Range and Content-Length, the object's ETag, and, for an object completed from a multipart upload, x-amz-mp-parts-count with the number of its parts. An object stored in one request is its own part 1, and part 1 of an empty object returns an empty 200 with no Content-Range, which no byte range describes. A part streams from storage like a range, so the parts of a large object can be downloaded one at a time. With checksum mode on, a part of an object with a COMPOSITE checksum gets the part's own checksum, of type COMPOSITE, and a part that is all of its object gets the object's checksum; any other part of a FULL_OBJECT multipart object gets none. A part the object does not have, including any part number above 10,000, returns 416 InvalidPartNumber naming PartNumberRequested and ActualPartCount, partNumber together with a Range header returns 400 InvalidRequest, and a part number below 1 returns 400 InvalidArgument.
An If-Range header makes the range conditional, so a resumed download never mixes two versions of an object. The range is served while the validator still describes the object: a quoted ETag equal to the object's (a multipart ETag compared whole, -N suffix included), or an HTTP-date equal to its Last-Modified, an exact match as RFC 9110 section 13.1.5 requires, so a later date does not qualify. Otherwise, including a weak (W/) or malformed validator, GetObject returns the whole object with 200. A presigned URL that signs Range must also sign an If-Range the request carries, or the request fails with 403 AccessDenied naming it in HeadersNotSigned.
S3 Select
SelectObjectContent runs SQL queries directly against S3 objects without downloading the entire file. Floci supports CSV, JSON Lines, JSON arrays, and Parquet inputs. The SQL dialect follows AWS S3 Select SQL reference.
Execution modes
Floci chooses the execution engine automatically based on the input format and whether the floci-duck sidecar is running:
| Condition | Engine | Notes |
|---|---|---|
| Input is Parquet | floci-duck (required) | DuckDB's read_parquet: sidecar must be available |
Input is CSV with FileHeaderInfo=USE and floci-duck is running |
floci-duck | Full DuckDB SQL: all operators, LIKE, BETWEEN, IN, IS NULL, AND/OR/NOT |
| Input is JSON and floci-duck is running | floci-duck | read_json_auto: supports JSON Lines and JSON arrays |
Input is CSV with FileHeaderInfo=NONE or IGNORE, or floci-duck is not running |
Java evaluator | Supports SELECT *, column projection, simple WHERE with =, !=, <, >, <=, >=, LIKE, BETWEEN, IN, IS NULL, AND/OR/NOT, LIMIT |
The floci-duck sidecar starts lazily on the first Athena query. Until then, isAvailable() returns false and S3 Select falls back to the Java evaluator for CSV and JSON. Once the sidecar is running, subsequent S3 Select calls route through DuckDB automatically.
If floci-duck is not running and the object is Parquet, S3 Select returns an error: Parquet decoding requires DuckDB.
The Java evaluator reads the object a record at a time and writes the result in Records events as
it goes, stopping once a LIMIT is reached, so neither the object nor the result is held in memory.
As on S3, a record can be split across Records events, and a record in the input or the result
can be at most 1 MB (1,048,576 characters): a longer one fails the query with OverMaxRecordSize,
which arrives as an error message at the end of the event stream once the response has begun.
BytesScanned and BytesProcessed in the Stats event count the bytes the query read, so one a
LIMIT stops early reports less than the object's size. A query floci-duck runs is read by DuckDB,
and its result is held in memory before it is returned.
FileHeaderInfo modes (CSV)
| Value | Behavior |
|---|---|
USE |
First row is the header; column names are available in WHERE and SELECT |
IGNORE |
First row is skipped and not included in output; only positional _N references work |
NONE |
All rows are data; only positional _N references work (e.g. WHERE _1 = 'Alice') |
Supported SQL operators
When using the Java evaluator (no floci-duck, or CSV with FileHeaderInfo=NONE/IGNORE):
- Comparison:
=,!=,<>,<,>,<=,>= - Pattern matching:
LIKE(supports%and_wildcards) - Range:
BETWEEN ... AND ... - Set membership:
IN (...) - Null checks:
IS NULL,IS NOT NULL - Logical:
AND,OR,NOT - Clauses:
SELECT *, column projection,LIMIT
Output formats
S3 Select supports cross-format output: a CSV object can produce JSON output and vice versa.
| Input | Output | Format |
|---|---|---|
| CSV | CSV | Default: comma-separated values |
| CSV | JSON | One JSON object per row: {"col1":"val1","col2":"val2"} |
| JSON | JSON | Default: one JSON object per line |
| JSON | CSV | Comma-separated values, values quoted when they contain commas or newlines |
Example
export AWS_ENDPOINT_URL=http://localhost:4566
# Upload a CSV file
printf 'name,age,city\nAlice,30,New York\nBob,25,\nCharlie,35,London\n' \
| aws s3 cp - s3://my-bucket/people.csv
# Query with WHERE and column projection
aws s3api select-object-content \
--bucket my-bucket \
--key people.csv \
--expression "SELECT name, city FROM S3Object WHERE age >= 30" \
--expression-type SQL \
--input-serialization '{"CSV":{"FileHeaderInfo":"USE"}}' \
--output-serialization '{"CSV":{}}' \
/dev/stdout
# IS NULL check
aws s3api select-object-content \
--bucket my-bucket \
--key people.csv \
--expression "SELECT name FROM S3Object WHERE city IS NULL" \
--expression-type SQL \
--input-serialization '{"CSV":{"FileHeaderInfo":"USE"}}' \
--output-serialization '{"CSV":{}}' \
/dev/stdout
# JSON Lines input
printf '{"name":"Alice","score":95}\n{"name":"Bob","score":72}\n' \
| aws s3 cp - s3://my-bucket/scores.json
aws s3api select-object-content \
--bucket my-bucket \
--key scores.json \
--expression "SELECT * FROM S3Object WHERE score > 80" \
--expression-type SQL \
--input-serialization '{"JSON":{"Type":"LINES"}}' \
--output-serialization '{"JSON":{}}' \
/dev/stdout
Mock mode note
When FLOCI_SERVICES_ATHENA_MOCK=true is set, Athena queries are stubbed but floci-duck does not start. In that configuration, S3 Select uses the Java evaluator for CSV and JSON. Parquet queries will fail unless FLOCI_SERVICES_DUCK_URL points to an already-running floci-duck instance.
Bucket Names
Floci does not hold bucket names to AWS's full DNS naming rules, so a name AWS would refuse is
usually accepted here. The exception is a name that would not stay a single directory under its
account's storage root: an empty name, ., .., a name containing a path separator, or one of
Floci's reserved storage directories (.accounts, .versions, .annotations) is rejected with
InvalidBucketName. On the persistent and hybrid backends such a name either climbs out of the
owning account's directory or collides with object version and annotation data.
Object Keys
S3 accepts keys of up to 1024 bytes, and so does Floci in every storage mode. On the disk-backed
modes the body file is laid out as <key>.s3data under the bucket's directory, and a single key
segment can run past the 255-byte file-name limit the filesystem enforces. Such a segment is
stored on disk as a readable prefix followed by the SHA-256 of the whole segment. A segment that
fits and can be represented as written keeps its raw path, unless it already looks like an
encoded name. Those names are encoded too, so they cannot overwrite an encoded object's body.
The mapping is deterministic, so the same key always resolves to the same file, and
listing answers with the original key, which the object index holds, never with the encoded
on-disk name.
When Floci runs on Windows, a segment Windows cannot hold as a file name is stored the same way:
one containing a character Windows refuses (<>:"|?*, \ or a control character), one ending in
a dot or a space, which Windows drops, and one naming a device that Microsoft's file naming rules
reserve, such as CON, NUL.txt or COM1. For an unversioned body, the last segment is judged
with .s3data appended, the name it is stored under, so a key such as logs/report. keeps its
earlier path. Versioned bodies use the key as a directory path, so its last segment is judged
without that suffix.
Older Windows releases of Floci could fold keys such as dir./file and dir\file onto another
key's path. Bodies at those ambiguous paths are not migrated automatically, because their intended
key cannot be determined safely. Existing raw filenames that now match the encoded-name pattern
are also not migrated automatically. Their metadata can remain while a read of the new encoded
path fails; these objects need to be rewritten from a known copy of their contents.
A key keeps the slashes it is sent with: PUT /bucket//file.txt writes /file.txt, and a key made
only of slashes is an object too, so PUT /bucket// writes the key / rather than creating the
bucket. One trailing slash after the bucket, /bucket/, is still the bucket. The key is read from
the path as it arrived wherever it is needed, so the object stored, the object an
s3:ExistingObjectTag condition describes and the object ARN the request is authorized against are
the same one (see IAM).
Bucket Regions
CreateBucket follows AWS's region rules, in every partition. The endpoint a request is signed for
decides what it accepts:
- The
us-east-1endpoint takes aLocationConstraintnaming any other region, and no constraint at all (the bucket then lives inus-east-1). A constraint ofus-east-1itself isInvalidLocationConstraint. - Every other regional endpoint requires a constraint naming exactly its own region. No constraint,
or a different region (including
us-east-1), isIllegalLocationConstraintException.
The AWS SDKs send the constraint for any region but us-east-1, so SDK clients are unaffected. A
raw-HTTP client that signs for another region and sends an empty body has to add the
CreateBucketConfiguration; the signing region alone does not place a bucket. GetBucketLocation
answers an empty constraint only for us-east-1. See AWS Partitions
for how this applies in China and GovCloud.
Global Bucket Namespace
Object deletion, including DeleteObjects batches, uses the named bucket's owning account for
stored objects, versions, annotations and notifications. Existing authorization checks still use
the original caller, and DeleteBucket remains scoped to that caller's account.
By default S3 buckets are isolated per account (see Multi-Account Isolation),
so two accounts can each hold a bucket with the same name. Set
FLOCI_SERVICES_S3_GLOBAL_BUCKET_NAMESPACE=true to make bucket and object resolution span every
account's partition: a bucket created in one account then resolves cross-account, matching real S3
where bucket names are globally unique. Enable this when a workload (for example an LZA log-archive or
CDK asset bucket) is created in one account and read from another.
Creating a bucket whose name another account owns returns 409 BucketAlreadyExists. Concurrent
creates and deletion share the same bucket-name lock. Recreating an owned bucket succeeds in
us-east-1 and returns 409 BucketAlreadyOwnedByYou in other regions. ListObjects and
ListObjectsV2 read the named bucket's owner's objects; ListBuckets stays caller-scoped.
With IAM enforcement enabled, a foreign bucket's policy neither grants nor denies creation of a
bucket with the same name. Caller permission checks, including requested creation settings, still
run before the collision check. A bucket owned by the caller retains its existing policy checks.
The default per-account namespace and authentication settings are unchanged. Bucket-name reuse
after deletion is immediate in Floci; AWS may temporarily return 409 OperationAborted.
Floci-managed staging buckets for Lambda packages, Spectrum, Firehose conversion and CUR remain
account scoped even when the global namespace is enabled. Their existing names and object paths are
preserved; the internal creation path does not change public CreateBucket collision handling.
Why the flag exists, and what it trades away. Floci partitions every resource by the caller's account, which is the right model for services whose names are account-scoped in AWS. S3 bucket names are not: they are unique across all accounts, a bucket lives in exactly one owning account, and whether another account may touch it is decided by policy rather than by the name being unreachable. A hard per-account partition therefore lets two accounts hold different buckets under one name (a state AWS cannot reach), and makes a legitimate cross-account call fail with NoSuchBucket instead of being authorized or denied on its merits. With the flag on, a caller that names another account's bucket resolves it and only IAM and bucket-policy evaluation stand between the caller and the object: AWS-faithful, but stricter than the implicit isolation Floci gives you elsewhere, provided both FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED (identity-policy checks, applied before the request reaches S3) and FLOCI_SERVICES_S3_ENFORCE_AUTH (the S3-specific flag that gates bucket-policy evaluation itself, in S3Service.authorizeS3Read/authorizeS3Write) are also true. Both default to false, and bucket-policy evaluation in particular is a no-op while FLOCI_SERVICES_S3_ENFORCE_AUTH is off, regardless of the IAM flag, so in the default configuration enabling the global namespace does not trade per-account isolation for a policy-gated boundary, it removes the only isolation Floci gives S3 for that bucket and replaces it with nothing. Mutations are written back to the bucket's owning account rather than forked into the caller's, and ListBuckets stays caller-scoped while ListObjects and ListObjectsV2 list the named bucket's owner-scoped objects, so the flag never reassigns ownership or leaks a bucket inventory. It is off by default, turn it on when emulating a multi-account estate whose real behaviour depends on the global namespace, and leave it off when the per-account partition is the isolation you are relying on. Account-scoped S3 state, notably the account-level Block Public Access configuration, is never widened by this flag. Note also that retrofitting a global namespace onto state that was previously partitioned per account means that if two accounts already hold a bucket under the same name (a state real AWS can't reach, but Floci's prior per-account isolation could), resolution picks whichever account's copy the backend happens to iterate to first; this is unlikely to be reachable on a fresh LZA-driven account structure, but is worth knowing if you enable the flag on an estate with pre-existing same-name buckets.
Signature Verification
AWS treats authentication and authorization as two steps. S3 first verifies the SigV4 signature of a signed request and refuses one that does not verify, whatever the bucket policy says. Only then does it decide, from IAM policies, the bucket policy and ACLs, whether the caller may perform the operation. An unsigned request is anonymous, and is allowed only when a policy or ACL grants access to everyone.
Floci verifies signatures when either of two flags is on:
FLOCI_AUTH_VALIDATE_SIGNATURES |
FLOCI_SERVICES_S3_ENFORCE_AUTH |
|
|---|---|---|
Authorization header signatures |
Verified | Verified |
Presigned URL signatures (X-Amz-Signature) |
Verified | Verified |
| Presigned POST signatures (browser form) | Verified when the form carries signature fields | Verified, and required |
| Bucket policy and ACL evaluation | Not evaluated | Evaluated |
| Unsigned (anonymous) requests | Allowed | Allowed only when a policy or ACL grants public access |
For a signed request, verification rebuilds the canonical request from the wire path, the query string, the headers named in SignedHeaders and the declared payload hash, derives the signing key from the access key's secret, and compares the signatures in constant time. It rejects:
- a signature that does not verify, or one computed for another key or method, with
403 SignatureDoesNotMatch - an access key that is neither
testnor registered in Floci IAM (CreateAccessKeykeys, and STS keys with their session token), with403 InvalidAccessKeyId. UnderFLOCI_AUTH_VALIDATE_SIGNATURESalone, a presigned URL from such a key falls back to the olderFLOCI_AUTH_PRESIGN_SECRETcheck instead, which an SDK-made URL fails with403 SignatureDoesNotMatch - temporary credentials past their expiry, sent with their own session token, with
400 ExpiredToken, whichever placement carries the signature. Once Floci has removed an expired session it no longer knows the key, and answers as for one that is not registered - a header-signed request more than 15 minutes from server time, with
403 RequestTimeTooSkewed - a body that does not match a declared
x-amz-content-sha256digest, with400 XAmzContentSHA256Mismatch - an
Authorizationheader whose algorithm is neitherAWS4-HMAC-SHA256norAWS4-ECDSA-P256-SHA256(SigV4A), with400 AuthorizationHeaderMalformed - a SigV4A request, which is verified the way AWS does it: Floci derives the ECDSA P-256 key pair from the access key's secret and checks the signature against the public half. The
Authorizationheader and a presigned URL (X-Amz-Algorithm=AWS4-ECDSA-P256-SHA256) are both verified. Its credential scope has no region, so the request must sign the region set: a header request without a signedx-amz-region-setis400 AuthorizationHeaderMalformed, and a presigned URL withoutX-Amz-Region-Setis400 AuthorizationQueryParametersError. A set naming one region serves that region, and a wildcard or a list serves the default region. Floci does not check the set against the bucket's region. A browser POST signed with SigV4A is not accepted - a presigned URL that carries any of
X-Amz-Algorithm,X-Amz-CredentialorX-Amz-Signaturebut lacks one ofX-Amz-Algorithm,X-Amz-Credential,X-Amz-Signature,X-Amz-Date,X-Amz-SignedHeadersorX-Amz-Expires, with400 AuthorizationQueryParametersError
FLOCI_AUTH_VALIDATE_SIGNATURES is authentication alone: use it when forged or stale credentials must fail but buckets should stay reachable without policies. It is not specific to S3: the same flag verifies the Authorization header signature of every other service (see Signature Validation). FLOCI_SERVICES_S3_ENFORCE_AUTH adds authorization, so it also decides anonymous access the way AWS does: a bucket that anonymous clients read needs a public-read bucket policy or ACL.
A workload must sign with credentials Floci can verify once either flag is on. A container Floci launches without a role, such as a Lambda function with no execution role, is given placeholder credentials that no flag accepts, so give it a role, or sign with test/test.
Presigned uploads ignore checksum values, algorithms, and types supplied only in the query
string, matching AWS S3. This includes the empty-body CRC32 (AAAAAA==) that the JavaScript SDK
adds when presigning before the file is available. These parameters still participate in signature
verification, but are not used to validate the body or stored as the object's checksum. Actual
checksum headers and streaming checksum trailers remain validated as usual.
Bucket Policy Enforcement
Floci evaluates S3 bucket policies for authenticated (signed) requests when policy enforcement is enabled. Two flags control this evaluation:
FLOCI_SERVICES_S3_ENFORCE_AUTH: gates bucket policy authorization directly withinS3Servicefor bucket read, object write, and bucket write operations. It also turns on signature verification.FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED: activates the global IAM request filter to evaluate identity-based policies and resource policies before requests reach the service layer.
Both flags default to false for backward compatibility.
Evaluated Policies and Operations
When FLOCI_SERVICES_S3_ENFORCE_AUTH or FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED is active:
- The bucket's attached policy document (configured via PutBucketPolicy) is retrieved.
- The caller's authenticated principal ARN is resolved from the signing credentials (for example, arn:aws:iam::<account>:user/<name>, arn:aws:iam::<account>:role/<name>, or arn:aws:iam::<account>:root).
- Policy statements are evaluated for the requested S3 action (e.g. s3:GetObject, s3:PutObject, s3:PutBucketPolicy) and resource ARN (arn:aws:s3:::bucket or arn:aws:s3:::bucket/key).
- An object request is checked as the actions IAM checks for it (see IAM): the version action when it names a versionId, and the actions it needs besides its own, such as s3:PutObjectTagging for an object created with tags. The policy is offered the tags and Object Lock settings the request sets (s3:RequestObjectTag/<key>, s3:RequestObjectTagKeys and the s3:object-lock-* keys) and the encryption it asks for (s3:x-amz-server-side-encryption and s3:x-amz-server-side-encryption-aws-kms-key-id, from its headers, or for a part or a completion from what its upload recorded; see IAM) besides the keys that say who is calling, but not the other S3 keys, such as s3:ExistingObjectTag or s3:versionid. An anonymous request's public-policy check is offered none. An anonymous object creation that a public bucket ACL's WRITE grant admits may also apply its ACL, tags and Object Lock settings, since an ACL grant names no actions, unless a public bucket policy decides the setting's action.
Supported Principal Types
Bucket policy statements can specify principals using Principal or NotPrincipal with scalar strings or arrays:
- Wildcard:
"*"or{"AWS": "*"}matches any caller. - IAM User:
{"AWS": "arn:aws:iam::<account>:user/<name>"}. - IAM Role:
{"AWS": "arn:aws:iam::<account>:role/<path><name>"}. A session of the role matches it through the role's own ARN, path included. - AWS Account:
{"AWS": "<12-digit-account-id>"}or{"AWS": "arn:aws:iam::<account>:root"}matches any principal belonging to the specified account. - Canonical user:
{"CanonicalUser": "<canonical-id>"}names an account by its S3 canonical user ID and matches any principal belonging to that account, as an account principal does. Floci derives the canonical ID deterministically from the account ID as its SHA-256 hex digest, a 64-character lowercase hex string, and reports it wherever S3 identifies an owner: theOwnerofListBuckets,ListPartsand both ACLs, theInitiatorofListParts, and everyCanonicalUsergrantee of an ACL Floci generates. The account ID itself still matches, for policies written when Floci reported the account as the canonical ID. - Service Principal:
{"Service": "<service>.amazonaws.com"}. - NotPrincipal: Inverts matching so the statement applies to any principal not matching the specified patterns. On a
Deny, an IAM caller is spared only when the list names its account as well as the caller, and never when it has a permissions boundary attached; see Resource-based policies. Each entry is written as a string, so aNotPrincipalwhose entries are no strings, an empty list among them, names nobody: it carves nobody out, so the statement covers nobody rather than everyone, on an anonymous request and a signed one alike. Block Public Access reads it the other way round, and deliberately: S3 "begins by assuming that the policy is public" and calls a policy non-public only when it grants "only to fixed values" of a principal or of one of the condition keys below, so aNotPrincipalleaves the policy public whether it names anybody or not.
Every check against a bucket policy, the anonymous paths and the Block Public Access status included, holds a statement's Effect to Allow or Deny as written, the IAM User Guide saying that "The Effect value is case sensitive", so a statement spelling either word another way states nothing and one bucket policy answers a signed request and an unsigned one the same way.
The same matching applies to every check S3 makes against a bucket policy, including the source of CopyObject and UploadPartCopy.
Evaluation Rules
- An explicit Deny always wins and rejects the request.
- An explicit Allow grants access.
- For bucket owners (principals within the account that owns the bucket), the default decision is Allow unless explicitly denied.
- For non-owner principals, an explicit Allow in the bucket policy is required; neutral evaluation results in Deny.
- Unsigned (anonymous) requests and ACL-based public access fall back to standard S3 ACL and anonymous authorization paths.
- Unknown access key IDs return HTTP 403 with
InvalidAccessKeyId.
With FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED also on, the bucket policy and the caller's identity policies are combined the way AWS combines them. For same-account access, an explicit Allow in either policy grants the request, so a principal that only the bucket policy names, by its user, role or session ARN or as "*", needs no identity policy at all. A statement that names only the account, by its id, its root ARN or its canonical user ID, delegates to the account, as on AWS, so within that account the caller's identity policy has to allow the request. For cross-account access, both the bucket policy and the caller's identity policy must allow the action. An explicit Deny in either policy denies the request in both cases.
Wire Response Shape
When a request is denied by bucket policy enforcement, Floci returns HTTP 403 with an S3 XML error body including the <Resource> element:
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<Resource>/my-bucket/my-key</Resource>
<RequestId>...</RequestId>
</Error>
Object Ownership and ACLs
New buckets start with BucketOwnerEnforced ownership, matching the current S3 default. GetBucketOwnershipControls returns that setting immediately after CreateBucket. Bucket and object ACL writes are disabled, while uploads without an ACL and uploads using bucket-owner-full-control remain supported. To use ACL grants, set x-amz-object-ownership to ObjectWriter or BucketOwnerPreferred during CreateBucket, or change the mode later with PutBucketOwnershipControls. Existing stored buckets keep their previously configured ownership mode.
Canned ACLs and grant headers
A request sets its ACL either with a canned ACL (x-amz-acl) or with grant headers (x-amz-grant-*), not both. As on AWS, CreateBucket, PutBucketAcl, PutObjectAcl, PutObject, CopyObject and CreateMultipartUpload refuse a request carrying both with InvalidRequest (400), Specifying both Canned ACLs and Header Grants is not allowed, and change nothing. A header with a blank value counts as absent.
Block Public Access
Floci enforces all four Block Public Access settings. A bucket's configuration is combined with the bucket owner account's and the most restrictive of the two wins, which for four independent booleans is a per-flag OR: a flag set at either level is in force.
With BucketOwnerEnforced object ownership, ACL writes are rejected and existing object ACL grants no longer provide access. Before enabling it, the bucket ACL must grant access only to the owner; otherwise PutBucketOwnershipControls returns InvalidBucketAclWithObjectOwnership (400).
| Setting | What it does | Needs enforce-auth |
|---|---|---|
BlockPublicPolicy |
PutBucketPolicy returns AccessDenied (403) when the submitted policy is public |
No |
BlockPublicAcls |
PutBucketAcl, PutObjectAcl and a PutObject carrying a public ACL return AccessDenied (403) |
No |
RestrictPublicBuckets |
A bucket whose policy is public serves only the owner account: anonymous and cross-account callers are denied | Yes |
IgnorePublicAcls |
A public ACL on the bucket or on an object stops granting access | Yes |
The split follows AWS. The first two reject the write that would introduce public access, and AWS applies them whoever the caller is, so they need no flag: put-public-access-block followed by a public put-bucket-policy returns 403 in Floci's default configuration. The second two suppress access that an existing policy or ACL would grant, which only has meaning once anonymous authorization runs at all, so they take effect when FLOCI_SERVICES_S3_ENFORCE_AUTH is on. With that flag off, Floci cannot tell an anonymous caller from a signed one and every request is authorized regardless.
A bucket nobody configured enforces the default GetPublicAccessBlock reports for it: all four settings on. DeletePublicAccessBlock or PutPublicAccessBlock replaces that default.
When upgrading, a workflow that already sets BlockPublicPolicy or BlockPublicAcls may now receive 403 on a later public policy or ACL write, even with enforce-auth off. A bucket saved by an earlier version, including one with no configuration, now refuses public policies and ACLs until its configuration is deleted or set. This matches AWS behavior.
As in AWS, enabling a setting never rewrites stored state. An existing public policy or ACL stays exactly as written, and clearing the setting makes the bucket public again.
The meaning of "public"
An ACL is public when it grants any permission to the AllUsers or AuthenticatedUsers predefined groups. This is broader than the grants anonymous authorization consults: a WRITE_ACP grant to AllUsers is public here even though it gives an anonymous caller no read.
A bucket policy is assumed public and then checked for a reason it is not. A statement with a wildcard principal ("Principal": "*", {"AWS": "*"}, or an Allow on NotPrincipal) is public unless a positive condition pins one of aws:PrincipalArn, aws:PrincipalAccount, aws:PrincipalOrgID, aws:PrincipalOrgPaths, aws:SourceArn, aws:SourceVpc, aws:SourceVpce, aws:SourceOwner, aws:SourceAccount, aws:userid, s3:DataAccessPointArn or s3:DataAccessPointAccount to a fixed value, one containing neither a wildcard nor an IAM policy variable. In a bucket policy, s3:DataAccessPointArn may contain a wildcard in the access point name if the account ID is fixed, for example arn:aws:s3:us-west-2:123456789012:accesspoint/*. aws:SourceIp counts too, but only for a range no broader than /8 (IPv4) or /32 (IPv6), matching AWS's treatment of very wide CIDR blocks as public.
For multivalued condition keys such as aws:PrincipalOrgPaths, ForAnyValue: with a positive operator can narrow the grant when each value is fixed. ForAllValues: alone does not narrow it because the condition also matches when the key is missing.
A single public statement makes the whole policy public. As on AWS, RestrictPublicBuckets then withholds even the non-public cross-account delegation another statement grants, until the public statement is removed.
Known gaps
- Floci does not model access points, so the access-point variants (
PutAccessPointPolicy, the VPC-origin rule, the differents3:DataAccessPointArntreatment) do not apply. - AWS's organization-level Block Public Access policies are not modelled; only the bucket and account levels combine.
CreateBucketapplies compatible ACL headers but rejects public ACLs with HTTP 400, including onObjectWriterbuckets, because new buckets start with Block Public Access enabled. After creation, anObjectWriterbucket without an explicitPutPublicAccessBlockalso refuses a publicPutBucketAclwithAccessDenied(403), because the default configuration is enforced.GetBucketAclreturns the stored ACL rather than the effective one, so it does not reflect anIgnorePublicAclsthat is suppressing a grant.RestrictPublicBucketsexempts AWS service principals on AWS. Floci reaches the signed path only for IAM principals, so there is nothing to exempt.
Account-level operations
The s3control PutPublicAccessBlock, GetPublicAccessBlock and DeletePublicAccessBlock operations are keyed by the x-amz-account-id header, stored per account, and persisted like the rest of S3's state so a restart does not silently drop the control. As in AWS, the header must name the caller's own account; a mismatch returns AccessDenied (403). Floci keeps one deliberate exception: the configured default (management) account may act on any account, because Floci's launched Lambdas run on placeholder credentials that resolve to the management account instead of assuming a role in the target account, which is how LZA's Custom::PutPublicAccessBlock resource reaches this API.
IAM enforcement of S3 Control
With IAM enforcement on, every /v20180820 route is authorized as the s3: action the Service Authorization Reference gives it, against the resource the route names, because S3 Control clients sign with the s3 scope. The s3control model's routes and the actions are vendored from AWS (rest-routes.json and iam-authorized-actions.json, regenerated by make aws-data-sync), and IamRestRouteCoverageTest checks every route Floci serves against them. The account-level Block Public Access is the one exception, written down where that data is checked: the reference files S3's and S3 Control's GetPublicAccessBlock, PutPublicAccessBlock and DeletePublicAccessBlock under the bucket actions, while the S3 User Guide's Block Public Access page gives the account level s3:PutAccountPublicAccessBlock to write and s3:GetAccountPublicAccessBlock to read, and a delete the same action as the write it undoes.
| Operation | Action | Resource |
|---|---|---|
ListTagsForResource, TagResource, UntagResource |
s3:ListTagsForResource, s3:TagResource, s3:UntagResource |
The ARN the route names, as written |
ListAccessPoints |
s3:ListAccessPoints |
All resources (*; AWS publishes no resource type, so a bucket-scoped statement grants nothing) |
PutPublicAccessBlock, DeletePublicAccessBlock |
s3:PutAccountPublicAccessBlock |
All resources (*) |
GetPublicAccessBlock |
s3:GetAccountPublicAccessBlock |
All resources (*) |
A tagging route carries the bucket's ARN in its path, and both documented forms arrive on the wire: the global form arn:<partition>:s3:::<name> that the Go SDK and Terraform's provider send, and the regional form arn:<partition>:s3:<region>:<account>:bucket/<name> that the operation's own request pattern and the AWS CLI examples use. The ARN as the caller wrote it is the resource the request is checked against, so a policy may name either form, and the ARN is read once, by the IAM check, the bucket-policy lookup and the handler together: the regional form's account is the account whose bucket the request acts on, so a request naming another account's bucket reads and writes that account's bucket, and answers NoSuchBucket when that account holds no bucket of the name, never the caller's same-named one. Query-string parameters the control plane ignores, such as ?tagging, do not re-point the authorization at a data-plane action.
A denial answers in S3 Control's own REST-XML <ErrorResponse> envelope, which follows the route rather than the resolved action, so the s3control SDK deserializers read the message that names the missing action, which Terraform's AWS provider matches to fall back from ListTagsForResource to GetBucketTagging.
Not Implemented
A bucket sub-resource the AWS model defines but Floci does not serve, such as S3 Metadata
(?metadataConfiguration, ?metadataTable and the journal, inventory and annotation table
updates), attribute-based access control (?abac) or CreateSession (?session), is answered
501 NotImplemented. A sub-resource sent with a method AWS does not define for it, such as
DELETE /bucket?versioning, is answered 405 MethodNotAllowed. Neither runs as the bucket's own
operation, so neither creates, lists nor deletes the bucket.
Configuration
| Variable | Default | Description |
|---|---|---|
FLOCI_SERVICES_S3_ENABLED |
true |
Enable or disable the service |
FLOCI_SERVICES_S3_DEFAULT_PRESIGN_EXPIRY_SECONDS |
3600 |
Default pre-signed URL expiry (1 hour) |
FLOCI_AUTH_PRESIGN_SECRET |
local-emulator-secret |
Secret used to sign pre-signed URLs |
FLOCI_SERVICES_S3_ENFORCE_AUTH |
false |
Verify SigV4 signatures and evaluate bucket policies and ACLs, including for anonymous callers (see Signature Verification) |
FLOCI_AUTH_VALIDATE_SIGNATURES |
false |
Verify SigV4 signatures without evaluating bucket policies or ACLs (see Signature Verification) |
FLOCI_SERVICES_S3_GLOBAL_BUCKET_NAMESPACE |
false |
Enable cross-account bucket and object resolution |
FLOCI_SERVICES_S3_LIFECYCLE_SWEEP_ENABLED |
true |
Apply bucket lifecycle expiration rules in the background (see Lifecycle Expiration) |
FLOCI_SERVICES_S3_LIFECYCLE_SWEEP_INTERVAL_SECONDS |
60 |
How often the lifecycle sweep runs |
Storage
Object bodies are files under the S3 data directory; the object index, s3-objects.json, holds
the metadata of every object in every bucket. Under persistent mode the index is journaled
instead of rewritten on every PutObject, CopyObject or DeleteObject: the change is appended
to s3-objects.wal, and s3-objects.json is rewritten from memory on the
FLOCI_STORAGE_WAL_COMPACTION_INTERVAL_MS cadence and at shutdown, so the cost of a write no
longer grows with the number of stored objects. Bucket definitions and the other S3 stores are
still rewritten on every change. See
Storage Modes.
In memory mode each object body is a single in-memory byte array, so an object can hold at most
about 2 GiB: a PutObject over that, an UploadPart or UploadPartCopy that takes a multipart
upload past it, or a CompleteMultipartUpload over it fails with EntityTooLarge. The disk-backed modes write a
PutObject or UploadPart body to disk as it arrives, working out its ETag and checksums on the
way, and assemble a multipart upload file to file, so no object passes through the heap; set
FLOCI_STORAGE_SERVICES_S3_MODE=persistent to keep only S3 on disk. Every request body is still
capped by FLOCI_PROTOCOLS_MAX_REQUEST_SIZE, 2048 MB by default: to accept the 5 GiB S3 allows in
one PutObject or UploadPart, raise it a little past 5120, since the aws-chunked framing SDKs
send makes a request slightly larger than its data. A request whose Content-Length is over the
limit gets an empty 413 Request Entity Too Large before S3 sees it. As on S3, more than 5 GiB of
data in one PutObject or UploadPart is rejected with EntityTooLarge, whatever the limit is
raised to: before any of it is read when the request declares its size, in Content-Length or, for
an aws-chunked body, x-amz-decoded-content-length, and otherwise as soon as it passes 5 GiB. As
S3's does, the error names the most allowed in MaxSizeAllowed and, when the request declared it,
the size proposed in ProposedSize. A request rejected before its body is read that sent
Expect: 100-continue is answered with Connection: close and its connection closed, since the
client never sends that body. With signature verification on, a request
signed over its whole payload, with a hex x-amz-content-sha256 rather than UNSIGNED-PAYLOAD or a
STREAMING- value, is still read into memory to check that hash, so it is limited to about 2 GiB.
So is a body sent with Content-Encoding: aws-chunked but neither a STREAMING-
x-amz-content-sha256 nor an x-amz-trailer, which is read whole to be decoded and fails with
EntityTooLarge past that. While it runs,
CompleteMultipartUpload needs free disk space for twice the object, its parts and the assembled
copy, until the parts are removed. Multipart uploads in progress do not survive a restart or a
reset, and their files are removed when either happens. UploadPartCopy reads only the copied
range from its source and, in the disk-backed modes, writes it straight into the part, so it can
copy a range of up to 5 GiB, S3's limit for one part, from a source of any size. A larger range
is rejected with EntityTooLarge. In the disk-backed modes
CopyObject stores the source's file for the copy without reading it into memory, and reads it
once only when the copy needs an ETag or checksum the source's cannot supply. As on S3, a
CopyObject source larger than 5 GiB is rejected with InvalidRequest; copy it in parts with
UploadPartCopy instead. S3 Select's Java evaluator reads a CSV or JSON object a record at a time,
so it takes a source of any size; see S3 Select.
Examples
export AWS_ENDPOINT_URL=http://localhost:4566
# Create bucket
aws s3 mb s3://my-bucket --endpoint-url $AWS_ENDPOINT_URL
# Upload a file
aws s3 cp ./report.pdf s3://my-bucket/reports/report.pdf --endpoint-url $AWS_ENDPOINT_URL
# Upload inline content
echo '{"hello":"world"}' | aws s3 cp - s3://my-bucket/data.json --endpoint-url $AWS_ENDPOINT_URL
# Download
aws s3 cp s3://my-bucket/data.json ./data.json --endpoint-url $AWS_ENDPOINT_URL
# Inspect object attributes without downloading the body
aws s3api get-object-attributes \
--bucket my-bucket \
--key data.json \
--object-attributes ETag ObjectSize StorageClass \
--endpoint-url $AWS_ENDPOINT_URL
# List
aws s3 ls s3://my-bucket --endpoint-url $AWS_ENDPOINT_URL
# Delete
aws s3 rm s3://my-bucket/data.json --endpoint-url $AWS_ENDPOINT_URL
# Enable versioning
aws s3api put-bucket-versioning \
--bucket my-bucket \
--versioning-configuration Status=Enabled \
--endpoint-url $AWS_ENDPOINT_URL
# Generate a pre-signed URL (valid for 1 hour)
aws s3 presign s3://my-bucket/report.pdf \
--expires-in 3600 \
--endpoint-url $AWS_ENDPOINT_URL
Addressing Styles
Floci supports both path-style and virtual-hosted style S3 addressing.
Path-Style (always works)
Path-style embeds the bucket name in the URL path:
Enable it in the SDK with forcePathStyle / pathStyleAccessEnabled:
Virtual-Hosted Style
Virtual-hosted style puts the bucket name in the hostname:
An exact configured FLOCI_HOSTNAME or floci.dns.extra-suffixes root is an
endpoint without a bucket. For example, with FLOCI_HOSTNAME=edge.localhost.floci.io,
ListBuckets at that hostname lists the account's buckets, while
my-bucket.edge.localhost.floci.io addresses my-bucket. Exact endpoint roots
take precedence over shorter DNS suffixes and existing bucket-name matches,
including controller fallback for object PUT and multipart requests.
Floci supports this natively, no forcePathStyle needed. The following wildcard DNS domains resolve to 127.0.0.1 via public DNS, so virtual-hosted requests reach Floci on the host machine automatically:
| Domain pattern | Resolves to |
|---|---|
*.localhost.floci.io |
127.0.0.1 |
*.s3.localhost.floci.io |
127.0.0.1 |
*.localhost.localstack.cloud |
127.0.0.1 |
*.s3.localhost.localstack.cloud |
127.0.0.1 |
Plain http://localhost:4566 also works without forcePathStyle, the SDK sends Host: my-bucket.localhost:4566 and Floci's filter extracts the bucket name from that header.
Configure the SDK endpoint to one of the base domains (no forcePathStyle):
Virtual-Hosted Style Inside Docker
Inside Docker containers, 127.0.0.1 resolves to the container itself, not Floci. Floci's embedded DNS server handles this automatically: it resolves *.localhost.floci.io, *.s3.localhost.floci.io, *.localhost.localstack.cloud, and *.s3.localhost.localstack.cloud to Floci's container IP on the Docker network.
To use virtual-hosted style from a test container, point its DNS at Floci and set the endpoint to Floci's service hostname:
FLOCI_IP=$(docker inspect -f '{{.NetworkSettings.Networks.floci_default.IPAddress}}' floci)
docker run --rm \
--network floci_default \
--dns "$FLOCI_IP" \
-e FLOCI_ENDPOINT=http://floci:4566 \
-e FLOCI_S3_VHOST_ENDPOINT=http://floci:4566 \
my-test-image
With FLOCI_HOSTNAME=floci set on the Floci container (default in the provided docker-compose.yml), the embedded DNS resolves my-bucket.floci to Floci's IP, and S3VirtualHostFilter extracts the bucket name from the Host header.
Object Attribute Notes
Floci now persists and returns the following object attribute state on S3 object APIs:
- user metadata from
x-amz-meta-* - storage class from
x-amz-storage-class Content-Encoding(without theaws-chunkedtoken) andCache-ControlfromPutObjectand multipart initiation, replayed onGetObjectandHeadObject- checksum metadata for object reads and
GetObjectAttributes - multipart part manifests for
GetObjectAttributes(ObjectParts) CompleteMultipartUploadvalidatesx-amz-mp-object-sizeagainst the total size of the selected parts. A mismatch returnsInvalidRequestwithout replacing the object or discarding the upload, so the request can be retried with the correct size. Blank, malformed, overflowing, or negative sizes also returnInvalidRequestand preserve the object and uploadCompleteMultipartUploadrequires every selected part except the last to contain at least 5 MiB (5,242,880 bytes). The last selected part, including a single-part upload, has no minimum size. A smaller non-final part returns400 EntityTooSmallwith its part number, ETag and proposed size. The existing object and uploaded parts are preserved so completion can be retried after replacing the undersized part.- multipart checksums follow the AWS checksum type:
COMPOSITE(<checksum of the part checksums>-<part count>) for SHA1, SHA256 and, by default, CRC32 and CRC32C;FULL_OBJECTfor CRC64NVME or whenx-amz-checksum-type: FULL_OBJECTwas requested onCreateMultipartUpload. The-Nsuffix is returned byCompleteMultipartUpload,HeadObjectandGetObject, and omitted byGetObjectAttributes, which lists part-level checksums only forCOMPOSITEobjects (aFULL_OBJECTmultipart object reports itsPartsCountalone, a single-part object has noObjectParts), as on AWS.UploadPartechoes the part checksum (x-amz-checksum-<algorithm>) when the upload declared an algorithm, and completing aCOMPOSITEupload requires the checksum of every part in theCompleteMultipartUploadbody (InvalidRequestotherwise), whileFULL_OBJECTuploads accept a body without them - canned object ACLs from
x-amz-aclonPutObject,CopyObject, and multipart initiation - explicit ACL grant headers (
x-amz-grant-read,x-amz-grant-write,x-amz-grant-full-control,x-amz-grant-read-acp,x-amz-grant-write-acp) withidandurigrantees onPutObject,CopyObject, and multipart initiation, where they apply when the upload completes - Object Lock headers (
x-amz-object-lock-mode,x-amz-object-lock-retain-until-date,x-amz-object-lock-legal-hold) onPutObject,CopyObject, and multipart initiation, where they apply when the upload completes. A mode without a date or a date without a mode, an unknown mode, or a date that is not an ISO 8601 timestamp is refused with400 InvalidArgument. In a bucket without Object Lock, a mode, a date or a legal hold ofONis refused with400 InvalidRequest, "Bucket is missing Object Lock Configuration", asPutObjectRetention,GetObjectRetention,PutObjectLegalHoldandGetObjectLegalHoldare there.GetObjectRetentionandGetObjectLegalHoldof an object without a retention or a legal hold answer404 NoSuchObjectLockConfiguration. APutObjectRetentionbody with a mode but no date, a date but no mode, or an unknown mode is refused with400 MalformedXML; one naming neither removes the retention - explicit object SSE headers from
x-amz-server-side-encryptiononPutObject,CopyObject, and multipart initiation, and form fields on browser/presigned POST, replayed onGetObjectandHeadObject - SSE-KMS key IDs from
x-amz-server-side-encryption-aws-kms-key-idonPutObject, browser/presigned POST, and multipart initiation, and fromCopyObject/UploadPartCopy(inherited from the source object when the request doesn't specify new SSE headers), returned byPutObject,CopyObject,UploadPartCopy,CompleteMultipartUpload,GetObject, andHeadObject. A bucket's default SSE-KMS key is not simulated:aws:kmswith no explicit key ID omits the header rather than returning a synthesizedaws/s3key ARN
Current limitations:
- checksum algorithms: CRC32, CRC32C, CRC64NVME, SHA1 and SHA256 are supported; the other AWS algorithms (SHA512, MD5, XXHASH3, XXHASH64, XXHASH128) are rejected with
InvalidRequest - copy-based metadata updates support
x-amz-metadata-directive: REPLACEfor user metadata and content type, but do not yet cover every AWS copy header emailAddressgrantees in thex-amz-grant-*headers are refused with501 NotImplemented- cross-account canned ACL variants collapse to the emulator's single synthetic owner where Floci does not model a distinct second principal
aws-exec-readis accepted for compatibility, but Floci does not yet model a distinct EC2 bundle-reader grantee inGetObjectAcllog-delivery-writegrants the well-known S3 log-delivery group (http://acs.amazonaws.com/groups/s3/LogDelivery)WRITEandREAD_ACP, matching AWS on the bucket path. Floci's shared canned-ACL helper also accepts it onPutObjectAcl, where AWS does not: the canned ACL is documented as bucket-only and the SDK'sObjectCannedACLenum omits it. That object path is a Floci leniency, not parity. It is the mirror image ofaws-exec-readabove, which botocore'sBucketCannedACLenum omits, though AWS's canned-ACL reference documents it as valid for both bucket and object; there the narrower party is the SDK model, not AWS
Notification event compatibility
Native queue, topic and Lambda notification records include both response identifiers and the
legacy bucket owner identity expected by existing AWS event consumers. Each destination receives
its configured rule ID, or emulator if the ID is null, empty or blank. Copies of the same operation
retain a common event time, response IDs, object details and sequencer, including version details
captured before delivery. Response IDs are synthetic emulator identifiers and do not correlate
with the initiating HTTP response headers.
The legacy ownerIdentity.principalId is the owning account's synthetic customer ID, separate
from the canonical user ID used by ACL APIs. Notification object keys use form URL encoding:
spaces become +, literal plus signs become %2B, and / is preserved. Consumers must ignore
additional fields as described in the
AWS event message structure.