S3
Protocol: REST XML
Endpoint: http://localhost:4566/{bucket}/{key}
Supported Operations
| Category | Operations |
|---|---|
| Buckets | ListBuckets, CreateBucket, HeadBucket, DeleteBucket, GetBucketLocation |
| Objects | PutObject, GetObject, GetObjectAttributes, HeadObject, DeleteObject, DeleteObjects, CopyObject |
| Listing | ListObjects, ListObjectsV2, ListObjectVersions |
| Multipart | CreateMultipartUpload, UploadPart, CompleteMultipartUpload, AbortMultipartUpload, ListMultipartUploads |
| Versioning | PutBucketVersioning, GetBucketVersioning |
| Tagging | PutBucketTagging, GetBucketTagging, PutObjectTagging, GetObjectTagging, DeleteObjectTagging |
| Annotations | PutObjectAnnotation, GetObjectAnnotation, ListObjectAnnotations, DeleteObjectAnnotation |
| Policy | PutBucketPolicy, GetBucketPolicy, DeleteBucketPolicy |
| CORS | PutBucketCors, GetBucketCors, DeleteBucketCors |
| Lifecycle | PutBucketLifecycle, GetBucketLifecycle, DeleteBucketLifecycle |
| ACL | PutBucketAcl, GetBucketAcl, PutObjectAcl, GetObjectAcl |
| Encryption | PutBucketEncryption, GetBucketEncryption, DeleteBucketEncryption |
| Notifications | PutBucketNotification, GetBucketNotification |
| Object Lock | PutObjectLockConfiguration, GetObjectLockConfiguration, PutObjectRetention, GetObjectRetention, PutObjectLegalHold, GetObjectLegalHold |
| Website | PutBucketWebsite, GetBucketWebsite, DeleteBucketWebsite |
| Pre-signed URLs | Generates and validates pre-signed GET/PUT URLs |
| S3 Select | SelectObjectContent |
| Public Access Block | PutPublicAccessBlock, GetPublicAccessBlock, DeletePublicAccessBlock |
| Metrics | PutBucketMetricsConfiguration, GetBucketMetricsConfiguration, ListBucketMetricsConfigurations, DeleteBucketMetricsConfiguration |
| Intelligent-Tiering | PutBucketIntelligentTieringConfiguration, GetBucketIntelligentTieringConfiguration, ListBucketIntelligentTieringConfigurations, DeleteBucketIntelligentTieringConfiguration |
| Analytics | PutBucketAnalyticsConfiguration, GetBucketAnalyticsConfiguration, ListBucketAnalyticsConfigurations, DeleteBucketAnalyticsConfiguration |
| Inventory | PutBucketInventoryConfiguration, GetBucketInventoryConfiguration, ListBucketInventoryConfigurations, DeleteBucketInventoryConfiguration |
| Replication | PutBucketReplication, GetBucketReplication, DeleteBucketReplication |
RestoreObject is accepted but stubbed: Floci validates the request and returns 202 Accepted, but no restore state machine runs.
Replication is configuration-only: the replication configuration is stored on the bucket and round-trips through GetBucketReplication, but no objects are actually replicated.
Event Notifications
Browser and presigned POST uploads emit s3:ObjectCreated:Post, matching AWS S3. They do not emit
s3:ObjectCreated:Put; use s3:ObjectCreated:* to subscribe to objects created by either method.
Annotation changes emit s3:ObjectAnnotation:Put and s3:ObjectAnnotation:Delete.
PutBucketNotificationConfiguration checks SQS, SNS, and Lambda destination ARNs and verifies
that each destination exists before replacing a bucket's configuration. A failed check leaves
the prior configuration unchanged. Accepted configurations send an s3:TestEvent message to
SQS and SNS destinations; unlike object notifications, this message has no Records array.
x-amz-skip-destination-validation: true skips existence checks and test messages, but still
requires well-formed destination ARNs. Destination resource-policy permissions are not checked.
Existing SNS topics in another account are checked for existence without receiving a test message,
regardless of IAM enforcement. Their object events are not delivered because SNS publish still
resolves topics in the caller's account. With IAM enforcement enabled, SQS queues in another account
are also checked without a test message. Object-event delivery is not yet subject to a shared service
authorizer. AWS's same-Region destination requirement is not enforced. Cross-account Lambda
destination routing is not emulated.
Object Annotations
Annotations are named UTF-8 text payloads (up to 1 MiB each, 1,000 per object version) attached to a
specific object version through the four ?annotation operations. Notes on the emulation:
- Annotation names allow letters (any language), digits,
_,., and-; names longer than 512 bytes, empty or whitespace-only names, names with other characters, and names starting withawsors3(case-insensitive) are rejected. - Payloads must be valid UTF-8 text between 1 byte and 1 MiB; anything else is rejected with 400,
and non-UTF-8 payloads return 415
UnsupportedMediaType. x-amz-object-if-matchis validated against the parent object's ETag on put and delete.- Versioning semantics match AWS: annotations attach to one object version, new versions do not inherit them, overwriting a non-versioned object or deleting it drops its annotations, a delete marker preserves the underlying version's annotations, and deleting a specific version deletes its annotations. Annotation deletion is permanent.
CopyObjectcopies annotations by default; thex-amz-object-annotation-directiveheader (as the AWS SDK sends it;x-amz-annotation-directiveis also accepted) set toEXCLUDEskips them.- Checksums are per-annotation and independent of the object checksum. The default algorithm is CRC64NVME. Supported: CRC32, CRC32C, CRC64NVME, SHA1, SHA256. SHA512, XXHASH64, XXHASH3, XXHASH128, and MD5 are rejected as unsupported.
- Annotations on SSE-C encrypted objects are rejected, as on AWS, and are not copied onto SSE-C copy destinations.
- Annotation operations serialize against object writes on the same bucket. On Object
Lock-protected versions, annotation put and delete follow the same rules as object delete:
governance retention requires
x-amz-bypass-governance-retention(put never takes the bypass), compliance and legal hold always block. - The literal
versionId=null(as reported by ListObjectVersions for pre-versioning objects) addresses the pre-versioning entry. - Annotations are stored per AWS account. With
globalBucketNamespaceenabled, object reads resolve cross-account but annotation reads, writes, and listings stay in the caller's account. - S3 Metadata annotation tables and annotation replication are not implemented.
Website Hosting
Static website requests use a bucket website hostname such as
http://my-bucket.s3-website-us-east-1.localhost:4566/. Floci supports:
- GET and HEAD requests through website hostnames
- index-document resolution for the site root and slash-terminated prefixes
302redirects that append a slash when an index document exists below a prefix- configured error documents and the default website error response
HEAD returns the same status and object metadata as GET without a response body. Redirect-all and advanced routing-rule configurations are not implemented.
Range Reads
GetObject supports standard Range: bytes= requests. Partial responses return 206, Content-Range, Content-Length, Accept-Ranges: bytes, and object metadata. Whole-object checksum headers are omitted on ranged responses because the stored checksum covers the full object, not the returned byte slice. Full-object and ranged reads stream object bytes from storage instead of materializing the response body first.
Suffix ranges against empty objects return an empty 200 response, matching AWS behavior used by clients that issue tail reads.
S3 Select
SelectObjectContent runs SQL queries directly against S3 objects without downloading the entire file. Floci supports CSV, JSON Lines, JSON arrays, and Parquet inputs. The SQL dialect follows AWS S3 Select SQL reference.
Execution modes
Floci chooses the execution engine automatically based on the input format and whether the floci-duck sidecar is running:
| Condition | Engine | Notes |
|---|---|---|
| Input is Parquet | floci-duck (required) | DuckDB's read_parquet — sidecar must be available |
Input is CSV with FileHeaderInfo=USE and floci-duck is running |
floci-duck | Full DuckDB SQL: all operators, LIKE, BETWEEN, IN, IS NULL, AND/OR/NOT |
| Input is JSON and floci-duck is running | floci-duck | read_json_auto — supports JSON Lines and JSON arrays |
Input is CSV with FileHeaderInfo=NONE or IGNORE, or floci-duck is not running |
Java evaluator | Supports SELECT *, column projection, simple WHERE with =, !=, <, >, <=, >=, LIKE, BETWEEN, IN, IS NULL, AND/OR/NOT, LIMIT |
The floci-duck sidecar starts lazily on the first Athena query. Until then, isAvailable() returns false and S3 Select falls back to the Java evaluator for CSV and JSON. Once the sidecar is running, subsequent S3 Select calls route through DuckDB automatically.
If floci-duck is not running and the object is Parquet, S3 Select returns an error — Parquet decoding requires DuckDB.
FileHeaderInfo modes (CSV)
| Value | Behavior |
|---|---|
USE |
First row is the header; column names are available in WHERE and SELECT |
IGNORE |
First row is skipped and not included in output; only positional _N references work |
NONE |
All rows are data; only positional _N references work (e.g. WHERE _1 = 'Alice') |
Supported SQL operators
When using the Java evaluator (no floci-duck, or CSV with FileHeaderInfo=NONE/IGNORE):
- Comparison:
=,!=,<>,<,>,<=,>= - Pattern matching:
LIKE(supports%and_wildcards) - Range:
BETWEEN ... AND ... - Set membership:
IN (...) - Null checks:
IS NULL,IS NOT NULL - Logical:
AND,OR,NOT - Clauses:
SELECT *, column projection,LIMIT
Output formats
S3 Select supports cross-format output: a CSV object can produce JSON output and vice versa.
| Input | Output | Format |
|---|---|---|
| CSV | CSV | Default — comma-separated values |
| CSV | JSON | One JSON object per row: {"col1":"val1","col2":"val2"} |
| JSON | JSON | Default — one JSON object per line |
| JSON | CSV | Comma-separated values, values quoted when they contain commas or newlines |
Example
export AWS_ENDPOINT_URL=http://localhost:4566
# Upload a CSV file
printf 'name,age,city\nAlice,30,New York\nBob,25,\nCharlie,35,London\n' \
| aws s3 cp - s3://my-bucket/people.csv
# Query with WHERE and column projection
aws s3api select-object-content \
--bucket my-bucket \
--key people.csv \
--expression "SELECT name, city FROM S3Object WHERE age >= 30" \
--expression-type SQL \
--input-serialization '{"CSV":{"FileHeaderInfo":"USE"}}' \
--output-serialization '{"CSV":{}}' \
/dev/stdout
# IS NULL check
aws s3api select-object-content \
--bucket my-bucket \
--key people.csv \
--expression "SELECT name FROM S3Object WHERE city IS NULL" \
--expression-type SQL \
--input-serialization '{"CSV":{"FileHeaderInfo":"USE"}}' \
--output-serialization '{"CSV":{}}' \
/dev/stdout
# JSON Lines input
printf '{"name":"Alice","score":95}\n{"name":"Bob","score":72}\n' \
| aws s3 cp - s3://my-bucket/scores.json
aws s3api select-object-content \
--bucket my-bucket \
--key scores.json \
--expression "SELECT * FROM S3Object WHERE score > 80" \
--expression-type SQL \
--input-serialization '{"JSON":{"Type":"LINES"}}' \
--output-serialization '{"JSON":{}}' \
/dev/stdout
Mock mode note
When FLOCI_SERVICES_ATHENA_MOCK=true is set, Athena queries are stubbed but floci-duck does not start. In that configuration, S3 Select uses the Java evaluator for CSV and JSON. Parquet queries will fail unless FLOCI_SERVICES_DUCK_URL points to an already-running floci-duck instance.
Bucket Names
Floci does not hold bucket names to AWS's full DNS naming rules, so a name AWS would refuse is
usually accepted here. The exception is a name that would not stay a single directory under its
account's storage root: an empty name, ., .., a name containing a path separator, or one of
Floci's reserved storage directories (.accounts, .versions, .annotations) is rejected with
InvalidBucketName. On the persistent and hybrid backends such a name either climbs out of the
owning account's directory or collides with object version and annotation data.
Bucket Regions
CreateBucket follows AWS's region rules, in every partition. The endpoint a request is signed for
decides what it accepts:
- The
us-east-1endpoint takes aLocationConstraintnaming any other region, and no constraint at all (the bucket then lives inus-east-1). A constraint ofus-east-1itself isInvalidLocationConstraint. - Every other regional endpoint requires a constraint naming exactly its own region. No constraint,
or a different region (including
us-east-1), isIllegalLocationConstraintException.
The AWS SDKs send the constraint for any region but us-east-1, so SDK clients are unaffected. A
raw-HTTP client that signs for another region and sends an empty body has to add the
CreateBucketConfiguration; the signing region alone does not place a bucket. GetBucketLocation
answers an empty constraint only for us-east-1. See AWS Partitions
for how this applies in China and GovCloud.
Global Bucket Namespace
By default S3 buckets are isolated per account (see Multi-Account Isolation),
so two accounts can each hold a bucket with the same name. Set
FLOCI_SERVICES_S3_GLOBAL_BUCKET_NAMESPACE=true to make bucket and object resolution span every
account's partition — a bucket created in one account then resolves cross-account, matching real S3
where bucket names are globally unique. Enable this when a workload (for example an LZA log-archive or
CDK asset bucket) is created in one account and read from another.
Why the flag exists, and what it trades away. Floci partitions every resource by the caller's account, which is the right model for services whose names are account-scoped in AWS. S3 bucket names are not: they are unique across all accounts, a bucket lives in exactly one owning account, and whether another account may touch it is decided by policy rather than by the name being unreachable. A hard per-account partition therefore lets two accounts hold different buckets under one name — a state AWS cannot reach — and makes a legitimate cross-account call fail with NoSuchBucket instead of being authorized or denied on its merits. With the flag on, a caller that names another account's bucket resolves it and only IAM and bucket-policy evaluation stand between the caller and the object: AWS-faithful, but stricter than the implicit isolation Floci gives you elsewhere — provided both FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED (identity-policy checks, applied before the request reaches S3) and FLOCI_SERVICES_S3_ENFORCE_AUTH (the S3-specific flag that gates bucket-policy evaluation itself, in S3Service.authorizeS3Read/authorizeS3Write) are also true. Both default to false, and bucket-policy evaluation in particular is a no-op while FLOCI_SERVICES_S3_ENFORCE_AUTH is off — regardless of the IAM flag — so in the default configuration enabling the global namespace does not trade per-account isolation for a policy-gated boundary — it removes the only isolation Floci gives S3 for that bucket and replaces it with nothing. Mutations are written back to the bucket's owning account rather than forked into the caller's, and ListBuckets/ListObjects stay owner-scoped, so the flag never reassigns ownership or leaks a bucket inventory. It is off by default — turn it on when emulating a multi-account estate whose real behaviour depends on the global namespace, and leave it off when the per-account partition is the isolation you are relying on. Account-scoped S3 state, notably the account-level Block Public Access configuration, is never widened by this flag. Note also that retrofitting a global namespace onto state that was previously partitioned per account means that if two accounts already hold a bucket under the same name — a state real AWS can't reach, but Floci's prior per-account isolation could — resolution picks whichever account's copy the backend happens to iterate to first; this is unlikely to be reachable on a fresh LZA-driven account structure, but is worth knowing if you enable the flag on an estate with pre-existing same-name buckets.
Signature Verification
AWS treats authentication and authorization as two steps. S3 first verifies the SigV4 signature of a signed request and refuses one that does not verify, whatever the bucket policy says. Only then does it decide, from IAM policies, the bucket policy and ACLs, whether the caller may perform the operation. An unsigned request is anonymous, and is allowed only when a policy or ACL grants access to everyone.
Floci verifies signatures when either of two flags is on:
FLOCI_AUTH_VALIDATE_SIGNATURES |
FLOCI_SERVICES_S3_ENFORCE_AUTH |
|
|---|---|---|
Authorization header signatures |
Verified | Verified |
Presigned URL signatures (X-Amz-Signature) |
Verified | Verified |
| Presigned POST signatures (browser form) | Verified when the form carries signature fields | Verified, and required |
| Bucket policy and ACL evaluation | Not evaluated | Evaluated |
| Unsigned (anonymous) requests | Allowed | Allowed only when a policy or ACL grants public access |
For a signed request, verification rebuilds the canonical request from the wire path, the query string, the headers named in SignedHeaders and the declared payload hash, derives the signing key from the access key's secret, and compares the signatures in constant time. It rejects:
- a signature that does not verify, or one computed for another key or method, with
403 SignatureDoesNotMatch - an access key that is neither
testnor registered in Floci IAM (CreateAccessKeykeys, and STS keys with their session token), with403 InvalidAccessKeyId. UnderFLOCI_AUTH_VALIDATE_SIGNATURESalone, a presigned URL from such a key falls back to the olderFLOCI_AUTH_PRESIGN_SECRETcheck instead, which an SDK-made URL fails with403 SignatureDoesNotMatch - a header-signed request more than 15 minutes from server time, with
403 RequestTimeTooSkewed - a body that does not match a declared
x-amz-content-sha256digest, with400 XAmzContentSHA256Mismatch
FLOCI_AUTH_VALIDATE_SIGNATURES is authentication alone: use it when forged or stale credentials must fail but buckets should stay reachable without policies. FLOCI_SERVICES_S3_ENFORCE_AUTH adds authorization, so it also decides anonymous access the way AWS does: a bucket that anonymous clients read needs a public-read bucket policy or ACL.
A workload must sign with credentials Floci can verify once either flag is on. A container Floci launches without a role, such as a Lambda function with no execution role, is given placeholder credentials that no flag accepts, so give it a role, or sign with test/test.
Bucket Policy Enforcement
Floci evaluates S3 bucket policies for authenticated (signed) requests when policy enforcement is enabled. Two flags control this evaluation:
FLOCI_SERVICES_S3_ENFORCE_AUTH: gates bucket policy authorization directly withinS3Servicefor bucket read, object write, and bucket write operations. It also turns on signature verification.FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED: activates the global IAM request filter to evaluate identity-based policies and resource policies before requests reach the service layer.
Both flags default to false for backward compatibility.
Evaluated Policies and Operations
When FLOCI_SERVICES_S3_ENFORCE_AUTH or FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED is active:
- The bucket's attached policy document (configured via PutBucketPolicy) is retrieved.
- The caller's authenticated principal ARN is resolved from the signing credentials (for example, arn:aws:iam::<account>:user/<name>, arn:aws:iam::<account>:role/<name>, or arn:aws:iam::<account>:root).
- Policy statements are evaluated for the requested S3 action (e.g. s3:GetObject, s3:PutObject, s3:PutBucketPolicy) and resource ARN (arn:aws:s3:::bucket or arn:aws:s3:::bucket/key).
Supported Principal Types
Bucket policy statements can specify principals using Principal or NotPrincipal with scalar strings or arrays:
- Wildcard:
"*"or{"AWS": "*"}matches any caller. - IAM User:
{"AWS": "arn:aws:iam::<account>:user/<name>"}. - IAM Role:
{"AWS": "arn:aws:iam::<account>:role/<path><name>"}. A session of the role matches it through the role's own ARN, path included. - AWS Account:
{"AWS": "<12-digit-account-id>"}or{"AWS": "arn:aws:iam::<account>:root"}matches any principal belonging to the specified account. - Canonical user:
{"CanonicalUser": "<canonical-id>"}names an account by its S3 canonical user ID and matches any principal belonging to that account, as an account principal does. Floci's canonical ID for an account is its account id, the owner IDGetBucketAclandGetObjectAclreport. - Service Principal:
{"Service": "<service>.amazonaws.com"}. - NotPrincipal: Inverts matching so the statement applies to any principal not matching the specified patterns.
The same matching applies to every check S3 makes against a bucket policy, including the source of CopyObject and UploadPartCopy.
Evaluation Rules
- An explicit Deny always wins and rejects the request.
- An explicit Allow grants access.
- For bucket owners (principals within the account that owns the bucket), the default decision is Allow unless explicitly denied.
- For non-owner principals, an explicit Allow in the bucket policy is required; neutral evaluation results in Deny.
- Unsigned (anonymous) requests and ACL-based public access fall back to standard S3 ACL and anonymous authorization paths.
- Unknown access key IDs return HTTP 403 with
InvalidAccessKeyId.
With FLOCI_SERVICES_IAM_ENFORCEMENT_ENABLED also on, the bucket policy and the caller's identity policies are combined the way AWS combines them. For same-account access, an explicit Allow in either policy grants the request, so a principal that only the bucket policy names needs no identity policy at all. For cross-account access, both the bucket policy and the caller's identity policy must allow the action. An explicit Deny in either policy denies the request in both cases.
Wire Response Shape
When a request is denied by bucket policy enforcement, Floci returns HTTP 403 with an S3 XML error body including the <Resource> element:
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<Resource>/my-bucket/my-key</Resource>
<RequestId>...</RequestId>
</Error>
Block Public Access
Floci enforces all four Block Public Access settings. A bucket's configuration is combined with the bucket owner account's and the most restrictive of the two wins, which for four independent booleans is a per-flag OR: a flag set at either level is in force.
| Setting | What it does | Needs enforce-auth |
|---|---|---|
BlockPublicPolicy |
PutBucketPolicy returns AccessDenied (403) when the submitted policy is public |
No |
BlockPublicAcls |
PutBucketAcl, PutObjectAcl and a PutObject carrying a public ACL return AccessDenied (403) |
No |
RestrictPublicBuckets |
A bucket whose policy is public serves only the owner account: anonymous and cross-account callers are denied | Yes |
IgnorePublicAcls |
A public ACL on the bucket or on an object stops granting access | Yes |
The split follows AWS. The first two reject the write that would introduce public access, and AWS applies them whoever the caller is, so they need no flag: put-public-access-block followed by a public put-bucket-policy returns 403 in Floci's default configuration. The second two suppress access that an existing policy or ACL would grant, which only has meaning once anonymous authorization runs at all, so they take effect when FLOCI_SERVICES_S3_ENFORCE_AUTH is on. With that flag off, Floci cannot tell an anonymous caller from a signed one and every request is authorized regardless.
When upgrading, a workflow that already sets BlockPublicPolicy or BlockPublicAcls may now receive 403 on a later public policy or ACL write, even with enforce-auth off. This matches AWS behavior.
As in AWS, enabling a setting never rewrites stored state. An existing public policy or ACL stays exactly as written, and clearing the setting makes the bucket public again.
The meaning of "public"
An ACL is public when it grants any permission to the AllUsers or AuthenticatedUsers predefined groups. This is broader than the grants anonymous authorization consults: a WRITE_ACP grant to AllUsers is public here even though it gives an anonymous caller no read.
A bucket policy is assumed public and then checked for a reason it is not. A statement with a wildcard principal ("Principal": "*", {"AWS": "*"}, or an Allow on NotPrincipal) is public unless a positive condition pins one of aws:PrincipalArn, aws:PrincipalAccount, aws:PrincipalOrgID, aws:PrincipalOrgPaths, aws:SourceArn, aws:SourceVpc, aws:SourceVpce, aws:SourceOwner, aws:SourceAccount, aws:userid, s3:DataAccessPointArn or s3:DataAccessPointAccount to a fixed value, one containing neither a wildcard nor an IAM policy variable. In a bucket policy, s3:DataAccessPointArn may contain a wildcard in the access point name if the account ID is fixed, for example arn:aws:s3:us-west-2:123456789012:accesspoint/*. aws:SourceIp counts too, but only for a range no broader than /8 (IPv4) or /32 (IPv6), matching AWS's treatment of very wide CIDR blocks as public.
For multivalued condition keys such as aws:PrincipalOrgPaths, ForAnyValue: with a positive operator can narrow the grant when each value is fixed. ForAllValues: alone does not narrow it because the condition also matches when the key is missing.
A single public statement makes the whole policy public. As on AWS, RestrictPublicBuckets then withholds even the non-public cross-account delegation another statement grants, until the public statement is removed.
Known gaps
- Floci does not model access points, so the access-point variants (
PutAccessPointPolicy, the VPC-origin rule, the differents3:DataAccessPointArntreatment) do not apply. - AWS's organization-level Block Public Access policies are not modelled; only the bucket and account levels combine.
CreateBucketdoes not apply anx-amz-aclat all in Floci, so there is no public bucket ACL at creation for the account-levelBlockPublicAclsto reject.GetBucketAclreturns the stored ACL rather than the effective one, so it does not reflect anIgnorePublicAclsthat is suppressing a grant.GetBucketPolicyStatushas no handler.RestrictPublicBucketsexempts AWS service principals on AWS. Floci reaches the signed path only for IAM principals, so there is nothing to exempt.
Account-level operations
The s3control PutPublicAccessBlock, GetPublicAccessBlock and DeletePublicAccessBlock operations are keyed by the x-amz-account-id header, stored per account, and persisted like the rest of S3's state so a restart does not silently drop the control. As in AWS, the header must name the caller's own account; a mismatch returns AccessDenied (403). Floci keeps one deliberate exception: the configured default (management) account may act on any account, because Floci's launched Lambdas run on placeholder credentials that resolve to the management account instead of assuming a role in the target account — which is how LZA's Custom::PutPublicAccessBlock resource reaches this API.
Not Implemented
These AWS S3 features have no handler in Floci. Calls will return an error (typically 404 or NoSuchBucket-style):
- Access logging (
PutBucketLogging,GetBucketLogging) - Request payment (
PutBucketRequestPayment,GetBucketRequestPayment)
Configuration
| Variable | Default | Description |
|---|---|---|
FLOCI_SERVICES_S3_ENABLED |
true |
Enable or disable the service |
FLOCI_SERVICES_S3_DEFAULT_PRESIGN_EXPIRY_SECONDS |
3600 |
Default pre-signed URL expiry (1 hour) |
FLOCI_AUTH_PRESIGN_SECRET |
local-emulator-secret |
Secret used to sign pre-signed URLs |
FLOCI_SERVICES_S3_ENFORCE_AUTH |
false |
Verify SigV4 signatures and evaluate bucket policies and ACLs, including for anonymous callers (see Signature Verification) |
FLOCI_AUTH_VALIDATE_SIGNATURES |
false |
Verify SigV4 signatures without evaluating bucket policies or ACLs (see Signature Verification) |
FLOCI_SERVICES_S3_GLOBAL_BUCKET_NAMESPACE |
false |
Enable cross-account bucket and object resolution |
Storage
Object bodies are files under the S3 data directory; the object index, s3-objects.json, holds
the metadata of every object in every bucket. Under persistent mode the index is journaled
instead of rewritten on every PutObject, CopyObject or DeleteObject: the change is appended
to s3-objects.wal, and s3-objects.json is rewritten from memory on the
FLOCI_STORAGE_WAL_COMPACTION_INTERVAL_MS cadence and at shutdown, so the cost of a write no
longer grows with the number of stored objects. Bucket definitions and the other S3 stores are
still rewritten on every change. See
Storage Modes.
In memory mode each object body is a single in-memory byte array, so an object can hold at most
about 2 GiB: an UploadPart that takes a multipart upload past that, or a
CompleteMultipartUpload over it, fails with EntityTooLarge. The disk-backed modes assemble a
multipart upload file to file, so the object never passes through the heap; set
FLOCI_STORAGE_SERVICES_S3_MODE=persistent to keep only S3 on disk. While it runs,
CompleteMultipartUpload needs free disk space for twice the object, its parts and the assembled
copy, until the parts are removed. Multipart uploads in progress do not survive a restart or a
reset, and their files are removed when either happens. UploadPartCopy reads only the copied
range from its source, so it can copy any part of a source of any size, but one copied part can
hold at most about 2 GiB; a larger range fails with EntityTooLarge. CopyObject and S3 Select
still read the whole source object into memory in every mode, so they need a source under 2 GiB,
and deleting the current version of a key in a versioned bucket reads the version that takes its
place the same way.
Examples
export AWS_ENDPOINT_URL=http://localhost:4566
# Create bucket
aws s3 mb s3://my-bucket --endpoint-url $AWS_ENDPOINT_URL
# Upload a file
aws s3 cp ./report.pdf s3://my-bucket/reports/report.pdf --endpoint-url $AWS_ENDPOINT_URL
# Upload inline content
echo '{"hello":"world"}' | aws s3 cp - s3://my-bucket/data.json --endpoint-url $AWS_ENDPOINT_URL
# Download
aws s3 cp s3://my-bucket/data.json ./data.json --endpoint-url $AWS_ENDPOINT_URL
# Inspect object attributes without downloading the body
aws s3api get-object-attributes \
--bucket my-bucket \
--key data.json \
--object-attributes ETag ObjectSize StorageClass \
--endpoint-url $AWS_ENDPOINT_URL
# List
aws s3 ls s3://my-bucket --endpoint-url $AWS_ENDPOINT_URL
# Delete
aws s3 rm s3://my-bucket/data.json --endpoint-url $AWS_ENDPOINT_URL
# Enable versioning
aws s3api put-bucket-versioning \
--bucket my-bucket \
--versioning-configuration Status=Enabled \
--endpoint-url $AWS_ENDPOINT_URL
# Generate a pre-signed URL (valid for 1 hour)
aws s3 presign s3://my-bucket/report.pdf \
--expires-in 3600 \
--endpoint-url $AWS_ENDPOINT_URL
Addressing Styles
Floci supports both path-style and virtual-hosted style S3 addressing.
Path-Style (always works)
Path-style embeds the bucket name in the URL path:
Enable it in the SDK with forcePathStyle / pathStyleAccessEnabled:
Virtual-Hosted Style
Virtual-hosted style puts the bucket name in the hostname:
Floci supports this natively — no forcePathStyle needed. The following wildcard DNS domains resolve to 127.0.0.1 via public DNS, so virtual-hosted requests reach Floci on the host machine automatically:
| Domain pattern | Resolves to |
|---|---|
*.localhost.floci.io |
127.0.0.1 |
*.s3.localhost.floci.io |
127.0.0.1 |
*.localhost.localstack.cloud |
127.0.0.1 |
*.s3.localhost.localstack.cloud |
127.0.0.1 |
Plain http://localhost:4566 also works without forcePathStyle — the SDK sends Host: my-bucket.localhost:4566 and Floci's filter extracts the bucket name from that header.
Configure the SDK endpoint to one of the base domains (no forcePathStyle):
Virtual-Hosted Style Inside Docker
Inside Docker containers, 127.0.0.1 resolves to the container itself — not Floci. Floci's embedded DNS server handles this automatically: it resolves *.localhost.floci.io, *.s3.localhost.floci.io, *.localhost.localstack.cloud, and *.s3.localhost.localstack.cloud to Floci's container IP on the Docker network.
To use virtual-hosted style from a test container, point its DNS at Floci and set the endpoint to Floci's service hostname:
FLOCI_IP=$(docker inspect -f '{{.NetworkSettings.Networks.floci_default.IPAddress}}' floci)
docker run --rm \
--network floci_default \
--dns "$FLOCI_IP" \
-e FLOCI_ENDPOINT=http://floci:4566 \
-e FLOCI_S3_VHOST_ENDPOINT=http://floci:4566 \
my-test-image
With FLOCI_HOSTNAME=floci set on the Floci container (default in the provided docker-compose.yml), the embedded DNS resolves my-bucket.floci to Floci's IP, and S3VirtualHostFilter extracts the bucket name from the Host header.
Object Attribute Notes
Floci now persists and returns the following object attribute state on S3 object APIs:
- user metadata from
x-amz-meta-* - storage class from
x-amz-storage-class - checksum metadata for object reads and
GetObjectAttributes - multipart part manifests for
GetObjectAttributes(ObjectParts) - multipart checksums follow the AWS checksum type:
COMPOSITE(<checksum of the part checksums>-<part count>) for SHA1, SHA256 and, by default, CRC32 and CRC32C;FULL_OBJECTfor CRC64NVME or whenx-amz-checksum-type: FULL_OBJECTwas requested onCreateMultipartUpload. The-Nsuffix is returned byCompleteMultipartUpload,HeadObjectandGetObject, and omitted byGetObjectAttributes, which lists part-level checksums only forCOMPOSITEobjects (aFULL_OBJECTmultipart object reports itsPartsCountalone, a single-part object has noObjectParts), as on AWS.UploadPartechoes the part checksum (x-amz-checksum-<algorithm>) when the upload declared an algorithm, and completing aCOMPOSITEupload requires the checksum of every part in theCompleteMultipartUploadbody (InvalidRequestotherwise), whileFULL_OBJECTuploads accept a body without them - canned object ACLs from
x-amz-aclonPutObject,CopyObject, and multipart initiation - explicit object SSE headers from
x-amz-server-side-encryptiononPutObject,CopyObject, and multipart initiation, replayed onGetObjectandHeadObject - SSE-KMS key IDs from
x-amz-server-side-encryption-aws-kms-key-idonPutObjectand multipart initiation, and fromCopyObject/UploadPartCopy(inherited from the source object when the request doesn't specify new SSE headers), returned byPutObject,CopyObject,UploadPartCopy,CompleteMultipartUpload,GetObject, andHeadObject. A bucket's default SSE-KMS key is not simulated:aws:kmswith no explicit key ID omits the header rather than returning a synthesizedaws/s3key ARN
Current limitations:
- checksum algorithms: CRC32, CRC32C, CRC64NVME, SHA1 and SHA256 are supported; the other AWS algorithms (SHA512, MD5, XXHASH3, XXHASH64, XXHASH128) are rejected with
InvalidRequest - copy-based metadata updates support
x-amz-metadata-directive: REPLACEfor user metadata and content type, but do not yet cover every AWS copy header - explicit ACL grant headers such as
x-amz-grant-readandx-amz-grant-full-controlare not modeled yet - cross-account canned ACL variants collapse to the emulator's single synthetic owner where Floci does not model a distinct second principal
aws-exec-readis accepted for compatibility, but Floci does not yet model a distinct EC2 bundle-reader grantee inGetObjectAcllog-delivery-writegrants the well-known S3 log-delivery group (http://acs.amazonaws.com/groups/s3/LogDelivery)WRITEandREAD_ACP, matching AWS on the bucket path. Floci's shared canned-ACL helper also accepts it onPutObjectAcl, where AWS does not: the canned ACL is documented as bucket-only and the SDK'sObjectCannedACLenum omits it. That object path is a Floci leniency, not parity. It is the mirror image ofaws-exec-readabove, which botocore'sBucketCannedACLenum omits, though AWS's canned-ACL reference documents it as valid for both bucket and object; there the narrower party is the SDK model, not AWS