Skip to content

Kinesis

Protocol: JSON 1.1 (X-Amz-Target: Kinesis_20131202.*) Endpoint: POST http://localhost:4566/

Supported Actions

Action Description
CreateStream Create a stream
DeleteStream Delete a stream
ListStreams List all streams
DescribeStream Get stream details and shard info
DescribeStreamSummary Lightweight stream description
RegisterStreamConsumer Register an enhanced fan-out consumer
DeregisterStreamConsumer Remove a consumer
DescribeStreamConsumer Get consumer details
ListStreamConsumers List consumers for a stream
SubscribeToShard Subscribe to a shard for enhanced fan-out
AddTagsToStream Tag a stream
RemoveTagsFromStream Remove tags
ListTagsForStream List tags
StartStreamEncryption Enable KMS encryption
StopStreamEncryption Disable encryption
SplitShard Split a shard into two
MergeShards Merge two adjacent shards
PutRecord Write a single record
PutRecords Write up to 500 records
GetShardIterator Get an iterator for reading
GetRecords Read records from a shard
ListShards -
IncreaseStreamRetentionPeriod Increase retention up to 8760 hours (365 days)
DecreaseStreamRetentionPeriod Decrease retention down to 24 hours
EnableEnhancedMonitoring -
DisableEnhancedMonitoring -
UpdateStreamMode -
UpdateMaxRecordSize -
UpdateShardCount -

Local Inspection Endpoints

These endpoints are Floci-local read-only helpers for the UI and tests. AWS SDK traffic should continue to use the JSON 1.1 Kinesis API on /.

Endpoint Description
GET /_aws/kinesis/streams List streams in the resolved region with shard, mode, tag, and record counts
GET /_aws/kinesis/records?StreamName=<name> Peek up to 100 stream records with shard attribution without consuming them
GET /_aws/kinesis/records?StreamName=<name>&Limit=<n> Peek up to n records, capped at 1000
GET /_aws/kinesis/records?StreamName=<name>&ShardId=<id> Peek records for one shard

Stream Addressing

Most actions accept either StreamName or StreamARN to identify a stream. When both are provided, StreamName takes precedence. CreateStream only accepts StreamName.

# By name
aws kinesis describe-stream --stream-name events --endpoint-url $AWS_ENDPOINT_URL

# By ARN
aws kinesis describe-stream --stream-arn arn:aws:kinesis:us-east-1:000000000000:stream/events --endpoint-url $AWS_ENDPOINT_URL

Shard Iterators

GetShardIterator supports all five iterator types: TRIM_HORIZON, LATEST, AT_SEQUENCE_NUMBER, AFTER_SEQUENCE_NUMBER, AT_TIMESTAMP.

A LATEST iterator is positioned at the shard tip at the moment the iterator is created, matching AWS: records written after the iterator was obtained are returned, records written before are not. This supports the standard tailing pattern: obtain a LATEST iterator, trigger the action that produces the record, then poll GetRecords following NextShardIterator.

Record Retention

A stream's retention period (default 24 hours, IncreaseStreamRetentionPeriod/DecreaseStreamRetentionPeriod up to 8760 hours) is enforced: a record older than the current retention period is not returned by GetRecords or the GET /_aws/kinesis/records inspection endpoint, and is actually removed from memory rather than merely hidden. Expiry is checked lazily on each stream's next PutRecord/PutRecords or read after a record ages out; there is no scheduled sweep. After DecreaseStreamRetentionPeriod, records outside the new window are inaccessible from the next put or read, in line with AWS making them inaccessible almost immediately. Shard iterators (including NextShardIterator continuation tokens and LATEST) resolve by sequence number, so they keep working correctly across pruning even if the exact record they were positioned at has since expired.

Record Routing

PutRecord and PutRecords honor ExplicitHashKey when it is provided. The value must be a decimal integer in the Kinesis hash-key space, and records are written to the open shard whose HashKeyRange contains that value. Without ExplicitHashKey, Floci keeps using the partition key to choose a shard.

Shard Scaling (UpdateShardCount)

UpdateShardCount enforces the documented default limits: at most double the current open shard count per call, at most ten calls per rolling 24-hour period per stream, a 10000-shard ceiling, and the asymmetric rule that a stream already above 10000 shards can only move to a target strictly below it.

The minimum bound (TargetShardCount cannot go below half the current open shard count) rounds up for an odd current count, so current=5 accepts target=3 but rejects target=2. AWS's documentation says only "below half" with no stated rounding, so which direction an odd count rounds is Floci's own call rather than observed AWS behavior; this is the more conservative reading (fewer shards allowed, not more).

AWS also documents a separate 10 TPS call-rate limit on this action ("Make over 10 TPS. TPS over 10 will trigger the LimitExceededException"), independent of the ten-calls-per-24h limit above. Floci does not simulate real-time request throttling for this or any other action, so that limit is intentionally unenforced here.

Configuration

Variable Default Description
FLOCI_SERVICES_KINESIS_ENABLED true Enable or disable the service
FLOCI_SERVICES_KINESIS_LIST_SHARDS_NEXT_TOKEN_TTL_MILLIS 300000 Lifetime of a ListShards NextToken in milliseconds; AWS expires tokens after 300000 ms. Lower it to exercise ExpiredNextTokenException without waiting

Enhanced Fan-Out (EFO)

SubscribeToShard uses a snapshot-and-close model: the server returns one batch of records as a binary EventStream response and closes the connection. The SDK resubscribes automatically using the ContinuationSequenceNumber from the last delivered record. All five StartingPosition types are supported: TRIM_HORIZON, LATEST, AT_SEQUENCE_NUMBER, AFTER_SEQUENCE_NUMBER, AT_TIMESTAMP.

export AWS_ENDPOINT_URL=http://localhost:4566
STREAM=my-stream

# Register a consumer
aws kinesis register-stream-consumer \
  --stream-arn $(aws kinesis describe-stream --stream-name $STREAM \
      --query StreamDescription.StreamARN --output text) \
  --consumer-name my-consumer

# Subscribe (AWS CLI streams events to stdout)
aws kinesis subscribe-to-shard \
  --consumer-arn <consumer-arn> \
  --shard-id shardId-000000000000 \
  --starting-position Type=TRIM_HORIZON

Examples

export AWS_ENDPOINT_URL=http://localhost:4566

# Create a stream
aws kinesis create-stream \
  --stream-name events \
  --shard-count 2 \
  --endpoint-url $AWS_ENDPOINT_URL

# Put a record
aws kinesis put-record \
  --stream-name events \
  --partition-key "user-123" \
  --data '{"event":"page_view","page":"/home"}' \
  --endpoint-url $AWS_ENDPOINT_URL

# Get a shard iterator
SHARD_ID=$(aws kinesis describe-stream \
  --stream-name events \
  --query 'StreamDescription.Shards[0].ShardId' --output text \
  --endpoint-url $AWS_ENDPOINT_URL)

ITERATOR=$(aws kinesis get-shard-iterator \
  --stream-name events \
  --shard-id $SHARD_ID \
  --shard-iterator-type TRIM_HORIZON \
  --query ShardIterator --output text \
  --endpoint-url $AWS_ENDPOINT_URL)

# Read records
aws kinesis get-records \
  --shard-iterator $ITERATOR \
  --endpoint-url $AWS_ENDPOINT_URL