Skip to content

Cloud Logging

floci-gcp emulates Google Cloud Logging over gRPC and REST using the real google.logging.v2.LoggingServiceV2 and sink-management methods of google.logging.v2.ConfigServiceV2. The primary use case is asserting what the app under test logged: write structured entries, then ListLogEntries back with a filter and verify the payload, severity, and resource.

Configuration

Variable Default Description
FLOCI_GCP_SERVICES_LOGGING_ENABLED true Enable/disable Cloud Logging

When disabled, sink management is unavailable: the gRPC ConfigServiceV2 API is not bound and returns UNIMPLEMENTED, while REST sink requests return HTTP 503 with UNAVAILABLE. Existing sink configurations remain stored.

Endpoint

Cloud Logging has no *_EMULATOR_HOST convention. Point the client at floci-gcp by overriding the API endpoint / transport channel and disabling credentials:

  • gRPC (Java/Python/Go/Node): build the v2 client with a plaintext channel to localhost:4588 and anonymous/no credentials (see Quick Start).
  • REST: POST http://localhost:4588/v2/entries:write and :list.

Quick Start

LoggingClient client = LoggingClient.create(
    LoggingSettings.newBuilder()
        .setTransportChannelProvider(
            InstantiatingGrpcChannelProvider.newBuilder()
                .setEndpoint("localhost:4588")
                .setChannelConfigurator(b -> b.usePlaintext())
                .build())
        .setCredentialsProvider(NoCredentialsProvider.create())
        .build());

String logName = "projects/floci-local/logs/my-log";
client.writeLogEntries(WriteLogEntriesRequest.newBuilder()
    .addEntries(LogEntry.newBuilder()
        .setLogName(logName)
        .setResource(MonitoredResource.newBuilder().setType("global"))
        .setSeverity(LogSeverity.ERROR)
        .setTextPayload("something failed")
        .build())
    .build());

for (LogEntry entry : client.listLogEntries(ListLogEntriesRequest.newBuilder()
        .addResourceNames("projects/floci-local")
        .setFilter("logName=\"" + logName + "\" AND severity>=WARNING")
        .build()).iterateAll()) {
    System.out.println(entry.getTextPayload());
}
import grpc
from google.cloud import logging_v2
from google.cloud.logging_v2.services.logging_service_v2.transports.grpc import (
    LoggingServiceV2GrpcTransport,
)

transport = LoggingServiceV2GrpcTransport(channel=grpc.insecure_channel("localhost:4588"))
client = logging_v2.services.logging_service_v2.LoggingServiceV2Client(transport=transport)

log_name = "projects/floci-local/logs/my-log"
client.write_log_entries(request={"entries": [{
    "log_name": log_name,
    "resource": {"type": "global"},
    "severity": "ERROR",
    "text_payload": "something failed",
}]})

for entry in client.list_log_entries(request={
    "resource_names": ["projects/floci-local"],
    "filter": f'logName="{log_name}" AND severity>=WARNING',
}):
    print(entry.text_payload)
import { v2 } from "@google-cloud/logging";
import * as grpc from "@grpc/grpc-js";

const client = new v2.LoggingServiceV2Client({
    servicePath: "localhost",
    port: 4588,
    sslCreds: grpc.credentials.createInsecure(),
});

const logName = "projects/floci-local/logs/my-log";
await client.writeLogEntries({ entries: [{
    logName,
    resource: { type: "global" },
    severity: "ERROR",
    textPayload: "something failed",
}]});

const [entries] = await client.listLogEntries({
    resourceNames: ["projects/floci-local"],
    filter: `logName="${logName}" AND severity>=WARNING`,
});
entries.forEach((e) => console.log(e.textPayload));

Filter subset

ListLogEntries supports a practical subset of the Cloud Logging filter language, combined with AND:

Field Operators Example
logName = logName="projects/p/logs/app"
severity = > >= < <= severity>=WARNING
resource.type = resource.type="gce_instance"
timestamp = > >= < <= timestamp>="2026-01-01T00:00:00Z"
labels.<key> = labels.env="prod"

An empty filter matches all entries in the requested resourceNames. OR/NOT, functions, regex, and nested parentheses are not supported; unrecognized clauses are ignored.

Entry semantics

  • Request-level logName, resource, and labels are applied to entries that omit them (entry labels win over request defaults).
  • Missing insertId and timestamp are filled in; receiveTimestamp is always set to now.
  • Entries are returned ordered by timestamp (ascending by default; order_by: "timestamp desc" reverses).
  • DeleteLog removes all entries for a log name (NOT_FOUND if the log has no entries).

Supported Operations

  • WriteLogEntries
  • ListLogEntries
  • ListLogs
  • DeleteLog
  • CreateSink, GetSink, ListSinks, UpdateSink, DeleteSink over gRPC and REST JSON
  • CreateLogMetric, GetLogMetric, ListLogMetrics, UpdateLogMetric, DeleteLogMetric over gRPC and REST JSON (see Log-based metrics)

Sink configuration

Sink management accepts projects/{id}, organizations/{id}, folders/{id}, and billingAccounts/{id} parents. Each parent has its own namespace. Parent names are validated syntactically; an organization, folder, or billing account does not have to be provisioned in a separate resource catalog.

Operation REST method and path
Create POST /v2/{parent}/sinks
Get GET /v2/{parent}/sinks/{sinkId}
List GET /v2/{parent}/sinks
Update PUT or PATCH /v2/{parent}/sinks/{sinkId}
Delete DELETE /v2/{parent}/sinks/{sinkId}

The LogSink.name field is the short sink identifier, such as workspace-events. Get, update, and delete requests identify the sink with the full path, such as organizations/123456789/sinks/workspace-events. List supports pageSize and pageToken. Non-positive page sizes are ignored, including when continuing from a page token.

The stored configuration includes the destination, opaque filter text, description, disabled flag, includeChildren, exclusions, and BigQuery options. Destination names can reference Cloud Storage, Pub/Sub, or BigQuery. Logging project and log-bucket destinations are unsupported. Destination existence and writer access are checked at delivery time, not when configuring a sink. Filter text is retained verbatim, including expressions outside the routing subset below.

Updates accept destination, filter, description, disabled, include_children, exclusions, and bigquery_options in the gRPC field mask, including valid nested BigQuery-option paths. REST uses the corresponding lowerCamelCase mask names, such as includeChildren. Omitting the mask uses the API's compatibility default: destination,filter,includeChildren. Fields outside the mask are preserved, and masked scalar fields can be cleared. Invalid, immutable, and output-only mask paths fail with INVALID_ARGUMENT; a failed update leaves the stored sink unchanged.

Writer identities are stable service-account principals. New unique writers are shared by sinks under the same parent, following Cloud Logging's resource-shared service-agent model. Project writers use an emulated project number; folder writers use the service-folder-{id} form observed against live GCP. Organization and billing-account allocation is deterministic emulator behavior, not a reproduction of Google's account provisioning. Existing persisted writer identities remain unchanged on reads and ordinary updates. New unique identities are computed directly from the parent. Deleting a sink does not revoke its writer's IAM grants.

Project sinks use the legacy serviceAccount:cloud-logs@system.gserviceaccount.com unless uniqueWriterIdentity=true is supplied. A legacy identity can be upgraded, but a unique project identity cannot be downgraded; updating it requires uniqueWriterIdentity=true. Non-project sinks always use a unique writer regardless of this flag. Live Cloud Logging v2 folder-sink checks on 2026-10-20 UTC confirmed creation with either flag and updates with false, true, then false retain the identity over REST and gRPC.

Writers are used directly by the internal IAM evaluator for destination writes. They do not mint OAuth tokens, inherit the ingestion caller's credentials, or call SignBlob. Sink creation does not create a user-managed IAM service account or signing key. customWriterIdentity is unsupported. Legacy project identities require Pub/Sub and BigQuery destinations in the same project as the sink. Legacy Cloud Storage delivery also requires a bucket owned by the sink's project; that check runs inside the destination policy locks at delivery time.

Sink state uses cloudlogging-sinks.json through StorageFactory and the configured global storage mode. Persistent, hybrid, and WAL storage retain configuration, writer identity, and deletion across restarts. Memory storage retains state only for the process lifetime.

GCS export

After accepting a non-dry-run WriteLogEntries request, both REST and gRPC ingestion select active Cloud Storage sinks and route matching entries. Request defaults, timestamps, insert IDs, and labels are applied before routing. Enabled exclusions remove matching entries. Disabled sinks and disabled exclusions are respected. Deleting a log does not delete objects already exported. Pub/Sub and BigQuery destinations are delivered as described under Pub/Sub export and BigQuery export.

Each local write produces immutable JSON-array batches of protobuf-JSON LogEntry objects, grouped by log ID and UTC timestamp hour. URL-encoded log IDs are decoded in object paths: {logId}/YYYY/MM/DD/HH:00:00_HH:59:59_S{shard}.json. Entries received in a different hour use _A{shard}:{unixTime}.json. Shards use random numeric identifiers and a create-only generation precondition, so batches never replace an existing object. Content type is application/json; internal storage fields are excluded.

This emulator delivers promptly instead of waiting for GCP's hourly batches and initial 2-3 hour latency. Failed deliveries are logged and do not fail accepted Logging writes or block other sinks. Outage buffering, automatic retries, export metrics, production shard allocation, and exactly-once delivery are not implemented. A failed batch is not replayed automatically after permissions are granted; later writes can export normally.

Routing filters and hierarchy

Routing uses a strict subset independent of the lenient ListLogEntries parser:

  • AND, implicit conjunction (including newlines), OR, NOT, and parentheses. As in Logging, OR binds more tightly than AND.
  • logName, severity, resource.type, resource.labels.<key>, labels.<key>, textPayload, jsonPayload.<path>, timestamp, receiveTimestamp, insertId, trace, and spanId.
  • Equality/inequality and ordering comparisons, string containment (:), and field presence (:*). JSON numeric fields compare numerically; incompatible object/scalar comparisons and failed scalar conversions do not match. Strings use Unicode NFKC case folding for equality, ordering, and containment. Containment against numeric, Boolean, and object values does not match. Timestamp operands accept RFC 3339 timestamps or ISO 8601 dates (YYYY-MM-DD); dates use midnight UTC. Quoted label keys and JSON path components are supported, as are JSON arrays. JSON presence distinguishes explicit nulls from missing keys. Missing scalar strings use LogEntry defaults; missing map/JSON fields fail comparisons. Severity presence (severity:*) is unsupported because the ingestion model does not preserve explicit default-enum presence.

Unsupported expressions, functions, regular expressions, global searches, and unavailable payload fields stop delivery for the affected sink and produce a diagnostic. This also applies to enabled exclusions: an unsupported exclusion cannot accidentally allow the entry to escape. Configuration CRUD still retains these expressions for future support.

A sink receives logs owned by its parent. Project ID/number aliases resolve through the project registry. For folder and organization sinks, includeChildren additionally routes projects with the matching direct parent recorded by Resource Manager. Billing account relationships do not establish Logging ancestry. Nested folder ancestry is not modeled; unrelated or unknown hierarchies never match implicitly.

Destination IAM

With iam.authorization-mode=enforce, grant the returned writerIdentity roles/storage.objectCreator on the destination bucket or its owning project. Export checks storage.objects.create against that principal, including supported IAM Conditions and project policy inheritance. A grant to the logging caller or source project does not grant access to a bucket owned by another project. Fresh shards need no delete permission. With enforcement disabled, export follows the emulator's usual IAM bypass.

Sink configuration, selection, and delivery share one lock. Delete, disable, filter, exclusion, and destination updates wait for any in-flight local delivery; subsequent writes see the changed configuration. Destination policy locks are held through authorization and object publication in the existing order: sink lock, IAM policy locks, bucket lock, object lock. A policy revocation is ordered before or after the publication, never between its permission check and mutation.

Sink management IAM

With floci-gcp.services.iam.authorization-mode=enforce, emulator-issued service-account credentials are checked for the following methods on both REST JSON and gRPC. Anonymous and external credentials retain the emulator's bypass behavior. Disabled mode skips these checks.

Method Permission Condition resource name
CreateSink logging.sinks.create The future .../sinks/{name} resource
ListSinks logging.sinks.list The parent
GetSink logging.sinks.get The sink name
UpdateSink, REST PUT or PATCH logging.sinks.update The sink name
DeleteSink logging.sinks.delete The sink name

Project sinks evaluate the project's IAM policy and inherit an organization policy when Resource Manager has recorded that organization as the project's direct parent. Project parents include domain-scoped IDs such as projects/example.com:my-project. For those IDs, evaluated callers cannot use the shared IAM mixin to read or write the project policy; trusted policy setup remains available. Organization sinks evaluate the policy stored under the organization name. Trusted setup can write that policy through the shared IAM policy gRPC mixin. Unknown or folder ancestry does not grant an organization policy, and project policy bindings do not grant access to organization sinks. roles/logging.configWriter and roles/logging.admin grant these five permissions; roles/logging.viewer and roles/logging.privateLogViewer grant get and list. Supported basic-role slices grant all five for owner, and get and list for editor and viewer. Active custom roles can grant individual permissions through their stored includedPermissions.

The restricted condition profile supports resource.name, resource.type, and resource.service comparisons, string prefixes, logical operators, and request-time comparison. The service and type attributes are logging.googleapis.com and logging.googleapis.com/LogSink for these checks. Create checks the future sink name, list checks the parent name, and get, update, and delete check the named sink. A condition naming one sink therefore permits create or operations on that sink while denying a sibling; it does not grant list. The published IAM Conditions resource.name support list names Logging buckets and views, but not sinks. Live project and organization sink checks using official Java REST and gRPC clients confirmed these resource names, service, and type in October 2026. The organization checks covered all five RPCs and all six REST method forms, including both PUT and PATCH, with exact sink-name, parent-name, service, and type conditions. These observations apply to the tested API version and principals; they do not establish general IAM Conditions or Organization Policy equivalence. Sinks expose no IAM policy methods in the Logging v2 API. Calls to GetIamPolicy, SetIamPolicy, or TestIamPermissions on a sink through the shared IAM gRPC mixin fail with FAILED_PRECONDITION in enforce mode. A sink inherits its parent policy rather than storing a sink policy. Sink permission denials use HTTP 403 or gRPC PERMISSION_DENIED, with the observed IAM_PERMISSION_DENIED error detail naming the denied permission and resource. An evaluated caller's organization policy read or write through the mixin retains the shared IAM FAILED_PRECONDITION result for an unmapped resource. Project policy methods remain in Resource Manager.

Folder and billing-account sink CRUD remain available when enforcement is disabled, but emulator-issued service-account requests for those parents fail closed in enforce mode because their IAM policy ancestry is not modeled. Other ConfigServiceV2 methods and Logging entry ingestion/read APIs are not IAM-enforced. Local tests do not establish equivalence with real GCP IAM Conditions or Organization Policy enforcement.

The wire contracts follow the ConfigServiceV2 proto, the Logging v2 REST sink methods, and the Logging v2 discovery document. Export semantics follow Cloud Storage routing, shared writer identities and destination permissions, and the Logging query language. Live verification covered folder identity behavior and date-only timestamp comparisons at midnight UTC. Export file bytes and exact validation messages were not verified live.

Pub/Sub export

Sinks whose destination is pubsub.googleapis.com/projects/{project}/topics/{topic} use the same routing as Cloud Storage sinks: the same filter subset, exclusions, includeChildren hierarchy, disabled flags and dry-run handling. Each routed entry is published as its own message, in the order the request supplied it, right after the write is accepted. The message data is the protobuf-JSON LogEntry (the same bytes a Cloud Storage export would hold for that entry, as a single object rather than an array), and the message carries the logging.googleapis.com/timestamp attribute with the entry's timestamp.

Only entries received after the sink exists are published; nothing is backfilled when a sink is created or re-enabled. A missing topic, a denied writer, or a disabled Pub/Sub service is logged, does not fail the accepted write, and does not affect other sinks. The failed entries are not retried. Real Cloud Logging also writes a logging.googleapis.com/sink_error entry and emails the project owners for these errors; the emulator does neither.

With iam.authorization-mode=enforce, grant the writerIdentity roles/pubsub.publisher on the topic or its project. Export checks pubsub.topics.publish against that principal. As with Cloud Storage delivery, the sink lock and then the topic and project policy locks are held through the check and the publication, so a revocation is ordered before or after it, never in between.

Deviations: real Pub/Sub sinks deliver within about a minute and some sample messages carry additional <service>/resource_type and <service>/resource_id attributes; the emulator publishes synchronously and sets only the timestamp attribute. Semantics follow Pub/Sub routing and sink troubleshooting.

BigQuery export

Sinks whose destination is bigquery.googleapis.com/projects/{project}/datasets/{dataset} use the same routing as the other sinks and stream routed entries into the dataset right after the write is accepted. The dataset must exist; tables are created on demand, under the dataset's project even when the sink lives in another project.

  • Tables: one per log, named after the log ID with -, ., / and other unsupported characters replaced by _, plus the entry timestamp's UTC date (syslog_20170523, compute_googleapis_com_activity_log_20171231). With bigqueryOptions.usePartitionedTables the suffix is dropped and the table is day-partitioned on the timestamp column.
  • Columns: LogEntry field names are kept (logName, insertId, receiveTimestamp, ...); timestamp and receiveTimestamp are TIMESTAMP. The keys of labels, resource.labels and jsonPayload are lowercased, unsupported characters become _, leading underscores are removed and names are capped at 128 characters. JSON objects become RECORD columns, arrays REPEATED columns, numbers FLOAT, booleans BOOLEAN and strings STRING. An array's columns cover the keys of all its items. Null values, empty objects and empty arrays add no column and are left out of the row.
  • Schema: the first entries of a table determine its schema, and later entries may add columns. An entry whose field changes type, or whose array mixes item types or holds arrays, is written instead to export_errors_YYYYMMDD (or export_errors for partitioned sinks) with logEntry (the entry as JSON), schemaErrorDetail, sink, logName, timestamp, receiveTimestamp, severity, insertId, trace and resourceType.

A missing dataset or a disabled BigQuery service is logged, does not fail the accepted write, and does not affect other sinks. BigQuery has no IAM enforcement in the emulator, so the writer identity needs no roles/bigquery.dataEditor grant here. Inserts run under the sink lock; with the DuckDB sidecar they add a network round trip to the write.

Deviations: real BigQuery sinks stream in small batches with some latency, shorten audit log payload names, store protoPayload and httpRequest, and email project owners on schema mismatches. The emulator inserts synchronously, has no protoPayload or httpRequest to store, and represents labels and resource.labels as RECORD columns keyed by label name (the saved documentation does not state their shape). Semantics follow BigQuery routing.

Log-based metrics

MetricsServiceV2 is served on gRPC and on REST JSON at /v2/projects/{project}/metrics. A metric identifier may contain /; REST callers send it as %2F (nginx%2Frequests). UpdateLogMetric replaces the metric and creates it when it does not exist. Duplicate creates fail with ALREADY_EXISTS, missing metrics with NOT_FOUND.

Validation follows the LogMetric reference:

  • name up to 100 characters from A-Z a-z 0-9 _-.,+!*',()%/, not starting with /; filter required, up to 20,000 characters, in the same strict subset sinks use; description up to 8,000.
  • Without metricDescriptor, the metric is a counter: DELTA, INT64, no labels, unit 1. metricDescriptor.name, type and description are output only and are filled from the metric: the type is logging.googleapis.com/user/<name>. Only DELTA with INT64 or DISTRIBUTION is accepted. Metric kind, value type and existing labels cannot change on update; new labels and label descriptions can.
  • Up to 10 labels, keys [a-zA-Z][a-zA-Z0-9_]* other than log (the default label), types STRING, BOOL or INT64. Every label needs an entry in labelExtractors, and every extractor needs a label.
  • Extractors are EXTRACT(field) or REGEXP_EXTRACT(field, "regex") with exactly one capture group, over the fields the routing filter supports. The field is converted to a string first, with whole numbers (also inside objects and lists) written as integers over both gRPC and REST. A distribution needs valueExtractor and bucketOptions (linear, exponential or explicit), with at most 200 buckets counting the underflow and overflow buckets.
  • bucketName is stored and must name a bucket in the metric's project; log buckets themselves are not emulated, so it does not scope what the metric counts.

Points in Cloud Monitoring

Each accepted, non-dry-run write is counted into every enabled metric of the entry's project, after sink export. Only entries received after the metric exists count; nothing is backfilled. Each series (metric labels plus the entry's monitored resource) gets one DELTA point per write: the number of matching entries for a counter, or a Distribution of the extracted values. Every series carries the default log label, the URL-encoded LOG_ID part of the entry's logName. A failed label extraction yields "", 0 or false; a value that fails extraction or conversion to a number is not recorded. Label values keep their first 1,024 characters.

Creating or updating a metric upserts its descriptor in Cloud Monitoring (with the log label), so ListTimeSeries and GetMetricDescriptor for logging.googleapis.com/user/<name> work as for any metric. When Cloud Monitoring is disabled, metrics are still managed but record nothing.

Deviations from real Logging, which flushes in-memory counters once a minute:

  • Points are written when entries arrive. A point's interval starts where the series' previous point ended (one minute before the end for its first point), so intervals never overlap and an aligned read (ALIGN_DELTA or ALIGN_SUM over 60s) gives the same totals. No zero points are written for idle minutes.
  • Regular expressions use java.util.regex, not RE2. Each match runs under a step budget, so a pattern that backtracks heavily (such as (a+){10}$) extracts nothing rather than stalling writes.
  • Deleting a metric leaves its Cloud Monitoring descriptor and points in place. Real Logging also refuses to delete a metric an alerting policy uses; alerting policies are not emulated yet.
  • IAM: with enforcement on, logging.logMetrics.{create,get,list,update,delete} are checked on the metric (create) or project (list), with roles/logging.admin and roles/logging.configWriter granting all five and roles/logging.viewer and roles/logging.privateLogViewer get and list, as the Logging roles reference lists.

Semantics follow the log-based metrics overview, labels and distribution metrics.

Not Yet Supported

TailLogEntries (streaming), protoPayload/httpRequest/operation/sourceLocation/split fields, monitored-resource-descriptor catalog, log-based metric backfill, log buckets and other ConfigServiceV2 methods, advanced filter operators, and retention/dedup semantics.