Telemetry protocol
This page contains the core schema for telemetry event data that gets exported from Sourcegraph.
To learn more, refer to our telemetry documentation.
Table of Contents
Event
| Field | Type | Label | Description |
|---|---|---|---|
| id | string | Generated ID of the event, currently expected to be UUID v4. | |
| timestamp | google.protobuf.Timestamp | Timestamp of when the original event was recorded. | |
| feature | string | Feature associated with the event in camelCase, e.g. 'myFeature'. | |
| action | string | Action associated with the event in camelCase, e.g. 'pageView'. | |
| source | EventSource | Source of the event. | |
| parameters | EventParameters | Parameters of the event. | |
| user | EventUser | optional | Optional user associated with the event. This field should be hydrated by the Sourcegraph server, and not provided by clients. |
| feature_flags | EventFeatureFlags | optional | Optional feature flags configured in the context of the event. |
| marketing_tracking | EventMarketingTracking | optional | Optional marketing campaign tracking parameters. π¨ SECURITY: This metadata is NEVER exported from single-tenant Sourcegraph instances, and is only exported for events tracked in the public Sourcegraph.com instance and managed services. |
| interaction | EventInteraction | optional | Optional metadata identifying the interaction that generated the event. |
EventBillingMetadata
| Field | Type | Label | Description |
|---|---|---|---|
| product | string | Billing product ID associated with the event. | |
| category | string | Billing category ID the event falls into. |
EventFeatureFlags
| Field | Type | Label | Description |
|---|---|---|---|
| flags | EventFeatureFlags.FlagsEntry | repeated | Evaluated feature flags. In Soucegraph we currently only support boolean feature flags, but in the API we allow arbitrary string values for future extensibility. This field should be hydrated by the Sourcegraph server, and not provided by clients. |
EventFeatureFlags.FlagsEntry
EventInteraction
| Field | Type | Label | Description |
|---|---|---|---|
| trace_id | string | optional | OpenTelemetry trace ID representing the interaction associated with the event. |
| interaction_id | string | optional | Custom interaction ID representing the interaction associated with the event. |
| geolocation | EventInteraction.Geolocation | optional | Geolocation associated with the interaction, typically inferred from the originating client's IP address (which we do not collect). |
| parent_interaction_id | string | optional | Parent interaction ID for tracking nested/sub-agent calls. |
| root_interaction_id | string | optional | Root interaction ID identifying the first interaction in the chain. If there is no parent, root_interaction_id equals interaction_id. |
| session_id | string | optional | Session identifier from sourcegraphSessionId cookie, used for session-level event correlation. |
EventInteraction.Geolocation
| Field | Type | Label | Description |
|---|---|---|---|
| country_code | string | Inferred ISO 3166-1 alpha-2 or alpha-3 country code |
EventMarketingTracking
Marketing campaign tracking metadata.
π¨ SECURITY: This metadata is NEVER exported from single-tenant Sourcegraph instances, and is only exported for events tracked in the public Sourcegraph.com instance and managed services.
| Field | Type | Label | Description |
|---|---|---|---|
| url | string | optional | URL the event occurred on. |
| cohort_id | string | optional | Cohort ID to identify the user as part of a specific A/B test. |
| referrer | string | optional | Referrer URL that refers the user to Sourcegraph. |
| last_page_seen_url | string | optional | URL the user last visited, in their current session. |
| most_recent_referrer_url | string | optional | Most recent referrer URL, in their current session |
| first_page_seen_url | string | optional | First URL the user visited, in their current session. |
| utm_campaign | string | optional | UTM campaign tracking parameters, in their current session. |
| utm_content | string | optional | UTM content tracking parameters, in their current session. |
| utm_medium | string | optional | UTM medium tracking parameters, in their current session. |
| utm_source | string | optional | UTM source tracking parameters, in their current session. |
| utm_term | string | optional | UTM term tracking parameters, in their current session. |
EventParameters
| Field | Type | Label | Description |
|---|---|---|---|
| version | int32 | Version of the event parameters, used for indicating the "shape" of this event's metadata, beginning at 0. Useful for denoting if the shape of metadata has changed in any way. | |
| legacy_metadata | EventParameters.LegacyMetadataEntry | repeated | Deprecated. DEPRECATED, legacy metadata format that only accepted int64 - use the new 'metadata' field instead, which accepts float values. Values sent through this proto field will be merged into the new metadata attributes. |
| metadata | EventParameters.MetadataEntry | repeated | Strictly typed metadata, restricted to integer values to avoid accidentally exporting sensitive or private data. |
| private_metadata | google.protobuf.Struct | optional | Additional potentially sensitive metadata - i.e. not restricted to integer values. π¨ SECURITY: This metadata is NOT exported from instances by default, as it can contain arbitrarily-shaped data that may accidentally contain sensitive or private contents. This metadata is only exported on an allowlist basis based on terms of use agreements and combinations of event feature and action, alongside careful audit of callsites. |
| billing_metadata | EventBillingMetadata | optional | Optional billing-related metadata. |
EventParameters.LegacyMetadataEntry
EventParameters.MetadataEntry
EventSource
| Field | Type | Label | Description |
|---|---|---|---|
| server | EventSource.Server | Information about the server that is publishing the event, based on RecordEventsRequestMetadata.Identifier. | |
| client | EventSource.Client | optional | Information about the client that generated the event. |
EventSource.Client
| Field | Type | Label | Description |
|---|---|---|---|
| name | string | Source client of the event. | |
| version | string | optional | Version of the client. |
EventSource.Server
| Field | Type | Label | Description |
|---|---|---|---|
| version | string | Version of the server emitting the event, corresponding to RecordEventsRequestMetadata.Identifier. For example, if the Identifier indicates the publisher is a Sourcegraph instance, the version represents the version of the Sourcegraph server. | |
| request_client | EventSource.Server.RequestClient | optional | Information about the original client that made the request that triggered this serverside event. |
EventSource.Server.RequestClient
| Field | Type | Label | Description |
|---|---|---|---|
| user_agent | string | The user agent of the original client that made the request that triggered this serverside event. | |
| api_client_name | string | The name of the original client that made the request that triggered this serverside event. | |
| api_client_version | string | The version of the original client that made the request that triggered this serverside event. | |
| requested_with | string | The X-Requested-With header from the original client that made the request that triggered this serverside event. | |
| api_client_feature | string | The client-reported feature from the original client that made the request that triggered this serverside event. |
EventUser
EventUser represents a user in the context of an event, and thus must be free of sensitive sensitive metadata and PII about the user.
PublishInstanceUserMetadata is used to allow instances to submit additional metadata about users for instances that consent to it.
| Field | Type | Label | Description |
|---|---|---|---|
| user_id | int64 | optional | Sourcegraph instance database user ID of the user. User IDs are specific to a Sourcegraph instance, and are not universal across Sourcegraph instances. We use an int64 as an ID because in Sourcegraph, database user IDs are always integers. |
| anonymous_user_id | string | optional | Randomized unique identifier representing the user (typically stored in localstorage in web clients, or similar mechanisms elsewhere). This is often used for unauthenticated users, but can persist to authenticated users as well. |
| sams_external_id | string | optional | Sourcegraph Accounts Management System (SAMS) account associated with the user, represented by a SAMS external user ID in a UUID format. This is only valid for services leveraging SAMS as an identity provider - in other words, traditional Sourcegraph instances will not provide this. Learn more about SAMS: https://handbook.sourcegraph.com/departments/engineering/teams/core-services/sams |
Identifier
| Field | Type | Label | Description |
|---|---|---|---|
| licensed_instance | Identifier.LicensedInstanceIdentifier | A licensed Sourcegraph instance. | |
| unlicensed_instance | Identifier.UnlicensedInstanceIdentifier | An unlicensed Sourcegraph instance. | |
| managed_service | Identifier.ManagedServiceIdentifier | A service operated and managed by the Sourcegraph team, for example a service deployed by MSP: https://handbook.sourcegraph.com/departments/engineering/teams/core-services/managed-services/platform/ Valid SAMS client credentials are required to publish events under a managed service identifier. The required scope is 'telemetry_gateway::events::publish'. See go/sams-client-credentials and go/sams-token-scopes for more information. | |
| workspace | Identifier.WorkspaceIdentifier | A workspace within a tenant host. Valid SAMS client credentials are required to publish events under a workspace identifier. The required scope is 'telemetry_gateway::events::publish'. See go/sams-client-credentials and go/sams-token-scopes for more information. |
Identifier.LicensedInstanceIdentifier
| Field | Type | Label | Description |
|---|---|---|---|
| license_key | string | License key configured in the Sourcegraph instance emitting the event. | |
| instance_id | string | Self-reported Sourcegraph instance identifier. | |
| external_url | string | Instance external URL defined in the instance site configuration. |
Identifier.ManagedServiceIdentifier
| Field | Type | Label | Description |
|---|---|---|---|
| service_id | string | Self-reported service identifier, for example 'my-service'. | |
| service_environment | string | optional | Self-reported service environment, for example 'prod' or 'dev'. |
Identifier.UnlicensedInstanceIdentifier
| Field | Type | Label | Description |
|---|---|---|---|
| instance_id | string | Self-reported Sourcegraph instance identifier. | |
| external_url | string | Instance external URL defined in the instance site configuration. |
Identifier.WorkspaceIdentifier
| Field | Type | Label | Description |
|---|---|---|---|
| workspace_id | string | Self-reported Sourcegraph workspace identifier, for example 'ws_$UUID'. | |
| external_url | string | The workspace's assigned external URL. |
InstanceOrganizationMetadata
π¨ SECURITY: This metadata is NOT exported from instances by default.
| Field | Type | Label | Description |
|---|---|---|---|
| id | int64 | Internal ID of the organization from the 'orgs' table. | |
| name | string | Name of the organization. | |
| display_name | string | Display name of the organization. | |
| user_ids | int64 | repeated | The IDs of users that are members of this organization. We allow a single repeated set here as the repeated ID-only format should be fairly compact for realistic scenarios. |
InstanceUserMetadata
π¨ SECURITY: This metadata is NOT exported from instances by default.
| Field | Type | Label | Description |
|---|---|---|---|
| id | int64 | Internal ID of the user from the 'users' table. | |
| username | string | Username of the user. | |
| primary_email | string | Primary email of the user. | |
| assigned_rbac_roles | string | repeated | Assigned RBAC roles for the user. |
PublishInstanceUserMetadataRequest
π¨ SECURITY: This metadata is NOT exported from instances by default.
| Field | Type | Label | Description |
|---|---|---|---|
| metadata | PublishInstanceUserMetadataRequestMetadata | Metadata about the events being recorded. | |
| user_metadata | PublishInstanceUserMetadataRequest.UserMetadataPayload | Batch of user metadata to publish in a single message. Clients should aim to batch large uploads into a series of smaller requests in the RecordEvents stream, being mindful of common limits in individual message sizes: https://protobuf.dev/programming-guides/api/#bound-req-res-sizes π¨ SECURITY: Callers exporting for single-tenant Sourcegraph should always respect in-instance toggles for what user metadata should be exported. | |
| organization_metadata | PublishInstanceUserMetadataRequest.OrganizationMetadataPayload | Batch of organization metadata to publish in a single message. Clients should aim to batch large uploads into a series of smaller requests in the RecordEvents stream, being mindful of common limits in individual message sizes: https://protobuf.dev/programming-guides/api/#bound-req-res-sizes π¨ SECURITY: Callers exporting for single-tenant Sourcegraph should always respect in-instance toggles for what user metadata should be exported. |
PublishInstanceUserMetadataRequest.OrganizationMetadataPayload
π¨ SECURITY: This metadata is NOT exported from instances by default.
| Field | Type | Label | Description |
|---|---|---|---|
| organizations | InstanceOrganizationMetadata | repeated | Organization metadata from the 'orgs' table in a Sourcegraph instance. |
PublishInstanceUserMetadataRequest.UserMetadataPayload
π¨ SECURITY: This metadata is NOT exported from instances by default.
| Field | Type | Label | Description |
|---|---|---|---|
| users | InstanceUserMetadata | repeated | User metadata from the 'users' table in a Sourcegraph instance. |
PublishInstanceUserMetadataRequestMetadata
| Field | Type | Label | Description |
|---|---|---|---|
| request_id | string | Client-provided request identifier for diagnostics purposes. | |
| identifier | Identifier | Telemetry publisher self-identification. Only LicensedInstanceIdentifier and UnlicensedInstanceIdentifier are accepted for this RPC. |
PublishInstanceUserMetadataResponse
RecordEventRequest
| Field | Type | Label | Description |
|---|---|---|---|
| metadata | RecordEventsRequestMetadata | Metadata about the events being recorded. | |
| event | Event | Event to record. |
RecordEventResponse
RecordEventsRequest
| Field | Type | Label | Description |
|---|---|---|---|
| metadata | RecordEventsRequestMetadata | Metadata about the events being recorded. | |
| events | RecordEventsRequest.EventsPayload | Batch of events to record in a single request. Clients should aim to batch large event backlogs into a series of smaller requests in the RecordEvents stream, being mindful of common limits in individual message sizes: https://protobuf.dev/programming-guides/api/#bound-req-res-sizes |
RecordEventsRequest.EventsPayload
| Field | Type | Label | Description |
|---|---|---|---|
| events | Event | repeated |
RecordEventsRequestMetadata
| Field | Type | Label | Description |
|---|---|---|---|
| request_id | string | Client-provided request identifier for diagnostics purposes. | |
| identifier | Identifier | Telemetry publisher self-identification - for example, a Sourcegraph instance of some other kind of service. |
RecordEventsResponse
| Field | Type | Label | Description |
|---|---|---|---|
| succeeded_events | string | repeated | IDs of all events that were successfully recorded in the request. Note that if succeeded_events is a subset of events that were submitted, then some events failed to record and should be retried. |
TelemeteryGatewayService
TelemetryGatewayService accepts telemetry and related data from Sourcegraph instances and managed services.
| Method Name | Request Type | Response Type | Description |
|---|---|---|---|
| RecordEvents | RecordEventsRequest stream | RecordEventsResponse stream | RecordEvents streams telemetry events in batches to the Telemetry Gateway service. Events should only be considered delivered if recording is acknowledged in RecordEventsResponse. This is the preferred mechanism for exporting large volumes of events in bulk. π¨ SECURITY: Callers exporting for single-tenant Sourcegraph should check the attributes of the Event type to ensure that only the appropriate fields are exported, as some fields should only be exported on an allowlist basis. |
| RecordEvent | RecordEventRequest | RecordEventResponse | RecordEvent records a single telemetry event to the Telemetry Gateway service. If the RPC succeeds, then the event was successfully published. This RPC currently ONLY accepts events published by ManagedServiceIdentifier, as this mechanism is intended for low-volume managed services. Higher-volume use cases should implement a batching mechanism and use the RecordEvents RPC instead. π¨ SECURITY: Callers exporting for single-tenant Sourcegraph should check the attributes of the Event type to ensure that only the appropriate fields are exported, as some fields should only be exported on an allowlist basis. |
| PublishInstanceUserMetadata | PublishInstanceUserMetadataRequest stream | PublishInstanceUserMetadataResponse | PublishInstanceUserMetadata uploads additional user metadata from a Sourcegraph instance. This matadata is used to enrich recorded events with attributes that are considered sensitive and thus not part of the RecordEvents and RecordEvent RPCs. π¨ SECURITY: Callers exporting for single-tenant Sourcegraph should always respect in-instance toggles for what user metadata should be exported. |