Sourcegraph 5.9.0

OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c

Cody Gateway: add new claude-3-5-haiku-latest & claude-3-5-opus-latest models to Cody Gateway allow list Cody Gateway: add new claude-3-5-haiku-latest dotcom models list
Backport 11e7481ba3c810ae5b47d32b8cf32066e2f0b2bb from #1470

OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c

• New OpenAPI API docs within the instance at the URL $SRC_ENDPOINT/api/openapi/public. Alternatively, reach the page at "User > Settings > OpenAPI Reference".

• Adds cost estimates to the ModelConfiguration returned by .api/llm/supported-models.json
• Adds llmTokenUsageCostEstimate telemetry value with estimates of each requests LLM cost in pennies

OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c

Long-term local retention of user telemetry as 'event logs' can now be disabled entirely via the telemetry: { disableLocalEventLogs } site configuration.

• Added embeddable page for file snippets

OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c

OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c

The sub repository permissions implementation now emits logs that describe the rules that were evaluated for a given request whenever tracing is enabled. (We limit these logs for only when tracing is enabled since they can be quite verbose).

• Add the sourcegraph API client user-agent to telemetry payload types under source.server.server_client.user_agent.

• The /.api/completions/stream API now includes used input/output tokens and the upstream model name in the response body when using the query parameter api-version=5 or higher. Does not apply to: Vertex Anthropic, Azure OpenAI (mo model name, and no token usage in streaming response), Gemini (no model name in streaming response)
• LLM chat completions now support stream: false when using Azure OpenAI and OpenAI-compatible providers.

• Propagates new X-Sourcegraph-API-Client-Name and X-Sourcegraph-API-Client-Version headers to request clients

• add library that writes to TEST_INFRASTRUCTURE_FAILURE_FILE if it is defined for test failures

feat(batches): show name of additional fields that cause errors

• The raw HTTP API now accepts the query paramter format=cody or header Accept: application/cody to render files and directories as context items for Cody. Example request path: /github.com/sourcegraph/cody/-/raw/agent/src/index.ts?format=cody.

fix(batches): enable request splitting by default

The documentation for gitserver's ReadDir method has been clarified to reflect that the return iterator's Name() method returns the full path of the file, as opposed to just the basename.

OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c

OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c

Fixes a bug where logs and traces were sometimes missing key-value pairs recorded alongside errors.

The documentation for the "exclude" section in the github code host connection schema has been clarified to explain that each block is OR'd together, and the expressions within each block are AND'd together.

• When there are multiple auth providers configured that point to the same GitHub URL, as can be the case when using private GitHub Apps, user external account tokens will now refresh correctly.

Prevents default browser shortcuts from modifying the file content locally.

fix(batches): handle spec being nil for suffix salt

Introduces new limits for auto-indexing inference to reduce the risk of continuously growing auto-indexing queues.

• The number of jobs spawned per round of auto-indexing inference per repo is capped (default: 100) to reduce risk of clogging of auto-indexing queues. Excess jobs will be discarded.
• The number of paths inspected for a single round of auto-indexing inference per repo is capped (default: 500) to reduce risk of timeouts. Excess paths will be discarded.

Fixes handling of SCIP uploads when the hash passed to the -commit flag of src-cli corresponds to the hash of an annotated tag instead of the hash of a commit. Previously, these uploads were not accessible for code navigation.

• GitHub code host connections using repositoryQuery will no longer delete repositories if the sync fails for reasons like a GitHub outage or a token expiring.

fix(batches): enable GQL request splitting via BATCH_CHANGES_REQUEST_SPLITTING

Added token usage information to .api/llm/chat/completions

Fixes a bug where hovers were not triggered for less mainstream languages like F#, Standard ML, Visual Basic, Pkl, Hack, MATLAB etc.

This fixes a bug where we would apply a phrase boost if a content: filter was specified with keyword search enabled. This led to inconsistent behavior (regexp vs keyword search) and it also rendered the content: filter ineffective, because we ran a general text search instead of a just a content search.

fix(code-monitor): fail monitors with missing/unverified primary email address

NA

fix(batches): display "Deleted Namespace" instead of throwing an error

Fixes a bug where the retention tab for precise indexes would show a nil pointer exception instead of retention information.

n/a

Cody Chat: the context engine now uses the reranker by default everywhere where Cody Gateway is enabled.

• adding code llama 7B for completions

OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c

A bug in configuration validation in the worker service has been fixed. Configuration mistakes are now correctly reported again.

• The prompt creation/edit page has been redesigned.

• Adds ability to mark prompts as recommended and then sort the recommended prompts at the top.

Added LLM token usage telemetry

• The environment variable KUBERNETES_SINGLE_JOB_STEP_IMAGE is now KUBERNETES_JOB_STEP_IMAGE.
• Both environment variables are read, with preference given to KUBERNETES_JOB_STEP_IMAGE, to preserve backward compatibility, but reading of KUBERNETES_SINGLE_JOB_STEP_IMAGE may be removed in a future version.

• Fixed a bug that would cause filters added from the search sidebar to not be correctly escaped

• Adds mode (CHAT, EDIT, INSERT) to prompt.

Added a new internal/tracelog package that provides a conditional logger for tracing. This logger only logs when tracing is enabled, improving performance by reducing unnecessary logging.

For the Perforce IP permissions implementation, this PR introduces machinery that implements the ability to parse the IP address expressions from the perforce protections table and compare them against the user's IP address (that's advertised from the incoming X-FORWARDED-FOR header.

Namely, this PR:

• Introduced parsePerforceIPString, which parses Perforce IP strings (e.g., CIDR expressions, specific IP addresses) into an ipMatcher for IP validation.

• Implemented multiple ipMatcher types, including:

  • addrMatcher for exact IP matches.
  • cidrMatcher for checking IPs within a CIDR range.
  • alwaysTrueMatcher for wildcard * expressions.

• Added toggleableIPMatcher, which either enables or disables IP enforcement based on site configuration settings.

• Created matcherCache for caching recent IP string to ipMatcher translations, improving efficiency.

• Updated NewRequestClientIPSource to properly derive IP from the x-forwarded-for header.

Added a new ipParseCacheSize configuration option for sub-repo permissions to control the caching of Perforce "Host" to IP address translations.

Added enforceIPRestrictions setting to the SubRepoPermissions section in site configuration, allowing IP-based enforcement using the X-FORWARDED-FOR header.

Updated JSON schema to require SubRepoPermissions to be enabled when enforceIPRestrictions is enabled.

Added validation in Perforce external service configuration to ensure IgnoreRulesWithHost and enforceIPRestrictions cannot be enabled simultaneously.

Updated internal implementation to associate each Perforce rule with an IPMatcher for IP-based rule enforcement.

This PR introduces logic for IP-based permissions in the authz provider for Perforce: 1. Modified interfaces in internal/authz/sub_repo_perms.go to accept caller's IP address for enforcing IP permissions. 2. Introduced a new IPSource interface to provide IP addresses for sub-repo permissions implementation. 3. Updated rule implementation to include an IPMatcher for comparing provided IP addresses with Perforce rule IP addresses.

This change lays the groundwork for supporting IP-based permissions in Perforce authz provider.

• Adds options to set prompts to "Auto Submit", which would automatically execute the prompts in one-click.

• Users can now duplicate existing prompts.

• Build containers using latest version of apko and rules_apko

• Fixes a bug where certain copy path buttons were only copying the visible path and not the full path of a file.