GitLab

Site admins can sync Git repositories hosted on GitLab (GitLab.com and GitLab CE/EE) with Sourcegraph so that users can search and navigate the repositories.

To connect GitLab to Sourcegraph:

1. Go to Site admin > Manage code hosts > Add code host
2. Select GitLab (for GitLab.com) or GitLab Self-Managed.
3. Set url to the URL of your GitLab instance, such as https://gitlab.example.com or https://gitlab.com (for GitLab.com).
4. Create a GitLab access token using these instructions with the read_api and read_repository scopes, and set it to be the value of the token.
5. Use the Repository syncing documentation below to select and add your preferred projects/repos to the configuration.
6. You can use the action buttons above the text field to add the fields, and additional fields can be added using Cmd/Ctrl+Space for auto-completion. See the configuration documentation below for additional fields.
7. Click Add repositories.

Example config:

SHELL
{
  "url": "https://gitlab.com",
  "token": "<access token>",
  "projectQuery": [
    "groups/mygroup/projects",
    "projects?membership=true&archived=no",
    "?search=<search query>",
    "?membership=true\u0026search=foo"
 ],
 "projects": [
  {
     "name": "group/name"
  },
  {
      "id": 42
  }
 ]
}

Supported versions

• GitLab.com
• GitLab CE/EE (v12.0 and newer)

Repository syncing

There are three fields for configuring which projects are mirrored/synchronized:

• projects: A list of projects in {"name": "group/name"} or {"id": id} format. The order determines the order in which we sync project metadata and is safe to change.
• projectQuery: A list of strings. Accepted values include:
  • The special value none, which will sync no repositories.
  • Query parameters for the GitLab Projects API endpoint. For example, ?archived=false.
  • A path and set of query parameters for any GitLab API endpoint that returns a list of repos. For example, groups/mygroup/projects?visibility=public.
• exclude: A list of projects to exclude which takes precedence over the projects, and projectQuery fields. It has the same format as projects.

Troubleshooting

You can test your access token's permissions by running a cURL command against the GitLab API. This is the same API and the same project list used by Sourcegraph.

Replace $ACCESS_TOKEN with the access token you are providing to Sourcegraph, and $GITLAB_HOSTNAME with your GitLab hostname:

SHELL
curl -H 'Private-Token: $ACCESS_TOKEN' -XGET 'https://$GITLAB_HOSTNAME/api/v4/projects'

Repository permissions

GitLab permissions can be configured in three ways:

1. Set up GitLab as an OAuth sign-on provider for Sourcegraph (recommended)
2. Use a GitLab administrator (sudo-level) personal access token in conjunction with another SSO provider (recommended only if the first option is not possible)
3. Assume username equivalency between Sourcegraph and GitLab (warning: this is generally unsafe and should only be used if you are using strictly http-header authentication).

NOTE: It can take some time to complete full cycle of repository permissions sync if you have a large number of users or repositories. See sync duration time for more information.

OAuth application

Prerequisite: Add GitLab as an authentication provider.

Then, add or edit a GitLab connection and include the authorization field:

JSON
{
  "url": "https://gitlab.com",
  "token": "$PERSONAL_ACCESS_TOKEN",
  // ...
  "authorization": {
    "identityProvider": {
      "type": "oauth"
    }
  }
}

In this case, a user's OAuth token will be used to get a list of repositories that the user can access. Repository-centric permissions syncing will be disabled.

Administrator (sudo-level) access token

This method requires administrator access to GitLab so that Sourcegraph can access the admin GitLab Users API endpoint. For each GitLab user, this endpoint provides the user ID that comes from the authentication provider, so Sourcegraph can associate a user in its system to a user in GitLab.

Prerequisite: Add the SAML or OpenID Connect authentication provider you use to sign into GitLab.

Then, add or edit a GitLab connection using an administrator (sudo-level) personal access token, and include the authorization field:

JSON
{
  "url": "https://gitlab.com",
  "token": "$PERSONAL_ACCESS_TOKEN",
  // ...
  "authorization": {
    "identityProvider": {
      "type": "external",
      "authProviderID": "$AUTH_PROVIDER_ID",
      "authProviderType": "$AUTH_PROVIDER_TYPE",
      "gitlabProvider": "$AUTH_PROVIDER_GITLAB_ID"
    }
  }
}

$AUTH_PROVIDER_ID and $AUTH_PROVIDER_TYPE identify the authentication provider to use and should match the fields specified in the authentication provider config (auth.providers). The authProviderID can be found in the configID field of the auth provider config.

$AUTH_PROVIDER_GITLAB_ID should match the identities.provider returned by the admin GitLab Users API endpoint.

Username

Prerequisite: Ensure that http-header is the only authentication provider type configured for Sourcegraph. If this is not the case, then it will be possible for users to escalate privileges, because Sourcegraph usernames are mutable.

Add or edit a GitLab connection and include the authorization field:

JSON
{
  "url": "https://gitlab.com",
  "token": "$PERSONAL_ACCESS_TOKEN",
  // ...
  "authorization": {
    "identityProvider": {
      "type": "username"
    }
  }
}

User authentication

To configure GitLab as an authentication provider (which will enable sign-in via GitLab), see the authentication documentation.

Internal repositories

GitLab also has internal repositories in addition to the usual public and private repositories. Depending on how your organization structure is configured, you may want to make these internal repositories available to everyone on your Sourcegraph instance without relying on permission syncs. To mark all internal repositories as public, add the following field to the code host connection:

JSON
{
  // ...
  "markInternalReposAsPublic": true
}

When adding this configuration option, you may also want to configure your GitLab auth provider so that it does not sync user permissions for internal repositories.

Rate limits

Always include a token in a configuration for a GitLab.com URL to avoid being denied service by GitLab's unauthenticated rate limits.

When Sourcegraph hits a rate limit imposed by GitLab, Sourcegraph waits the appropriate amount of time specified by GitLab before retrying the request. This can be several minutes in extreme cases.

Internal rate limits

See Internal rate limits.

Configuration

admin/code_hosts/gitlab.schema.json

JSON
{
	// If non-null, enforces GitLab repository permissions. This requires that there be an item in the `auth.providers` field of type "gitlab" with the same `url` field as specified in this `GitLabConnection`.
	"authorization": null,
 
	// TLS certificate of the GitLab instance. This is only necessary if the certificate is self-signed or signed by an internal CA. To get the certificate run `openssl s_client -connect HOST:443 -showcerts < /dev/null 2> /dev/null | openssl x509 -outform PEM`. To escape the value into a JSON string, you may want to use a tool like https://json-escape-text.now.sh.
	"certificate": null,
	// Other example values:
	// - "-----BEGIN CERTIFICATE-----\n..."
 
	// Only used to override the cloud_default column from a config file specified by EXTSVC_CONFIG_FILE
	"cloudDefault": false,
 
	// When set to true, this external service will be chosen as our 'Global' GitLab service. Only valid on Sourcegraph.com. Only one service can have this flag set.
	"cloudGlobal": false,
 
	// A list of projects to never mirror from this GitLab instance. Takes precedence over \"projects\" and \"projectQuery\" configuration. You can exclude projects by: name ({"name": "group/name"}), ID ({"id": 42}), regular expression matching pattern ({"pattern": "^group/project-.*"}), or by excluding empty repositories ({"emptyRepos": true}).
	"exclude": null,
	// Other example values:
	// - [
	//     {
	//       "name": "group/name"
	//     },
	//     {
	//       "id": 42
	//     },
	//     {
	//       "emptyRepos": true
	//     }
	//   ]
	// - [
	//     {
	//       "name": "gitlab-org/gitlab-ee"
	//     },
	//     {
	//       "name": "gitlab-com/www-gitlab-com"
	//     }
	//   ]
 
	// The type of Git URLs to use for cloning and fetching Git repositories on this GitLab instance.
	//
	// If "http", Sourcegraph will access GitLab repositories using Git URLs of the form http(s)://gitlab.example.com/myteam/myproject.git (using https: if the GitLab instance uses HTTPS).
	//
	// If "ssh", Sourcegraph will access GitLab repositories using Git URLs of the form [email protected]:myteam/myproject.git. See the documentation for how to provide SSH private keys and known_hosts: https://sourcegraph.com/docs/admin/repo/auth#repositories-that-need-http-s-or-ssh-authentication.
	"gitURLType": "http",
 
	// Deprecated and ignored field which will be removed entirely in the next release. GitLab repositories can no longer be enabled or disabled explicitly.
	"initialRepositoryEnablement": null,
 
	// If true, internal repositories will be accessible to all users on Sourcegraph as if they were public, and user permission syncs will no longer check for public repositories. This overrides repository permissions but allows easier discovery and access to internal repositories, and may be desirable if all users on the Sourcegraph instance should have access to all internal repositories anyways. Defaults to false.
	"markInternalReposAsPublic": false,
 
	// An array of transformations will apply to the repository name. Currently, only regex replacement is supported. All transformations happen after "repositoryPathPattern" is processed.
	"nameTransformations": null,
	// Other example values:
	// - [
	//     {
	//       "regex": "\\.d/",
	//       "replacement": "/"
	//     },
	//     {
	//       "regex": "-git$",
	//       "replacement": ""
	//     }
	//   ]
 
	// An array of strings specifying which GitLab projects to mirror on Sourcegraph. Each string is a URL path and query that targets a GitLab API endpoint returning a list of projects. If the string only contains a query, then "projects" is used as the path. Examples: "?membership=true&search=foo", "groups/mygroup/projects".
	//
	// The special string "none" can be used as the only element to disable this feature. Projects matched by multiple query strings are only imported once. Here are a few endpoints that return a list of projects: https://docs.gitlab.com/ee/api/projects.html#list-all-projects, https://docs.gitlab.com/ee/api/groups.html#list-a-groups-projects, https://docs.gitlab.com/ee/api/search.html#scope-projects.
	"projectQuery": [
		"none"
	],
	// Other example values:
	// - [
	//     "?membership=true\u0026search=foo",
	//     "groups/mygroup/projects"
	//   ]
 
	// A list of projects to mirror from this GitLab instance. Supports including by name ({"name": "group/name"}) or by ID ({"id": 42}).
	"projects": null,
	// Other example values:
	// - [
	//     {
	//       "name": "group/name"
	//     },
	//     {
	//       "id": 42
	//     }
	//   ]
	// - [
	//     {
	//       "name": "gnachman/iterm2"
	//     },
	//     {
	//       "name": "gitlab-org/gitlab-ce"
	//     }
	//   ]
 
	// Rate limit applied when making background API requests to GitLab.
	"rateLimit": {
		"enabled": true,
		"requestsPerHour": 36000
	},
 
	// The pattern used to generate a the corresponding Sourcegraph repository name for a GitLab project. In the pattern, the variable "{host}" is replaced with the GitLab URL's host (such as gitlab.example.com), and "{pathWithNamespace}" is replaced with the GitLab project's "namespace/path" (such as "myteam/myproject").
	//
	// For example, if your GitLab is https://gitlab.example.com and your Sourcegraph is https://src.example.com, then a repositoryPathPattern of "{host}/{pathWithNamespace}" would mean that a GitLab project at https://gitlab.example.com/myteam/myproject is available on Sourcegraph at https://src.example.com/gitlab.example.com/myteam/myproject.
	//
	// It is important that the Sourcegraph repository name generated with this pattern be unique to this code host. If different code hosts generate repository names that collide, Sourcegraph's behavior is undefined.
	"repositoryPathPattern": "{host}/{pathWithNamespace}",
 
	// A GitLab access token with "api" scope. Can be a personal access token (PAT) or an OAuth token. If you are enabling permissions with identity provider type "external", this token should also have "sudo" scope.
	"token": null,
 
	// The OAuth token expiry (Unix timestamp in seconds)
	"token.oauth.expiry": null,
 
	// The OAuth refresh token
	"token.oauth.refresh": null,
 
	// The type of the token
	"token.type": "pat",
 
	// URL of a GitLab instance, such as https://gitlab.example.com or (for GitLab.com) https://gitlab.com.
	"url": null,
	// Other example values:
	// - "https://gitlab.com"
	// - "https://gitlab.example.com"
 
	// An array of webhook configurations
	"webhooks": null
}

Native integration

To provide out-of-the-box code navigation features to your users on GitLab, you will need to configure your GitLab instance. If you are using an HTTPS connection to GitLab, you will need to configure HTTPS for your Sourcegraph instance.

The Sourcegraph instance's site admin must update the corsOrigin site config property to allow the GitLab instance to communicate with the Sourcegraph instance. For example:

JSON
{
  // ...
  "corsOrigin":
    "https://my-gitlab.example.com"
  // ...
}

Configuration Notes

GitLab connections provide flexible configuration options for different deployment scenarios:

• Project Selection: Use projects for specific repositories, projectQuery for API-based filtering, and exclude for fine-grained exclusions
• API Queries: The projectQuery field supports GitLab API parameters like ?membership=true&archived=false and group paths like groups/mygroup/projects
• Path Patterns: The repositoryPathPattern field controls repository naming in Sourcegraph - ensure uniqueness across code hosts
• Internal Repositories: GitLab's internal repository visibility can be handled via markInternalReposAsPublic for easier access

Rate Limiting Configuration

GitLab connections include rate limiting to manage API usage:

• Default: 36,000 requests per hour with rate limiting enabled
• GitLab.com enforces stricter rate limits for unauthenticated requests
• Self-managed instances may have different limits based on configuration

Token Configuration

The token field supports different token types:

• Personal Access Token (PAT): Recommended for most use cases with api and read_repository scopes
• OAuth Token: Used with OAuth authentication flows, includes refresh token support
• Project Access Token: GitLab 13.10+ feature for project-specific access

Security Considerations

Token Security and Scoping

Personal Access Tokens: Grant broad access equivalent to the user's permissions:

• Use dedicated service accounts rather than personal tokens when possible
• Required scopes: api for repository discovery, read_repository for cloning
• Add sudo scope only when using external identity providers for permission syncing
• Regularly rotate tokens and audit their usage

OAuth Tokens: Provide more secure, time-limited access:

• Tokens automatically refresh when expired
• Support for fine-grained permissions
• Better audit trail through OAuth application logs

Certificate Validation

For self-managed GitLab instances with custom certificates:

• Use the certificate field to specify custom CA certificates
• Obtain certificates using: openssl s_client -connect HOST:443 -showcerts
• Ensure certificate chain is complete for proper validation

Repository Permissions

GitLab permission syncing has specific security requirements:

• OAuth Method: Most secure, uses user's OAuth token for permission checks
• External Identity: Requires admin token with sudo scope, matches users via external auth providers
• Username Method: Least secure, assumes username equivalency - only use with http-header auth

Internal Repository Access

GitLab's internal repositories require careful configuration:

• Setting markInternalReposAsPublic: true makes internal repos visible to all Sourcegraph users
• Consider organizational policies before enabling this setting
• Alternative: Use proper permission syncing to maintain GitLab's access controls

Common Examples

Basic Project Sync

JSON
{
  "url": "https://gitlab.com",
  "token": "glpat-xxxxxxxxxxxxxxxxxxxx",
  "projects": [
    {"name": "gitlab-org/gitlab"},
    {"name": "my-group/my-project"}
  ]
}

Organization-wide Sync with Exclusions

JSON
{
  "url": "https://gitlab.example.com",
  "token": "glpat-xxxxxxxxxxxxxxxxxxxx",
  "projectQuery": [
    "groups/engineering/projects?archived=false",
    "?membership=true&visibility=private"
  ],
  "exclude": [
    {"name": "engineering/deprecated-project"},
    {"pattern": "^experimental/.*"}
  ]
}

Self-managed GitLab with Authorization

JSON
{
  "url": "https://gitlab.company.com",
  "token": "glpat-xxxxxxxxxxxxxxxxxxxx",
  "projectQuery": ["?membership=true"],
  "authorization": {
    "identityProvider": {
      "type": "oauth"
    }
  },
  "certificate": "-----BEGIN CERTIFICATE-----\n..."
}

Advanced Query with Custom Path Pattern

JSON
{
  "url": "https://gitlab.com",
  "token": "glpat-xxxxxxxxxxxxxxxxxxxx",
  "projectQuery": [
    "groups/kubernetes/projects?visibility=public&order_by=last_activity_at",
    "?search=docker&sort=desc"
  ],
  "repositoryPathPattern": "gitlab/{pathWithNamespace}",
  "markInternalReposAsPublic": true
}

High-volume Configuration

JSON
{
  "url": "https://gitlab.enterprise.com",
  "token": "glpat-xxxxxxxxxxxxxxxxxxxx",
  "projectQuery": ["groups/bigorg/projects"],
  "rateLimit": {
    "enabled": true,
    "requestsPerHour": 20000
  },
  "exclude": [
    {"emptyRepos": true}
  ]
}

Best Practices

Token Management

1. Use dedicated service accounts instead of personal tokens for production deployments
2. Implement token rotation policies to regularly refresh access tokens
3. Audit token permissions regularly to ensure least-privilege access
4. Use OAuth tokens when possible for better security and automatic refresh

Repository Organization

1. Use consistent naming patterns with repositoryPathPattern to maintain organization
2. Leverage project queries for dynamic repository discovery rather than static lists
3. Implement exclusion patterns to filter out unwanted repositories (archived, empty, experimental)
4. Group related projects using GitLab groups and subgroups for better organization

Performance Optimization

1. Configure appropriate rate limits based on your GitLab instance's capacity
2. Use specific project queries rather than broad searches to reduce API calls
3. Monitor API usage to avoid hitting GitLab's rate limits
4. Enable proper caching by configuring reasonable sync intervals

Permission Management

1. Choose the right permission method for your authentication setup
2. Test permission syncing thoroughly before deploying to production
3. Use OAuth authentication when possible for better security and user experience
4. Regularly audit repository access to ensure permissions remain appropriate

Maintenance

1. Monitor connection health through the site admin interface
2. Keep certificates updated for self-managed GitLab instances
3. Review and update project queries as your repository landscape changes
4. Test configuration changes in a staging environment when possible

Access token scopes

Sourcegraph requires an access token with api permissions (and sudo, if you are using an external identity provider type). These permissions are required for the following reasons:

We are actively collaborating with GitLab to improve our integration (e.g. the Sourcegraph GitLab native integration and better APIs for querying repository permissions).

Webhooks

Using the webhooks property on the external service has been deprecated.

Please consult this page in order to configure webhooks.