Configuring Maven to Use a Private Repository for Auto-Indexing

Configuring Maven to use a private repository, such as Nexus or Artifactory, with Sourcegraph is essential when dependencies are retrieved from private repositories that require authentication (as opposed to public repositories such as Maven Central). This guide covers steps to set up Maven for scip-java indexing with a private repository, ensuring secure and consistent dependency resolution.

Testing the Configuration on a Single Repository

To test and validate the Maven configuration, modify a single repository’s auto-indexing settings to include a custom settings.xml file. Refer to (Maven's official docs)[https://maven.apache.org/settings.html#quick-overview] for an overview of how this file is used to configure Maven repositories and other settings.

Add Custom Index Job Configuration

• Access the repository’s index settings in Sourcegraph and open the “Raw” configuration panel.

• Insert the following configuration:

  JSON
  {
    "steps": [],
    "local_steps": [
      "mkdir -p ~/.m2",
      "echo '<settings xmlns=\"http://maven.apache.org/SETTINGS/1.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd\"> <servers> <server> <id>repo</id> <username>$ARTIFACTORY_USER</username> <password>$ARTIFACTORY_PASSWORD</password> </server> </servers> </settings>' > ~/.m2/settings.xml"
    ],
    "root": "",
    "indexer": "sourcegraph/scip-java:latest",
    "indexer_args": [
      "scip-java",
      "index",
      "--build-tool=auto"
    ],
    "outfile": "index.scip",
    "requestedEnvVars": [
      "ARTIFACTORY_USER",
      "ARTIFACTORY_PASSWORD"
    ]
  }

Set Up Executor Secrets

Before triggering the indexing process, make sure the environment variables $ARTIFACTORY_USER and $ARTIFACTORY_PASSWORD are created as Executor Secrets.

Trigger Indexing

After configuring the repository, navigate to the "Precise Indexes" tab and click "Enqueue" to start indexing.

Automating the Configuration Across All Repositories

After verifying the configuration on a single repository, you can automate the setup across all repositories by modifying the inference configuration using a Lua script.

Create or Update the Lua Script

Navigate to site-admin -> Code Graph -> Inference, and replace or add the following Lua script for scip-java indexing with Maven’s settings.xml setup:

LUA
local path = require("path")
local pattern = require("sg.autoindex.patterns")
local recognizer = require("sg.autoindex.recognizer")
local patterns = require "internal_patterns"
 
local new_steps = {
    'mkdir -p ~/.m2',
    [[echo "<?xml version=\"1.0\"?><settings xmlns=\"http://maven.apache.org/SETTINGS/1.0.0\"><servers><server><id>repo</id><username>$ARTIFACTORY_USER</username><password>$ARTIFACTORY_PASSWORD</password></server></servers></settings>" > ~/.m2/settings.xml]]
}
 
local new_requested_env_variables = {'ARTIFACTORY_USER', 'ARTIFACTORY_PASSWORD'}
local java_indexer = require("sg.autoindex.indexes").get "java"
 
local custom_java_recognizer = recognizer.new_path_recognizer {
    patterns = {
        pattern.new_path_basename("pom.xml"),
        pattern.new_path_basename("build.gradle"),
        pattern.new_path_basename("build.gradle.kts"),
 },
    generate = function(api, paths)
        api:register({
            local_steps = new_steps,
            requested_envvars = new_requested_env_variables,
            root = path.dirname(paths[1]),
            outfile = "index.scip",
            indexer = java_indexer,
            indexer_args = { "scip-java", "index", "--build-tool=auto" }
 })
    end
}
 
return require("sg.autoindex.config").new({
 ["custom.java"] = custom_java_recognizer,
 ["sg.java"] = false
})

Verify the Configuration

Once the Lua script is applied, you can verify that the configuration works by using the "Preview results" button in the Lua script editor under the "Inference Configuration" section. This will display the inferred index jobs for your repositories, showing details such as the root directory, indexer, indexer arguments, and environment variables used for each job.

If everything is configured correctly, the dependencies will be pulled from the specified private repository without issues.