Sourcegraph 3.16: Search past releases, get started with campaigns, and enjoy syntax highlighting improvements

Crhstina Forney

May 20, 2020

Explore, navigate, and better understand all code, everywhere, faster with Sourcegraph Universal Code Search. Uber, Lyft, Yelp, and other enterprises rely on Sourcegraph to improve developer productivity and manage large-scale code changes.

📌 Introducing version contexts to search past releases

📣 Smoother getting started experience for campaigns

🎨 Best-in-class syntax highlighting

👀 Easier to read queries in plain text mode

🧠 Performance improvements for precise code intelligence

🕵️‍♀️ Discover useful scopes with repogroup autocompletion

🔑 Check syncing status of repository and user permissions

📊 Understanding user activity of your instance

ⓘ Improvements to basic code intelligence for several languages

📝 Changelog Every detail that changed in this release

🎖️ Thank you Sourcegraph couldn't be what it is without the community.

Deploy or upgrade: Local | AWS | DigitalOcean | Kubernetes cluster

Go in-depth with the 3.16 release by watching the recorded livestream with Product Manager Christina Forney and Software Engineers Thorsten Ball, Loïc Guychard, and Eric Fritz.

Introducing version contexts to search past releases

Many organizations have old releases running in production and need to search across all the code for a release. Sourcegraph 3.16 introduces version contexts as an experimental feature. This introduces the ability to create a context consisting of many repositories with specific revisions to be searched. As you explore search results, you might need to navigate your code. Version contexts are respected by code intelligence hovers (go to definition and find references), and allow you to quickly traverse the specific code revisions you care about.

In this release, any non-master branches that are specified in the context will use our non-indexed search. This means that results may be slow for larger deployments. We are working on adding support for indexing additional branches that are specified within the context, and will include this in a future release.

To take advantage of of this feature, add a versionContexts block to the experimental features in your site configuration:

    

"experimentalFeatures": [  "versionContexts": [    {      "name": "srcgraph 3.16",      "revisions": [        {          "repo": "github.com/sourcegraph/sourcegraph",          "rev": "v3.16.0"        },        {          "repo": "github.com/sourcegraph/zoekt",          "rev": "b56036a3b745033badc48807bd67e91cd8d73bdf"        }      ]    }  ]]

We are excited to hear your feedback as we continue to work on this feature! Get in touch on Twitter @sourcegraph, file an issue in our public issue tracker, or email [email protected].

Smoother getting started experience for campaigns

Sourcegraph 3.16 includes multiple smaller bug fixes and performance improvements to campaigns. In addition, we focused on making it easier and more straightforward to start using campaigns on your Sourcegraph instance. We addressed this in several ways: by providing clearer instructions in the onboarding process, improving the webhooks setup process, and by improving our documentation.

We highly recommend configuring campaigns webhooks for your Sourcegraph instance to reduce the load on the code host, and to improve the syncing performance of campaigns updated in Sourcegraph with the code host.

The documentation for campaigns has also undergone a complete overhaul and includes the video above that explains what campaigns can do.

Campaigns are currently in beta. During the beta period, campaigns are free to use. After the beta period, campaigns will be available as a paid add-on. Please contact us for more information.

Best-in-class syntax highlighting

Sourcegraph 3.16 includes major improvements to syntax highlighting throughout Sourcegraph. In addition to general improvements, six new languages are supported (Smarty, Ethereum / Solidity / Vyper, Cuda, COBOL, vb.NET, and ASP.NET), and 30 new file extensions are now detected.

Additionally, this release adds best-in-class syntax highlighting to code diffs. This richer view of diffs can easily be accessed from your pull request (or merge request) using the Sourcegraph browser extension by clicking on the Sourcegraph button in a file header. Alternatively, you can compare revisions directly from Sourcegraph by going to the compare section from a repository page. You can also see this in action on your campaigns to make changes across all the repositories at your organization.

For an example of syntax highlighting on a diff, check out a recent change in the Sourcegraph repository. Make sure to try it out in both dark and light mode!

Easier to read queries in plain text mode

We introduced the smart search bar for plain text mode that provides syntax highlighting, hover tooltips, and diagnostics on search queries in Sourcegraph 3.13. This means that Sourcegraph can provide useful hints, such as fixing invalid quotes, to get to search results faster. Sourcegraph 3.16 removes the feature flag and makes this the standard experience for all users in plain text mode.

Performance improvements for precise code intelligence

Our precise code intelligence backend has been rewritten from TypeScript to Go. This change is part of a larger effort to aggressively optimize conversion and querying of LSIF data, which will be more apparent in upcoming releases. Even with this initial improvement, we’re seeing improvements on precise code intelligence query performance. The following graph shows the five-run average duration of our integration test before and after the rewrite.

Discover useful scopes with `repogroup` autocompletion

The repogroup filter is useful for creating collections of repositories to search over within Sourcegraph. However, there wasn’t an easy way for users to learn what repogroups were available. Now, Sourcegraph provides autocompletion to help you discover available options. This happens automatically when you start typing, or can be triggered using the ctrl-spacebar keyboard shortcut. Try it out in the new smart search bar for plain text mode or in interactive mode!

Check syncing status of repository and user permissions

Sourcegraph 3.14 and 3.15 introduced a more performant background permissions syncing solution for Bitbucket Server, GitLab, and GitHub repository permissions. Sourcegraph 3.16 adds new permissions pages to user and repository settings when background permissions syncing is enabled. These pages allow admins to check the last time permissions were synced and manually trigger the syncing process for an immediate refresh to the user or repository’s permissions.

Enable this feature by adding "permissions.backgroundSync": {"enabled": true} in your site configuration settings.

Understanding user activity of your instance

We want to better understand the activity of users on a Sourcegraph instance while still upholding our commitment to only collecting anonymized, aggregated, and non-specific information (see our ping philosophy). Many admins have asked to better understand the engagement of users on their instances, and in Sourcegraph 3.16 we provide a way to export this data. We will be adding an activity histogram and other visualizations to the admin area in future releases. You may also opt to share this data with Sourcegraph to help us make product decisions that make Sourcegraph even better, and we’ll share the results with you.

Improvements to basic code intelligence for several languages

The out-of-the-box search-based code intelligence was broken on indexed commits for Haskell, JSONNet, Kotlin, Scala, Swift, Thrift, and TypeScript for several releases. Our symbol indexer did not know how to extract symbols for these languages, even though our unindexed symbol service did. This has been fixed and users with code in these languages should expect to see code intelligence hovers for their code once again!

Additionally, Cobol and Tcl have been added to our growing list of languages supported via Sourcegraph’s basic code intelligence. Every Sourcegraph instance comes enabled with basic code intelligence provided by search-based heuristics. For fast and precise code intelligence, see our LSIF documentation.

Changelog

Added

Autocompletion for repogroup filters in search queries. #10141
If the experimental feature flag codeInsights is enabled, extensions can contribute content to directory pages through the experimental ViewProvider API. #10236
- Directory pages are then represented as an experimental DirectoryViewer in the visibleViewComponents of the extension API. Note: This may break extensions that were assuming visibleViewComponents were always CodeEditors and did not check the type property. Extensions checking the type property will continue to work. #10236
Major syntax highlighting improvements, including:
- 228 commits / 1 year of improvements to the syntax highlighter library Sourcegraph uses (syntect).
- 432 commits / 1 year of improvements to the base syntax definitions for ~36 languages Sourcegraph uses (sublimehq/Packages).
- 30 new file extensions/names now detected.
- Likely fixes other major instability and language support issues. #9557
- Added Smarty, Ethereum / Solidity / Vyper), Cuda, COBOL, vb.NET, and ASP.NET syntax highlighting.
- Fixed OCaml syntax highlighting #3545
- Bazel/Starlark support improved (.star, BUILD, and many more extensions now properly highlighted). #8123
New permissions page in both user and repository settings when background permissions syncing is enabled ("permissions.backgroundSync": {"enabled": true}). #10473 #10655
A new dropdown for choosing version contexts appears on the left of the query input when version contexts are specified in experimentalFeatures.versionContext in site configuration. Version contexts allow you to scope your search to specific sets of repos at revisions. #10606
Campaign changeset usage counts including changesets created, added and merged will be sent back in pings. #10591
Diff views now feature syntax highlighting and can be properly copy-pasted. #10437
Admins can now download an anonymized usage statistics ZIP archive in the Site admin > Usage stats. Opting to share this archive with the Sourcegraph team helps us make the product even better. #10475
Extension API: There is now a field versionContext and subscribable versionContextChanges in Workspace to allow extensions to respect the instance's version context. #10601 and #10606
The smart search field, providing syntax highlighting, hover tooltips, and validation on filters in search queries, is now activated by default. It can be disabled by setting { "experimentalFeatures": { "smartSearchField": false } } in global settings. #10648

Changed

The userID and orgID fields in the SavedSearch type in the GraphQL API have been replaced with a namespace field. To get the ID of the user or org that owns the saved search, use namespace.id. #5327
Tree pages now redirect to blob pages if the path is not a tree and vice versa. #10193
Files and directories that are not found now return a 404 status code. #10193
The site admin flag disableNonCriticalTelemetry now allows Sourcegraph admins to disable most anonymous telemetry. Visit https://docs.sourcegraph.com/admin/pings to learn more. #10402

Fixed

In the OSS version of Sourcegraph, authorization providers are properly initialized and GraphQL APIs are no longer blocked. #3487
Previously, GitLab repository paths containing certain characters could not be excluded (slashes and periods in parts of the paths). These characters are now allowed, so the repository paths can be excluded. #10096
Symbols for indexed commits in languages Haskell, JSONNet, Kotlin, Scala, Swift, Thrift, and TypeScript will show up again. Previously our symbol indexer would not know how to extract symbols for those languages even though our unindexed symbol service did. #10357
When periodically re-cloning a repository it will still be available. #10663

Removed

The deprecated feature discussions has been removed. #9649

The changelog for this and previous releases is available on GitHub.

Thank you

Thank you to the many people who contributed to Sourcegraph since the last release!