Customer stories

Qualtrics speeds up unit test creation and understanding code with Cody

Qualtrics logo

Qualtrics is a global Experience Management (XM) company building software the world's best brands use to deliver exceptional frontline experiences, build high-performing teams, and design products people love. Qualtrics serves over 19,000 clients and has an Engineering team of over 1,000 software developers.

28% fewer trips out of the IDE

Qualtrics engineers report leaving their IDE to browse the web 28% less often using Cody.

25% faster code understanding

Qualtrics engineers report they can understand code and answer questions about code 25% faster.

Improved unit test coverage

Engineers reported that unit tests, which took devs a full day to write, now take roughly 10 minutes with Cody, leading to improved unit test coverage.

Try Sourcegraph with your team

Search, write, and understand code faster with Code Search + Cody.

Book a demo

With a large global Engineering organization of over 1,000 software developers, Qualtrics' Developer Experience (DevX) team is responsible for delivering solutions and processes to those developers to improve their day-to-day happiness and productivity. DevX also runs programs to raise the bar on engineering quality at Qualtrics.

Qualtrics' AI Initiative

When generative AI and ChatGPT quickly rose in popularity in 2023, Qualtrics' DevX team immediately saw the potential to use AI within their development teams. The team took a primary initiative to evaluate AI coding assistants in the summer of 2023. Godwin Babu, Sr. Manager and leader of Qualtrics' DevX team, hypothesized that AI usage could significantly improve productivity within their engineering teams.

Why Cody?

Qualtrics evaluated several AI solutions in the market before deciding on Cody, and security was a primary priority. Godwin stated, “When choosing a coding assistant, we prioritized DLP [data loss prevention] and assurance of the security of our intellectual property. Cody worked seamlessly with the systems we already had in place.”

Qualtrics' security protocols extend to their code host, which is why they run their own self-hosted GitLab instance. Cody works with all major code hosts, both self-hosted and cloud-based, so it could plug into Qualtrics' existing setup. Per Godwin, “We run our own GitLab instance within our own data centers, and Sourcegraph works seamlessly with it. That made signing up for Cody easy.”

DevX team also wanted a solution supporting the latest LLMs so their teams could always access the fastest and most accurate models. This led Qualtrics back to Cody, which supports several LLMs as options. “Cody's ability to switch backends, from Claude to GPT, is very attractive to us. This is a fast-moving field, and we see updates to LLMs constantly, so having a tool that can react to new LLMs quickly is important to us.”

Lastly, Qualtrics was already a customer of Sourcegraph with extensive usage of Code Search for their engineering teams. According to Gordon Fu, Quality Engineering Manager, Qualtrics' existing trust and confidence in Sourcegraph as a vendor led to Cody quickly bubbling to the top of their AI evaluation.

Qualtrics ultimately decided on Cody as their AI coding solution based on its security, LLM interoperability, and ability to work with their existing code host setup.

Results

Since bringing Cody to the engineering organization, Qualtrics engineers report having to leave their IDE to find information on the web 28% less often when using Cody, and they can understand code 25% faster. The majority of their surveyed engineers also report between 10 and 30 minutes of time savings per day by using Cody, which the DevX team estimates to be roughly 10% of development time.

Looking forward, Godwin also sees places where Cody will drive even more progress across their engineering teams: “One of the places we're thinking of using Cody is connecting it with other initiatives. For example, in DevX, we have one program to improve code coverage: setting goals for all teams to achieve certain coverage with unit tests. We see a direct correlation with Cody here since Cody has proven adept for creating tests and improving code coverage.”

Brendan Doyle, Senior Software Engineer at Qualtrics, already sees value in writing unit tests with Cody. Brendan works on a team building a wrapper library for AWS Lambda and other internal Infrastructure as Code (IaC) services.

“It's not a secret for developers that writing unit tests is a necessary time sink in the day-to-day life of a developer for writing reliable software,” per Brendan. “Generating unit tests, by far, is my number one favorite feature and time saver for Cody. I can spend my brainpower somewhere else instead of spending it figuring out how to write tests.”

Something that would've taken me multiple dev days was done in an hour with Cody. Cody can generate a template for a test, and then I can prompt it to make adjustments to get the test to cover exactly what I'm looking for.

Brendan Doyle,Senior Software Engineer, Qualtrics

Brendan's team also gets value from using Cody to ramp up on big codebases and existing projects, especially for the younger engineers. “One of the most daunting things as a junior engineer is working on a large, existing codebase. There is always a ton of domain knowledge about that code that's restricted to the people who wrote it, no matter how well it's documented. There are always nuances that only the code authors know. But if developers know how to prompt Cody, Cody can find context and explain the code to them.”

You can get an understanding of a large codebase a lot faster with Cody.

Brendan Doyle,Senior Software Engineer, Qualtrics

Explore other customer stories

Indeed logo

Indeed keeps code up to date and accelerates development velocity.

Read the case study
FactSet logo

FactSet migrates from Perforce to GitHub.

Read the case study
HashiCorp logo

HashiCorp streamlines cross-repository code search and fixes with Sourcegraph.

Read the case study
CERN logo

Sourcegraph empowers CERN to tackle code reuse and code changes in mission-critical applications.

Read the case study