Lessons on UX, security, and scale when building an enterprise-grade Slack agent

In modern software development, context switching is a productivity killer. Engineers constantly jump between their IDEs, CLI, code host, documentation, and communication platforms like Slack. We recognized an opportunity to embed Deep Search directly into the developer workflow. We wanted to save users from constantly switching between Slack and the Deep Search UI.
Building an agent for enterprise customers requires a focus on user experience, security, and scale. This post explains the technical details and design choices behind our agent.

Designing a great user experience

A good Slack bot does more than just answer your commands. It takes action and communicates naturally. We focused on a few user experience areas to make it fit right in.

Communicating the agent's state

When processing complex queries like those in Deep Search, response times can vary. Keeping the user informed is paramount. We implemented a multi-stage communication flow:

Acknowledge and Initiate: When a user @mentions the agent, we immediately tell them the search is running. We use emojis, such as the 👀emoji, to communicate status in a way that feels native to Slack, enhancing the feeling of an active, responsive agent.

Updates and Escalation: If a search is taking longer than expected, more than a few minutes, we proactively send an update, directing the user to the external link for real-time viewing.

Handling verbose AI responses and follow ups

The age of AI-powered search means responses can be large and highly detailed. We found that providing a quick, dense summary is crucial. We prioritize giving a TL;DR summary while still offering users good options to see the full, verbose response, typically structured using Slack features like Block Kit or threads for readability.

The agent is also designed to manage conversational context, allowing for follow-up questions within the Slack thread to maintain a stateful session despite the platform's API interactions often being stateless.

Implementing rich link previews (unfurling)

To truly minimize context switching, we made Sourcegraph links actionable directly within Slack. When a user pastes a Sourcegraph link, whether it's a file, a standard search, or a Deep Search, the integration automatically "unfurls" it with a rich preview, showing relevant context without requiring a click-through.

The technical process for link unfurling is driven by Slack events:

1. Slack sends a link_shared event to the Sourcegraph service, containing the URLs.
2. The integration parses each URL to determine its content type: file, search, or deep search.
3. A unique entity ID is generated using a SHA256 hash, prefixed with a type identifier (e.g., f-, s-, or the conversation's readToken).
4. The service returns a chat.unfurl response with rich metadata, leveraging Slack's work objects API.
5. When users click these unfurled links, Slack fires an entity_details_requested event, which triggers the display of full details in a flexpane sidebar.

All unfurl and flexpane views generate telemetry events, allowing us to track success and error states for continuous improvement.

Clicking on a card also allows you to view the entire Deep Search (DS) conversation within Slack's flexpane component.

Enterprise-grade security and deployment

For large organizations, an agent must be secure by default and seamlessly integrate into complex environments like Slack Enterprise Grid.

Supporting Slack Enterprise Grid

Enterprise Grid is essential for large enterprises, as it supports a one app to many Slack workspaces setup. Our application is specifically designed to handle this multiple workspace configuration, ensuring scalability and centralized management.

Identity and authorization trick

A major barrier to adoption is requiring users to manually "sign into your Sourcegraph account" after installing the Slack app. We eliminated this friction by implementing a key authentication trick:

We tie Slack users to their corresponding Sourcegraph accounts via their verified email addresses in both systems. Once the integration is turned on by an administrator, any user with a matching, verified Sourcegraph account automatically gains access to the integration. This ensures that the agent's permissions align with the individual user's code access rights, a critical component of data authorization, without any additional sign-in steps.

Enterprise deployment and health checks

As Sourcegraph is enterprise software, we favor a 'create the app yourself' workflow, guiding the site administrator through the app creation process. This ensures the admin controls the app and correctly configures their Sourcegraph instance URL, which is vital for secure credential storage and managing OAuth flows.

To verify basic connectivity during setup, we leverage Slack's event URL test ping. We force Slack to send a new HTTP request to our service by appending a custom ?retry=<random number> query parameter. A successful response to this modified ping serves as a reliable signal that "Slack can indeed contact our service and the integration is set up appropriately".

Scaling with Redis rate limiting

Scalability and reliability demand a robust technical stack and disciplined approach to resource management. To prevent abuse and ensure system stability, we implemented a sophisticated, yet fast, rate-limiting mechanism using Redis.

Our rule set is straightforward: users are limited to a fixed number of requests over a small interval. The system tracks activity using simple counters based on the user and team IDs. When a request is made, Redis checks the current count. If the limit is reached, the new request is blocked, and the user is told when to try again. If the user is under the limit, the counter is incremented, and the system resets the count after the period ends.

This system was engineered with three non-negotiable goals:

1. Fail open: If Redis experiences downtime, the system fails open, meaning requests still go through. Infrastructure errors will not break the core Slack integration functionality.
2. Isolated limits: Limits are isolated by user and team ID. If one user sends too many requests, it does not affect the performance of any other user or workspace.
3. Speed: We rely on basic counters instead of complex, latency-inducing algorithms to stop search spam.

The system checks these limits immediately using our key-value store, stopping heavy search tasks before they consume significant resources. This architecture is crucial for maintaining an "always-on" service and handling high concurrency across large, multi-workspace environments.

Thanks for reading. If you found this interesting then consider reading more about another Slack Bot we built.

---

A special thanks to Justin Dorfman for his contributions to this blog post.