Compliance-Driven Development or the Story Behind Swiftype’s SOC2 Certification

Based on my experience, just a decade ago not many people within the Silicon Valley startup community considered compliance an important stepping stone in a company’s development roadmap. And when it came to compliance for startups, it was nearly synonymous with PCI/DSS — mandatory certification used by the credit card industry. Over the last few years though, the rise in the number of startups working with large amounts of private and confidential data (fintech, healthcare, etc) and subsequently the rise in the magnitude of data breaches, led our industry to accept the idea that compliance and certifications are not just for the “big guys”. Nowadays, even very small companies are pressed to go through formal certifications if they want people to trust them with private or confidential data.

That is exactly what happened to Swiftype at the beginning of 2017. While preparing for a public release of our latest product (Swiftype Enterprise Search), we understood that it was going to involve a lot of confidential information and we would need to be able to assure our customers of our capabilities to protect their data. In addition to the marketing aspect, there was a security angle to the problem as well: we were looking for a standard framework that could be used by our small team to ensure the safety of customer data, guiding us through the process. Based on those considerations, we decided to go through a formal SOC 2 certification. In this article, I will describe our journey towards the certification and our findings along the way.

Different Ways of Approaching Certification

Based on my experience with various certifications (during my career, I’ve had a chance to participate in PCI/DSS, ISO27001 and SOC 2), there are at least two primary ways of looking at and preparing for an audit.

A Passive Approach

You hire an auditing company, wait for them to hand you a list of requests, scramble to collect some evidence and keep going back and forth with them until you have a required minimum — just enough to pass the certification. If you miss some controls, the auditor often provides you with generic examples to be used. This approach is typical for large (and old-school) companies who view certification and compliance as an obligation.

Even though this is the cheapest (in the short term) way of achieving a certification, I am not a fan of this approach for multiple reasons:

• You often end up with a random set of processes and controls designed to fit a generic large company that are handed over to you by an outside consultant. Those processes often feel very formal and alien to a culture of a fast-moving startup.
• By viewing the certification process as an obligation (something “we have to do”), you miss out on the opportunity to use it as a driver for improvement in all areas of your business: from infrastructure security to onboarding processes, to accounting, HR and so on.
• Being generic, a lot of processes you introduce by being passive during a certification end up relying heavily on bureaucratic approach and are very manual. This may be too painful for a small company with a limited staff juggling different roles and responsibilities.

In general, I do not believe a small to medium-sized company should use this approach when preparing for and going through any certification. By doing so, you are missing out on many potential benefits of compliance-related efforts.

An Active Approach

Another, drastically different approach to compliance is based on a simple idea — you have decided to get certified because you understand the value behind it, you’re spending resources on doing it, so you better use it as an opportunity to improve your business, your infrastructure, your team and gain as much as possible from the process. Here is how it works:

• First, you collect the list of requirements and criteria that your company will be tested against during the audit.
• Then, you analyze the current state of the company according to each of the criteria.
• And, finally, you carefully map those lists to each other, looking for any gaps that you will need to fill.

By actively analyzing the state of your company, you will understand what is missing and will be in a position to fix those issues by changing your infrastructure, introducing new processes, etc. You will understand not only what needs to be done, but why it is being done and, since you know how your company operates, you should be able to tailor those changes to your unique environment and make them feel much more natural. Another very important aspect of the active approach to compliance in a modern technical startup is that you should be able to automate a lot of your compliance needs, significantly reducing the additional workload caused by compliance-related changes within your company.

I was lucky to get introduced to this approach at Eligible by Aaron Bedra, and it has changed my view on compliance forever. Applying the same method at Swiftype only reinforced my conviction that compliance-related work in a small company may be hugely beneficial while being done in a non-disruptive way.

Initial Gap Analysis

As described above, the first step in our preparation for SOC 2 certification was so-called “gap analysis” — a process of mapping the list of criteria for a certification with the current state of the company and finding all of the criteria that require additional controls and processes to be introduced within the company.

The most useful document during this step was the official list of Trust Services Criteria published on AICPA website. The document contains a list of all criteria used during the audit and, what was enormously helpful, a list of illustrative risks and illustrative controls associated with each criterion. Those illustrative examples helped us better understand the reasoning behind each criterion and define what our internal controls should look like.

Based on the official documentation from AICPA, we created a large spreadsheet, which mapped SOC 2 criteria to illustrative risks, then each risk was mapped to an internal control we already had or needed to implement.

Here is a small snippet from the spreadsheet:

As you can see, control activities (specific things we do within the company to address different risks) within the document end up repeating, since one control activity often addresses multiple risks and hence applies to different criteria. After we collected a list of control activities, we grouped them into logical collections — internal controls within our company.

When the document was ready, it became the main tool used for preparation for the audit, helping us track the progress of implementation for each control activity and each control within the company.

Designing Controls

As I already mentioned before, the illustrative controls provided by AICPA within their Trust Services Criteria list were really helpful for guiding the process of designing our own internal controls. We would take each criterion, look at each illustrative risk and illustrative control and ask ourselves a simple question: how could we address the illustrative risk in the most efficient way using internal automation and other technological solutions at our disposal? The result would very often be much simpler than the illustrative control, but as long as it addressed the underlying risk, we were confident enough it would work for us.

Just as with any other aspect of building a company, there are many people who have designed and implemented compliance and security controls before, and it is always useful to understand what other people did and why before you make your own decisions. While designing our own controls, we looked at available public information on SOC 2 and ISO 27001 controls, talked to our peers within the industry and researched security controls used by our vendors (many companies publish information about their security controls or are willing to provide you with their SOC 2 reports). But we always strived to make controls our own — make sure they would fit within our existing culture, our existing processes, etc. Changing existing processes that worked for us for years was the last option, and we only did it when we could see a clear improvement for the company as the result of the change.

Tracking the Implementation

After we performed the initial gap analysis, we ended up with a very long list of control activities designed for our company. To make it easier to control the process of implementing those controls, we used Jira. Here are some ideas that helped us along the way:

• First, we created a dedicated Jira project for tracking our internal controls — each logically grouped set of control activities from the gap analysis document ended up being represented with a single CONTROL Jira issue, that would contain information about those control activities, map them to relevant SOC 2 criteria, etc.
• Then we created Jira issues for our engineering and operations teams (TECHOPS and ENG in our case); one issue per control activity we needed to implement — this made sure that we would not drop anything on the floor during the implementation process and would be able to incorporate compliance-related work into our normal business operations and Jira-based sprint planning. All implementation Jira issues have been tagged with a “SOC2” label to make it easier to find them across different team projects.
• Finally, we linked implementation Jira issues to controls by making implementation issues “block” controls — this made it very easy to tell what needed to be done before a control would be considered fully implemented at Swiftype by looking at a specific CONTROL issue and seeing all blocking implementation items on it.

Here is an example snippet of a CONTROL issue from our internal Jira tracker:

Internal Pre-Audit Control Testing

As I explained at the beginning of the article, ever since the beginning of the process we aimed to pass the audit without any exceptions — we wanted to be sure our system was designed properly without any gaps between the compliance criteria and our internal controls. To ensure our audit would go as smoothly as possible, we performed internal control testing during the final stretch before our on-site visit from the auditors.

The testing was done using the following simple process:

1. For each CONTROL issue listed in our Jira
2. Make sure all blocking implementation issues have been completed
3. Check each SOC 2 criteria related to a specific CONTROL, get a list of illustrative risks listed in the original document and make sure we:

• Have relevant policies in place addressing the risk
• Have automation in place providing us with alerts and audit trails addressing the risk
• Could provide evidence of both the policies and automation if asked during the audit

When we tested a control, we would mark it as Done in Jira, helping us track the controls that still needed attention.

This phase is where our very active and involved process finally met the old-school evidence-based process typically used by large companies. Since certification is based on providing auditors with evidence, they always require you to upload hundreds of pieces of content (documents, screenshots, logs, etc.) to their portals before the on-site audit. In turn, while doing our internal testing, we would always make sure we could provide any evidence required. This meant that after each test had been finished, we would upload relevant evidence pieces into the auditor portal and make notes on how it was obtained so that we could quickly do it again during the on-site audit.

Our Findings and Future Plans

During the on-site audit, we were often complimented by the auditor’s staff for being so well prepared and having our controls so well laid out. But most importantly, we have not noticed any slowdown in our team’s day-to-day operations after implementing all of our internal controls. I believe that is a true testament to the active and involved preparation process we have gone through and to all the automation we put in place, allowing our small team to continue focusing on building the product and the company instead of being slowed down by the old, paper-heavy process often associated with compliance.

We believe that many aspects of our company improved thanks to the compliance-related efforts of our team and all the changes we were able to make while guided by the reliable, industry-tested framework of SOC 2. We’re looking forward to providing safe and secure services to our customers and are happy to have AICPA certify our ability to do so.