Before you get started..

• Before you get an audit, you should ideally do some preliminary checks yourself first. This saves time and money as we’d be doing that first anyway. Check our open source best security practices guide here on Github.

• The costs of an audit varies but it’s typically tied to the complexity, labor hours, and type of audit (e.g. is there complex cryptography involved? does it have lots of integrated parts? how much testing has already been done?). As initial audits could range anywhere from $30k to $200k, we have helped many projects raise funds and source sponsors to cover the entire costs or to split them.

• An audit isn’t just a single process or service, it’s a relationship with auditors that leads to discoveries and improved processes now and in the future. This means recurring audits would ideally find fewer vulnerabilities, and future updates would be written more securely based on learned experience.

• The most critical step of any audit is reaching out to talk to us about what you actually need and how we can help. Even if you don’t end up doing a full audit, we still want to hear from you and what you’re working on, give any advice we can, and help you find the resources you need.

 

How does it work?

Introduction

A focused and correctly-scoped security review executed by an experienced team typically results in significant and long-lasting improvements to software and is one of the best ways to improve security posture in open source projects.  According to “Zero Days, Thousands of Nights: The Life and Times of Zero-Day Vulnerabilities and Their Exploits” by Lillian Ablon and Andy Bogart, “products that have a longer life, are more complex, are popular with a large market share, or are high revenue generators, more people have evaluated the code bases, and finding vulnerabilities often requires more in-depth auditing, logic review, and source code analysis, in order to go several layers deep.” This has been the case with OSTIF, as our expert reviews have resulted in hundreds of bug patches, including the patching of over 100 Critical/High Impact Vulnerabilities.  This document serves as a guide for projects that undergo security review through OSTIF.

The Audit Process 

	Step 1. Coordinate OSTIF meets with you one-on-one to get to know you and understand your needs. Once a preliminary scope is defined, OSTIF sources professional auditors and experts from a diverse network. Bids are collected and analyzed based on cost and team expertise. After coordination and approval, OSTIF organizes communication between you and the audit team and moves the review forward.
	Step 2. Audit The audit team gets to work.  Code is evaluated for vulnerabilities based on the scope of the review.  You are provided with updates as the review progresses. OSTIF manages the process and acts as a neutral party should any questions or concerns arise.
	Step 3. Patch Auditors supply you with the results of the evaluation and assist with fixes and strengthening the code. This process allows for a lasting impact on the software’s security.
	Step 4. Release Report and Maintain The updated code and audit report is released to the public. This provides assurance to users that the software has been expertly reviewed. Further maintenance via supplemental reviews of new updates and additional audits are available through OSTIF.

Audit Process Timeline

Pre-Audit – The following need to be completed by OSTIF prior to the beginning of the audit.

Name	Action
Initial Meeting	Designate and establish communication channels/contacts.
Finalize scope and audit team.	get SOW and RfP together and signed, connect with the audit team and schedule the audit.

 

Audit – The following are completed throughout the audit process by the Audit Team.

Name	Action
Audit Kickoff Meeting	Designate representatives and contacts, frequency of meetings, kick off review.
Sync Meetings and Calibration Sessions	Provide updates on audit progress, address any issues or concerns.
Pre-Closing Meeting	When a report is drafted or nearly completed, meet with all stakeholders to discuss and address any issues or concerns.

 

Post Audit – The following are completed by everyone when the audit work is completed. 

 

Name	Action
Closing Meeting	When the report is finalized, meet with all stakeholders to discuss and address any issues or concerns.
Coordinate Announcement	Meet with stakeholders to discuss timeliness and content of report publishing
Solicit Feedback	(Project maintainers only) Fill out Feedback Form

 

Who’s Who

OSTIF: We are directors for the engagement, working to bring all necessary parties together on a timeline to initiate, pursue, and complete the work. In designing the engagement, OSTIF works in the front and back ends of the consultation to plot the undertaking from start to finish in the best way for all invested parties. 

Security Team: This organization will perform the engagement work. Their responsibilities include carrying out the SOW, writing and presenting a final report of the work, and collaborating with the other parties to create a comprehensive understanding of the Project. They are not only auditors, but a great resource for the Project to learn more about security as it specifically relates to the Project and open source. 

Project/Organization: A Project is the subject of the work. The Project’s responsibilities include a detailed questionnaire on behalf of the project, as well as whatever amount of time they’d like to contribute to the audit’s success. This includes participation in meetings throughout the process of the engagement, responsivity to questions via preferred communication methods, and the resolution of any findings with security impact to the satisfaction and approval of the Project. 

How To Prepare 

A Security Review through OSTIF is intended to be a collaborative effort between Project Representatives (such as core maintainers, creators, community managers, or corporate representatives) and the Review Team. OSTIF undergoes a rigorous RFP and coordination process to ensure that the most qualified and well-suited teams are selected for Security Review. The Review Team works closely with you throughout the engagement and provides insight on findings and remediation recommendations. The end result is a documented audit report that outlines the findings and remediation strategies, along with documentation of any and all fixes made throughout the engagement. 

Project Documentation

The following is basic information necessary for any security review:

• Project Information: Typically in the form of a github link or link to source code and documentation.
• Project Documentation: Any and all documentation on the project, practices, and processes.

Tools for the Audit Team

• Any previous reviews of your code base or components you depend on. 
• Any documentation regarding security practices (i.e security.md) 

• Security Scorecard: A tool to check project’s security practices. OSTIF recommends that you check your scorecard and do your best to adhere with the practices checked by the tool. 
• CII Best Practices Badge Program: Get acquainted (if you haven’t already) with the Program and follow the best practices.

Contacts

Establishing clear contacts and communication channels between the Project and Audit Team is essential to a successful Security Review. It is equally important to designate who the contacts are, and ensure that current information is up-to-date and accurate. Due to the international location of some teams, ensure that information is reaching its intended recipient at their business hours. 

Security Practices

Knowledge and documentation of the following security practices help auditors with scoping and test coverage:

• Components and Dependencies: Are any components not open source? 
• Security Practices: Do you fuzz your code routinely? What practices and processes do you have in place for secure code? 
  • Fuzzing Tools (i.e oss fuzz)
  • Dynamic Analysis Tools  
  • CI/CD Tools
• Encryption: What encryption library(ies) are used by your project? 

What to Expect

Auditees can expect the following as part of an OSTIF Security Review:

• Reachout and Introductions: We want to get to know you! OSTIF asks for a brief meeting for introductions and a deeper context of your security needs. 
• Audit Questionnaire Form: After introductions, we ask that you collaborate with us on a simple questionnaire form. The objective of the form is to capture basic information to use for scoping and building a Request for Proposals (RfP).
• RfP: Once OSTIF has a better understanding of your project and security goals, a draft RfP is built for you to review and approve. 
• Sourcing an Independent Review Team: OSTIF puts the RfP out to its network of thousands of open source security experts, analyzes the responses, and chooses the best proposal for your project. 
• Connecting to Review Team and Audit Kickoff Meeting: Once OSTIF has sourced a review team, introductions are made and the review team kicks off the audit with you. The process is designed to make it as easy as possible for project maintainers. 
• Completing the review and making fixes: The review team works alongside the project maintainers and reports findings as they are made. They collaborate on remediation and disclosure. (NOTE – OSTIF follows the Google Project Zero disclosure policy of 90 days (+30 days if needed and requested in advance)). Thankfully due to the help and collaboration provided by the Review Team, remediation and fixes happen simultaneously as part of the audit. However, the policy sets a precedent to complete remediation immediately.

 OSTIF works with you and champions the security review process. The result is fixed vulnerabilities and improvement in tooling and security posture. 

Reach out to us!

Let’s have a discussion, we’d love to hear about your project.

CLICK HERE to go to our intake form