Posts

post

Projects

project

Videos

video

Podcasts

podcast

People

person

Events

event

Publications

publication

Solutions

solution

Services

service

Pages

page

Partners

partner

Case Studies

case-study

Magazine Issues

magazine-issue

Magazine Articles

magazine-article

Quotes

quote

Tabs

Default Confirmation

General Contact form

Newsletter form

Project Management Office Contact Form

Newsletter subscription

EGI DIH and Business Contact Form

Share your news

Services, Solutions and Support Team Contact Form

Communications Team Contact Form

Director Finances and HR Contact Form

Training Team Contact Form

EGI DIH Application form

Form Test

Service order form

test

EOSC DIH Tender

core/freeform

This article was written by <a href="https://www.egi.eu/about/egi-foundation/team/bruce-becker/">Bruce Becker</a>, Senior Operations Officer at the EGI Foundation, and re-posted here with permission.&nbsp;You can read the original article on <a href="https://medium.com/@brusisceddu/devsecops-1223732b8086" target="_blank" rel="noopener noreferrer">Medium</a>.&nbsp;
&nbsp;
<h5 id="018f" class="graf graf--p graf-after--h3">I have been thinking about how to include a vulnerability scan in a pipeline to deliver applications to EGI FedCloud sites. It goes a little like this.</h5>
<h4 id="e468" class="graf graf--h3 graf-after--p">The big&nbsp;picture</h4>
A CSIRT’s job is never done! A distributed computing platform is inherently open to risks, even more so a collaborative platform. Ensuring that the platform is safe and secure for users is a thankless, full-time job. The attack surface can be very large indeed when one considers a platform like&nbsp;<a class="markup--anchor markup--p-anchor" href="https://www.egi.eu/federation/egi-federated-cloud/" target="_blank" rel="nofollow noopener noreferrer" data-href="https://www.egi.eu/federation/egi-federated-cloud/">FedCloud</a> — there are several layers to it which may provide vectors for exploits. While the majority of these can be locked down by the operators of these sites, at the end of the day, they are still used by&nbsp;… well,&nbsp;users.
<h4 id="7402" class="graf graf--h3 graf-after--p">Whose cloud is it anyway&nbsp;?</h4>
There is the usual thin line to tread between usability, ease of access, and security. Much of the appeal of a thing like FedCloud is the freedom of users — and the communities which they belong to — to define their own applications and workload scenarios, molding the basic infrastructure into something they are comfortable with. In essence, by providing a common IaaS or PaaS layer, FedCloud allows users to deploy arbitrary applications, at their own speed, under their own control. Of course, with great freedom comes great responsibility.
There is an inherent difference between users and operators&nbsp;: the former are trying to optimise their&nbsp;usage&nbsp;of an infrastructure, while the latter are trying to optimise the&nbsp;stability&nbsp;thereof. It’s not that either of these player is malicious&nbsp;per se, but their different priorities generate a natural conflict — one which perhaps cannot entirely be&nbsp;removed, but which can be&nbsp;mediated.
<h4 id="99a5" class="graf graf--h3 graf-after--p">Something to talk&nbsp;about</h4>
How could subtle changes to the environment improve the relationship between operators and users&nbsp;? Perhaps the first positive step is to surface issues before they become a problem. The second is to provide a common language for announcing vulnerabilities and clear, easy-to-execute instructions for mitigating them when detected. With Dev (the users), Sec (the CSIRT) and Ops (the infrastructure) all on the same page, any conflicts can be discussed in an objective manner.
<h4 id="aace" class="graf graf--h3 graf-after--p">Prevention is better than&nbsp;cure</h4>
Currently,&nbsp;<a class="markup--anchor markup--p-anchor" href="http://csirt.egi.eu/" target="_blank" rel="noopener nofollow noreferrer" data-href="http://csirt.egi.eu/">EGI CSIRT</a>&nbsp;does a great job of scanning the endpoints of the infrastructure to detect vulnerabilities. This is necessary, since these vulnerabilities are not a static set, but new ones are being found continuously. Fixing machines that are already deployed is a necessity in order to provide a secure&nbsp;platform, but what about the new applications that are built and deployed by user communities&nbsp;? Wouldn’t it be nice if these applications could be checked&nbsp;before&nbsp;they were deployed<a class="markup--anchor markup--p-anchor" href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:maybe" target="_blank" rel="noopener nofollow noreferrer" data-href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:maybe">1</a>? In a perfect world, there would be a more-or-less well-defined pipeline through which applications could flow, before landing in the production environment.
Typically, this would include all the great things that you can imagine in a pipeline — continuous integration, testing, and delivery. It would be awesome if we could surface the vulnerabilities or security risks&nbsp;at the same time&nbsp;as surfacing run-time or deploy-time issues. Put differently, we wouldn’t deploy a broken application, so why deploy an insecure one&nbsp;?
<h4 id="0989" class="graf graf--h3 graf-after--p">How to spot a vulnerability</h4>
That of course begs the question:&nbsp;How do we know that an application is insecure&nbsp;?
A naive answer appears to be&nbsp;“Just scan the damn things for known vulnerabilities”. Indeed there are several tools out there for doing this, including&nbsp;<a class="markup--anchor markup--p-anchor" href="https://github.com/CESNET/pakiti-server" target="_blank" rel="nofollow noopener noreferrer" data-href="https://github.com/CESNET/pakiti-server">Pakiti</a>,&nbsp;<a class="markup--anchor markup--p-anchor" href="https://github.com/coreos/clair" target="_blank" rel="nofollow noopener noreferrer" data-href="https://github.com/coreos/clair">clair</a>&nbsp;and others. There are also tools for delivering “compliance as code” — particularly&nbsp;<a class="markup--anchor markup--p-anchor" href="https://inspec.io/" target="_blank" rel="nofollow noopener noreferrer" data-href="https://inspec.io/">InSpec</a>,&nbsp;<a class="markup--anchor markup--p-anchor" href="https://testinfra.http//testinfra.readthedocs.io/" target="_blank" rel="nofollow noopener noreferrer" data-href="https://testinfra.http//testinfra.readthedocs.io/">TestInfra</a>. These tools typically compare installed&nbsp;packages&nbsp;against a vulnerability database. They are designed to check OS packages. They may also work with some language ecosystems like Ruby<a class="markup--anchor markup--p-anchor" href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:Gemnasium" target="_blank" rel="nofollow noopener noreferrer" data-href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:Gemnasium">2</a>and Node<a class="markup--anchor markup--p-anchor" href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:AppCanary" target="_blank" rel="nofollow noopener noreferrer" data-href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:AppCanary">3</a>&#8211; maybe not so much with Python or Go — but that relies on specific packages which can be matched against a vulnerability database. These scanners do not actually do penetration testing, as is usually the case in network scanning systems, or more advanced penetration testing systems.
And herein lies the catch&nbsp;:&nbsp;What if your applications are not delivered with packages. This is indeed the case with,&nbsp;e.g.&nbsp;CODE-RADE, where everything is built from scratch and no “packages” are installed. We could, for example, tag builds according to the versions of the source code built, and then match&nbsp;thoseagainst the CVE databases, perhaps.
Although this is a&nbsp;design&nbsp;feature of CODE-RADE, it may be a mere convenience for many use cases. Users may simply hack their application into shape until it works, then tag a VM or container and call it a day. Detecting vulnerabilities introduced in such applications is going to be a risky business&nbsp;unless a true penetration testing suite can be introduced to the delivery pipeline. There are some tools, again mostly language specific, which can be called to the cause of keeping applications in the production infrastructure safe,&nbsp;e.g.&nbsp;<a class="markup--anchor markup--p-anchor" href="https://www.owasp.org/index.php/OWASP_Dependency_Check" target="_blank" rel="nofollow noopener noreferrer" data-href="https://www.owasp.org/index.php/OWASP_Dependency_Check">OWASP Dependency Check</a><a class="markup--anchor markup--p-anchor" href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:OWASP" target="_blank" rel="nofollow noopener noreferrer" data-href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:OWASP">4</a>.
<h4 id="7616" class="graf graf--h3 graf-after--p">Adding Sec to the DevOps&nbsp;pipeline</h4>
Let’s face it, we’re not running a Fortune500 company here. You can’t have 100% secure applications, but we can do a damn sight better than what we’ve got right now&nbsp;! I propose a shift from vulnerability monitoring of the infrastructure to&nbsp;vulnerability testing&nbsp;of the applications before they even get there.
If development of research applications follows a continuous integration, adding compliance and vulnerability testing to the pipeline represents just another step for the application to pass. Sure,&nbsp;there is a conceptual leap to make: from “Trust me, Ops&nbsp;! This opaque blob of data is totally benign!” to “Ok Ops, you’ve got your tests, I’ve got mine and they’re all passing”&nbsp;is perhaps a big one for many. In order to have people adopt this way, Ops needs to deliver a smoother experience and better support than they are currently delivering to users&nbsp;: builds for arbitrary applications, arbitrary environments and configurations and what Devs love about&nbsp;<a class="markup--anchor markup--p-anchor" href="http://12factor.net/" target="_blank" rel="nofollow noopener noreferrer" data-href="http://12factor.net/">12 Factor Apps</a>&nbsp;:&nbsp;Dev/Prod Parity<a class="markup--anchor markup--p-anchor" href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:DevProd" target="_blank" rel="nofollow noopener noreferrer" data-href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:DevProd">5</a>.
<h4 id="4a7a" class="graf graf--h3 graf-after--p">Something we can all&nbsp;trust</h4>
Once we have a pipeline, we can and should raise security or vulnerability issues wherever we can, along with all the infrastructure tests. Furthermore, these tests should be separated from the application tests themselves. In other words, if Ops provides a tested and immutable environment for Dev to build on, then the application should:
<ul>
<li>Ensure that it can build — Are the compilers and dependencies available&nbsp;? Are the relevant infrastructure services available&nbsp;?</li>
<li>Ensure that it is correct — Have errors been introduced into the environment in recent commits&nbsp;? Does the application maintain internal consistency in the build and execution environment provided by Ops&nbsp;?</li>
<li>Ensure that it will run — Does the execution environment permit the proper execution of the application, with access to relevant backing<a class="markup--anchor markup--li-anchor" href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:backing" target="_blank" rel="nofollow noopener noreferrer" data-href="https://brucellino.github.io/blog/blog/2018/04/08/DevSecOps/#fn:backing">6</a>services&nbsp;?</li>
<li>Ensure that infrastructure remains immutable — Has the application made detectable changes to our infrastructure&nbsp;?</li>
</ul>
That last point is key. The rest of the tests (integration tests, unit or functional tests) are up to Dev. But trusting Dev to ensure that Prod is ok is like trusting the fox with the chickens — there’s an inherent conflict of interest, even if there is, as is most often the case, no malice. No, these tests need to be maintained by Prod, in collaboration with Sec.
<h4 id="da5a" class="graf graf--h3 graf-after--p">Asserting Compliance</h4>
We then come full-circle. EGI has an extensive list of security policies which can be used as a basis for writing compliance as code. They need to get out of whatever format they’re in now, and into something that can be executed. To quote the&nbsp;<a class="markup--anchor markup--p-anchor" href="https://www.chef.io/inspec" target="_blank" rel="nofollow noopener noreferrer" data-href="https://www.chef.io/inspec">Chef pitch for Inspec</a>:
<ul>
<li>Transform your requirements into versioned, executable, human-readable code.</li>
<li>Detect Fleet-wide Issues and Prioritize Their Remediation</li>
<li>Reduce Ambiguity and Miscommunication Around Rules</li>
<li>Keep up with Rapidly Changing Threat and Compliance Landscapes</li>
</ul>

DevSecOps: Including Security in the Continuous Delivery Pipeline

<a href="https://www.egi.eu/terms-of-use/">terms of use</a>  |  <a href="https://www.egi.eu/privacy-notice/">data protection</a> | <a href="https://www.egi.eu/egi-foundation/certifications">certifications </a>