Security Testing Part 2 - Lab Work

Jarlsberg Turns to Gruyere

As of July 13, 2010, the Google Codelab formerly known as Jarlsberg is now called Gruyere. This training aid is a knowingly vulnerable application, which can be used to learn and understand web vulnerabilities.

http://google-gruyere.appspot.com/

This is a particularly useful experience for test engineers, since one of its main aims is to simulate the activity of penetration testing - simulating attacks from malicious sources (known as Black Hat Hackers, or occasionally Crackers). The process involves an active analysis of the system for potential vulnerabilities resulting from poor or improper system configuration, from known and unknown hardware and software flaws, or from operational weaknesses in process and/or technical countermeasures.

Gruyere is excellent at what it does, in its tutorial-structured, hint-led way. The contents sidebar is the best reference to what classes and detailed subtypes of vulnerability are covered. Cross-Site Scripting, Script Inclusion and Request Forgery (XSS, XSSI, XSRF) are particularly well served, pardon the pun. Other inclusions include: data tampering, information disclosure, denial of service, remote code execution and elevation of privilege.

For each vulnerability, you assume the role of a black hat (malicious hacker) and uncover the related exploit. One particularly well realised aspect of this Codelab is the way in which you are encouraged - just like a real world security researcher - to combine approaches of both major kinds: black box (e.g. probe it with some bad input data) and white box (e.g. read its source code).

What Can't Gruyere Do?

There are some significant omissions. For starters, any popular C/C++ attack with "overflow" in the name. Gruyere is written in Python, which by design, prevents any attempt at reading and writing outwith an array's bounds. For that reason, the popular attack known as buffer overflow is not covered by this lab. Similarly, Python prohibits integer arithmetic overflow, and so Gruyere includes no examples of integer overflow exploits.

It is important to note that these observations do not guarantee the immunity of the website from such attacks. All applications are exposed to so-called platform vulnerabilities. These can be security weaknesses in the web browser or other client side code, or in the underlying Python runtime. The lab doesn't cover such issues, even though sometimes you might avoid platform vulnerabilities by making changes to the app, so as to alter its platform dependencies, or its resource usage.

By far the biggest omission from this lab has to be SQL Injection - for the admittedly very good reason, that Gruyere does not use SQL. Again, there are plenty of well researched cases elsewhere on the web. For example this very readable account by Steve Friedl is a model of clarity.

Microsoft 's extensive and very professionally produced Security Virtual Labs are also well worth a look. This one on SQL Injection Vulnerabilities can take up to 90 minutes to complete (the standard allowance for these labs). It requires JavaScript plus "IE6 or above"; and you'll have to install an ActiveX control to connect to and run the lab.

A Note On Scope

While security testing obviously needs to be targeted on a specific set of possible exploits, given the vulnerabilities and their mitigations in the application or system under test, the umbrella undertakings of security training, and more generally education, must entertain no such restrictions.

Take Banned APIs for example. These are closely related to the subject of buffer and integer overflows described above, and to other vulnerabilities specific to code in languages like C/C++ but not found for example in managed code. Why should a Test Department dealing with C# WinForms apps care about these? - Well, it could happen at any time, that the Development Department will decide to slip some C/C++ code, or a legacy library, into the mix. Maybe to avoid injecting a .NET or Windows Shell dependency somewhere. Suddenly, your attack surface quadruples. Shouldn't you be prepared for that? Certainly, to the extent that you should immediately and instinctively be at least aware of the implications of that change.

Another example. Why worry about XSS attacks, when no such vulnerabilities can be exploited, due to the particular nature of our app? - Because tomorrow, the Marketing Department will demand that lucrative third party ads be served up in IFrames beside your primary content. Next day, you could be serving malware as blithely as the BBC. Clearly, just-in-time training is woefully inappropriate here.

Finally, here is the definitive word on the issue from Michael Howard, architect of Microsoft's Security Development Lifecycle, and principal security program manager on their Trustworthy Computing (TwC) Security team. In his article The SDL and the CWE/SANS Top 25 Most Dangerous Programming Errors 2010, he writes:

Even CWE 98, "PHP File Inclusion," is covered by the SDL in our required security training classes, which is especially remarkable when you consider that virtually no PHP code is written at Microsoft!

The reason that we address issues like PHP file inclusion in the SDL is that we don't simply wait for new vulnerability taxonomies to be released and then rush to add mitigations to our security processes; rather, we structure the SDL to provide developers with fundamentally sound, secure programming practices. As a result, we cover not just the known vulnerabilities of today (like the Top 25) but also many of the unknown vulnerabilities that will be discovered tomorrow. The fact that all of the Top 25 are addressed by the SDL is a great validation, but it is the result of the content of our process and not the cause.

Now Hack!

Once you've read the introductory material, including the warning paraphrased in red below, you can start your Gruyere adventure here:

https://google-gruyere.appspot.com/start

One final thing, do remember to concentrate exclusively on the suggested attacks. Seriously. Any deviation from the vulnerability cases which Google have expressly authorized here could have dire consequences of almost unlimited badness for you.

Previously:

Part 1 - Overview