Case Study: Auntie's Virus

When Free AV Goes Wrong

When there's PC trouble chez Aunt M, I usually have up to a week to sort it out, before it becomes critical - before, that is, her next weekly Sainsbury's Online order. Such was the case this week, when I battled bravely against Pakes.AV, a Trojan she'd managed to acquire from an XSS attack at some Canadian newspaper's site.

For her Internet connectivity requirements, Aunt M has always been with Virgin Media. They've offered free antivirus software for several years, and we've both found it quite reliable in the past. But then it began to get into a bit of trouble. Resource usage began climbing, and even the expedient of replacing both our PCs with brand new ones couldn't shake off the barnacles.

Eventually, about a year ago, her PC could run little else except a background scan: she was effectively locked out of her PC for hours on end. But still it got worse: the free AV software updates began failing. At this point, the PC's behaviour on power up became utterly monotonous and predictable; thrashing and churning, crashing and burning. Big red button. Repeat.

At one stage I had installed 8GB of RAM into her new machine. Yes, apparently in 2010, Windows needs more than eight million kilobytes of memory, if you want to browse to an occasional more or less static website, or to send or receive one or two emails a day, without having your bank account emptied by Russians.

Free As In Beer

After researching the various free antivirus alternatives available on the web (okay, after searching for "best free antivirus", then selecting the top result from the top comparison site), I had installed AVG Free, whose website trumpets Trusted by Over 110 Million Users!

Also as an experiment, and because the purchase of a new Dell had resulted in my 12-month free McAfee evaluation becoming lost or truncated to 3 months (leaving Sky Broadband's and McAfee's licensing support departments pointing ineffectually at each other), I had also installed AVG Free on my own computer.

Today, my Windows 7 installation continues to work flawlessly, but as I said, Auntie acquired this little nasty bit of code last week. Over the telephone I got her to run a manual scan, which appeared to clean up the problem. As I half expected, the infection reappeared next day.

Another manual scan, and another successful outcome. Left instructions not to visit any websites except Sainsbury's and BBC News. As I three-quarters expected, the infection reappeared next day. She is now unable to start another scan, or do anything else really, including making a Remote Assistance request.

Isn't It Ironic

Remote Assistance can be a great help at times. But on this occasion, the frequent and insistent popup behaviour of the AVG threat notifier itself made the "cure" at least as intrusive and debilitating as the malware. This is also hindered by the relatively loud UI design, the strong colours of which make it difficult for her even to identify the windows on the screen; selecting one of these windows, such as the XP Remote Assistance dialog; bring it to the front (impossible as it happens, since the notification window has the Topmost property); and then use it to allow me remote control of her PC.

I'm not saying that all free software is worth what you pay for it. Regular readers know I'm a huge Linux fan! But the usability requirements of an elderly user seem to dovetail with those of the support engineer attempting to extend help without the benefit of remote assistance and control. It's become clear to me that certain free AV offerings are still designed using a 1990s approach, agnostic to accessibility constraints, and in fact with cursory regard to usability.

Anyway, I'll have to go over there to take a look...

Site Visit #1

Pakes.AV is essentially a search redirection infection. So my first problem, as I began surfing for advice or support from the AVG Free forums, was trying to deflect the incessant torrent of popup ads.

My second problem was discovering that, after battling a path clear through these, I'd been rerouted in flight to a collection of pages offering loans, mortgages, pharmaceuticals and pornography. Quickly scribbling down all of these useful URLs, I proceeded to the AVG site...

My third problem was finding that the AVG Free forums were an unhealthy mess of diagnostic dumps, cross-posted bad advice usually related to quite unrelated issues, poorly written and/or indecipherable instructions, and a proliferation of people with a polymorphic variety of official-sounding titles, making it hard to identify any authority.

I did manage to start a full scan with some additional options switched on. Then I left to continue my researches at an uninfected workstation.

Reflections

What little I could decipher online about the problem seemed to mention the Trojan hiding in Windows System Restore, which should therefore be disabled. Hmm, suspicious, I thought: an antivirus forum, recommending that I disable a feature designed to recover from trouble?

But as I was assuming the worst anyway, i.e., that a complete reinstall of Windows would be at the end of this trail, so I first tried restoring the system to a date in May, when I figured there had to be no infection. Sure enough, as I'd seven-eighths expected, that Trojan horse came right back. So then I bit the .22 slug, disabled System Restore, and scanned again. Did that work? Neigh.



Let's Go To Work

After some consideration that night, I'd decided that a 30 day evaluation of Kaspersky would be the way forward, so I downloaded that on to a USB key drive, ready to take on site and install next day.

Installation on the uncleaned system proved to be a bit of a challenge, as the machine was now in a fit of spawning, flinging up hundreds of command windows on startup. But I persevered, reasoning that Pakes.AV was so long in the tooth that Messrs Kaspersky would by now have a comprehensive treatment for it. And if not, then I could consider the more clinical cleansing and/or full Windows reinstallation options later.

My confidence proved well placed, as a preliminary scan resulted in two, reset-requiring, "special disinfection" procedures becoming invoked. One full scan later, it emerged that some 30+ assorted viruses and Trojans had been jostling and writhing on Auntie's PC, for who knows how long; all unseen by AVG Free.

Conclusions

Most of which were already known, and obvious in hindsight:

1. As with baked goods and beer, so with antivirus software: you get what you pay for.
2. Sometimes Remote Assistance alone just isn't up to a disinfection.
   
3. Never assume that the visible threat is the only one present.
4. Have a security strategy - and review it regularly.
   

I'd also like to add my personal opinion, that Kaspersky is a class act. I've had experience with, and eventually reasons to drop, antivirus software companies such as Trend Micro and McAfee. Admittedly these were reasons related to licensing and support, rather than the technical performance and update issues that have forced me to abandon the several free offerings that I've also tried. But then again: support's the bit you pay for!

Everything about Kaspersky's software, from its UI design to the company's web presence and security lab blog, inspires confidence. I wouldn't be surprised if I never have to contact them for support; but if I do, I'm also confident that experience will be exemplary. Thirty days hence, they'll have two new paying customers.

Bejeweled Again

Unshakeoffable!

Earlier this week, colleague C inflicted a couple of earworms on me. First there was Tangerine Dream's soundtrack from the motion picture The Thief (particularly Diamond Diary and Burning Bar). Then - new to me - came Psychotic Photosynthesis, by Omar-S.

Knowing C regularly reads this blog, I've decided to exact my revenge on him, using Skaven's music from the appropriately addictive Bejeweled 2: "Beyond The Network" (2004).



It takes a couple of playthroughs. And then, it becomes a big mystery, how Skaven manages to evoke an ethereal 80s quite so strongly: a decade of mediocre electronica, set in a new romantic milieu. I thought I'd managed to avoid it entirely; abstract orchestral music and I were exclusive in those days. And yet somehow, listening to this sprawling and hypnotic, 40 minute ambient synth suite, I can hear the entire early catalog, not of say Kraftwerk, as might have been expected; but of Simple Minds.

Often I've been tempted to ask Skaven, real name Peter Hajba, real country Finland, "What's your secret?" But it would be insulting, even to suggest that any one "secret" might account for the multifarious effects of this tour-de-force, with its many hooks, its forgivably cheesy bits, the remorseless grip of that so-well informed nostalgia. And besides, from Future Crew News at the time of the game's release, we have from Skaven's own honest account: the music "represents literally years of work, bringing together new ideas, old ideas, inspiration and horrible clichés."

A flawed masterwork, then (my highest accolade).

Common Table Expressions

Waiter, there's a Table on the Fly

Common Table Expressions (CTEs) have been around for a while, since having been introduced into SQL Server 2005. Basically they provide a way to create a temporary working table, stuffed with useful calculated data, which can then be accessed from an adjoined query just as though it were a real physical table in the database.

Now, the particular power of Microsoft CTEs comes from the fact that they can be built recursively. This fact has made them a very popular choice for interrogating hierarchical, tree-structured data, for example in tables using the ID : ParentID method of auto foreign keys. A few lines, a mere couplet of SQL, can conjure up a temp table containing any given row together with all of its descendants.

The question of how best to implement storage of hierarchical, tree-structured data is an ancient one, and there have of course been many solutions to it. Joe Celko's SQL For Smarties describes in detail an alternative, fascinatingly mathematical approach with some important performance benefits. I have used this on some occasions, recursive CTEs on others, and still other alternative approaches as appropriate when non-functional requirements have dictated.

Lighten Up

But I've probably been a bit blinded by the efficacy of CTEs in this particular context, to their other potential uses in simpler scenarios. This occurred to me today, when a colleague asked for some advice in building certain dashboard metrics as SQL queries. After a little too much black coffee, I found that while part of my brain idled along a path of conventional JOINs and nested SELECTs, another part instead began to wonder how a procedurally oriented developer, habitually reluctant to think in relational terms, might be able to build such a compound query out of its perceived components.

Set-relational operations are combined in a fundamentally different way from sequential ones. This has ever been clear to the SQL school, but is only now, with the advent of LINQ, beginning to receive much more attention from the procedural. It seemed to me at that moment, that CTEs might have a role to play in this ongoing adoption.

Quite often, people might correctly identify and perform a promising initial SELECT query, projecting their data from source tables into a "shape" somewhat closer to their desired output; but then they immediately run up against SQL's unfamiliar syntax and context restrictions, as soon as they try to take step 2. This appears to be an exact analogue of the notorious "impedance mismatch" between the coding requirements of relational storage, and those of the business object environment.

But what if we simply take that intermediate projection, construct a CTE out of it, and use that as our starting point for the next step?

Composable Queries Again

In practice, there are no significant limitations upon what can be done with the CTE contents, at least as far as our procedural developer is concerned. It really can be treated as just another table, containing data structured in a slightly more goal-friendly way than was previously available. Now just lather, rinse and repeat.

The forementioned colleague's problem was: obtain a result set containing the percentages of cases successfully closed, after a given number of communications, where the communications dimension was striated into "less than two", "two or three", and "four or more".

From the database, we use only an intermediate Communication table, which connects cases to communications (so each row has an integral Comm_CaseId and Comm_CommunicationId). Here's how CTEs allow the answer to be built up piecemeal. The first CTE, "Totals", simply obtains a count of all the cases, to allow us later to calculate proportions. The second, "Comms", aggregates the cases as required by our striation, delivering each case ID in one column, together with a corresponding category letter in another. The third CTE, "Cats", then further aggregates these categories. Finally, our main SELECT combines this output with the previously computed totals to arrive at the required output:

WITH Totals(Total) AS
(
  SELECT COUNT(DISTINCT Comms.Comm_CaseId)
  FROM Communication Comms
),
Comms(CaseID, Cat) AS
(
  SELECT
    Comms.Comm_CaseId,
    CASE COUNT(Comms.Comm_CommunicationId)
      WHEN 0 THEN 'A' -- Category A means 0 or 1 communication(s).
      WHEN 1 THEN 'A'
      WHEN 2 THEN 'B' -- Category B means 2 or 3 communication(s).
      WHEN 3 THEN 'B'
             ELSE 'C' -- Category C means 4 or more communication(s).
    END
  FROM Communication Comms
  WHERE Comms.Comm_CaseId IS NOT NULL
  GROUP BY Comms.Comm_CaseId
),
Cats(Num, Cat) AS
(
  SELECT COUNT(CaseID), Cat
  FROM Comms
  GROUP BY Cat
)
SELECT
  Cat AS 'Category',
  100 * Num / (Total + 0.0) AS '%'
FROM Cats, Totals

Non Hollywood Ending :-(

In the end, my colleague reverted to traditional Join based ANSI SQL code for her problem, as that turned out to be easier to decorate with the auxiliary columns she needed for the purposes of her BI analysis. Still, it was an enlightening exercise.

I Killed My Wife...

...'s Interest in Cookery Programmes...

...by the simple expedient of pointing out - with the honourable exceptions of Keith Floyd R.I.P., and Rick Stein - how frequently every TV chef utilises (and almost always incorrectly) the word "literally". For example: "Then the Kiev will literally explode with a garlic aroma." Sounds more like Chicken Chernobyl, than Kiev.

It really does appear to be the sole quadrisyllabic entry in their cooking lexicon. And now, when any of her favourite cookery programmes comes on the telly, she just can't stop herself from playing "Literally Bingo". Truly I have completely ruined one of her favourite pastimes.

TV chefs aren't the only people who both habitually overuse and entirely misapply this word. There are also those AWFUL Americans (that's not just some random insult, incidentally; if you haven't met it before, it's the acronym from Americans Who Figuratively Use "Literally").

Here's David Mitchell complaining just a little about their Queen's English usage habits:



Actually I think I've finally made my peace with the American "I could care less"; let's just assume they've dropped an initial "As if", leaving it now silent, and implicitly understood. Hereby is logic painlessly restored. This was not easy for me to do, because a favourite variant (to which I claim authorship) runs thus: "I could care less, but it would take more effort."

Yet one greater bugbear of mine, with these darn Yanks, still remains: their use of "dumb" (mute, or vocally challenged) to mean "stupid" (Republican)...

Anyway, good luck in the 2010 World Cup, America, and always try to remember that in spite of the name, other nations are also competing. Who's that you're playing first? Engerland?

Go USA!

Update: Though we may have hoped and dreamt, still we never truly believed you would deny our Auld Enemy their opening win - especially after you went down in the 4th minute. Apparently. I was watching on HD, where it actually ended USA 1-0 England! So, For The Win: well done guys.

Also, still finding it hard to believe that we willingly sat through an England football match, while Scotland's rugby squad - away! - were so historically hammering Argentina on the other channel. I could care less! Literally!

Algebra of Functions: Pause for Breath!

Previously...

In Part 1, we started out with just the operation of functional composition, and derived the Monoids - composable functions of a single input and output parameter type.

In Part 2, we generalised these, by allowing as much type variation as practicable. Retaining only the stricture that a given function's input be obtainable from the amplified output of its predecessor, we got ourselves the Monads.

In Part 3, we "discovered" the relationships between the monadic bind operation, the SelectMany method of LINQ, and C#'s way cool Query Comprehension from and select syntax.

Next...

There are a couple of candidates for Part 4. One article on parsers, another favourite monad example, is almost complete. Another on continuations is also on the starting block. I'd have to be honest, and say that the subject of continuations is of much more interest to people working in areas like language compiler design, and other more esoteric, academical researches, than to us average app-writing Joes. On the other hand, continuations certainly provide the best available proof that the applications of monads aren't confined to the "amplified data" related contexts we've looked at so far.

So I reckon it'll probably be parsers next, with continuations making an appearance after them. Meantime however, while I'm polishing off my parsers, I thought we might stop for a little bit of perspective.

What's The Point?

The earlier articles in this series have all been repeatedly revised, and usually extended, since their initial publication. The reason for this has been the absolute failure of people to stop asking me, "What's the point of this functional stuff?" Or, to rephrase that: my own personal failure, to provide sufficient background.

I thought I was being clever! - just revealing enough about the functional motivation, to cover each item (monoid, monad, or comprehension) as it emerged. In my experience, exposing a procedural or object-oriented programmer, even to the very jagged word "Haskell" alone, would send them running to the bar to order the next round. And while that behaviour has its own undeniable appeal (and obvious applications), it doesn't address the issues that functions do.

So, in a nut:

1. To make efficient use of multiple cores, we must parallel program.
2. Threading is difficult.
   
3. But multiple cores are not the future: they're here now. One week ago, Intel revealed the Knights Corner, pictured above: a fifty-core, 22nm processor. Try to program that without parallelism, and your code will never use more than 2% of the available hardware. Do you consider that acceptable?
   
4. The big, error-prone problem with parallel code, is the controlled synchronisation of state.
5. Functional techniques sidestep this issue extremely effectively, by quite simply avoiding state and mutability.
   

That's the case for functional programming: it's finally come into its own, out of academia and into the world of commercial software writing, because there's no mutable state to synchronise.

Monadic design quickly comes into this equation. We now need to rediscover how to program an entire swathe of tasks, which had previously made apparently irreducible use of mutable state. These situations include input, output, and concurrency; synchronous and asynchronous exceptions; and interfacing with libraries written in other languages, functional or otherwise. Haskell (and most other functional) programmers know these as "the awkward squad", and are fully conversant with their common solution, the I/O monad.

Back On The C# Track

So there it is, that's how I see our immediate multicore future panning out. This little series will continue for a bit, attempting to introduce a procedural, object-oriented audience to functional techniques, using the methods and facilities that have recently become available in C#.

However...

We used to claim that object-oriented programming could be done in almost any language. In fact I once proved this to myself, many decades back, by writing a commercial ATE system (Automated Test Equipment) in a purely object-oriented style, using nothing but the Microsoft procedural GW-Basic interpreter. Yes, seriously! The result was extremely satisfying, and an undeniable air of maintainability exuded from its conventions, when compared with operationally similar procedural scripts from other jobs. But it was also notably verbose, tedious in the extreme to write, and certainly could not be said to represent an experience which I would ever wish to repeat.

Today, we seem to be saying that functional coding can be practised in certain suitably enhanced O-O languages, such as C#. And it's equally true. It's just that you often have to jump through some rather ugly syntactic hoops, to arrive at an implementation that would be infinitely more tidy, elegant and pleasing to the eye and brain, if executed using a purpose-designed and built, function-oriented language.

Yes, it's fascinating that monadic C# functions can be composed just as readily as monoidal ones, despite the apparent impedance mismatch between their inputs and outputs. And yes, it's initially impressive to see these constructions massaged into (query) comprehension syntax, certainly an improvement on fluent method chaining. But that only goes so far, and even achieving what it does, is seen by many as nothing more than a mis-use of a comprehension syntax originally designed exclusively for SQL-style queries, rather than fully generalised relational domains.

There are strict and severe limits to what can be accomplished with ad hoc functional extensions to O-O languages generally, and C# in particular. For that reason, I give this series two, maybe three further articles at most, before its inevitable defection to the wonders of F#.

Update: Eric White has just published Recursive Descent Parser: A Simple Grammar, which is part 3 of his series on "using LINQ to write a recursive-descent parser for SpreadsheetML formulas" [sic]. I'm going to wait until the next two parts come out, as there seems to be every possibility that Eric is about to cover exactly the same ground as (and who knows, maybe even do it a little better than?) me.

Security Digest #9

Seriously, June already?

MSF-Agile + Security Development Lifecycle Process Template for VS 2010

VS 2010 shipped recently. If you've already upgraded, and want to integrate SDL processes into the development environment, then you'll be happy to learn that the MSF-Agile+SDL Process Template for TFS 2010 is now available for download.

MSF-A+SDL is a TFS process template incorporating the SDL for Agile process guidance into the MSF Agile framework. With the MSF-A+SDL template, any code checked into the VSTS repository is analyzed, to ensure it complies with SDL practices. The template also automatically creates security workflow tracking items for manual SDL processes, such as threat modeling, to ensure that these important security activities are not accidentally skipped or forgotten.

New features:

• Security dashboard giving at-a-glance summary of a project's current security development state;
• Bugs-By-Origin chart, analyzing effectiveness and return-on-investment of the organization’s security tools;
• Integrated security bugbar, like that described in the March MSDN Magazine Security Brief, to help non-expert users triage security bugs.
  

Full details from Bryan Sullivan at the SDL Blog.

Security Runtime Engine : May 2010 Preview now on CodePlex

The CodePlex WPL site now has the May 2010 CTP code only release for the Web Protection Library, and a Word document introducing the new extensibility points for the Security Runtime Engine.

What it doesn't have is binaries. This is a preview, not a production-ready release. Its chief reason to exist is as a kind of request for comments. The Security Tools Team is looking for feedback. The release represents a rewrite of the Security Runtime, and a new way to write plug-ins for it. Rather than simply decide what’s best for users, they want to show us the direction they’re taking, and give us a change to influence it.

Just Published: Silverlight Security Overview

Nick Kramer has officially published his paper on Silverlight Security. As Nick says, there is a lot of information on MSDN about Silverlight and security. However it can be hard to know where to start. This document describes how Silverlight protects end-users from attack by malicious web sites, and how to build a secure Silverlight application.

Nick's treatment gives us the lay of the land, to help orient ourselves, and figure out what details are important. It also gives an introduction to Silverlight's unique security thinking, for example why it's safe to allow sandboxed apps to open files (using the OpenFileDialog and isolated storage).

Here's the 212KB docx:

http://download.microsoft.com/download/A/1/A/A1A80A28-907C-4C6A-8036-782E3792A408/Silverlight%20Security%20Overview.docx

CERT Releases Basic Fuzzing Framework

(Via threatpost, the Kaspersky Lab Security News Service) Carnegie Mellon University's Computer Emergency Response Team (CERT) has released a "basic fuzzing framework to help identify and eliminate security vulnerabilities from software products". The Basic Fuzzing Framework (BFF), available here, is described as a simplified version of automated "dumb fuzzing" and includes a Linux virtual machine that has been optimized for fuzz testing and a set of scripts to implement a software test.

This is the second public release of a fuzz testing tool by CERT. Last year, the group released a tool called Dranzer that lets software developers test ActiveX controls for vulnerabilities before the software is released to the public. Dranzer remains available as an open-source utility.

A full explanation of the Basic Fuzzer Framework is available on the CERT blog.


No Hacking Required

Bernd Marienfeldt, Information Security Officer at LINX (London Internet Exchange), writes a private blog about subjects including IT and security. As a member of the Security Bloggers Network, he regularly covers such topics as Apple and the iPhone, as well as Ubuntu and other Linux flavours.

So it wasn't a complete surprise to see him credited for the discovery of this scoop:

Ubuntu Lucid Lynx allows full access to a non-jailbroken, PIN locked iPhone

It's a sobering revelation of a frequently recurring theme, namely the misapplication of security technology and the concomitant false sense of security that it engenders.

Bernd's own report upon this research is of course the most authoritative and utterly fascinating version.


That's all from The Padlock for now, so get out there and enjoy the summer - with apologies to my two Brazilian readers. Hey, that's a lot of readers!

Tweets - May 2010

40 years ago today, 4 unarmed students were murdered by soldiers in America for daring to protest an unjust war. Don't forget.

— Warren Ellis (@warrenellis) May 4, 2010

Farewell John MasonGlasgow East MPYour by-election winTook a hammerin'Red ClydesidersProtest insteadThe likes ofBloody David Cameron

— John Kerr (@dogbiscuituk) May 11, 2010

@wilw That's not a corollary, that's a coronary.

— John Kerr (@dogbiscuituk) May 29, 2010