Dear Charlotte (Part 1 of 2)

A Love Letter

Charlotte Hatherley, King Tut's Wah Wah Hut, 18 September 2009.

Thank you Charlotte, for a wonderful evening; I had the time of my life.

When we arrived around 8:30, my wife Linda and I were greeted at the door by half of the other members of the Charlotte Hatherley Fan Club that I've started in my office, and his pal. Let me say right now, your fan base expanded this night.

I do hope you enjoyed our humble Glasgow venue, despite a couple of teething troubles getting the sound set up for you. In addition to being a great wee pub with its own special honey-flavoured lager and a great line in curry, King Tut's Wah Wah Hut is "a legendary showcase for new and emerging talent", local and otherwise. Tonight you were preceded by Uddingston's finest: the jazz influenced, piano led and bassy Nespresco, who were followed in turn by the very freshest cream of Auld Reekie's School of Art: the acclaimed, happy electro Futuristic Retro Champions. Their irresistibly singalong Jenna was, as always, such a feelgood highlight of their set - hope you had a chance to catch it.

But this night was yours, and from the moment you came on stage, cursing like a trooper against those equipment problems, then like a trooper, playing through it all regardless, I felt in awe; the presence of rock royalty.

Dreamatis Personae

Supported by The Crimea’s lead guitarist Andy Norton on bass / guitars, and the brilliant Alex Thomas (Squarepusher, etc) on drums, you flew rapidly through a quartet of new songs: Colours, Full Circle, New Worlds, Little Sahara.

I sang along with you on all but the last of these, having stalked these songs continuously on the internet for many months, checking for posts and torrents several times each day, until I'd build up about half an album's worth...

My amazing wife Linda is very well trained. Her slight frame flew back and forth repeatedly at the front of the stage, ensuring that I would at the end of the evening, with zero effort on my part, have literally dozens of great photographs of not only "my girl" as she calls you, but also your accompanying musicians, in fast and fluent action.

"Those were new songs, this one's older. Wounded Sky," you announced, as I yelled "Oh, yes, please!" recognising a personal favourite. The softer tone continued back into your new material with the intro to forthcoming single Alexander, followed by Straight Lines, then back out for a trio of older favourites, Behave, Sister Universe (wondrous surprise!) and - after leaving the stage and teasing us with feedback for a few minutes - the brilliantly rocking and immortal, kilogram plus of perfect pop-rock, Summer.

Encore!

My beautiful wife Linda, dedicated companion and veteran of many a Yes gig, is very well trained, as I might have mentioned already. Knowing my memory for a setlist, she made sure to snap the one on the stage front. Not to mention snaffling it the very second the gig ended (erm, yeah, sorry about that...)

The encore continued with pace through another couple of new songs, the masterfully hook-laden White, and the beautiful Cinnabar, finally ending with your first ever solo single, 2004's tribute to Kim Wilde.

You were so very gracious during signing after the show, where I waited at the end of the line, to have my earlier purchase New Worlds autographed. It was particularly pleasing to see half of the other members of the Charlotte Hatherley Fan Club that I've started in my office, and his pal, clutching a just-bought copy of Grey Will Fade and queued up for signing!

My devoted wife Linda is very well trained. Who else would have pestered you repeatedly, against my disingenuous protestations, so that I could have a couple of pictures beside "my girl"?

After The Garage And Mr Chips

We were still playing New Worlds on the car stereo, as we pulled in to the Barrbridge McDonald's 24-hr drive through, around 2am (it's a long story, but see para header for a hint). Linda remarked that the CD lyrics could have been printed a little more clearly, for lighting conditions such as these, while I tried to explain how I love your music for the music - though your lyrics are certainly a lot better than most, they're not part of my infatuation with your work. See, it's all about the choppy chords, creative fills, beautiful original melodies, and the geeky patterns hidden behind all that.

So, next time: exactly how I fell for Charlotte, in excruciating detail.

All photographs copyright © 2009 by Linda Kerr.

Security 101: Part 3

An Introduction To The Microsoft Security Development Lifecycle (SDL) (Concluded)

Just before we jump into the second half of this introductory article, did you know that the Microsoft Security Development Lifecycle has its own blog? Written by an 8-strong team of SDL / security development managers, this is probably the best place available to keep up to date with the latest news regarding the lifecycle itself, and the tools and resources available to help us with its integration and day-to-day use.

Phase 3: Implementation

"During the implementation phase, the product team establishes and follows best practices for development then enforces the best practices during software development."

• Specify tools
• Enforce banned functions
• Static analysis
  

Take tools, for example. There is available, a constantly expanding array of second- and third-party tools which can help secure development; build tools, code analysis and coverage tools, and so on. Microsoft use and recommend a set of FXCop security rules (actually different sets, depending on your development environment version). It's one aspect of best practices, to keep up to date with what's available in these areas. Similar remarks apply to the available guidelines on the use of these tools.

Choice of a managed code language and environment is another such aspect. When this is not an option, one way in which Microsoft has tried to reduce the vulnerable surface area, has been to ban certain frequently exploited APIs - such as unsafe string buffer handling functions - in unmanaged C or C++ code. Published lists of such recommendations are available.

Rules to prevent SQL injection attacks are also included in this development phase. In a similar vein, JavaScript developers should generally avoid the use of the eval() function!

Web Development

The current MSDN documentation for this area of the SDL contains many additions labelled "(New for SDL 4.1)", and it is interesting to see just how many of these relate to web development - it's a very high proportion. This is still more interesting in the light of a recent SANS Institute Report, which found that most organisations presently focus on OS patching, whereas 60% of all attacks are on web apps.

Documentation

Trustworthy Computing mandated that the default configuration for a software package, landing on a user's desk (or lap!) for the first time, should be a secure configuration. However, users also need the ability to mess with their security settings, for example to change their defaults to something that better suits their particular environment. Documentation of the security settings, in both configuration and deployment, therefore becomes a deliverable, allowing such decisions to be made in a safe and informed way.

Phase 4: Verification - Fuzz is the Buzz

"The verification phase is the point at which the software is functionally complete and is tested against security and privacy goals outlined in the requirements and design phases."

• Dynamic/Fuzz testing
• Verify threat models and attack surface
  

To ensure that code meets the security and privacy targets set in earlier phases, we require thorough security and privacy testing, and a security push, followed by a privacy review of the release candidate. Here we are concerned with the classic CIA of information - Confidentiality, Integrity, and Availability.

Only the test process is capable of guaranteeing that the system will remain secure in the wild, and one key to this process, given some emphasis in the MS scheme, is Fuzzing. Once again, there are tools available which will randomly generate more test data than you ever wanted to see, and do it intelligently and with expert knowledge of the type of channel: command line, file, database, URL, script, image, and so on. That's what fuzzing's all about. There are RPC fuzzers. There are ActiveX fuzzers (yuk). I could go on...

This SDL blog article contains a link (.zip) to a simple file fuzzer, suitable for use by novices. It also has another to the "BinScope Binary Analyzer", which MS teams have used in one form or another since 2002. This little beauty checks for loads of security-related stuff in your binaries, and integrates well with both VS2008 and Team Foundation Server. That, and the price tag (it's free), should just about sell it.

Phase 5: Release

"The release phase is when you ready your software for public consumption, and you create plans for post release servicing of the software."

• Response plan
• Final security review
• Release archive
  

The product is subjected to a final security review, and a final privacy review, prior to its release. The results of these activities are fed into a Response Plan - that's the plan of action implemented when post-release vulnerabilities are discovered.

[Don't you mean "if"? - Ed.] [Only joking. - Ed.]

The privacy review may take its structure from a preset SDL Privacy Questionnaire. It may require validation by a privacy advisor or legal representatives, and the drafting of a privacy disclosure statement or statement of compliance.

The Final Security Review should be a timely (say 4-6 weeks before release) and comprehensive review of known threat model vulnerabilities.

Response Planning

Even when your release is into a world containing no particular threats to your new system, such threats can emerge later. And similarly, the privacy goalposts can be moved by the emergence of some new privacy advocacy. In short, you have to plan for contingencies.

Know and document publicly who is responsible for dealing with the different types of issues that may arise. Have a policy in place to handle cases where these issues involve third-party code components rather than your own.

Don't forget to lock all the doors, and cancel all the holidays.

Response: Examples

For examples of security response at Microsoft, visit the Microsoft Security Response Center (MSRC) Website: http://www.microsoft.com/security/msrc/default.aspx

Next time, and for the remainder of this series of articles, I'll be looking at some particular vulnerabilities, some quite public and occasionally spectacular, and their responses. I'll also be covering various security- and SDL-related tools and other resources. This is where it starts to get really interesting...

How To Get Phished

Too Much Information

Bosnia and Herzegovina, Croatia, Macedonia, Montenegro, Slovenia and Serbia, including the autonomous provinces of Vojvodina and Kosovo, were until 1991 all grouped together under a single country name, Yugoslavia.

They had one Air Force, in which my friend was a jet fighter pilot. Around the time of the great breakup, he moved to Scotland. Here he spent some time as a local council gardener, before starting, along with two boring accountant types (their own words), his own Computer Systems sales company.

With that background, you'll be unsurprised to hear, he was indisputably the most eccentric member of that group. And so yesterday, I was equally unsurprised to receive the following MSN message from him⁽¹⁾:

HAHAHA LOL ?!?! OMG!!!
http://Uncovered-Photos.com/?user=john.kerr&img=DSC134.JPG
http://Uncovered-Photos.com/?user=john.kerr&img=DSC137.JPG
http://Uncovered-Photos.com/?user=john.kerr&img=DSC140.JPG

Here we go, I thought. He's Photoshopped my face into some German watersports pictures, or something similar. That crazy guy, always SHOUTING, this type of nonsense is just absolutely typical of him!

I forgot about it until today, when I noticed him logging in. It had been a recognisably genuine message from a known, reliable source; so I clicked on the first link.

Hello, what's this?

"Reported Web Forgery!" replied FireFox⁽²⁾.

Remarkably, I'd already become so convinced that the original MSN message was real, that I then took note of the warning, and still clicked through (using the handy "Ignore this warning" link at the bottom right); fully expecting to discover some new Web 2.0 mashup or spoofing technique he'd recently mastered, and wanted to show off. What I found instead was a login screen. Only then did the proverbial penny drop!


The slightly ungrammatical prompt wasn't really a giveaway, since of course English isn't my friend's first language. It was just the fact that I was being asked to provide my login details, without having any clear understanding of exactly why these would be needed in order to show me, what I'd assumed was going to be, a few vaguely dirty and not-very-funny pictures.

Well, no thanks...

I tried contacting my friend by phone, but there was no answer. Using MSN, I then got him to identify himself by answering a couple of questions, after which I conveyed that his MSN account was compromised, and he should change his password.

Later, after researching - ok, Googling - the issue, I went through the handshake protocol again, this time advising an immediate and full antivirus scan. The exploit already seems to have quite a number of variations, some of which might be more malicious than others.

As an example of social engineering, this exploit owes much of its near-success with me, to sheer luck. The style of the SHOUTING, the rest of the message, and the implied content, all of these were just so absolutely typical of the person that the message purported to be from. That was a pure coincidence: nobody else I know could even conceivably have sent that particular message. Still, it reinforces the need to be on guard - at all times.

⁽¹⁾ Obviously I've mangled the actual content, including the site address!
⁽²⁾ The same operation in IE8 gave no such warning.

New Porcupine Tree Leaked

The Incident

After this, things will never be the same again.

Edited 23 Sep 2009 to add: Fame (and chart success) at last!

About bloody time! The new Porcupine Tree album, The Incident, has finally leaked all over the hinterpipes.

I won't lie to you Marge, but having already pre-ordered the £60 Limited Deluxe Edition Box Set some months ago, I felt just a little entitled - and what's more, thanks to that meddling 3-strike buffoon Lord Mandy, very motivated indeed - to fire up ye olde Bit Torrent client, and deeply to inhale⁽¹⁾.

This is fantastic. Steven Wilson's songwriting skills have taken an upturn from the band's previous outing, the Grammy-nominated Fear of a Blank Planet. There's great stylistic variation among both the fourteen parts of the main 55-minute song cycle on CD1, and the four "bonus" songs on CD2. Most noticeably he has averted his course ever further away from the lure of predictable, empty commercial formulae, writing solidly and innovatively, and perhaps benefiting from a confidence boost after the recent release of his own momentous solo effort, Insurgentes.

Drummer Gavin Harrison and bassist Colin Edwin are both typically understated, though perhaps not apparently so. Gavin has won Modern Drummer magazine's readers' poll, "best progressive drummer of the year", in 2007, 2008 and 2009. He plays rock percussion in a very jazz-influenced mode, so even his restrained playing can sound quite busy. Colin is also reined-in with Porcupine Tree, compared with his solo work: prohibited from using both hands on the fretboard, using the Chapman Stick, and so on.

Ambience comes courtesy of ex-Japan keyboard player Richard Barbieri, whose accompaniments add texture and continuity to the song arrangements. And there are plenty of harmony vocals too, which means lots for "fifth member" John Wesley to do, as he joins them once more on tour.

Steven's production work is slick as a Steely Dan - unsurprisingly, as this is his favourite part of the creative endeavour - and doggedly old school. The Grammy nomination garnered by FOABP was for his masterful surround-sound mix. Since then, he has begun the job of remixing classic King Crimson into surround sound releases. One more compelling reason to go for the deluxe package here!

Now, if only there was a comparable demand for the new Charlotte Hatherley, we could all stop worrying about the future of great songwriting.

⁽¹⁾ Please support your favourite artists by always making legal purchases of their official releases!

Poetry Corner

Broken Haiku available too - phone for a quotation

A software developer one day
Thought limericks might be his forte
But he'd no patience for
Specifications
.

1st Principles: Curve Fitting (2)

Loose Ends of Quintic Trends

I was asked to fill in the blanks left at the end of the previous Curve Fitting article, so just quickly, here goes. Last time we assumed a linear mathematical model, y = mx + c. This time we start off with a more general one, y = p(x), where function p is assumed to have one or more adjustable parameters that we can tweak to find the best approximation.

As before, we first plot all n data points (x,y) on a graph, and conceptually draw the curve represented by our function p(x) somewhere through the middle of them. Then the sum of the squares of the error terms is,

∑(∆y)² = ∑(p(x) - y)².

For each parameter t in model p, we partially differentiate this sum and look for turning points:

∂∑/∂t = 2∑(p(x) - y)(∂p/∂t), which equals zero when

∑(∂p/∂t)p(x) = ∑(∂p/∂t)y.

Now let's plug in a concrete example, in fact let's make a beeline for that bloody quintic. This has six independent parameters, which we'll label a through f:

p(x) = ax⁵ + bx⁴ + cx³ + dx² + ex + f

We simply read off the six partial derivatives, and substitute each of these in turn into the above formula.

∂p/∂a = x⁵; ∂p/∂b = x⁴; ∂p/∂c = x³;
∂p/∂d = x²; ∂p/∂e = x; ∂p/∂f = 1.

a∑x¹⁰ + b∑x⁹ + c∑x⁸ + d∑x⁷ + e∑x⁶ + f∑x⁵ = ∑x⁵y
a∑x⁹ + b∑x⁸ + c∑x⁷ + d∑x⁶ + e∑x⁵ + f∑x⁴ = ∑x⁴y
a∑x⁸ + b∑x⁷ + c∑x⁶ + d∑x⁵ + e∑x⁴ + f∑x³ = ∑x³y
a∑x⁷ + b∑x⁶ + c∑x⁵ + d∑x⁴ + e∑x³ + f∑x² = ∑x²y
a∑x⁶ + b∑x⁵ + c∑x⁴ + d∑x³ + e∑x² + f∑x = ∑xy
a∑x⁵ + b∑x⁴ + c∑x³ + d∑x² + e∑x + fn = ∑y

There we have our six explicit simultaneous equations in six parameters; solve 'em! Use any decent math library. Or, if there's a cow peering into your Portakabin™, and all you have is Commodore Pet BASIC, take advantage of that 30KB of RAM to knock up a quick Gaussian elimination.

From memory, of course.

So Long PCW, and belatedly, Sub Set

It's The End Of The World

Groundbreaking and trailblazing, Personal Computer World magazine, at age 31 the UK's first and oldest, has finally ceased to be.

The August 2009 issue (published on 8th June) gave no indication that it would be the last. It even contained the usual advertisement for the "next" issue.

Lifelong contributor Guy Kewney has already provided the definitive, if typically idiosyncratic, personal obituary. But I want here to say a few words about one of the magazine's lesser known innovations.

I'm not going to claim that PCW was my lifelong companion. My friends and I were already a few years into our careers in microelectronics and computing, thanks to the National Semiconductor SC/MP prototyping board system (1974-6) among others, well before PCW first appeared on the newsstands in February 1978.

Needless to say, the "Scamp" was uniquely innovative for its time. For example, it was the first system to provide for multiprocessor architecture--up to three of these processors could share the same address and data buses. Not that we first year undergraduates could afford more than one between three of us!

Through magazines like Elektor and Practical Electronics, carefully collected and stored in custom binders, we all learned the black art of machine code programming. And while I still owe PCW's editors my greatest debt in that lowest basement of Babel, actually they gave me my first "break" by publishing articles in various other areas; articles about micros, pocket computers, and higher-level technical subjects. In 1980, I bet a friend one pint of Guinness, that I could get a line drawing of a locomotive engine published in PCW. Maybe you can guess the subsequent shape of the state diagram in my "Complex Number Calculator" for the Sharp PC-1211 pocket computer. [Update, 12 March 2011: see it here]

I Know All About Them

Thus spake Spock, when asked "…You know a great deal about computers, don't you?"

The instruction sets of these early up-to-8-bit processors, while not exactly RISC, were quite manageable. By which I mean, anyone could quickly become a master machine code programmer; able to write on a first pass, a perfectly optimized program for any given task. And like Spock, to do it directly in binary, without the need for an assembler (compilers were things the universities used on their minis and mainframes). The "secret" was simply to take the full set of opcodes for the processor, each corresponding to some ordered pair of hexadecimal digits, and construct an "instruction map" - a 16x16 chart - with the digits 0-9, A-F along each axis. This could then be memorised quite easily.

The Zilog Z80 (Tandy TRS-80, Sinclair ZX-80/81/Spectrum, Amstrad CPC-464/PCW) yielded the largest map; firstly because it used up all 256 available opcode bytes, and secondly because it used several of these as prefixes for 16-bit instructions, which obviously opened up more maps (more pages), extending the instruction set of its progenitor, the Intel 8080, e.g. to include new operations utilising its new index registers. Other favourites were the 6502 (Aim-65, Commodore Pet/Vic-20/64, BBC Micro) and of course the Motorola 6809 (in Mettoy's Dragon 32 home computer), the first to include a Multiply instruction.

PCW Sub Set

Apart from this token attempt by the 6809, none of these 8-bit processors had much out-of-the-box support for adding and subtracting numbers above 127, never mind multiplying and dividing them, or allowing fractions. Even when supplied in the form of a home computer, i.e. with interpreted BASIC in ROM, their arithmetic and other subroutines were often very far from optimal.

Personal Computer World provided a great service to the micro development community via its Sub Set series of articles, curated initially by Alan Tootill, joined later by David Barrow. Here we could all meet to set down the gold standard for 32-bit arithmetic routines, signal processing and interfacing, random number generation, development tooling, and a hundred other hotly contested library projects. Simultaneously, the Datasheet format provided a respectable standard for documenting material of this type.

I became a regular contributor, Alan and David frequent pen pals. When I eventually landed my first full-time job in the industry, it was purely 100% as a result of the work published here, and 0% thanks to my undergraduate degree.

Later I'd be tempted more than once, while en route to maybe York or London, to arrive on David's West Yorkshire doorstep with an oft-promised bottle of Talisker. As yet undelivered, I regret. Well, that would have looked like bribery, wouldn't it?

Sub Set was a viciously competitive environment, a forum for developers throughout the UK and beyond, where regular challenges were posted, and we raced to submit the fastest or shortest solution, or to improve on an earlier one. Having one byte trimmed, one clock cycle shaved, from your code - this could mean utter, instant, humiliation.

Naming Isn't Everything

Once I complained to David about his choice of a variable name. His response was the most beautiful utility routine I have ever seen for any processor. It was for the 6502, a device severely impoverished in terms both of instruction set, and of memory addressing modes. It virtually rewrote the silicon. What it did was to expose full leverage of an obscure debug mode "Break" instruction, transparently to add (1) streamlined interrupt processing, and (2) a brand new, unbelievably useful, relative address mode "Call" pseudo-instruction. Not content to name his routine BIRCH (for Break, Interrupt & Relative Call Handler), David went on to supply five-letter tree names, each somehow meaningful, for every label in that code.

Century Communications eventually published their Best of PCW: Assembler Routines for the 6502, and the companion volume for the Z80, in book form. For a little while, I knew the pleasure of getting regular royalty cheques through the post, thanks to machine code subroutines... that I'd already been paid for!

Again, thank you PCW. I'm still pleasantly surprised even today, when surfing for fruit scone recipes, to chance upon namechecks and acknowledgements in projects like MAMEs and homebrew emulators (although "Contains the John Kerr disassembler" nearly scared the shine off of me, first time I saw it).

Coda

Like type-in listings of text-only adventures, Sub Set was discontinued some years ago. Since then, machine code has actually remained surprisingly manageable on a human scale, partly due to the emergence of RISC and the ARM, P-Code, and now MSIL and the Jitter; but the quantity has changed, and all the short essential jobs have been done and documented. Much of what we did back then, is now performed in multicore cathedrals of silicon, and even embedded systems developers have any number of compilers to choose from.

For a lot of hobbyists, the demise of PCW Sub Set was one sign of maturity in the industry, similar to the emergence of the standards-defining IBM PC and its Microsoft operating system, begetter of many clones, that signalled time to move on to something more interesting. Leaving the machines, and their league of magazines, to continue a headlong rush into their demarcated business and gaming arenas. And their ultimate obsolescence.