Secondary Feed

Software

Here’s an old NBC news report speculating about technology in the year 2000. Apparently “something called the Internet” will be important. HT: Sorting out Science.

The Mono project, an open source rewrite of Microsoft’s .NET framework, is more mature than I thought. From Hanselminutes.

“Cloud” is a good metaphor for most of what I hear about “cloud computing” because it’s so nebulous. But Michael Stiefel has some solid things to say on the subject.

Python Infrequently Answered Questions

Quote from Word Aligned blog “One day software will be the most reliable component of every product which contains it.” — Tony Hoare. I’m not as optimistic as Mr. Hoare, or I at least thing “one day” is far away.

Joakim Karlsson says in his post The Locality of Code Changes “The probability that you will change a piece of code in the near future increases when you make changes to that code or to code in its vicinity.”

Economics

The best explanation I’ve seen for why newspapers are dying

Malcolm Gladwell’s rebuttal to Chris Anderson’s “Free” thesis

EconTalk interview with Mark Helprin on copyright

Math and statistics

In Is P = NP an ill-posed problem? Dick Lipton contrasts the Riemann hypothesis and the question of whether P = NP.

Visualizing correlations

Music, coffee, and physics

Classical music in cartoons

Latte art

Fun with an MRI machine. NB: The block is aluminum, not iron. Magnets don’t attract aluminum. But aluminum can conduct a current induced by a magnetic field. HT: Ovablastic.

What Makes Conferences Less Attractive

I generally eliminate conferences that

- overschedule event with no time or place for spontaneous meetings

- overcrowd, usually to maximize profits, with just too many people, which encourages people to hang out only with their old pals

- lack adaptability so opportunities pass by without notice or care

- offer too much lecturing, not enough interaction, and insufficient experiential work--or none at all

- invite presenters of widely varied and untested skill and preparation

- do not name their presenters in advance, or give biographical information

- provide insufficient time and space for socializing, meeting new people

- allow little or no interaction with the presenters (In some conferences, presenters eat in a special area, intentionally separated from the participants. In others, presenters speak and run.

- schedule sales pitches instead of teaching presentations

- schedule canned pitches instead of original material

- offer too many plenary sessions, when participants have no choice of what to attend

CAST

[http://www.associationforsoftwaretesting.org/drupal/CAST2009]

The sessions I attended were all of high quality and interest to me. Also, it's a reasonably small conference with numerous opportunities to participate in spontaneous hall sessions.

BizConf

[http://bizconf.heroku.com/]

AYE (Amplifying Your Effectiveness)

[http://www.ayeconference.com]

The June-pictures and Cottage-Life threads intersect in a photograph of a stiffly-serpentine beast that appeared on our beach. With a true story about a real serpent.

OK, I confess to finding and placing the little stone representing the critter’s eye.

Boys and Beasts

One time last summer, our son invited a friend over for a day of Cottage Life. They vanished into the woods, as is entirely proper for children of that age. I was under the cottage considering a recalcitrant water heater when I heard their voices, shrilly excited, approaching rapidly. Then Lauren’s, cool but firm: “That’s nice; now take it outside, boys.”

A few minutes later, a frightened shriek from not too far away, and the boys’ chatter returning to the cottage, faster this time. My son saying in a practical tone of voice: “If it was poisonous, you’ll die in about an hour.”

It seems they’d caught some poor little garter snake and, after showing it off to Lauren, his friend did something that provoked it to turn around and bite him — well, gum him, they have no teeth to speak of — hard enough to make him let go. His skin wasn’t even broken, and once they realized our sympathies were with the snake, the boys went back to the woods.

That’s the XML Summer School in September at St. Edmund. I can’t make it, in part because my wife is co-ordinating which means I do child-care. I’ve been to these and they’re totally great, intense and interactive and focused; then you get to go drinking around Oxford in the evening. If you’re within reach and work with XML and want to upgrade yourself, I totally recommend it.

• Zend Framework 1.8.0 Released

Share This

SonyInsider dug up an FCC filing that indicates that the forthcoming PSP Go will have a significantly faster top processor speed than than current PSP models. Specifically, the Go's CPU can clock up to 480MHz, compared to the 333MHz speed of the existing models.

The site ends the post by asking the obvious question: "What will a 480MHz PSP Go bring to the table?" I suspect the answer to this is, "Nothing that hasn't already been announced." Let me explain.

WE HEAR plenty of whinges about customer service in the airline industry, particular at the low-cost end. So it seems only fair to report on the good times, and that is why Gulliver would like to share a recent experience he had with easyJet.

I was part of a group of five passengers who had paid extra for “Speedy Boarding” on a return flight. On the outward leg we used the dedicated check-in line, and at the gate a public announcement told us Speedy Boarders to move forward so we could board the plane first. But on the return leg, at Lyon airport, there was no such announcement. We queued to pass through passport control, expecting to be fast-tracked on the other side. That never happened, though, and having waited with the other passengers we did not get on the plane first as we had paid to do.

Giving a yahoo e-mail address I wrote to easyJet—no easyTask, the website is a warren—requesting reimbursement for the Speedy Boarding part of our return leg (about £9 per passenger). Having sent my note at 5pm on a Monday, I was impressed to get a response the following morning.

The “Customer Experience Champion” (easyJet is to the naming of its customer-service representatives what Starbucks is to the naming of coffee-cup sizes) who dealt with my case offered sympathy, but no money:

I am also concerned to read that there was no announcement for Speedy Boarders, and that you had to board with the remaining passengers. However, I regret that I am unable to refund the cost of Speedy Boarding, on this occasion.

I hope that you will continue to be a loyal easyJet customer, and give us an opportunity to serve you again in future.

Moral of this story: if at first you don't succeed, write a tetchy letter. And credit to easyJet for being so rapid with their replies and for doing what I considered the right thing, at the second time of asking.

In a response to Richard Stallman's open letter warning about introducing Mono into Linux, Debian made it clear that no plans are indeed forthcoming to adopt Mono-based programs into the default installation of its free Linux platform.

Diamonds in the Sky

100 Most Beautiful Words

Thomas Christensen's Glossary of Book Publishing Terms

Another Glossary of Book Publishing Terms

In an age where your boss, coworkers, parents, and even (*gasp*) grandparents are finally joining social networks, we are all more aware than ever that we had better keep things relatively clean. And if you were someone who joined MySpace, Facebook, Flickr, or a number of other sites years ago, you may have more cleaning up to do than usual—after all, back then, you were probably young(er) and dumb(er), posting silly pics of your drunken escapades or questionable updates regarding your unusual interest in English cucumbers.

If you delete questionable images of yourself, you may be in the clear—or you may not, depending on the social network. As it turns out, some social networks delete your images right away while others hold onto them even after claiming they've been deleted. This was the discovery made by researchers at Cambridge University last month when they found that images deleted from social media sites are often left on the server, ripe for anyone to embed elsewhere or link up.

We put this finding to the test and found that some of the most popular sites on the Internet do, in fact, keep images on their servers after you delete them. On May 21, 2009, we deleted photos from four of the networks most used by the Ars staff and readership and monitored them for six weeks. The four networks we checked were Flickr, Twitter, MySpace, and Facebook.

The MILEPOST project for intelligent compilers promoted by the European Community is in a stable version 4.4 and a developer project was started to fine-tune the software.

Firefox 3.5 and Chrome already support the new video tag of the HTML5 specification. Before it can become a standard, though, more browsers makers have to agree on the codec.

I spent part of today driving on Interstate 5 up into Northern California. Most of the day was a blue sky day. Not a cloud in sight. I stopped off in Weed to get some dinner and when I got back outside and hit the road again, the sky had just erupted into this crazy sunset cloudscape.

iPhone 3GS • ©2009 James Duncan Davidson

This photo was a quick iPhone shot from the rest area just north of Weed. It’s a pretty impressive sight, no? To think, if I had spent any more time eating dinner, I would have missed the show. The iPhone photo here doesn't quite do the scene justice, however. The auto white balance algorithm (which always works against sunsets) slurped quite a bit of the intensity out of the yellows and pushed in too much blue. I’ve tweaked things back and forth a bit in Lightroom, but since it’s a JPG image, the basic color balance decisions are baked in and there’s not a whole lot of leeway.

Never fear. I didn’t just rely on the iPhone for this one. After I shot the photograph above, I pulled out the D700 and managed to get a few frames off before the sky opened up and dumped rain on me. And, of course, I shot in 14-bit RAW so that I could fine tune the white balance and capture a lot more of the range out of the scene. Here’s the result:

Nikon D700 • ©2009 James Duncan Davidson

That’s more like it. That’s the scene I remember looking at just a few hours ago. And yes, it was that intense. The iPhone did a pretty good job (especially for a tiny chip cell phone cam), but at some point, you gotta pull out the big guns. I only wish I had found a better place to stop than a rest area. Sometimes you gotta take what nature gives you.

Tomorrow, I’m probably cutting over the mountains and out to the coast. It was hot inland today and I could use a bit of time time sitting above breaking waves.

GULLIVER blogged earlier in the week about the Economist Intelligence Unit's business-travel ranking. Vancouver topped this annual survey, which assesses both cost-related and environmental factors in order to determine a city’s desirableness for a business traveller.

But if it's details you're after, it's details we've now got. We're sharing the full ranking—all 132 cities—here. (The methodology is viewable here.) You'll discover that Panama City is rated higher than Dubai, that Manila soars above Mexico City, and that Caracas is, erm, eight times dearer than Almaty.

There's no denying that gaming publisher Midway has had a rough time in the past year. After an insane saga of strange twists, turns, accusations, and increasingly dire news, most of us weren't entirely certain that the beleaguered publisher would actually survive to see 2010. Despite our doubts, it turns out that Midway is living to see another day, having just been acquired by Time Warner for $33 million.

For those of you new to the situation: after the company's much-hyped Mortal Kombat vs. DC Universe earned lukewarm reviews and reasonable (though not amazing) sales numbers, Sumner Redstone sold his controlling interest in Midway for $100,000, and the publisher wound up laying off roughly 25 percent of its workforce and killed many games that were currently in development. It was then revealed that, even though employees weren't getting paid what was owed to them and the publisher was filing for bankruptcy, executives were still raking in a great deal of cash during all this.

Specifically, the World Rose Festival; a large not-very-well-lit room in the bowels of Vancouver’s très chic new Convention Centre full of flowers, arrangements, and paraphernalia. Alex Waterhouse-Hayward, who’s a serious rose geek, wrote about it in The Rose Expert.

Rose people have their own vocabulary and highly-specialized sensibilities. I entirely failed to see the essential difference in quality between the “Best rose in show” (which was indeed very pretty) and a hundred others of roughly the same shape and tint. I thought a few of my own inexpertly-husbanded flowers would compare well with the show’s red-and-blue beribboned blossoms; silly me.

My favorite bit was a white sort-of hallway made of hanging sheer fabric filled end-to-end with big ambitious arrangements; the first photo is a close-up of one. The second photo is of a prize-winner, but I entirely forget the category and the color of the ribbon.

Knowing the light would be poor, I took the Sigma 30mm F1.4. It’s a lovely lens, but its laughable (by design) depth-of-field makes it devilishly difficult to work with. I consider my relationship with it as an ongoing project.

I've been buried in a big project on campus for the last few months. Yesterday, we delivered our report to the president. Ah, time to breathe, heading into a holiday weekend! Of course, next week I'll get back to my regular work. Department stuff. Cleaning my desk. And thinking about teaching software engineering this fall.

A bit of side reading found via my Twitter friends has me thinking about testing, and the role it will play in the course. In the old-style software engineering course, testing is a "stage" in the "process", which betrays a waterfall view of the world even when the instructor and textbook say that they encourage iterative development. But testing usually doesn't get much attention in such courses, maybe one chapter that describes the theory of testing and a few of the kinds of testing we need to do.

It seems to me that testing can take a bigger place in the course, if only because it exemplifies the sort of empiricism that we should all engage in as software developers. When we test, we run experiments to gather evidence that our program works as specified. We should adopt a similar mindset about how we build our programs. How do we know that our design is a good one? Or that our team is functioning well? Or that we are investing enough time and energy in writing tests and refactoring our code?

That's one reason I like Joakim Karlsson's post about the principle of locality in code changes. There may be ways that he can improve his analysis, but the most important thing about this post is that he analyzed code at all. He had a question about how code edits work, so he wrote a program to ask subversion repositories for the answer. That's so much better than assuming that his own hypothesis was correct, or that conventional wisdom was.

In regard to the testing process itself, Michael Feathers wrote a piece on "canalizing" design that points out a flaw in how we usually test our code. We write tests that are independent of one another in principle but that our test engines always run in the same order. This inadvertent weakness of sequential code creates an opportunity for programmers to write code that takes advantage of the implicit relationship between tests. But it's not really an advantage at all, because we then have dependencies in our code that we may not be aware of and which should not exist at all. Feathers suggests putting the tests in a set data structure and executing them them from there. At least then the code makes explicit that there is no implied order to the tests, which reminds the programmers who modify the code later that they should not depend on the order of test execution.

(I also like this idea for its suggestion that programs can and other should be dynamic structures, not dead sequences of text. Using a set of tests also moves us a step closer to making our code work well in a parallel environment. Explicit and implicit sequencing in programs makes it hard to employ the full power of multicore systems, and we need to re-think how we structure our programs if we want to break away from purely sequential machines. The languages guy in me sees some interesting applications of this idea in how write our compilers.)

Finally, I enjoyed reading Gojko Adzic's description of Keith Braithwaite's "TDD as if you mean it" exercise. Like the programming challenges I have described, it asks developers to take an idea to its extreme to break out of habits and to learn just how the idea feels and what it can give. Using tests to drive how the writing of code is more different from what most of us do than we usually realize. This exercise can help you to see just how different -- if you have an exercise leader like Keith to keep you honest.

However, I disagree with something Keith said in response to a comment about the relationship between TDD and functional programming:

I'm firmly convinced that sincere TDD leads one towards a functional style.

TDD will drive you to the style whose language you think.

There will be functional components to your solution to support the tests, and some good OOP has a functional feel. But in my experience you can end up with very nice objects in an object-oriented program as a result of faithfully-executed TDD.

Another of Braithwaite's comments saved the day, though. He credits Allan Watts for this line that captures his intent in designing exercises like this:

I came not as a salesman but as an entertainer. I want you to enjoy these ideas because I enjoy them.

Love this! He has a scholar's heart.

There is a lot more to testing that unit tests or regression testing. Finding ways to introduce students to the full set of ideas while also giving them a visceral sense of testing in the trenches is a challenge. I have to teach enough to prepare a general audience and also prepare students who will go on to take our follow-up course, Software Testing. That's a course that undergraduates at most schools don't have the opportunity to take, a strong point of our program. But that course can't be an excuse not to do testing well in the software engineering course. It's not a backstop; it's new ballgame.

NASA's Mars Phoenix Lander, which spent the summer in Mars' northern polar regions performing a variety of science experiments, caused quite a stir when rumors circulated that it had discovered signs of life on the Red Planet. NASA eventually held a press conference to dispel the rumors, promising that more details would eventually be revealed when scientists got around to publishing papers that would describe the experiments in detail. That day has finally arrived; today's issue of Science contains four papers that describe various findings from the mission. There's no sign of alien life, but the studies do reveal an active water cycle on Mars—including night-time snowfall.

The papers rely on evidence from a variety of the instruments on the lander, and the description of the data provides an impressive catalog of the various ways that Phoenix could prod and query the Martian pole. In the months before Martian winter shut the lander down, it managed to dig a dozen trenches, taking soil samples from each. These samples went into wet and dry chemistry labs, had their conductivity tested, and were even examined using an atomic force microscope. Meanwhile, cameras and a LIDAR system (a laser-based range detector) scanned the surroundings.

We buy cloth diapers for our baby, as a greener, cheaper and healthier alternative to disposables. Several companies make cloth diapers with snaps or Velcro fasteners, but those can hit $20 apiece or more.

Flat diapers are much cheaper, and can be folded to fit any size baby, but there’s no built-in fastener. The traditional approach used to be safety pins, but it’s a daunting task to pin a diaper without stabbing the baby or yourself with the sharp point.

The Snappi diaper fastener is a rubber elongated “T” with plastic teeth at each of the three ends. The teeth hold the diaper securely, but are too short to go through the diaper and into the baby. Putting the Snappi on is about as easy as using Velcro, and taking it off is even easier. It’s simple to clean and has a lifespan of about six months.

We tried an off-brand version first, and it nearly sent us back to pins -- the teeth wouldn't hold, and the plastic bits that connect the teeth to the stretchable body of the “T” always separated from the rubber. The Snappi brand fasteners never gave us any trouble.

Snappi Diaper Fasteners
$2

Manufactured by Snappi Baby

Available from Amazon

HEAVEN forfend that you have failed to read the new edition of The Economist in its entirety. But just in case, Gulliver would like to draw attention to a piece on the travails of the airline industry.

The problem, according to Brian Pearce, IATA’s chief economist, is that “fares and yields are still collapsing” in most of the world because capacity cuts have yet to catch up with the fall in demand.

Combine that with the recent rise in the price of jet fuel and you have an industry in severe difficulties. Many airlines are haemorrhaging cash—British Airways was getting through £2.7m ($4.4m) a day in February—but the price of raising more money "could cripple them for years". So who's the happiest man in aviation at the moment? It might well be Sir Michael Bishop, who has left it, having forced Lufthansa to honour a decade-old put option to add his 50% stake in BMI to the 30% the German carrier already held.

Glynn Foster from Sun talks about OpenSolaris on FLOSS Weekly episode 75. After explaining how Solaris has always been a robust, scalable operating system, Foster brags that now on a Toshiba laptop with OpenSolaris pre-installed ” … the volume works, and the keys work…” Then host Jono Bacon laughs “The keys work?!” The dialog starts at about 23:30 into the podcast.

The other host, Leo Laporte, mumbled “so cool” after Glynn Foster says “the volume works” and apparently would have let him get away with saying “the keys work.” But Jono Bacon is the community manager for Ubuntu, a Linux distribution that cares more about whether the volume and keyboard work than whether the OS scales.

It was amusing to listen to Glynn Foster and Jono Bacon personify their respective operating system’s priorities, server performance for Solaris and desktop experience for Ubuntu. Foster says that OpenSolaris used to be a royal pain to install and configure but now it has gotten much better. I don’t know how well Ubuntu scales — I imagine it’s not nearly as scalable as OpenSolaris — but it was designed from the beginning to be easy to install.

"4 Sale by Owner" -- as found on Main Street in Grand Junction, Colorado. Stephanie Walker

 

Folks, we're off for the July Fourth holiday. We wish you iced tea and fireworks, and whatever American dream makes most sense to you.

(Thanks to Stephanie Walker of Love in the Time of Foreclosure for the picture, which she took with her husband on their way to move in with her parents for a while.)

» E-Mail This     » Add to Del.icio.us

THE INFRASTRUCTURIST website recently featured a fascinating interview with William Lind, the author of Moving Minds: Conservatives and Public Transportation. In it, Mr Lind explains that conservatives shouldn't, and sometimes don't, necessarily oppose public transportation. That shouldn't come as too much of a surprise—after all, President Barack Obama's Secretary of Transportation is former Republican congressman Ray LaHood. But Mr Lind goes further, explaining how one might convince a reluctant conservative to support government investment in public transportation. "What does a transit-loving liberal need to know when approaching an auto-loving conservative," the Infrastructurist asks. Mr Lind responds:

The most important thing that a liberal needs to know in talking to conservatives about public transportation is not to use liberal arguments.  You can’t argue for transit on the basis that the poor need it. Conservatives aren’t particularly interested in that. On the other hand, when you start talking about things like promoting and shaping economic development and redevelopment, that’s a big interest to conservatives. When you talk about offering transit that is of a quality that conservatives would actually want to use–which usually means rail transportation–they’re interested, because conservatives are just as tired as everybody else of sitting stuck in traffic.

Mr Lind also argues that America's backwardness in public transportation is a glaring national security weakness:

National Security is always a big interest to conservatives and any time you can talk in those terms, you’re going to have their attention. Virtually every American knows that our greatest single national security vulnerability at the moment, the one that has enmeshed us in the middle east, is our dependence on foreign oil, most of it coming from unstable parts of the world. And this can drag us into unwanted wars, as it has it can result not only in high gas prices, like we had last summer, but in complete cutoffs like we had in ‘73 and ‘79, where events halfway around the world suddenly leave our gas stations without any gas to sell. And at present, if that happens, most Americans have no backup. Approximately half of Americans have no public transportation.

These are important points to consider, but they're just the beginning of the interview. Go read the whole piece and tell us what you think.

VANCOUVER, we salute you. Not only is it the world’s most liveable city, it is also apparently the best venue for business travellers. The Canadian city has topped the Economist Intelligence Unit’s business-travel ranking of 136 world cities for the third year in a row, with Toronto second, Adelaide third, Honolulu fourth and Perth fifth (the same top five as last year). New York and London were 62nd and 65th respectively, with New York the lowest ranked of the 16 American cities examined.

The ranking takes into consideration both cost-related and environmental factors in assessing a destination's desirableness for a business traveller. Cities are rated in 31 categories, which are grouped and weighted in five sectors thus:

1. Stability (25%)
2. Healthcare (10%)
3. Culture and environment (25%)
4. Infrastructure (20%)
5. Cost (20%)

Marks for the first four sectors derive from the liveability ranking that we have already examined. The "cost" mark derives from the city's per-diem rate: the price of a basket of goods and services typically required during a short business trip. So that's hotels, restaurant meals, car rental, taxi rides, etc. You can read the full methodology here, alongside the 2006 figures.

When we revealed the recent liveability rankings, plenty of commenters suggested that Vancouver was a city of the "nice but boring" type. The rankings clearly cannot cater to all tastes, but the emphasis on stability and infrastructure means that nice but boring cities do indeed do well. The same sadly cannot be said of cities in the world's more troubled regions, and Karachi, Algiers and Lagos bring up the rear.

Business Travel Ranking (December 2008, zero = ideal) 1 Vancouver 8.0% 2 Toronto 8.8% 3 Adelaide 9.6% 4 Honolulu 10.9% 5 Perth 11.5% 6 Auckland 11.5% 7 Detroit 11.9% 8 Atlanta 12.1% 9 Montreal 12.4% 10 Vienna 12.5%

(Photo credit: Shutterstock)

Some of the most arresting analysis of the causes and consequences of the financial crisis is being made by Andrew Haldane, the executive director of what the Bank of England calls - with no hint of irony - "financial stability".

His latest speech, "Small Lessons from a Big Crisis" [pdf link], is grist for those who believe top bankers are being paid far too much (although this is not a conclusion he draws himself).

First, Haldane looks at the returns generated by UK banks and financial institutions since 1900, to see whether shares in the financial sector have performed better than the market in general.

What this shows is that from 1900 to 1985, the financial sector produced an average annual return of around 2% a year, relative to other stocks and shares.

So for 85 years investing in bank shares was "close to a break-even strategy" (his words), nothing special.

But in the subsequent 20 years, from 1986 to 2006, returns went through the roof: the average annual return soared to more than 16%, which was the best performance by financial-sector shares in UK financial history.

And it's no coincidence that the pay of top bankers also zoomed up to the stratosphere. Which at the time upset only a few, because the bankers seemed to be enriching the owners of the banks, their shareholders (millions of us through our pension funds).

That, of course, is not the whole story.

The collapse of banks' share prices in the past two years has wiped out most of those gains: to March this year, when the low point was touched, the fall in UK bank share prices was more than 80%, an all-time record plunge.

What this means is that in the full period from 1900 to the end of 2008, the annual average return on financial shares was less than 3%, almost identical to the market as a whole.

Which is what common sense would predict should have happened, since banks are to a large extent a utility, serving the needs of the wider economy, and its difficult to see how banks in general can therefore grow significantly faster than the wider economy.

What went so right in 1986 to 2006? Had top bankers become much more brilliant than their predecessors, such that they deserved disproportionate rewards?

Haldane answers this question by breaking down banks' return on equity - the return generated on ordinary shareholders' capital - into its two component parts, which are the return on gross assets and the leverage employed by the bank.

This is slightly complicated, but bear with me, because it is absolutely central to assessing whether bankers merited their lavish remuneration.

Now if you want to know whether bankers are particularly skilful, you have to look at the return on gross assets. If one bank earns consistently bigger margins on the loans and investments it makes, that tells you it is probably doing something cleverer than its rivals.

By contrast, leverage - or the ratio between a bank's gross assets and its stock of shareholders' equity - is the Las Vegas part of the return on equity, the contribution made by a punt or a gamble.

Here's the important point: for any rate of return earned per unit of a bank's gross assets, the return on shareholders' equity rises as the assets-to-equity ratio rises - or, to use the jargon, as leverage rises.

Which is easier to grasp by way of a practical illustration.

Suppose a bank has lent £1,000 and earns a 1% net return on this, or £10. If that £1,000 is backed by £50 of shareholders' equity - which is a leverage multiple of 20 - the return on equity is 10 divided by 50, or 20% (which, for what it's worth, is a handsome rate of return).

Now, suppose another bank lends £1,000 on a leverage multiple of 50, or supported by just £20 of shareholders' equity. In this case, the return on equity is 10 divided by 20, or 50%. So the return to shareholders is a stupendous 50%.

Or to put it another way, increasing leverage is a simple and automatic way of increasing returns to shareholders. And as I hope you've noticed, there's nothing terribly clever about it.

But if all you care about is fat returns, and you're not interested in how they're earned, you'd give the boss of the highly leveraged bank a cigar, a bottle of Krug and a £5m bonus.

As I've observed many times in this column, maximising leverage is the equivalent of buying a house with the maximum amount of debt: it looks like an awfully smart thing to do when everything's going up up up, but is the fastest way to lose money when the economy turns.

Just to prove the point: if our banks were to lose £20 on their £1,000 of loans, the bank with just £20 of equity would be wiped out, it would be bust (a big hello to Royal Bank of Scotland, which at the peak of its lending and investing had a balance sheet that was indeed 50 times the size of its core equity).

So what has Haldane discovered about the golden banking years from 1986 to 2006? Were the super-normal returns of banks the consequence of management skill, viz high returns on gross assets? Or were they casino profits, generated because banks in general increased their leverage, their ratio of assets to equity?

This is what Haldane says:

"Since 2000, rising leverage fully accounts for movements in UK banks' ROE [return on equity] - both the rise to around 24% in 2007 and the subsequent fall into negative territory in 2008."

In other words, in the seven years before the crash, British banks' bumper profits were in aggregate generated wholly by a massive increase in leverage by the industry: and in Haldane's view, these would be returns generated by gamblers' luck, the jackpot from the roulette ball landing on black.

What follows?

Well, it's uncontroversial that we all paid something of a price, in the form of the worst global recession since the 1930s, when the bankers' luck ran out, when the wheel spun to red.

Which means that we all have an interest in preventing bankers from repeating these reckless gambles.

These would be a few useful lessons.

1) The overall level of bankers' pay was inflated over the past few years by the rewards they scooped from the leverage gamble. It should be cut to a level commensurate with an industry that's closer to a boring utility than to a wealth-creating, entrepreneurial venture. This has not happened yet. In fact, if anything, bankers are pumping up their pay packages again (the recent remuneration deal made by Royal Bank of Scotland with its chief executive, Stephen Hester, would not have looked mean in the boom-boom era).

2) Regulators should impose a legally binding maximum - and at a relatively modest level - for the ratio of a bank's gross assets to its equity, the leverage multiple, to restrict bankers' freedom to gamble.

3) Owners of banks should be very cautious indeed about rewarding bankers for the returns they generate on equity, and should focus rather more on the returns earned on gross assets.

If you're still with me (wakey, wakey), there's one other important related issue I want to explore, which is how to re-introduce moral hazard into banking, how to persuade bank chief executives that they'll really suffer if they place reckless bets that go wrong.

The problem is that no one can possibly any longer believe that there are any circumstances in which our government will let one of our biggest banks collapse.

Which is an enormous comfort to the chief executive of a bank. It means he or she can do something spectacularly stupid, safe in the knowledge that taxpayers will bail out the bank as and when it all goes wrong.

The best deterrent against greed-fuelled gambling by banks is the threat of being sacked when it all goes pear-shaped. But that's not a particularly scary threat to any banker who's earned enough in the preceding years never to need to work again.

That rather implies that bankers should be paid a decent wage, but should not be able to get their mits on any serious wealth for years and years and years.

Arguably they shouldn't be allowed the big haul till they retire and it's clear beyond a scintilla of doubt that they haven't dangerously over-mortgaged their respective institutions.

And once again we're back to the serious critique of Royal Bank of Scotland's board for sanctioning Sir Fred Goodwin's never-have-to-work-again pension.

But Sir Fred is just one embodiment of how banking became a casino run for the benefit of bank executives: the sucker punters were the shareholders and - little did we know it - taxpayers.

Market research firm Gartner sees Intel's mobile processor platform as a serious contender with Qualcomm and Texas Instruments due to its recent collaboration with Nokia.

Think about Wikipedia, what some consider the most complete general survey of human knowledge we have at the moment. Now imagine squeezing it down to fit comfortably on an 8GB iPhone. Sound daunting? Well, that's just what Patrick Collison's Encylopedia iPhone application does. App Store purchasers of Collison's open source application can browse and search the full text of Wikipedia when stuck in a plane, or trapped in the middle of nowhere (or, as defined by AT&T coverage...) Collison will be presenting a talk on how he did it at OSCON, O'Reilly's Open Source Convention at the end of July, and he spent some time talking to me about it recently.

James Turner: Why don't you start by talking about your background a bit and how you got involved with working with the Wikipedia?

Patrick Collison: I guess I've always been pretty interested in Wikipedia, and I ran my own MediaWiki installations back when I was in school in Ireland. We had our own personal ones and all of the rest. Then in November of 2007, I went to visit my friend in Japan for a month. And in Japan they have all of this incredibly advanced cellular technology and all of the rest. And so because of that, they had very few wireless networks, and my phone didn't work. As a result, I actually had very little access to the Internet. I sort of realized without Wikipedia how little I really knew. And I had just got an iPhone, so I decided to try basically putting a copy of Wikipedia on the phone, so that I'd have it as I was walking around in Japan. Then basically, I spent a significant fraction of my time there in Japan, again, in 2007 writing those applications, say maybe two or three weeks, just firstly trying to decide if it was possible and putting it all together. And then it was released, I think, January of 2008.

James Turner: Now you've also worked on getting it onto the OLPC I understand. How did that occur?

Patrick Collison: I actually didn't do much of the work for this. It was actually a project led by Chris Ball who works both with FreeBSD and with the OLPC project. But I released the code to this application; it was open source from the very start. So it was pretty easy for them to take it and to port it to the OLPC. I mean there are already some applications that allowed you to put a copy of Wikipedia on your computer or something like that, but none had really been optimized for embedded or low power devices or anything like that, which obviously Wikipedia for the iPhone had to be. I think it took about two or three weeks to take the code that ran on the iPhone and then to bring it to the point where it'd run on the OLPC.

James Turner: There are obvious benefits to having Wikipedia on the OLPC, because connectivity is very important in some of those areas. So you'd want to have it local, but outside of the experience that you were just describing, isn't the point of the iPhone that you can just access Wikipedia? What are kind of the advantages of having it locally?

Patrick Collison: I actually find that you spend, or I certainly spend a surprising amount of my time without access to the internet, even with the iPhone. Say for start if you were abroad, I mean everyone knows the horror stories of the data changes AT&T will issue you with if you're roaming. But also just stuff like personally, I find that on a plane or something you have eight hours to not do much. And so I actually end up doing a lot of my Wikipedia browsing there. But even aside from connectivity issues, it actually turns out to be quite a bit faster to use the built-in, cached Wikipedia application as opposed to the website. I mean you can search in real-time with the applications. You just type a couple of characters and tap into your article, rather than firing up Safari or searching for the article in Google; then zooming in so you can tap in, et cetera, et cetera. I and most of the people I know who use the application actually end up using it even when they have internet connectivity. And maybe 20 percent of the time it's pretty useful because it's the only choice.

James Turner: Now just as a point of interest, is this an App Store app or do you have to have a jail-broken phone for it?

Patrick Collison: It was released back when only the jail-broken SDK existed. It was in that initial sort of surge of early applications. I guess the first jail-broken iPhone app, I think, happened in August, and so this was released just under six months later. And then when Apple announced the SDK, I actually originally did not intend to port it to the App store, just because I was just working on other things at the time and my company had just been bought and so it seemed like a lot of work. But then over the summer, I started getting a huge amount of email from people who had upgraded to the new version of the iPhone OS, and were now missing Wikipedia. And I started getting 20 or so emails from people per day saying they love this application and they were really missing it. Or even people saying they were continuing to use the old version of the OS just for this application. And they really hoped that I would port it so they could eventually upgrade. After receiving these emails for a while, I eventually felt too bad about not porting it. So I spent a couple of days porting it and then released it in the App Store. I wrote it and finished the port in August. And then it took about three months to wade through Apple's approval process. Around the end of October, it was released in the App Store.

James Turner: Now most apps that you see in the App Store are relatively small in the couple of megabyte to tens of megabyte ranges. I understand this is about two gigabytes. Does that make it kind of unique or difficult as an App Store?

Patrick Collison: Yeah. I mean when I first went to submit it to the store I had done quite a bit of work getting it down to just marginally under two gigabytes, because two gigabytes was Apple's stated limit. But it actually turned out that Apple's infrastructure and their software was not able to handle two gigabyte applications or anything even close to it. I don't know, but a couple hundred megabytes was the cutoff. That three-month approval process included them having to fix bugs and me having to change how the application worked and all the rest just so I could physically get it into the store. And so the way it actually works today is the application itself is extremely small. I mean just a couple of hundred K. And then you download the application. And then when you first run it, it includes its own sort of embedded downloader thing that allows you to download the Wikipedia from within the application. And it allows you to pause and resume the download and all of the rest. So this actually ended up being the only reliable way of making the download work.

James Turner: And presumably you want to do that on a Wi-Fi network if you don't want to eat up most of your monthly bandwidth from AT&T?

Patrick Collison: Right, right, right. Or I mean I guess if you want, you can really test how honest AT&T are being by saying they'll give you unlimited data.

James Turner: So how big is the uncompressed original source data for the Wikipedia? And how do you cram it down to two gigabytes?

Patrick Collison: So my memory is that it's around 12 or 13 gigabytes uncompressed. The very first thing we do is it comes in this very verbose XML format. We have our own custom format that just includes the bare minimum amount of metadata. And then compacts it in this fairly space efficient binary way. We manage to strip out 20 or 30 percent of the content just by doing that. And then we apply bzip compression, which gets a pretty good compression ratio because, obviously, it's text. Then we also remove some of the content from the applications, the kind of stuff that's not particularly useful on the phone, We strip out the links, for example, to the article in other languages because those links don't work in an offline application. We don't have the other languages. We add links to pictures because we're not storing the pictures. We strip out references because I'm assuming you're not too interested in analyzing the minutia of the references when you're using the phone, and that kind of stuff. And so that, again, saved us another 20 to 30 percent or so.

And really, that's it. I mean what ends up being transferred to the phone is just this huge two gigabyte bzip2 encoded text file that we then index in various ways to allow it to selectively decompress various chunks when a user wants to load an article.

James Turner: So one other question I have is: how about the little things at the top of the article telling you why it's a bad article or things like that? Do you keep that in?

Patrick Collison: Oh, the info boxes? Right. No, we actually don't keep those. And one thing we're considering right now, we're actually working on a fairly substantial update at the moment, is kind of recognizing those -- okay, to backtrack one step, I mean one thing we do not do at the moment is load any templates because it's too computationally expensive on the phone. The way the Wikipedia website works is that you have this article and it has links to all of these other articles which are kind of special articles, you know, templates. And so to load one article might actually load say 20 articles. But loading one individual article on the phone takes about -- well, certainly on the iPhones before the 3GS, takes around five to ten seconds. If you were to do that for ten articles, it would become unusably slow. And so because of that, we don't load any templates. And, therefore, we don't show stuff like the stub message or something like that or the neutrality being contested or that kind of stuff. But what we're working on at the moment is making it, when we're creating that specially prepared dump for the iPhone, that we recognize some set of templates and then include a special flag with the article title or something that notes for the really important info boxes like say neutrality is contested or stub whatever, that we'd then be able to display something on the phone.

James Turner: Wikipedia is obviously a very dynamic thing. It's literally updated second-to-second. How do you deal with that?

Patrick Collison: Right. So, again, this was one of my concerns in the beginning. I mean I was unsure how useful it would be for that exact reason. But actually, it turns out, to be honest, in practice not to matter all that much. I mean the dump I currently have on my phone is actually from August of '08, so it's pretty old. And that causes obvious problems, like if I go to Barack Obama or take a look at something, there's quite a lot that it does not have. But, at the same time, it's pretty rare that I have the experience of wishing that my dump had something that it does not in fact have or something has happened since then. We considered doing updates in the form of deltas, where you could download say 100 megabytes per week to bring your dump totally up-to-date. But that actually ended up not being possible because Wikipedia's own infrastructure can't handle or at least was not able to handle weekly updates. And so they were releasing dumps every couple of months. And so there wasn't really much point in us putting together a really advanced update schedule.

They seem to have kind of improved things over the last couple of months. And so I'm noticing that they are releasing dumps now it seems every week or two. And so we're thinking again about the question of updates and what we could do. But we're hesitant to put too much work into it just because it seems that the usefulness of the application isn't affected that much by whether or not it's six months old or six weeks old. I guess people are pretty used to the idea of any particular reference document not being totally up-to-date. And Wikipedia is really the exception by being up-to-date. But I mean the application for the iPhone, I guess, is conceptually more like a conventional encyclopedia, and that seems to sort of work out okay.

James Turner: How often do you take dumps? Oh, boy. That sounds like the wrong question.

[Laughter]

James Turner: How often do you generate dumps?

Patrick Collison: Right, right. So it varies actually for the different languages. For English, just because it's all so complex, quite infrequently. The latest one available right now is from, I believe, October. And we're actually working on a new one at the moment. For other languages, it's quite a bit more frequent. And I think most of the ones available at the moment are from March or thereabouts. And, like I say, we're hoping to speed that up.

James Turner: You're going to be speaking about this at OSCON in about three weeks. I was curious; is there anything else at OSCON that's really caught your eye or you're interested in?

Patrick Collison: Yeah. So I know that the Cloudera guys are going to be there. And so I'm really looking forward to that. I think they're working on some pretty cool stuff with Hadoop. And so I'm really interested in having a look at that. And then, also, I'm a huge fan of the GitHub and what those guys are doing with kind of changing the dynamic of open source software and interaction and all of the rest. And I'm sort of interested to see -- like for a long time, I feel like open source software generally was held back by SourceForge and its ilk. And sort of the -- well, it's frequently just not a particularly good infrastructure. Like for a long time, we never really moved beyond Source Forge and mailing lists and CVS or, I guess, now subversion. And it's really interesting how Git has, through technology, started to change the sociology or social dynamics of open source software. And GitHub seems to be kind of continuing that with kind of the ecosystem of forks of different projects and allowing those to be later reconciled and all of that kind of stuff. And so I'm pretty excited about that. I'm looking forward to talking to those guys.

James Turner: All right. Well, Patrick Collison, thank you so much for talking to us. And we look forward to seeing you at OSCON.

Patrick Collison: Thank you.

• eMusic Q&A: Mark Everett of Eels – eMusic Spotlight
  "What were you not willing to do?

  Put out Beautiful Freak Volume Two and more $300,000 videos. The way that music is made now is basically for people who don't like music. It's made by focus groups. I would go insane. The only person I'm thinking about listening to a song when I'm making a song is me. I'm just trying to impress myself. If other people like it, that's great. But I'm not going to write a song to try and please a certain kind of person."

  (tags: eels music commercial success recordindustry mre e)

Share This

As Jimmy blogged, "

The IronRuby team is pleased to announce a new release of IronRuby: IronRuby 0.6!

Download IronRuby 0.6

You can also get the source code for this release"

On Wednesday EFF called the move "outlandish" and "a ploy to squeeze more money out of the mobile phone companies." The advocacy group filed a friend of the court brief with the United States District Court for the Southern District New York this week, which is hearing the dispute between ASCAP, AT&T, and Verizon over whether the telcos have to pay the music licensing body royalties for wireless ringtones. Joining the amicus brief are Public Knowledge and the Center for Democracy and Technology. Meanwhile CTIA - The Wireless Association, to which the big telcos belong, has also filed an amicus brief in the case.

“My challenge to everyone competing with Amazon, Google and Microsoft is to remember that you’re competing with Amazon, Google and Microsoft. These are strong technology companies, and if you’re going to compete with them, open source is the only way to do that. Otherwise, you have no leverage.” – Matt Mullenweg

Let’s accept up front that the next Amazon, Google or Microsoft is not going to be able to purchase hardware as cheaply as the last Amazon, Google and Microsoft. That’s strike one. Bandwidth is also going to be a bit more dear. Strike two. Consider the challenges of managing all of the above, and that’s strike three.

But before we call them – and count them – out, let’s consider for a moment the history of the software industry. Before the cloud, before software as a service there was this weird little trend called open source. This bizarre practice involved opening (read: giving away) your source code (read: your software) so that anyone, your competitors included, could use it. For free.

Odd as this might have seemed at the time, of course, open source allowed the small to compete with the big by leveraging rather than submitting to their weaknesses. It’s sometimes difficult to remember in this Google-obsessed age, but during Windows 95’s heyday, it was natural to conclude that Microsoft was the once and future provider of all the technology that one might reasonably require. Of course we’d once thought the same about IBM, but this was different. Microsoft was different.

My how things change. And stay the same, to be fair, as Microsoft hasn’t exactly gone the way of SGI. But anyone who’s watched the Microsoft business over the past decade or so will tell you that open source has been a disruptive influence on the firm, top to bottom. As if it wasn’t enough that monopolies like the browser and operating system markets were threatened by open source alternatives, its biggest and most terrifying competitors were building their own businesses on software they didn’t have to develop. Not that Microsoft’s been alone in feeling the corrosive disruption of free software, of course; it could and has been argued, in fact, that the biggest single reason that Sun is about to be subsumed into Oracle is the LAMP stack.

David versus Goliath, indeed.

To explore the specifics of how open source might impact the cloud, let’s indulge in a bit of Q&A.

Q: Before we begin, do you have anything to disclose?
A: Yes indeed. Folks with relevant technologies like Canonical, Cloudera, Convirture, Dell, IBM, Reductive Labs, Red Hat, Microsoft, Sun and so on are RedMonk customers, while we ourselves are customers of providers of Amazon and Google.

Q: To continue the above: could history repeat itself? Could David beat Goliath – again – in the cloud space, on the backs of free software?
A: Frankly, I doubt it.

Free software is not, by itself, enough to overcome the aforementioned economy of scale advantages enjoyed by the Amazon’s, Google’s, and Microsoft’s of the world, let alone the larger, enterprise focused systems players like HP, IBM, and Oracle (why not you too, Cisco?). But that, to me, is not the interesting question.

Q: What is the interesting question, then?
A: What we should be asking is not whether free software can replace Amazon et al, but whether or not it can power a viable cloud alternative. An alternative sufficiently viable to keep the big guys honest and prevent lockin. On the answer to that question, to me, hinges nothing less than the future of the cloud market.

Q: Why is that question so important?
A: First, there’s the aforementioned question of lockin. Neither customers nor the governments that tax them can be trusted to stave off damaging monopolies, in my opinion. History demonstrates conclusively that IT staffs, necessarily focused on the present, will happily sacrifice the future for the sake of Getting Things Done today. Equally clear is the fact that governments, when finally awakened to anticompetitive threats, generally do too little, too late. Meaning that the best hope for an open and vibrant playing field – i.e. a market of cloud providers not intent on locking you in at the first opportunity – in future is competition for the existing players.

Besides their monopoly-resistant properties, open source cloud software could play an important role in the rise of so-called private clouds – cloud infrastructures that are run on-premise. Whether one agrees or disagrees with the concept of private clouds or not, they’re coming. For compliance, privacy, uptime and a host of other reasons. Given that one can’t replicate the platforms of an Amazon, a Google or a Microsoft internally, it would seem to make public to private or vice versa transitions challenging in the extreme.

(Re)enter open source. Though some might point to interoperability and standards conversations as the most promising candidates for ensuring adequate competition in the cloud space, my experience in other standards arenas leads me to assign greater value to reference implementations of said standards. Open source implementations, more specifically, because at the end of the day the entire interoperability and standards discussion is about ensuring a level playing field. Throw in the fact that open source could potentially allow replication of the public cloud stack privately and you might yet see enterprises and governments pushing for open source.

Q: Are the benefits for open source cloud offerings strongest within the customer, then?
A: Not at all. Lost in discussion of cloud development has been the fact that the development platforms and targets are changing, and quickly. The level of interoperability that even unwieldy standards like J2EE offer is generally absent in the cloud. Platform as a service (PaaS) customers are writing applications, typically, to a completely proprietary abstraction layer, whether it’s offering by Google, Salesforce or someone else. And even Infrastructure as a Service (IaaS) customers deploying to enterprise standard platforms like RHEL will find their deployments regrettably unique, be that in the way that storage is accessed or the instances themselves are managed.

As Matt points out above, then, open source is going to be the primary mechanism with which startups compete, in my view. In the two primary styles of cloud implementations, IaaS and PaaS – what I’ve previously termed instance and fabric – we’ve dramatically different economic opportunities. With IaaS, the opportunities for developers and vendors has typically been to abstract the infrastructure via management, clustering and provisioning type applications. These opportunities are, frankly, likely to dwindle as Amazon increasingly offers these services itself. Within PaaS ecosystems such as Google or Salesforce, there is even less opportunity, in that the fabric is responsible for many of the tasks currently being serviced by vendors operating in the IaaS ecosystem. Most cloud vendors are building on Google or Salesforce rather than around them.

In other words, it’s a competitive market, and it’s only going to get more competitive as the bigger systems players rapidly pivot and reposition their wares for use in the cloud.

So how do you compete? Realistically, unless you’re company letterhead reads Google, IBM, Microsoft, Oracle or Salesforce, you’re probably going to have a hard time convincing even medium size cloud customers to write to something other than Amazon.

Unless, of course, you can develop a credible alternative that is popular enough to assuage concerns about longer term viability. Which pretty much means you’re going the open source route, in my view. Thus it is that the combination of open source and cloud is even more important for developers than it is for customers.

Q: Are any developers seeing things in those terms?
A: Sure. Take Cloudera, who’s offering a suite of commercial services around the open source Hadoop platform. Or the folks from Reasonably Smart – recently acquired by the folks from Joyent – who offer up the code from their Git and Javascript based PaaS layer with the following explanation: “we see [open source] as the only real way to make our platform truly attractive. Other Platform-as-a-Service providers may state a desire to be open, we’ve been that way from day one.”

Elsewhere, Red Hat is throwing a virtual conference strictly on the topic of open source cloud computing.

Q: Is the cloud a natural ally for open source?
A: Not at all. One of the godfathers of the free software movement, Richard Stallman, has called cloud computing “stupidity.” Others have argued that software deployed to the cloud obsoletes open source licenses, undermining the point of the software itself, with some even going so far as to call the loophole that permits this “a cancer.”

Q: Is there evidence to support these concerns?
A: Not much that I can see, candidly. Though the thinking is sound, in practice there are a great many healthy open source projects that are primarily deployed in network settings. From Hadoop to WordPress, well managed open source projects are succeeding without resorting to the more severe restrictions of the AGPL.

Q: Besides customers like enterprises and governments, who might most benefit from an open source cloud stack?
A: In a word: hosts. Given the stark economic reality that the major provider cloud providers’s economic advantages will expand with the growth they’re currently experiencing, what are smaller providers to do? Embracing open source seems to be the clearest response. Much as smaller and medium sized hosts worldwide today run Debian, Fedora, CentOS or Ubuntu as a means of minimizing their expense, so too are tomorrow’s would be cloud providers likely to embrace open cloud stacks in an effort to remain competitive in the burgeoning cloud market.

Besides, it’s not clear how big a cloud market will be left when the big guys are finished carving it up. If you assume (as you probably should) that IBM customers are more than likely to leverage an IBM cloud, HP customers an HP cloud and so on, you’ve already lost an important portion of the Global 100. Then consider the entrenched strength of the category’s market pioneer in Amazon and the relative strengths of communities that the likes of Google and Salesforce.com can sell into, and the addressable market is dwindling rapidly.

LAMP, with its flexibility, simplicity and perhaps most importantly – lack of upfront licensing costs – fueled an explosion in the hosting services market once upon a time. It’s entirely possible that a similarly open source cloud stack could do the same, particularly since far more software is delivered via the network than when the hosting industry first expanded.

Q: What is this cloud LAMP stack going to look like?
A: What we’re going to see, what we’re beginning to see, I think, is a loose coalition or confederation of projects and vendors that will together comprise an increasingly viable top to bottom alternative to some of the cloud providers today. We’re clearly not going to see an Amazon or a Google spring forth, complete, overnight, but the fact is from management to virtualization to operating systems to cloud provisioning the open source alternatives to the current proprietary cloud stacks are more credible by the day.

Q: Which projects and vendors will be part of this “coalition?”
A: Ultimately, there will have to be a variety of participants with varying aims and interests, but they’re probably going to look a lot like the recent Eucalyptus/Ubuntu partnership. Besides Linux (all flavors) and Eucalyptus, examples of projects I would expect to see considered for various roles in an open source cloud stack would be things like ConVirt, Drizzle, Hadoop, Puppet, Reasonably Smart and so on. Which is not to mention critical enabling technologies like KVM or potential API candidates like the one GoGrid made available under a CC license.

As you can tell, it’s far too early to begin casting for the new acronym, but it’s clear to me that there are going to be options for those that wish to pursue open source cloud computing. Which should be obvious, since most of the existing clouds are built on open source.

Q: What about timeframes: what are your expectations in terms of when the open source cloud will arrive?
A: It’s far too early to tell. What I would say instead is that the clock is ticking, and that the network effects favor the incumbents, so if I were an open source provider with cloud ambitions, I’d be ramping up the partnership and alliance conversations as quickly as possible.

If you happen to be one such developer or vendor, drop us a line and we’ll do what we can to help connect you to similarly interested parties.

Share This

The title about says it; both kinds of purple are flowers.

The little ones are a ground-cover recommended by a professional gardener for our front yard; since the kids play out back, we don’t want to be defending a grass lawn from moss and creeping buttercup and dandelions and all the other enemies. This stuff just spreads out and covers up and you can walk on it a bit while you’re gardening. Don’t know what it’s called.

Don’t know what the larger purple flowers are either.

We’re working on a fairly substantial revision of the Sun Cloud API, motivated by this problem: In a RESTful context, how do you handle state-changing operations (POST, PUT, DELETE) which have substantial and unpredictable latency?

What we’ve learned, from work with our own back-end based on the Q-layer technology and with some other back-ends, is that Cloud operations are by and large not very fast; and that the latencies show up in weird places. Here’s an example: in our own implementation, creating a Virtual Machine from a template or by copying another VM instance is very snappy. But weirdly, connecting a network (public or private) to a VM can sometimes be extremely slow. Go check out other implementations like EC2 and you see a similar unpredictable-latency narrative.

The idiom we’d been using so far was along these lines:

• As with both AtomPub and Rails, when you want to create something new you POST it to a collection of some sort and the server comes back with “201 Created” and the URI of the new object.

• When you POST to some controller (for example “boot a machine”) or do a DELETE, the server comes back with “204 No content” to signal success.

This is all very well and good; but what happens when some of these operations take a handful of milliseconds and others (e.g. “boot all the VMs in this cluster”) could easily go away for several minutes.

The current thinking is evolving in the Project Kenai forums, and was started up by Craig McLanahan in PROPOSAL: Handling Asynchronous Operation Requests. Check it out, and put your oar in if you have something better in mind.

To summarize: For any and all PUT/POST/DELETE operations, we return “202 In progress” and a new “Status” resource, which contains a 0-to-100 progress indicator, a target_uri for whatever’s being operated on, an op to identify the operation, and, when progress reaches 100, status and message fields to tell how the operation came out. The idea is that this is designed to give a hook that implementors can make cheap to poll.

We also thought about a Comet style implementation where we keep the HTTP channel open, and that can be made clean but support for it in popular libraries is less than ubiquitous. My personal favorite idea was to use “Web hooks”, i.e. the client sends a URI along with the request and the server POSTs back to it when the operation is complete. But every time I started talking about it I ran into a brick wall because it probably doesn’t work for a client behind a firewall, which is where most of them will be. Sigh.

There are a few points that are still troubling me, listed here in no particular order:

• When an operation is finished and you want to provide a Status code, we’re re-using HTTP status codes. Which on the one hand seems a bit outside their design space, but on the other hand maybe it’s a wheel we don’t have to re-invent.

• Instead of having the “op” field, we could have a different media-type for each imaginable kind of Status resource. That might be a bit more RESTful but seems a less convenient to use for client implementors.

• This whole notion of the target_uri makes me wonder if we’re missing a generalization. The most obvious role is when the Status is that of a create operation, for example Create New VM; then the target_uri is the new resource’s URI, what would come back in the Location HTTP header in a synchronous world.

  And in a few cases you might want more than one target, for example when you’re attaching an IP address to a VM.

  Hmmm.

• Speaking of generalization, I wonder if this whole “Slow REST” thingie is a pattern that’s going to pop up again often enough in the future that we should be thinking of a standardized recipe for approaching it; the kind of thing that has arisen for CRUD operations in the context of AtomPub and Rails.

What do you think?

"MySpace mom" Lori Drew has had her misdemeanor guilty verdict overturned by the federal judge handling the case, the LA Times reports. Violating a website's terms of use is not, it seems, a federal crime after all.

Horrible things aren't always crimes

The guilty verdict against Lori Drew, prosecutors crowed, would send an "overwhelming message" to online bullies. Though she escaped conviction on felony charges, the 49-year-old Missouri mom could have still faced three years in prison or fines of up to $300,000 for launching an online harassment campaign that ended in the suicide of a teenage neighbor. Drew was due to be sentenced today.

But the "message," legal observers worried, may be that anyone who uses a website without paying close attention to those ubiquitous Terms of Service risks committing a federal crime. The judge shared those concerns.

Click to play. Chana Joffe-Walt

 

Last week I spent a day at the Port of Tacoma watching a bunch of longshoremen unload a ship full of 700 Kias. These guys race the cars off the boat one by one. Then they drive them to an enormous parking lot about a half a mile away, where the cars sit and wait for their next stop. These days that wait is much longer than usual.

The port tells me that Kia sends cars ahead of its orders, so these cars aren't necessarily spoken for. I rode along in one as it was off-loaded -- just once.That was enough. It felt sort of like being on a race track.

» E-Mail This     » Add to Del.icio.us

1. OECD Factbook -- Flash-built impressive data explorer from OECD. Go to Indicators > Load and, in the words of Ben Goldacre, "prepare for nerdgasm". (via bengoldacre on Twitter)
2. James Boyle is on Twitter -- author of the book The Public Domain.
3. Sewers and Startups (Pete Warden) -- designing to last, reminds me of Saul Griffith's heirloom design riff. When I joined Apple back in 2003, the central build farm for all projects had both PowerPC and x86 Darwin boxes, and our code had to compile on both. Steve was playing a long game, years before the Intel switch he was obviously planning for it, (though I only caught the significance in retrospect).
4. Open Data Makes Garbage Collection Sexier, Easier, and Cheaper -- pragmatic use for open government data. For more on the author of this post, see Hello World for Open Data by Tim Bray.

‘Rock Star’ is perhaps the most abused phrase in the history of job listings. Nobody should be looking for a “rock star” accountant, HR recruiter or janitor. Whomever is posting these jobs is grossly misinformed as to the nature of rock stardom. Or accounting. Or both.

—AvoidThisJob.com on the differences between a Rock Star and a Planet Funk Store Manager

• New Software Disrupts Datacenter, BI Markets - InternetNews.com
• Coding Horror: Oh, You Wanted "Awesome" Edition
  As more developer start taking on the operations roles - where pricing is completely differently done than for developers (read: not free) - expect to see a lot of coders going, "WTF?" like this.
• Austin Entrepreneur Network Blog: Jason Cohen of SmartBear Talks about His Company
• Survey Results: Java EE Containers – Heaven or Hell? | ZeroTurnaround.com
• Can the iPhone Replace Your Laptop? | The iPhone Blog
• Austin Public Companies Stock price growth or decline over the past quarter/3 months
• [whatwg] Codecs for <audio> and <video>
  "After an inordinate amount of discussions, both in public and privately, on the situation regarding codecs for <video> and <audio> in HTML5, I have reluctantly come to the conclusion that there is no suitable codec that all vendors are willing to implement and ship."

Share This

Anemone is a free, multi-threaded Ruby web spider framework from Chris Kite, which is useful for collecting information about websites. With Anemone you can write tasks to generate some interesting statistics on a site just by giving it the URL.

Its only dependency is Nokogiri (an HTML and XML parser). Other than that, you just need to install the gem to get started using Anemone's simple syntax which, among other things, allows you to tell it which pages to include (based on regular expressions) or define callbacks.

This example taken from Anemone's homepage prints out the URL of every page on a site:

require 'anemone'

Anemone.crawl("http://www.example.com/") do |anemone|
  anemone.on_every_page do |page|
    puts page.url
  end
end

The bin folder in the project contains some more in-depth examples, including tasks for counting the number of unique pages on a site, the number of pages at a certain depth in a site, or a list of urls encountered.  There's also a combined-task which wraps up a few of these, intended to be run as a daily cron job.

You can install Anemone as a gem or get the source from Github of course, and there's some fairly comprehensive RDoc documentation available in the source or online.

Also worth seeing.. Mobile Orchard's Beginning iPhone Programming Workshop. Bay Area/July 30-31. Seattle/Aug 20-21. Ruby Inside discount of $200 -- use "ri" discount code.

I seem to be caught between two IDEs: Eclipse and IntelliJ. I abandoned Eclipse a couple of years back, partly based on wide spread recommendations from many different people, and partly because Eclipse just stopped working for me (it crashed out).

After I got started with IntelliJ I started to appreciate its merits, despite a generally clunky interface (with lots of modal windows), truly awful documentation. Many things are streamlined and only a ctrl-alt-shift-coke-bottle-touch-your-nose away.

However, over time, using IntelliJ got slower and slower and slower. It also started running the Tapestry test suite horrifically slowly: 40 minutes and up (it should be about five). It would often go away, even when memory wasn't tight. Indexing? Checking Repositories? Computing primes? No way to tell.

Meanwhile, Eclipse has been moving forward, with Eclipse Galileo being a Cocoa (not a Carbon) application. Critical plugins such as M2Eclipse have gotten nice, and the Clojure plugin is mostly better than the IntelliJ one (though both are very early).

For a while I was using IntelliJ when teaching Tapestry (as part of the VMWare image I use when training) ... and I got a lot of resistance. People were much happier with Eclipse on the last couple of go-rounds, and I'm sticking with it.

Overall, I'm feeling that most of what I've grown used to in IntelliJ is present in Eclipse, just handled a bit differently. The Clojure plugins are a wash; IntelliJ has the edge on the Git plugin. I think Subversion inside Eclipse is actually better.

I've even cranked up NetBeans but didn't find anything there compelling enough to switch.

It seems like all my major tools (Firefox, Firebug, Eclipse, IntelliJ) are in the habit of growing too complex, and doing too much stuff in the background that I don't care about. All those intentions in IntelliJ that you have to turn off (for performance reasons), and all those extra plugins for Eclipse that you need to not download in the first place ... they're all getting in my way.

I think a lot of this falls into the general category of accidental complexity ... to address the limitations of the Java programming language, all this extra stuff is coming into play: tools and wizards and plugins and indexes and whatnot. I find it pretty pleasant to work with Clojure instead, where the accidental complexity of Java is managed and isolated and the IDE doesn't feel the need to be overly ambitious. That's the Clojure concept right there ... grow the language to your needs, rather than building up tools. I think that's the Tapestry ethic as well.

John Adams at Velocity 2009 • ©James Duncan Davidson

One of the guys I’m always happy to run into at conferences is John Adams (@netik) who—for the last year or so—has been helping keep Twitter going. He took the job last year at Velocity and this year spoke a bit about what it takes to keep the fail whale at bay. He's also a fellow photographer which means that we always geek out a bit about photography whenever we bump into each other.

You can see more photos in the Flickr set for the Velocity Conference.

The casualty list is appalling: Antique plastic dolls at the National Museum of Denmark have begun to peel and flake; classic furniture at the Victoria & Albert Museum in London might as well have been left out in the sun for years; the first-ever plastic toothbrush, at the Smithsonian, is collapsing into a pile of crumbs; etc. A whole generation of irreplaceable items that are as representative of our culture as pottery or flintheads were of ancient ones are dying -- and many people charged with their care have no idea how to stop further damage.

One of the recently leaked builds of Windows 7 has more juice in it than just a new default wallpaper. In the Windows 7 Home Premium edition—as noticed by Kristan Kenney—, the Microsoft Software License Terms has an additional clause that mentions a Family Pack licensing plan that would cover up to three computers in a household. This is no accident: other editions like Professional and Ultimate do not contain the Family Pack wording. 

Here's the whole clause:

Ignite is coming to LA! As always speakers will get 20 slides that auto-advance every 15 seconds. We're going to be holding the geek event at Cinespace in Hollywood on 7/21. Submit a talk now.

This will be the first Ignite in Los Angeles; it is co-hosted by LA Geek Dinner. The LA G33k dinner was kind enough to let us take over their July dinner to host the first Ignite. LA G33k Dinner, founded by Heathervescent brings people with a passion for technology and the internet together over a meal where conversations happen, friendships form, and collaborations on various projects occur. L.A. Geek Dinner is an inclusive event. Find them on Facebook.

The event is free. We're hosting it at Cinespace on July 21:

6:30pm Geek Dinner starts

8pm-9:30 Ignite talks

10pm Cinespace opens to the eneral public for Dim Mak (you're welcome to stay for the band)

While the event is free, you are responsible for paying for your own food/drinks from Cinespace if you want 'em. Please RSVP to the Geek Dinner list on Upcoming.

If you're working on an interesting project, have an unusual skill, or just some interest that would be fun to share with everyone, please submit a proposal to: http://bit.ly/IgniteLA

Ignite LA is being organized by Brady Forrest, Matt Forrest, Dan Gould, and Heathervescent. If you're not familiar with Ignite check out some videos on the Ignite Show.

I put the exclamation points there because, OK, it sounds dull. Bonds, you think, are dull. International Monetary Funds, probably more so.

But this is one of those stories that brushes up against the profound.

The news: The International Monetary Fund has decided to issue what could be $70 billion dollars worth of bonds. It's the first time the IMF has raised money through bonds. The fund has been short of cash to lend out because of the financial crisis.

The interesting bit: The bonds are denominated in a kind of synthetic currency called Special Drawing Rights (SDRs). SDRs are used internally at the IMF as units of account. But China has proposed turning them into a new kind of global currency that could one day replace the dollar's central role.

The IMF's official position: "The dollar is the principal reserve currency in the global economy and will remain so as far as we can see."

We'll have a story on All Things Considered tonight, with commentary from Eswar Prasad who used to run the IMF's China division. About the bonds, he writes "A Win-Win Proposition? Almost, but Not Quite"

» E-Mail This     » Add to Del.icio.us

I've been doing a fair bit more reading on my Android phone, although recently I've switched from FBReaderJ to Aldiko. Each new release of FBReaderJ has gotten better, but Aldiko includes an actual CSS-based renderer1 and presently provides a much smoother reading experience2. I feel a little guilty using a piece of commercial software when a free-as-in-freedom solution exists, but not yet guilty enough to try hacking on FBReaderJ.

Guilt aside, reading more on a smaller screen has had me thinking again about the tension between book creator and reader in e-book formatting and layout. EPUB on the mobile phone highlights this tension more than e-ink devices if for no other reason than that a phone screen even less resembles a book page than an e-ink screen does. One conclusion I've come to is that it's somewhat unfortunate that Adobe has thus far been the biggest contributor to EPUB as a commercial e-book format.

On the one hand, someone had to do it, and it's good that someone has done it. Even the DRM thing, to some degree — most publishers aren't ready to do without it, and at least Adobe had the good grace to use an easily circumventable system. The major concern I have is with some aspects of the apparent mindset behind Adobe Digital Editions.

At present, Digital Editions is the EPUB viewer to beat — I have no idea what the actual usage figures look like, but it's the only viewer one can legally use for all those commercially-sold Adobe-DRMed EPUB books, so one has to imagine that DE commands the lion's share of the market. Adobe represents Digital Editions as being more than just an EPUB viewer. The advertising copy on the DE web site touts it as "offer[ing] an engaging way to view and manage eBooks and other digital publications." To this end, DE supports not only EPUB, but also the document format much more central to Adobe's business — PDF. And because PDF is so much more important to Adobe, DE caters to PDF at the expense of EPUB.

It's no surprise to the average e-book enthusiast that PDF's fixed-page nature makes it a poor e-book format. The most obvious reason is that PDF files can't be cleanly reflowed to alternative page sizes. A perhaps less obvious corollary is that PDF leaves little room for user control of basic formatting parameters such as font size, line height, text alignment, and paragraph marking. But via one or more chains of causality, because PDF rendering does not allow user-control of these properties, Adobe DE doesn't allow setting them for EPUB either. This yields an EPUB viewer where the reader has control over only the font size and page size. And even then only partial control! — DE will not rescale any size a book specifies as an absolute size, and DE allows books to provide "page template" files which control how the available screen area is divided into text regions and body columns.

The most generous interpretation of this decision is interface consistency. As long as Adobe is attempting to present PDF as an e-book format and provide a viewer which handles "digital publications" regardless of format, it only complicates that viewer's interface to provide options which apply to some formats but not others. Somewhat less charitably, Adobe's focus on PDF may have led to an unconscious bias toward creator control of document formatting, to the extent of perhaps not even considering placing control beyond font size (a.k.a. "zoom") in the readers' hands. And way over on the conspiracy-theory side of the fence, perhaps it represents a conscious decision on the part of Adobe to limit the usefulness (and adoption) of EPUB by making it only a small improvement over PDF even for the documents most suited to reflowable formatting.

So for whatever reason, the most popular EPUB viewer on the market has limitations which severely restrict the usefulness of the format. But EPUB is an open standard, which means other, better viewers are free to compete with Digital Editions and supplant it, right? Except they aren't really, because of DRM.

As I mentioned above, it seems that most publishers are not yet ready to do without DRM, despite the lesson of the music industry. The overwhelming majority of commercially-sold books are encumbered with DRM, and all of the DRM-encumbered EPUB books sold use Adobe's ADEPT DRM. It would be technically possible for a competitor to begin offering a different EPUB DRM scheme, but I can only imagine the degree of confusion mutually incompatible "EPUB" books would cause among average consumers. So any successful EPUB viewer device or application needs to license the ADEPT DRM technology from Adobe.

To facilitate this, Adobe has begun offering the "Adobe Reader Mobile 9 SDK." The SDK is available by license agreement only4, so we can but speculate from the marketing copy on the capabilities and interfaces it provides. The "features" list in the SDK FAQ focuses entirely on features of the "Reader Mobile document rendering engine," suggesting that SDK primarily/only provides a rendering engine. If this is the case, then Adobe is not expecting — or potentially allowing — other vendors to write competing ADEPT-compatible EPUB renderers. Instead, all the available and announced EPUB reader apps/devices using the Adobe SDK5 will simply repackage the Adobe renderer and the paucity of options for user control it provides.

One example in support of this theory is Amazon's PDF support in the Kindle DX. Although not widely advertised, Amazon is apparently using the Adobe SDK, just integrating only the PDF renderer. Notably missing in the DX's PDF support vs. all the Kindles' Mobipocket support is the ability to add annotations to documents. It seems to me that this would be a "must have" feature, not only for parity with Mobipocket support, but also for the target market as a device for textbooks and technical documents. To me the most obvious explanation for this feature's lack is that there isn't an easy way to add it while still using the SDK to allow rendering of DRMed PDFs. There are plenty F/OSS PDF renderers to which Amazon could have (comparatively) easily added annotation support, which suggests that the Adobe SDK's DRM support is an implicit part of the included renderer, and that SDK licensees cannot use the SDK to read DRMed documents independently of the renderer.

Another interesting angle is to compare the Adobe approach with how other book formats/viewers handle the creator-reader tension.

The desktop version of MSReader allows setting only the font size, but a significant number of users seem to regard it as still the best desktop e-book viewer available. A major component of this seems to be very well-chosen defaults for properties like line-height, and the infrequency with which books alter those properties. I have seen much more mixed reactions to the Mobile version, perhaps because on the smaller screens of mobile devices tuning the line-height, margins, etc to an individually comfortable size is much more important. Perhaps a commenter could fill me in?

The various Mobipocket viewers support differing assortments of user-controlled properties. Most support setting font-size, line-height, and paragraph alignment. Interestingly, Mobipocket's treatment of these properties demonstrates all three possible resolutions of the creator-reader formatting tension. Line-height may be set by the reader, but not by the book creator — the format simply provides no way to specify it. The font-size may be set by the book creator, but only in terms of a size relative to the reader-selected base size. And paragraph alignment may be specified by either the book creator or reader, but the creator's setting overrides the reader's when specified.

Correspondingly, paragraph alignment is the subject of one of Mobipocket's most forceful formatting recommendations: "alignment must NOT be set if it is not strictly needed." In contrast, the EPUB specification documents contain little resembling formatting guidelines (beyond an admonition against using absolute positioning). The one purely formatting recommendation in Adobe's "EPUB Best Practices Guide" is "use spacing that looks more like a book," suggesting including CSS rules to eliminate default space around block-level elements. It does contain some sensible recommendations e.g. against using
tags in most contexts, but otherwise the "Guide" documents Adobe's EPUB extensions and DE's quirks more than actual "best practices." Which means that EPUB combines the most extensive e-book formatting capabilities with the fewest guidelines for producing actually readable books. And this is something of a challenge for authors of EPUB viewers.

Coming full circle to the beginning of this post, the Aldiko EPUB viewer does allow the user to set some basic formatting properties, including margins, font, font-size, and line-height6. The first three work seamlessly, but line-height somewhat less so. Aldiko handles books which don't specify their own line-height fine, but any book-specified line-height "wins." Which isn't intended as a slight on Aldiko — this is a difficult problem to solve.

Something like page margin can really only be set via CSS in one "most sensible" way7, and thus is easy to override coherently and consistently in a viewer. Font-size is potentially trickier, but the most obvious ways of specifying font sizes (relative sizes and the CSS named absolute sizes) make a simple solution fairly straightforward. Properties like 'line-height' unfortunately lack such a solution — all the allowed relative values for the CSS 'line-height' property are interpreted as relative to the 'font-size', not the 'line-height' of the parent element. This means that the font-size solution of "change the base and respect subsequent relative changes" doesn't work for line-height. Without doing some sort of layout analysis, all a viewer can do is either ignore all book-specified line-heights or respect all book-specified line heights.

Solution? I'm not sure. One possible solution would be for book producers and viewer author to agree on guidelines which allow viewers to consistently override some set of formatting properties. Most of these could be fairly simple, like Mobipocket's "alignment must not be set" rule. Another solution would be for e-book reader apps to do some sort of pre-rendering layout analysis which allows them to produce automatically produce a per-book user stylesheet. I like hands-off technological solutions, but I'm not sure how feasible that will be on mobile devices.

Other ideas?

1 To my surprise, a good enough one to handle the 'max-width' property on images.

2 Quite literally, in the case of the page-turn animation.

4 Probably to stop people from reverse-engineering the DRM system.

5 The ones I'm currently aware of: the Sony Reader, Lexcycle Stanza, the Bookeen Cybook, and the Elonex ebook.

6 Not paragraph alignment yet, but via e-mail the author has said it'll probably be added soon.

7 One could set left and right margins on every block level element, but hopefully no one actually does that.

Download the episode directly right here, subscribe to the feed in iTunes or other podcatcher to have episodes downloaded automatically, or just click play below to listen to it right here:

John and I caught up earlier in the week. Despite it being a short time between this episode and the last, we found plenty to talk about:

• Man! It’s hotinhur!
• John’s Cloud Week videos - John Adams from Twitter, Chris Wensel on Cascading, ControlTier.
• The “science” meme at Velocity - R, Mathmatica, and doing stats. Coté also remembers that Sara Dornsife now works for the R company, and the Zed Shaw talk that included R. I ask John what you’d use this stat stuff for, obviously charting historic data, but also doing predicting.
• Chef vs. Puppet - also cfengine is mentioned in this context. What other “alliances” does OpsCode have around Chef? EngineYard, (maybe, if John remembers) RightScale. Also, they got $2.5M in funding (we mistakenly remember $5M).
• Eucalyptus - talking through their layered up, (seemingly) “swappable” cloud platform and how they’re thinking of making money off it.
• IBM CloudBurst demo (this one, or this one, or another one? - how was that? John likes the Service Management hook-in, self-service portal.
• Is “private cloud” slowing down “cloud computing.” And getting at traditional IT shops “hav[ing] that ability to, kind of, ‘API’ themselves.”
• Are SLAs and KPIs calling out for “better” operational stuff in IT like private cloud?
• This gets us into webappVM - deep instrumentation; “webappVM is building a self-monitoring application cloud for web applications, bringing enterprise-level application management capabilities to public and private clouds” from their site.
• FiveRuns Dash - generic-ish, Web 2.0 style metrics. This gets us onto FiveRuns memories. Also, former FiveRuns CEO OT is now at Zenoss as their CMO.
• Google Ops Head goes after blood at Structure 09, .Net at MySpace
• Kuleto’s review
• John [was] helping run CloudCampColumbus this week. He’s trying to get some Tivoli folk out there.
• Coté interested in pictures and anecdotes of Detroit. [We'll see if John snatches some.]
• I mention that I’m thinking of strumming up interest in barcampESM II (see the first for reference) - is there enough interest for that kind of thing this year? Also, I’m lazy.

Disclosure: Reductive Labs (Puppet), IBM, and Zenoss are client.

Share This

Behavioral advertising, in which users are fed ads based on the interests revealed by their Web browsing habits, has an obvious appeal to advertisers, as it will ostensibly allow them to serve ads to the most relevant audiences. It also raises a host of privacy concerns—to work effectively, the Web surfing histories of consumers have to be aggregated and analyzed by those providing the ads. 

Both the Federal Trade Commission and Congress have asked questions about whether advertisers were doing enough to protect and inform consumers, raising the prospect that regulation of behavioral advertising was only a matter of time. In an attempt to head off the government, a coalition of advertising groups that includes Google has now issued a series of principles that will guide their self-regulation.

The industry didn't need to look far to see the downsides of a failure to respond to public concerns. One of the more aggressive approaches to behavioral advertising, the deep packet inspection used by NebuAd, saw the company's CEO dragged before Congress, and the resulting bad publicity turned the company into a pariah. It ultimately closed its doors last month.

In this interview from RSC 2009, I talk with IBM’s Neeraj Chandra, who you may recall from two previous interviews at RSDC 2008 and Innovation 2008, last year.

We start out talking about what exactly a “Smart Product” is and how Rational fits into the overall IBM “Smart Planet” vision. Here, the discussion gets into one of the recent Rational tenants: businesses should not only be looking to software for differentiation and value, but are indeed forced to.

The question then, is how IBM helps companies do this: the goals are, of course, desirable, but the devil is always in the details. Part of the story here is the need to bring more discipline to the software creation process as it raises is criticality to the business.

While the IT-side of the equation has to change, there’s also much needed from the business side. We discuss how the business-side needs to change and adapt to these scenarios as well. Neeraj points out that much of this change is enabled by upping the collaborative aspects in the overall Rational portfolio - enabled, of course, by the Jazz platform.

We get back to to how business strategy and objectives map down to IT and the development of software - I ask Nerraj to go over how bodies of practice like Rational’s MCIF are used to map between the two sets of objectives.

Disclosure: IBM is a client and sponsored these videos.

Share This