Neil Mix

When is it appropriate to implement a programming language to run in a web page?

I’ve been contemplating this lately as the number of programming-language-in-web-page projects (henceforth referred to as PLIAWPs) has multiplied. I really enjoy language implementations, especially in a language like JavaScript where doing so can be exceedingly concise. So I certainly understand the motivation for writing your own PLIAWP. But is it useful?

On the down side, PLIAWP implementations carry heavy baggage with them:

• They don’t integrate into native JavaScript debugging facilities. You have to create your own facilities instead. This is the most often overlooked aspect of creating your own language. Writing a parser and interpreter are only the beginning. In order to use your PLIAWP, developers will need error messages with meaningful line numbers, stack traces, syntax-aware editing, graphical debuggers, etc. These are must have features if you want your PLIAWP to achieve anything better that esoteric status.
• Performance is going to take a hit. If you write an interpreter, you’re looking at an order-of-magnitude slowdown, minimum. If you compile to JavaScript, you’ll do a bit better. But JavaScript’s string-handling facilities vis-a-vis parsing aren’t speedy. Plus, for any language meaningfully different from JavaScript, there’ll be a performance tax trying to fit your language’s runtime into the JavaScript runtime.
• JavaScript is already a powerful language. What does your language do that JavaScript can’t? Is it really worth forcing developers to learn a new syntax or runtime while giving up the comfort of their existing developer tools?

But it’s not all bad. There are definitely some plusses to building a PLIAWP:

• It’s fun.
• It expands the shared knowledge of what’s possible on the web, pushing the limits of what we know how to do.
• It’s incredibly educational.
• Did I mention that it’s fun?

But I didn’t mention useful, now did I.

So when does a PLIAWP emerge graduate from interesting to useful? There’s no easy formula, but here’s a few standards I’m sticking to these days:

1. The PLIAWP must be significantly different from JavaScript. (Think Java versus Haskell different.) While nice syntax has its appeal, that’s not enough to justify the overhead of a full language implementation.
2. A program written in the PLIAWP must not be trivially convertible into a JavaScript program. (A different way of stating the above point.)
3. The PLIAWP must solve a specific problem that JavaScript doesn’t solve easily. (If the term “Domain-Specific Language” is coming to mind right now, that’s no coincidence.)
4. The PLIAWP must be developer-friendly, or at least clearly on the path to developer friendliness.

A while back I built my very own PLIAWP, plus a few similar experiments since. With a couple years reflection since that time, I find it interesting to consider which PLIAWPs meet my standard of usefulness and which fail. GWT passes, generic Ruby/Python/LISP/whatever retreads fail. John Resig’s fabulous processing.js almost passes (more on that in a second), Narrative JavaScript fails.

That’s right, my own pet project fails my own standard. I started Narrative JavaScript with the intent to build something useful. In the end, it turned out esoteric. It addressed in part each of the above benchmarks, but not strongly enough to warrant the effort required for adopting my little PLIAWP. I’ve since learned new programming techniques to mitigate the problem I set out to solve, such that the burden of the developer-unfriendly PLIAWP world makes Narrative JavaScript untenable for real-world projects.

Does that mean I regret building it? Absolute not! I learned a *ton* about programming, computers, and computation from my failed experiment. And I had fun in the process. It was absolutely worth it, and I hope to try again someday when the demands of fatherhood and a successful startup subside enough to give me a little breathing room. Language implementation is a thrill.

So it was with great joy that I watched John launch his processing.js project. He is a budding (if not already great) master at work, unleashing creativity meshed with engineering know-how. It’s truly great stuff, a pleasure to watch. As stated above, I’m not sure the PLIAWP meets my usefulness standard, but I don’t think that matters. I don’t want to rain on this parade. In time, he and others may realize that processing.js would be better off as a highly-tuned JavaScript API than an in-page DSL. But why hurry to that point. Let’s explore this world a bit longer, push the limits a bit further.

If it’s a good idea, go ahead and do it. It is much easier to apologize than it is to get permission.

- Admiral Grace Hopper

Auto-updating software is neat. It saves the customer from painful manual updates, and it helps the vendor by keeping people up-to-date with their software.

Unfortunately, the auto-update feature got off to a bad start in UI. Due to mindless follow-the-leader mentality, it’s had a hard time recovering ever since.

Case in point:

While this is an extreme case, it also illustrates a problem that we’ve all had with auto updates, namely how they manage to interrupt at the most inopportune times.

The problem boils down to 2 factors: a) permission, and b) timing. A little forethought solves both problems fairly easily, but bad precedent of previous implementations casts a long shadow.

Programmers often get snagged on the concept of permission. The (non-conscious) line of thought is: we wouldn’t want to disappoint someone, so let’s make sure it’s OK before we proceed. What’s missing is opportunity cost: people like having their chores done for them. By thinking that doing-without-consent is risky, we miss the opportunity to make the customer to be pleased as punch that someone is quietly getting-things-done for them.

Early on in the history of software auto-updates, someone decided that it was very important to interrupt the customer to make sure that they don’t object to having their software updated. Early versions of auto-updates went something like this:

• start the app
• app says, “Updates to your software may be available. Would you like me to go check?”

First off, how lazy! This comes straight from the playbook of how to ruin your career: when a simple but helpful task needs doing, check with your manager before doing anything. He or she will be impressed with the amount of reassurance and micro-management you require.

At some point someone realized, “hey, that’s kind of annoying.” And thus the auto-update question was no longer asked on startup, but instead asked at some random time on some random interval — but the same time and interval each time, so as not to be confusing.

Did you see the bait-and-switch? The problem was with asking too much permission. But “fix” was to change the timing. I quoted the word “fix” because the solution didn’t improve things much. We went from an annoying every-time-I-start-the-app question to an annoying randomly-interrupt-me-in-the-middle-of-my-workday question.

After a while, someone made the startling realization, “hey, we could check for updates automatically!” No doubt someone objected that permission should be asked first, but thankfully the objections were overridden. So now we have: randomly-interrupt-me-when-updates-are-available. Which is better. And this is pretty much state-of-the-art for auto-updates these days.

But we’re not quite there yet, are we?

So what to do?

1. Just grab the update already! Beg forgiveness rather than ask permission.
2. Don’t interrupt me while I’m working! Wait until the next time the app is started, then perform the update and let your customer know what you’ve changed.

So now we’re at: annoying-interruption-on-startup-only-when-updates-have-been-made. But how annoying is the interruption, really? Presumably you’ve got great new features for your customer that they’ll be happy to hear about. Perhaps rather than being annoyed, your customers will feel pleasantly surprised by the gift of new features they’ve received this morning. Your software just made their lives easier, on multiple levels.

Still sound too aggressive, too risky? Consider this: there’s already strong precedent for the auto-update process I described, just not on the desktop. It’s on the web. Websites don’t ask permission before updating the software. (They can’t, really.) In some cases, when the updates are really big, they might let you know about the changes your first time in. (And occasionally they may provide a graceful “downgrade” to the old version if you protest.)

But most of us never really noticed this subtle distinction between desktop and web auto-updates. That’s because we’re too sympathetic to the underlying technology. But when we wipe the technical details away (as we should when designing our UIs), the truly-auto-update isn’t so scary after all, now is it?

Here’s a little something I’ve seen around the web a few times:

What does this mean to you? What pictures form in your head when you see this?

Now before anyone accuses me of being naughty, I can assure you that this is a Safe-For-Work post. The story behind this image is hardly controversial.

If the phrase “rollover for more” seems perfectly innocuous to you, stop for a moment and consider how your grandmother would react if you told her to “rollover for more.” Mmmmmaybe you aught to find a way to reword this.

One of Joel’s recent posts has been tumbling around in the back of my head. His observations are, as always, excellent. But unlike his usual posts, the conclusions drawn in this one mystify me. I see his point, but it feels…crazy.

One important truth he reveals is that software projects need not run optimally on today’s hardware; Moore’s law and the passage of time guarantee that when your product hits its prime market share, the hardware will be capable of supporting it.

Apply this maxim to development platforms, flip things around a bit, and we get this:

Any software development platform incapable of fully utilizing its hardware will surely fail.

This is true because there will always be some other platform that is capable of soaking up all that power, and that platform is going to have whiz-bang features that everyone’s gotta have right now that can’t be done in the old-fashioned turtle-speed platform.

Java is a great example. For those of us who used it back in the mid-90’s when it was brand new, it was slooooow. It was hard to believe that this was a “real” development platform. But it had some great features that outweighed the perf issues. And 10 years later, after 5 or so revs of Moore’s law and some nice optimization, Java is king. (Or at least queen.)

Case study #2: Ajax. Remember Ajax back in 2000? Yeah, I do too. If there’s a lesson to be learned from Java and Ajax, it’s this: your development platform should aim to run on tomorrow’s hardware. Worried that RoR is too CPU intensive? Relax, Moore’s Law will take care of it.

But…

Moore’s law isn’t quite the same as it used to be, now is it? As you might have heard (sarcasm), the multi-core world is coming, and a few people are a bit riled up about it.

Back to Joel and Ajax. Ajax depends on JavaScript, and JavaScript — as currently spec’d and implemented in browsers — is a single-core beast. The new version — supercharged kitchen-sink-of-features, still in its final spec’ing stages — will probably appear in the wild sometime late 2008. And what’s the new version doing to integrate well into the new multi-core world?

Well, nothing really. And it’s too late to change that.

So now we’re looking at version 3 of JavaScript before multi-core deficiencies will be addressed. Meanwhile it’s at least a few years before version 2 is established enough for Ajax apps and frameworks to depend on its features. Lather, rinse, repeat, it’s another couple years after that before theoretical multi-core Ajax apps are viable.

Repeat: the performance of Ajax apps will remain little-changed for the at least the next 3 to 5 years, unable to utilize multi-core processing power. (Same thing applies to Flash too, you’ll notice.) By that time, current processors will likely have 16+ cores. That’s a whopping amount of processing power lying dormant.

Do you think 5 years enough time for someone to build a platform that contains all the benefits of Ajax, plus some new great stuff, that lets developers soak up all the processing power in those extra cores, and deploy it to 90%+ of machines out there? Do you think maybe some of the big players have efforts in this space already underway? (hint: Gears is a start.) It’s just a question of who and how soon.

So what does this mean for Ajax? For one thing, it means that Ajax — as it currently exists is — will have to evolve for the long term. One of the major browser vendors is going to have break ranks and extend the browser platform to go multi-core (it won’t happen in lock-step — design-by-standards-committee is too slow and will inevitably lead to a back-stab). Either that or something new will emerge. (Or more likely, many new things will emerge.)

Get ready to brush up your skills…

It happens on every flight. The lights go off briefly as the pilots prepare for takeoff. Instinctively, passengers reach up for the button that turns on the reading light. It’s a red, rectangular button directly overhead…it’s hard to see (because the lights are off)…is this the one?…*bing*

The big red button is alluring, all-powerful in it’s ability to draw you toward it. Place it with care — the best placement is not intuitive. Most often, the big red button belongs somewhere you wouldn’t normally find a button. Consider: the airplane (wrong), versus the subway (correct). Easily accessible: yes, prominently visible: yes, intuitive: no. The idea is that in a panic, you’ll find it, but in your daily routine it’s out of your way.

Draw parallels to your communications. How many times have you inadvertently started an email panic by placing an unqualified big-red-button in plain site? Conversely, when was your big-red-button missed because it was buried deeply in amidst complex prose?

Imagine you need to tell a story, to walk someone through a coherent thought beginning to end. Now imagine that you can’t use words, and you can’t use visuals. You can use sound, but you can’t use spoken language of any kind, and your audience will be blindfolded. Where would you start? How would you proceed? What messages would you be able to convey? And how limited would your message be?

At first glance, it may seem this format is too limiting for meaningful communication. But that’s not the case: the composers of classical music were great masters of the design and presentation of complex narratives, and they did so without words or visuals. For a master class in story telling, study classical music.

It’s no easy trick. You need to repeat your themes, but not too much. Themes must be presented in different ways, but not too different. And you need to evoke recognition of archetypal structures, but do so without belaboring the obvious.

A well-written classical music piece contains the plot of a great novel or movie: it has a well-defined flow and structure, it tells you where you are going without giving away the ending, and it’s so interesting that you’ll enjoy the ride even if you’ve experienced the story many times previously. When done well, the removal of any one section renders it incomplete. The narrative takes on a life of its own beyond the composer, as if it had always existed but no one ever pulled back the curtain to reveal it.

The sad truth about classical music in the modern world is that most people consider it “background” music. It’s fairly easy to hear simple beauty in classical music moment by moment, one theme at a time. But this is a shallow understanding of the music. In order to grasp its true depth (and unimaginable beauty), you have to “listen big”. You need to pay attention and remember each moment, and then you must reflect on how each moment impacts the next. In the end, the structure becomes clear. A beautiful architecture is revealed, and a deep emotional understanding is imparted. Rachmaninoff described this moment in a piece of music as “the point”. It’s presence is the hallmark of a masterful presentation.

Unfortunately, the attention requirements of classical music don’t mesh well with today’s sound-byte driven world. My college music history professor spent a year walking us on a musical journey through centuries, and his final lecture led to the conclusion that classical music (or more specifically, The Tradition of Western European Art Music) was terminally ill, if not already dead. It was a great moment, and remarkable in that he employed the master’s device on us — he built us up to a culminating “point” where a profound message became clear. Stunned by his prognosis, we were deeply saddened. And we applauded.

The next time you hear a bit of classical music in the background, consider scratching a little deeper. You might be surprised at what you learn.

I once had the pleasure of meeting the UI designer for Tivo at a party. Sadly, this was before it had become popular and I was not yet a customer, so I don’t have any great stories to tell about her philosophy toward UI design. In any case, I remember that some time later I mentioned this encounter to a software engineering friend of mine, and he responded with great distaste, “Boy would I like to give her an earful!”

To this day I am stunned by my friend’s callous disregard for the Tivo UI. Of course Tivo isn’t perfect — no UI is — but there’s so much they could have done wrong, that most companies would have done wrong, that they got right. I still regard Tivo as a landmark product that changed people’s perception of what’s possible with technology.

So what of my friend and his disdain for Tivo? Sadly it’s not too uncommon to hear techies skewer products with otherwise good UIs. And that is the crux of what I call the Elegance Paradox.

I often say that it would be awfully painful to be a top-notch designer whose boss doesn’t understand design. The design process is about whittling away distractions, making the obscure feel obvious, making the obvious feel implicit, and doing it without anyone noticing. To the untrained eye, your best work looks like you’ve done no work at all. If you’ve done a stellar job, then your design will feel utterly obvious.

The Elegance Paradox is this: to create elegance requires entirely inelegant preparation, but nobody should be able to see that. How many times have you said to yourself “that person makes it look so easy.” That’s a pretty common reaction to seeing an expert in a field you’re not familiar with — their presentation looks smooth, spontaneous, and simple. But the more familiar you become with a particular field, the more you recognize how difficult it all is. As you get more proficient, you realize that creating elegance isn’t smooth and effortless, it’s full of strains, grunts, gasps, hard work and complexity.

And once you’ve whittled away and acheived some modicum of simplicity, what’s left for the outside observer to see? “Obvious” flaws.

When I first became interested in UI programming (and more specifically, sane user interfaces), I thought that I was one of the rare programmers who “got it.” I would look at the world of tech products out there, and I could see their numerous flaws instantly. I was certain that, given the chance, I would be able to do better.

Several years later I’ve learned the toughest best lesson I could: it’s easier to see the flaws than it is to see the elegance. As much as I’d like to believe otherwise, I often don’t “get it” when it comes to UI design, which is why I rely heavily on the opinion and judgement of my designing coworkers. Anytime I might pick at something I perceive as a UI flaw, I remind myself that there are a myriad of design decisions that I can’t see. If I were put in the same position as the designer, would I have made the right choice on all those decisions? Likely not.

So now we come back to my friend who thought the Tivo UI designer needed a good talking to. What’s his problem? His problem is that he doesn’t understand the Elegance Paradox. The simplicity of the design led him to notice only that which was missing. To him the design felt entirely obvious, and he mistakenly therefore assumed creating such a design is an straightforward process.

Amongst software engineers, the designer of Rails or Python or jQuery will receive much credit for their work (and rightfully so) because we’re all familiar with the problem space, and we all recognize that we likely couldn’t (or didn’t) make the same good design decisions when put in their position.

For UI builders, it’s a different story. All too often, the problems of UI elegance are dismissed as “something we just need to do better,” or “an area where we need to pay more attention.” Well-intentioned critique, no doubt, but not remotely grasping the scope of the problem.

Good UI is hard. Not just the design, but the programming too. Coding an elegant UI is an endless process of fixing twitches, hitches, and glitches that no one will ever see or know about. When done well, the end result is a smooth, elegant, effortless UI interaction, but your customer won’t recognize the tireless effort you put in to make it happen. And that’s how it should be. Is there any better way to serve your customer?

If you want a responsive software engineering team with reliable development schedules, here’s one simple rule:

Make sure that every developer’s checkin leaves your product in a state where it can be compiled, built, and demoed.

That’s it. Focus on that and the rest works itself out.

Why? Because you and your team are smart — if you can see problems, you’ll fix them. But you can’t react to what you don’t know. For every week/day/hour your gestating product is AWOL, your team ventures further into the unknown, adding unpredictability to your development cycle.

Sound difficult? It’s not. It takes effort and diligence, but pays back in spades.