The Devel!

hquery: an unobtrusive server script implementation

I'd previously blogged about some ideas to resolve the developer-designer gulf regarding templates. I'm happy I've finally found time to code up a proof-of-concept:

hquery - the unobtrusive server script from choonkeat on Vimeo.

Q: What is "unobtrusive server script"?

If "unobtrusive javascript" adds interaction to a web page without being inside the markup - then "unobtrusive server script" replaces data in a web page without being inside the markup (and design).

You'll get a clean separation of 1) designed web page is constructed, then 2) server data is weaved in, and finally 3) javascript hooks up the interactive portions.

The key thing is that all 3 layers are stringed together by CSS path - a concept / language that both designers and developers can communicate with.

Q: Is this a silver bullet?

No. There are constraints: everything that needs to be dynamic, need its own CSS/XPath. e.g. Google's result page markup isn't very suitable:

  <div>this is abstract<cite>citation</cite></div>

Notice how we can't address the abstract content without stepping on <cite>. When using USS, we'll need the markup to be:

  <div><span>this is abstract</span><cite>citation</cite></div>

Then both abstract and citation can be targeted independently. Furthermore, to be totally isolated from markup changes, it is best to only use classes and ids without the tag names.

Q: Is USS a Ruby on Rails thing?

Nope. "hquery" is just a library I wrote to do USS in Rails. I believe any language / framework can have its own USS renderer.

Q: How do I write javascript that needs dynamic values?

Instead of writing mixed-up code like this:

<script>
  alert("I am <%= user.name %>");
</script>

Try doing it this way (separating data from logic):

<div id="json"><script>var user = {name: 'foo'};</script></div>
<script>
  alert("I am " + user.name);
</script>

Then in your script, you can swap out the JSON data:

select "#json script" do |ele|
  ele.html "var person = " + current_user.to_json   
end

Q: Where next?

I dunno. Since hquery script is declarative, we can compile to regular <% messy %> template as runtime? Or, have build tasks that can check that corresponding HTML contain necessary elements - i.e. a new design don't break/miss out anything?

Anything else? Love to hear your thoughts!

PS: I find myself trusting Yahoo search results when its in Google skin!

Something's Gotta Give

Saw this clarification on TC regarding Twitter

the issue certainly isn’t with creating new messages, the issue is when you check your own messages For eg, if I am following 200 people, the query needs to check the new messages from all those people and sort them into date order. That query would be really intense ^{link »}

- from Twitter At Scale: Will It Work? via sharedcopy.com

I'd heard that earlier in the Gilmor Gang : Blaine clarified that static resources (e.g. profile images) are no issue at all, and that the the bigger issue is with each person's "home page", where multiple streams are joined and ordered chronologically. When 1 of the connected people update, all related cached "home pages" are obsolete - hence they are generated painfully often. (so don't refresh your own page so much!)

I'm not a scaling expert. Nor do I know anything about Twitter's real problems. So treat this as noise maybe.

Anyways, ignoring other things unknown and purely focusing on this particular aspect... I thought it'd be interesting to consider what we might gain, if we're just willing to give up certain things?

So.. assuming static resources ain't a problem, lets try to have more of those! A single person's stream (without friends) looks cachable - like a blog that generates html instead of serving dynamic. A single person's social graph (following who) looks cachable as well. If assumptions are wrong here, then exit(1); So a person's homepage is his own static stream with static external references to static streams of ppl he follows. Like a html page referencing a bunch of CSS. No server side join or sort. Instead, the browser, with the help of some static javascript, pulls in those resources, does the merging and sorting (and paging) and spit a pretty display similar to the current. i.e. every user's browser does its own merging and sorting, you know... like, help out will ya?

Would that be better? And what did we give up? The ability to properly browse your own Twitter homepage on a browser without JS & clients using API need a few more pulls and do merging / sorting themselves too.

Can we live with that?

Update: Slim pointed to an article in the comments, and "Part III" of that article, Eran had talked abt the same alternative:

if you only ask this once, the server side batch solution will be faster. But when the client starts asking this once a minute or more repeatedly, resources go to waste and scalability is more expensive. ^{link »}

Once the server is only serving data with limited scope, usually providing the messages of a single person with the optional perspective of the reader (to enforce access control), scaling becomes a much easier task as data can be segmented easier. ^{link »}

- from Hueniverse: Scaling a Microblogging Service - Part III via sharedcopy.com

Update: Then again, you have scoble who follows 30k other people making it impractical for the client to fetch those streams independently... I guess most systems would decide to place limits and "punt" the issue, e.g. you can have at most 5000 friends. And that's something Twitter is unwilling to let go.

A Clearer Layout For Viewing Timezones

After putting up a job post for SharedCopy at the 37signals job board, I've gotten quite a fair bit of response and in turn, had to propose timings for Skype calls very often.

I hate timezones.

For my purpose (i.e. propose a time where all parties are at least awake!) I figure I'd just need a simple view of everybody's clock over a period of 1 day - which can be done in Ruby:

24.times do |hh|
  puts [+8, -7, -5].collect {|tz| Time.now.utc + hh + (tz * 3600) }.inspect
end

Functional, but a chore to run and the output is still a mess to pick out information from. Then I did a proper one in Javascript, in case I ever need to use it while on the move... ;-)

[Update] A few notes:

• The darker/shaded areas are ZZZ hours - so just look for horizontals where the "whites" align.
• Your own timezone should automatically show up - so most usage is only 1 click.
• URL changes as you add timezones - so you can even bookmark the page and remember a standard set of timezones.
• Day-of-week is subtly indicated just so you can tell if its the same day or otherwise.
• Name of location is used instead of numeric offset (and auto resized to fit) for a more humane experience

 

Try it out at http://dev.choonkeat.com/js/cleartz/index

Singapore-RB meetup: OAuth4R Presentation PDF

Here are my slides for the OAuth and OAuth4R presentation I made this evening. Thanks folks.

| View | Upload your own

The "No Special Case" Pattern

Very often I'll write (or use) APIs that look like this (translate accordingly for Ruby / Javascript code)

Thing findOneThing(Key  key);
// if nothing is found, throw an Exception or return null

Collection findManyThings(Collection keyList);
// if nothing is found, return null

All well and good? I'd known the textbook-bullet-points for this, but for the longest time, I didn't realize how much tax I'm paying subconsciously.. and just how many APIs are ignoring this wisdom:

Nulls are awkward things in object-oriented programs because they defeat polymorphism. ^{link »}

Instead of returning null, or some odd value, return a Special Case that has the same interface as what the caller expects. ^{link »}

Not all language can do this easily ^{link »}

- from P of EAA: Special Case via sharedcopy.com

Finding yourself having to check for null return values? That's not "the way it is", that's tax.

Ruby helps a bit by making "nil" an object, paving way for the much loved duck-typing. Whereas some languages settle for dead bodies equivalent nulls. Especially in these languages, having to "return a empty object with the same interface" is a real chore. Person? EmptyPerson. Record? EmptyRecord.. ad infinitum

Still, a language can really only help so much. And it is up to the library authors (i.e. you) to put this wisdom into practice. "Practice?". Now comes the funny part.

When talking about "design patterns", usually images of "Enterprise", "UML", "Configuration files" start appearing in my mind. So I find it ironic that the best real example of this pattern comes from the lowly-regarded (by many "real" engineers) realm of Javascript. Its called jQuery.

In fact, jQuery takes it even further  by simplifying Martin Fowler's pattern into a single practical use-case: Returning 1 object is equivalent to returning a List containing 1 object. By always returning a List (which can be empty, duh), it has even avoided the need to conjure special "null object with equivalent interface"! And most importantly, "practical" because all languages can do empty list.

Its amazing how simple your train of thought (and code!) becomes when everything is a list (tiny voice: uh.. Lispers knew that since before you were born... blah blah). There's nothing to decide, you always loop through the return value. 0 time? 1 time? N times? Imagine all those other idioms & body of code, now becoming unnecessary... its applying liposuction for your source code!

I don't suppose its too far fetched to apply some jQuery-ism to ActiveRecord API is it?

Update: Changed title from "Unspecial Case" to the "No Special Case", which is more correct.

Learning Web Development

Much in the spirit of "Skinny Controllers and Fat Models", I'm coming round to the fact that Single-Table Inheritance for is the evil best avoided.

All that code-reuse might look sexy and get all the OOP noddings. But in the longer run, the risk is that my code (esp. domain/business logic) end up as big balls of mud. But this "risk" is taken in the bid to stave off something more imminent - code repetition. SQL everywhere? No, wrap in a function! No, wrap in a Class! No, inherit from that BaseClass! Fewest blocks to build the tallest cathedrals - architectural elegance.

But buildings don't evolve. When requirements evolve, aka "reveal their true selves", the models start to diverge bit by bit and my OOP-instinct intuitively hold them together. Then one day, often when its too late or a new team member joins, I realise the models are really not of the same family anymore. To my chagrin, the glue holding the inheritance were more about preventing them from stepping over each other. And the result is worse than the repetition I'd avoided.

Another related issue is RESTful controllers - "Oh, so 2006" you say. Sorry I'd only drank half the CRUD kool-aid. Now I'm feeling the pain on those parts where I'd chosen to go the other way when there's a Model involved. Its the kind of pain you know only when you're straddling between 2 worlds. Bottom line: not worth over-thinking about controllers. I'm also certain those content-type negotiation can be abstracted out of sight. What's the fun in repeating this:

responds_to do |format|
  format.js { render :json => @object.to_json }
  format.xml { render :xml => @object.to_xml }
  format.html {  }
end

And if I jump both feet into doing "Presentation Layer" in CSS/Javascript only, a lot of "view" can actually go away - leaving only layouts.

So yes, bliss will be the day I only define schemas, write Models, sprinkle CSS and my app is done. Like my colleague (and Prolog admirer) always say, "we only need defined data; everything else is easy". 

PS: I used to roll my eyes at that, saying it is too academic. In the real world, we often don't have the luxury of data being fully & correctly defined for us. Requirements change, stay agile, live with it! But I should've looked at it in another way: "we only need defined [to be defining] data; everything else is easy [can always write themselves]"

True False parameters

Every so often, I get the chance to use functions that accepts a nondescript boolean as parameter... or *gasp* a sequence of nondescript booleans.

api.connect( true )

It's a chore for the maintainer (usually my future self) to have to look up what "true" meant. IDEs might be helpful at this point, bringing up code hints and what not.. unless of cos the function declaration itself isn't exactly perfect...

def connect(bool)
  ...
end

I've finally come round to do myself a favor (ok, I AM going to be the maintainer) by using self-explanatory truthy / falsey values instead

api.connect(  :auto_retry ) # true
api.connect( !:auto_retry ) # false

Reads better. No?

Drop that AJAX.. step back and listen up. NOW

Before you get too dangerous... get properly lectured on Javascript.

This is really good stuff. I'm floored by Video #3. Watch all of it. Don't skip. With the slides.

Proudly use your office hours to watch it. And tell your boss, "You're welcome."

PS: I feel like I just discovered the doormat I've been stepping on for years is actually a magic carpet...  

Yet Another Tool To Program In Workflow

Nice interface - it looks vaguely familiar actually. I wonder if Coghead intend to woo the enterprise crowd and do a Save-as BPEL ;-)

If a developer using a Coghead application can switch (with a click on a bookmarklet, and authentication) into inspect/debugging mode on the current application's current page that could be swell.

Designer-Developer Gulf

Vitamin has an article about understanding & working with designers. I'm not sure I took away anything other than knowing I'm not alone afterall.

The gap between the creative and developers is actually quite substantial. If I could naively reduce the problem down to tools, then this analogy could succintly describe the issue: You are tasked to co-author a book using Wordstar2000 with somebody using Microsoft Word Powerpoint - somewhere along the line, one party will need to close that gap. In this part of the world, more often than not, its the developers that closes this gap (not always doing a good job, me included). Otherwise, the designer and developer will be the same person. That's where you get flash-only websites (those that lets you click Skip flash presentation using a flash button), and freshmeat.net.

A site's design is carefully thought out, planned and template pages are created, "here's the main page, inner pages, forms and blah. now, go forth and multiply!". The programmer takes this goodie bag of hair, nose, eyes, ears and proceeds to construct the face.. do some code to make it blink... sprinkle the eyes here.. do some code for smiling... plaster the lips there... voila! At this point, the creative side has 2 options, iterate over the design or throw his hands up and the product ships.

Through iteration, hopefully the programmer learns design better and designer understands technical constraints better. You'd think that after many iterations, projects after projects, this gap ought to come pretty close right? Strangely, it often doesn't. We all know programmers has no taste, so that's a dead end. And I know of online marketing companies that has designated "html cutters" (in case you can't picture it, their fulltime job is to take a piece of museum-ready artwork done by the creatives, and html-ize it. Man... )

What will it take for the situation to improve?

Few months back, during RailsConf, I had the good fortune to chat with Bryan Wood of Lendmonkey. He related to me how great the process was to work with Amy Hoy for their Railsday project: Freckle.

1. Start coding up some pages, forms and the skeleton app
   
2. Commit the files into the repository.
   
3. Continue to work on the applications (like, erm, make it actually work)
4. Time to time, synchronize your code with the repository
5. After a while, during one of the updates, lots of files and images are downloaded... you refresh your page and WOAH! Your web app looks pretty!
   

This 5 bullet points covers the web development process from the developer's point of view. A designer that closes the gap. Sounds surreal to me. A dream come true. I have never seen it in action.