Javascriptguy's blog

Long poll jsonp

admin — Tue, 04 May 2010 08:06:45 +0000

UPDATE: Implemented changes to the code as suggested by Akeru, thanks!
UPDATE2: Updated to the newer version of jQuery-jsonp, thanks Julian, the Ugly spinner loading is now gone!

Go straight to the code
or
Download it in a zip file

So, whatever happened to comet? Wasn’t it supposed to save the masses from wasteful ajax polling? I certainly appreciate Alex Russel’s work on the matter, but it really seems as though this very useful technology is falling to the wayside. I think this is mainly due to the difficulty and requirements of a successful implementation on the most popular web server, the main problem is Apache’s use of threading, this post explains the problem eloquently. Of course Apache have a whole project that deals with this issue to a degree, but I don’t think that it is easy to implement, once you dig a little into the documentation, you quickly see that this is not really a thing that just works out of the box… You need to read a book on the subject first. That is a bit of a deal-breaker for me; after all, the technique is not that difficult to understand, so why should you need to read a whole book to implement it?

So is comet dead?

No, not at all! As a matter of fact, cometd implements the bayeux protocol, and the technology is very much alive; I’m sure that it is very useful, for those who have the time, resources and patience to implement it, but where does this leave the rest of us? You know, the guys who want a quick way to get up and running with some instantly useful code that they can understand and tinker with? Well, comet-d does implement something they call “callback-polling”, and other people seem to call “script tag long polling”, though I prefer the term “long poll jsonp”.

Right, so now what?

Let us look at what exactly “long poll jsonp” is. I’m assuming you already know what jsonp is, so that just leaves “long polling”. To describe long polling, let us look at the backend and front-end parts in general terms.

Back end

On the back-end we simply have a program that doesn’t finish till it has some data for us. That’s all there is to it.

Front end

On the front end we make a jsonp request, which when finished, starts up again. We are of course passing a token that indicates what data we received, for example a timestamp.

That describes long polling in a nutshell; of course there are some pitfalls and issues to watch out for with this technique, such as error handling, (server timeout), and connection timing.

Show me the money code!

It is time to look at some code! Most comet example implement a simple chat client, so I’ll do the same here. First lets looks at the backend, as it is very simple.

/*
jQuery Long Poll plugin 0.0.1 (02-05-2010)

Note: requires jQuery-JSONP: http://code.google.com/p/jquery-jsonp/

This script is licensed as free software under the terms of the
LGPL License: http://www.opensource.org/licenses/lgpl-license.php

*/
(function($){

$.longPoll = function( args ) {
args = args || {};
var self = $.extend( {
url: ”, // URL To connect to
tsParameter: ‘ts’, // Parameter used for the timestamp
reconnect: true, // If we’re automatically reconnecting
errorTimeout: 60, // Timeout for errors, eg: server timeout, execution time limit, etc.
errorTimeoutDelay: 5, // Seconds to delay, if the connection failed in an error
hasConnection: false, // If we have a connection
timestamp: 0,
error: false,
request: null,
reconnectFunc: function() {
if( self.reconnect ) {
self.disconnect();
// Reconnect – with longer error timeout delay (in case we have an intermittent connection)
if( self.error ) {
setTimeout( function() { self.connect(); }, self.errorTimeoutDelay * 1000 );
} else {
setTimeout( function() { self.connect(); }, 100 );
}
}
},
connect: function() {
self.reconnect = true;
// self.request = $.ajax( {
self.request = $.jsonp( {
url: self.url + ‘?’ + self.tsParameter + ‘=’ + self.timestamp + ‘&callback=?’,
timeout: self.errorTimeout * 1000, // Timeout and reconnect
error: function(XHR, textStatus, errorThrown) {
self.reconnectFunc();
self.error = true;
},
dataType: "jsonp",
success: function( data ) {
self.timestamp = (data[self.tsParameter])? data[self.tsParameter]: self.timestamp;
self.success( data );
self.error = false;
},
complete: function( data ) {
self.reconnectFunc();
}

} );
self.hasConnection = true;
},

// Aborts request
disconnect: function() {
self.reconnect = false;
if( self.request )self.request.abort();
self.hasConnection = false;
},

// User defined
success: function( data ) {}

}, args );
return self;
};

})(jQuery);

So in the above plugin code, we create a longpoll object, and then setup a bunch of parameters and functions. The reconnectFunc is used to handle reconnection on error or completion of calls; the connect function uses the jQuery-JSONP library to connect to our url. The reason we’re using the jQuery-JSONP library is that natively, (as of version 1.4.2), jQuery’s support for jsonp is flawed. First of all it calls the success method twice (see: http://dev.jquery.com/ticket/6451), and secondly, it doesn’t support abort() correctly, which is a required function for our code to work. I’m sure that this will be fixed in the next version of jQuery.
So, as you can see in the code, we’re handling connection timeouts and general connection errors gracefully; of course I recognise that the comet-d library with the implementation of the bayeux protocol does this too, but on a much grander scale; my library isn’t intended to replace comet-d, it is simply here to provide an alternative implementation of long poll jsonp, as that is the most useful part of comet, to most people.
Let us look at the rest of the chat app, here is my HTML code:

JSONP Long Poll chat demo

So as you can see, we simply create an input field for the message, and a button to send the message; we then hook up some simple code to each part of the functionality.

Ugly spinner loading in some browsers

~~Of course this is just a technique, it isn’t perfect – in some browsers, the loading indicator continually shows that a page is loading; a future enhancement will probably fix that… probably.~~
Updated to the newer version of jQuery-jsonp, (thanks Julian!), the Ugly spinner loading is now gone!

There is a demo here, this is all served off the paxjs domain, to emphasise the cross-site nature of the technique:

Chat demo

CSS Formatter

admin — Mon, 01 Feb 2010 21:25:27 +0000

So I created a CSS parser and formatter on the weekend; I think it’s kinda neat as it formats CSS “just the way I like it”.
Why would I do this, when there are so many CSS formatters available? Well, the aim of most formatters is to “optimise” the CSS
(ie: make it as small as possible), whereas the aim of my formatter is to make it as readable as possible.
How many times have you inherited a project, only to find the CSS is a giant mess?
Many of the available parsers don’t have the option to display each entry on a single line, and most don’t keep the comments!
And none of them allows you to format the CSS so that each entry is on a seperate line, and the entries are
preceeded by X tabs, as long as the entry itself is less than X tabs long (though you may be able to configure CSSTidy to do this?). Let me demonstrate, take this CSS:

/* Spinner classes */
.narrowDark {
width: 16px;
height: 16px;
background: url( ../img/load_narrow_dark.gif ) no-repeat top left;
}

/* Validation classes */
.formField {
width: 1300px;/* The surrounding container for each form field */
clear: left;
}

.narrowDarkOLD { height: 16px; background: url( ../img/load_narrow_dark.gif ) no-repeat top left; width: 16px; }

.formField {
height: 32px; }

/* Menu */
.paxMenuDropDown{ background-color: #eee; border-bottom: 1px solid #aaa }

.paxMenuDropDown li.active a:hover {
text-decoration:none;
background-color: #00f;
color: white; border: none;
}

What I want to do is turn it into this:

/* Spinner classes */
.narrowDark,
.narrowDarkOLD { background: url( ../img/load_narrow_dark.gif ) no-repeat top left; height: 16px; width: 16px; }
/* Validation classes */
/* The surrounding container for each form field */
.formField { clear: left; height: 32px; width: 1300px; }
/* Menu */
.paxMenuDropDown { background-color: #eee; border-bottom: 1px solid #aaa; }
.paxMenuDropDown li.active a:hover{ background-color: #00f; border: none; color: white; text-decoration: none; }

Here we have:

One entry per line
All the comments are intact, and moved outside the class definition (you probably don’t want to do this with IE6 hacks)
Each property is sorted alphabetically
Duplicate entries are merged (narrowDark and narrowDarkOLD)
Repeated classes are consolidated (formField)
Nicely indented at 6 tabs

Check out the CSS formatter here.

WSG meeting was fun!

admin — Sat, 14 Nov 2009 07:16:50 +0000

Had a ton of fun presenting, and met some interesting people. Even some that want to help out with PAX which would be great!

Thanks very much to Russ for organising the night!!!

The future of Mozilla

admin — Tue, 03 Nov 2009 14:32:09 +0000

I attended the free Mozilla ~~booze-up~~ Labs event the other month; it was quite interesting; Ben Galbraith spoke about what Mozilla is up to, and what they are planning in the near future… It was interesting because I got the feeling that whilst they have a few interesting projects in the pipeline, they don’t really have anywhere specific to take their browser engine; it has gotten to a point where it performs on par with all the other browsers, and all the other browsers perform on par with it.
So where should Mozilla go from here? I think it is blindingly obvious what they could do, though I don’t think they’re gonna like my suggestion… Before I mention what it is, lets take a closer look at Mozilla.

What is Mozilla’s greatest asset?

Ben touched on it briefly in his presentation, but mostly he spoke about the other side-projects and other new and exciting bits and pieces…
It’s the browser, right!? … Isn’t it???
Well, no it isn’t the browser, there are plenty of alternatives which perform just as well, and do exactly the same thing; Mozilla’s greatest asset is their add-ons. This is their only unique selling point; there are literally thousands of high quality plugins that are very easy to install, and make the browser do things Mozilla could never have imagined, and best of all, they didn’t have to do any of the development for it! I see Mozilla’s add-ons a lot like Apple would see their app store for the iPhone, a free asset that others keep maintaining for them, with a common goal of market domination. No other browser has an add-on library like this… At least till now!

Enter Google Chrome extensions

Yes, google just launched their Chrome extensions, and Mozilla should be suffering in their jocks – as soon as Google have all the most handy extensions, there is not much of a reason to stay with firefox, it will be especially difficult, as Chrome will no doubt be pre-installed on many devices and OS.

So what should mozilla do then?

To put it plainly: drop their browser engine, and concentrate on their add-ons. In the ideal world we would have one browser engine, so that developing websites and javascript would be so much easier. Chrome and Safari already use the web-kit engine, so I’m suggesting that Mozilla write XUL to work on top of the web-kit engine, and concentrate on making their plugins work on it. I think Galbraith felt that he had done all he could at Mozilla, and the realisation that the future is not in the browser itself, but in the add-ons might be why he is no longer with Mozilla, and instead working on an App store for Palm…

WSG Nov 11th

admin — Mon, 19 Oct 2009 04:45:55 +0000

I’m speaking at the WSG about my PAXJS framework next month! You can register at the WSG website! I always enjoy the WSG meetings, there is a great mix of people from different areas and backgrounds, and much discussion anout web development in general. Note that the cost is $10 to attend.

Manage your data with the jQuery Collections plugin

admin — Sun, 20 Sep 2009 23:10:58 +0000

Download the script

It’s a classic problem: you have a small set of data records that you want your users to interact with at the same time, usually with a need for ordering of the records. I’ve seen this exact problem pop up in just about every project that I’ve every worked on. The most obvious solution to the problem is an inline-editable datagrid-style widget. Often this provides a “good enough” solution, and many UI designers convince their users to integrate this type of widget, as it is relatively easy to do.
This sometimes leads to dis-joined user experiences, inconsistent look & feel, depending on what solution is employed and how much time you spend on integrating it.

First, here are some examples of data grids:

Datagrid from extjs.com

Datagrid based on mootools

Datagrid from paxjs.com

These widgets have the following in common:

Data is displayed in a single row by default
When editing a row, the data from other rows is not always accessible
The look and feel is “like a spreadsheet”, and usually doesn’t offer too much in the way of customisation.

This is of course not too bad, as long as it is consistent with the rest of your application, and performs the necessary
tasks for your user. Some datagrids allow you to customise colours and fonts, but that is usually about as far as you can take
it without dwelving deeply into customisation and spending lots of time setting up work arounds for how the
grid is laid out and interacts with your data.

So how do you create a good solution, which easily fits in with your design, and back-end api without spending a
bunch of time customising the datagrid script, and laboriously hacking at the CSS?

The collection manager

The premise is quite simple, it works a little like a datagrid in that there is a concept of data-rows, but
it doesn’t matter what HTML you use for each row; this gives us flexibility to customise exactly how the layout
looks, and at the same time has enough smarts to help you manage the collection of items very easily.

To create a collection, you simply setup a template, a target element, a add button and initialise
the collection using the collections manager script.

Let us start with the most simple of examples.

Example: Hello world

In this example, you can add names – enter a few, and click the “Submit form” button to see how the values map to back-end objects.

The html

 id=‘collectionTarget0′ class=‘clearfix’>

class=‘clear’>

type=‘button’ id=‘addButton0′>Add item

We have a div as the target to render the collection manager into, plus an “add” button. The clearing div is optional,
depending on your layout requirements. This HTML is inside a standard form tag that submits to collections.php, which simply
shows you the contents of the submitted form.

The template

 id=‘collectionTemplate0′ style=‘display: none’>

Name:  name=‘name’ type=‘text’>

The template is where things get interesting; it is plain html, BUT, (and it’s a big BUT), there are no element IDs.
This is because the collections manager script takes the template, and adds IDs to the form elements as needed; it uses
the collection manager’s namePattern to name the field and give it a unique ID. This is a requirement, as we can
have multiple data rows.

The initialisation script

jQuery( document ).ready(function(){
jQuery(‘#collectionTarget0′).collections( {

template: jQuery(‘#collectionTemplate0′),

addButton: jQuery(‘#addButton0′)
} );

} );

The initialisation script simply brings everything together, it uses the template as the query target, specifies the
template element, and then the add button.

Let us look at a slightly more useful example, a customer profile – lets assume we have a data structure that maps to this JSON:

var customerProfile = {
name: ”,
comments: ”,

customerType: ['n', 'r', 'v'],

newsletter: true
};

The customerType maps to one of the following values: n = new, r = repeat, v = vip; news letter is true or false.

Example 1: Collection manager for customer profiles

In this example, we start with two rows of example data, and use the default settings for everything else. We also include a
delete button, so that you can remove collection items. Change some of the values, or perhaps add a new row, then click the
“Submit form” button to see how the back-end sees your values.

The html

 id=‘collectionTarget1′ class=‘clearfix’>

class=‘clear’>

type=‘button’ id=‘addButton’>Add

Again, we have a div and an “add” button. Also, the HTML is inside a standard form tag that submits to collections.php.

The template

 id=‘collectionTemplate1′ class=‘clearfix cp_collection’ style=‘display: none’>
 name=‘id’ type=‘hidden’>

 class=“span-3″>
 class=“span-3″>

 class=‘above’ for=‘name’>Name
 class=‘coll medium’ name=‘name’ type=‘text’>

class=“span-3″>
class=‘above’ for=‘customerType’>Customer Type

class=‘coll’ name=‘customerType’>
value=‘n’>New

value=‘r’>Repeat
value=‘v’>VIP

class=“span-9″>
class=‘above’ for=‘comments’>Comments

class=‘mediumText coll’ name=‘comments’>

class=“span-3″>
class=‘above’>Send newsletter

class=‘coll’ name=’spam’ type=‘radio’ value=‘true’>Yes

class=‘coll’ name=’spam’ type=‘radio’ value=‘false’>No

class=“span-2 last”>

class=‘above’>Action/>
type=‘button’ name=‘deleteButton’>Delete/>

The template is more complex than the hello world example, but it is just plain HTML with no IDs, just names for the form
fields. You will note that there is also a “Delete” button in this template, which allows us to remove items.

The initialisation script

jQuery(document).ready(function(){
jQuery(‘#collectionTarget1′).collections( {

template: jQuery(‘#collectionTemplate1′),

data: [
{ id: 1, name: 'David Lann', comments: 'David gets a discount at your discression', customerType: 'v' },
{ id: 2, name: 'Jill Jones', comments: 'call on 0295551231, or 0405551599', spam: 'true' }
],

addButton: jQuery(‘#addButton’),

deleteButtonName: ‘deleteButton’
} );

} );

The initialisation script now contains some initial data and a reference to the “Delete” button. The use of the
deleteButtonName parameter specifes that the script will attach a delete function to that button for each
record.

Example 2: collection manager with order

Here we expand on the previous example by adding ordering and move up / down buttons; this is a unique feature that the collection
manager has, compared to standard data grids: a concept of order.
As you move a field up or down, we persist the value of the order fields, so assuming the fields were in the correct order to start
with, we should have the correct order once we submit the form. Note that usually the order field would be hidden, but we’re showing
it here to make it easier to see what is happening.
Change some of the values, and move the fields around, then click the “Submit form” button to see the values, taking
note of the order field values.

The html

 id=‘collectionTarget2′ class=‘clearfix’>

class=‘clear’>

type=‘button’ id=‘addButton2′>Add

Again, we have a div and an “add” button. Also, the HTML is inside a standard form tag that submits to collections.php.

The template

 id=‘collectionTemplate2′ class=‘clearfix cp_collection’ style=‘display: none’>
 name=‘id’ type=‘hidden’>

 class=“span-3″>
 class=“span-3″>

 class=‘above’ for=‘name’>Name
 class=‘coll medium’ name=‘name’ type=‘text’>

class=“span-3″>
class=‘above’ for=‘customerType’>Customer Type

class=‘coll’ name=‘customerType’>
value=‘n’>New

value=‘r’>Repeat
value=‘v’>VIP

class=“span-9″>
class=‘above’ for=‘comments’>Comments

class=‘mediumText coll’ name=‘comments’>

class=“span-3″>
class=‘above’>Send newsletter

class=‘coll’ name=’spam’ type=‘radio’ value=‘true’>Yes

class=‘coll’ name=’spam’ type=‘radio’ value=‘false’>No

class=“span-3″>
class=‘above’ for=‘order’>Order

class=‘coll short orderField’ name=‘order’ type=‘text’>

class=“span-2 last”>

class=‘above’>Action/>
type=‘button’ name=‘deleteButton’>Delete/>

type=‘button’ name=‘moveUpButton’>Up/>

type=‘button’ name=‘moveDownButton’>Down

The template is almost the same as the previous example, except that we now have an order field.

The initialisation script

jQuery( document ).ready(function(){
jQuery(‘#collectionTarget2′).collections( {

template: jQuery(‘#collectionTemplate2′),

//  Initial data
data: [

{ id: 1, name: 'David Lann', customerType: 'v', order: '1' },
{ id: 2, name: 'Jill Jones', customerType: 'n', spam: 'true', order: '2' }
],

addButton: jQuery(‘#addButton2′),

deleteButtonName: ‘deleteButton’,
beforeAdd: function( data, args ) {

//  Set the order, using the collection count
var collectionCount = parseInt( jQuery( args.addButton ).attr( ‘collectionCount’ ), 10 );
data['order'] = data['order'] || collectionCount;

return data;
},

bindings: {

//  Move up collection, freezing the order field values; you could
//  optionally freeze other fields here

‘moveUpButton’: {
event: ‘click’,

func: function( e, collection ) {
jQuery.fn.moveUpCollection( collection, ['order'] );
}

},
//  Move down collection, freezing the order field values

‘moveDownButton’: {
event: ‘click’,

func: function( e, collection ) {
jQuery.fn.moveDownCollection( collection, ['order'] );
}

}
}

} );

} );

Example 3: collection manager for customer profiles with ajax delete

Yet again we expand on the previous example, this time we add ajax delete…
Change some of the values, and move the fields around, then click the “Submit form” button to see the values, taking
note of the order field values. We have also included the “namespace” parameter, which affects how the data is presented
to the back-end, but more about this later.

The html

 id=‘collectionTarget3′ class=‘clearfix’>

class=‘clear’>

type=‘button’ id=‘addButton3′>Add

Again, we have a div and an “add” button. Also, the HTML is inside a standard form tag that submits to collections.php.

The template

 id=‘collectionTemplate3′ class=‘clearfix cp_collection’ style=‘display: none’>
 name=‘id’ type=‘hidden’>

 class=“span-3″>
 class=“span-3″>

 class=‘above’ for=‘name’>Name
 class=‘coll medium’ name=‘name’ type=‘text’>

class=“span-3″>
class=‘above’ for=‘customerType’>Customer Type

class=‘coll’ name=‘customerType’>
value=‘n’>New

value=‘r’>Repeat
value=‘v’>VIP

class=“span-9″>
class=‘above’ for=‘comments’>Comments

class=‘mediumText coll’ name=‘comments’>

class=“span-3″>
class=‘above’>Send newsletter

class=‘coll’ name=’spam’ type=‘radio’ value=‘y’>Yes

class=‘coll’ name=’spam’ type=‘radio’ value=‘n’>No

class=“span-3″>
class=‘above’ for=‘order’>Order

class=‘coll short orderField’ name=‘order’ type=‘text’>

class=“span-2 last”>

class=‘above’>Action/>
type=‘button’ name=‘deleteButton’>Delete/>

type=‘button’ name=‘moveUpButton’>Up/>

type=‘button’ name=‘moveDownButton’>Down

The template is structurally the same as the previous example; of course the names of the fields are different.

The initialisation script

jQuery( document ).ready(function(){
var config = {

template: jQuery(‘#collectionTemplate3′),

//  Initial data
data: [

{ id: 1, name: 'David Lann', customerType: 'v', order: '1' },
{ id: 2, name: 'Jill Jones', customerType: 'n', spam: 'true', order: '2' }
],

addButton: jQuery(‘#addButton3′),

deleteButtonName: ‘deleteButton’,
beforeAdd: function( data, args ) {

//  Set the order, using the collection count
var collectionCount = parseInt( jQuery( args.addButton ).attr( ‘collectionCount’ ), 10 );
data['order'] = data['order'] || collectionCount;

return data;
},

bindings: {

//  Move up collection, freezing the order field values
‘moveUpButton’: {

event: ‘click’,
func: function( e, collection ) {

jQuery.fn.moveUpCollection( collection, ['order'] );
}

},
//  Move down collection, freezing the order field values

‘moveDownButton’: {
event: ‘click’,

func: function( e, collection ) {
jQuery.fn.moveDownCollection( collection, ['order'] );
}

}
},

//      Override the delete function, to send ajax request if necessary

deleteFunction: function( template, args, data, deleteButton ) {
jQuery( deleteButton ).text( ‘Deleting…’ ).attr( ‘disabled’, ‘disabled’ );

//      If we have data, send it to the server to delete, otherwise simply remove
var dataLength = 0;

for( var x in data )dataLength++;

if( dataLength > 0 ) {
//      Add delete parameter to the data

data['action'] = ‘delete’;

var request = jQuery.ajax( {
type: ‘POST’,

data: data,
url: ‘collections.php’,

dataType: “text”,
error: function() {

alert( “Ajax error occured” );
},

success: function( html ) { //      Remove collection
jQuery.fn.removeCollection(template, args, data);
}

} );
} else {

jQuery.fn.removeCollection( template, args, data );
}

},

namePattern: “contact{number}[{name}]“
};
jQuery(‘#collectionTarget3′).collections(config);

} );

The initialisation script now also has a deleteFunction parameter, with a function that ensures that we only
access the back-end if we need to; this allows the back-end to only

Back end integration

Of course the collections manager is useless without data from the back-end; it is very
easy to integrate the collection manager, thanks laregly to the namePattern parameter, which allows tight
control of how the fields are named; out of the box, you can quickly integrate with just about any php, python, java
(struts, springBeans), .NET, etc…

The namePattern parameter allows you to specify where the record number and name appears; in the last example,
we use a namePattern like so:

namePattern: “contact{number}[{name}]“

This creates individual objects for each data row, rather than the default of one array of data rows. This can be particularly
convenient if you have a set number of a type of object, for example if you have exactly four contact objects named “contact1″,
…, “contact4″, this will allow you to easliy map directly to those objects.
The default namePattern is:

namePattern: “collection[{number}][{name}]“

Which will map to one array named “collection” of data rows.

API

Here is a list of options, and what you can do with the collections manager

target – The target of the collection manager (defaults to jQuery result)
template – Element to use for cloning. Note: do NOT give any items IDs, they will be removed
data – Optionally include some data, default is 1 record
startCount – What number to start the count from.
minItems – Minimum number of items to show at any time (default 1)
maxItems – Maximum number of items to show (default 10)
namePattern – Pattern for naming the form fields (for mapping to back-end objects)
deleteButtonName – Name for delete button to use in the template
deleteFunction – Delete function override, params: ( template, args, data, deleteButton )
addButton – Button to assign the add function to.
bindings – name: {event:function}. WIP: we may be able to get rid of the specific addButton and DeleteButton events here.
beforeAdd – Function that runs before an item is added, params: ( data, args ), if we return anything, it will be used as data
afterAdd – Function that runs after an item is added, params: ( data, args )
beforeRemove – Function that runs before an item is removed – return false to cancel removal of item. Params: ( data, args )
afterRemove – Function that runs after an item is removed, params: ( data, args, wasRemoved )