2010-10-04

Global Pub-Sub

I've started working on a universal messaging bus that would allow any number of components to broadcast "events" to any other number of listeners (the publication-subscription, or observer, pattern.) Since I'm a sucker for playing with new tools, I've decided to build the bus in node.js. The basic idea is to allow a client in one process to subscribe to events broadcast by other clients in other processes by subscribing to a "central" bus.

In practice, each broadcaster will have a local bus object to which they publish. That bus serializes the message and sends it to a central server. The central server checks the message for validity, then pushes the message to any connected listeners. There will also be an HTTP connector that will store events for a period of time to allow AJAX listeners to periodically connect and pull down waiting events.

The listeners will have a local bus object to which they can subscribe. When an event is pushed to the listener object by the server (or pulled down in a long poll by an AJAX client), the bus un-serializes the message and pushes it to the registered listeners for that event type.

To the publishers and subscribers, everything appears to be happening local to the process.

What does this look like codewise?
// Publisher client
var bus = new Bus(server, port);
bus.broadcast('someEvent', arg0, arg1);

// Listener client
var bus = new Bus(server, port);
bus.addListener('someEvent', function (arg0, arg1) {
    // do something with the information
});


Ideally, I'd like to include publisher-subscriber clients in several languages. A PHP subscriber would use a callback:
function handleSomeEvent($arg0, $arg1) {
    // do something with the information
}

$oBus = new Bus($server, $port);
$oBus->addListener('someEvent', 'handleSomeEvent');

The publisher-subscriber client could use a TCP, WebSocket, or HTTP long-poll to communicate with the remote bus. A future iteration might allow for multiple remote bus nodes to push events to each other to then be pushed down to individual client listeners.

More information as it develops.

2 comments:

  1. Hey Josh,

    This is really similar to what we hope people will be using Kwwika for - a global data sharing bus. Great idea!

    Cheers,

    Phil

    ReplyDelete
  2. @Phil: I was really doing it for more of a learning exercise. I recognize there are libraries out there that do this already, and probably in a better way. I'll definitely check out Kwwika for a future project that has a need for a global bus. Thanks for the input!

    ReplyDelete