Distributed Applications and Node.js

"My name is Legion; for we are many"

Created by @ajlopez

https://github.com/ajlopez/Talks/NodeDistributedApps

Using reveal.js

I am so old...

http://ajlopez.wordpress.com/2008/12/30/thirty-years-of-software-development/

JavaScript is Butter in Your Hands

Node.js: The next big thing!!

since sliced bread

http://www.toaster.org/slicingbread.html

Skynet Will Be A Node.js Distributed Application

http://terminator.wikia.com/wiki/Skynet

Agenda

  • Distributed Applications
  • Node.js features for distributed applications
  • Implementation blocks
    • Messaging
    • RPC
    • Broadcast
    • Bus
    • ...
  • Experiments

Some Notes

Alan Kay

The deep insights of Alan Kay
"I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages"

The big idea is messaging

Richard Feynman

"If you can't explain it, you don't really understand it." "What I cannot create, I do not understand."

Try yourself

Michael Faraday

"But still try, for who knows what is possible..."

Do experiments

So, Bring me a Brain, Igor!

Distributed Applications

Questions

  • Elements that produce messages?
  • Elements that consume messages?
  • A message is consumed by only one element?
  • Or by many elements?
  • A message is consumed locally or remotely?
  • How to send a message to other process/machine?
  • How to select the remote element?
  • Nodes can be added/removed dinamically?

A Typical Sample: Web Crawler

The logical elements can reside in multiple nodes

The Elements

ObjectStream

https://github.com/ajlopez/ObjectStream
Object to JSON and JSON to Object

SimpleMessages

https://github.com/ajlopez/SimpleMessages
Bidirectional Message Channels

SimpleRemote

https://github.com/ajlopez/SimpleRemote
Remote Objects with Callbacks

SimpleQueue

https://github.com/ajlopez/SimpleQueue
In-memory in-process queues

The Frameworks

Distributed Actors a la Akka

https://github.com/ajlopez/SimpleActors
  • Inspired by Akka (Java/Scala)
  • Each node has an actor tree
  • Each actor can be references by address (host@name/name)
  • An actor can create other actors
## System and Actors ```js // System var webcrawler = node.create('webcrawler'); // Actors var downloaderref = webcrawler.actorOf(downloader, 'downloader', { router: true }); var resolverref = webcrawler.actorOf(resolver, 'resolver'); var harvesterref = webcrawler.actorOf(harvester, 'harvester'); var serverref = webcrawler.actorOf(server, 'server'); ```

Spouts and Bolts a la Storm

Storm (Java Project)
  • A spout is a source of streams in a computation
  • A bolt processes any number of input streams and produces any number of new output streams

Storm in Node

https://github.com/ajlopez/SimpleStorm
  • Topology by Node
  • Inter-node Communication using Queues

Publish and Subscribe

https://github.com/ajlopez/SimpleBus
  • Publishers and Subscribers
  • A message can be received by many targets
  • Selective subscription (using a filter)

Fabriq in Node

Original Fabriq by @asehmi
https://github.com/ajlopez/AjFabriqNode
  • Each node has message processors
  • Routing of message based on content
  • Each node has its own message processors

Demo Experiments

Distributed Genetic Algorithms

https://github.com/ajlopez/SimpleGA

Distributed Fractal

https://github.com/ajlopez/NodeSamples/FractalSimpleGA

Distributed Backgammon (WIP)

https://github.com/ajlopez/SimpleGammon

Brain and Body

Try and take over the world!!

The End?

BY Angel 'Java' Lopez / @ajlopez