After searching the InterWeb for a decent MapReduce example coded in CoffeeScript I came up blank and decided to write my own. This one uses Mongoose too - well why use anything else?
I haven't written a whole lot of explanation in the text, but commented the code quite heavily so you can copy and paste and still have it with you.
Load Mongoose and connect to your database.Describe your schema and model and then load up some dummy data.
Now for the MapReduce stuff. If you still don't get MapReduce then you will once you read this Star Trek based article.
Define a Map function.
Define a Reduce function.
Get some output
The results will now be in a collection called "results" which you can look at with JMongoBrowser or whatever.
I hope this is of some use to you. If there are any corrections or improvements then please post something below.
Wednesday, 19 October 2011
Monday, 17 October 2011
Staying Up Late - CoffeeScript and Zappa
Over the last couple of years NodeJS has become the darling of "coding for pleasure" crowd. All that cross fertilization from the client side buffs into the server side and the server side coders ranting about the whole event-driveness of NodeJS.
Yes, it's the next big thing. Well, it's probably not, but it's certainly "A" next big thing or maybe we'll all get bored dream up something else with a catchy name... "perl for parallelism" or "Friendly Java (without the memory leaks)", or what about "C+=3" or "Oil for Android", "Lube for Palm OS". Well you know what I mean.
We see something; we explore it and once it starts getting a bit boring we convince ourselves and then our customers that we need to use some new technology or other. It's all true of course, or partly true, or lies.
Anyway, I'm not here to rant about that, after all I as guilty as anyone else.
What I want to rant about is Zappa which is my own personal cure for technical disfunction at the moment. Well that sounds real bad, but I've been doing this job for years and it's only these little affairs over the years that keep me coming home; "Assembler", "C", "C++", "Ruby", "Python", how I remember their sweet names, but sooner or later they all start to look like their mother and have to go.
Zappa is a young virgin of a technology, only now reaching a pubescent version 0.3. Essentially it's a lightweight framework built on top of known technologies such like NodeJS, CoffeeScript, Express and Socket.IO in much the same way as Python's Flask is built on top of Werkzeug, Jinja 2 and so on.
It's that Socket.IO bit that makes this one so sexy (I'm overdoing the procreation metaphors aren't I?). Socket.IO provides cross-browser WebSocket goodness which means you can do stuff like stop using the Model-View-Controller paradigm and start thinking outside the cubical with your "Model-View-Controller-View-Controller-Controller-View Paradigm" or whatever. Mind you, that's when you'll realise that MVC wasn't about limiting your options, it was about simplifying them.
Anyway, back to Zappa itself. Go and have a look, there are great examples that will get your inventive juices flowing (stop it!) and inspire you to spend a little time alone locked it the bathroom... I mean office. The community is starting to grow and the mailing list is big enough to be useful, but not so big as to be faceless and scarey.
Have fun but take precautions.
Yes, it's the next big thing. Well, it's probably not, but it's certainly "A" next big thing or maybe we'll all get bored dream up something else with a catchy name... "perl for parallelism" or "Friendly Java (without the memory leaks)", or what about "C+=3" or "Oil for Android", "Lube for Palm OS". Well you know what I mean.
We see something; we explore it and once it starts getting a bit boring we convince ourselves and then our customers that we need to use some new technology or other. It's all true of course, or partly true, or lies.
Anyway, I'm not here to rant about that, after all I as guilty as anyone else.
 |
| A Messy Affair |
Zappa is a young virgin of a technology, only now reaching a pubescent version 0.3. Essentially it's a lightweight framework built on top of known technologies such like NodeJS, CoffeeScript, Express and Socket.IO in much the same way as Python's Flask is built on top of Werkzeug, Jinja 2 and so on.
It's that Socket.IO bit that makes this one so sexy (I'm overdoing the procreation metaphors aren't I?). Socket.IO provides cross-browser WebSocket goodness which means you can do stuff like stop using the Model-View-Controller paradigm and start thinking outside the cubical with your "Model-View-Controller-View-Controller-Controller-View Paradigm" or whatever. Mind you, that's when you'll realise that MVC wasn't about limiting your options, it was about simplifying them.
Anyway, back to Zappa itself. Go and have a look, there are great examples that will get your inventive juices flowing (stop it!) and inspire you to spend a little time alone locked it the bathroom... I mean office. The community is starting to grow and the mailing list is big enough to be useful, but not so big as to be faceless and scarey.
Have fun but take precautions.
Wednesday, 13 July 2011
Back of an Envelope Node.js Framework - Discuss
I had a quiet afternoon today and while "keeping up with recent developments in my field" or as you might call it "wasting my afternoon googling" I got to thinking what I'd like a Node.js MVC framework should or could contain.
Firstly, I've been playing with Node.js for about three weeks now and I'm impressed, but I can't seem to find a framework that I'm happy with. Backbone looked good, but I wasted three days on it and it was too much like hard work. Similarly Spine which tried to be less complicated, but eventually failed. Finally I really wanted Brunch to work but it was a release behind backbone (which is a major component of it) and I didn't want to get lost in the mismatching documentation (the Backbone release was pretty major).
I have spent a whole lot of time over the last couple of years using python frameworks, especially Flask and I like their approach of starting with a good foundation (in their case Werkzeug) and adding the bits that are still needed. Could that work with Node? I mean writing a basic HTTP server looks pretty straight forward and there are tons of modules to add the other functionality.
So with this possibly naive starting position I grabbed my pen and scribble down some stuff I thought I'd need. Now bear in mind that I haven't much used any of these modules yet, and I pinched several of them from the bits that go to make up Brunch.
I'd appreciate any feedback, suggestions and so on that those more learned than I might care to offer. Few of you will be less learned I imagine.
Here are the transcribed notes with some links:
Server
Models
-CSS
-Templating
Controllers
If you spot any typos, inaccuracies or just plain lies then please let me know.
 |
| How design shouldn't be done |
Firstly, I've been playing with Node.js for about three weeks now and I'm impressed, but I can't seem to find a framework that I'm happy with. Backbone looked good, but I wasted three days on it and it was too much like hard work. Similarly Spine which tried to be less complicated, but eventually failed. Finally I really wanted Brunch to work but it was a release behind backbone (which is a major component of it) and I didn't want to get lost in the mismatching documentation (the Backbone release was pretty major).
I have spent a whole lot of time over the last couple of years using python frameworks, especially Flask and I like their approach of starting with a good foundation (in their case Werkzeug) and adding the bits that are still needed. Could that work with Node? I mean writing a basic HTTP server looks pretty straight forward and there are tons of modules to add the other functionality.
So with this possibly naive starting position I grabbed my pen and scribble down some stuff I thought I'd need. Now bear in mind that I haven't much used any of these modules yet, and I pinched several of them from the bits that go to make up Brunch.
I'd appreciate any feedback, suggestions and so on that those more learned than I might care to offer. Few of you will be less learned I imagine.
Here are the transcribed notes with some links:
Server
- Simple Node.js
- Express
- choose a module
- Express?
Models
- JS Object/JSON?
- ORM?
-CSS
-Templating
Controllers
- Plain Old JS
- CoffeeScript?
- jQuery
- Underscore.js
- Quip (responses) ?
- how? Express
- node-boilerplate?
- Stitch (glue)
- Cake
If you spot any typos, inaccuracies or just plain lies then please let me know.
Monday, 27 June 2011
Binding Updates to Model Changes in Backbone/CoffeeScript
My previous post showed a View that looked like this:
But what happens if we have a more complicated app and something else changes the Model?
Well as I said I'm just learning myself here, co the correct way that should be handled is to bind Model changes to the Views render function which looks like this:
Note the bind instruction which means that every time the counter changes, it calls the render function. You might also notice the fat arrow now on the render which make "this" still mean the View rather than the caller (which would be the Model otherwise.)
The explanation for all this is at http://jashkenas.github.com/coffee-script under "Function Binding".
But what happens if we have a more complicated app and something else changes the Model?
Well as I said I'm just learning myself here, co the correct way that should be handled is to bind Model changes to the Views render function which looks like this:
Note the bind instruction which means that every time the counter changes, it calls the render function. You might also notice the fat arrow now on the render which make "this" still mean the View rather than the caller (which would be the Model otherwise.)
The explanation for all this is at http://jashkenas.github.com/coffee-script under "Function Binding".
Sunday, 26 June 2011
Simplest Ever Backbone.js/CoffeeScript Example
I'm teaching myself to use Backbone because after having done quite a lot of JavaScript in work lately, it has become obvious to me that I need a framework.
I'm also learning how to use CoffeeScript, which is close enough to python/ruby for me to understand.
First thing that I noticed is that most of the examples for Backbone out there are in pure JavaScript, so I thought I'd do my bit by providing something in CoffeeScript.
To that end, here is a very simple app which displays a number. You can increase or decrease the count by clicking buttons.
The app.coffee code:
and the html to go with that:
In detail
At the top, we set up global hashes to store our objects. In a bigger app this would make referencing bits and pieces much easier, although we don't really need it in this example.
Next the controller. This one is almost empty except it has two routes both taking you to the home control which is also part of the controller. The route with no reference (the empty string) and the one for #home tagged on the end of your URL (by way of an example) both call the home function in this case.
The home control just tells the view to render itself.
Next the Counter model which has one attribute called count which defaults to zero. In reality I think you would want to synchronize your models with a database, but we'll do that in another post soon.
And now for the View. The View is obviously what you are seeing in the browser. In our trivial example we only have one view, but a real app would probably have several. A view lumps together a unit of the app, sometimes nowadays we call these widgets, or blocks or display regions or whatever. You get the idea.
In the initialize function, we just connect the view to the model that backs it. The events connect the button to the functions that they call. The render function draws the result into the #count div; the @ symbol at the end is shorthand for this. In CoffeeScript (like Ruby) the last line of a function gets returned even if you don't specifically say "return". So in other words it means "return this". This is handy since it lets us chain together functions.
The inc and dec functions increase and decrease the count in the model and then update the display. Please see the update here.
Finally the equivalent of the main() function which instaniates the bits and bobs and then starts the history. I've no idea why they called it history, but basically it starts up all the listeners.
That's all for now. Next time I'll show you how to connect all this to your database using python and Flask.
If you see anything wrong with what I've written here or if you spot spelling or other mistakes then please let me know.
I'm also learning how to use CoffeeScript, which is close enough to python/ruby for me to understand.
First thing that I noticed is that most of the examples for Backbone out there are in pure JavaScript, so I thought I'd do my bit by providing something in CoffeeScript.
To that end, here is a very simple app which displays a number. You can increase or decrease the count by clicking buttons.
The app.coffee code:
and the html to go with that:
In detail
At the top, we set up global hashes to store our objects. In a bigger app this would make referencing bits and pieces much easier, although we don't really need it in this example.
Next the controller. This one is almost empty except it has two routes both taking you to the home control which is also part of the controller. The route with no reference (the empty string) and the one for #home tagged on the end of your URL (by way of an example) both call the home function in this case.
The home control just tells the view to render itself.
Next the Counter model which has one attribute called count which defaults to zero. In reality I think you would want to synchronize your models with a database, but we'll do that in another post soon.
And now for the View. The View is obviously what you are seeing in the browser. In our trivial example we only have one view, but a real app would probably have several. A view lumps together a unit of the app, sometimes nowadays we call these widgets, or blocks or display regions or whatever. You get the idea.
In the initialize function, we just connect the view to the model that backs it. The events connect the button to the functions that they call. The render function draws the result into the #count div; the @ symbol at the end is shorthand for this. In CoffeeScript (like Ruby) the last line of a function gets returned even if you don't specifically say "return". So in other words it means "return this". This is handy since it lets us chain together functions.
The inc and dec functions increase and decrease the count in the model and then update the display. Please see the update here.
Finally the equivalent of the main() function which instaniates the bits and bobs and then starts the history. I've no idea why they called it history, but basically it starts up all the listeners.
That's all for now. Next time I'll show you how to connect all this to your database using python and Flask.
If you see anything wrong with what I've written here or if you spot spelling or other mistakes then please let me know.
Saturday, 25 June 2011
Set Up Your Own Git Repository Hosting Service in Three Minutes
You might have seen my earlier post "A Review of Free Code Repositories".
Well I forgot about hosting your own. Really handy for hobbyist development. I have one of those free developer servers from Amazon out there somewhere. I don't use it much except to demo the occasional site and so on.
I decided that it was a good idea to set it up as a git repository and i show you here how to do this. If you are using something Ubuntu-ish remotely then this should take about three minutes.
A Word of Explanation
This is the simple setup, it avoids creating 'git' user on the remote server, but you DO need to use a seperate key for using the git service and I show you that first.
On your local machine:
On your remote host you are now logged in as 'ubuntu':
On your local machine:
That's it really. To create a new repository called dummy and initialize it, you can do something like this.
On your local machine:
You'll be using that last code section again, so you might want to make it into a script.
Never type anything you don't understand.
If I've made any mistakes then please let me know.
Well I forgot about hosting your own. Really handy for hobbyist development. I have one of those free developer servers from Amazon out there somewhere. I don't use it much except to demo the occasional site and so on.
I decided that it was a good idea to set it up as a git repository and i show you here how to do this. If you are using something Ubuntu-ish remotely then this should take about three minutes.
A Word of Explanation
This is the simple setup, it avoids creating 'git' user on the remote server, but you DO need to use a seperate key for using the git service and I show you that first.
On your local machine:
# echo "Host gitserver Hostname ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com User git-col PreferredAuthentications publickey IdentityFile ~/keys/git-col.pub " >> ~/.ssh/config # mkdir ~/keys
# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/col/.ssh/id_rsa): git-col.pub # do NOT accept the default
# cp git-col* ~/keys # scp git-col.pub ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com: # ssh ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com
On your remote host you are now logged in as 'ubuntu':
# sudo apt-get install git # git clone git://github.com/sitaramc/gitolite # ./gitolite/src/gl-system-install # gl-setup -q ~/git-col.pub # exit
On your local machine:
# git config --global user.name "Your Name" # git config --global user.email your@email.address # git clone gitserver:gitolite-admin # exit
That's it really. To create a new repository called dummy and initialize it, you can do something like this.
On your local machine:
# set the new repository name
export repo=dummy
# update the gitolite conf file with your new repository
cd ~/gitolite-admin
echo " repo ${repo}" >> conf/gitolite.conf
echo " RW+ = @all" >> conf/gitolite.conf
# push the config to your server
git commit -m "added new repository ${repo}" conf/gitolite.conf
git push
# make sure you cd to an existing directory!
mkdir ~/Projects
cd ~/Projects
# pull back the new empty repository
git clone gitserver:${repo}
# go to repository and make any change
cd ${repo}
echo "*.pyc" >> .git/info/exclude # I use python
touch README
# do your first push back to master
git add .
git commit -am "repository ${repo} pushed to origin on gitserver"
git push origin master
You'll be using that last code section again, so you might want to make it into a script.
Never type anything you don't understand.
If I've made any mistakes then please let me know.
Tuesday, 21 June 2011
Flask-Coffee - Fill your flask with coffee
Flask-Coffee is a Flask extension that compiles .coffee CoffeeScript files into .js JavaScript files for you if they have changed before your app renders.
This is a nice idea in development and not so nice an idea in production, so please keep that in mind.
Prerequisites
You will need to have to have Node.js installed with NPM and coffee-script.
You can follow the instructions in my previous article to install Node.js and then it's just
Installation
Install Flask-Coffee with pip:
Usage
This will watch your app’s static media directory and automatically render .coffee files into .js files in the same (sub)directory.
The best way to incorporate Flask-Coffee into your development is as described for flask-lesscss in my earlier flask-script tutorial.
Contribute
If you want to contribute email me at the email at http://pypi.python.org/pypi/Flask-Coffee after checking out the code at http://bettercodes.org/projects/flask-coffee.
This is a nice idea in development and not so nice an idea in production, so please keep that in mind.
Prerequisites
You will need to have to have Node.js installed with NPM and coffee-script.
You can follow the instructions in my previous article to install Node.js and then it's just
npm -g install coffee-script
Installation
Install Flask-Coffee with pip:
pip install flask-coffee
Usage
from flaskext.coffee import coffee coffee(app)
This will watch your app’s static media directory and automatically render .coffee files into .js files in the same (sub)directory.
The best way to incorporate Flask-Coffee into your development is as described for flask-lesscss in my earlier flask-script tutorial.
Contribute
If you want to contribute email me at the email at http://pypi.python.org/pypi/Flask-Coffee after checking out the code at http://bettercodes.org/projects/flask-coffee.
Subscribe to:
Posts (Atom)