Install Google App Engine on Ubuntu 12.10

Google App Engine  is currently on version 1.7.4 and Ubuntu has recently released Ubuntu 12.10 (Quantal Quetzal). Quantal Quetzal comes with Python 2.7 installed, and App Engine has been providing that version of Python as an option since February 2012. So if you are starting a new App Engine project, it's probably a good time to move to Python 2.7.

I'll explain briefly how you start a new project and there's a nice clean copy of the code at the bottom that you can cut and paste.

Let's get the show on the road. Choose a name for the project and create and switch to a virtual environment:

PROJ=gae_project
mkvirtualenv ${PROJ}
cdvirtualenv

Note that note that "--no-site-packages" and "--distribute" are now the defaults for mkvirtualenv. You don't even need to use "--python=python2.7" on Ubuntu 12.10.

Now we need to know what the latest version of App Engine is, but as of writing it's 1.7.4:

GAE=1.7.4
wget -O /tmp/gae.zip http://googleappengine.googlecode.com/files/google_appengine_${GAE}.zip
unzip /tmp/gae.zip

Now let's create an App Engine app. The app will need a name that has been created in the App Engine Console:

GAE_APP_NAME=dummy
mkdir -p gae_app/static

Now create the app.yaml file:

echo """application: ${GAE_APP_NAME}
version: development
runtime: python27
api_version: 1
threadsafe: true

default_expiration: 7d

handlers:
- url: /static
  static_dir: static
- url: .*
  script: wsgi_app.app
""" > gae_app/app.yaml

And finally the app itself:

echo """import webapp2

class MainPage(webapp2.RequestHandler):
  def get(self):
      self.response.headers['Content-Type'] = 'text/plain'
      self.response.out.write('Please replace me with a decent WSGI App Framework such as Flask')

app = webapp2.WSGIApplication([('/', MainPage)],
                              debug=True)
""" > gae_app/wsgi_app.py

And finally to run the development server:

python ./google_appengine/dev_appserver.py gae_app/

I hope that this has all been of some help to you. Did I miss anything? Please comment below.

	PROJ=gae_project
	mkvirtualenv ${PROJ}
	cdvirtualenv
	GAE=1.7.4
	wget -O /tmp/gae.zip http://googleappengine.googlecode.com/files/google_appengine_${GAE}.zip
	unzip /tmp/gae.zip
	GAE_APP_NAME=dummy
	mkdir -p gae_app/static
	echo """application: ${GAE_APP_NAME}
	version: development
	runtime: python27
	api_version: 1
	threadsafe: true
	default_expiration: 7d
	handlers:
	- url: /static
	static_dir: static
	- url: .*
	script: wsgi_app.app
	""" > gae_app/app.yaml
	echo """import webapp2
	class MainPage(webapp2.RequestHandler):
	def get(self):
	self.response.headers['Content-Type'] = 'text/plain'
	self.response.out.write('Please replace me with a decent WSGI App Framework such as Flask')
	app = webapp2.WSGIApplication([('/', MainPage)],
	debug=True)
	""" > gae_app/wsgi_app.py
	python ./google_appengine/dev_appserver.py gae_app/

view raw install.sh hosted with ❤ by GitHub

A Review of the Nagare Microframework

The analytics tell me that the keywords for this blog are "Best Python Framework" which is odd because I don't think I can pretend to have that answer. What I can do is point out what the differences are and let you find the one that suits you best. Nagare for example offers some concepts that you won't be used to if you come from using Django or Flask and the like.

If you have read Install Stackless Python on Ubuntu and could get stackless running the examples I showed you, then just follow these commands to get a stackless virtual environment running a nagare project called 'nagaredemo':

	stackless /usr/lib/python2.7/dist-packages/virtualenv.py ~/Projects/nagaredemo
	workon nagaredemo
	cdvirtualenv
	pip install nagare
	nagare-admin create-app demos
	cd demos/
	stackless setup.py develop
	nagare-admin serve demos

view raw nagare.sh hosted with ❤ by GitHub

In the examples below, paste the code directly into the file

//negaredemo/demos/demos/app.py

and be aware that when you change the code you need to  restart the development server (Ctrl-C stops it.)

Hello World!

	from nagare import presentation
	class Welcome(object):
	message = "Hello World!"
	@presentation.render_for(Welcome)
	def render(welcome, h, *args):
	"""
	HTML fragment is associated with `Welcome` class as its default view.
	- `welcome`: an object of type Welcome
	- `h`: HTML renderer, it exposes HTML namespace
	"""
	with h.div:
	h << h.span(welcome.message, class_='welcome-class')
	return h.root
	app = Welcome

view raw example1.py hosted with ❤ by GitHub

Okay, so example1 shows the basics of rendering a Welcome object.

http://localhost:8080/demos/

You'll notice immediately that we're not using a templating language here, but rather the built in Domain Specific Language (DSL). In reality you would use the DSL to build xhtml snippets and then use meld to insert them into templates. I've not used meld before, but it looks very straight forward - neat in fact. Now onto example2.

Callbacks

	from nagare import presentation
	class Counter(object):
	def __init__(self):
	self.val = 0
	def increase(self):
	self.val += 1
	def decrease(self):
	self.val -= 1
	@presentation.render_for(Counter)
	def render(counter, h, *args):
	h << h.div('Value: ', counter.val)
	# action method allows to register callbacks on HTML elements
	# that can have actions (links, inputs, etc.)
	h << h.a('++').action(counter.increase)
	h << '|'
	h << h.a('--').action(counter.decrease)
	return h.root
	app = Counter

view raw example2.py hosted with ❤ by GitHub

Here we've registered two callbacks on the DOM without having to explicitly declare the URL mappings. have a look at the rendered HTML to see how this has been implemented. Now, that's pretty handy.

Tiresome RESTful URLs

	from nagare import presentation
	class Counter(object):
	def __init__(self):
	self.val = 0
	def increase(self):
	self.val += 1
	def decrease(self):
	self.val -= 1
	@presentation.render_for(Counter)
	def render(counter, h, *args):
	h << h.div('Value: ', counter.val)
	h << h.a('++').action(counter.increase)
	h << '|'
	h << h.a('--').action(counter.decrease)
	return h.root
	@presentation.init_for(Counter, 'len(url) == 1 and url[0] == "Counter"')
	def init(self, url, comp, http_method, request):
	"""
	Meaningful URLs are uses to initialize a component with the received URL.
	Other than that application works the same as it used to do.
	"""
	value = request.params.get('value')
	if value and value.isdigit():
	self.val = int(value)
	app = Counter

view raw example3.py hosted with ❤ by GitHub

Example 3 takes that same code and augments it to allow us to have more ReSTful URLs. I'm not sure this is as obvious as I'd like, but it works I suppose.

http://localhost:8080/demos/Counter?value=7

Saving State Automatically

	from nagare import component, util
	import random
	class GuessNumber(component.Task):
	"""
	`component.Task` are used to provide a sensible way to program web applications in
	direct style, i.e. as a sequence of actions.
	"""
	def go(self, comp):
	number = random.randint(1, 10)
	while True:
	x = comp.call(util.Ask('Try: '))
	if not x.isdigit() or int(x) != number:
	comp.call(util.Confirm('Choose another number'))
	if int(x) == number:
	comp.call(util.Confirm('Congratulations !'))
	break
	app = GuessNumber

view raw example4.py hosted with ❤ by GitHub

Nagare can hide the request/response object and global session from the developer. Object states are automatically saved in a server side session during the request/response round-trip. This allows for a more direct style of programming like in a desktop application.

And there's more

There's more documentation on the Nagere website itself, so scoot along there and see what else there is to offer. Nagare is built upon ConfigObj, WebOb, Paste, PEAK-Rules and lxml, so it's on good foundations. I'm told by Hervé Coatanhay that the next release will introduce non stackless support using pypy or CPython.

Conclusion

Nagare is able to implement Continuations and that's a great thing. However I'm not sure I like the way it obfuscates the implementation. I'd much rather be able to see what was going on.

For example, saving state between requests is nice, but is it secure? Do I need to plough through the code to find out? In my opinion it would have been better to have offered a structure that allows for others to plug in solutions/implementations.

The documentation is somewhat limited, which is probably because the community is still fairly small. However sinceteh project has been running since 2008, perhaps that's something to be concerned about.

Thanks to Hervé Coatanhay for suggestion the illustrative code snippets.

An Alternate Explanation of Continuations

What's a continuation?

If you read the Wikipedia item on continuations, make sure you understand the "continuation sandwich" paragraph which is supposed to be the definitive example of what a continuation is. I like to think of it the other way around - with my patent "what a continuation is not" example.

What a continuation is not?

Imagine you have a bookies slip with a four horse accumulator bet (parlay) on it. Your first three horses win and you are in a situation (the state of the bet) where you have a small fortune going onto the last horse. The horse falls.

Now, if the bookies slip had been a continuation it would have maintained the state after the third race and you could have retrieved that state and tried another horse on the last race. But it isn't and you can't, so tough.

Continuations are an important concept in the Nagare Microframework which I'll be looking at next. Continuations allow Nagare to maintain state in an exciting way which may seem unusual to you if you have used some of the other python frameworks like Flask or Bottle or Django.

Good luck fellow travellers.

Stackless Code - The Short Version

So my last couple of posts have been on installing Stackless Python and what exactly the Stackless bit of Stackless Python means. Now it's time for the short version of the example code and an alternate explanation of continuations. Prepare to be underawed.

With Stackless ou get tasklets and channels. Channels let you communicate between tasklets.

	#!/usr/bin/env stackless
	import stackless
	def sendKidsToShops():
	kids = 3
	ch.send(kids)
	print kids, "kids sent to the store"
	def buySweeties():
	n = ch.receive()
	while n:
	print "kid #", n, "is spending your money on candy."
	n -= 1
	#stackless.schedule()
	ch = stackless.channel()
	stackless.tasklet(sendKidsToShops)()
	stackless.tasklet(buySweeties)()
	stackless.run()

view raw stackles.py hosted with ❤ by GitHub

Running this you get:

	kid # 3 is spending your money on candy.
	kid # 2 is spending your money on candy.
	kid # 1 is spending your money on candy.
	3 kids sent to the store

view raw gistfile1.txt hosted with ❤ by GitHub

Now the third and final thing that Stackless offers is cooperative multitasking. To see how this works, just uncomment the line with stackless.schedule() in it and rune it again.

	kid # 3 is spending your money on candy.
	3 kids sent to the store
	kid # 2 is spending your money on candy.
	kid # 1 is spending your money on candy.

view raw gistfile1.txt hosted with ❤ by GitHub

That's about it really, but as you know it's not what you got it's how you use it. There's a much more detailed explanation of what we just did here and the point is that this allows you to use continuations.

See my next post An Alternate Explanation of Continuations

Good luck fellow travellers.

Python Rocks - So what is Stackless Python?

Python may be one of the most widely learned and used languages today, but it was conceived in the late 1980's when if you hadn't got a mainframe, you almost certainly were running your code on a single CPU computer of some sort.

For this reason the original implementation of Python was written with the understanding that it was perfectly sensible to use the same single execution stack that C used - after all Python was written in C. Despite being on version 2.7/3.3 nowadays, the standard Python is still written in C, still uses a single execution stack design and is often known as CPython.

English: CPU Zilog Z8 (Photo credit: Wikipedia)

The execution stack - or call stack - or just stack is like a big spike that you stick messages on in a last on, first off way and it's where the low level machine code subroutines used to stick the current code  address before going away to do some jiggery pokery. When the subroutine was finished it returned by pulling the last address off the stack and then execution continued from there. It was all fairly simple in the days of the Z80 and as I understand it, that's still essentially how a single CPU - or Core - works.

AMD Athlon™ X2 Dual-Core Processor 6400+ in AM2 package (Photo credit: Wikipedia)

The problem is that sometime in the early 2000s, dual-core, and then multi-core chips started to be become increasingly affordable and therefore available. Most of you will be using a multi-core system to read this post. This means that your systems are capable of running more than one process a time - what is called concurrency.

This is a bit of a pain for CPython because it only knows how to use a single stack, i.e. a single core, and that is just a bit of a waste of those other cores which are just itching to make it all run super fast.

So Stackless Python is essentially a redesign of CPython which avoids using the call stack and instead uses something called microthreads to get around the problem. This means four things to you:

• Concurrent programming is possible.
• Concurrency can improve on execution time if done properly.
• You need to learn some new concepts: tasklets and channels.
• You get to use some new stuff: tasklets and channels.

I'll introduce those next time.

There's a very informative interview with the creator of Stackless Python here.

You may like to be ready by reading my post about installing Stackless Python.

Good luck fellow travellers.

Install Stackless Python on Ubuntu

I'm just about to write a couple of posts on Stackless Python and the Nagare Micro Framework which runs on it. So I've been installing Stackless on my Ubuntu 12.04. Here are some nice copy and paste instructions if you want to play along.

First install the required libraries and get stackless itself:

	sudo run apt-get update
	sudo apt-get install libreadline-dev
	sudo apt-get build-dep python2.7
	cd /tmp
	wget http://www.stackless.com/binaries/stackless-272-export.tar.bz2
	bunzip2 stackless-272-export.tar.bz2
	tar xf stackless-272-export.tar

view raw install.sh hosted with ❤ by GitHub

Now install stackless :

	cd /tmp/stackless-272-export/
	./configure --prefix=/opt/stackless --enable-unicode=ucs4
	make
	sudo make install

view raw bash.sh hosted with ❤ by GitHub

After the "make" you'll see some failures as below. Just ignore them.

	Python build finished, but the necessary bits to build these modules were not found:
	_bsddb bsddb185 dl
	imageop linuxaudiodev ossaudiodev
	sunaudiodev
	To find the necessary bits, look in setup.py in detect_modules() for the module's name.
	Failed to build these modules:
	_curses _curses_panel

view raw gistfile1.txt hosted with ❤ by GitHub

Now it's time to link your standard (CPython) packages so that they can be used with stackless:

	sudo ln -s /usr/lib/python2.7/dist-packages/ /opt/stackless/lib/python2.7/site-packages
	sudo ln -s /usr/local/lib/python2.7/dist-packages/ /opt/stackless/lib/python2.7/dist-packages
	sudo ln -s /opt/stackless/bin/python /usr/bin/stackless
	sudo vi /opt/stackless/lib/python2.7/site.py

view raw bash.sh hosted with ❤ by GitHub

...and edit the paths in the site.py file. At about line 300, edit the file to look like this. It's the second sitepackages.append bit we're adding here:

	elif os.sep == '/':
	sitepackages.append(os.path.join(prefix, "lib",
	"python" + sys.version[:3],
	"site-packages"))
	sitepackages.append(os.path.join(prefix, "lib",
	"python" + sys.version[:3],
	"dist-packages"))
	sitepackages.append(os.path.join(prefix, "lib", "site-python"))

view raw site.py hosted with ❤ by GitHub

That should be it! Let's test it:

	$stackless
	Python 2.7.2 Stackless 3.1b3 060516 (default, Jan 2 2013, 22:39:14)
	[GCC 4.6.3] on linux3
	Type "help", "copyright", "credits" or "license" for more information.
	>>> exit()

view raw gistfile1.txt hosted with ❤ by GitHub