tag:blogger.com,1999:blog-23680864901796321652024-03-05T21:32:45.566-08:00Terse WordsGetting stuff done using open source tools.tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.comBlogger48125tag:blogger.com,1999:blog-2368086490179632165.post-35428804288133963392018-09-01T02:35:00.000-07:002018-09-01T02:35:34.485-07:00VuePress ExamplesI Had great fun writing playing with vuepress and netlify putting together <a href="https://vuepress-examples.netlify.com/">https://vuepress-examples.netlify.com/</a> however as usual I have to move onto other things and stop having fun.<br />
<br />
Still a really useful way to put quick sites together especially for documentation and so on.tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com1tag:blogger.com,1999:blog-2368086490179632165.post-20867450343973359212013-09-03T05:01:00.003-07:002013-09-03T06:19:26.672-07:00Who is Installing gbrainy? While I was flicking through the metrics from <a href="http://www.installion.co.uk/" target="_blank">installion.co.uk</a> last night I was shocked by just how many hits the <a href="http://www.installion.co.uk/debian/squeeze/main/g/gbrainy/index.html" target="_blank">Debian </a>and <a href="http://www.installion.co.uk/ubuntu/raring/universe/g/gbrainy/index.html" target="_blank">Ubuntu installation</a> pages for <a href="https://wiki.gnome.org/gbrainy" target="_blank">gbrainy</a> had had. It seems that from virtually out of nowhere a while back, an increasing number of people are trying to install it on a linux box and it made me wonder why.<br />
<br />
The way I look at it there can only be two reasons:<br />
<br />
<ol>
<li>It's just really good and so is spreading virally.</li>
<li>Everyone is being asked to install it by some educational or training body - maybe it's on some curriculum somewhere.</li>
</ol>
<div>
<br /></div>
<div>
Having Goggled around and wasted more time than I really should have I came up with nothing except that <a href="http://www.edubuntu.org/" target="_blank">Edubuntu </a>comes with it installed (and surely schools are using that?)</div>
<div>
<br /></div>
<div>
Anyway, if you have any ideas then let me know. Meanwhile I'm just sitting here playingr <a href="https://wiki.gnome.org/gbrainy" target="_blank">gbrainy</a> like the rest of them.</div>
<div>
<br /></div>
<div>
There's a neat article <a href="http://www.makeuseof.com/tag/train-brain-stay-sharp-gbrainy-linux/" target="_blank">here</a>. </div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-9253997997339850892013-05-16T13:15:00.002-07:002013-05-16T13:15:20.760-07:00Python _imaging cannot open shared object fileOkay so I'm using a 64 bit Linux and I was having a lot of trouble getting Calibre ebook tool to convert books into Kindle loving mobi format. I was getting a python error:<br />
<br />
<pre>ImportError: libjpeg.so.62: cannot open shared object file: No such file or directory</pre>
<br />
Turns out all you need to do is<br />
<br />
<pre>sudo apt-get install libjpeg62</pre>
<br />
You can test this by starting python and doing:<br />
<br />
<pre>import _imaging</pre>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-75020998938543160802013-04-20T01:12:00.000-07:002013-04-20T01:12:58.290-07:00Fastest Way to Burn an ISO to a USB Flash DriveIf your using Linux:<br />
<br />
<br />
<script src="https://gist.github.com/colwilson/5425208.js"></script>tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-33286756039712883322013-03-01T05:31:00.000-08:002013-03-01T05:31:49.687-08:00Installing Git and Virtualenv on Linux or Mac<h3>
For Debian/Ubuntu Linux:</h3>
<pre class="prettyprint sh">sudo apt-get install git python-setuptools</pre>
<h3>
For Redhat/Fedora Linux:</h3>
<pre class="prettyprint sh">sudo yum install git python-setuptools</pre>
<h3>
For Mac:</h3>
Install <a href="http://www.macports.org/">MacPorts</a> and then:<br />
<pre class="prettyprint sh">sudo port install git python-setuptools</pre>
<h3>
Continuing on for all of the above</h3>
<pre class="prettyprint sh">sudo easy_install pip
sudo pip install virtualenv virtualenvwrapper</pre>
<br />
Now, you may want to set some defaults in your ~/.bashrc. My relevant entries look like this:<br />
<br />
<pre class="prettyprint sh"></pre>
<pre class="prettyprint sh"></pre>
<pre class="prettyprint sh">export WORKON_HOME=$HOME/Projects
export VIRTUALENVWRAPPER_HOOK_DIR=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_LOG_DIR=$HOME/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh
git config --global user.name "Terse Col"
git config --global user.email myemail@gmail.com
git config --global http.sslVerify false</pre>
<br />
Now away you go:<br />
<br />
<pre class="prettyprint sh">mkdir $HOME/.virtualenvs
source $HOME/.bashrc
mkvirtualenv mynewproject
cdvirtualenv
</pre>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com1Exeter, Devon, UK50.727273946806477 -3.47275257110595750.72601744680648 -3.4752740711059569 50.728530446806474 -3.4702310711059572tag:blogger.com,1999:blog-2368086490179632165.post-80999171299887462142013-01-29T06:04:00.000-08:002013-01-29T06:04:03.216-08:00Install Google App Engine on Ubuntu 12.10<b><a href="https://developers.google.com/appengine/" target="_blank">Google App Engine</a> is currently on version 1.7.4 and Ubuntu has recently released <a href="http://www.ubuntu.com/download/desktop" target="_blank">Ubuntu 12.10 (Quantal Quetzal)</a>. Quantal Quetzal comes with Python 2.7 installed, and App Engine has been providing that version of Python as an option since <a href="http://googleappengine.blogspot.co.uk/2012/02/announcing-general-availability-of.html" target="_blank">February 2012</a>. So if you are starting a new App Engine project, it's probably a good time to <a href="https://developers.google.com/appengine/docs/python/python27/newin27" target="_blank">move to Python 2.7</a>.</b><br />
<br />
I'll explain briefly how you start a new project and there's a nice clean copy of the code at the bottom that you can cut and paste.<br />
<br />
Let's get the show on the road. Choose a name for the project and create and switch to a virtual environment:<br />
<br />
<pre>PROJ=gae_project
mkvirtualenv ${PROJ}
cdvirtualenv
</pre>
Note that note that "--no-site-packages" and "--distribute" are now the defaults for mkvirtualenv. You don't even need to use "--python=python2.7" on Ubuntu 12.10.<br />
<br />
Now we need to know what <a href="https://developers.google.com/appengine/downloads" target="_blank">the latest version of App Engine</a> is, but as of writing it's 1.7.4:<br />
<br />
<pre>GAE=1.7.4
wget -O /tmp/gae.zip http://googleappengine.googlecode.com/files/google_appengine_${GAE}.zip
unzip /tmp/gae.zip</pre>
<br />
Now let's create an App Engine app. The app will need a name that has been created in the <a href="https://appengine.google.com/" target="_blank">App Engine Console</a>:<br />
<br />
<pre>GAE_APP_NAME=dummy
mkdir -p gae_app/static
</pre>
Now create the app.yaml file:<br />
<br />
<pre>echo """application: ${GAE_APP_NAME}
version: development
runtime: python27
api_version: 1
threadsafe: true
default_expiration: 7d
handlers:
- url: /static
static_dir: static
- url: .*
script: wsgi_app.app
""" > gae_app/app.yaml
</pre>
<br />
<div>
And finally the app itself:</div>
<div>
<br /></div>
<pre>echo """import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('Please replace me with a decent WSGI App Framework such as Flask')
app = webapp2.WSGIApplication([('/', MainPage)],
debug=True)
""" > gae_app/wsgi_app.py</pre>
<pre></pre>
<br />
And finally to run the development server:<br />
<br />
<pre>python ./google_appengine/dev_appserver.py gae_app/
</pre>
<br />
I hope that this has all been of some help to you. Did I miss anything? Please comment below.<br />
<br class="Apple-interchange-newline" />
<script src="https://gist.github.com/4664418.js"></script>tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com1tag:blogger.com,1999:blog-2368086490179632165.post-21252148861052362822013-01-13T02:05:00.000-08:002013-01-13T02:05:08.098-08:00A Review of the Nagare Microframework<b>The analytics tell me that the keywords for this blog are "Best <a class="zem_slink" href="http://en.wikipedia.org/wiki/Python_%28programming_language%29" rel="wikipedia" target="_blank" title="Python (programming language)">Python</a> Framework" which is odd because I don't think I can pretend to have that answer. What I can do is point out what the differences are and let you find the one that suits you best. <a href="http://www.nagare.org/" target="_blank">Nagare</a> for example offers some concepts that you won't be used to if you come from using <a class="zem_slink" href="http://www.djangoproject.com/" rel="homepage" target="_blank" title="Django (Web framework)">Django</a> or <a class="zem_slink" href="http://flask.pocoo.org/" rel="homepage" target="_blank" title="Flask (programming)">Flask</a> and the like.</b><br />
<br />
If you have read <a href="http://terse-words.blogspot.co.uk/2013/01/install-stackless-python-on-ubuntu.html">Install Stackless Python on Ubuntu</a> and could get stackless running the <a href="http://terse-words.blogspot.co.uk/2013/01/so-my-last-couple-of-posts-have-been-on.html">examples</a> I showed you, then just follow these commands to get a stackless <a href="http://pypi.python.org/pypi/virtualenv" target="_blank">virtual environment </a>running a nagare project called 'nagaredemo':<br />
<br />
<script src="https://gist.github.com/4517252.js"></script>
In the examples below, paste the code directly into the file<br />
<pre>//negaredemo/demos/demos/app.py</pre>
and <i>be aware that when you change the code you need to restart the development server</i> (<a class="zem_slink" href="http://en.wikipedia.org/wiki/Control-C" rel="wikipedia" target="_blank" title="Control-C">Ctrl-C</a> stops it.)<br />
<br />
<h4>
Hello World!</h4>
<div>
<br /></div>
<script src="https://gist.github.com/4517903.js"></script><br />
Okay, so example1 shows the basics of rendering a Welcome object.<br />
<pre>http://localhost:8080/demos/</pre>
You'll notice immediately that we're not using a templating language here, but rather the built in <a class="zem_slink" href="http://en.wikipedia.org/wiki/Domain-specific_language" rel="wikipedia" target="_blank" title="Domain-specific language">Domain Specific Language</a> (DSL). In reality you would use the DSL to build xhtml snippets and then use <a href="http://plope.com/software/meld3/" target="_blank">meld</a> to insert them into templates. I've not used meld before, but <a href="http://www.nagare.org/trac/wiki/PresentationTier" target="_blank">it looks very straight forward</a> - neat in fact. Now onto example2.<br />
<br />
<h4>
Callbacks</h4>
<br />
<script src="https://gist.github.com/4517977.js"></script><br />
Here we've registered two callbacks on the DOM without having to explicitly declare the URL mappings. have a look at the rendered HTML to see how this has been implemented. Now, that's pretty handy.<br />
<br />
<h4>
Tiresome RESTful URLs</h4>
<br />
<script src="https://gist.github.com/4523031.js"></script><br />
Example 3 takes that same code and augments it to allow us to have more ReSTful URLs. I'm not sure this is as obvious as I'd like, but it works I suppose.<br />
<pre>http://localhost:8080/demos/Counter?value=7</pre>
<h4>
Saving State Automatically</h4>
<div>
<br /></div>
<div>
<div>
<script src="https://gist.github.com/4523186.js"></script></div>
<div>
Nagare can hide the request/response object and global session from the developer. Object states are automatically saved in a server side session during the request/response round-trip. This allows for a more direct style of programming like in a desktop application.</div>
</div>
<br />
<h4>
And there's more</h4>
<div>
<br /></div>
There's <a href="http://www.nagare.org/trac/wiki/NagareTutorial" target="_blank">more documentation</a> on the Nagere website itself, so scoot along there and see what else there is to offer. Nagare is built upon ConfigObj, WebOb, Paste, PEAK-Rules and lxml, so it's on good foundations. I'm told by Hervé Coatanhay that the next release will introduce non stackless support using pypy or CPython.<br />
<br />
<h4>
Conclusion</h4>
<div>
<br /></div>
Nagare is able to implement <a href="http://terse-words.blogspot.co.uk/2013/01/an-alternate-explanation-of.html" target="_blank">Continuations</a> and that's a great thing. However I'm not sure I like the way it obfuscates the implementation. I'd much rather be able to see what was going on.<br />
<br />
For example, saving state between requests is nice, but is it secure? Do I need to plough through the code to find out? In my opinion it would have been better to have offered a structure that allows for others to plug in solutions/implementations.<br />
<br />
The documentation is somewhat limited, which is probably because the community is still fairly small. However sinceteh project has been running since 2008, perhaps that's something to be concerned about.<br />
<br />
Thanks to <a href="https://twitter.com/alzakath" target="_blank">Hervé Coatanhay</a> for suggestion the illustrative code snippets.<br />
<br />
<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=ccc571a0-4d3f-4a40-808f-5d40a0126e26" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com10tag:blogger.com,1999:blog-2368086490179632165.post-50267907664838355082013-01-07T15:36:00.001-08:002013-01-07T15:39:16.724-08:00An Alternate Explanation of Continuations<br />
<h4>
<b>What's a continuation?</b></h4>
<div>
If you read the <a href="http://en.wikipedia.org/wiki/Continuation" target="_blank">Wikipedia item on continuations</a>, make sure you understand the "continuation sandwich" paragraph which is supposed to be the definitive example of what a continuation is. I like to think of it the other way around - with my patent "what a continuation is not" example.</div>
<br />
<h4>
What a continuation is not?</h4>
Imagine you have a bookies slip with a four horse <a class="zem_slink" href="http://en.wikipedia.org/wiki/Parlay_%28gambling%29" rel="wikipedia" target="_blank" title="Parlay (gambling)">accumulator bet</a> (parlay) on it. Your first three horses win and you are in a situation (the state of the bet) where you have a small fortune going onto the last horse. The horse falls.<br />
<br />
Now, if the bookies slip had been a continuation it would have maintained the state after the third race and you could have retrieved that state and tried another horse on the last race. But it isn't and you can't, so tough.<br />
<br />
Continuations are an important concept in the <a href="http://www.nagare.org/" target="_blank">Nagare Microframework</a> which I'll be looking at next. Continuations allow Nagare to maintain state in an exciting way which may seem unusual to you if you have used some of the other python frameworks like <a href="http://flask.pocoo.org/" target="_blank">Flask</a> or <a href="http://bottlepy.org/docs/dev/" target="_blank">Bottle</a> or <a href="https://www.djangoproject.com/" target="_blank">Django</a>.<br />
<br />
Good luck fellow travellers.<br />
<div>
<br /></div>
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=b38e7dd7-d1ea-4d3e-b2c9-68382bbb8c23" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-4678091902946549372013-01-07T15:28:00.000-08:002013-01-07T15:37:56.977-08:00Stackless Code - The Short Version<b>So my last <a href="http://terse-words.blogspot.co.uk/2013/01/install-stackless-python-on-ubuntu.html">couple</a> of <a href="http://terse-words.blogspot.co.uk/2013/01/python-rocks-so-what-is-stackless-python.html">posts</a> have been on installing <a class="zem_slink" href="http://en.wikipedia.org/wiki/Stackless_Python" rel="wikipedia" target="_blank" title="Stackless Python">Stackless Python</a> and what exactly the Stackless bit of Stackless Python means. Now it's time for the short version of the example code and an alternate explanation of <a href="http://en.wikipedia.org/wiki/Continuation" target="_blank">continuations</a>. Prepare to be underawed.</b><br />
<br />
With Stackless ou get <i>tasklets</i> and <i>channels</i>. Channels let you communicate between tasklets.<br />
<br />
<script src="https://gist.github.com/4479022.js"></script>
Running this you get:<br />
<br />
<script src="https://gist.github.com/4479084.js"></script>
Now the third and final thing that Stackless offers is <a class="zem_slink" href="http://en.wikipedia.org/wiki/Computer_multitasking" rel="wikipedia" target="_blank" title="Computer multitasking"><i>cooperative multitasking</i></a>. To see how this works, just uncomment the line with <i>stackless.schedule()</i> in it and rune it again.<br />
<br />
<script src="https://gist.github.com/4479109.js"></script>
That's about it really, but as you know it's not what you got it's how you use it. There's a much more detailed explanation of what we just did <a href="http://dalkescientific.com/StacklessPyCon2007-Dalke.pdf" target="_blank">here</a> and the point is that this allows you to use <i>continuations</i>.<br />
<br />
See my next post <a href="http://terse-words.blogspot.co.uk/2013/01/an-alternate-explanation-of.html">An Alternate Explanation of Continuations</a><br />
<br />
Good luck fellow travellers.<br />
<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=b38e7dd7-d1ea-4d3e-b2c9-68382bbb8c23" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-5858527283622703872013-01-05T14:26:00.000-08:002013-01-06T01:10:00.165-08:00Python Rocks - So what is Stackless Python?<b><a class="zem_slink" href="http://www.python.org/" rel="homepage" target="_blank" title="Python (programming language)">Python</a> may be one of the most widely learned and used languages today, but it was conceived in the late 1980's when if you hadn't got a mainframe, you almost certainly were running your code on a single <a class="zem_slink" href="http://en.wikipedia.org/wiki/Central_processing_unit" rel="wikipedia" target="_blank" title="Central processing unit">CPU</a> computer of some sort.</b><br />
<br />
For this reason the original implementation of Python was written with the understanding that it was perfectly sensible to use the same single execution stack that C used - after all Python was written in C. Despite being on version 2.7/3.3 nowadays, the standard Python is still written in C, still uses a single execution stack design and is often known as <a class="zem_slink" href="http://en.wikipedia.org/wiki/CPython" rel="wikipedia" target="_blank" title="CPython">CPython</a>.<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container zemanta-img" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><div class="zemanta-img">
<div class="zemanta-img">
<div class="zemanta-img">
<div class="zemanta-img">
<a href="http://commons.wikipedia.org/wiki/File:KL_Zilog_Z8.jpg" imageanchor="1" style="margin-bottom: 1em; margin-left: auto; margin-right: auto; text-align: clear:right;"><img alt="English: CPU Zilog Z8" border="0" class="zemanta-img-inserted" height="95" src="http://upload.wikimedia.org/wikipedia/commons/thumb/e/ed/KL_Zilog_Z8.jpg/300px-KL_Zilog_Z8.jpg" style="border: none; font-size: 0.8em;" width="200" /></a></div>
</div>
</div>
</div>
</td></tr>
<tr><td class="tr-caption zemanta-img-attribution" style="text-align: center; width: 300px;">English: CPU Zilog Z8 (Photo credit: <a href="http://commons.wikipedia.org/wiki/File:KL_Zilog_Z8.jpg" target="_blank">Wikipedia</a>)</td></tr>
</tbody></table>
The execution stack - or call stack - or just stack is like a big spike that you stick messages on in a last on, first off way and it's where the low level <a class="zem_slink" href="http://en.wikipedia.org/wiki/Machine_code" rel="wikipedia" target="_blank" title="Machine code">machine code</a> subroutines used to stick the current code address before going away to do some jiggery pokery. When the subroutine was finished it <i>returned</i> by pulling the last address off the stack and then execution continued from there. It was all fairly simple in the days of the <a href="http://en.wikipedia.org/wiki/Z80" target="_blank">Z80</a> and as I understand it, that's still essentially how a single CPU - or <i>Core</i> - works.<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container zemanta-img" style="float: right; text-align: right;"><tbody>
<tr><td style="text-align: center;"><div class="zemanta-img">
<div class="zemanta-img">
<div class="zemanta-img">
<div class="zemanta-img">
<a href="http://commons.wikipedia.org/wiki/File:Athlon64x2-6400plus.jpg" imageanchor="1" style="margin-bottom: 1em; margin-left: auto; margin-right: auto; text-align: clear:right;"><img alt="AMD Athlon™ X2 Dual-Core Processor 6400+ in AM..." border="0" class="zemanta-img-inserted" height="134" src="http://upload.wikimedia.org/wikipedia/commons/thumb/f/fb/Athlon64x2-6400plus.jpg/300px-Athlon64x2-6400plus.jpg" style="border: none; font-size: 0.8em;" width="200" /></a></div>
</div>
</div>
</div>
</td></tr>
<tr><td class="tr-caption zemanta-img-attribution" style="text-align: center; width: 300px;">AMD Athlon™ X2 Dual-Core Processor 6400+ in AM2 package (Photo credit: <a href="http://commons.wikipedia.org/wiki/File:Athlon64x2-6400plus.jpg" target="_blank">Wikipedia</a>)</td></tr>
</tbody></table>
The problem is that sometime in the early 2000s, dual-core, and then multi-core chips started to be become increasingly affordable and therefore available. Most of you will be using a multi-core system to read this post. This means that your systems are capable of running more than one process a time - what is called concurrency.<br />
<br />
This is a bit of a pain for CPython because it only knows how to use a single stack, i.e. a single core, and that is just a bit of a waste of those other cores which are just itching to make it all run super fast.<br />
<br />
So <i>Stackless</i> Python is essentially a redesign of CPython which avoids using the call stack and instead uses something called <a href="http://en.wikipedia.org/wiki/Microthread" target="_blank">microthreads</a> to get around the problem. This means four things to you:<br />
<br />
<ul>
<li>Concurrent programming is possible.</li>
<li>Concurrency can improve on execution time if done properly.</li>
<li>You need to learn some new concepts: tasklets and channels.</li>
<li>You get to use some new stuff: tasklets and channels.</li>
</ul>
I'll introduce those next time.<br />
<br />
There's a very informative interview with the creator of Stackless Python <a href="http://gnosis.cx/download/charming_python_10_outtakes.html" target="_blank">here</a>.<br />
<br />
You may like to be ready by reading my post about installing <a href="http://terse-words.blogspot.co.uk/2013/01/install-stackless-python-on-ubuntu.html" target="_blank">Stackless Python</a>.<br />
<br />
Good luck fellow travellers.<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=cdbc4c00-6251-404c-928c-c460d0249fc3" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0Dunkeswell Airfield, Honiton, Devon EX14, UK50.860554 -3.232221999999978850.8405095 -3.2725624999999789 50.8805985 -3.1918814999999787tag:blogger.com,1999:blog-2368086490179632165.post-17955360756718361362013-01-02T15:36:00.000-08:002013-01-07T15:06:51.251-08:00Install Stackless Python on Ubuntu<b>I'm just about to write a couple of posts on <a class="zem_slink" href="http://www.stackless.com/" rel="homepage" target="_blank" title="Stackless Python">Stackless Python</a> and the <a href="http://www.nagare.org/" target="_blank">Nagare Micro Framework </a>which runs on it. So I've been installing Stackless on my <a class="zem_slink" href="http://www.ubuntu.com/" rel="homepage" target="_blank" title="List of Ubuntu releases">Ubuntu 12.04</a>. Here are some nice copy and paste instructions if you want to play along.</b><br />
<br />
First install the required libraries and get stackless itself:<br />
<br />
<script src="https://gist.github.com/4439310.js"></script>
<br />
<div>
Now install stackless :</div>
<div>
<br /></div>
<script src="https://gist.github.com/4439332.js"></script>
<br />
<div>
After the "make" you'll see some failures as below. Just ignore them.</div>
<div>
<br /></div>
<script src="https://gist.github.com/4439348.js"></script>
<br />
<div>
Now it's time to link your standard (<a class="zem_slink" href="http://en.wikipedia.org/wiki/CPython" rel="wikipedia" target="_blank" title="CPython">CPython</a>) packages so that they can be used with stackless:</div>
<div>
<br /></div>
<div>
<script src="https://gist.github.com/4439359.js"></script>
</div>
<div>
...and edit the paths in the site.py file. At about line 300, edit the file to look like this. It's the second sitepackages.append bit we're adding here:<br />
<br /></div>
<div>
<script src="https://gist.github.com/4439372.js"></script></div>
<div>
That should be it! Let's test it:</div>
<div>
<br /></div>
<script src="https://gist.github.com/4439385.js"></script>
<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=b38e7dd7-d1ea-4d3e-b2c9-68382bbb8c23" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com1tag:blogger.com,1999:blog-2368086490179632165.post-6567457394197448712012-12-05T05:58:00.001-08:002012-12-05T05:58:35.993-08:00Aggregating Everything - Map/Reduce and Camel?If you are used to Map/Reduce you will be used to the idea of breaking tasks down into little chunks and then collecting the partial results together into some final format.<br />
<br />
So, recently when I was parsing zillions of rows of data and aggregating related data into partial CSV files and then aggregating the bits of partial of data to reports I thought - Aha! MapReduce.<br />
<br />
For a whole bunch of good design decisions I was using <a href="http://camel.apache.org/" target="_blank">Apache Camel</a> - a neat pipelining tool which with a bit of help from <a href="http://activemq.apache.org/" target="_blank">ActiveMQ</a> provides the sort of long running stability that I needed. Camel however does not do Map/Reduce, but it does have the <a href="http://camel.apache.org/aggregator2.html" target="_blank">Aggregator Integration pattern</a>, which you can use to so a similar thing.<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLFPpqfmeacYbzFs5BwVNXTEXhXhQisedp4D1D8ai01vloXQ5W_2Hs2rQk4k8myCRITrtme5KXLij5tEZm8SWIiOAFvaKWGQ7z9ThEAJ_J_SHfCADoAyyk7d06scfcMVF5pSw1KJ_I-Jc/s1600/1403392_moedas.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="182" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLFPpqfmeacYbzFs5BwVNXTEXhXhQisedp4D1D8ai01vloXQ5W_2Hs2rQk4k8myCRITrtme5KXLij5tEZm8SWIiOAFvaKWGQ7z9ThEAJ_J_SHfCADoAyyk7d06scfcMVF5pSw1KJ_I-Jc/s200/1403392_moedas.jpg" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Image courtesy of <a href="http://www.sxc.hu/profile/caboco" target="_blank">Carlos Oliveira</a></td></tr>
</tbody></table>
Imagine you empty your jar of loose change on a table. You stack the nickles in stacks of ten coins, the dimes is stacks of ten coins and the quarters in stacks of ten coins. You add up all the 50 cents, $1s and $2.50s and you know how much you have. That's Map/Reduce.<br />
<br />
Now, imagine you empty your jar of loose change into one of those coin counting machines in the Mall. Internally all the coins are sorted by falling through a hole which is either nickle, dime or quarter shaped and as they emerge from the other side they are counted*. That's aggregation Camel style.<br />
<br />
I did hit a bit of a snag. I couldn't work out how to tell the <a href="http://camel.apache.org/aggregator2.html" target="_blank">Aggregator Integration pattern</a> that there were no more files to come... Stop... Woaa-there... Desist!<br />
<br />
It turns out that hidden away (in the middle of the <a href="http://camel.apache.org/batch-consumer.html" target="_blank">docs</a>) the File endpoint rather usefully sets a flag in the headers called <i>CamelBatchComplete </i>which is just what I was looking for:<br />
<br />
<script src="https://gist.github.com/4205772.js?file=context.xml"></script>
Good luck fellow travelers.<br />
<br />
* I have no idea how a coin counting machine works.<br />
<br />tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-83383731641262109132012-11-23T08:02:00.001-08:002012-11-23T08:03:22.973-08:00Simple Camel Configuration of a Twitter EndpointI was asked just now how my got the Twitter Stream working in my new Camel based project and how I managed the credentials.<br />
<br />
The Twitter endpoint works like a dream and this is essentially what my code looks like. All you need is a <i>secrets.properties</i> file in alongside your java file.<br />
<script src="https://gist.github.com/4136224.js?file=routes.java"></script>
<br />
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-89568067452745593522012-11-17T04:39:00.003-08:002012-11-17T04:41:05.563-08:00Apache Camel - Connection Beans Without SpringWhile writing some tests for an Apache Camel project, I just spent rather longer than I'd have liked trying to work out how to configure a connection bean for Mongo without using Spring.<br />
<br />
Since I hide my embarrassments in public I thought I'd best share with anyone else with brain freeze.<br />
<br />
<script src="https://gist.github.com/4095633.js?file=test.java"></script>
<br />
Don't forget you need the camel-mongodb artifact in your pom.xml file.
Good luck fellow travellers.tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-37713368598180257502012-11-05T14:53:00.001-08:002012-11-05T14:55:58.690-08:00Python Web Microframeworks - Take Your PickYou may read my post "<a href="http://terse-words.blogspot.co.uk/2012/11/python-web-frameworks-revisited.html">Top Python Web Frameworks - Today</a>" in which I took a fresh look at what which Python Web Frameworks were still around and still maintained.<br />
<br />
In this post I give a quick overview of about half of those which I have loosely designated as "Microframeworks" - regardless of what the authors have called them. Wikipedia doesn't have a definition for microframework - I just looked - so what I really mean here is anything which let's you get started without having to learn a whole bunch of syntax and convention. Right on sister!<br />
<br />
Let's get going:<br />
<br />
<script src="https://gist.github.com/4011036.js?file=micros.sh"></script>
Now that's done a little code from each -<br />
<h3>
<a href="http://bottlepy.org/" style="font-size: x-large;" target="_blank">Bottle</a><span style="font-size: large;"> (v0.11.13)</span></h3>
<div>
<br /></div>
<script src="https://gist.github.com/4015996.js?file=main_bottle.py"></script>
<br />
<div>
Bottle doesn't rely on any other packages at all, which means it's a great framework to use if you want to see all the working parts as they're all in the one file. That being said it can offer client-side sessions and compression and even WebSockets straight out of the box so it's not just a toy by any means.<br />
<br />
<h3>
<a href="http://flask.pocoo.org/" style="font-size: x-large;" target="_blank">Flask</a><span style="font-size: large;"> (v0.9)</span></h3>
<script src="https://gist.github.com/4016000.js?file=main_flask.py"></script><br />
Flask is dependent on <i>Werkzeug</i> for all the WSGI stuff and upon <i>Jinja2</i> as a template library. It comes with client-side sessions, a built in debugger and is totally unicode friendly. I love Flask and use it often as my other posts will testify.<br />
<br />
<h3>
<a href="https://github.com/stanislavfeldman/kiss.py" style="font-size: x-large;" target="_blank">kiss.py</a><span style="font-size: large;"> (v0.4.9)</span></h3>
<script src="https://gist.github.com/4016002.js?file=main_kiss.py"></script><br />
The first of the new boys, kiss.py is certainly not package independent! It requires <i>Werkzeug</i> for WSGI, <i>requests</i> for http, <i>Beaker</i> for sessions, <i>Elixir</i> and <i>SQLAlchemy</i> for an ORM (PostgreSQL, MySQL and SQLite), <i>Jinja2</i> for templates, <i>gevent</i>, <i>pev</i> and <i>greenlet</i> for events as well as <i>compressinja</i>, <i>jsmin</i>, <i>jsonpickle</i>, <i>putils</i> and <i>pyScss</i> which add various other niceties. Almost all well known and trusted libraries.<br />
<div>
<br />
<h3>
<span style="font-size: large;"><a href="http://webpy.org/" target="_blank">web.py</a> (v0.37)</span></h3>
</div>
<script src="https://gist.github.com/4016007.js?file=main_web.py"></script><br />
Again, web.py doesn't rely on any other packages at all, but to me it's not as useful as flask or kiss.py and not as simple to study as bottle, so I can't see the point, although according to the site, it's well used by others.<br />
<br />
<div>
<h3>
<span style="font-size: large;"><a href="https://bitbucket.org/akorn/wheezy.web" target="_blank">wheezy.web</a> (v0.1.307)</span></h3>
</div>
<div>
<script src="https://gist.github.com/4016013.js?file=main_wheezy.py"></script><br />
pip installs wheezy.web, wheezy.core, wheezy.caching, wheezy.html, wheezy.http, wheezy.routing, wheezy.security, wheezy.validation which to me looks like the developers have taken a sensible approach to the development cycle by splitting everything up into independent code units.<br />
<br />
According to the site, functionality includes routing, model update/validation, authentication/authorization,content caching with dependency, xsrf/resubmission protection, AJAX+JSON, i18n (gettext), middlewares, and more.<br />
<br />
<h3>
In Summary</h3>
No I haven't tested them to death and no I haven't even tried out kiss.py and wheezy.web in a real world app, although I will do. I certainly have not done either load or concurrency testing on them. You can do that and I'll read your blog.<br />
<br />
Goo luck fellow traveller.</div>
</div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com2tag:blogger.com,1999:blog-2368086490179632165.post-91155263517033030662012-11-02T06:44:00.001-07:002012-11-04T01:33:12.877-07:00CoffeeScript Love: Backbone.js Tutorials in CoffeeScriptAhhhh enjoy the aroma, inhale the caffeine. If you're looking for a nice list of Backbone tutorials that use CoffeeScript then look no more - <a href="http://www.coffeescriptlove.com/2012/06/backbonejs-tutorials-in-coffeescript.html">CoffeeScript Love: Backbone.js Tutorials in CoffeeScript</a>.<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=07678bfd-dd79-43bb-a26a-478d9bd9df9d" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-84453752892940548682012-11-01T13:03:00.001-07:002012-11-04T01:32:45.607-07:00Top Python Web Frameworks - Today<div style="text-align: right;">
</div>
Every once in a while it's nice to rediscover old friends. Over the last few years I've had a play with quite a few <a class="zem_slink" href="http://www.python.org/" rel="homepage" target="_blank" title="Python (programming language)">Python</a> Web Frameworks, but one does tend to get a favourite and stick with it. Same as other areas I suppose - cars, dogs, beers, partners... Still, it's good to have a look at what's new and fresh once in a while (my wife doesn't care, but don't tell my dog.)<br />
<br />
<div style="text-align: right;">
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container zemanta-img" style="float: right; text-align: right;"><tbody>
<tr><td style="text-align: center;"><div class="zemanta-img">
<div class="zemanta-img">
<div class="zemanta-img">
<div class="zemanta-img">
<a href="http://commons.wikipedia.org/wiki/File:Python_add5_syntax.svg" imageanchor="1" style="margin-bottom: 1em; margin-left: auto; margin-right: auto; text-align: clear:right;"><img alt="example of Python language" border="0" class="zemanta-img-inserted" height="149" src="http://upload.wikimedia.org/wikipedia/commons/thumb/e/e1/Python_add5_syntax.svg/300px-Python_add5_syntax.svg.png" style="border-bottom-style: none; border-color: initial; border-image: initial; border-left-style: none; border-right-style: none; border-top-style: none; border-width: initial; font-size: 0.8em;" width="200" /></a></div>
</div>
</div>
</div>
</td></tr>
<tr><td class="tr-caption zemanta-img-attribution" style="text-align: center; width: 300px;">example of Python language (Photo credit: <a href="http://commons.wikipedia.org/wiki/File:Python_add5_syntax.svg" target="_blank">Wikipedia</a>)</td></tr>
</tbody></table>
If you look around the internet, there are probably thirty or so Python Web Frameworks around, but not all of them are actively supported. This means that they are likely to be missing functionality in some of the latest latest areas of interest - OAuth2, HTML5 or mobile support for example. Or maybe they just don't make it easy to stuff that you have become used to over the last few years - AJAX or web services maybe.<br />
<br class="Apple-interchange-newline" />
In my previous post "<a href="http://terse-words.blogspot.co.uk/2011/06/top-ten-python-web-frameworks.html">My Top Ten Python Web Frameworks</a>" from about 18months ago, I gave you my opinion of what was hot, or not, at that time. Some of those seem to have stalled now - Tipfy, GAE framework, Weblayer; while others have appeared or matured - kiss.py, web.py, wheezy.web.<br />
<br />
Below is an alphabetic list of the active (updated this year) which I know about. There may be others out there, so please let me know in the comments at the bottom.<br />
<ul>
<li><a href="http://bottlepy.org/" target="_blank">Bottle</a> now on v0.11</li>
<li><a href="http://www.cubicweb.org/" target="_blank">CubicWeb</a> now on v3.15.5</li>
<li><a href="http://www.djangoproject.com/" target="_blank">Django</a> now on v1.4</li>
<li><a href="http://flask.pocoo.org/" target="_blank">Flask</a> now on v0.9</li>
<li><a href="https://github.com/stanislavfeldman/kiss.py" target="_blank">kiss.py</a> now on v0.4.9</li>
<li><a href="http://www.nagare.org/" target="_blank">Nagare</a> now on v0.4.1</li>
<li><a href="http://pylonsproject.org/" target="_blank">Pyramid</a> now on v1.3</li>
<li><a href="http://www.turbogears.org/" target="_blank">TurboGears</a> now on v2.2.0</li>
<li><a href="http://webpy.org/" target="_blank">web.py</a> now on v0.37</li>
<li><a href="http://www.web2py.com/" target="_blank">web2py</a> now on v2.2.1</li>
<li><a href="https://bitbucket.org/akorn/wheezy.web" target="_blank">wheezy.web</a> now on v0.1</li>
</ul>
<div>
In my next few posts I'll be giving you a run down of what state each of them is in and try to give you some idea of how they fit your requirements.</div>
<div>
<br /></div>
<div>
<div>
Good luck fellow traveller.</div>
</div>
<div>
<br /></div>
<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=07678bfd-dd79-43bb-a26a-478d9bd9df9d" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-52996946747595341112012-07-17T15:42:00.001-07:002012-07-29T02:08:37.802-07:00A First Look at Flask-Evolution<b>While poking around in <a href="http://flask-ahoy.com/" target="_blank">Flask-Ahoy</a> this morning I came <span style="background-color: white;">across </span><span style="background-color: white;"><a href="http://flask-ahoy.com/package/Flask-Evolution/0.6#.UAW21xXezbw" target="_blank">Flask-Evolution</a> which I hadn't previous noticed. It offers the Django idea of migrations to the <a href="http://flask.pocoo.org/" target="_blank">Flask</a> community and so I thought I'd give it a go and see how it shapes up.</span></b><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;">So let's set up a virtual environment and get the basics in place.</span>
<script src="https://gist.github.com/3132389.js?file=evo.sh">
</script><br />
Now the contents of the two files you touched. First the "hello world" Flask app goes in the myapp.py file:<br />
<br />
<script src="https://gist.github.com/3132440.js?file=myapp.py ">
</script><br />
<span style="background-color: white;">Next we we take the </span><a href="http://packages.python.org/Flask-Script/" style="background-color: white;" target="_blank">tutorial Flask-Script manage.py file</a><span style="background-color: white;"> and adapt it very slightly:</span><br />
<br />
Now we are ready to create a simple migration for a Post model for a blog:<br />
<br />
<script src="https://gist.github.com/3132466.js?file=create.sh">
</script><br />
<br />
and you get the template file which looks like this:<br />
<br />
<script src="https://gist.github.com/3132482.js?file=0001_post.py ">
</script><br />
<br />
In the file put your Post model and modfy "up" and "down":<br />
<br />
<script src="https://gist.github.com/3132493.js?file=0001_post.py ">
</script><br />
<u>Our First Migration</u><br />
<br />
<script src="https://gist.github.com/3132533.js?file=run.sh">
</script><br />
<u style="background-color: white;">Our Second Migration</u><br />
Now if we want to modify that Post model we can again do:<br />
<br />
<script src="https://gist.github.com/3132539.js?file=create.sh">
</script><br />
<span style="background-color: white;">modify your model and modfy "up" and "down", I've left some comments so you can see what I did:</span><br />
<br />
<script src="https://gist.github.com/3132557.js?file=0002_post.py ">
</script><br />
<span style="background-color: white;">and run it again.</span><br />
<br />
<script src="https://gist.github.com/3132533.js?file=run.sh">
</script>
<br />
You should also play around with the "redo" and "undo" commands. That's about it really.<br />
<br />
<span style="font-size: large;"><u>Conclusion</u></span><br />
This is only version 0.6 of Flask-Evolution and it's got a way to go, but I can see some use in using it, but it's far from seamless and it's a long way from <a href="http://south.aeracode.org/" target="_blank">South</a>.<br />
<br />
Documentation is woeful and I'm not convinced the developer <a href="https://github.com/adamrt" target="_blank">Adam Patterson</a> has the time to do the job he'd like to.<br />
<br />
You could always offer to give him a hand!<br />
<br />
<b>If you see anything wrong with what I've written here or if you spot spelling or other mistakes then please let me know.</b><br />
<div>
<b><br /></b></div>
Good luck fellow traveller.<br />
<br />tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com1tag:blogger.com,1999:blog-2368086490179632165.post-75815871042834608742012-07-08T15:15:00.001-07:002012-11-04T01:31:32.980-07:00Hosting Domains on Your Domestic Broadband Connection<b>It's not ideal, but it is possible to host your own domain(s) on your home broadband connection. Find out how.</b><br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBr0iscHlYf7aPwodbV0WEcW1zzuHR6gqchxiMn7ZPnGWVHctPm3EtQlOrD_626yTy_LbXdnVtE2_p7AfN6vdpcMk0fLQamxDdMZmBthA0M-TcQInQQpv8rsBEPsu-zs4E564cvJ7IHmA/s1600/dodgy_connection.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBr0iscHlYf7aPwodbV0WEcW1zzuHR6gqchxiMn7ZPnGWVHctPm3EtQlOrD_626yTy_LbXdnVtE2_p7AfN6vdpcMk0fLQamxDdMZmBthA0M-TcQInQQpv8rsBEPsu-zs4E564cvJ7IHmA/s320/dodgy_connection.jpg" width="264" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Hosting on a Budget?</td></tr>
</tbody></table>
I showed you in my previous last <a href="http://terse-words.blogspot.co.uk/2012/07/nginx-uwsgi-living-together-in-your.html" target="_blank">Nginx & uWSGI Living Together in Your Shed</a> how to set up your own Nginx server to serve WSGI applications like Flask, Django, Bottle, Pyramid etc. This allows you to host <span style="background-color: white;">your own applications on the cheap. All you need is any old server right?</span><br />
<br />
Well sort of. I mean it's cheap enough to buy your own domain name and you probably will want to, but when you are filling in the registration what do you enter when it asks for your DNS servers?<br />
<br />
<span style="background-color: white;"><i>Static</i> IPs can be bought or leased from IP providers and this makes it possible to host your own DNS services. However most </span><span style="background-color: white;">of us have <i>dynamic</i> IP addresses at home because our IP providers keep their costs down by buying big blocks of addresses and dynamically allocating us one from the deck each time we need one, and sometimes for their own maintenance reasons too. </span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">This means your IP address will change from time to time. </span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">To get around this you can use a <i>dynamic DNS</i> service such as <a href="http://freedns.afraid.org/" target="_blank">freedns.afraid.org</a> who will point your domain name at your new IP address every time it changes. Cool. Their service is simple, clear and cheap (or even free). I'll let you investigate that yourself, but the main thing is that when you buy your domain name, you need to fill in the DNS bit using the </span><i>dynamic</i><span style="background-color: white;"> DNS server names that they provide.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">The final thing is: How do you know when your dynamic IP address changes and how do you inform your </span><span style="background-color: white;">dynamic</span><i style="background-color: white;"> </i><span style="background-color: white;">DNS</span><span style="background-color: white;"> </span><span style="background-color: white;">service</span><span style="background-color: white;"> when it happens? Well, for my own purposes I wrote a little script which will do this for you, it checks if your IP has changed and if it has it calls the special callback URL that freedns and others supply you with. </span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">You can find it here: </span><span style="background-color: white;"><a href="https://github.com/colwilson/dynamic-dns-updater">https://github.com/colwilson/dynamic-dns-updater</a></span><br />
<span style="background-color: white;"><br /></span>
Good luck fellow traveller.<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=07678bfd-dd79-43bb-a26a-478d9bd9df9d" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-18511022198446758632012-07-04T06:52:00.001-07:002012-11-04T01:30:58.633-07:00Nginx & uWSGI Living Together in Your Shed<b>Let's assume you have just bought the memorable domain name idrinkink.org and you want to host your lovely web app on that old server in the shed. </b><br />
<br />
I recently set up both <a href="http://www.flask-ahoy.com/" target="_blank">Flask-Ahoy!</a> and <a href="http://www.django-ahoy.com/" target="_blank">Django-Ahoy!</a> on a five year old desktop my daughter was chucking out, so I can guarantee you that this configuration can handle enough traffic to get you going, or to demo products to your customers.<br />
<br />
Before we start, I'm assuming you are familiar with python web frameworks (Django, Flask, Bottle etc) and with using virtualenv to segregate your environments. I usually use <a href="http://flask.pocoo.org/" target="_blank">Flask</a> as my framework of choice, so I'm using their "Hello World" app here.<br />
<br />
Let's get started:<br />
<br />
<script src="https://gist.github.com/3046998.js?file=new.sh">
</script>Now the contents of those files:<br />
<br />
<script src="https://gist.github.com/3047037.js?file=app.py">
</script>
<script src="https://gist.github.com/3047044.js?file=uwsgi.sh">
</script>
<script src="https://gist.github.com/3047061.js?file=idrinkink.org.nginx.conf">
</script>Finally copy the config into place and start the uwsgi daemon and the server up:<br />
<br />
<br />
<script src="https://gist.github.com/3047206.js?file=place.sh">
</script>This of course is just the basics, but it will hopefully give you a starting point from which you can fiddle to your hearts content.<br />
<br />
Good luck fellow traveller.<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=07678bfd-dd79-43bb-a26a-478d9bd9df9d" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-91794329338037980972012-07-02T14:28:00.000-07:002012-11-04T01:29:53.832-07:00Using Twitter As Your Flask Login Provider in 6 LinesFirst off, I lied about the 6 lines, but it got you here and now you can see how simple it is yourself. There are actually 40 lines I think.<br />
<br />
When I was building <a href="http://www.flask-ahoy.com/" target="_blank">Flask Ahoy!</a> I wanted to use Twitter as the login provider. I could spend a while boring you about why (as opposed to say Google, Facebook, OpenID or Roll-Your-Own), but I'l save that for another exciting post.<br />
<br />
I have used the extremely handy <a href="http://www.flask-ahoy.com/package/Flask-OAuth" target="_blank">Flask-OAuth</a> to do all the legwork so you need to install that:<br />
<br />
<script src="https://gist.github.com/3035674.js?file=gistfile1.sh">
</script>
Next we need some html for users to click on to sign in and sign out. Try this simple block :<br />
<br />
<script src="https://gist.github.com/3035603.js?file=gistfile1.mustache">
</script>
Next the real code begins. In your views module you need three views:<br />
<br />
<ul>
<li><span style="background-color: white;">One to send you off to Twitter to get authorized (<i>login</i>)</span></li>
<li><span style="background-color: white;">One to get the callback from Twitter and store your authorization credentials (</span><i>twitter_authorized</i>)</li>
<li><span style="background-color: white;">One to log you out and tidy up (<i>logout</i>)</span></li>
</ul>
<br />
<script src="https://gist.github.com/3035708.js?file=gistfile1.py">
</script>
Finally we need a special little method which remembers your credentials:<br />
<br />
<script src="https://gist.github.com/3035729.js?file=gistfile1.py">
</script>
<br />
Now for the sake of safety I'm going to point out a couple of things here:<br />
<br />
<ul>
<li>This scenario uses flask sessions (encrypted cookies) to store your Twitter OAuth token.</li>
<li>When you logout, you are only destroying the <i>twitter_user</i> flag, you should probably destroy your credentials as well.</li>
</ul>
<br />
<br />
Good luck fellow travelers.<br />
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=07678bfd-dd79-43bb-a26a-478d9bd9df9d" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com6tag:blogger.com,1999:blog-2368086490179632165.post-49961790062076219642012-05-04T23:46:00.001-07:002012-07-09T10:10:56.329-07:00Install Google App Engine on Ubuntu 12.04If you want to use <a href="https://developers.google.com/appengine/" target="_blank">Google App Engine</a> on Ubuntu there are a couple of things to note. Firstly GAE has recently moved to version 1.6.5 and secondly Ubuntu has recently released <a href="http://www.ubuntu.com/download" target="_blank">Ubuntu 12.04 (Precise Pangolin)</a>. Precise Pangolin is a <a href="https://wiki.ubuntu.com/LTS" target="_blank">Long Term Support (LTS)</a> version of Ubuntu to come with Python 2.7 installed, and App Engine has been providing it as an option since <a href="http://googleappengine.blogspot.co.uk/2012/02/announcing-general-availability-of.html" target="_blank">February 2012</a>. So if you are starting a new App Engine project, it's probably a good time to <a href="https://developers.google.com/appengine/docs/python/python27/newin27" target="_blank">move to Python 2.7</a>.<br />
<br />
I'll explain briefly how you start a new project and there's a nice clean copy of the code at the bottom that you can cut and paste.<br />
<br />
Let's get the show on the road. Choose a name for the project and create and switch to a virtual environment:<br />
<br />
<pre>PROJ=py27_gae_project
mkvirtualenv ${PROJ}
cdvirtualenv
</pre>
Note that note that "--no-site-packages" and "--distribute" are now the defaults for mkvirtualenv. You don't even need to use "--python=python2.7" on Ubuntu 12.04.<br />
<br />
Now we need to know what <a href="https://developers.google.com/appengine/downloads" target="_blank">the latest version of App Engine</a> is, but as of writing it's 1.6.5:<br />
<br />
<pre>GAE=1.6.5
wget -O /tmp/gae.zip http://googleappengine.googlecode.com/files/google_appengine_${GAE}.zip
unzip /tmp/gae.zip</pre>
<br />
Now let's create an App Engine app. The app will need a name that has been created in the <a href="https://appengine.google.com/" target="_blank">App Engine Console</a>:<br />
<br />
<pre>GAE_APP_NAME=dummy
mkdir -p gae_app/static
</pre>
Now create the app.yaml file:<br />
<br />
<pre>echo """application: ${GAE_APP_NAME}
version: development
runtime: python27
api_version: 1
threadsafe: true
default_expiration: 7d
handlers:
- url: /static
static_dir: static
- url: .*
script: wsgi_app.app
""" > gae_app/app.yaml
</pre>
<br />
<div>
And finally the app itself:</div>
<div>
<br /></div>
<pre>echo """import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('Please replace me with a decent WSGI App Framework such as Flask')
app = webapp2.WSGIApplication([('/', MainPage)],
debug=True)
""" > gae_app/wsgi_app.py</pre>
<pre></pre>
<br />
And finally to run the development server:
<br />
<br />
<pre>python ./google_appengine/dev_appserver.py gae_app/
</pre>
<br />
I hope that this has all been of some help to you. Did I miss anything? Please comment below.<script src="https://gist.github.com/2600447.js?file=py27_gae_project.sh">
</script><br />
<br />tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com6tag:blogger.com,1999:blog-2368086490179632165.post-75792123728245236872012-02-27T15:10:00.000-08:002012-11-04T01:35:31.815-07:00Generating Gibberish in Python - Markov Chains<b>Markov Chains allow us to generate letter sequences which can contain sensible keywords and bulk up your blog. </b><br />
<div>
<br /></div>
<div>
I was once taught to speed read. A sham, a party trick, but interesting too. Essentially you just have to practice gliding over pages in a book or whatever and pick out key information (as opposed to just words). You do this by focusing on starts of paragraphs, which tend to be information rich, and capitalized words which tend to be names and useful information and ignore the rest.</div>
<div>
<br /></div>
<div>
What I'm suggesting here is almost the opposite, generating key words which are beefed up by noise in such a way that, I believe, most search engines will not see the join.</div>
<div>
<br /></div>
<div>
<a href="http://en.wikipedia.org/w/index.php?title=Markov_chain&oldid=477343426" target="_blank">Markov Chains</a> have been around for the most of a hundred years and work by splitting a text into groups of letters of a fixed length and record which groups follow that group anywhere in the text. To generate gibberish all you have to do is choose any group of letters and randomly pick one of the other groups that you know can follow that group; loop until you get enough text. If you use short groups of letters (say two) the text contains less information from the original text than if you use longer groups (say six), because with short groups there are more choices available. You should try this yourself to see what I mean, but I find that a group length of about four is a good start.</div>
<div>
<br /></div>
<div>
Now to the code. Firstly I would love to see your improvements, so please let me know or branch it from <a href="http://github.com/colwilson/Gibberish" target="_blank">GitHub</a>. And don't forget to read the extra information at the bottom of this page to understand how it works.</div>
<div>
<br /></div>
<script src="https://gist.github.com/1927802.js?file=gibberish.py">
</script>
<br />
<div>
<div>
Essentially you just have to practice gliding over pages in a book or whatever and pick out key information (as opposite, generate gibberish all you use short groups of letters of the join. Firstly ins have been around for the most of a hundred years and use longer groups (say six), because obviously there are more choices available. Essentially you just have to do is choose any group of letters and randomly pick one of the other groups that you know can follow that groups of letters of a fixed length and randomly pick one of a fixed length and randomly pick one of a fixed length and randomly pick one of the code. </div>
<div>
<br /></div>
<div>
Chains have been around for the most of a hundred years and words which are beefed up by noise in such a way that, I believe, most the join. GitHub op until you have to do is choose any group of letter sequences which can contain sensible keywords and capitalized words which groups follow that groups of letters (say two) the text contains less information from the original text that you know can follow that groups of letters (say two) the text contains less information from the original text. </div>
<div>
<br /></div>
<div>
What up by noise in such a way that, I believe, most the other groups than if you get enough text than if you use short group anywhere information rich, and capitalized words which tent to the opposed to just words which tent to be names and work by splitting a text into groups of paragraphs, which tend to be information from the original text than if you use short group anywhere in the most of a hundred years and words which are been around for the most of a hundred years and work by splitting a text into group; loop until you have to do is choose any group of letter sequences which can contain sensible keywords and bulk up your blog. Chains have beefed up by noise in such a way that, I believe, most the opposed to just words which groups (say six), because obviously there are more choices available.</div>
</div>
<div>
<br /></div>
<div class="zemanta-pixie" style="height: 15px; margin-top: 10px;">
<img alt="" class="zemanta-pixie-img" src="http://img.zemanta.com/pixy.gif?x-id=27209b2b-df5c-44a2-8ea7-cf6a5a4ad650" style="border: none; float: right;" /></div>
tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-79878542457724165552012-01-22T15:14:00.000-08:002012-01-22T15:24:54.487-08:00Get Real Data from the Semantic Web - Finding Resources<b>In my <a href="http://terse-words.blogspot.com/2012/01/get-real-data-from-semantic-web.html" target="_blank" title="Get Real Data from the Semantic Web">last article</a>, I briefly explained how to get data from a resource using python and SPARQL. This article explains how to find the resource in the first place.</b><br />
Have you ever been taught how to knit? I you have, then you'll know that you are not usually taught how to <em>cast on</em> (or start off) on your first lesson. That's because it much easier to learn how to knit than it is to cast on.<br />
<br />
So it is with the Semantic Web. Once you have a resource URL, it's reasonably easy to extract information linked to that resource, but finding the starting resource is a bit trickier.<br />
So let's just recap how we might get the abstract description for <a href="http://dbpedia.org/page/London" target="_blank">London from DBpedia</a>.<br />
<br />
If we know the URL then that's pretty straight forward:<br /><script src="https://gist.github.com/1659179.js"> </script>
(If you want to follow this tutorial, then you had better copy the sparql.py file from <a href="http://terse-words.blogspot.com/2012/01/get-real-data-from-semantic-web.html" target="_blank" title="Get Real Data from the Semantic Web">there</a>.)<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaKGcKsCSbbt7aKILcr4QgRFRiU6r_9Nu3p-Ct8HhujHLIZ-cPIPUqr5-9qeIQ5mcBvpd0E2A2_A_uNeyPbgtDzTHPaByVGu0prbON3YWnC2eK4J-w9Lkwnk04zpKSJwGk1fqlD_-01F4/s1600/types.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="183" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaKGcKsCSbbt7aKILcr4QgRFRiU6r_9Nu3p-Ct8HhujHLIZ-cPIPUqr5-9qeIQ5mcBvpd0E2A2_A_uNeyPbgtDzTHPaByVGu0prbON3YWnC2eK4J-w9Lkwnk04zpKSJwGk1fqlD_-01F4/s400/types.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">RDF types for the DBpedia entry for London</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<b></b></div>
If you don't however, then you'll have to search for it. According to the dbpedia entry, London is many things, including a <b>owl:Thing</b>, there are a lot of Things out there, probably enough to make even the DBpdia endpoint time out, so let's choose something more restrictive such as <b>yago:Locations</b> but not too restrictive, for example <b>yago:BritishCapitals</b>.<br />
<br />
<script src="https://gist.github.com/1659199.js">
</script>
<br />
Just to be a smart ass as I finish off, you can get both at the same time by doing this, but don't forget that doing this will stress the SPARQL endpoint more than is probably necessary. Be kind.
<script src="https://gist.github.com/1659213.js">
</script>tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0tag:blogger.com,1999:blog-2368086490179632165.post-83906231974842075282012-01-19T13:53:00.001-08:002012-01-19T14:24:11.644-08:00Get Real Data from the Semantic Web<p><strong>Semantic Web this, Semantic Web that, what actual use is the Semantic Web in the real world? I mean how can you actually use it?</strong></p><p>If you haven't heard the term "Semantic Web" over the last couple of years then you must have been in... well somewhere without this interweb they're all talking about.</p><p>Basically, by using metadata (see <a title="RDF" href="http://en.wikipedia.org/wiki/Resource_Description_Framework" target="_blank">RDF</a>), disparate bits of data floating around the web can be joined up. In otherwords they stop being disparate. Better than that, theoretically you can query the connections between the data and get lots of lovely information back. This last bit is done via <a title="SPARQL" href="http://en.wikipedia.org/wiki/SPARQL" target="_blank">SPARQL</a>, and yes, the QL does stand for Query Language.</p><p>I say <em>theoretically</em> because in reality it's a bit of a pain. I may be an <em>intelligent agent</em> capable of finding linked bits of data through the web, but how exactly would you do that in python.</p><p>It is possible to use <a title="rdflib" href="http://code.google.com/p/rdflib/" target="_blank">rdflib</a> to find information, but it's very long winded. It's much easier to use <a title="SPARQLWrapper" href="http://sparql-wrapper.sourceforge.net/" target="_blank">SPARQLWrapper</a> andin fact in the simple example below, I've used a SPARQLWrapperWrapper to make asking for lots of similarly sourced data, in this case DBPedia, even easier.</p><p><script src="https://gist.github.com/1643191.js"> </script></p><p>To use this try importing the DBpediaEndpoint and feeding it some SPARQL:</p><p><script src="https://gist.github.com/1643199.js"> </script></p><p>Your homework is - How do you identify the resource_uri in the first place?</p><p>That's for another evening.</p>tersecolhttp://www.blogger.com/profile/07607140110549407614noreply@blogger.com0