Chisimba JabberBlog

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

HOWTO set up and install a Sphinx Search system on top of a Chisimba database Sphinx is a full-text search engine, distributed under GPL version 2. It is not only fast in searching but it is also fast in indexing your data. Currently, Sphinx API has binding in PHP, Python, Perl, Ruby and Java. Recently, in order to improve search times and results in the Chisimba framework (http://www.chisimba.com) I had a look at the Sphinx Search code (http://www.sphinxsearch.com). This is a quick HOWTO to get Sphinx search up and running on your Ubuntu system to take advantage of the remarkably fast and easy to use search capabilities of Sphinx. NOTE: The Ubuntu packages for Lucid Lynx are not great IMO. It is actually easier to compile from source. I installed everything in /opt but you are more than welcome to install it wherever makes sense to you. 1. Download Sphinx from sphinxsearch.com. For this tutorial I sed the latest version (1.10-beta) [code]wget http://www.sphinxsearch.com/downloads/sphinx-1.10-beta.tar.gz[/code] Note that the "beta" bit does not refer to the stability of the overall package, just the new features, so this is a pretty safe bet. 2. Open up a terminal and extract Sphinx: [code]tar -xvf sphinx-1.10-beta.tar.gz[/code] 3. Sphinx needs the MySQL client headers to build against if you are going to be indexing MySQL databases (or PostgreSQL) [code]sudo aptitude install libmysqlclient15-dev[/code] 4. Install Sphinx: [code]cd /opt/sphinx-1.10-beta/ ./configure --with-mysql --with-pgsql make make install [/code] Note: the above assumes you are building for both MySQL and for PGSQL! 5. Test your installation! [code]search Sphinx 1.10-beta (r2420) Copyright (c) 2001-2010, Andrew Aksyonoff Copyright (c) 2008-2010, Sphinx Technologies Inc (http://sphinxsearch.com) Usage: search [OPTIONS] Options are: -c, --config use given config file instead of defaults -i, --index search given index only (default: all indexes) -a, --any match any query word (default: match all words) -b, --boolean match in boolean mode -p, --phrase match exact phrase -e, --extended match in extended mode -f, --filter only match if attribute attr value is v -s, --sortby sort matches by 'CLAUSE' in sort_extended mode -S, --sortexpr sort matches by 'EXPR' DESC in sort_expr mode -o, --offset print matches starting from this offset (default: 0) -l, --limit print this many matches (default: 20) -q, --noinfo don't print document info from SQL database -g, --group group by attribute named attr -gs,--groupsort sort groups by --sort=date sort by date, descending --rsort=date sort by date, ascending --sort=ts sort by time segments --stdin read query from stdin This program (CLI search) is for testing and debugging purposes only; it is NOT intended for production use. [/code] If you get the above, you are in business! OK so you have Sphinx up and running. The next step is to get some data from your database to populate the index with. As an aside, Sphinx installs 4 programmes to your environment which work off of a sphinx.conf file. Let us create a very basic conf file to work from in your sphinx-1.10-beta directory create a file called sphinx.conf: [code] source tweetgator { type = mysql sql_host = localhost sql_user = root sql_pass = s3cr3t sql_db = tweetgator sql_port = 3306# optional, default is 3306 sql_query = SELECT puid, tweet, tstamp, screen_name, name, location, id from tbl_twitterizer sql_query_info = SELECT * FROM tbl_twitterizer WHERE puid=$id } index tweet { source = tweetgator path = data/tweetgator docinfo = extern charset_type = sbcs } indexer { mem_limit = 64M } searchd { port = 3312 log = log/searchd.log query_log = log/query.log read_timeout = 5 max_children = 30 pid_file = log/searchd.pid max_matches = 1000 } [/code] From the above, you can see that we are indexing the "Tweetgator" module running on our existing Chisimba database. Save that file and exit your editor. Next up, we need to create some directories to hold our index data and log files: [code]mkdir log mkdir data [/code] We then run the indexer to go and grab all the data: [code]indexer --all[/code] This will produce some informative output to tell you how many records have been indexed etc. [code] Sphinx 1.10-beta (r2420) Copyright (c) 2001-2010, Andrew Aksyonoff Copyright (c) 2008-2010, Sphinx Technologies Inc (http://sphinxsearch.com) using config file './sphinx.conf'... indexing index 'tweet'... collected 8673424 docs, 1405.1 MB sorted 176.3 Mhits, 100.0% done total 8673424 docs, 1405117673 bytes total 168.080 sec, 8359780 bytes/sec, 51602.73 docs/sec total 445 reads, 6.806 sec, 1979.5 kb/call avg, 15.2 msec/call avg total 1741 writes, 3.157 sec, 1014.4 kb/call avg, 1.8 msec/call avg [/code] The above will obviously be different according to the size of your tweetgator table! You can now use the built in search program to go and query your spanking new index! Go on, give it a try! I know that my tweetgator table has a bunch of World Cup references in it, so I tried: [code]search vuvuzela[/code] and got back a bunch of results. You are now ready to move onto the next step which I will outline later (installing and configuring the Chisimba Sphinx module)

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

HOWTO get the Yaz libraries up and running in your PHP5 installation. YAZ is a programmers’ toolkit supporting the development of Z39.50/SRW/SRU clients and servers. Z39.50-2003 (version 3) as well as SRW/SRU version 1.1 are supported in both the client and server roles. The current version of YAZ includes support for the industry standard ZOOM API for Z39.50. This API vastly simplifies the process of writing new clients using YAZ, and it reduces your dependency on any single toolkit. The PECL extension is described as: PHP/YAZ is an extension to the popular web server language PHP that implements Z39.50 origin (client) functionality as well as the SRW/SRU protocols. The following Z39.50 services are supported by this extension: init, search, present, scan and sort. The extension can handle GRS-1, MARC, SUTRS and XML. The module also allows you to convert MARC in ISO2709 to MARCXML on the client side. The PHP/YAZ extension is part of PHP 4.0.1 and later but has now been moved to PECL. As a PECL module, PHP/YAZ is now independent of PHP versions. It works with both PHP 4 and PHP 5. You need to install the YAZ dev libraries as well to get the headers so that the PECL extension can build against them. So now we install the lot with: [code] aptitude install libyaz3 libyaz3-dev yaz-doc pecl install yaz [/code] Once the YAZ extension is built properly, you must then remember to add your new extension to the php.ini file so that PHP knows about it. I am using Apache, so I add it to the apache2/php.ini file. [code] extension=yaz.so [/code] Restart/Reload Apache and then check for the new extension with [code] php -i | grep yaz [/code] on the commandline, or use a regular phpinfo() call within a PHP script. Remember that if you use the commandline only, you will also need to add the yaz extension line to your cli.ini file! With that, you should be up and running with YAZ in PHP5

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

HOWTO install MongoDB on Ubuntu Lucid Lynx 1. Try the regular package. If you cannot connect via the commandline with an error saying: [code]mongo: error while loading shared libraries: libmozjs.so: cannot open shared object file: No such file or directory[/code] It simply means that the XULRunner package is still messed up. If you can connect, you may stop reading and enjoy your new MongoDB installation. Other symptoms that you have hit a block are if you are using PHP5 (as I am in this case) you get an error that looks like: [code]MongoConnectionException: localhost: Transport endpoint is not connected[/code] 2. I am presuming you had a problem as described in [1]... OK so, you now need to change your sources in Ubuntu and grab a later version (snapshot is probably the one for you). To do this, simply open up a root terminal (or sudo) with the following commands: [code] sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10 sudo apt-add-repository "deb http://downloads.mongodb.org/distros/ubuntu 10.4 10gen" sudo aptitude update sudo aptitude install mongodb-snapshot [/code] You should now be able to open up a terminal and use the MongoDB shell: [code] mongo MongoDB shell version: 1.5.7-pre- connecting to: test > [/code] If you see the above, you are ready to go! To test your shiny new MongoDB install with PHP, make sure that you have the mongo PECL extension installed, and then try out the following script: [code] comedy; $collection = $db->cartoons; // add an element $obj = array( "title" => "Calvin and Hobbes", "author" => "Bill Watterson" ); $collection->insert($obj); // add another element, with a different "shape" $obj = array( "title" => "XKCD", "online" => true ); $collection->insert($obj); // find everything in the collection $cursor = $collection->find(); // iterate through the results foreach ($cursor as $obj) { echo $obj["title"] . "\n"; } // disconnect $m->close(); ?> [/code]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

The Chisimba team is finally back from yet another extremely successful OSCON, held this time back in Portland, Oregon in the USA. We hosted a very popular booth in the exhibition hall, as well as attended many interesting sessions! I gave a 45 minute talk on Creating business opportunities through the use of the Chisimba Tribe module, which was very well received overall! A copy of the presentation slides (CC BY-SA license as usual) is available on http://presentations.chisimba.com as well as on http://slideshare.net Here are the slides from the talk rendered via the Chisimba filter: [WPRESENT: type=byurl, url=http://presentations.chisimba.com/usrfiles/webpresent/gen14Srv43Nme33_21550_1279914144/gen14Srv43Nme33_21550_1279914144.swf] Between Wesley Nitsckie and myself, we also managed to capture some images of the booth and other OSCON things. My photo set is available at http://www.flickr.com/photos/paulscott56/sets/72157624510526629/ and some pictures of other things that happened on the trip are available at http://www.flickr.com/photos/paulscott56/sets/72157624501977882/ Enjoy and I am certainly already planning my talk for next year!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Yesterday, I was struggling to access my banks' online banking portal. I was not the only one. I tweeted the official user for the bank and received the following reply: "Hi there, we're having a problem with our site due to excessive traffic. It will be sorted ASAP" OK, fine. I guess that can be an excuse - well for about 5 minutes. The fact here is that the site was down for a long time (almost all afternoon) and failing a full scale DDOS attack, this is not an excuse at all. What I am talking about here is that a properly architected modern application should be able to scale almost indefinitely. There is no excuse for traffic volumes being an issue at all. All operations centres should at least have the following in place: 1. A scalability plan 2. Extra caching servers available 3. Additional redundancy on core services 4. Either a HA set up or at least a round robin DNS set up to cater for these kinds of situations. Chisimba on the other hand [b]can[/b] scale properly in these situations. Using a proper architecture for a modern web application, we are able to do all of the above quickly and in an agile manner. Using software like MemcacheD, APC and LVS, we can scale almost indefinitely. In most cases where we have had to deal with massive loads, the network gives out (saturated) well before the application. With things like memcache we can dynamically add caching servers to our cache pool (cluster) to ensure a good experience to all. Chisimba also makes use of 3 levels of object caching, to ensure the best possible caching structure and the most efficient operation and performance of the application. The truly cool thing here is that even if you don't have enough server infrastructure to handle the loads, you are easily able to use desktop machines and commodity hardware as your additional capacity until you can replace them! Cool huh?

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Broadband for all? Finland has been in the news again with their broadband as a legal right initiative, where each and every single person has a legal right to at least 1mbps broadband internet. This is all fine for a highly developed country and I congratulate the Finnish people and government on making it so. The next question on almost everybody's lips though, is "What about us?". Indeed, what about South Africa? I wrote an article about this last year, but failed to find it in my backup archives after a quick look at where I thought it was, so I am going to paraphrase that text here. Thanks to @yossihasson for reminding me of this stuff! The main issue that we have in South Africa is that there is not really enough internal connectivity. Yes, sure, people who can afford it can get local only ADSL lines etc, but the majority of folks still have no internal connectivity at all (or external for that matter). The key to stimulating growth (economic as well as technological) is to foster greater communication infrastructures and connectivity to [b]other South Africans[/b]. Think about it. International connectivity is great for people that do business overseas, and for other folks in the tech industry that collaborate with people in different timezones, but for the average South African? They really don't need that very much. Facebook and Google are popular, but are mainly used to communicate with friends inside the country anyway! What I propose for South Africa is to provide free local networking, local being South Africa. If everyone could at least talk to everyone else, imagine the business opportunities! Now, most of you will be shaking your head by now and saying "Well who will pay for this?" The answer to that question is [b]NOT[/b] government/councils/rich folks/big companies etc. It is the entrepeneur. If someone can be taken through basic GNU/linux system admin and provided with a small loan to buy a server, some cabling and some cheap devices to provide a basic network in an area, the collective can use a high bandwidth backhaul to the next server. Entrepeneurs could easily pay for this kind of infrastructure quickly by providing value added services on their network (like buying airtime, advertising etc) as well as providing people with a means to connect to the network. I am not talking here about expensive mobile phones or desktop computers, I am talking about enabling your current television set with a wireless keyboard and mouse and a small device to provide the network (they exist, I just forget the name for now). This will enable a household to interact with local SA businesses, other people, services etc. Those people can also [b]provide[/b] services that they can do to others. This will stimulate the economy locally, which will stimulate on a national basis. eGov services and VoIP can also be provided to these devices, as well as full HD tv and video on demand. Seeing some opportunity yet? Once people are working efficiently, they may in fact opt to purchase international bandwidth, which is OK. That is their choice. The main thing here is that you don't need laws to do this, you need banks to understand that a few small loans of approx R30k or so can change the country. Telkom and other cable providers will also need to come to play for the backhauls, but that infrastructure is now mostly in place. Again, the long term benefits to them of doing this is quite obvious! So do we need 1mbps bandwidth for all? No, these devices provide on average 10GBps between each other and on a fiber based backhaul that can be maintained throughout the country. On the amount of money that I can make providing services over that network or saving on my landbased phone calls or my satellite based TV I can easily buy 1 or even 4mbps international connections! We can make this happen, it is simple and useful.

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Project 365 #365 post I will use the Chisimba Flickr filter to grab all of my Project 365 posts and display them as a slideshow here [FLICKR: type=slideshow,userid=72787861@N00,tag=project365,width=600,height=600]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Twitpic Project 365 So there seems to be some interest in this, so it can start tomorrow. The way to participate is to upload a photograph to twitpic (http://twitpic.com) every day for 365 days (1 year). It really is that simple. The photographs can be of anything you choose. It is [b]your[/b] project 365! The only thing that is required to participate is to have a twitter account (http://twitter.com) a twitpic account (or one of the other image sharing services for twitter) and to tag your photographs with #365 Enjoy! Have fun! There are no hard and fast rules. Use anything from high end cameras to your mobile phone to create. Mix it up or keep it pure! All up to you! Most importantly, have fun!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Some pictures that I (@paulscott56) took today. @wesleynitsckie took some too (see his Flickr account) [FLICKR:type=slideshow,userid=72787861@N00,tag=uwc,width=600,height=600]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Why microformats are probably the most important thing you can do for your web right now Microformats are great. They allow you to publish information snippets through both machine readable as well as human readable text, as well as consume information published by others. Take, for example the vCard. Many people use the vCard and hCard formats as well as the hCalendar format to share information via email. These things are also often added to their web sites as well for additional info. The next step in this evolution would be to also tie these things into the FOAF (Friend of a Friend) protocol to publish and share much richer information. This information can be generated by large social networks quite easily and published as a microformat enabled export. What this means is that if you have "friends" or "followers" using the Facebook.com and Twitter.com nomenclature, and you sign up to another site that implements hCard import, the "friends" that you have on one site can be set as "friends" on the new site automatically. This means also that there is no need to re-friend people and start from scratch every time you add a new service. This also means that smaller, more niche networking sites can easily become part of a larger, easier to maintain and add to community and can start pubishing additional information in the form of say activity streams to all the other sites, making for a better web all round (more social, semantic, temporal and spatial). This means that specialized services (such as geolocation as an example) can be offset to smaller specialized service sites instead of tying up resources on more people-centric sites. Chisimba already has a full spec FOAF module, as well as produces microformats. I have done the basics on the import (user import via microformat) but it is still a work in progress. I will keep you updated as things progress!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Steal this idea (#sti on Twitter) Semantically interconnected tagging of things. OK that sounds a bit of a mouthful, but the idea is relatively simple. Create tags (sticker or plaque that can be riveted to a device like a TV or something that simply has a QRCode or some other identifying tag on it. The codes can be generated from a web based application that builds up a database of tagged devices and who they belong to. When selling a device, the db is updated via mobile or web using the QRCode in a secure manner, and if the device is stolen, a reward can be offered via the site for its safe return to the rightful owner. Ownership can be determined by a simple read of the QRCode by mobile phone. Devices can also be tracked by geographic location, so if something is found outside of a specific area, police can be flagged to check it out etc. So, what do you think?

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

This looks pretty interesting! Description from announcement... PiCloud, a cloud computing platform for the Python Programming Language, has released version 1.9 of its client library, cloud. PiCloud enables Python users to leverage the power of an on-demand, high performance, and auto scaling compute cluster with as few as three lines of code! No server management necessary. You can find out more here: http://www.picloud.com What's New: * Store your data files on the cloud using our cloud.files interface! * Greatly reduced the cloud library's memory and cpu usage when sending large data. * Map jobs begin processing before cloud.map() returns--large performance gains. * Persistent user processes has in many cases reduced function overhead by over 50%. * Increased network protocol reliability. * Profiling is now disabled by default, but can be enabled with the _profile keyword. * Bug fixes, and much more! Full service description: PiCloud is a cloud computing platform that integrates into the Python Programming Language. It enables you to leverage the compute power of Amazon Web Services without having to manage, maintain, or configure virtual servers. PiCloud integrates seamlessly into your existing code base through a custom Python library, cloud. To offload the execution of a function to the cloud, all you must do is pass your desired function into the cloud library. PiCloud will then run the function on its high-performance and automatically-scaling cluster. We quickly scale our server capacity to meet your computational needs, and only charge you for the resources you actually consume. Getting on the cloud has never been this easy! PiCloud improves the full cycle of software development and deployment. Functions that are run on PiCloud have their resource usage monitored, performance analyzed, and errors traced; we further aggregate all your functions to give you a bird's eye view of your service. Through these introspective capabilities, PiCloud enables you to develop faster, easier, and smarter. Common use cases for our platform: * Scientific computing * Video and image encoding * Statistical analysis of data sets * Real-time data processing * Charts and graphs generation

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

A big word of thanks to @andrevr and @fionaby for helping to test the @twitter @anywhere hovercard functionality! You should also follow them if you are a twitter user!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Instead of doing a follow Friday tweet (or 6), I have used @anywhere to enable folks to follow people from my site. Hover over the Twitter usernames to get the hypercard that allows you to follow these great people: @cazpi @dkeats @charlvn @jdtangney @wesleynitsckie @thebrianmanley @capetown @brandmonday @karitz @hennokruger @smikobus @corrie206 @alastairotter @kmf @janbraai @nnenna Much easier and cleaner yes?

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Yesterday I conducted the Chisimba training at the Kofi Anan Centre of ICT excellence in Accra, Ghana. There was an excellent turnout of delegates, with a full room of 30 people attending. We were forced to turn a few people away due to space requirements. I decided to keep the training session as agile as possible, although the session was branded as coding and deploying Chisimba applications. It turned out that most delegates were interested in looking at eLearning as well as content managemant. It was for this reason that we started with eLearning, and then took a look at setting up and customizing the CMS module as well as the news module (the module running this site is the news module). There were some interesting questions at the beginning in the form of "Drupal does x, can Chisimba do the same?", but towards the end of the session those had changed to "Why doesn't Drupal/Joomla! also do this? Chisimba is way more powerful!" All in all it was a very successful day and it was enjoyed by all. Everything from installation to final deployment and customization was covered in a single intensive day (next time, I am sure that someone will remind me to break for lunch!) Towards the end of the day, I was interviewed by Ghana National television as well on AVOIR and Chisimba, as well as the work that we are doing in Africa for the world.

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

THIS IS A MILD RANT Yet another idea that I proposed and implemented (to a degree) has been adapted and stolen by a large corporate [b]without any attribution[/b] Don't get me wrong, I fully support competition, co-opetition and collaboration, but give attribution where it is due is all. I will not be naming and shaming anyone, mainly because they have lawyers, and I am a nice person. The point here is that as anyone, be it a multinational multimillion Rand/Dollar company, or a mom and pop store type company, if you use one of my ideas, please give me a line of credit at least, somewhere. Something along the lines of "based on an original idea by..." or something is fine, nothing huge, no royalties or anything else, I don't care. I do care about Fair Use and attribution where it is due though. So, if you are going to use something that I have proposed, then let me know. Don't implement the idea and then have the cheek to get your new product to follow me on twitter as well! I tell people about big ideas, I want people to make money from them. I would prefer that smaller smartups make money from them, but there are no set rules there. All I ask is that you are transparent about it, and that you give me a teeny weeny bit of credit. I guess I have the option of NOT telling anyone any ideas, but that defeats the purpose of ideas and community surely? I have ideas that could quite easily turn into very lucrative products almost every day. I like to share them. I know that I am not the only person in this situation, so large corporates: It will not impact your business in any significant way to add in a line of credit for the idea mongers. Do it, it really will make you less evil.

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

HTML5 Elements - Gradients WebKit started the support of CSS gradients and Mozilla have followed suit and implement a slightly different way of generating them but support is there. Make sure to set a standard background colour for browsers that don’t support RGBa. The example below shows a linear gradient going from bottom-to-top. h1 a { background: -webkit-gradient( linear, left bottom, left top, color-stop(0.23, #c34000), color-stop(0.62, #ff5400) ); background: -moz-linear-gradient( center bottom, #c34000 23%, #ff5400 62% ); }

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Retweet button for the Chisimba blog Yesterday, I was asked to do a quick retweet button for the Chisimba blog module. I thought that it should be relatively easy to do, and wanted the following characteristics: 1. It should increment the retweet number in an AJAXy style, so as not to reload the page 2. It should be configurable in style to match skins. 3. It should include an icon due to the multilingual capabilities of Chisimba, where the concept of "retweet" may be difficult to understand So, I opened up a browser and did a quick search and came up with http://ejohn.org/ There is a great example there that did everything that I wanted. Within about 20 minutes, the blog had fully configurable retweet functionality. For those using the Chisimba blog, please take note of the new config options (found in sysconfig - blog): 1. Retweet_style - which is the style that you would like your retweet button to look like. Valid styles are retweet and retweet vert. Retweet makes a horizontal button with the number of retweets to the left of the button, while retweet vert will make a vertical style button with the number of retweets above the button. 2. Retweet_Status - this is the status that is automatically added to your retweets. Can be anything as long as the URL + Status is less than 140 chars. Nice ones are along the lines of "Interesting read" and "Check out this post" etc.

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Document Freedom day March 31st is Document Freedom Day (DFD), a day for document liberation marked around the world. It will be a day for us all to educate our friends and neighbours of the importance of open document formats. OpenOffice.org uses the OpenDocument Format (ODF), and that means at least one hundred million people around the world use it at home, at work, at school: everywhere. What is more, an impartial open standard like the ODF can be implemented by any application, free or not. It eliminates vendor lock-in and gives users real choice today and tomorrow. OpenOffice.org is proud of being a part of DFD, and joins many other groups, vendors, and projects in the global effort to educate the world of the freedoms given by open documents and standards. Document Freedom Day unites us all. Open standards, and especially the ODF opens the world to all. Details on the Document Freedom Day can be found in our wiki at http://wiki.services.openoffice.org/wiki/Dfd and on the official campaign website at http://documentfreedom.org Join us, join the future.

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Why NoSQL is NOT a replacement for your RDBMS Been reading some disturbing posts about NoSQL recently. This one http://teddziuba.com/2010/03/i-cant-wait-for-nosql-to-die.html just seemed to be the nail in the coffin for me. Hopefully this will clear up some of the FUD and misconceptions around NoSQL (as I use and understand it) Why on earth would you want to [b]replace[/b] your RDBMS with a NoSQL store is beyond me. This is akin to driving in a screw with a hammer. Should work, but why do it when you have a full set of decent screwdrivers? NoSQL is [b]NOT[/b] a replacement for RDBMS, it is merely another tool in a toolset that can be used for a specific purpose. Examples of Walmart and Google AdWords have been cited as running just fine off RDBMS (MySQL in these cases), which is just fine. Why that is an argument [b]against[/b] the use of NoSQL is beyond me. NoSQL is about a document store, or a schemaless data store. Nobody ever said it was a relational data store at all. Businesses (both large and small) need relational data. Some businesses also need schemaless data stores (not all businesses mind you, some). NoSQL [b]does not[/b] mean No SQL, it means [b]Not ONLY SQL[/b]. Remember this well. Take it as "NoSQL stores should be a compliment to your RDBMS store(s), not a replacement" Do not let the FUD become myth, which becomes reality. Two different tools, fit for different purpose. Let me remind the naysayers of a famous quote: "Nobody will ever need more than 640k RAM"... Can't wait for NoSQL to die? I can't wait for FUD to die

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Slides from my presentation at IS on Fri 26th March 2010: [slideshare id=3585270&doc=ispres-100329091556-phpapp02]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Why I won't be signing up for uncapped ADSL anytime soon Yes, I know, I am a geek and I probably am one of the few people in SA that actually [b]need[/b] uncapped ADSL, but I won't be signing up for it soon at all. This is why: 1. Every kid with a Warez/Pr0n/Pirate MP3 fetish will be saturating the networks 2. Every adult will suddenly be able to safely update their 2 year old unmaintained Windows installation and saturate the networks 3. Every gamer (adult or not) will be suddenly inspired(?) to game as much as humanly possible over the next few months, incite their gamer friends to do the same and saturate the networks 4. Every geek will suddenly feel that it is OK to download [b]every[/b] GNU/linux distribution ever made so that they have it, you know, just in case thereby saturating the networks 5. Every YouTube video WILL be watched, probably twice, shared and watched again. This was never really possible until now, so the networks will be saturated. There are a bunch of other reasons, but almost all of them end in "... and so the networks will be saturated" It is for that, that I have decided that we shall wait for the novelty to wear off and for the petabytes of wasted bandwidth to be wasted before we take the plunge. I just really hope that the ISP's that are offering these packages are mindful of this, and are putting provisions in place to ensure that I can still get stuff done on a reasonable network.

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

I am speaking again this year, so I thought I would get the promo done early

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Skipfish web application security scanner Last night, approximately 13 hours after release, I decided to try out the new security scanner from Google called Skipfish. Installation was simple, just download the gzipped tarball and extract it somewhere useful and run make on the already generated Makefile. A skipfish binary is created that does all that you need. Skipfish uses dictionaries of extensions and common SQL injection vectors etc, so you need to tell it where to get that information. Luckily it comes with a few different dictionaries already that you can get going immediately. Please do read the README files for both the application AND the dictionaries to get the best from it! Basic usage is in the form of: ./skipfish -W dictionaries/[dictionary].wl -o [output_dir] http://[url] The test that I ran I also used the -A switch which provides basic auth into your app. Tests take quite a while to grind through, especially if the network connection to your server is slow. One thing that I did run into was that I tried it on localhost in a specific directory, which didn't work as skipfish tests the [b]domain[/b]. This means that in oder to test http://127.0.0.1/myapp/ you need to set up a virtual host quickly as http://myapp.localhost otherwise your report will be polluted with myapp/ myotherapp/ and whatever else you are running off localhost. Overall, this looks like a very useful tool and I recommend adding it to your toolset.

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

How to get Apache Cassandra up and running in a few easy steps This is an Ubuntu/Debian HOWTO, so if you use anything else, I recommend that you grab the Cassandra binary and go with that rather First off, we need to add some sources to our apt sources. You can either do this by editing /etc/apt/sources.list or adding a software source via System -> Administration -> Software Sources Add the following sources (or the latest sources from http://wiki.apache.org/cassandra/DebianPackaging ) [COLORBOX: boxtype=greenbox] [code type="bash"] deb http://www.apache.org/dist/incubator/cassandra/debian unstable main deb-src http://www.apache.org/dist/incubator/cassandra/debian unstable main [/code] [/COLORBOX] Once that is done, go ahead and refresh your sources. The GUI manager will ask you to do this automatically, but on the terminal you will need to sudo apt-get update Once done, you are ready to install! [code type="bash"] sudo aptitude install cassandra [/code] will do it. It is about a 4MB download. NOTE: Your configuration files are in /etc/cassandra and start-up options (heap size, etc) can be configured in /etc/default/cassandra Once you are installed we can run a quick test: [code type="bash"] cassandra-cli --host localhost --port 9160 [/code] will start up the cassandra CLI interface for us to run a couple tests on. Typing help or ? will give you more help You can see your cluster name with the following: [code type="bash"] show cluster name [/code] Now lets set a value: [code type="bash"] set Keyspace1.Standard1['pscott']['first'] = 'Paul' set Keyspace1.Standard1['pscott']['last'] = 'Scott' set Keyspace1.Standard1['pscott']['age'] = '32' [/code] After each addition you will get a notification that the value was inserted. Now, lets get all the data back as our final test: [code type="bash"] get Keyspace1.Standard1['pscott'] [/code] If all the results that you have just inserted come back to you, WIN! you have a working single node cassandra install! Feel free to play around with more options, then go for a multinode install and start having some real fun!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

From http://blog.stuartherbert.com/php/2010/02/26/getting-gearman-up-and-running-on-ubuntu-karmic/ How to get Gearman up and running on Ubuntu in a few easy steps [COLORBOX: boxtype=greenbox] Getting Gearman up and running on Ubuntu 9.10 (Karmic Koala) is very straight-forward and only takes a few minutes, but oddly not clearly documented on Gearmanâ€™s own wiki at the time of writing. 1. Add â€œppa:gearman-developers/ppaâ€ as a software source. 2. sudo apt-get install gearman-job-server 3. sudo apt-get install libgearman-dev 4. sudo apt-get install uuid-dev 5. pecl install â€œchannel://pecl.php.net/gearman-0.6.0â€³ 6. Add a gearman.ini to /etc/php5/conf.d, with â€œextension=gearman.soâ€ as the contents 7. sudo /etc/init.d/apache2 restart 8. Check phpinfo() to make sure gearman extension loaded Youâ€™ll find that the gearmand process is already up and running, and listening on port 4730 on localhost. All you need to do now is to write some code to take advantage of it :) [/COLORBOX]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Why appstores destroy your freedom Would you buy a computer that you were only allowed to run applications that the manufacturer said you were allowed to use? Maybe, for some, but what if that same manufacturer said that you weren't allowed to put any documents, images (even your own), sound files, movies, music, or text that they didn't specifically sanction onto your shiny new computer? Hmm, not so great eh? Appstores are a great idea for a connected world, but as with many other things, appstores have the potential to be great, or the potential to once again become problematic in a Free world. Lets take a small example: A garage band decides to try and make it in th ebig time and would like to hack together a demo recording to present to the record labels. The band realises already that the record companies control almost everything we listen to already (including the radio stations - even the "independant" ones) so they need to take this route. They would like to record and then edit using the high quality sound production software that they have heard of that also happens to be Free software. Unfortunately, they all own devices that cannot be used without the use of an AppStore, and the AppStore has a policy that you can only use their own expensive software. This particular device also only connects to a single appstore, and you cannot change that, because the device manufacturer said so. Some of the band members end up spending most of their savings and the other band members start mowing lawns instead of jamming to make enough to buy the sound editing app. This takes a few months to do, because the application is extremely expensive. Eventually, the application is bought and the disc is edited and ready to burn to a CD. Hang on, the device has no FireWire (IEEE1394) port, and, get this, no USB port even! Wow, that seems a little strange, how do we attach a portable CD/DVD burner to this device to burn the music? We don't! We are forced to try and sell our art in the device manufacturer's AppStore. We attempt that. The track is rejected as it does not belong to a recognised label (one of the labels that the AppStore guardians have an agreement with), so we are now in a cyclic hell and our band will never amount to anything due to silly draconian policy and loss of Freedom. In the recent past, bands, movie producers and artists have become superstars without the use of the traditional media labels. This has all been made possible by the freedom of the web. Anyone can relatively cheaply bang together a web site and publish their art to it. Anyone on the web is then able to find it, enjoy it and possibly buy it too. This works, and it works well! Why then, would we want to take 10 steps backward and make sure that there is no new or innovative art produced? No new books that may not be published due to some reason or another that we will loose, no anything except the artistic fodder that we are force fed daily through television, radio and big company advertisements. Sad world! I also mentioned that appstores could be great. They could allow any content with user ratings. They could allow devices to connect to different appstores than their own. They could allow freedom. So far none have to my knowledge. Where is the FOSS appstore? Where is the Indy Artist Appstore? Where is the CC licensed community art Appstore? How do I donate $50 to a phoitographer whose work I love on Flickr that uses a CC license? Nope, no appstore for that... You get the point no? Watch out for your Freedoms. Great advertising and passionate salespeople talk a good game. Make sure you don't stop thinking for yourself!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

AVOIR and Chisimba in Cuba AVOIR was recently invited to Universidad 2010 in Havana Cuba through the VLIR sponsored North South South project that was established between VUB (Vrije Universiteit Brussels), UWC (University of the Western Cape) and UoN (University of Nairobi). Elijah Omwenga respresented UoN and I attended as the UWC delegate. Georges Eisendrath was representing both VLIR and VUB at the conference. The first day of the conference (after a very long trip!) was spent on a VLIR partners workshop. This was to enable all the VLIR sponsored projects in Latin America and elsewhere to meet each other and form possible connections and further collaborations. Each institution was given a short presentation slot to let everybody know what they were doing in this space. Elijah Omwenga presented the UWC/UoN/VUB NSS project and the Chisimba based repository system that the project entails, while I was given an opportunity to present AVOIR as a network for collaboration in Africa and beyond. Both presentations were very well received and much interest was sparked in AVOIR and what we do. We also were interesting in the fact that AVOIR could easily be used as a model in other places and a global network should be established, building on to what AVOIR has already achieved in Africa. I was then given the opportunity on the Wednesday to present, with Frederick Questier from the VUB, an hour long workshop on Free and Open Source Software. Many network collaborations were established and a number of new opportunities were created! Chisimba and AVOIR is now set to become a major player in Southeast Asia as well as some parts of Latin America. Many thanks to VLIR, Georges Eisendrath and many others for this opportunity! Images will be posted later to the AVOIR photo pool on Flickr

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

EFF Defending your rights for 20 Years! Support them by buying some stuff from http://www.eff.org/ or donating! [img]http://farm5.static.flickr.com/4043/4325854994_2e9635e35e.jpg[/img]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

HipHop the PHP Compiler for/from Facebook So I have seen literally thousands of tweets about HipHop from many many (reputable) sources and it seems that they are mostly wrong, FUD or oversimplified. The next few lines will dispell those myths and will make you understand what it is, and if it is good for you or not. First, some background: HipHop is a C++ compiler. It converts PHP code into C++ and compiles that into a binary using GCC. This is where you get the speed increase (Facebook now claim 50%), which makes perfect sense. OK great, sounds wonderful, lets grab the source code and deploy like mad! Err, no, lets not, and here is why not: 1. HipHop works on a certain custom set of extensions only 2. HipHop doesn't work with PHP-5.3.x (yet) 3. It works with a very limited set of functions and extensions, all of which require maintenance, yes, by you! 4. If you make a change, you need to rebuild and recompile. This is OK if you do that once every now and then, but remember how long it takes to recompile C++? Yeah, a loooooong time. That means that for your smaller project(s) this is completely useless and lets face it, most PHP projects are not huge (except the huge ones like the frameworks and Yahoo! and Facebook) 5. If you are runnin your PHP from Windows, you are out of luck (twice IMO, but that is another post). 6. If you use eval() see point 5 above. (Thank goodness IMO) So, if you are willing to put up with all of the above, it is a great product (in theory). Some larger frameworks could benefit from using it, but I doubt they will. The divergence between "native" PHP and it's own evolution and the HipHop'ified PHP will get bigger and bigger until there is a fork and well, you have to choose. In summary, you are going to run faster, much faster and use less resources (besides compile time), but there is quite a big tradeoff. Be wary of that! There are other points, of course, but I will leave that for someone else for now. (Think, hmm, PHP is C, this is C++...)

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Collecta Widget in Chisimba I saw from a tweet in the past 20 minutes that Collecta (http://www.collecta.com) has released a quick widget builder to get realtime results in pages. I decided that this was such a good idea that I would make a quick Chisimba filter to use the functionality so that any user could add any custom Collecta widget whenever they needed to. Think also of the eLearning implications here! Anyway, to use the widget (I guess it is a widgetWidget) use the following syntax after updating the filters module: [ COLLECTAWIDGET: term=searchterm,title=title_for_your_widget, width=width_in_px,height=height_in_px ] [b](Remember to delete the spaces in the square brackets)[/b] So a real example would be: [ COLLECTAWIDGET: term=Chisimba,title=Chisimba,width=300,height=500 ] Which renders like: Enjoy!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

AR and why we will (probably) wait a little Lots of people have started talking in earnest in the past year or so about Augmented Reality or AR. If you don't know what AR is, then please see the following Wikipedia article for a refresher: See Wikipedia article at http://en.wikipedia.org/wiki/Augmented_reality (I am not using the Chisimba Wikipedia filter as the above article is quite long) What most of what is available now is the so called "Spatial display" stuff. This is all about having a spatial mashup with whatever is currently being displayed by your camera phone/device. That is all fine and well, except: 1. It relies on a digital compass (only very high end devices have these, e.g. iPhone 3GS, but NOT iPhone 3G) 2. It relies on a decent, fast network connection. 3. It relies on local content being updated frequently for usefulness. These are all very big issues. In South Africa, our terrestrial network is hardly up to the task, and in most places around the country, the mobile networks data offering is shoddy at best (mostly only get a GPRS connection), and then when you do get a connection, it is phenomonally expensive. The other major thing is access to devices. Most people have a mobile phone capable of data these days, but certainly not a high end phone by any stretch of the imagination! Digital compasses and other necessary components are few and far between still. This will take a few more years until it is ready for what I call mainstream (If I can't afford it, it certainly is not mainstream). I [b]could[/b] see possibilities for museums or large libraries or possibly even large university campuses using this type of tech though. Let's say you enter a museum, hire a device for the morning and have an updated display within the walls of the museum. This could also work off the existing wifi infrastructure with the database and software hosted onsite, mitigating most of the problems. People could then wander around and get a 5 Megapixel annotated view of the artifacts instead of looking at them. Same could be done on a large campus. This one may even be useful, as more students could find lecture venues and attend more lectures. Again, we have had this technology available for a few years already. Only now are ARM and other mobile processors capable of dealing with the data volumes, and only now do we have ubiquitous mobile networks to facilitate this. I can certainly bulid the infrastructure for you as a localised service and deploy some amazing applications, but are they really useful yet, and do people know how to use them?

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Mobile location based advertising OK, so we all get the annoying mobile ads and we all wonder where the heck they got our number from right? Right! So now, I think it is time to take the power back! Our mobile phones are extremely personal devices. We have our contacts (addresses, numbers, web etc) of all of our friends/acquiatances/business people, we have personal messgaes as SMS/MMS (some of which contain personal images) and we have other media like sound recordings, video, mp3's, personal ringtones etc. We keep all of this stuff on these devices, but are only mildly annoyed when operators push SMS adverts to our phones? Wow! I say no. There are two options as far as I see it: 1. Location based messages with context. Say I am walking through a shopping mall (location) and I have previously set up that I may be interested in ads from shop X or product range Y (context), I will be notified of possible sales or specials [b]in that particular mall[/b] (location) [b]at that time[/b] (temporal). I will only receive these messages if I want to, or have bluetooth/wifi/IM switched on and opted in. 2. The other option (one which I think I may prefer) is to have a series of automated billboards/signs that are connected to your device via bluetooth and send out personalised ads according to something like your IMEA number. This implies that you have [b]actually opted in[/b] to receive adverts and that you actually want them. The signs can then be personalised to such a degree that they can greet me by name and offer me stuff I actually may want to buy. With both methods, there are privacy issues that I don't have the brains to think through right now, but both are very do-able with technology that has now been around 10 years (implies some stability, right? :-) )

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Using mobile tech in Africa for good So, the recent earthquake in Haiti got me thinking, "what would happen if we had a major disaster here in SA or elsewhere in sub-Saharan Africa?" Then I thought OK, we have at least 67% penetration on mobile phones (all of which support SMS, and all of which support the "emergency messaging service/dial" thing). What would happen if the mobile operators would allow traffic on those specific services during crises? The answer, probably a whole lot. Imagine that we could send SMS at any time during a crisis period on the emergency number and have it posted to a web site, or twitter, or both so that folks could act on it if needs be. We also have millions of folks using mobile data in sub-Saharan Africa, this could be leveraged as well. I am pretty sure that mobile operators are able to turn off the charges for data during a crisis? Tihs all equates to a sort of crowdsourced e-911 system, coupled with Location Based Services (LBS) or your higher end phone's GPS receiver, it would really help in finding people stuck in landslides, trapped in trees during floods, stuck in rubble etc. Again, we have all this cool stuff available, but as usual, we are beholdent to the mobile operators. SMS, as you all should know, is sent on the carrier signal/band, which means that it costs the mobile operators NOTHING, nada, zilch to send. Think about that next time you say to someone "Just SMS me, it is cheaper"...

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

The Chisimba blog Having a somewhat anecdotal look at the download graphs from the Chisimba package servers, I see that the blog module is probably the most popular module of all. It is certainly one of the most downloaded and used modules, outside of UWC's eLearning set up (where it is used too). This, to me is somewhat interesting for the following reasons: 1. Blog was one of the first Chisimba modules that I wrote, right after releasing the basic Chisimba framework. This makes it one of the oldest still working and in use modules in the system 2. Blog has been hacked upon by almost everyone that has written a Chisimba module, therefore becoming one of the most "community oriented" modules in the system. Most of the senior develelopers have had a hand in some aspect of blog, as well as a number of junior devs, interns and other folks just interested in the way that things work. (This has created a little bit of code bloat, which will be factored out ASAP though) 3. Popularity of blog always surprises me, with the ubiquity of blogging platforms such as Wordpress, LiveJournal and Blogger (both hosted and self hosted). I think that the difference is that the Chisimba blog makes use of interop filters that work with Wordpress et al. as well as providing much content and an open API to interact further with all the platforms. Chisimba blog also has many more features than the others although not many people make full use of them all. With the above in mind, what can we do to make things better? How can we make the Chisimba blog even better and more popular? Some example of Chisimba blogs: http://www.dkeats.com/ http://www.cazpix.co.za/ http://eteaching.uwc.ac.za/ (UWC eLearning) and many more...

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

How to use the Activity Streamer module The activity streamer module allows for certain events to be recorded by the system in the background and then re-mixed into other forms and formats further upstream. This may mean that the activities are streamed to services such as Twitter or Facebook, or it could simply mean adding the activities to a database table for later analysis. To use the Activity streamer, most of your code will reside in your module's controller, as this is where your control functions exist within Chisimba. The best way to demonstrate this will be to outline a basic example: The first order of business is to get and instantiate the activity streamer module classes if the module is installed or exists: [code lang="php"] if($this->objModuleCat->checkIfRegistered('activitystreamer')) { $this->objActStream = $this-getObject('activityops','activitystreamer'); $this->eventDispatcher->addObserver(array($this-objActStream, 'postmade' )); $this->eventsEnabled = TRUE; } else { $this->eventsEnabled = FALSE; } [/code] $this->eventsEnabled is simply a property to hold whether or not we will be using the streamer later on in the code, instead of re-doing the check. You will also notice that we have added an observer too to the event Dispatcher. This is a framework object, but it takes a callback function that you will set up. The postmade() function that has been set as the callback above, simply adds the data as an array to the global activitystreamer data table. You are free to override this, of course, but it is preferred that all data is also sent to the global data table for posterity. The next step is to actually update the activity streamer in your code. In a controller action, you need to add a few variables that must exist for a meaningful data entry: [code lang="php"] // update the activity streamer $title = $this->objLanguage-languageText("mod_events_newconnection", "events"); $link = $this->uri(array('action' = '#')); $contextCode = NULL; $author = $this-objUser->fullName($myid); $message = $author." ".$this->objLanguage->languageText("mod_events_isnowfriendswith", "events")." ".$this-objUser->fullName($fid); [/code] The above code was taken from the "events" module when a person adds another as a friend in the social network portion of the code. You will notice that there is a title to the activity, a link back to it, a context code (not used in this case), an author and a message. These are all required for a successful data entry. The next step is to actually create the entry via our controller code. To accomplish this, we use the event dispatcher's post method: [code lang="php"] $this-eventDispatcher->post($this-objActStream, 'events', array('title' = $title, 'link' = $link, 'contextcode' = $contextCode, 'author' = $author, 'description' = $message)); [/code] That will then create the entry and you can then carry on with life. Remember, that every time this event is fired, a data entry will be produced, so please ensure that you code for error conditions and fringe cases too (such as a user is logged out just before the activity is completed etc).

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Someone posted a link to this MySQL set of tips by Mike Chirico, which I have pasted here for your viewing pleasure. If you have anything to add, please contact him on his Sourceforge page, and if you find this useful, please consider donating to SF.net to keep great people like this working. Interesting Things to Know about MySQL Mike Chirico (mchirico@users.sourceforge.net) Copyright (c) 2004 (GNU Free Documentation License) Last Updated: Sun Jan 21 09:44:21 EST 2007 The latest version of this document can be found at: http://souptonuts.sourceforge.net/readme_mysql.htm

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Beautiful! [YOUTUBE]http://www.youtube.com/watch?v=17jymDn0W6U[/YOUTUBE]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Companies and social networks/web presence Recently the bank that I have been using all of my life has started a Twitter presence. There have been some ups and downs as could be expected, but it has mostly been a very positive experience. That is, until I really needed them. Correct me if I am wrong here, but if you are going to use Twitter/Facebook/Whatever to "engage" with your customers, either do it very well, or not at all. Let us return to the bank. The first example of this that really got my goat was that the bank in question asked for feedback via tweets on what they could improve in their internet banking system. I suggested a whole lot of stuff and even provided URL's. Now, I am not really naive enough to think that a large corporate with international shareholders would actually take architectural decisions based on feedback via twitter, but they could have said something along the lines of "KTHNKSBAI you nutjob" or something at least. Other suggestions like creation of "folders" on internet banking were replied to and lauded. This makes me wonder if the person asking the question actually understands the question in the first place! Scary! The second example happened today, and brings me more toward the tagline of this post: web presence. Presence being the keyword. If banks/corporates have a twitter/web/facebook/anything presence, you MUST understand that this is then a 24/7/365 job. If I can use internet banking or an ATM or my phone on a public holiday, I MUST be able to get information that I need from their twitter presence on a public holiday too! It is not a holiday in Uganda or any of the other countries that this bank operates in, and guess what, online nobody cares if it is a holiday or not. When I talk about webservices as a geek, I understand that whatever service I deploy MUST run at all times (at least a 99.99% uptime) and takes no holidays. Corporates in an online space must understand the same. This is not negotiable. Banks are not too cash strapped to enable customers on a 24 hour basis. In SA we pay extortionist rates in banking fees and charges, so I want to see where that money is going. I also pay an internet banking charge, which in my mind is insane, so what is that for? Bandwidth? Why do I pay to send an email notification? I want some service for my money now thanks! I would appreciate it if some banks or bank representatives have some thoughts around this. Please do leave a comment and lets make SA a better place to do business!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Cloud computing There has been a lot of talk/hype around "Cloud computing" over the last year, and, seeing as though this has been something that I have inadvertantly (read I didn't know it was called Cloud Computing) for the past few years, I decided that it was time to say something. So, as far as the situation is in 2009, "Cloud" extends to one of two basic services. 1. Mega (elastic) storage and 2. Mega (elastic) Computing both of which are stunningly simple to deploy using a variety of Free Software tools that have now been around for years and work really well! I am actually really glad that we have started to talk about "Cloud" as it gives a nice marketing spin to some really great concepts. Anyway, in this post, I would like to focus a little on the concept of "Personal Cloud". I have seen loads of stuff on Private and Public clouds, but not on the personal cloud. So, what is it I hear you ask? Well, simply put, think of a private cloud (lets take storage as an example) in your broom cupboard. Sound nuts? Not really if you think about it! So, you have a bunch of desktops, laptops and netbooks on your home LAN. You may use a desktop at home (you know the one that the kids punish during games, the recipe browser and the household finances machine), a laptop for work, and a netbook that you now have to do presentations or when travelling. You may also have kids that require their own resources for homework, video editing and whatever else. The point here is that in most homes, we have a wireless network, or at least an ADSL connection and multiple computers downloading stuff from the internet. This makes for a lot of data, seeing as though the average sizes of hard drives these days are getting bigger and bigger (starting to be in the order of TB). All of this data needs somewhere to live, especially if it is important data (think home video of little Joey's 1st birthday, or your slide collection or the scanned recipes of Great Great Aunt Mildred etc...) you will want it kept in a place that you can access easily (i.e not worry about chewing up an entire ADSL account if your PC dies). Enter the personal storage cloud system. The really cool thing about this type of thing is that you can deploy it (yes, you!) on cheap commodity hardware and Free Software. You can quite easily hook up a PB of storage in your garage in a single 4u server enclosure, which scales with the addition of more enclosures - there you have it, elastic storage on a budget. The same applies for SMME's and other businesses. As soon as you reach a threshold (say 10PB) we can then call it "private cloud" or reach 1 EB and call it "Enterprise cloud" or something, they are just names! The point here is: It is all the same technology, whether 1TB or 100EB. Compue clouds are very similar. What you may need an elastic compute cloud for at home is probably only the domain of geeks, film makers and musicians, but you never know right? I see a future business opportunity to sell preconfigured "Cloud Appliances" for home, SOHO and enterprise. We have all the bits, we just need to put them together, slap on a funky label, make some flyers and go for it!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

HOWTO install a Chisimba site on a serv.co.za hosting plan When using serv, I use the S1 plan (cheapest one), but this will work for all of their plans equally well 1. Do a checkout or download Chisimba from http://avoir.uwc.ac.za 2. GZip and Tar or Zip up the code and upload it to your domain via the cPanel File Manager. 3. Extract the code to your domain/subdomain 4. Click back to your cPanel main interface and click on the PEAR manager (PHP Pear Packages icon) 5. Take note of the path that is then provided (you will need this just now). It will look like /home/yourdomain/php 6. Start installing the necessary PEAR packages: 6.1 MDB2 6.2 MDB2_Driver_MySQL 6.3 MDB2_Driver_MySQLi 6.4 MDB2_Schema-0.8.2 (sometimes you may need to get the required dependencies by attempting to install mdb2_schema-beta first) 6.5 Config 6.6 Log 7. Go back to the cPanel home page by clicking the little house icon on top left. 8. Your uploaded code should all be extracted now and ready to go, but first check that the permissions are correct. All files and folders should be 755! NOTE: It is probably a MUCH better idea to do a chmod -R 755 /path/to/your/downloaded/chisimba/files/ first. That way everything should be peachy on the server already! If all is OK, then go back to the main cPanel page, but DO NOT CLOSE your filemanager, you will need it just now! 9. Next step is to prepare a database for the install to use: 8.1 On the cPanel front page, go to the databases section and click on MySQL Databases icon 8.2 Create a database. Call it whatever you like, I will call mine chisimba 8.3 Click Go Back and add a user to your new database. Create the user and give a password. 8.4 Add the user to the database and assign all rights to the user. 10. You now need to edit 4 files. Go back to your file manager window and edit installer/dbhandlers/dbhandler.inc 11. Add the following line to the top of the file: ini_set("include_path", ini_get("include_path").":/home/servuser/php"); Where the /home/servuser/php part is the bit you got in step 5 12. Do the same for dbhandlermanager.inc in the same directory, index.php in the installer directory and then for engine_class_inc.php in /classes/core/ too. 13. OK, you are now ready to start the installation. Open up another browser window and navigate to your new domain. It will automatically redirect to the installer and start it. 14. Accept the license and work through the wizard. When you get to the PEAR bits, enter the pear path that you saved from step 5. 15. Carry on with the install. Fill in your database details with the username and password you created. Remember that your serv username is appended to the db username, so it will be servuser_username 16. Fill in your database name (will also be appended with the servusername_). Check the create database checkbox. Click next. 17. Complete the rest of the installer and log in to your new install.

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Chisimba installation HOWTO (Video) from one of our community members, David Janks: [YOUTUBE]http://www.youtube.com/watch?v=68t_Jp8DQxk[/YOUTUBE]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

I was thinking a bit about stuff last night, and especially the fact that I am under pressure from a lot of folks to move away from Tweetdeck (which I think is awesome) to a web based client like HootSuite, Seesmic web or any of the other AJAXY things out there (there are now loads to choose from). The bottom line is, yes, they probably do work well, and yes, that is great, but there are very distinct disadvantages to using a browser based client: You need an open browser window, "maximised" to be useful. IMO browser windows take up a LOT of desktop real estate, so not great when you have 6 shells open too, and, and, and... I realise that Tweetdeck does take up quite a lot of resources, although it seems MUCH better in its latest incarnation, but I really like it. Code wise, would you rather write a bunch of JavaScript and PHP or something to do a simple AJAX based Twitter client, or would you rather write a hundred or so lines of python to do the same thing on your desktop? Remembering that in a web environment you have to hack thread-like behaviour, while on desktop threads are a no-brainer to do background, non-blocking updates etc. Basically, what I am saying is that RIA (Rich Internet Applications) rock. They bring the best of both worlds to the user in a controlled and resource happy environment. In a web sense, if all my users were using desktop clients, I would be over the moon, as the only thing I would have to serve is JSON or something similarly lightweight, saving my bandwidth bill and my hardware bill, as I could support way more folks than a heavier interface could! The trick though is to save yourself time too. Write simple, cross platform desktop apps that are easily maintainable and package-able. I chose wxPython to demo with below. You don't need whole whacks of code you just need a minimal interface to hold together some glue code making use of existing packages. The code below is not pretty, but it is terse and to the point. If you would like to improve the code, please feel free, but please try keep it less than 100 lines... So can you write a simple, but useful Twitter client in less than 100 lines? Here is one in 98, including shebang... [code type="python"] #!/usr/bin/env python import wx import twitter import time import sys class RedirectText(object): def __init__(self,aWxTextCtrl): self.out=aWxTextCtrl def write(self,string): self.out.WriteText(string) class MyFrame(wx.Frame): def __init__(self, *args, **kwds): kwds["style"] = wx.ICONIZE|wx.CAPTION|wx.MINIMIZE|wx.CLOSE_BOX|wx.MINIMIZE_BOX|wx.MAXIMIZE_BOX|wx.SYSTEM_MENU|wx.RESIZE_BORDER|wx.CLIP_CHILDREN wx.Frame.__init__(self, *args, **kwds) self.textUpdate = wx.TextCtrl(self, -1, "", style=wx.TE_PROCESS_ENTER|wx.TE_MULTILINE|wx.TE_AUTO_URL|wx.TE_LINEWRAP|wx.TE_WORDWRAP) self.updateButton = wx.Button(self, -1, "Update", style=wx.BU_EXACTFIT) self.panel = wx.Panel(self, wx.ID_ANY) self.log = wx.TextCtrl(self.panel, wx.ID_ANY, "", style=wx.TE_MULTILINE|wx.TE_WORDWRAP) #self.log.Disable() self.__set_properties() self.__do_layout() self.Bind(wx.EVT_TEXT_ENTER, self.onUpdate, self.textUpdate) self.Bind(wx.EVT_BUTTON, self.onUpdate, self.updateButton) def __set_properties(self): self.username = 'paulscott56' self.password = 'password123' self.encoding = None self.SetTitle("Twitrrr") self.SetSize(wx.DLG_SZE(self, (205, 234))) self.textUpdate.SetFocus() self.fetchTwitter(self) def __do_layout(self): sizer_1 = wx.BoxSizer(wx.VERTICAL) sizer_2 = wx.BoxSizer(wx.VERTICAL) sizer_3 = wx.BoxSizer(wx.HORIZONTAL) sizer_2.Add(self.textUpdate, 1, wx.ALL|wx.EXPAND, 4) sizer_2.Add(self.updateButton, 0, wx.ALL|wx.EXPAND, 4) sizer_3.Add(self.log, 4, wx.EXPAND|wx.ALIGN_CENTER_HORIZONTAL|wx.ALIGN_CENTER_VERTICAL, 4) self.panel.SetSizer(sizer_3) sizer_2.Add(self.panel, 1, wx.EXPAND, 0) sizer_1.Add(sizer_2, 1, wx.EXPAND, 0) self.SetSizer(sizer_1) self.Layout() self.Centre() def onUpdate(self, event): self.api = twitter.Api(username=self.username, password=self.password, input_encoding=self.encoding) message = self.textUpdate.GetValue() try: status = self.api.PostUpdate(message) self.textUpdate.Clear() except UnicodeDecodeError: print "Your message could not be encoded. Perhaps it contains non-ASCII characters? " self.textUpdate.Clear() print "%s just posted: %s" % (status.user.name, status.text) event.Skip() def fetchTwitter(self, event): self.api = twitter.Api(username=self.username, password=self.password, input_encoding=self.encoding) statuses = self.api.GetFriendsTimeline(user=self.username) self.log.Clear() statuses.reverse() for s in statuses: # make the text thing sn = s.user.screen_name txt = s.text time = s.relative_created_at stringthing = sn+": "+txt+" "+time+'\r\n' # redirect text redir=RedirectText(self.log) sys.stdout=redir print stringthing # make a timer here timeLapse = 60000 self.timer = wx.Timer(self) self.timer.Start(timeLapse) self.Bind(wx.EVT_TIMER, self.fetchTwitter, self.timer) # end of class MyFrame if __name__ == "__main__": app = wx.PySimpleApp(0) wx.InitAllImageHandlers() Twitrrr = MyFrame(None, -1, "") app.SetTopWindow(Twitrrr) Twitrrr.Show() app.MainLoop() [/code]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

I see that both http://www.brandminus.co.za and http://www.brandplus.co.za have been registered by two different people! I like to take this as an indication of great success of the original site at http://www.brandmonday.biz To the two folks that have done this, why not focus on creating and bolstering awareness of the original site so that it becomes a truly great resource for consumers? Thanks and congratulations to @metameerkat (http://twitter.com/metameerkat), @cazpi (http://twitter.com/cazpi) and the rest of the Brandmonday team for this great initiative, and carry on the great work!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

The new web portal for the University of the Western Cape (UWC) was launched yesterday. This is another #Chisimba application that will be built upon over the coming months to encompass even more cool functionality. [IFRAME]http://www.uwc.ac.za|600|800[/IFRAME] This is another small example of how Chisimba can be used to create dynamic applications that both consume and provide many services (web and otherwise). Chisimba is also interacting with proprietary administration systems in this case, showcasing the fact that even this is possible with this great piece of African born and built software!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

In Africa, network latency with respect to cloud applications is an especially large threat. Tom Croucher (@sh1mmer if you are a Twitter user) outlines this in the following Slideshare presentation: [slideshare id=2542191&doc=web20talk-091119215420-phpapp02] The presentation also outlines in images and fewer words what I have been plotting for the last few weeks, so please do leave a comment if you like!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

Would you like to become a Chisimba ambassador? If yes, read more to find out how! An ambassador is a person that represents the Chisimba project wherever they are, promotes the use of Chisimba as well as the development of Chisimba by talking about the project. This includes presentations at conferences (including references to Chisimba in your presentation), informal discussions, meetings, government fora and others. An ambassador is someone, in other words, that values African development above all else and is passionate about what we do. You are aware of the AVOIR manifesto, the four cornerstones of AVOIR and the AVOIR project in general. You are also aware of how Chisimba fits into the AVOIR project and how to foster participation. A Chisimba ambassador should also have a passion and deep knowledge of Freedom and Free and Open Source Software, and the ideals associated with that. Duties of a Chisimba ambassador are: 1. Write an article or short "story" for http://avoir.uwc.ac.za http://avoir.uwc.ac.za/ periodically (once a week is good!) 2. Speak about Chisimba at every opportunity. 3. Actively participate in the Chisimba family (Mailing lists, Twitter, AVOIR site, Forum etc) 4. Introduce or speak about AVOIR and Chisimba in your own networks. Actively build the community. In return, Chisimba ambassadors will be the first in line to receive benefits like: 1. Trips to conferences (OSCON, PHP Conference, etc) 2. Help with running workshops and other initiatives 3. Potential travel to some of the more interesting places of the world (we promise no tourist traps!) 4. Good, restful sleep on a daily basis, knowing that you are doing the right thing for Africa, humanity, ubuntu (not the OS, the concept) and the world. 5. A premium spot and bio on the AVOIR site and blogging by the Chisimba community to reach worldwide fame. Interested? Please contact me to find out how to get started please simply leave a comment!

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

A few sites that are using the Chisimba framework. If you would like your site to be added to the list, please leave a comment! * African Virtual Open Initiative and Resources (http://avoir.uwc.ac.za) * Angel (http://angel.elearn.org.af/) * #BrandMonday: Tweet about Brands on Mondays (http://www.brandmonday.biz) * Derek Keats (http://www.dkeats.com) * E-Learning at the University of Nairobi (http://elearning.uonbi.ac.ke/) * E-Learning at the University of the Western Cape (http://elearn.uwc.ac.za) * E-Learning at University of Dar es Salaam Computing Center (http://elearn.ucc.co.tz/) * Innovation Symposium (http://innovationsymposium.wits.ac.za/) * Institute for Human Evolution (http://kim.wits.ac.za/wip/ihe/) * This site (http://www.paulscott.za.net) * SANORD (http://sanord.uwc.ac.za/) * University of the Western Cape (http://www.uwc.ac.za) * Wits: Knowledge and Information Management (http://kim.wits.ac.za) * Wits Presentations Repository (http://presentations.wits.ac.za) * Moi University Presentations Repository (http://41.204.167.5/webpresent/) * Moi University eLearning Portal (http://41.204.167.5/chisimba/) * Chisimba Real time Twitter Aggregator (http://tweetgator.peeps.co.za)

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

My most popular image of the last 24 hours. Not a particularly great pic, but OK... [img]http://farm3.static.flickr.com/2426/4102658717_10f3aa96d0_m.jpg[/img]

pscott209@gmail.com

admin — Thu, 01 Jan 1970 00:00:00 +0200

I recently got a new phone. A shiny new Nokia 6303 Classic. Normally, I see "Classic" and it is an old phone wrapped in a new shell, but they said it did 3G/Edge and that is all I was really looking for. So I unboxed the thing and set everything up as a phone. Next step was to use it in its primary role, that of a data modem for my travels. The functionality described above was advertised as "out the box" and "Hell yeah, it works!" so I thought nothing of it until now. I plugged the data cable into the phone (which was really hard btw, it's almost impossible to open the little USB flap thing with fingers larger than a small child's) and the other end into my laptop's USB slot. Nothing. The phone didn't peep, spawn off some magical configuration wizard, or connect to the internet. The laptop did about the same. Hmm, I hear you say, hmm... Plan B. I read the manual. You are REQUIRED to use something called the Nokia PC Suite on your Windows box to do anything with this device (and I presume most other Nokia devices). This software does not exist for linux and is not supported by Nokia. Wow! OK, so Nokia, although I have now handed over a large proportion of my salary to them over the years, refuse to care about me and the other 4 million odd users out there. Grrr! Right, time to doff the user hat and don the hacker one it seems... OK so the first thing was to see what was happening on the USB port. I fired up a terminal and banged in: [code]lsusb[/code] The response should include something along the lines of: [code]Bus 002 Device 004: ID 0421:01b0 Nokia Mobile Phones[/code] Which is the stupid phone. Right, so the trick with USB devices is that they have a vendor ID and a product ID, both shown above. Once you have those two bits of info, you are able to probe for the device and set it up: [code]sudo /sbin/modprobe usbserial vendor=0x0421 product=0x01b0[/code] You will notice that the vendor and product ID's are as above, but with a 0x added to it. Do this for your device too. It will probably be different, so make sure you don't just copy and paste the line! You now need to create a wvdial config: [code]wvdialconf create[/code] You should see quite a bit of output while wvdial checks and configures your phone as a modem: [code] Editing `create'. Scanning your serial ports for a modem. Modem Port Scan*1: S0 S1 S2 S3 WvModem*1: Cannot get information for serial port. ttyACM0*1: ATQ0 V1 E1 -- OK ttyACM0*1: ATQ0 V1 E1 Z -- OK ttyACM0*1: ATQ0 V1 E1 S0=0 -- OK ttyACM0*1: ATQ0 V1 E1 S0=0 C1 -- OK ttyACM0*1: ATQ0 V1 E1 S0=0 C1 D2 -- OK ttyACM0*1: ATQ0 V1 E1 S0=0 C1 D2 +FCLASS=0 -- OK ttyACM0*1: Modem Identifier: ATI -- Nokia ttyACM0*1: Speed 4800: AT -- OK ttyACM0*1: Speed 9600: AT -- OK ttyACM0*1: Speed 19200: AT -- OK ttyACM0*1: Speed 38400: AT -- OK ttyACM0*1: Speed 57600: AT -- OK ttyACM0*1: Speed 115200: AT -- OK ttyACM0*1: Speed 230400: AT -- OK ttyACM0*1: Speed 460800: AT -- OK ttyACM0*1: Max speed is 460800; that should be safe. ttyACM0*1: ATQ0 V1 E1 S0=0 C1 D2 +FCLASS=0 -- OK WvModem*1: Cannot get information for serial port. ttyUSB0*1: ATQ0 V1 E1 -- failed with 2400 baud, next try: 9600 baud ttyUSB0*1: ATQ0 V1 E1 -- failed with 9600 baud, next try: 9600 baud ttyUSB0*1: ATQ0 V1 E1 -- and failed too at 115200, giving up. Found an USB modem on /dev/ttyACM0. createWarn: Can't open 'create' for reading: No such file or directory createWarn: ...starting with blank configuration. Modem configuration written to create. ttyACM0Info: Speed 460800; init "ATQ0 V1 E1 S0=0 C1 D2 +FCLASS=0" [/code] You now have a modem connected to your laptop! W00t! But wait, we need to configure it: [code]sudo gedit /etc/wvdial.conf[/code] OK so we need to edit the default wvdial generated config to something more useful. Copy and paste the following, changing if needs be: [code] [Dialer Defaults] Modem = /dev/ttyACM0 Baud = 460800 Init1 = ATZ Init2 = ATQ0 V1 E1 S0=0 C1 D2 +FCLASS=0 ISDN = 0 Modem Type = Analog Modem Phone = *99# Username = username Password = password Stupid Mode = 1 [/code] Once all that is done, you can type the following to connect to the internet (assuming the mobile network is useful): [code]wvdial[/code] It will chug away a while, and the phone will say something along the lines of "Packet Data connection established" and you will be online. Check that you get an IP address and a DNS address to use with your connection, or else try again. To disconnect, simply issue a: [code]ctrl-c[/code] to break the connection. You may then want to go and create a desktop launcher or something, but that is up to you. To end off, I am really disappointed that this was hard and that I had to think further than "Plug it in". Hope this is useful for someone else too though!