Paul Scott on 2009-05-26 07:21:43 Comments (2) Tweet this

YQL and XMPP

I created a quick XMPP (Jabber) service running through the #Chisimba #tribe module last night that basically creates an interface for folks to use Yahoo!'s most excellent YQL or Yahoo Query Language through an XMPP connection and client. This was basically done so that multiple #Chisimba sites (and others I guess, but I don't really care for them too much) could use the "service" to query the Yahoo! tables and user data.

The basic usage is covered well in the excellent Yahoo! docs, so I won't get too much into that right now, but for the *completely* uninitiated, YQL is basically like SQL for the web! Yahoo have made a bunch of data "tables" available, so that you can do crazy stuff like SELECT * FROM rss WHERE url="http://someurl.net/"

This is really great, but the cumbersome signup process would be a severely limiting factor in Africa, as well as the fact that a lot of these types of services are firewalled to death to "save bandwidth" while XMPP very rarely is firewalled off...

With the above in mind, I decided that it would be useful for sites to get site data from the service via quick and easy XMPP messages. It is trivial to set up an XMPP client programatically in #Chisimba, making use of the awesome XMPPHP library.

In early tests, I realised that it would be stupid to try and return a PHP class as an XMPP message string, so went the following route instead:

1. message is received from the XMPP client
2. Query is executed against YQL
3. PHP Object is returned
4. Object is serialized
5. Serialized object is base64 encoded to a string
6. XMPP message containing the string is sent back to client

I had previously done the code to connect and auth to Yahoo in the #Chisimba #yapi module, so that part was easy. All I had to do was to pull in the Yahoo API code with:


$this-objYapiOps = $this-getObject('yapiops', 'yapi');


This then set up the classes and authourised me against the Yahoo oAuth container using the keys that I had entered into my #Chisimba #dbsysconfig module for the yapi module.
Now that I have an auth token and the yapi code at my fingertips, all I had to do was to get the message to the API and back. I decided to use the keyword "yql:" for YQL queries, as the XMPP message handler in this particular module provides many other services, such as a dictionary lookup and Wikipedia searches as well. It was relatively easy to parse the yql: keyword out and get the raw query from the message string, which I then sent to the yapi module via the query function:


$ret = $this-objYapiOps-executeYQL($query);


$ret or the return is a PHP stdClass in all cases.

The next steps were to then prepare the class ($ret) to be transmitted back over XMPP and used with serialize and base64_encode()


$message = base64_encode( serialize($ret) );
// now send the message back to client via the payload 'from' key
$this-conn-message($pl['from'], $message);


OK, so to put this all in perspective, lets do a simple example using the Yahoo spelling checker through YQL:
The raw YQL query is: select * from search.spelling where query="sekratari"
The string that we enter into my XMPP client is: yql: select * from search.spelling where query="sekratary"
I send the message to peepscoza@gmail.com via my favourite XMPP client pidgin.
I get a response of: Tzo4OiJzdGRDbGFzcyI6MTp7czoxMDoic3VnZ2VzdGlvbiI7czo5OiJzZWNyZXRhcnkiO30=
Which is the base64_encoded text of the serialized PHP object.
Bearing in mind that this should all be done programatically, we will assume that this is a script running the query, and not a person!
The script would then be required to do the following to access the data:


$class = unserialize(base64_decode($str));


The PHP class is then useful. You can access properties etc as you would with any other object! The class looks like the following:

object(stdClass)[1]
public 'suggestion' = string 'secretary' (length=9)



$word = $class-suggestion;


Simple and effective! Have fun!