Pyrex and libyahoo2 (or not)
I had something I wanted to try in Python running against Yahoo! Messenger. The obvious choice of library for talking to Yahoo! Messenger was libyahoo2. I could not find a Python binding for it, so I started sketching one together with SWIG. The first step is creating a bunch of empty callbacks. The libyahoo2 in Ubuntu’s package is compiled without USE_CALLBACK_STRUCT, so libyahoo2 expects to find a bunch of extern functions defined to interact with the host. I made empty callbacks in a C file and started reading more about the API.
It rapidly became clear that I was going to want a layer on top of
the library to make interacting with it from Python more
palatable. I switched to Pyrex, since I wanted to write that
wrapper in Python (or something Python-like) rather than building a
straight-C wrapper so using SWIG would continue to make sense.
SWIG’s big benefit in my mind over Pyrex is easy support for more languages and better tools for defining straight wrappers. I wasn’t going to get “free” use in other languages and it wasn’t going to be a straight wrapper now but rather a module that exposed the functionality of libyahoo2 to Python.
I kept the callbacks.c I had defined but started migrating the definitions to the Pyrex file as I implemented. This way my library would continue to link without complaint about functions I didn’t have yet.
Following the usual pattern for Python bindings to libraries that need wrappers to be more Pythonic, I planned to have a ‘yahoo2’ module in Python and a ‘_yahoo2.so’ extension module. The _yahoo2 module is written in Pyrex.
libyahoo2 appears not to adhere to the documentation it defines. It’ll call ext_yahoo_remove_handler with a tag that was never returned by ext_yahoo_add_handler… (0). It looks like an undocumented (that I found) part of the charter of ext_yahoo_add_handler is not to add a given handler more than once.
This also made defining the callbacks the way libyahoo2 expected easier.
D’oh, I got it far enough along to get this:
libyahoo2.c:620: debug: Key: 4 Value: Yahoo_Messenger libyahoo2.c:620: debug: Key: 5 Value: hodorbot libyahoo2.c:620: debug: Key: 14 Value: This version of Messenger expired on April 2, 2008. Please upgrade now to the latest supported version: http://messenger.yahoo.com Learn more: http://messenger.yahoo.com/eol libyahoo2.c:620: debug: Key: 15 Value: 1225167367 libyahoo2.c:620: debug: Key: 97 Value: 1
I’ve learned what I wanted to, so instead of seeing if a more recent libyahoo2 than what’s in Ubuntu works (> 0.7.5+dfsg-3), I’m just going to call it here. I’ll post it in case someone can learn something useful from it.
You can pull a working repository with a command like:
git clone git://github.com/xythian/python-yahoo2.git
Update: Moved this to github from where it was.
There’s three files that do anything:
- yahoo2.py - wraps some of the lower level details from the Pyrex layer, including exposing the IO bits as an asyncore dispatcher
- _yahoo2.pyx - is the binding
- callbacks.c - exists to have empty functions defined to satisfy the linker until those have implementations in the Python binding
Not much works, really; there’s implementations of connect and async_connect, but only async_connect is called by libyahoo2 before it gets far enough along to not log in with the error message above. There’s a wrapper for setting the log level (which was key to discovering the above fact…).
This is a typical definition of one of the library callbacks:
cdef public int ext_yahoo_connect_async(int id, char *host, int port, \ yahoo_connect_callback callback, void *callback_data): cdef ConnectionHandle handle handle = ConnectionHandle(id, host, port) handle.connect_callback = callback handle.connect_data = callback_data handle.async_connect(callback, callback_data) HANDLER_MAP[id].connections.append(handle) MANAGER.add(handle) return handle.fileno()