Reader Macros: A cool feature, frustration waiting to happen

One of the neat features in Common Lisp is that any program can register a new reader macro that will read from the input stream when it sees some character. A good example of where this is useful and well implemented is Edi Weitz’s :cl-interpol.  This library allows you to, for any given file, declare that you want to read phrases that begin with #? as a interpolated.

For example:
#?"${name} is coming over for ${event}"
#?/\d{3}(-|\.|\s)\d{3}(-|\.|\s)\d{4}/ ;creates a regular expression

This can be a wonderful time saver and can be quite a bit easier on the eyes than a serious format.  However, these reader syntaxes can be good and bad.  Many library authors include them by default and they are enabled when the package is loaded.  The reader syntaxes are not bound by packages (because the readtable is global).  This has the downside of your code randomly ceasing to work based on some vagary of library load order and which reader syntax you expect to be enabled.

I had problems when loading LISA into an application for the first time because it stomped my :cl-interpol file-level reader declaration and kept stomping it.  No amount of reevalution seemed to provide reliable results as to which of the two syntaxes were enabled.  I finally found where in the LISA config to disable that syntax, which allowed me to work around the conflict.

I had a similar experience when trying to use :cl-mediawiki in a project for the first time.  CL-MediaWiki used to use of the :cl-unification #T reader macro which enabled a template engine to ease unification.   (Now it uses the make-template function).  When I added cl-mediawiki to my projects dependancy list and started to use it, it failed utterly.  The reason? Apparently Closure-HTML has a reader syntax defined and enabled for #T.  Both of these libraries add this syntax as part of their standard load.  This leaves the end user completely unable to use your library (or forcing them to patch it) if it stomps on existing reader-syntaxes that they have enabled.

My suggesstion would be to follow Edi’s example and allow the end user to enable a syntax on a perfile level (at least) and have the syntaxes disabled by default.  This allows the user to decide when and where they use your handy syntax without frustrating them along the way. Also as best I can tell, following Edi’s example is always safe and good advise.

CL-MediaWiki in Action

As mentioned in a previous post, we heavily utilize a MediaWiki instance in to manage a lot of small bits of data.  Last night I worked :cl-mediawiki into our internal billing system.  For many months it has been a manual process to create a new page and category for a client in our billing system.  We had a report that would give you a link and some default wiki text to type in.  With the desire to bring more notes into the wiki, and the advent of the edit-api available in MediaWiki 1.13, I set about writing cl-mediawiki to allow us to more directly integrate.

Now when a new client is entered into our billing system, we call out to the wiki to automatically create a wikipage and category for that client.  When a new order gets added to our system, we will add a new order page.  When that order is attached to an account or moved to a new account, we will replace the account number on their wiki page with the new account number.  When the order gets canceled, we can add the cancelation notes directly to the wiki.

CL-MediaWiki now also requires :cl-ppcre.  The upshot of this is a regex-replace-all function that will replace the content of a wikipage (with an optional value to set the page to if it doesnt exist).  This was used to keep the account number up to date on the order page.

link to the paste – Scroll down to see all the examples mentioned

Announcing CL-MediaWiki: A Common Lisp interface to the MediaWiki API

UPDATE: Changed hosting information at the bottom

We use a MediaWiki instance at work to manage a tremendous amount of random facts.  We have it setup with Sphinx FTI to get good search results and have found the wiki to be a huge help in managing our data.  In fact, via a bit of reverse proxy magic, some of the notes fields in our internal billing application are chunks of a wiki page for that client with links to all of that client’s other attached wiki pages

Media Wiki Notes Field Screen Shot

Media Wiki Notes Field Screen Shot

We wanted the ability to interact with the wiki in a more programatic manner (such as making cancelation notes), so I started checking into the media wiki api.  It gives you the ability to do almost anything you would want to do with the Media Wiki engine programatically (except I cant figure out how to get it to render wiki markup for me).  I wanted a wrapper around this in Common Lisp which could take care of formulating the messages to the wiki engine and determining whether or not we succeeded preferably leaving these as just function calls to the rest of application.  Because I could find no mention of a library like this elsewhere, I decided to go ahead and whip one up.

Using :cxml :drakma, & :cl-unification, I have written a very incomplete wrapper around media wiki remote api.  The basic call cycle so far is as follows:

  • Build a list of parameters / values based on the api documentation
  • Feed these into Drakma to make the request of the media wiki api
  • Use CXML to parse the returned XML string into an XML-S tree
  • Use CL-Unification to match the response XML and extract data for return (when successful) and error conditions which we signal (if it is not).