Symphony as a data preservation utility

18^th February 2009, 3 comments, tagged with SymphonyWith the recent disappearance of Magnolia there has been renewed talk of archiving data from third party sites into a structured repository. Symphony might hold the answer.

Reading Jeremy Keith’s ongoing ideas about data storage, and Paul Mison’s subsequent thoughts:

Now I’m wondering: is there a space for a piece of user-installable software, like Movable Type or Wordpress, that aggregates their data from sites across the web, and then presents it as a site? If there is, is it even possible to write it in a way that anyone who couldn’t have written it themselves can even use it? Can I write it just for myself in the first place?

Movable Type, maybe. Wordpress, certainly not. Expression Engine, close. I’m tempted into thinking that Symphony could well solve this conundrum. As I have posted before, it is a relatively trivial task to set up a Symphony build to cache and thereby archive data returned from an XML feed. Symphony stores content in a completely abstracted manner. One can recreate the data structure of any third party site, both fields and relationships, and easily populate the model with a single HTTP POST.

Almost all sites we use regularly provide an open access API (Facebook excluded). There is massive scope to write an Extension for Symphony that will interpret incoming XML (whether it be Atom, RSS, SOAP, custom API responses… anything) and map them onto Symphony sections. In fact this functionality has been requested several times.

Here’s a very brief explanation of how we already do this on the website for Battlefront.

The benefit of caching this data into Symphony is that you then have the ability to manipulate it in powerful ways. Symphony can query, sort and merge the data and output plain XML. The views are written in XSLT, so you can transform the data into any format you wish: a merged RSS feed, a customised XML schema, HTML, JSON, CSV or even SVG or PDF if you fancy.

A Symphony Extension would cover all of your data — Twitter, Last.fm, Flickr, YouTube, Vimeo, Dopplr. Easy to configure, you can sleep easy knowing that as you slumber your data is periodically archived into your own normalised relational database.

I urge the Archive Team to consider this as a viable option to backing up personal data strewn across the web. The building blocks are already there.