A month ago there would have been no reason to write this post, because Google Reader made it’s own RSS feed of the posts you wanted to share. See, Google wants to drive people to use Google+, and they seem to be doing this by crippling their other services that have even a smidgen of social usefulness. The thing is, I liked the social bits of Google Reader. Sure, they were a bit of an afterthought, and mildly dysfunctional, but I got a lot of value from reading what other people shared, and I liked that when I shared something I was creating a kind of archive of stuff I liked on Reader, with an RSS feed of its very own.

The great thing about RSS is that it can be consumed by arbitrary 3rd party applications. The likes of Google, Facebook and Twitter don’t like this, because they want control over the 3rd parties that can access their stream. So they are quite prepared to kill off RSS services in their applications, because it does not serve their mission. You can no longer get an RSS feed of your Twitter stream (as far as I know), there is no RSS built into Google+ (which I have yet to have the time to fully grok).

My needs are small, I want an RSS feed of the stuff I want to share from Google Reader, so that other people can follow the things I share in Reader (if they want), and I can pipe that information elsewhere (I use dlvr.it to post selected RSS feeds into Twitter). Google doesn’t want to provide that anymore, so I’ll hack something together.

The ingredients:

1. These simple instructions for how to render an RSS feed from a MySQL backend.
2. The instructions for how to create your own “Send to:” item in Google Reader
3. My rudimentary PHP hackery skills

The code:
All source is available on BitBucket.

First, we need a database connection. The database is set up exactly as described in (1), above.

<?php
DEFINE('DB_USER', 'db_user');
DEFINE('DB_PASSWORD', 'db_password');
DEFINE('DB_HOST', 'localhost');
DEFINE('DB_NAME', 'db_name');
// Make the connnection and then select the database.
$dbc = @mysql_connect(DB_HOST, DB_USER, DB_PASSWORD) OR die(mysql_error());
mysql_select_db(DB_NAME) OR die(mysql_error());
?>

Now, when the page is visited, we want to render what is in the database as an RSS feed (again, this is a simple adaptation of the code in (1)):

<?php
  class RSS {
        public function RSS() {
                require_once ('mysql_connect.php');
        }
        public function GetFeed() {
                return $this->getDetails() . $this->getItems();
        }
        private function dbConnect() {
                DEFINE('LINK', mysql_connect(DB_HOST, DB_USER, DB_PASSWORD));
        }
        private function getDetails() {
                //header of the RSS feed
                $detailsTable = "webref_rss_details";
                $this->dbConnect($detailsTable);
                $query = "SELECT * FROM ". $detailsTable;
                $result = mysql_db_query (DB_NAME, $query, LINK);
                while($row = mysql_fetch_array($result)) {
                        //fairly minimal description of the feed
                        $details = '<?xml version="1.0" encoding="ISO-8859-1" ?>
                                <rss version="2.0">
                                        <channel>
                                                <title>'. $row['title'] .'</title>
                                                <link>'. $row['link'] .'</link>
                                                <description>'. $row['description'] .'</description>
                                                <language>'. $row['language'] .'</language>
                                                ';
                }
                return $details;
        }

        private function getItems() {
                //return all the items foe the RSS feed
                $itemsTable = "webref_rss_items";
                $this->dbConnect($itemsTable);
                $query = "SELECT * FROM ". $itemsTable;
                $result = mysql_db_query(DB_NAME, $query, LINK);
                $items = '';
                while($row = mysql_fetch_array($result)) {
                        $items .= '<item>
                                <title>'. $row["title"] .'</title>
                                <link>'. $row["link"] .'</link>
                                <description><![CDATA['. $row["description"] .']]></description>
                        </item>';
                }
                //close the feed
                $items .= '</channel>
                                </rss>';
                return $items;
        }
}
?>

Finally, we need a method for adding new stuff for the feed. This code takes the GET variables passed to it by Google Reader, and stores them in the DB:

<?php
if ($_GET['url']) {
        //receive google reader 'send to' items, and store in mysqldb
        $url = $_GET['url'];
        $source = $_GET['source'];
        $title = $_GET['title'];
        $simple_check = $_GET['check'];
        //stops anyone adding new items to your feed unless they have the key
        if ($simple_check == 'uniquepasscodehere') {
                //statement adds new item to RSS database
                $insert_statement = "INSERT INTO webref_rss_items(title, description, link) VALUES('$title', '$source', '$url')";
                require_once('mysql_connect.php');
                $result = mysql_query($insert_statement, $dbc);
                if ($result) {
                        echo "<p>Success!";
                        //would be nice to close the window automatically after a couple of seconds
                }
                else {
                        die('<p>Invalid query: ' . mysql_error());
                }
        }
}
else {
        //render everything in the db as RSS
        header("Content-Type: application/xml; charset=ISO-8859-1");
        include("RSS.class.php");
        $rss = new RSS();
        echo $rss->GetFeed();
}
?>

Now, I can set up the Send To: item in Google Reader:

Finally, click ‘Send To: -> Readershare’ in the footer of an item in Google Reader, and it is rendered into my RSS feed, which can then be consumed by other applications, including Google Reader itself (so if you want to subscribe to my Google Reader shared items feed, you can find it at http://fuzzierlogic.com/readershare). Oh, and I can pipe my Google Reader shares back into Twitter again.

An announcement courtesy of Colin Gillespie, a lecturer in Maths & Stats here in Newcastle:

The School of Mathematics & Statistics at Newcastle University, are
again running some R courses. In January, 2012, we will run:

• January 16th: Introduction to R;
• January 17th: Programming with R;
• January 18th & 19th: Advanced graphics with R.

The courses aren’t aimed at teaching statistics, rather they aim to go through the fundemental concepts of R programming.

Further information is available at the course website.

It is hard not to get carried away in a room full of people who seem mostly to want the same things. You come away from a conference like Science Online thinking that the open science revolution is inevitable, and there is nothing anyone can do to stop it. Then you get back to your day job and talk of REF and impact factors and get bought back to earth with a bump.

The take home message of the conference this year seemed to be this: for open science to work, long term, reward mechanisms within the profession have to change in a comprehensive and profound way. Do I think this is possible? Of course. Do I think this is inevitable? Not by a long chalk. There are too many parties with a vested interest in things remaining the same, some of whom were represented here, despite all of the talk being about openness.

NPG certainly don’t seem that interested in opening things up too far, as the breakout session on APIs demonstrated. Nothing outside of their paywall was discussed, even more broadly applicable tools, like Connotea, seem to be quietly dropped in the background. The research councils are still more interested in “impact” (whatever that means) than genuinely original thinking.

But for all this pessimism, there are interesting things happening, and a mainstream breakthrough becomes more likely as the volume of those agitating for change grows. MaryAnn Martone‘s keynote was genuinely inspiring, a clear case for breaking down the garden walls. Michael Nielsen made a compelling case for wholesale revolution (however unlikely I think this sort of change may be). We showed that in an afternoon, you can set up a collaborative blog and populate it with interesting scientific content, using freely available tools. The interest we always encounter for the Knowledgeblog project enthuses me, and encourages me that something similar will make hay someday soon (even if we don’t manage to be the people who make the breakthrough).

It may be difficult for me to get to SoLo12, but I will try very hard to return, because I always leave with a smile on my face.

Writing papers. It’s a pain, right? Journals are finicky about formatting. You write the content and then the journal wants you to make it look right. You finally get the content in the right shape and then they tell you that you’ve formatted the bibliography wrong. Your bibliography is clearly in Harvard format, when the journal only accepts papers where the bibliography is formatted Chicago style. Another hour or two of spitting and cursing as you try to massage the citations and bibliography into the “correct” format. You’re not even allowed to cite everything you want to, because the internet is clearly so untrusted a resource.

I’m of the opinion that publishing should be lightweight, the publishers should get out of the way of the author’s process, not actively get in the way. Working on the Knowledgeblog project has only reinforced this opinion. Why should I spend days formatting the content, when any web content management system (CMS) worth its salt will take raw content and format it in a consistent way? Why should I process all the citations and format the bibliography when it should be (relatively) simple to do this in software? Why should I spend time producing complicated figures that compromise what I am able to show when data+code would give the reader far more power to visualise my results themselves?

This document is written in Word 2007 on a Windows 7 virtual machine. On this virtual machine I have also installed Standalone Zotero. The final piece of this particular jigsaw is a Citation Style Language (CSL) style document I wrote (you can download it from the Knowledgeblog Google Code site) that formats a citation in such a way that KCite, Knowledgeblog’s citation engine, can understand it. Now, when I insert citations into my Word document via the Zotero Add-In, I can pick the “KCite” style from the list, and the citation is popped into my document. Now when I hit “Publish” in Word, the document is pushed to my blog, KCite sees the citation as added by Zotero, and processes it, producing a nicely formatted bibliography. We are working on the citeproc-js implementation that means the reader can format this bibliography any way they choose (Phil has a working prototype of this). The biggest current limitation is that your Zotero library entry must have a DOI in it for everything to join up.

So, here is a paragraph with some (contextually meaningless) citations in it (http://dx.doi.org/10.1006/jmbi.1990.9999) . All citations have been added into the Word doc via Zotero, and processed in the page you’re viewing by KCite (http://dx.doi.org/10.1073/pnas.0400782101) . Adding a reference into the document from your Zotero library takes 3-4 clicks, no further processing is needed (http://dx.doi.org/10.1093/bioinformatics/btr134) .

Other popular reference management tools, such as Mendeley and Papers, also use CSL styles to format citations and bibliographies, so this same style could be employed to enable KCite referencing with those tools as well. This opens up a wide range of possible tool chains for effective blogging. Mendeley + OpenOffice on Ubuntu. Papers + TextMate on OS X (Papers can be used to insert citations into more than just office suite documents, more on that in a later post). The possibilities are broad (but not endless, not yet anyway). Hopefully this means many people’s existing authoring toolchain is already fully supported by Knowledgeblog.

Image credit: http://www.flickr.com/photos/sybrenstuvel/2468506922/ (Sybren Stüvel on Flickr)

Bibliography

Over the weekend I saw this tweet from Stack Overflow/Exchange founder Joel Spolsky. The content of the link he posted has served to crystallise some of my thinking of the last couple of weeks with relation to the Bioinformatics question and answer site BioStar.

The link Spolsky posted in the tweet was to a failed Stack Exchange proposal, and I found the page interesting not for the proposal, or the fact that it failed, but the clearly enumerated reasons for why it failed. Here’s a screenshot:

To clarify the procedure here, new Stack Exchange sites are proposed by a community of users. That community was originally drawn from Stack Overflow, the extremely successful programming Q&A site, but now that there are nearly 50 active sites, the available community of proposers is much larger. Newly proposed sites have to overcome a series of hurdles before they go live, from proposal, through commitment, to a private beta, a public beta, before finally becoming a fully-fledged SE site. At the end of each of these stages, sites are assessed for the likelihood that they will become a healthy and active site. Crucially, this assessment appears to not be an individual process. It is obviously the view of the SE powers-that-be that all Q&A sites are created equal, and what works for one will work for all of them. What is worrying about this attitude is that sites that are genuinely niche and likely to have a small, but active and dedicated, community will be left by the wayside, since presumably they will be unable to generate the kind of ad-revenue that Spolsky at al are going to require to repay their investors.

BioStar is a web community reaching a crossroads. The site is running on the now-free, but inevitably unsupported Stack Exchange 1.0 platform (the process discussed above is for the SE 2.0 community). To continue to thrive, I firmly believe the site needs to move on from this platform, since it is almost certainly going to be closed down from under it within the next 12-18 months. This presents the site owners (and us, the community) with a choice.

1. Migrate the site to SE 2.0
2. Change to an open-source alternative Q&A platform
3. Roll-our-own site, with the functionality we require

I will start by ruling out option 3. Bioinformatics teaches us the perils of reinventing the wheel when it is not necessary. An effort to write a custom-built platform for BioStar would be almost entirely redundant, undertaken on the free time of the community (free-time which could be better spent answering questions on BioStar), and almost certainly offer no tangible benefit over using one of the already available Q&A engines. (Think Facebook-for-Scientists…)

I used to be firmly in the camp supporting option 1. I genuinely love Stack Overflow. I have found great utility in some of the Stack Exchange family of sites. However, the attitude betrayed in both Spolsky’s tweet and the closure notice on the Atheism Stack Exchange site makes me think that BioStar would be left out in the cold if we attempted this migration. Let’s look at how BioStar measures up to these numbers:

• Questions per day (SE 2.0 recommends – “15 questions per day on average is a healthy beta”)
  • Since 30th September 2009 BioStar has received 1,681 questions – that’s 3.13 questions per day
• Percentage answered (SE 2.0 – “90% answered is a healthy beta”)
  • BioStar does well here. There are currently 47 questions with no upvoted answers – about 2.8%
• User group (SE 2.0 – 150 users with 200+, 10 with 2,000+, 5 with 3,000+)
  • We have 14 users with 3,000+, 24 with 2,000+ and (by my count) 142 with 200+. But BioStar has been going for 18 months, the atheism SE site was shut down after 2 months in public beta
• Answer ratio (SE 2.0 – “2.5 answers per question is good”)
  • I don’t have easy access to precise numbers for this, but it’s around 3 answers per question on BioStar
• Visits per day (SE 2.0 – “1,500 visits per day is good, 500 visits per day is worrying.”)
  • I have no stats at all for this, but I’m willing to put good money on the fact that daily numbers are much closer to 500 than 1,500.

By these criteria, and judging by the Atheism Stack Exchange linked to by Spolsky, BioStar would fail to emerge from SE 2.0 beta, based on current numbers, and any effort the existing community put in to get it that far would be wasted. And I don’t think the audience of the site would be grown dramatically by it being a Stack Exchange 2.0 site. I think we have to accept that Bioinformatics is a niche subject with a relatively small potential audience, one that is not going to be especially interesting to a commercially driven exercise (such as Stack Exchange necessarily has to be).

So that leaves us with migration to an OSS alternative as the only remaining option. There are a number of platforms available, some of which offer an experience extremely close to ‘real’ Stack Exchange. I would pick one of these that allows an existing SE XML dump to be imported, and migrate the site as soon as possible, certainly within the next 6 months. There is no question that the change over will be painful, and will probably cost the site a few users, and some traffic in the first instance (the biostar.stackexchange.com URL will have to go, for example), but I am confident in the community that has been built around the site – it will survive, and will be all the stronger for the change.

Besides, if we look at the facts in the cold, hard light of day, we really have no choice.