BDP RSS Aggregator: read me first

The BDP RSS Aggregator is a plugin for us with the Wordpress publishing platform.

Contents

Usual (but important) warnings

The software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software.

If after using this plugin your computer or server belches blue smoke, eviscerates the blogbase and melts into a puddle, you were warned.

Distributed under GPL. Copyright: Bryan Palmer 2005-07.

Installation

  1. Download the ZIP file. Unpack the ZIP file. Upload the bdprss directory (and its contents) to your plugins directory.

    (Note: if you are upgrading, you should first delete all old versions of the plugin).

  2. Activate the plugin (and the associated widgets plugin) from the WordPress dashboard.

  3. Go to the Manage > RSS Feeds option from the dashboard and add the RSS feeds you want.

  4. Then from the Manage > RSS Feeds option on the dashboard, you must create a list format. The list format will will have an identifier that you will use for the Widget number or as the argument to BDPRSS2::output(listID) in a couple of steps. So write this number down.

    If you are using widgets. You will want to structure the list with each feed element being wrapped in list item tags (

  5. ...
    6. ).

  6. Go back to the Manage > RSS Feeds page. Set the polling frequency.

  7. Go back to the Presentation > Widgets page. And put an RSS Aggregator widget in your sidebar.

  8. If you don't have widgets, edit the sidebar.php file from your theme (in /wp-content/themes/your-theme-name) and add code along the following lines.
    <li><h2>RSS Feeds</h2>
    <ul>
    <?php BDPRSS2::output(1); ?>
    <li>About <a href="http://www.ozpolitics.info/blog/2005/03/28/aggregated-blog-feeds/"> the RSS Aggregator</a></li>
    </ul>
    </li>

  9. If you don't have any sidebars, you can do what I do. I use the Exec-PHP plugin to put the output on a page. Simply create a page and put the following code on the page.
    <?php BDPRSS2::output(1); ?>

    <h2> Archive </h2>

    <?php BDPRSS2::archiveList(1); ?>

    About <a href="http://www.ozpolitics.info/blog/2005/03/28/aggregated-blog-feeds/">the RSS Aggregator</a>

    To get it to work you may need to turn off the XHTML correction features in Wordpress. You can turn them on again once everything is working.

  10. From version 0.6.0, you can provide access to the archive of old feeds on a date basis using the following function: BDPRSS2::archiveList(listID, mode, period, format); Where:
    1. listID -- the list identifier
    2. mode -- the archive period is a 'day' or 'month'
    3. period -- the number of archive periods (eg. 7 days)
    4. format -- the PHP date format code (eg. 'd M Y' or 'M Y')

    You can use the BDPRSS2::viewCache(); function on a stand alone page, along with Exec-PHP, to give people access to the cache record for a single item.

    And you can use the BDPRSS2::rssLink(listID, linkText) function to provide a link to an RSS feed for the aggregated feed.

Formatting output

Once you have installed the plugin and selected your feeds you will need to get ready to format your output. In this section we look at some styles you might want to consider.

Recent entries

The first format is recent entries. It looks like this.

The XHTML settings for this follow:

Feeds by site - long version

An example is on this page.

The XHTML settings for this follow:

Feeds by site - short version

An example is on the right hand column of this page

The XHTML settings for this follow:

Timing issues

One of the options is to display the age of the post.

This is tricky stuff. Too many feeds simply do not include the date and time of a post. Some feeds only include the date. In these cases the Aggregator uses the time the feed was polled as the date and time.

Another problem is with date and time formats. Some dates and times are encoded with a standard unix string. Some are encoded with the W3C date and time format. Some use non-standard encoding. The Aggregator uses the PHP function strtotime() if the date was passed in a <pubDate>date field</pubDate>. If the Dublin Core <dc:date>date field</dc:date> is used, the Aggregator decodes it with an adaptation of the Wordpress function parse_w3cdtf().

Quite a number of feeds have the date and time, but express it incorrectly in terms of the GMT offset. If, at the time of polling, a new post has a date/time that is more than 5 minutes into the future or 5 minutes older than the last update time, then the Aggregator uses the poll time. The aggregator will slowly seek to correct the GMT offset for posts over time.

How did you do the feed page

The secret is to use a plugin that allows you to include PHP on a page.

There are a number of such plugins for Wordpress.

I previously used PHP Exec. I now use Exec-PHP.

Change log

Version 0.6.2 - 20 May 2007
Widgetized to work with WordPress version 2.2.

Version 0.6.1 - 17 April 2007
Minor bug fix.

Version 0.6.0 - 17 March 2007
Further minor bug fixes. Should work better with permalink structures.

Warning: If you upgrading from 0.6.0-pr2, you will need to update the database and re-enter your feeds etc. Sorry.

Version 0.6.0 (pre-release 3) - 18 February 2007
Further bug fixes. Should now work with UTF-8 databases. Restructured the classes. Now converts relative addresses for images and links, to absolute addresses.

Changed the archive arrangement so that feed history can be archived by day or month.

Added the capacity to provide an RSS link for a feed list.

Warning: If you upgrading from 0.6.0-pr2, you will need to update the database and re-enter your feeds etc. Sorry.

Version 0.6.0 (test pre-release 2) - 4 February 2007
Minor bug fixes - seems to work fine with Wordpress 2.1. As 0.6.0-pr1 has worked on my box for three weeks I have downgraded the build status from experimental to vanilla alpha.

Remember: If you upgrading from 0.5.3 or earlier, you will need to re-enter your feeds etc.

Version 0.6.0 (test pre-release 1) - 13 January 2007
Major re-write to implement:

Note: this should be considered an experimental build. This version necessitated a complete overhaul of the underlying database. It uses differently named database tables to the old version (so if it fails, reverting to the previous version should not be too difficult). If you are upgrading, you will need to re-enter your feeds and lists.

Version 0.5.3 (alpha) - 5 January 2007
Tidy-up of a couple of errors following significant internal changes in 0.5.2. This is really a checkpoint build towards a much more configurable aggregator.

Version 0.5.2 (alpha) - 31 December 2006
Mega changes to the internals; it should be more efficient now. No significant change to functionality. This is really a checkpoint build towards a much more configurable aggregator.

Version 0.5.1 (alpha) - 18 December 2006
Fixed to work with the Tiger Administration plugin. Increased the range of XHTML tags that can be captured in a feed. Fixed a multibyte encoding bug found by Knut Sparhell. A couple of other minor tweaks.

Version 0.5.0 (alpha) - 10 December 2006
Three changes: First, I am now fetching feeds using the Snoopy PHP class that comes with WordPress. Second, I have changed the input method for multiple URLS from a comma separated list to a white-space separated list. Third, I have introduced an errors page (in the dashboard) that lists the most recent feed errors, including for malformed feeds. (This last change saw a change in the underlying mysql tables, so I have incremented the version number from 0.4.x to 0.5.x).

Version 0.4.10 (alpha) - 8 October 2006
It looks I will be able to capture images in feeds without modifying the database. This release is a significant step closer to that objective. Hopefully images with indirect references are now converted to direct references.

Version 0.4.9 (alpha) - 3 October 2006
A few tweaks - the image stuff appears fine for complete references, but needs more work for indirect references. This will necessitate a change to the database. I will get to it in a few weeks.

Version 0.4.8 (alpha) - 2 October 2006
A moderate upgrade:

Version 0.4.7 (alpha) - 2 October 2006
A moderate upgrade:

Version 0.4.6 (alpha) - 24 September 2006
Minor tweaks - cut very long words back to 25 characters - this will need further work

Version 0.4.5 (alpha) - 22 September 2006
Minor tweaks - can now list feeds that do not include item links (this version not released)

Version 0.4.4 (alpha) - 17 September 2006
Fixed a glitch introduced in 0.4.3

Version 0.4.3 (alpha) - 17 September 2006
Further minor fixes/tweaks

Version 0.4.2 (alpha) - 16 September 2006
Minor fix - allow for URLs with ampersands

Version 0.4.1 (alpha) - 16 September 2006
More minor fixes - with time stamping

Version 0.4.0 (alpha) - 16 September 2006
A few changes - Restored old feed reader from version 0.2.2 but upgraded to use cURL if normal read fails - Used the link if the title is not in the channel information. - other minor tidy-ups

But still uses the same database structure as 0.3.x

Version 0.3.2 (alpha) - 29 July 2006
Minor tidy ups - fixed the missing variable to BDPRSS2::print_item_set() - trimmed output to the last word.

Version 0.3.1 (alpha) - 22 July 2006
Minor tidy ups - nothing consequential.

Version 0.3.0 (alpha) - 21 July 2006
This version has been updated to use MagpieRSS that comes with Wordpress. It now allows for automatic fractional GMT adjustments. There is additional information on the control panels, including the age of the most recent post from each feed. Links can be launched in new windows. Feed names can be permanently changed (for example, from the pompous "Jerry's cornucopia of wonderful ideas and ethical advice for modernity" to the simpler "Jerry's site").

If you are upgrading: you will need to hit the delete-everything-button-of-death on the old version and then completely remove the old plugin from the plugin directory. Finally, install the new plugin and activate it.

Before hitting the button of death, you may want to take notes on the format layout you use. Also, if you view the source of that page, you will find a comma separated list of all your feeds. Copy it to a file, you will be able to use that to enter your feeds into the new version.

Apologies for the kludgy upgrade instructions, but this is test software. You are using it in full recognisition of all the risks.

Version 0.2.2 (test) - 30 Dec 2005
When editing a list, there is now a variable to prevent printing the site name. To use this update you should delete the old bdprss_lists table from your database. But if you are feeling particularly brave, use myPHPadmin to add a field to the wp_bdprss_lists table. The field is named 'output_site_name'. It is an ENUM field. The enumeration is ('Y', 'N'). The default value is 'Y'. The field is not null. (Warning, if you do the wrong thing in myPHPadmin, you can destroy your entire blog).

Version 0.2.1 (test) - 18 Dec 2005
When editing a list, there is now a variable to set the length (in characters) for each item displayed. To use this update you must delete the old bdprss_lists table from your database.

Version 0.2.0 - pre-release 3 - 3 Dec 2005
Fixed a bug that became evident with Firefox 1.5

Version 0.2.0 - pre-release 2 - 7 Sept 2005
Minor tweaks

Version 0.2.0 - pre-release 1 - 4 Sept 2005
Complete rewrite. Still only alpha stable code

Version 0.1.3 - 13 July 2005
Added $table_prefix code. This means more than one wordpress blog in a database is now supported.

Note: this necessitates re-enetering your feed lists.

Version 0.1.2 - 13 July 2005
Made the RSS polling frequency a variable that can be changed from the dashboard.

Version 0.1.1 - 10 July 2005
Tidy-ups.

Version 0.1 - 10 July 2005
Completed the usability upgrade.

Version 0.0.7 - 10 July 2005
Vastly improved method for adding and removing feeds. Much friendlier to use now.

Version 0.0.6 - 2 July 2005
Minor bug fixes. Removed some of the debugging code. Fixed a bit of phpexec that was not interacting well with wpautop() in WordPress.

Version 0.0.5 - 25 June 2005
Added the delete sites code to the set function.

I am a little more comfortable that the code is becoming stable.

Version 0.0.4 - 23 June 2005
Minor bug fix. You can now include your own site in the list of feeds.

Version 0.0.3 - 19 June 2005
Played with the phpexec plugin to make a stand alone feed page. I have tweaked the plugin to do what I wanted. (http://priyadi.net/archives/2005/03/02/wordpress-php-exec-plugin/)

Also changed the calling arrangements. So you will to change the code in your sidebar.php to upgrade to this version.

Still a prototype version!

Version 0.0.2 - 9 April 2005
Played with character encoding. Some other minor changes. Still a prototype version.

Version 0.0.1 - 29 Mar 2005
Prototype version released. This should be considered alpha development software.