News Feeds

Forum Resource Management & Use (Policy, How To Post)

Moderators: OpenUru.org Moderators, Domain Managers

Post Reply
admin
Site Admin
Posts: 194
Joined: Thu Nov 13, 2008 6:33 pm

News Feeds

Post by admin »

News feeds are enabled. Please advise on any useful feed tweaks - currently set to 5 at the forum level.
User avatar
Mac_Fife
Member
Posts: 1239
Joined: Fri Dec 19, 2008 12:38 am
Location: Scotland
Contact:

Re: News Feeds

Post by Mac_Fife »

At the moment, 5 seems fine. If things were to start jumping around here then I suspect you'll need to double that.
Mac_Fife
OpenUru.org wiki wrangler
Christian Walther
Member
Posts: 317
Joined: Sat Dec 13, 2008 10:54 am

Re: News Feeds

Post by Christian Walther »

I have recently resolved to start following OpenUru.org again, and thought that the feeds would be very helpful with that. Unfortunately, I have discovered that they have the same problem as those of the MOUL forum:

Why the RSS feeds lose posts
Christian Walther wrote:I finally figured out why I miss so many posts when I only read this forum through the RSS feeds! :D

At first I thought I just wasn't refreshing them often enough, but that wasn't it - I refresh them every hour, a feed carries 15 posts, and we have much less than 15 new posts per hour per subforum.

The problem is that the feed does not contain the 15 most recent posts in the subforum, but the last messages of the 15 most recently posted-to threads. In other words, each feed never has more than one post from the same thread. If there are several posts to a thread in the interval between refreshings, you miss all but the last of them.

At least it's not possible to miss any threads this way, unless you wait long enough that more than 15 new ones are opened. :?

I wonder how hard it would be to get this fixed. Any phpBB experts around here who can give their judgement, before I bother the Cyantists with it?
Is this something that could easily be fixed, or am I just misusing the feeds for a purpose they weren’t intended for?
User avatar
Mac_Fife
Member
Posts: 1239
Joined: Fri Dec 19, 2008 12:38 am
Location: Scotland
Contact:

Re: News Feeds

Post by Mac_Fife »

The news feed generation in phpBB3 (as used here) is a built-in feature and the scope for customising it's behaviour is fairly limited. On the MO:UL forums, the news feeds are a "mod" that has been applied, and I think it's largely coincidental that it seems to work almost identically, as there are several RSS mod packs for phpBB2 that could have been chosen.

You've noted that the feeds deliver only the latest post in any recently updated thread, and I think that's intentional: On a busy forum, the number of posts even in a hour could be substantial - We've seen on the MO:UL forums a particular "hot topic" pop up on occasion that can attract 100 or more replies in that time. So either a) you allow the feed to grow to a very large size so that all the updated posts are delivered, or b) you limit the number of posts in the feed to the most recent N posts, irrespective of which thread they're in, or c) limit the number of posts and only report the most recent post from any thread.

If you choose a) then you could end up with several tens (or hundreds) of posts mentioned in feed, many of which may be from threads you're not interested in, and it could be hard to find the ones that are really important to you. Some people may want to read every post, but a lot of people won't. For b) a recently busy thread could end up using up the post limit, so other threads don't get mentioned in the feed at all. As you say c) won't return all the new posts, only the most recent one in any one thread, but should at least draw attention to the updated threads (provided that the number of updated threads doesn't exceed the post limit in the feed). I guess c) was seen as the best compromise for most uses.

What it means is that you can't use the feed itself as a means to keep up to date with the discussion - you have to follow the links back to the forum and catch up there, and I guess you could argue that this means you'd be a swell skimming the forums as using the feed. On the other hand, you can probably filter the news feed for the threads that interest you faster than you can go through the forums.
Mac_Fife
OpenUru.org wiki wrangler
Christian Walther
Member
Posts: 317
Joined: Sat Dec 13, 2008 10:54 am

Re: News Feeds

Post by Christian Walther »

I would obviously prefer b), but I see your point. Also, let me make clear that I’m not expecting anyone to reprogram the feed generator here if it’s not a simple configuration option, even if the way it works now wasn’t on purpose, so take what follows as idle musing in response to your detailed explanation.

For one, I could think of more sophisticated compromises – considering old threads, no matter how old, more important than new posts seems pretty radical to me. There should be some compromise between that and the other extreme (b)) of always preferring new posts, at the expense of almost-new threads. On the other hand, “more sophisticated” can also mean “less predictable”.

As to having posts in the feed that I’m not interested in, I think that’s handled well enough by having a separate feed for each subforum, and in any case, I’d prefer to have the filtering into things I want to read or not happen on my end.

Apart from saving time by being able to read several forums in a central, uniform place instead of having to visit a lot of web sites, one of the things I was hoping for when I started using feeds to read forums was to build a local archive of the forum, but that’s not possible with a feed of type c), except in the limit of refreshing it infinitely frequently (luckily for your webserver, my feed reader (Vienna) doesn't allow me to set different refresh rates for individual feeds, so I have all my feeds at 1 hour, otherwise I’d use 10 minutes or so for the forum feeds ;)). Many of the things I dislike about web forums in comparison to newsgroups and mailing lists are connected to the lack of (the easy possibility of) a local archive: You are at the mercy of the forum operator regarding whether what you write stays accessible – if the forum goes down, all your work is lost. You are limited to the built-in search function, while you could search a local archive in any way necessary. You may need to log in to see certain posts, which may have undesired side effects (like marking all other posts as unread).
User avatar
Mac_Fife
Member
Posts: 1239
Joined: Fri Dec 19, 2008 12:38 am
Location: Scotland
Contact:

Re: News Feeds

Post by Mac_Fife »

I think the bottom line, at least for this site, is that the feeds from the forum won't change significantly. The number of items on the feed may increase, but that's probably about it.

I think part of the problem with the "more sophisticated compromises" is that there's no practical way for a piece of software to evaluate the relevance of a post, and I think that's what you mean by "less predictable": Does a post to an old thread indicate a simple "bump", maybe a result of someone finding it for the first time, or has something happened to suddenly re-ignite that topic afresh? Most of the heuristic ways I can think of how this might be done would require a lot of tracking of the thread usage history, and that would probably mean that performance would take a bit of a hit for people using the forums. How much I can only guess, but I suspect we wouldn't be far off talking about doubling the volume of database SQL queries.

You're right, taking feeds for only the forums you're interested will help to ensure relevance of the material you receive, but I guess a lot of people will take only the overall forum feed. There is an additional problem with closed or private sub-forums in that to get the feed for those, you need to have a valid persistent login cookie for the forum. There were a few phpBB2 RSS/Atom forum mods that didn't account for forum permissions and exposed private posts to anyone who took the news feed. I think the same problem also affected some of the mods that were available for phpBB3 prior to version 3.0.6.
Mac_Fife
OpenUru.org wiki wrangler
Christian Walther
Member
Posts: 317
Joined: Sat Dec 13, 2008 10:54 am

Re: News Feeds

Post by Christian Walther »

I was more thinking of “less predictable” for the human sitting in front of his feed reader and thinking “how the heck is this server deciding which posts to give me in the feed?” But probably I’m the only one doing that, so it doesn’t matter. :)

My “more sophisticated” ideas didn’t go as far as heuristically analyzing the thread history, but you are right that some of them, like using some kind of weighting factor between the age of a post and its position in the thread to determine its “fitness” for the feed, might be too hard on the database. On the other hand, I’d expect a simple thing like “give me the five newest posts plus the latest posts each of the newest five threads, with duplicates removed” (which would give you a fluctuating number of entries in the feed, but I don’t think that’s a problem) to be only marginally more complex than the current system. But that’s hard to tell without knowing the database structure.
User avatar
Mac_Fife
Member
Posts: 1239
Joined: Fri Dec 19, 2008 12:38 am
Location: Scotland
Contact:

Re: News Feeds

Post by Mac_Fife »

I'm not volunteering to write a phpBB mod to do what you proposed Christian, but your suggestion is quite workable: The data base schema for phpBB is a lot simpler that it is for the wiki which does revision tracking, etc., so without looking at it in detail, I don't see it being too hard to do. Just don't expect it to happen here anytime soon - we're trying to keep the software here as "vanilla" as possible to minimise the hassle when it comes to applying critical updates.

I'd guess that on a busy forum, there may be a risk of dropping quite a few noteworthy threads, but nothing's perfect. I could suggest a minor amendment to your strategy:
Give me the N newest posts plus the latest non-duplicate post from as many of the most recent threads as will make up the maximum feed item count.
A small change, and not much more complex, but you'd get a consistent number of items each time.
Mac_Fife
OpenUru.org wiki wrangler
Post Reply

Return to “Forums (phpBB)”