22 April 2007

1.3.x Progress

The 1.3 development is coming along very well. With todays release 1.3.3 there is a pretty reliable implementation for the old problem of how to determine the number of unread items of each feed and folder in the feed list.

What is missing is the calculation of the unread count for search folders which is pretty problematic. Search folders are internally realized with sqlite views and every item modification can change the unread count of each search folder. So the only way to update the unread count of all search folders after a user interaction is to recount it for each search folder or to determine which of the search folders was affected and to increment/decrement the unread count appropriately.

The disadvantage of both solutions is that the unread count update processing time increases directly with the number of search folders. Choosing to recount on each update additionally decreases performance with a growing total number of items, while the delta processing depends on the matching rule complexity.

Anyone knows a good design pattern for realizing such a dynamic counting functionality?

7 comments:

Eric said...

You could have one list for each item that can be used in a rule (i.e., Item title, Item body, Feed title, etc.). Each list would contain search folders that use that item in its search criteria.

When something changes, iterate through the appropriate list and only check for the one item that changed if the criteria that use the field that changed are all valid.

This has the problem of you having to know if the item was already in the list, but how to correct that is probably best for you to say, since it seems like it could change with implementation.

This solution would have some more logic for each of the cases, but as far as scalability, it would be good.

Aristotle Pagaltzis said...

Constain the view to a criterion matching the changed item(s). Eg. after

UPDATE items SET read=1 WHERE item_id=13882;

you then do something like

SELECT SUM(read) FROM vfolder_03 WHERE item_id=13882;

and use that for your calculation. Assuming there is an index on the id column, that should be fast (since views are nothing more than pre-defined SELECTs that are combined with the user-supplied SELECT to form the actual SELECT executed).

intangible said...

I don't have much experience with them, but I think you could use triggers to update a count somewhere on update of a table:

http://www.sqlite.org/lang_createtrigger.html

Lars said...

Thanks for your hints. I think I got some new ideas I would not have got on my own!

Most important is to find a way to represent (and have fast access) to the "item is in search folder" relation. Currently it is only given implicitely therefore I cannot use triggers to calculate deltas for the different search folder views.

But having an extra relation for the item - search folder relation will allow counting.

Therefore I see now two possibilities:

1.) To have a merged match list as eric implied, where one can easily count the unread items of a search folder.

2.) Or to have a separate view for each search folder like aristotle suggested that simply selects the unread count aggregate. This solution seems easier but its performance propably depends on how the DB implements views.

But this is good because now I have something to think about again!

Stefan said...

Hi,

I found a bug. Try to get the feed from the Jamestown University http://www.jamestown.org/terrorism/
(url=http://www.jamestown.org/terrorism/news/rss.php)

The feedback from the feedvalidator:
http://feedvalidator.org/check.cgi?url=http://www.jamestown.org/terrorism/news/rss.php

It looks like a UTF-ASCI-problem.

Sorry, I didn't see another way to get in contact with you.

cu

Anonymous said...

is there better support for favicons in 1.3?

For examples the feed for http://www.isp-control.net/ and www.lawblog.de don't show any favicons in liferea, but they have one in firefox :(

would be REALLY COOL to see this fixed :)

Anonymous said...

It would be better if liferea would search the favicon in the feed url (example.com/xml/rss/feed.xml), but on the main page/domain (example.com)