Unable to Update Newsgroups / retrieve Set Information - Session timeout

[hawklan]

Hi all!

Shortly I installed URD Version 0.6.0 (downloaded from http://urdland.com/cms/index.php?option=com_content&task=view&id=13&Itemid=27) on my Debian rootserver.

Everything seems to be working fine exept that I always get an error while trying to retrieve the set information from a big newsgroup.

After installation completed I connected to my newsserver and retrieved the newsgroup list.
On that list I now have lots of newsgroups - big ones with lots of posts and small ones with only a few posts.
Example:
a.b.german.divx with 46889 Posts (according to my newsgroup list)
a.b.dvd.german with 7310304 Posts

When I now subscribe to a.b.german.divx URD starts to retrieve the sets from that group and after some time (minutes) the process is finished an I have the sets available in "Browse sets" an I can download binarys from this newsgroup.
So far everything works great!

But when I now try to subscribe to a.b.dvd.german with its milions of posts the set retrieval process starts, tells me that it will take several houres to complete but after some time it suddenly stopps and in the log file I find the following:

Code:

Error while updating a.b.dvd.german: Connection terminated by server (news.aeton.de: Session timeout.) (400)

At first glance this seems to be a problem with the news server but I have tried with two providers now (aeton and usenext) and both times the same problem.
Is there anything I can do about that?

Thanks for any help!

[spearhead]

Probably just restarting the update will continue dl'ing more headers. But then again, that shouldn't be necessary as URD should restart automatically... I'll look into it.

Thanx for using URD and reporting the issue.

[hawklan]

Hi Spearhead,

thanx for your quick response!

URD in fact is restarting the update automatically after some time.
But to me it looks like it is downloading the headers from the beginning over and over and not continuing at the point at which the timeout occurred.
I had one such update running all night and in the morning it still had not finished though the estimated time for the process was only about two houres.
Also the size of the MySQL DB had not changed much compared to the evening (normally when importing set info for a new newsgroup results in significant increase of DB size).

If you need any further info let me know

[spearhead]

Well it may seem like it restarts over again, but it doesn't remember where it started the last time, it just starts a new action, but all the previously downloaded headers are in the database. It does wait a while before reconnecting, that's correct.

The table to look at is parts_XXX where XXX is the id of the newsgroup. Other tables are updated after the update completes successfully.

If it doesn't finish at all or if it doesn't have any articles when it finishes, please post again, and I'll look into it.