Appears to be fixed (went the editing route) in testing - A small group with binaries semi-randomly picked a binary to auto-download (in settings/preferences/downloading/auto highlight) to nzb and the nzb showed up after manual updating.

Thanks much!

They were a bit different each time like failure to add or remove columns/keys.  If the errors had been identical each time or it failed in general to eventually run (4 failures iirc, though hours between some of those) I might have kept them...  I didn't notice the auto-download failure until about 2 days after I started the upgrade process.



At this point the only thing not working (as near as I can tell) is the auto-download, I managed to stumble my way through the others.

I've been having troubles since upgrading, this one I can't seem to get past.
I can't seem to get autodownload working again.  I referred to the link below a bit.
http://urdland.com/forum/index.php?topic=539.0

I have autodownload enabled in all 3 places (admin/config/global, admin/users/<account> and settings/preferences/downloading).
I have search terms set in downloading which when I browse a relevant group get highlighted.  There are items which do not indicate they have been read.  I also have download as nzb enabled, did try disabling that, still didn't autodownload.

When I do update a group, which has matching sets that have not been auto-downloaded or manually selected yet, I do not see an "fn:auto_download" in the log (set to debug before doing the update).


The following 2 items may not be relevant...
I had several errors with the automated update (uncaught exceptions I think) but eventually after several attempts I got to the normal interface.

I had an odd problem where when I updated groups it would say that the group didn't exist.  I updated the group list (added 2 items) and group updates started working.

The main thread of this does appear to be resolved.

Thanks.

OK.  I have updated sets twice in a row (nzb auto-download), moving the nzb files out of the way before the 2nd update.  It is creating duplicate sets (made sure).  Notably it is not generating nzb files for everything, just items that haven't been auto-downloaded (nzb option OFF) yet.

If there are any other code bits you would like me to test I can.  The thing I was keen about is in fact working (no thanks to my activities).  Thanks much!

Lets not discount the slowness of the system.  For reference it can't even use half the bandwidth available to it on header downloading, even with new, empty, groups.  Also I didn't expect the process to take a couple of days but leaving myself room to be able to monitor the process in person.

I am confident that it was my own doing (at least the auto-download stuff).  Basically I was killing the process before it could complete and being daily/weekly items (and in a state of concern to keep the system from virtually halting) I didn't catch that the sets were in fact newer than the previous ones.

Sorry for dragging this on.

I am going to wait for the groups to populate with something new, update with auto-nzb and then delete the created nzbs and do another update (I still have that line in the functions.php).

Code:

$sql .= "(usersetinfo.\"statusnzb\" != 1 OR usersetinfo.\"statusnzb\" IS NULL) AND ";

[edited, see below]

Update: Not so good.
A) I unchecked the option to halt unknown passworded downloads and not the NZB downloads so it was correct behavior downloads weren't starting (at least that day) since I had NZB turned on.

B) I had a lot of nzb files, this was with the extra line of code instead of just a few from the last few days.

C) I correctly turned off NZB auto-download leaving regular auto-download on and ended up with the large number of items again so whatever it is doesn't seem unique to nzb files.  Thankfully I now know where that clear all downloads button is (and that it's there) .

[edit]
I got a database query set up to count off the number of unread user sets...  I have a suspicion that may take a few days to check...  I think my activity in response to the seemingly duplicate auto-download entries (stop and clear all downloads) might be messing things up.  NOTE that this is not my reaction with the nzbs since that's a negligible load on the system.

Specifically I noticed that the count is dropping and I am not 100% certain the added downloads are duplicates.
<shakes head>...

[/edit]

Sorry for the delay... didn't notice the new post for some reason.
Tested it out after making the change to functions.php - also restarted urd, apache, and mysql for good measure.  Still generated a ton of NZB files.  I will try it again without the nzb option and see what happens. Another refresh of headers today should generate no new sets so this should be a good test (at least from not downloading everything).

[update] no new downloads with the manual update and auto-download nzb disabled (but auto-download enabled).  Will put in a new post if the next test works - auto-update headers and auto-download. [/update]

I have been fiddling with the new version for a bit now (surprised about another speed change in the positive).  I didn't run the update.sh script previous to looking through the new version though after having run the script it did nothing (already updated message).  My guess it the install process updated the database (ran the script).

Automatic set downloading (matched strings) is indeed working... Though is it supposed to get everything that matches every time?  I was expecting that new sets would be checked for auto-download...

(artificial) Example: group set to 1 month before expiring sets. Group set to auto-update daily.  Group expected to have 1 new (matching) set added each day.  After updating the new item plus the previous month's matching sets are added to download. - so instead of 1 new item get 30 new items set to download (the previous 29 were already selected for download previously).

btw: love the nzb feature - has made testing much less painful...

Made sure I got the relevent check boxes checked - at the admin level and at the user level.

Put in some search terms.

Browsing the group shows highlighted items so that part seems to work fine.

Updated the group and after the update steps are completed (update headers, generate sets, expire) I have the following pop up into my log:

Code:

Something went wrong while auto-downloading: Could not execute SQL query "SELECT "name", "pass" FROM users WHERE "ID" = '8'" mysqli error: [2006: MySQL server has gone away] in EXECUTE("SELECT "name", "pass" FROM users WHERE "ID" = '8' LIMIT 1")

It seems to hang for a LONG time on expiring btw... I don't remember this long of a hang before enabling auto download.  I noticed that while the long hang is present in expiring phase it doesn't error on small groups.  I tried doubling the default_socket_timeout (to 120) in php.ini (in cli)... didn't seem to help.

I don't know if a timeout is the right thing to be chasing from that error it only occurred to me from some time on google and an odd thread found.  I also doubled the memory_limit to 256.

Just for fun I tried modifying the timeout to 1 hour (3600 seconds) and still got the error...

I am feeling vaguely dumb for not trying to just run the daemon... (that seems to have done it).

For reference - running the sql queries by hand gave errors that basically amounted to "can't do that as it's already been done" (extreme simplification).

The error I got with the cookie to auto-login:
Fatal error: Exception thrown without a stack frame in Unknown on line 0

anyway - all done and working again w/o having to reset the db .

a little feedback (while I'm here) - new interface is kinda neat, loading screens a plus and loading times are indeed *MUCH* faster for big groups.

Thanks much

urd daemon process was off when I started the upgrade.

nothing besides the error message is on the webpage (so no ability to restart via the lightbulb icon... or start in the first place).

sh update.sh 1.0.5 output:
ERROR 1050 (42S01) at line 1: Table 'categories' already exists
Query could not be executed
exit: 366: Illegal number: -1


I will run through the script and try to pick out the commands that need to be run by hand...

Had 1.0.5 running nice for quite a while... noticed the out of date light in the corner so downloaded the 1.0.6 deb package, installed it, and got some strange errors.

Realized the problem was probably due to having cookies from urd set to auto-login so I deleted those cookies.

Currently getting "Interesting... Preference not found('modules')".

I saw a post about deleting .installed in the urd folder however this seems to lead to a requirement to start over with the urd mysql database and having taken a month to get a few newsgroups into the database I would much rather not start over.

I have been hunting around and didn't find anything good (but I am still learning about urd... and linux... and php).  I would prefer shutting down urdd gracefully...  I suppose a non-graceful method would be to shutdown web services (apache)...

Some peaking around I see that 17 is passed to urdd to cause a startup/restart but 17 is COMMAND_PASS while COMMAND_RESTART is 55 and COMMAND_SHUTDOWN is 23... I am not quite connecting the dots.

If shutting down apache closed urdd gracefully (and starting apache up started urdd if it was running) that would be sufficient but somehow I don't think that is how it works (and it shuts down other things I would rather not turn off).

Mkay, so (after I update urd) I can create a subset of what I want to browse/search by only checking the groups I want visible... That should work out fine.

Thanks much!

Hidden checkbox?  The subscription -> newsgroups area with the visible checkboxes?  Those don't actually seem to do anything.  I tried hiding (unchecking) all but one group and they all still show up in browse sets -> group sets.

As for size - I have read that there is a filesystem that would work with linux to provide real time compression on the databases (yes, more degradation of performance).  I would assume the degradation is no where near as bad as newsleecher though since urd is using a well developed database program (mysql).

For comparison newsleecher crashes often and when it crashes it tends to 'forget' all the headers for the active group(s) when it crashes.

That raises a question though - Does performance degrade due to all the groups or individual groups?  Put another way, is the performance only taking a hit when accessing the big groups or any group accesses?

[Completely working:]

Here's the list:

Completely working:
- Newszilla (xs4all)
- Newszilla6 (xs4all, requires IPv6)
- Eweka over IPv6 (requires IPv6)
- Giganews
- http://www.hitnews.eu (SSL)
- Astraweb (http://www.news.astraweb.com)
- PowerUseNet (http://www.powerusenet.com/)
- Usenetserver (http://www.usenetserver.com)
NZB only:
- download2day
- XSNews   

I am still fiddling with urd.  I rather like it so far.  Trying to move off of Newsleecher as it can't handle large numbers of headers well and have been looking for something server side.  I rather stumbled on this, happily.

Anyway I am trying to figure out if there is some way of combining multiple groups into 1 searchable set.  That was a feature we used rather frequently to limit just how much we were searching/browsing.

On the side question - I just want to confirm that my configuration is what I think (hope) it is.
server1 connecting via ssl to usenet with primary index not checked.
server2 connecting to the same usenet server via stunnels on the local machine, primary index is checked.

I am hoping to confirm that server1 wont be downloading any headers and server 2 wont be downloading any binaries.

I have it setup this way because the usenet server supports ssl-zlib compression but as near as I can tell urd doesn't.  From reading ssl-zlib can be a bit expensive cpu time on their end so I don't want to 'waste' their time trying to compress binaries.

My usenet server claims 500 or so days retention but if I have any groups expiration over 365 I get a message: "Error: Bogus expiry time entered" and the group isn't added.

I have observed the less active groups actually have around 400 days.

Is this a technical limitation in URD or more bug worth reporting?

Trying it out on Ubuntu 9.10...