Allways on Monday's

[henk54]

So now and then, strange things happens here, but only on the Monday's.

Sometimes automatic-update refuses (not on the other days, but again not always).
Yesterday, yes on Monday, a.b.multimedia newsgroup could not be updated anymore.
The Updated-Kolom shows that there was actualy no new postings.
I know what that means, it can't update again ever.
If I invoke a Manually-update no tasks are shown!
This event occur fore some time ago at other news-group(s) and
what in the past happens from january(?) on...
But I can't memorized which, but it happens in time.

So I deleted a.b.multimedia, and guess what URD can't find the news-group anymore
(I started to add the newsgroup a.b.multimedia in Grabbit, and it shows up)
I'll see many NG. to search again, but not a.b.m., so this is very strange.

Which LOG is important for me to investigate? and
what must I looking fore..?

[spearhead]

One thing to look into is the jobs page (under the admin menu). It shows all scheduled tasks. and the Tasks page. If there is a thread running (paused, queue) like update or generate sets on a particular group it won't add a second until it is completed. Cancelling and deleting such a task may sometimes help .

additionally, one problem I am aware of is that the index on the server can sometimes overruns (as it is a 32 bit number) this shouldn't really happen but some servers seem to have this problem, esp large groups like boneless. I can't really reproduce it here so it's hard to test. The only solution atm is to purge the group and re-index it.

There is only one log file important to urd and that is the one that usually can be found in /tmp/urdd.log (also shown under admin/log). Urd doesn't do anything with other logs (but other used programs may do, like mysql or php - depends on the config).

I can't really say what the monday thing is about, but perhaps it has something to do with the database optimisation. This should be run weekly to keep performance up,  but you toy around with it more often or less. Or change it to another day. Better run it after all updates have completed.

[henk54]

....additionally, one problem I am aware of is that the index on the server can sometimes overruns (as it is a 32 bit number) this shouldn't really happen but some servers seem to have this problem, esp large groups like boneless.

That could be happen today, and also couple of days ago.
News-servers re-indexed postings (I'll think) there was a large amount of postings.
URD stucks on that, it needed over 7day index it all.
So I'll stopped URD-sever to do that.

The other nasty things I'll try to investigate more, and study the logs.
I'll come back if I have more insights.

And I look intoo the Job-things if URD must do something more on the monday's,
such as optimising....

[spearhead]

....additionally, one problem I am aware of is that the index on the server can sometimes overruns (as it is a 32 bit number) this shouldn't really happen but some servers seem to have this problem, esp large groups like boneless.

That could be happen today, and also couple of days ago.
News-servers re-indexed postings (I'll think) there was a large amount of postings.

It shouldn't happen ofter 2^32=4 milliard articles - that is quite a lot. Re-indexing shouldn't really happen as it confuses clients. I don't know what server you are using, but e.g. giganews has switched to 64 bit numbers. But not all clients like that. The rfcs are a bit sketchy on this issue. I guess the original rfc never foresaw that 2^32 articles would be reached. The successor just made it worse really.

[henk54]

See what happens on Sunday, and then on Monday

Code:

**Monday**
Updating a.b.multimedia 0% Cancelled 10/24/2011 15:50:00 10/24/2011 18:08:56 Could not execute SQL query "LOCK TABLE parts_3 write" mysql er

**Sunday**
Generating sets for a.b.multimedia 100% Finished 10/23/2011 15:59:45 10/23/2011 16:00:28 Generated sets data complete
Updating a.b.multimedia 100% Finished 10/23/2011 15:50:00 10/23/2011 15:59:45 737 articles/s - total 426687 articles

This happens also in the past with other (news) groups.
But I'm not sure it is/was the same error!!!
So I removed/deleted a.b.multimedia and then I couldn't retrieve it from the search-option
List newsgroups, again as before, newsgroup vanished if it never was listed ever on usenet!

When a hugh posts are passing by (in URD and in Unison) I'm  talking about some what above 1M postings instead of normal 10K-100K posts, this happens last week.
Newserver is eweka.

**Update1**
After the Maintenance ---> Update newsgroup list: 15:31, a.b.multimedia newsgroup is listed again, thats odd, really odd!
**Update2**
There 470M. (new)posts, that means it take just about 5 days to update.
Is there a way to limit this lets say about a few days (eq. 1M posts in total) in history, instead of....

[spearhead]

What does this line say further? It seems chopped off at the end:

Code:

Updating a.b.multimedia 0% Cancelled 10/24/2011 15:50:00 10/24/2011 18:08:56 Could not execute SQL query "LOCK TABLE parts_3 write" mysql er

You can limit the amount of artilces by the expire date. It defaults to 5 days (all articles newer than 5 days are indexed) There is also a hard limit on the number of articles "Maximum articles downloaded per update: " in admin/config/urd daemon.

[henk54]

I'll forgot to explain this a bit.

What does this line say further? It seems chopped off at the end:

The line is cropped, there's no more info, because (I was lucky) in the Job-history that
problem occur (yes on monday, and not on sunday) the big question is how did this happens.
If I started a Manully update, nothing happens no sign of something like 'Satus (1)'
only 'Status (0)' and stay that way.

Looks like you haven't not enough info on this, I was afraid of.

[spearhead]

Hmm the error should(!) be in the log file too. Or perhaps you can query the database directly:

Code:

select * from queueinfo where comment like "%Could not execute SQL query%";

AFAIK a lock table should not fail usually, just be suspended. So that is a little odd.

[henk54]

Hmm the error should(!) be in the log file too. Or perhaps you can query the database directly:

Code:

select * from queueinfo where comment like "%Could not execute SQL query%";

AFAIK a lock table should not fail usually, just be suspended. So that is a little odd.

The LOG-file on Monday giv's not any info, no warning, no error etc...
only the Job-History had some  limit info about that.

The good thing is IT will happens again, I'm sure of, and I hope it give more info.

The SQL code thing; Please, explain a bit more eq. the complete code, to investigate, Thanks.

[spearhead]

If you've no experience with mysql, it might nog be a good idea... but here goes anyway

run
cat dbconfig.php
and get the db_password value (between the single quotes)
login to mysql
mysql -u urd_user -p urddb
you'd be asked for the password, enter it
enter the query
type quit to exit

[henk54]

If you've no experience with mysql, it might nog be a good idea...

When I understand SQL a little bit more I try this, better be warned then...

New problem;
failed: Bogus expiry time entered

This occur when I click on Apply on the main-screen Settings -> Newsgroup.
Even if I change nothing (it wouldn't help BTW), this not funny.

Please giv' me some advise (maby an another Topic).

[spearhead]

Interesting? I'd have to look into that. What expire times do you have ?
Also what does this value in admin/config/globalsettings say:   

Code:

Maximum expire time:

expire times should be between 1 and that value (inclusive)

[henk54]

Interesting? I'd have to look into that. What expire times do you have ?
Also what does this value in admin/config/globalsettings say:   

Code:

Maximum expire time:

expire times should be between 1 and that value (inclusive)

Well don't look further!

Max.epx.time was 2!
I tried some numbers (5,10,100) then at 360 the error was gone.
So there's a relationship between exp.time set for a Newsgroup let's say 30 or 180 day's
and the setting under Admin, the value must be higher as the max. days to expire for a newsgroup.
In my case I used the max value of 180 days.
So 360 is on the save side.

How it come's, well in a rush (read the monday's problem here) I change that value by accident, not aware of the problems.

Thank you.