Error downloading articles

[jkv]

After downloading newsgroups list I try to update the newsgroups I selected, but when I do the following error occurs:

2008-11-24    20:18:01    ERROR    Error while updating alt.binaires: Cannot select next article for alt.binaires
2008-11-24    20:18:01    NOTICE    Cannot select next article for alt.binaires
2008-11-24    20:18:01    NOTICE    Determining first articles number for alt.binaires

Any idea where I should look to solve this?

Thanks,

JkV   

[spearhead]

You're probably using a server that doesn't support getting the headers. Which server are you using? Download2day doesn't support getting headers, e.g.. If you can try and telnet (or netcat) to the server and run the following commands (maybe you need to authorise first.

group alt.binaries.boneless
stat
next
list

if you only get errors from the last 3 (420, 421 msgs) your server doesn't support getting headers.

You can still use urd to download by way of NZB files tho. But you need to get the nzb files elsewhere

[jkv]

I have an account on Yabnews.nl

[spearhead]

I have an account on Yabnews.nl

I don't have access there, but it seems many commercial usenet providers don't give the possibility to run the list command. You can try it out by running from the command line

telnet news.yabnews.com 119

then type the following commands
authinfo user USERNAME
authinfo pass PASSWORD
group alt.binaries.boneless
list


if yet a list of articles sth else is wrong. If you get a reply code of 400 something, indexing isn't allowed on the server

[jkv]

Done as you wrote and list command works fine.
So indeed there must be something else causing the problem

telnet news.yabnews.com 119

then type the following commands
authinfo user USERNAME
authinfo pass PASSWORD
group alt.binaries.boneless
list


if yet a list of articles sth else is wrong. If you get a reply code of 400 something, indexing isn't allowed on the server

[/quote]

[spearhead]

what version do you use 0.5.5 or 0.5.6 from subversion - if the latter, which revision? (run svn info)

Does it happen every time you run update?

Can you set the log level to debug (in admin/configuration) and post some more output?

[jkv]

0.5.5

[spearhead]

does this group "alt.binaires" actually exist. And does it have any articles? The newsgroup overview has a post count, if it's zero there are no posts on the server (if the group is not subscribed, if it is, the number is the number of articles urd has stored for that group)

[jkv]

I tried it on different groups with the same result.
you can download my log on www.deplecht.nl/debug.zip

perhaps this clears things up.

Thanks,

JkV

[spearhead]

a quick glance at the log ...
I only see you update ngs that are non-existent or (nearly) empty on my news server. Not all groups generate the errors... the ones that do, don't exist, the other ones are I presume empty as I don't see any msg like "adding set...". Can you try it with a group that does have msgs for sure, like alt.binaries.boneless or  a.b.movies.divx. If these also give the same error or don't get any articles something is definitively wrong

[jkv]

Did that and it looked if he was updating.
But then the same error unfortunately and this one keeps popping up as well:

"2008-11-25    20:02:32    WARNING    Warning on line 437 in file /var/www/html/urd/urdd/urdd.php: socket_select(): unable to select [4]: Interrupted system call (2)"

you can view my config at www.deplecht.nl/config.zip

[spearhead]

that warning is normal. Except that I can't seem to get php to suppress it properly.

the config file is empty.

[spearhead]

can you add a line to functions/nntp.class.php right after line 98 where it reads :

Code:

} catch (exception $e) {
$article = FALSE;
}

add inside the catch{} block:

Code:

write_log ($e->getMessage() . '(' . $e->getCode() . ')', LOG_NOTICE);
write_log ($e->getTraceAsString(), LOG_NOTICE);

and post the output?

[jkv]

Did a whole new installarion including database but with the same result


Debug output of URD settings

--------------- copy from here----------------
Global configuration

array(61) {
  ["URD_version"]=>
  string(5) "0.5.5"
  ["admin_email"]=>
  string(15) "jkv@deplecht.nl"
  ["auto_expire"]=>
  string(1) "1"
  ["auto_reg"]=>
  string(1) "0"
  ["cksfv_path"]=>
  string(14) "/usr/bin/cksfv"
  ["default_expire_time"]=>
  string(1) "5"
  ["default_language"]=>
  string(14) "nederlands.php"
  ["default_template"]=>
  string(7) "default"
  ["dir_cd"]=>
  string(0) ""
  ["dlpath"]=>
  string(9) "/wd/news/"
  ["forumpassword"]=>
  string(6) "xxxxxx"
  ["forumusername"]=>
  string(0) ""
  ["global_hidden_files_list"]=>
  string(2) "  "
  ["global_hiddenfiles"]=>
  string(1) "0"
  ["group"]=>
  string(0) ""
  ["log_level"]=>
  string(1) "6"
  ["maxbuttons"]=>
  string(2) "10"
  ["maxdl"]=>
  string(3) "400"
  ["maxul"]=>
  string(3) "100"
  ["period_cd"]=>
  string(1) "0"
  ["period_getinfo"]=>
  string(1) "0"
  ["period_ng"]=>
  string(1) "0"
  ["period_opt"]=>
  string(1) "0"
  ["period_sendinfo"]=>
  string(1) "0"
  ["period_update"]=>
  string(1) "8"
  ["permissions"]=>
  string(4) "0600"
  ["register"]=>
  string(1) "0"
  ["run_update"]=>
  string(1) "0"
  ["scheduler"]=>
  string(2) "on"
  ["shaping"]=>
  string(1) "0"
  ["tar_path"]=>
  string( "/bin/tar"
  ["time1_cd"]=>
  string(1) "0"
  ["time1_getinfo"]=>
  string(1) "0"
  ["time1_ng"]=>
  string(1) "0"
  ["time1_opt"]=>
  string(1) "0"
  ["time1_sendinfo"]=>
  string(1) "0"
  ["time1_update"]=>
  string(2) "12"
  ["time2_cd"]=>
  string(1) "0"
  ["time2_getinfo"]=>
  string(1) "0"
  ["time2_ng"]=>
  string(1) "0"
  ["time2_opt"]=>
  string(1) "0"
  ["time2_sendinfo"]=>
  string(1) "0"
  ["time2_update"]=>
  string(2) "00"
  ["trickle_path"]=>
  string(0) ""
  ["un7zr_path"]=>
  string(0) ""
  ["unace_path"]=>
  string(14) "/usr/bin/unace"
  ["unarj_path"]=>
  string(14) "/usr/bin/unarj"
  ["unpar_path"]=>
  string(0) ""
  ["unrar_path"]=>
  string(14) "/usr/bin/unrar"
  ["unzip_path"]=>
  string(14) "/usr/bin/unzip"
  ["update_text"]=>
  string(0) ""
  ["update_type"]=>
  string(1) "0"
  ["update_version"]=>
  string(0) ""
  ["urdd_host"]=>
  string(9) "localhost"
  ["urdd_maxthreads"]=>
  string(1) "5"
  ["urdd_path"]=>
  string(25) "/var/www/html/urd/urdd.sh"
  ["urdd_port"]=>
  string(5) "11666"
  ["urdd_restart"]=>
  string(1) "1"
  ["url"]=>
  string(23) "http://192.168.1.1/urd/"
  ["usenet_server"]=>
  string(1) "4"
  ["yydecode_path"]=>
  string(17) "/usr/bin/yydecode"
}

User preferences

array(24) {
  ["button1"]=>
  string(4) "none"
  ["button10"]=>
  string(4) "none"
  ["button2"]=>
  string(4) "none"
  ["button3"]=>
  string(4) "none"
  ["button4"]=>
  string(4) "none"
  ["button5"]=>
  string(4) "none"
  ["button6"]=>
  string(4) "none"
  ["button7"]=>
  string(4) "none"
  ["button8"]=>
  string(4) "none"
  ["button9"]=>
  string(4) "none"
  ["defaultsort"]=>
  string(11) "Subject ASC"
  ["delete_files"]=>
  string(1) "1"
  ["hidden_files_list"]=>
  string(0) ""
  ["hiddenfiles"]=>
  string(1) "0"
  ["language"]=>
  string(14) "nederlands.php"
  ["mail_user"]=>
  string(1) "0"
  ["maxsetname"]=>
  string(2) "80"
  ["minngsize"]=>
  string(1) "1"
  ["minsetsize"]=>
  string(1) "0"
  ["search_terms"]=>
  string(0) ""
  ["setsperpage"]=>
  string(3) "100"
  ["template"]=>
  string(7) "default"
  ["unpar"]=>
  string(1) "0"
  ["unrar"]=>
  string(1) "1"
}

URDD Status

array(5) {
  

• =>

 string(21) "Uptime: 0 days 0:8:41"
  [1]=>
  string(9) "Queued: 0"
  [2]=>
  string(10) "Running: 0"
  [3]=>
  string(12) "Scheduled: 1"
  [4]=>
  string( "Users: 1"
}

URDD settings

array(22) {
  

• =>

 string(17) "urdd_port : 11666"
  [1]=>
  string(28) "urdd_listen_host : 127.0.0.1"
  [2]=>
  string(23) "urdd_listen_host6 : ::1"
  [3]=>
  string(28) "urdd_logfile : /tmp/urdd.log"
  [4]=>
  string(32) "urdweb_logfile : /tmp/urdweb.log"
  [5]=>
  string(22) "urdd_log : file|stderr"
  [6]=>
  string(21) "urdd_min_loglevel : 6"
  [7]=>
  string(22) "urdd_need_auth : FALSE"
  [8]=>
  string(21) "urdd_daemonise : TRUE"
  [9]=>
  string(22) "urdd_debug_level : 255"
  [10]=>
  string(17) "urdd_timeout : 30"
  [11]=>
  string(23) "urdd_queue_size : 20000"
  [12]=>
  string(14) "maxmsgs : 2000"
  [13]=>
  string(14) "urdd_uid : urd"
  [14]=>
  string(14) "urdd_gid : urd"
  [15]=>
  string(23) "group_filter : alt.bin*"
  [16]=>
  string(16) "scheduler : TRUE"
  [17]=>
  string(24) "log_file : /tmp/urdd.log"
  [18]=>
  string(19) "urdd_restart : TRUE"
  [19]=>
  string(19) "urdd_maxthreads : 5"
  [20]=>
  string(19) "nntp_maxthreads : 4"
  [21]=>
  string(24) "orig_nntp_maxthreads : 4"
}

URDD version

array(1) {
  

• =>

 string(12) "0.5.5 (beta)"
}

URDD jobs, queue, threads, users

array(6) {
  

• =>

 string( "Threads:"
  [1]=>
  string(6) "Queue:"
  [2]=>
  string(5) "Jobs:"
  [3]=>
  string(78) " #1.  @ 11/30/08 12:00:00 (repeat every 604800 sec) 1 u:jkv cmd:CHECK_VERSION "
  [4]=>
  string(6) "Users:"
  [5]=>
  string(9) " #1.  jkv"
}


--------------- copy to here----------------

[jkv]

what do you mean exactly "add inside the catch{} block     

can you add a line to functions/nntp.class.php right after line 98 where it reads :

Code:

} catch (exception $e) {
$article = FALSE;
}

add inside the catch{} block:

Code:

write_log ($e->getMessage() . '(' . $e->getCode() . ')', LOG_NOTICE);
write_log ($e->getTraceAsString(), LOG_NOTICE);

and post the output?

[spearhead]

what do you mean exactly "add inside the catch{} block     

The code at line 97-99 in functions/nntp.class.php reads:

Code:

} catch (exception $e) {
$article = FALSE;
}

please change it to :

Code:

} catch (exception $e) {
$article = FALSE;
        write_log ($e->getMessage() . '(' . $e->getCode() . ')', LOG_NOTICE);
        write_log ($e->getTraceAsString(), LOG_NOTICE);
}

and post the output?

[jkv]

2008-11-25    22:36:06    NOTICE    Cannot select next article for alt.binaries.boneless
2008-11-25    22:36:06    ERROR    Error while updating alt.binaries.boneless: Cannot select next article for alt.binaries.boneless
2008-11-25    22:35:06    INFO    Connecting to NNTP server: news.yabnews.nl
2008-11-25    22:35:06    INFO    using authentication as ynxxxxxx
2008-11-25    22:35:06    NOTICE    Determining first articles number for alt.binaries.boneless

[zx81]

Hi,
can you try to connect via netcat to your provider ? Just like I did with download2day. For me this looks like the same problem.

-Markus

[spearhead]

atm I don't have any idea really what goes wrong.

Maybe you can try upgrading throughsubversion to the current dev version and see if that helps. I fixed some stuff in the guess...() function plus some timeout issues.

this should do it (you may have to install subversion first) :
# svn checkout http://urd.googlecode.com/svn/trunk/trunk urd

[jkv]

svn checkout http://urd.googlecode.com/svn/trunk/trunk urd
svn: OPTIONS of 'http://urd.googlecode.com/svn/trunk/trunk': could not connect to server (http://urd.googlecode.com)

[spearhead]

svn checkout http://urd.googlecode.com/svn/trunk/trunk urd
svn: OPTIONS of 'http://urd.googlecode.com/svn/trunk/trunk': could not connect to server (http://urd.googlecode.com)

probably a temporary error

[spearhead]

I have the same problem. Unable to get headers.

I tried the SVN, but in the config where you add the server it fails with...

'Fatal error:
Preference not found('preferred_server')

(Click the close-button in the top right to continue)'

so I'm unable to test if it works.

You can set the preferred server in the admin/usenet_servers page. There is a column of red/green bullets, that selects a server for downloading, the checkbox after it is used to set the preferred server. I'm still a bit working on the page... docuentation is also still missing.

If the update isn't working try telnetting to the newsserver and run these commands (you may need to log in first).

group alt.binaries.boneless
stat
next
list

if you only get errors from the last 3 (420, 421 msgs) your server doesn't support getting headers. Not all commercial servers support indexing.

[spearhead]

'Fatal error:
Preference not found('preferred_server')

this has been fixed in svn.

[Kane]

I installed the stable onto a Debian etch system, works without any problems. The system it wouldn't work on was a Gentoo box.

When I first ran the urd installer it choked on missing stuff in php (on Gentoo), after a few flag changes and a recompile urd installed fine, but gives the error above. Maybe its something not compiled the way urd wants it? I'll have a poke around tonight.

[spearhead]

I installed the stable onto a Debian etch system, works without any problems. The system it wouldn't work on was a Gentoo box.

When I first ran the urd installer it choked on missing stuff in php (on Gentoo), after a few flag changes and a recompile urd installed fine, but gives the error above. Maybe its something not compiled the way urd wants it? I'll have a poke around tonight.

Most of the testing has been done on ubuntu, so on debian it should give little trouble. The preference not found error should be fixed if you get the latest version from svn. It is something I'm still working on a bit tho

If you run into any trouble, just let me know and I'll see what I can do.