Determine if a set has a password during downloading

[jdroflet]

Passworded sets can be annoying even malicious when clicking through to the link or in the contents of the file.

If there is already a method to determine if the rar is encrypted then  this kind of a wash for me. Otherwise here is how I detect if a set needs a password to extract.

Start the download:

Browse to the URD  /tmp/<uusername> folder
The running download will be the newest created folder, open it
Hit refresh until  rar files appear and have some data.
Open (do not extract) one of the rar file - I use 7-zip in windows
If the contents of the file name end in the expected file format extension EX: .avi .iso .mp3 then the file is not likely encrypted.
Close 7-zip - again do not extract, save or modify with the 7-Zip.

If the contents of the file have a .rar or unexpected extension it could require a password and you can cancel the download in URD,  save the bandwidth.

Feature request submitted
http://urdland.com/forum/index.php?topic=277.msg1850#msg1850
Regards,
John

[thorwak]

I've only DLed passworded crap by mistake once myself. Usually a good rule of thumb is to get the first version posted (on the right day), unless there is a PROPER. Fakes usually appear later, and have different size, names can hint also. It's not really a big issue, at least I haven't been suffering from it.

That said, I'll have a look at rar binaries to see if one can spot a flag that indicates encryption or not.

Looking at filenames inside archives is not really an option if you ask me, depends way too much on what you're downloading...

[thorwak]

Seems I won't have to do research myself

Here's the header spec. I haven't verified it myself though.

http://www.win-rar.com/index.php?id=24&kb_article_id=162

[thorwak]

I did some quick tests with encrypted split archives I created myself and a hex editor, and I can easily detect if there are encrypted files inside the archive. Usually an archive extracts to one large and few small files. Any part of the split archives can be VERY easily checked if the first file in that part of the archive is encrypted, next to no overhead since one only has to do read a very small part of the beginning. Since the large file will span over many archives it will be quickly spotted (unless it's rar insides rar I guess, but even that could be check for)

Spearhead, I'm not really a PHP guru and usually I would do sth like this as a separeate utility,c or perl, (it's funny that "file" doesn't report whether an archive is encrypted or not), but I could try adding a function (in functions.php or sth) you could call with a filename and i returns 0 is no encryption is found, -1 if encryption flags set?

It would create next to no overhead I'd say.. Doesn't matter which order files are downloaded in. In fact, best would be to check all rar files as they are assembled, and then have a config setting to abort and perhaps completely nuke a download if encryption is found.

You'd probably have to hook it up to the downloading/assembling code and add a config option for it, but I could write the checker.

Do you want it?

[spearhead]

It wouldn't fit into the downloading code, as all file interaction is handled by yydecode at that point. Also these may be in random order. So it would have to go into the unrar code run afterwards. There would be a slight benefit perhaps by indicating it contains a pw. 

Anyway, usually there is a crap prog in there installing malware or whatever, that contains the password. It would be more interesting if it would be able to detect that prog and extract the password from there. Dunno atm if that is feasible tho. URD has support for password encoded files tho. I'd have to look into that.

[thorwak]

There would be no reliable way to extract a password from an executable since they would have to be analyzed manually for every version of the malware out there. I've done my fair share of playin with "crackmes" ( ) and so on so I guess I could do it in theory, if there is one or two really common ones, but it feels kind of pointless since there will just be new ones coming out, I'm guessing.

My thought was more like this:
As files are downloaded and assembled rars start appearing. there could be 100 of them for a single download. We don't need all of them to check for this - the flag can be checked for in in the header in any one of them. So, as soon as 1 of them is there, URD could detect this (periodic checking for rar files in download directory as last resort, if URD has no real idea what's actually ending up in the download temp directory, or after yydecode has been run. It doesn't matter if we check x.rar or x.r34, x.03, the flags are in all of them. In fact we'd need just the the first couple 100 bytes of one the files, but easier to wait until at least one rar-part is completely assemebled.

Checking after the download is done wouldn't be worth the effort since the time and bandwidth it already wasted at that point.

But if you see no point in it, or if it's too messy having urdd "polling" the DL dir or whatever we'll scrap the idea, was just an idea Always fun to play around with binary files every now and then


EDIT: For the hell of it, if someone has a couple of links to malware appearing again and again in downloads containing obfuscated password, I could take a look at it. I like sticking it to the maleware dudes whenever I can Could perhaps end up to be a separate tool to safely extract passwords

[thorwak]

As an update, I've been working on this for a couple of days, hex analyzing various types of encrypted rars and studying specs.

I have working detection for:

- Normally encrypted RARs
- Encrypted RARs inside unencrypted RARs
- File encryption method for both of the above
- Block encryption method for both of the above (this would be impossible to detect by looking at filenames as even they are encrypted)

Both encryption types tested and detected successfully from external NZB files.

It works by looking at the actual flags in the RARs, and any RARs inside RARs, analyzing data on-the-fly so a signal can be sent back to the "downloader" to abort the task if the user so wishes.

There's no gurantee it'll make it into URD since there may be other considerations and ulimately not up to me. Also some slight work is still needed to make download tasks cancel gracefully.

I did it for fun, but if it ends up being used, and useful to someone (I have a hard time finding encrypted posts even when looking for them) - even better.