Log in

Thu, Oct. 13th, 2011, 09:51 pm
Amanda K: Question!!!

Has anyone found a good way to categorize their fics? Like if someone comes on here and says "I need a fic that involves kids" do you have a way to quickly find the name and link? I am becoming frustrated because I see requests I could answer, but I cant keep it all straight. Any suggestions? There are simply too many to keep them in my mind.

Fri, Oct. 14th, 2011 05:07 am (UTC)

Is there a installation guide? I've downloaded Perl, but I'm not sure what to do next.

Fri, Oct. 14th, 2011 05:20 am (UTC)

What operating system do you use? MS-Windows, Mac or Linux?

Fri, Oct. 14th, 2011 05:31 am (UTC)

I use Windows Vista.

Fri, Oct. 14th, 2011 06:31 am (UTC)

I gather that ActiveState Perl is the one to use on MS-Windows: http://www.activestate.com/activeperl/downloads
Presumably there are installation instructions on that site.

Then you need to install WWW-FetchStory from CPAN. This article tells you how to install CPAN modules: http://www.activestate.com/blog/2010/10/how-install-cpan-modules-activeperl

I'm afraid I can't give much more help than that; I don't use MS-Windows myself. If you have further problems, there are a few forums you could check out:
ActiveState Perl forums at http://community.activestate.com/forum
Windows Perl Scripting Forums at http://www.roth.net/forums/

Sat, Oct. 15th, 2011 09:39 am (UTC)

Phew, so I've finally managed to get Perl installed, as well as the FetchStory module. Now, my question is haha: how do I use the script?

Sat, Oct. 15th, 2011 06:17 pm (UTC)

I'm sorry, I don't understand the question. You use it like you would use any Perl script; by typing on the command-line, and finding the usage information by using perldoc.
I have a feeling that this is a Windows-versus-Linux question; something that I can't answer because I don't know how to do it on Windows.
Have you tried the forums I suggested?

Sat, Oct. 15th, 2011 07:44 pm (UTC)

Whoops, lol, I was half asleep when I asked the question. The answer is so obvious now :P

Thank you for such an awesome tool! It's really saved me hours and hours of manual copy-pasta!!

Sun, Oct. 16th, 2011 12:18 am (UTC)

Oh hooray! I was afraid you might not be able to get it to work, and then it wouldn't be helpful to anybody but me.

Sun, Oct. 16th, 2011 02:42 am (UTC)

I'm having some trouble getting it to download multi-chapter stories from Ashwinder (it'll only dl the first chapter), though it's worked like a dream on those sorts of stories on Fanfiction.net.

Sun, Oct. 16th, 2011 02:54 am (UTC)

It probably is a login issue; sites like Ashwinder often block stories unless you are logged in. The way the script gets around it is to trick the site into thinking that it is your browser, by using the "cookies" that the browser has which record that you've logged in to the site.

What I use is the Cookie-Exporter add-on for Firefox:

First, I log in to the site in question, clicking on the "remember me" (or equivalent) option, so that it sends cookies to the browser; then I use the cookie-exporter to save my cookies to a "cookies.txt" file; and then give the name of that file to the "--wget_cookies" option in the fetch_story script. (The option is only called "wget" cookies because the file is in the same format that wget understands, not that you need to have wget in order to use that option).

No, I don't know why they're called "cookies".

Sun, Oct. 16th, 2011 04:42 am (UTC)

This is so weird, the cookie solution worked for several four to eight chapter long stories, but anything longer and it only exports the first chapter.

Haha, I was so going to ask why they're called cookies, thanks for heading me off :P

Sun, Oct. 16th, 2011 04:59 am (UTC)

Can you give me the URLs of some of the ones that don't work? I'll try downloading them myself. If they work for me, I don't know how to help; if they don't work for me either, then at least I can reproduce the problem and I'll be able to debug it later.

Sun, Oct. 16th, 2011 06:15 am (UTC)

Hmmm, I'm not having any problems with those.
The only thing I can suggest is to log out of Ashwinder, log in again, export your cookies again, and try downloading them again.

Mon, Oct. 17th, 2011 12:28 pm (UTC)

Thanks for all of your help! :D:D

Mon, Oct. 17th, 2011 12:44 pm (UTC)

So, did that work?

Mon, Oct. 17th, 2011 06:36 pm (UTC)

It's not working, even after starting a new firefox session and doing the cookies, but I didn't want to keep bugging you about it, lol.

Mon, Oct. 17th, 2011 09:14 pm (UTC)

Rats. Because I can't reproduce the problem, any suggestions I give you can only be guesses on my part.

The symptoms look like what happens when the script isn't getting the cookies, and is trying to download a story that is rated Lascivious or Wanton. But just because that's what it looks like, doesn't mean that that is the cause. As an analogy of this, consider that a cold and whooping cough have the same initial symptoms but different causes. Diagnosis requires more data.

A few questions for you (in the interests of collecting more data):

* What is the exact command that you are using? (cut and paste it please)
* What are the ratings of the stories that downloaded successfully?
* What are the ratings of the stories that did not download successfully?
* Is Firefox still running when you try to download stories?
* Look at the Firefox settings: Privacy -> History -> Use Custom Preferences For History -> Cookies; what does it say?

Tue, Oct. 18th, 2011 02:06 am (UTC)

God, I am so dumb. This is what I have been doing:

C:\Users\Lily>fetch_story --wget_cookies cookies.txt

C:\Users\Lily>fetch_story [insert a link]

Just now, as I was typing out a response to you, this occurred to me:

C:\Users\Lily>fetch_story --wget_cookies cookies.txt [insert a link]

THAT worked! I didn't realize that I had to fetch the cookies and insert the link in the same command. I can now get the stories that require login!!!

Tue, Oct. 18th, 2011 02:10 am (UTC)

Sometimes it's the simple things...

Yes, the poor script needs to know where the cookies are and what the URL is at the same time, because it can't remember anything between one invocation and the next.

Glad it's working for you now.

Tue, Oct. 18th, 2011 02:42 am (UTC)

This is really interesting. What other types of tasks can Perl scripts perform?

Thank you for being so patient with me! <3<3

Tue, Oct. 18th, 2011 04:20 am (UTC)

All sorts of things! Everything from web applications to system administration. Why? Thinking of learning Perl yourself?

Tue, Oct. 18th, 2011 08:36 pm (UTC)

I was just curious :D, if only we could automate everything!