?

Log in

Thu, Oct. 13th, 2011, 09:51 pm
Amanda K: Question!!!

Has anyone found a good way to categorize their fics? Like if someone comes on here and says "I need a fic that involves kids" do you have a way to quickly find the name and link? I am becoming frustrated because I see requests I could answer, but I cant keep it all straight. Any suggestions? There are simply too many to keep them in my mind.

Sun, Oct. 23rd, 2011 10:40 am (UTC)
vkjrock_gakuhai

Hi, I've installed perl and the script, but whenever I run cmd
C:/users/gakuhaitsu>fetch_story link
it always comes up as failed fetching [link] 400/ URL must be absolute at C:/Perl/sit/lib/WWW/FetchStory/Fetcher.pm line 526
or 404 not found... line 526
another questions is for:
C:\Users\gakuhaitsu>fetch_story --wget_cookies cookies.txt [insert a link]
where are we supposed to place the cookies.txt file? I d/led the mozilla add-on and extracted the cookies.txt file, but where are we supposed to place/indicate that?
So far I've tried multiple ashwinder (all different ratings) and fanfiction sites, but all of them have returned as errors. May you enlighten me as what I'm doing wrong? Brilliant, brilliant program by the way, reading through all your documentation and source files, you've made so many useful options/data to extract for us, and included so many sites. Gah, too bad it wasn't in java/phython/c/turing... the only languages I'm fluent in.
If you have time, can you type out an example command line w/ the site and using the toc and --wget_cookies command? Thanks, oh, I just checked out your site from your help file, and found that it's been one of the SSHG fanfic recs site I've been using for quite a while. ^_^ You must dedicate a whole lot of time to your multiple fandoms.

Sun, Oct. 23rd, 2011 08:32 pm (UTC)
vkjrock_gakuhai

okay, that was really weird. Everything worked fine this morning... Hmm... and never mind about the cookies question, I definitely wasn't in my right mind last night at 5 am. Whoops.
Um, question though. Everything seems to be working okay (except TOC chapter titles every once in a while and ashwinder characters are returned as ARRAY(0x31149e4)^_^) but every ashwinder story returns HTML parser error: Unexpected end tag: i "...."</i> or something like that. Like starting or ending of italized stuff, or bolded
stuff. Oh, and there's also Unexpected end tag: font. The resulting htmls retrieved are fine though, so no matter. Once again, thank you for making this awesome, awesome program. Think I'll go give perl a shot (or a try at least), after I master C++ ^_^ Thank you <3

Mon, Oct. 24th, 2011 01:58 pm (UTC)
kerravonsen

except TOC chapter titles every once in a while
I think that's because it gets the titles from the "title" tag and sometimes people don't put the right info in there.

and ashwinder characters are returned as ARRAY(0x31149e4)
Ouch. That's definitely a bug.

but every ashwinder story returns HTML parser error: Unexpected end tag: i "...." or something like that. Like starting or ending of italized stuff, or bolded stuff. Oh, and there's also Unexpected end tag: font.</i>

That is probably because the actual HTML on the site is malformed; it's good enough for most browsers to cope with, so one doesn't notice. However, I found when building EPUB files that EPUB browsers are more fussy, so I needed to add in some code that would force the HTML to be syntactically correct, and once I'd added that code in, I figured it made sense to fix up the HTML all the time, rather than just when creating EPUB files. But when the tidy-the-HTML module encounters incorrect HTML, it complains. Hence the "HTML parser error" messages.

Once again, thank you for making this awesome, awesome program.

\o/ I'm so glad someone besides me is finding it useful.

Think I'll go give perl a shot (or a try at least), after I master C++
(makes encouraging noises)

Once you've done C and C++, I think Perl will be easy. I like Perl's objects more than C++ objects; while in some ways C++ is more powerful in that regard, that power makes it more complicated, and thus more difficult to debug when things go wrong.
Other things I like about Perl are
1. self-documenting code is easy to do with Perl.
2. http://www.cpan.org/

Mon, Oct. 24th, 2011 08:12 pm (UTC)
vkjrock_gakuhai

wow. thanks for your lengthy reply below ^_^ Second Life is one awesome story, although I've yet to catch up on this exchange round stories. Generally, I like to wait until the author's been revealed, so that my database and naming system for the story authors don't need to be corrected the second time around. Although, some of the prompts and summary look really, really intriguing. :)

Hmm... I don't think its always the url typo, since I was copying and pasting the stuff... but it was just my desktop acting up that night? Since the same command worked the next day.

Oh, and thanks for your explanations of the parser errors, everything makes a lot more sense now. Especially since I went through and read your code and comments again. Very user friendly.

I've been using it all yesterdays, going back and downloading some insanely long stories from ashwinder (before I was saving individual html pages, or copying and pasting into one document before converting to a html). So yep, THANK YOU.

cpan looks really useful, so many codes that people have already made which can be implemented into your own stuff.

Question: Is there a way to generate the entire story into one html file with links to each chapter on the top of the page (TOC area)? I've been downloading with the html then putting the TOC page into Calibre, converting and exporting into htmlz (it automatically imports/generates/merges the rest of the chapter files), unzipping that file, and grabbing the index.html file from one of the subfolders in the unzipped folder. This works well, just, a bit lengthy. Is there a section of the code which can do the above?
(There is also a program VHtmlMerger which does the merging of the files, but the TOC references don't work since it'll link to the old individual html chapter file, instead of a position in the new file which has the entire story)... Sorry, if that was confusing or lengthy. You don't have to reply if you don't have time, or didn't understand my rambling.

(makes encouraging noises)
Accepts the noises ;)

Mon, Oct. 24th, 2011 10:27 pm (UTC)
kerravonsen

Generally, I like to wait until the author's been revealed, so that my database and naming system for the story authors don't need to be corrected the second time around.

Since I file my fic by fandom, it isn't so much of a hassle changing an author from "Unknown" to the real author name. Besides, I'm playing exchange bingo this time around... (but I'm getting further and further behind! I was four stories behind for a while, now I'm eight stories behind!)

but it was just my desktop acting up that night? Since the same command worked the next day.

Sounds like it. Perhaps your network connection was playing up, what with all those 404-not-found errors.

Oh, and thanks for your explanations of the parser errors, everything makes a lot more sense now.

I should probably put a note in the docs about it.

Especially since I went through and read your code and comments again. Very user friendly.

Thank you!

cpan looks really useful, so many codes that people have already made which can be implemented into your own stuff.

Yes, exactly!

Is there a way to generate the entire story into one html file with links to each chapter on the top of the page (TOC area)?

I'm afraid I haven't implemented anything like that. I found it easier to create an EPUB file instead (the --epub option). If you're using Calibre, maybe you could generate an EPUB file instead, and import that into Calibre?

Accepts the noises ;)
(grin) My work here is done! (billows away)

Mon, Oct. 24th, 2011 01:43 pm (UTC)
kerravonsen

it always comes up as failed fetching [link] 400/ URL must be absolute at C:/Perl/sit/lib/WWW/FetchStory/Fetcher.pm line 526
or 404 not found... line 526


That sounds like you aren't giving the full, correct URL. For example, to download one of the recent Exchange fics, I would do this:

fetch_story http://sshg-exchange.livejournal.com/262715.html

Another thing to keep in mind is that sites like Ashwinder that have "?" in the URL, it may work better to put the URL inside single quotes, because on some systems, a question mark is a special character which is treated differently by the command interpreter.

where are we supposed to place the cookies.txt file?

Anywhere you like, so long as you can remember the full path to its location.
Myself, I just put it in my home directory.

So, for you, you might save it as C:\Users\gakuhaitsu\cookies.txt

Then you would go into that directory and...

C:\Users\gakuhaitsu> fetch_story --wget_cookies cookies.txt 'http://ashwinder.sycophanthex.com/viewstory.php?sid=17863'

(that's the URL for "Second Life", BTW)

Brilliant, brilliant program by the way, reading through all your documentation and source files, you've made so many useful options/data to extract for us, and included so many sites.

Thank you!
It began as so many projects do; I wanted to make downloading fanfic easier for myself, so I wrote a script, and then I re-wrote it, and then I added to it, and then I realized other people might find it useful, so I re-wrote it again to make it more generic, and then I added to it some more...

Gah, too bad it wasn't in java/phython/c/turing... the only languages I'm fluent in.

Well, Perl isn't so hard, especially if you know C...
But, yeah, after a while one doesn't want to learn yet another language... that's why I haven't learned Python, because things I could do in Python, I could do equally well in Perl, and I already know Perl.

I just checked out your site from your help file, and found that it's been one of the SSHG fanfic recs site I've been using for quite a while. ^_^ You must dedicate a whole lot of time to your multiple fandoms.

I like reading fanfic! And I like sharing good fanfic with others. And I've been doing that since 1999...