| View previous topic :: View next topic |
| Author |
Message |
Nightstar
Joined: 14 Nov 2003
Posts: 11031
Location: Somewhere between here and there
|
 Posted: Tue Mar 08, 2011 8:39 pm Post subject: Need Back-Up Advice |
 |
|
In light of the recent TAC scare--how can I go about backing up threads or snatching and saving threads? I'm hoping to find all the stuff I've written about farming in order to use it for a book.
I'd be mighty appreciative of advice on how I can go about finding all the relevant posts, and then what I can do to save them.
_________________
Masquerading as a normal person day after day is exhausting.
Magpie House Design: The Moon And Beyond! http://www.hirezfox.com/km/ |
|
| Back to top |
|
 |
LionHeart
Joined: 10 Sep 2008
Posts: 667
Location: Behind a keyboard, in Australia
|
| Posted: Tue Mar 08, 2011 11:46 pm Post subject: |
|
|
I'd suggest using the forum search function, and then copy-pasting the text of the posts into a word processor (or simply a plain old text file) to start with.
When TAC went down, I was able to save the text of the most recent thread here (NeoCTC #159) by using Google and selecting the cached version of the page, then copy-pasting into an OpenOffice word processor file.
You can see the results of my efforts here:
http://www.users.on.net/~richrocks/files/CTC/
_________________
"We demand rigidly defined areas of Doubt and Uncertainty!"
My Scribble Pad - where I write things. |
|
| Back to top |
|
|
mako
Joined: 26 Jul 2003
Posts: 1318
Location: Left Coast
|
| Posted: Wed Mar 09, 2011 1:21 am Post subject: |
|
|
Clever! 
I would use wget:
http://www.ehow.com/how_7211150_download-pages-linux.html
Anyone who has a linux box and some time should be able to do this for you and send you a .zip of the entire Magpie House forum.
Alternatively, you can pull on Scott's g33k-fu and ask him to install wget onto one of your Windows PCs and run it from there.
CYa!
Mako
_________________
More German Shepherds in Comics! |
|
| Back to top |
|
|
LionHeart
Joined: 10 Sep 2008
Posts: 667
Location: Behind a keyboard, in Australia
|
| Posted: Wed Mar 09, 2011 1:48 am Post subject: |
|
|
Well I just tried right-clicking on the page, and using "Save page as..." in Firefox, which got me an HTML file of the current page. Seemed to work OK...
_________________
"We demand rigidly defined areas of Doubt and Uncertainty!"
My Scribble Pad - where I write things. |
|
| Back to top |
|
|
Nighthawke
Joined: 28 Oct 2007
Posts: 2054
Location: In your hard drive, raising all sorts of hell.
|
| Posted: Wed Mar 09, 2011 1:53 am Post subject: |
|
|
Considering the lack of other activity on TAC, it maybe prudent to set up an ace in the hole and back up the data we have on this forum. His TAC blog has not budged for 3 years, maybe longer.
_________________
NH
 |
|
| Back to top |
|
|
Won-Tolla
Joined: 11 Feb 2004
Posts: 5647
Location: Norway
|
| Posted: Wed Mar 09, 2011 2:04 am Post subject: |
|
|
| mako wrote: | I would use wget:
http://www.ehow.com/how_7211150_download-pages-linux.html
Anyone who has a linux box and some time should be able to do this for you and send you a .zip of the entire Magpie House forum.
Alternatively, you can pull on Scott's g33k-fu and ask him to install wget onto one of your Windows PCs and run it from there. |
Or just try the windows version. Or one of the many alternatives that exist for different OSes.
(I had Wget on my Amiga too. Now I use other tools...) |
|
| Back to top |
|
|
Nightstar
Joined: 14 Nov 2003
Posts: 11031
Location: Somewhere between here and there
|
| Posted: Wed Mar 09, 2011 11:46 am Post subject: |
|
|
Good advice. Nowe if we all did it, we'd have multiple copies...
"Jesus Saves. Often."
_________________
Masquerading as a normal person day after day is exhausting.
Magpie House Design: The Moon And Beyond! http://www.hirezfox.com/km/ |
|
| Back to top |
|
|
Nighthawke
Joined: 28 Oct 2007
Posts: 2054
Location: In your hard drive, raising all sorts of hell.
|
| Posted: Wed Mar 09, 2011 12:46 pm Post subject: |
|
|
| Nightstar wrote: | Good advice. Nowe if we all did it, we'd have multiple copies...
"Jesus Saves. Often."
|
It's best to stay consistent and allow the forum managers to perform this work.
Also, someone needs to notify the forum admin as to our intents so they do not become alarmed at the level of database activity as the backup applications spiders it.
_________________
NH
|
|
| Back to top |
|
|
marmoe
Joined: 17 Oct 2003
Posts: 2918
Location: Germany
|
| Posted: Wed Mar 09, 2011 1:58 pm Post subject: |
|
|
I'm on it. However, wget on its own has problems with the multiple links to the same content. The link and naming structure of a phpBB board is wget unfriendly. You need to tell it exactly what to download and keep it wellbehaved. We're talking several thousand pages and a few 100MB for the HTML. Can you say wget DOS "attack". Oh, and I'm on it.
_________________
TAC registration help squad. Just drop me a line here.
How-to: Killfile |
|
| Back to top |
|
|
wizzard_o
Joined: 04 Jun 2006
Posts: 4481
Location: On top of a hill in Buckinghamshire, England.
|
| Posted: Wed Mar 09, 2011 2:08 pm Post subject: |
|
|
It's a shame we just cant get a link to the database and download that instead 
Wizz.
_________________
Avatar by Koz, Signature by White Pony. |
|
| Back to top |
|
|
Nightstar
Joined: 14 Nov 2003
Posts: 11031
Location: Somewhere between here and there
|
| Posted: Thu Mar 10, 2011 5:23 pm Post subject: |
|
|
Marmoe, you da man! *hugs!*
Can we start a fund to help defray the cost of ghosting the Forums?
_________________
Masquerading as a normal person day after day is exhausting.
Magpie House Design: The Moon And Beyond! http://www.hirezfox.com/km/ |
|
| Back to top |
|
|
marmoe
Joined: 17 Oct 2003
Posts: 2918
Location: Germany
|
| Posted: Thu Mar 10, 2011 6:35 pm Post subject: |
|
|
None necessary, Kathy, but thank you anyway. The scripts run in the background and I'm using my flatrate for the internet access. Getting everything working took something like four hours time.
_________________
TAC registration help squad. Just drop me a line here.
How-to: Killfile |
|
| Back to top |
|
|
pirx
Joined: 28 Nov 2005
Posts: 121
Location: Hamburg, Germany
|
| Posted: Fri Mar 11, 2011 1:40 am Post subject: |
|
|
| marmoe wrote: | | None necessary, Kathy, but thank you anyway. The scripts run in the background and I'm using my flatrate for the internet access. Getting everything working took something like four hours time. |
This weekend, I'll try to do a complete mirror with httrack (which also fixes any links) on my Linux server. If that works as it should, we'll have a browsable archive. I would make that available on the net.
Now this would NOT have the user login data, so It would just be open for all (as the forum now is anyway) for reading, end completely read-only. Does any artist out there have an issue with that? In the end, I would be republishing copyrighted content, although only the forumites here would likely know as I do not intend to advertise the link (nor do any advertising myself; either keep the ad links that are there on TAC as is or put grey boxes there).
Pirx |
|
| Back to top |
|
|
Nightstar
Joined: 14 Nov 2003
Posts: 11031
Location: Somewhere between here and there
|
| Posted: Fri Mar 11, 2011 10:40 am Post subject: |
|
|
The more back-ups we can get, the better off we'll be; a read-only archive would at least preserve the history of the forums, and I presume could be periodcally updated?
_________________
Masquerading as a normal person day after day is exhausting.
Magpie House Design: The Moon And Beyond! http://www.hirezfox.com/km/ |
|
| Back to top |
|
|
mako
Joined: 26 Jul 2003
Posts: 1318
Location: Left Coast
|
| Posted: Mon Mar 21, 2011 5:18 am Post subject: |
|
|
| marmoe wrote: | | None necessary, Kathy, but thank you anyway. The scripts run in the background and I'm using my flatrate for the internet access. Getting everything working took something like four hours time. |
Marmoe,
You da gecko!
wget has throttling built in to avoid blasting the target server with a ginormous amount of work and bandwidth burn.
If you're available to zip or tgz the backup, I'd be happy to stuff it onto the HRF server for handy/safe keeping.
I could also just run a cron job from the server directly using your script(s) and tarball the result. The box is running CentOS 5 (a Redhat RHEL clone).
Let me know if this is a workable idea...
Thanks!
Mako
_________________
More German Shepherds in Comics! |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
