mirror a site with wget and download static media

Posted by mountainswhim on Server Fault See other posts from Server Fault or by mountainswhim
Published on 2010-02-25T02:20:01Z Indexed on 2010/04/17 21:43 UTC
Read the original article Hit count: 188

Filed under:

wget

I'd like to mirror a site with wget and convert all the links to the local copies I've downloaded. So far that's easy all I have to do is wget -mk http://site.com

However all of the static media is located in a different domain, if I follow all of the foreign hosts using the -H option it will download that static media files, but it will also download anything else externally linked.

Is there a way to specify 'only download foreign hosts specified in list X'?

Related posts about wget

Make wget not download files larger than X size

as seen on Super User - Search for 'Super User'
Okay, I give up. How do I size limit which files are downloaded, like say I don't want any files bigger than 2 MB? >>> More
How to start using Wget?

as seen on Super User - Search for 'Super User'
Please, forgive me for asking this question. Usually I would try to learn thisngs myself first before bothering others, but my situation is urgent - if I don't act now and don't download all my family pictures from this website, it will be closed in about two weeks from now and I will loose all of… >>> More
wget mirroring, subdomains and directories and cookies

as seen on Server Fault - Search for 'Server Fault'
Hi all, I have an account on a web page that is now "full" (ie I have used up all my allocated space) and I would like to make a mirror of that site. wget seems like the thing to use. The problem is that I would only like to mirror the sites the lie within this directory http://user.domain.com/room/2324343/transcript/… >>> More
How can I install things in Linux with *no yum* and *no wget*?

as seen on Super User - Search for 'Super User'
I'm a newbie to Linux (that mainly uses Windows and Mac OS X) needing some advice. I was trying to install git on a Linux machine today, and encountered some problems: Not knowing the version of the installed OS, I've opened the /proc/version file which said: Linux version 2.6.9-42.0.2.ELsmp (bhcompile@ls20-bc1-13… >>> More
Getting wget to dowload only files with specific name patterns

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to use wget to DL some files. I want to DL only files whose name that fit a certain pattern, e.g. ???.txt and not any other *.txt files. Can this be done with wget? I could only find a way to --accept/--reject files based on the extension. Thanks! >>> More

Developer IT

mirror a site with wget and download static media - Developer IT

mirror a site with wget and download static media

wget

Related posts about wget

Make wget not download files larger than X size

How to start using Wget?

wget mirroring, subdomains and directories and cookies

How can I install things in Linux with no yum and no wget?

Getting wget to dowload only files with specific name patterns

Categories cloud