Register
It is currently Mon Nov 24, 2014 3:22 am

website map


All times are UTC - 6 hours


Post new topic Reply to topic  [ 3 posts ] 
Author Message
 PostPosted: Thu Nov 22, 2012 4:06 am   

Joined: Thu Nov 22, 2012 4:03 am
Posts: 1
I have to do script in bash who do in file website map.

For example:
I use this script:

Code:
./website.sh www.adress.com


as te result I see in new file all links in pages in this domain.
for example:

Code:
www.adress.com/index.htm

www.adress.com/page1.htm
.
.
.
etc.


I don't now how do it.


Top
 Profile  
 PostPosted: Wed Jan 02, 2013 1:04 pm   

Joined: Fri Jul 30, 2010 3:19 pm
Posts: 6
Use lynx:

Code:
lynx -dump "$url" |
   sed  -e '1,/^References$/d' -e 's/^ *[0-9]*. //'


Top
 Profile  
 PostPosted: Sun May 05, 2013 9:39 pm   
Site Admin
User avatar

Joined: Sun May 15, 2005 9:36 pm
Posts: 673
Location: Des Moines, Iowa
wget -r --spider -b -o /var/tmp/wget.log http://target.site.com/

-r - Recursive download
--spider - Check downloaded content for new links, then discard
-b - Run in the background
-o /var/tmp/wget.log - Target log file
http://target.site.com/ - Replace with the site you want to check


Top
 Profile WWW  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Yahoo [Bot] and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP