I have written a script to download and web links that are pasted in a simple text file.
It works through the list, downloading each one to a directory (created by date) and deleting the line of the downloaded link.
My aim is to have this run every night via 'cron',
Here it is:
Code:
#!/bin/bash
#
# ----------
# Successively download each link in a file,
# and remove each link after downloading.
# --------------------
# VARIABLES
# --------------------
#
# set the download file
links=~/wget.links
i=1
# set the download directory
directory=$(date "+%a.%d.%b.%y")
#
# --------------------
# FUNCTIONS
# --------------------
#
# set the function to download each line of the download file
function download_links()
{
echo "---------- downloading "$each" ----------"
echo ""
wget -c --tries=5 --waitretry=1 --directory-prefix $directory $each
echo ""
}
#
#
# set the function to delete the links just downloaded
function delete_lines()
{
echo "removing "$each" from file"
sed -i ''$i'd' $links
echo ""
}
#
# --------------------
# MAIN BODY
# --------------------
#
# for each line fron file...
for each in $(cat $links)
do
# call the function to download each line
download_links
# if wget doesn't return true then don't delete the link from the file
# ------------------------
# if [ ]
# then
# delete_lines
# else
# i=`expr $i + 1`
# fi
# -------------------------
# call the function to remove downloaded file each time
delete_lines
# if there are no links left to download...
if [ 0 -eq $(cat $links | wc -l) ]
# then print the following
then
echo "all files finished downloading"
# otherwise, we still have the following to download
else
echo "still have the following to download:"
cat $links
fi
done
It works well, except if the download doesn't work then I would like it not to delete the link and print an error message, skipping to the next one and so on.
Is there any way I can get wget to return 'false' to the "if [ ]" condition?
Or is there a better way to do it?
Cheers
lol
