BashScripts.org
http://bashscripts.org/forum/

Bash sorting script for large data volumes
http://bashscripts.org/forum/viewtopic.php?f=16&t=501
Page 1 of 1

Author:  rr997 [ Sun Apr 13, 2008 3:41 pm ]
Post subject:  Bash sorting script for large data volumes

Hello,
I have a question before i start doing any work on my own and possibly waste time. I'm looking to create a scripts (bash/ruby/perl) i previously worked and with simple scripts grep/awk/ls basic loops. It need to go through for example /home/user, find all types of extensions, create a directory for each one and then move any file that has that extension to it, .doc(s) would to /home/user/sorted/doc for example. Is bash best choice for it ? In terms of speed of course, i would be looking to sort anywhere between 50GB-4000GB of files from windows profiles. Thanks.

Author:  jbsnake [ Mon Apr 14, 2008 9:46 am ]
Post subject: 

bash would do this SUPER easy. we are talking something like:
Code:
mv *.doc /home/user/sorted/doc/
mv *.mp3 /home/user/sorted/mp3/

you could even make it a little better than that... something like:
Code:
#!/bin/bash
# filename: sortit.sh

extenstion="$1"
mkdir -p /home/user/sorted/${extension}
mv *.${extension} /home/user/sorted/${extension}/


then you could run the script like:
Code:
./sortit.sh doc

as long as you ran the script in the directory the files were in...

Author:  Daenyth [ Mon Apr 14, 2008 12:13 pm ]
Post subject: 

If they weren't in the same directory:

Code:
if [ -n "$1" ]; then
    EXTENSION="$*" #CLI arguments
else
    EXTENSION='doc xls mp3' #Default arguments
fi

for ext in $EXTENSION; do
    dir="${HOME}/sorted/${ext}"
    mkdir -p "$dir"
    find . -name "*.$ext" -exec mv "{}" "$dir" \;
done


Note that the extensions given are case sensitive, so you might want to run as sortext doc DOC for example.

Page 1 of 1 All times are UTC - 6 hours
© 2000, 2002, 2005, 2007 phpBB Group • http://www.phpbb.com