Register
It is currently Mon Jul 28, 2014 4:20 am

Bash sorting script for large data volumes


All times are UTC - 6 hours


Post new topic Reply to topic  [ 3 posts ] 
Author Message
 PostPosted: Sun Apr 13, 2008 3:41 pm   

Joined: Sun Apr 13, 2008 3:35 pm
Posts: 1
Hello,
I have a question before i start doing any work on my own and possibly waste time. I'm looking to create a scripts (bash/ruby/perl) i previously worked and with simple scripts grep/awk/ls basic loops. It need to go through for example /home/user, find all types of extensions, create a directory for each one and then move any file that has that extension to it, .doc(s) would to /home/user/sorted/doc for example. Is bash best choice for it ? In terms of speed of course, i would be looking to sort anywhere between 50GB-4000GB of files from windows profiles. Thanks.


Top
 Profile  
 PostPosted: Mon Apr 14, 2008 9:46 am   
Site Admin

Joined: Tue May 17, 2005 7:31 pm
Posts: 251
Location: Georgia
bash would do this SUPER easy. we are talking something like:
Code:
mv *.doc /home/user/sorted/doc/
mv *.mp3 /home/user/sorted/mp3/

you could even make it a little better than that... something like:
Code:
#!/bin/bash
# filename: sortit.sh

extenstion="$1"
mkdir -p /home/user/sorted/${extension}
mv *.${extension} /home/user/sorted/${extension}/


then you could run the script like:
Code:
./sortit.sh doc

as long as you ran the script in the directory the files were in...


Top
 Profile  
 PostPosted: Mon Apr 14, 2008 12:13 pm   

Joined: Tue Apr 01, 2008 10:19 am
Posts: 49
If they weren't in the same directory:

Code:
if [ -n "$1" ]; then
    EXTENSION="$*" #CLI arguments
else
    EXTENSION='doc xls mp3' #Default arguments
fi

for ext in $EXTENSION; do
    dir="${HOME}/sorted/${ext}"
    mkdir -p "$dir"
    find . -name "*.$ext" -exec mv "{}" "$dir" \;
done


Note that the extensions given are case sensitive, so you might want to run as sortext doc DOC for example.


Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP