Register
It is currently Sat May 18, 2013 5:12 pm

Bash sorting script for large data volumes

Board index » Bash Shell Basics » Programming

All times are UTC - 6 hours


Post new topic Reply to topic  Page 1 of 1
  [ 3 posts ] 
  Print view Previous topic | Next topic 
Author Message
rr997
 PostPosted: Sun Apr 13, 2008 3:41 pm   

Joined: Sun Apr 13, 2008 3:35 pm
Posts: 1
Hello,
I have a question before i start doing any work on my own and possibly waste time. I'm looking to create a scripts (bash/ruby/perl) i previously worked and with simple scripts grep/awk/ls basic loops. It need to go through for example /home/user, find all types of extensions, create a directory for each one and then move any file that has that extension to it, .doc(s) would to /home/user/sorted/doc for example. Is bash best choice for it ? In terms of speed of course, i would be looking to sort anywhere between 50GB-4000GB of files from windows profiles. Thanks.
Top
 Profile  
jbsnake
 PostPosted: Mon Apr 14, 2008 9:46 am   
Site Admin

Joined: Tue May 17, 2005 7:31 pm
Posts: 251
Location: Georgia
bash would do this SUPER easy. we are talking something like:
Code:
mv *.doc /home/user/sorted/doc/
mv *.mp3 /home/user/sorted/mp3/

you could even make it a little better than that... something like:
Code:
#!/bin/bash
# filename: sortit.sh

extenstion="$1"
mkdir -p /home/user/sorted/${extension}
mv *.${extension} /home/user/sorted/${extension}/


then you could run the script like:
Code:
./sortit.sh doc

as long as you ran the script in the directory the files were in...
Top
 Profile  
Daenyth
 PostPosted: Mon Apr 14, 2008 12:13 pm   

Joined: Tue Apr 01, 2008 10:19 am
Posts: 49
If they weren't in the same directory:

Code:
if [ -n "$1" ]; then
    EXTENSION="$*" #CLI arguments
else
    EXTENSION='doc xls mp3' #Default arguments
fi

for ext in $EXTENSION; do
    dir="${HOME}/sorted/${ext}"
    mkdir -p "$dir"
    find . -name "*.$ext" -exec mv "{}" "$dir" \;
done


Note that the extensions given are case sensitive, so you might want to run as sortext doc DOC for example.
Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 1
  [ 3 posts ] 

Board index » Bash Shell Basics » Programming

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 4 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP