Register
It is currently Sat Oct 25, 2014 9:41 pm

Loop and output to multiple files


All times are UTC - 6 hours


Post new topic Reply to topic  [ 8 posts ] 
Author Message
 PostPosted: Fri Aug 27, 2010 6:48 am   

Joined: Fri Aug 27, 2010 6:26 am
Posts: 4
Hello - I'm a real beginner with Bash scripting, as you will see : ) but maybe someone can help me out, I have a script as below

#! /bin/bash
ls | grep '\.txt' | sort > filelist
filecontent=( `cat "filelist" `)
for i in "${filecontent[@]}"
do echo $i >> filelist_a
done

I have a text file called "filelist" with a list of sequential names in i.e
file0001
file0002
file0003
etc

I want to output to a set of other files i.e filelist_a, filelist_b etc
I need the script to loop through the "filelist" file and append to "filelist_a", "filelist_b" etc
At the moment I can do it for the one file "filelist_a", but thats as for as I can get at the moment. I'm thinking I need to put in an array for these filelist i.e declare - a filelists=('filelist_a' 'filelist_b' 'filelist_c')

Can someone give me an example of how i can amend my current script to do that


Last edited by daddysauce on Fri Aug 27, 2010 1:39 pm, edited 1 time in total.

Top
 Profile  
 PostPosted: Fri Aug 27, 2010 11:38 am   
User avatar

Joined: Sun Jun 27, 2010 12:57 am
Posts: 192
That's a very complex way of copying lines of text to another file or files.
I would suggest something more simple like this:
Code:
cat filelist.txt | tee -a "filelist_a.txt" "filelist_b.txt" >> "filelist_c.txt"


Top
 Profile  
 PostPosted: Fri Aug 27, 2010 1:11 pm   

Joined: Fri Aug 27, 2010 6:26 am
Posts: 4
Hi Patsie, thanks for the reply
simple would definitely be good,
the list needs to be divided between the files though - this code seems to copy the whole list to each one


Top
 Profile  
 PostPosted: Fri Aug 27, 2010 1:26 pm   
User avatar

Joined: Sun Jun 27, 2010 12:57 am
Posts: 192
If you want to split the contents of one input file over multiple output files, then please provide the 'algorithm' of how you want to distribute them.
Should it be a simple round robin, purely random, based on some calculation of the string? Without that information there is little to suggest to you.


Top
 Profile  
 PostPosted: Fri Aug 27, 2010 1:42 pm   

Joined: Fri Aug 27, 2010 6:26 am
Posts: 4
round robin sounds right - like a dealer handing out the cards to the players, till there are no cards remaining.


Top
 Profile  
 PostPosted: Fri Aug 27, 2010 1:52 pm   
User avatar

Joined: Sun Jun 27, 2010 12:57 am
Posts: 192
Here's a way to distribute each line of an input file in a round-robin manner over a number of output files

Code:
#!/bin/bash

ifile="filelist"
ofile=("filelist_a" "filelist_b" "filelist_c")

numfiles=${#ofile[@]}

while read line; do
  echo "$line" >> "${ofile[$((num++))]}"
  ((num%=numfiles))
done < "$ifile"


Top
 Profile  
 PostPosted: Fri Aug 27, 2010 3:38 pm   

Joined: Fri Aug 27, 2010 6:26 am
Posts: 4
whoop! whoop! that works beautifully patsie - i owe you a beer! - thanks for taking the time.


Top
 Profile  
 PostPosted: Mon Aug 30, 2010 4:34 am   
User avatar

Joined: Tue Apr 27, 2010 2:28 pm
Posts: 172
Location: Czech Republic
daddysauce wrote:
whoop! whoop! that works beautifully patsie - i owe you a beer! - thanks for taking the time.

Or, if the filelist tends to be long, you can try a much faster solution (approx. 200 times faster on my machine):
Code:
#! /bin/bash

ifile=$1
ofile=("filelist_A" "filelist_B" "filelist_C")

numfiles=${#ofile[@]}

for ((n=1;n<=numfiles;n++)) ; do
    sed -n "$n~$numfiles"p $ifile > ${ofile[n-1]}
done


Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP