Register
It is currently Thu Dec 18, 2014 6:46 pm

FileList and sed


All times are UTC - 6 hours


Post new topic Reply to topic  [ 7 posts ] 
Author Message
 PostPosted: Sat Jul 17, 2010 5:08 am   

Joined: Sat Jul 17, 2010 4:50 am
Posts: 3
Hello,

First time on this forum. Hope to be back here soon...

I'am doing simple scripts in bash. I use bash because it is the only language I kind of know.

I have 2 directories,called /input and /output. The first one /input is régularly updated with new files from an ftp. I need to copy the file from /input to /output if a value, the $ValueF, is smaller in the newer file. In order to do this I've created a text file called FileDatabase.txt in /output directory where I store the filename and the corresponding ValueF. My script check the ValueF in the FileDatabase.txt and then copy in /output if needed. My problem is that for some reasons I have hard time updating the ValueF in FileDatabase.txt. I'am using a sed command. But this command only works when I am pasting it in the terminal but not inside my script :-/ :-/ :-/
Really !

Here is my script I have simplify for clarity.

Code:
#!/bin/bash

# List all the files in INPUT directory. At each file correspond an ValueF. If the Value F is smaller
# in the INPUT directory, cp file from input directory to OUTPUT directory

partern=pattern
#Directories parameters
input_dir=/home/Data/INPUT
output_dir=/home/Data/OUTPUT


# On va d'abord looper sur tout les fichiers.
find ${input_dir} -name "$pattern" | sort -r > ${input_dir}/Input_filelist.txt
while read input_file;
do
  # Extract filename
  basefilename=`basename "$input_file"`
  echo "$basefilename"
  #Extract ValueF
  Input_ValueF=`grep ValueF "$basefilename" | cut -d"=" -f2`
 
  # remove leading 0
  ((Input_ValueF=10#${Input_ValueF}))

  # Output file (same name as input file, different directory
  output_file=${output_dir}/$basefilename
 
  if [ ! -f ${output_file} ]; then
    # First time, just copy to output dir
    echo "First Time"
    cp ${input_file} ${output_file}
    # File up database
    echo "${output_file} ${Input_ValueF}" >> ${output_dir}/FileDatabase.txt
    sort -o ${output_dir}/FileDatabase.esv ${output_dir}/FileDatabase.txt
  else
    echo "Check FValue"
    # Output file already exists.
    # Check ValueF reading FileDatabase.txt
    ValueF=`grep ${output_file} ${output_dir}/FileDatabase.txt | cut -d" " -f2`

    echo "$Input_ValueF vs $ValueF "
   # Compare both Value F
    if [[ "$Input_ValueF" -lt "$ValueF" ]]; then
      echo "better one"
      # Overwrite old file with new one
      cp ${input_file} ${output_file}
      # Update database using sed command
      sed  '/${output_file}/ s/ ${ValueF} / ${ValueF} /' ${output_dir}/FileDatabase.txt > new
      #wait ${!}
      mv new ${output_dir}/FileDatabase.esv
    fi
  fi
  echo
done < ${input_dir}/Input_filelist.txt


What is wrong with my sed command ??
If you have a better way and more reliable way to do this kind of thing with file, I'll be pleased to learn. ( shall I use a database ? how does it work ? )

THanks in advance

RENOO


Top
 Profile  
 PostPosted: Sat Jul 17, 2010 5:27 am   

Joined: Mon Mar 02, 2009 3:03 am
Posts: 579
the problem comes from sed's separator, change to some character that is not in file's path (something like @)


Top
 Profile  
 PostPosted: Sat Jul 17, 2010 11:28 am   

Joined: Sat Jul 17, 2010 4:50 am
Posts: 3
Hi,

Thank you very much for your reply.
You've mentionned a very good point here. I thought we had it.

It did the following switch

Code:
sed  '/${output_file}/ s@ ${ValueF} @ ${Input_ValueF} @' ${output_dir}/FileDatabase.txt > new



As modifying the 2 firsts "/" gives an error.
Unfortunatly, it is still not working. The "new" file is similar than the FileDatabase.txt ??

What am I doing wrong ?


Top
 Profile  
 PostPosted: Sat Jul 17, 2010 1:04 pm   
User avatar

Joined: Sun Jun 27, 2010 12:57 am
Posts: 192
RENOO wrote:
Code:
sed  '/${output_file}/ s@ ${ValueF} @ ${Input_ValueF} @' ${output_dir}/FileDatabase.txt > new

In your sed there are spaces before and after both ${ValueF} and ${Input_ValueF}. Are there spaces there in the FileDatabase.txt file (especially the spaces behind the values).


Top
 Profile  
 PostPosted: Sat Jul 17, 2010 2:03 pm   

Joined: Mon Mar 02, 2009 3:03 am
Posts: 579
Code:
sed  '\@${output_file}@s@${ValueF}@${Input_ValueF}@' ${output_dir}/FileDatabase.txt > new


Top
 Profile  
 PostPosted: Thu Jul 22, 2010 12:27 pm   

Joined: Sat Jul 17, 2010 4:50 am
Posts: 3
Hi there,

I confess i have not been very smart here. I don't need to create the databased.txt as the FValue is contain inside the file itself. I just need to read the file itself instead of creating another file and modifying the FValue.
My mistake here. But still I very thank you for you help. With the time I am getting better...

@Watael
Why do you only "escape" (\) the first @ in your code ? How shall I understand this ?

Thanks again all of you.
RENOO


Top
 Profile  
 PostPosted: Fri Jul 23, 2010 2:18 am   

Joined: Mon Mar 02, 2009 3:03 am
Posts: 579
GNU Sed Manual:
Quote:
3.5 The s Command
[...]The / characters may be uniformly replaced by any other single character within any given s command. The / character (or whatever other character is used in its stead) can appear in the regexp or replacement only if it is preceded by a \ character.


Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Google [Bot] and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP