BashScripts.org
http://bashscripts.org/forum/

Merge AVIs with space in filenames
http://bashscripts.org/forum/viewtopic.php?f=8&t=1172
Page 1 of 1

Author:  partimers [ Sat Aug 21, 2010 4:37 pm ]
Post subject:  Merge AVIs with space in filenames

Hello,

I am having a bad time working around space in filenames trying to merge AVIs with avimerge. Let me describe what I want.

Suposing I have files like this in a directory:
Code:
090103 Arriving at Home.avi
090110 01 Barbecue at Sam's.CD01.avi
090110 01 Barbecue at Sam's.CD02.avi
090110 01 Barbecue at Sam's.CD03.avi
090110 06 Pier!.avi
090110 07 Boat ride.CD01.avi
090110 07 Boat ride.CD02.avi

Files named *.CD0X.avi are parts of a same movie that I want to join in a single avi. That can be accomplished in the command-line using avimerge with this command:
Code:
avimerge -i 090110\ 07\ Boat\ ride.CD01.avi 090110\ 07\ Boat\ ride.CD02.avi -o 090110\ 01\ Barbecue\ at\ Sam\'s.avi

I wrote a script that would identify all *.CD01.avi files in the directory, count how many *.CD* would there be for that particular movie (2 for the Boat ride case above), and iterate a loop to build the command-line for avimerge. The problem is that I have no clue on how to tackle the dreaded spaces and other special chars, and avimerge is not receiving the file names properly to do its work. Please do not tell me "get rid of spaces and other special chars" as they are mandatory.

Here is my pathetic try:
Code:
#!/bin/sh

find . -name "*.CD01*" | while read FILE
do   
   NAME=${FILE%.CD01.avi}
   NUMFILES=`ls -1 "$NAME"* | wc -l`
   OUTPUTFILE=$NAME.avi
   INPUTFILES=''
   COUNTER=1
   while [  $COUNTER -le $NUMFILES ]; do
      THISFILE=$NAME".CD"`printf "%02d" $COUNTER`".avi"
      INPUTFILES=$INPUTFILES" ""$THISFILE"
      COUNTER=`expr $COUNTER + 1`
   done
   avimerge -i $INPUTFILES -o $OUTPUTFILE
done


I would be very grateful if somebody could help me with this one.

Author:  Patsie [ Sun Aug 22, 2010 12:48 am ]
Post subject:  Re: Merge AVIs with space in filenames

Yeah, with spaces, quoting can be a real b*tch ;)
I've adapted your original script a bit and hope it works for you. (couldn't really test it here without avimerge, but I think it should do the trick)

Code:
#!/bin/sh

for FILE in *.CD01.avi
do
   NAME="${FILE%.CD01.avi}"
   OUTPUTFILE="$NAME.avi"
   avimerge -i "${NAME}".CD[0-9][0-9].avi -o "$OUTPUTFILE"
done

Author:  partimers [ Sun Aug 22, 2010 6:56 am ]
Post subject:  Re: Merge AVIs with space in filenames

Thank you very much! I worked like a charm. Elegant and simple as all good scripts should be.

For my future knowledge, a question: what happens exactly in the line, regarding the expansion of "${NAME}".CD[0-9][0-9].avi in several file names? :
Code:
avimerge -i "${NAME}".CD[0-9][0-9].avi -o "$OUTPUTFILE"

Author:  Patsie [ Sun Aug 22, 2010 8:15 am ]
Post subject:  Re: Merge AVIs with space in filenames

partimers wrote:
For my future knowledge, a question: what happens exactly in the line, regarding the expansion of "${NAME}".CD[0-9][0-9].avi in several file names? :
Code:
avimerge -i "${NAME}".CD[0-9][0-9].avi -o "$OUTPUTFILE"


Before calling avimerge, your shell will first evaluate all variables.
First it will expand ${NAME} to for example 'My Movie File'.
Then it will evaluate "My Movie File".CD[0-9][0-9].avi to all matching filenames, like an 'ls' command.
Finally it will start avimerge with all expanded filenames found.

Author:  partimers [ Sun Aug 22, 2010 7:45 pm ]
Post subject:  Re: Merge AVIs with space in filenames

Patsie wrote:
Before calling avimerge, your shell will first evaluate all variables.
First it will expand ${NAME} to for example 'My Movie File'.
Then it will evaluate "My Movie File".CD[0-9][0-9].avi to all matching filenames, like an 'ls' command.
Finally it will start avimerge with all expanded filenames found.


Hi Patsie,

Thanks a lot for your help and explanation. I need one more thing before my script is like I need it to be. I need to process the files differently if they are multi-parts (*.CD??.avi) or not. For the the multiparts I need to perform the merge and then encode. For the single parts, there is no merging, just the encoding. Basically I need:

- Match anything named <some name>.CD01.avi, then merge all <some name>.CD??.avi and encode the avi
- Match anything not name *.CD??.avi (excluding *.CD02.avi, *.CD03.avi, etc) and just encode.

This was my try (real action is commented out, ffmpeg and avimerge, as I am still strugling to get the right OUTPUTFILENAME):
Code:
#!/bin/sh
echo "Processing Multi-parts"
for FILE in $(ls | egrep '\.CD01\.avi');
do
   NAME=${FILE%.CD01.avi}
   OUTPUTFILE="$NAME".mp4
   echo $OUTPUTFILE
   #avimerge -i "${NAME}".CD[0-9][0-9].avi -o "$OUTPUTFILE"
   #ffmpeg -y -i "$FILE" -acodec libfaac -vcodec libx264 -b 5000k -threads 0 -r 30 "$NAME".mp4
done

echo "Processing Single-parts"
for FILE in $(ls | grep avi | egrep -v '\.CD[0-9][0-9]\.avi');
do
   NAME="${FILE%.CD01.avi}"
   OUTPUTFILE="$NAME.mp4"
   echo $OUTPUTFILE
   #ffmpeg -y -i "$FILE" -acodec libfaac -vcodec libx264 -b 5000k -threads 0 -r 30 "$NAME".mp4
done



I am sure I messing up with the escapes and ls command, because I am not getting correctly at the files. For example, if I run the scripts againts these files:
Code:
090110 01 Barbecue at Sam's.CD01.avi
090110 01 Barbecue at Sam's.CD01.txt
090110 01 Barbecue at Sam's.CD01part.avi
090110 01 Barbecue at Sam's.CD02.avi
090110 01 Barbecue at Sam's.CD03.avi
090110 06 Pier!.avi
090110 07 Boat ride.CD01.avi
090110 07 Boat ride.CD02.avi
_ENC_
testCD01.avi


I get this if I echoed $NAME
Code:
Processing Multi-parts
090110 01 Barbecue at Sam's.CD01.avi 090110 07 Boat ride.mp4
Processing Single-parts
090103 Arriving at Home.avi 090110 01 Barbecue at Sam's.CD01part.avi 090110 06 Pier!.avi testCD01.avi.mp4


If I change the for statement to something like
Code:
for FILE in $(ls | egrep '\.CD01\.avi');
...
for FILE in $(ls | grep avi | egrep -v '\.CD[0-9][0-9]\.avi');


then I get
Code:
Processing Multi-parts
090110.mp4
01.mp4
Barbecue.mp4
at.mp4
Sam's.mp4
090110.mp4
07.mp4
Boat.mp4
ride.mp4
Processing Single-parts
090103.mp4
Arriving.mp4
at.mp4
Home.avi.mp4
090110.mp4
01.mp4
Barbecue.mp4
at.mp4
Sam's.CD01part.avi.mp4
090110.mp4
06.mp4
Pier!.avi.mp4
testCD01.avi.mp4

:|

Author:  Patsie [ Sun Aug 22, 2010 10:45 pm ]
Post subject:  Re: Merge AVIs with space in filenames

If you really want to know what is happening and why your script is behaving like it is, you can put 'set -x' before a part you want to debug and 'set +x' to turn debugging off again. Turning debugging on will generate a lot of output and can look quite scary, but it's faster than waiting for someone to answer on a forum :)

Using '$(ls)' in your for loop will execute first and all filenames, including the spaces are then passed to the for loop which will see them as separate parts.
I strongly suggest getting rid of the spaces in your filenames first, before processing them any further. It really makes scripting against it hell.
Just replace them with i.e. underscores and after merging/encoding replace them again with spaces if you must.

Author:  partimers [ Mon Aug 23, 2010 6:32 am ]
Post subject:  Re: Merge AVIs with space in filenames

Hi Patsie,

thanks for the tip on set -x. Will use it from now on to understand better my scripts.

I realized it is much simpler to use this for the multiparts:
Code:
for FILE in *.CD01.avi


What I really need is to figure out a way to match files not containing .CD??.avi in their names, with proper escaping. Substituting the spaces seems to me a little bit scary. This script will be handling movies from my family members, and I cannot force them to not use special chars, particularly as we are non-english speakers (lots of cedillas, etc). Will keep trying and will post the final script here for reference when I eventually get at it.

Author:  Patsie [ Mon Aug 23, 2010 3:12 pm ]
Post subject:  Re: Merge AVIs with space in filenames

In ksh and bash you have to turn on 'extended file globbing'
Check if it's on/off with 'shopt extglob' and turn it on with 'shopt -s extglob'
Then you can use '!(*.CD[0-9][0-9].avi)' to do an inverse of a fileglob.
You can turn it off again with 'shopt -u extglob'

Code:
shopt -s extglob
for file in !(*.CD[0-9][0-9].avi); do
  echo "$file"
done
shopt -u extglob

Author:  partimers [ Tue Aug 24, 2010 7:16 pm ]
Post subject:  Re: Merge AVIs with space in filenames

Well, after some more learning, here is my final complete script. It suits my needs very well. Thanks Patsie for your help.

Code:
#!/bin/sh

##############################################################################
# Script to encode home movies recorded by a Canon IXUS 860 IS to H.264
#
# Author: [email protected]
#
#   Requires: ffmpeg and mencoder on your $PATH
#
#    * This script will encode all *.avi movies on the current dir. It does
#     not recurse sub-directories
#
#   * Multi-part movies will be merged with mencoder and then encoded.
#     Multi-parts should be named like this:
#
#      Movie Name.CD01.avi
#      Movie Name.CD02.avi
#      Movie Name.CD03.avi
#
#    * At the end, the encoded movies will be abailable on directory ./_ENC_
#
##############################################################################

mkdir -p _ENC_

for FILE in *.avi;
do
   # Firt process the multi-parts
   if [[ "$FILE" =~ \.CD[[:digit:]][[:digit:]]\.avi ]]
   then
       set +x
      # Match the first part of the multi-part movies
      if [[ "$FILE" =~ \.CD01\.avi ]]
      then
         NAME=${FILE%.CD01.avi}
         OUTPUTFILE="$NAME".mp4
         NUMFILES=`ls "$NAME".CD* | wc -l`

         # We tell the user what will happen
         echo "##############################################################################################"
         echo "MOVIE: $NAME"
         echo "----------------------------------------------------------------------------------------------"
         echo "MERGING! : $NUMFILES files into $NAME.avi"
         echo "ENCODING : $NAME.mp4"
         echo "##############################################################################################"
         echo

         # Merge multi-parts
         echo "###########################################"
         echo "MERGING the following files"
         ls -1 "${NAME}".CD[0-9][0-9].avi
         echo "into"
         echo $NAME".avi"
         echo
         mencoder -forceidx -ovc copy -oac copy -o "$NAME".avi "${NAME}".CD[0-9][0-9].avi

         # Encode the merged AVI
         echo
         echo "###########################################"
         echo "ENCODING "$NAME.mp4
         echo
         START=`date +%s`
         ffmpeg -y -i "$NAME".avi -an -pass 1 -vcodec libx264 -b 5000k -threads 0 -vpre slow_firstpass -r 30 "$NAME".mp4
         ffmpeg -y -i "$NAME".avi -acodec libfaac -ab 192k -pass 2 -vcodec libx264 -b 5000k -threads 0 -vpre slow -r 30 "$NAME".mp4
         END=`date +%s`
         ELAPSED=`expr $END - $START`
         echo "----------------------------"
         echo "Time Elapsed in Seconds: $ELAPSED"
         echo

         # Remove the merged AVI, and ffmpeg temporary files and logs
         #rm -f "$NAME".avi
         rm -f ffmpeg2pass*.log
         rm -f x264_2pass.log
         rm -f x264_2pass.log.mbtree

         # Move the finished MP4 to _ENC_
         mv "$NAME".mp4 _ENC_
      fi
   else
      NAME=${FILE%.avi}
      OUTPUTFILE="$NAME".mp4
   
      # Process the single-part movies
      echo "##############################################################################################"
      echo "MOVIE: $NAME"
      echo "----------------------------------------------------------------------------------------------"
      echo "ENCODING : $NAME.mp4"
      echo "##############################################################################################"
      echo

      # Encode the AVI
      echo "###########################################"
      echo "ENCODING "$NAME.mp4
      echo
      START=`date +%s`
      ffmpeg -y -i "$FILE" -an -pass 1 -vcodec libx264 -b 5000k -threads 0 -vpre slow_firstpass -r 30 "$NAME".mp4
      ffmpeg -y -i "$FILE" -acodec libfaac -ab 192k -pass 2 -vcodec libx264 -b 5000k -threads 0 -vpre slow -r 30 "$NAME".mp4
      END=`date +%s`
      ELAPSED=`expr $END - $START`
      echo "----------------------------"
      echo "Time Elapsed in Seconds: $ELAPSED"
      echo

      # Remove the merged AVI, and ffmpeg temporary files and logs
      rm -f ffmpeg2pass*.log
      rm -f x264_2pass.log
      rm -f x264_2pass.log.mbtree

      # Move the finished MP4 to _ENC_
      mv "$NAME".mp4 _ENC_
   
   fi   
done

Page 1 of 1 All times are UTC - 6 hours
© 2000, 2002, 2005, 2007 phpBB Group • http://www.phpbb.com