Register
It is currently Fri Aug 01, 2014 10:31 pm

My first (useful?) script


All times are UTC - 6 hours


Post new topic Reply to topic  [ 1 post ] 
Author Message
 PostPosted: Tue Jan 29, 2008 8:39 pm   

Joined: Tue Jan 29, 2008 7:22 pm
Posts: 2
Hi, I'm new in bash and looks really great to automatize different tasks, but it's funny too! :)
I make this script because read somewhere that with 'dd' it's possible to test different block-sizes to find wich one is the best for a hard drive, so I give it a try.
Finally I make a script to test it with different parameters, and here we have:

Code:
#!/bin/bash

###################################
## Quicker blocksize test
## By Terseus 30/01/2008
###################################

#######################################
## I have learned so much with this script, and have feed my curiosity about the block-size importance :)
## Hope someone find it useful.
#######################################


## Needed regular expressions
readonly DEVICE_DETECT_REGEX="[0-9]\?[ ]*[0-9]\?[ ]*[0-9]\+[ ]*\([a-z]\{3\}[0-9]\?\)"
readonly TIME_REGEX="real [0-9]\+\(.[0-9]\+\)\+"

readonly ACCEPTED_BS="512 1024 2048 4096 8192 16384 32768 65536" ## List of the accepted block-sizes.
BS_LIST=$ACCEPTED_BS ## List of the block-sizes to test.
FILELEN=104857600 ## The size of the file to use in the tests, in bytes.
DEVICE_INPUT="" ## The input device to use, by default use the first detected.
FILE_OUTPUT="./test-file-hdd-bs" ## The test file to use.
LOG_FILE="./test-hdd-bs.log" ## The log file to use, if already exist the old it's numbered.
COUNTER=5 ## The number of tests that will be do for every block-size.
WRITE_BEST=0 ## The block-size with the best write average time.
READ_BEST=0 ## The block-size with the best read average time.
DELETE_BEST=0 ## The block-size with the best delete average time.
PARAMS=$@ ## The parameters of the script.

## Check if is root.
if [ "$(whoami)" != "root" ]; then
echo "This script needs root privileges to run."
exit 1
fi

## Begin to process the parameters.
if [ "$#" -gt 0 ]; then
STATUS=0
for PARAM in $@; do
  case $STATUS in ## The STATUS var point out if we're checking a parameter or a parameter's value.
   0)
    case $PARAM in ## Here are detected the parameters.
     -b|--block-size) STATUS=1 ;;
     -s|--file-size) STATUS=2 ;;
     -d|--device) STATUS=3 ;;
     -l|--log-file) STATUS=4 ;;
     -c|--counter) STATUS=5 ;;
     -h|--help) ## Parameter to show help.
      echo -e "This script make some tests using different block-sizes in the input/output, \nmaking an average of the duration of the tests with every block-size and saving \nthe results, this can help to choose the best block-size for your system.\n\nAccepted parameters:"
      echo -e "  -b [list], --block-size [list]\n\tSet the block-sizes to test, where [list] is the list of the block-sizes\n\tseparated by semicolon(,)\n\tThe accepted values come from 1/2kb to 64kb\n\tExample: -b 512,1024,2048\n\tDefault: 512,1024,2048,4096,8192,16384,32768,65536"
      echo -e "  -s [size], --file-size [size]\n\tSet the size of the testfile, where [size] is the size of the file \n\tfollowed by the measure unit, espressed by b (byte), k (Kb), m (Mb), g\n\t(Gb)\n\tExample: -s 512m\n\tDefault: 100m"
      echo -e "  -d [dev], --device [dev]\n\tSet the input device to use, where [dev] is the absolute path of the \n\tdevice.\n\tExample: -d /dev/hda1\n\tBy default choose the first device of /proc/partitions."
      echo -e "  -l [file], --log-file [file]\n\tSet the logfile to use, if already exist will be renamed to [file].0, \n\tthe script will save up to 5 logs by this way.\n\tExample: -l /home/user/test-hdd-bs.log\n\tDefault: ./test-hdd-bs.log"
      echo -e "  -c [num], --counter [num].\n\tSet the number of tests repetitions for every block-size.\n\tExample: -c 3\n\tDefault: 5"
      echo -e "  -h, --help\n\tShow this help and exit."
      exit 0
     ;;
     *) echo "Unknown parameter: $PARAM"; exit 1 ;;
    esac
    ;;
    1) ## Block-sizes list parameter.
    BS_LIST="$(echo "$PARAM" | sed "s/,/ /g")"
    for BS_CHECKING in $BS_LIST ## Check every block-size in the list.
     do
     if [ -z "$(expr "$ACCEPTED_BS" : ".*\($BS_CHECKING\).*")" ]; then ## If it isn't a valid block-size, show it and exit.
      echo -e "Invalid block-size value: $BS_CHECKING\nThe accepted block-sizes are $ACCEPTED_BS"
      exit 1
     fi
    done
    STATUS=0
   ;;
   2) ## Testfile size parameter.
    ## Check that the readed input is a number.
    if [ $(expr "${PARAM:0:${#PARAM}-1}" : ".*[^0-9]") -ne 0 ]; then
     echo "Invalid testfile size: $PARAM"
     exit 1
    fi
    ## Check the used measure unit and calculate the new size.
    case ${PARAM:0-1:1} in
     b|B) FILELEN=${PARAM:0:${#PARAM}-1} ;;
     k|K) FILELEN=$(( ${PARAM:0:${#PARAM}-1} * 1024 )) ;;
     m|M) FILELEN=$(( ${PARAM:0:${#PARAM}-1} * 1048576 )) ;;
     g|G) FILELEN=$(( ${PARAM:0:${#PARAM}-1} * 1073741824 )) ;;
     [^bBkKmMgG])
      echo "Invalid testfile size: $PARAM"
      exit 1
     ;;
    esac
    STATUS=0
   ;;
   3) ## Input device parameter.
    DEVICE_INPUT=$PARAM
    ## Check that is a valid block-device
    if ! [ -b "$DEVICE_INPUT" -a -r "$DEVICE_INPUT" ]; then
     echo "The device $DEVICE_INPUT is not valid, please set another device."
     exit 1
    fi
    STATUS=0
   ;;
   4) ## Logfile parameter.
    LOG_FILE=$PARAM
    ## Add './' if there are no '/' in the value and check the directory afterwards.
    if [ $(expr "$LOG_FILE" : ".*\/") -eq 0 ]; then
     LOG_FILE="./$LOG_FILE"
    fi
    STATUS=0
   ;;
   5) ## Number of test repetitions parameter.
    COUNTER=$PARAM
    ## Check that is a valid number.
    if [ $(expr "$COUNTER" : ".*[^0-9]") -gt 0 ]; then
     echo "The value $COUNTER is not a valid number."
     exit 1
    fi
    STATUS=0
   ;;
  esac
done
if [ "$STATUS" -ne 0 ]; then
  echo "Incomplete parameter: $PARAM"
  exit 1
fi
fi


## If there are no input device set, make the autodetection and get the first detected.
if [ -z "$DEVICE_INPUT" ]; then
## Extramemos la primera unidad detectada.
DEVICE_INPUT="/dev/""$(cat /proc/partitions | grep -o -m 1 "$DEVICE_DETECT_REGEX" | sed "s/[0-9]\?[ ]*[0-9]\?[ ]*[0-9]\+[ ]*//g")"
if ! [ -b "$DEVICE_INPUT" -a -r "$DEVICE_INPUT" ]; then ## Comprobamos que la unidad se pueda leer.
  echo "The autodetected device $DEVICE_INPUT is not valid, you need read privileges."
  exit 1
fi
fi

## Check that we have read privileges in the log's directory.
DIR_LOG="$(echo "$LOG_FILE" | sed "s/[^/]\+\$//")"
if ! [ -w $DIR_LOG ]; then
echo "You don't have write privileges in the log's directory $DIR_LOG, specify another directory for the log."
exit 1
fi

## Check that we have write privileges in the logfile.
if [ -e "$LOG_FILE" ]; then
if ! [ -w "$LOG_FILE" -a -f "$LOG_FILE" ]; then
  echo "The logfile $LOG_FILE is not valod, specify another logfile."
  exit 1
fi
fi

## Check that the testfile does not exist.
if [ -e "$FILE_OUTPUT" ]; then
echo "The destination of the testfile $FILE_OUTPUT already exist, must be \ndeleted before can continue."
## Read the input until we get a valid response.
INPUT=""
while [ -z "$INPUT" ]; do
  echo -n "¿Do you want to delete it and continue? (y/n) "
  read -n 1 INPUT
  echo ""
if ! [ "$INPUT" = "y" -o "$INPUT" = "n" ]; then
  INPUT=""
fi
done
if [ "$INPUT" = "n" ]; then exit 0; fi
rm -rf $FILE_OUTPUT
fi

## Show the configuration of the script.
echo "Input device: $DEVICE_INPUT"
echo "Testfile: $FILE_OUTPUT"
echo "Logfile: $LOG_FILE"
echo "Block-sizes to test: $BS_LIST"
echo "Testfile size: $FILELEN"
echo "Number of tests/block-size: $COUNTER"
## Read the input until we get a valid response.
INPUT=""
while [ -z "$INPUT" ]; do
echo -n "¿Do you want continue? (y/n) "
read INPUT
if ! [ "$INPUT" = "y" -o "$INPUT" = "n" ]; then
  INPUT=""
fi
done

if [ "$INPUT" = "n" ]; then exit 0; fi;

## If the logfile exist, rotate all the existents.
if [ -e "$LOG_FILE" ]; then
LOG_LIST=$(ls -r $LOG_FILE*) ## Get the inverse arranged loglist.
for LOG_FILE_OLD in $LOG_LIST; do
  case $LOG_FILE_OLD in
   *[5]) ## The one that finish with 5 is deleted.
    rm -f $LOG_FILE_OLD
   ;;
   *[0-4]) ## The one that finish with a number between 0 and 4 are uppered one number.
    mv $LOG_FILE_OLD $LOG_FILE.$(expr $(expr "$LOG_FILE_OLD" : ".*\([0-4]\)") + 1)
   ;;
   *) ## The one that last (the one without number) is added .0 to the finish.
    mv $LOG_FILE $LOG_FILE.0
   ;;
  esac
done
fi

for BLOQUE in $BS_LIST; do ## Make a test for every block-size.
## Initialize the addition vars.
READ_FINAL[$BLOQUE]=0
WRITE_FINAL[$BLOQUE]=0
DELETE_FINAL[$BLOQUE]=0
echo "========== with blocks of $BLOQUE bytes =========="
for FORVAR in $(seq 1 $COUNTER)
  do
  echo -n "Test nº $(($FORVAR))... "
  ## Write test.
  COMANDO="$({ time -p dd if=$DEVICE_INPUT of=$FILE_OUTPUT bs=$BLOQUE count=$(($FILELEN / $BLOQUE)); } 2>&1)"
  RESULT="$(echo "$COMANDO" | grep -o -m 1 "$TIME_REGEX" | sed "s/real //")"
  WRITE_FINAL[$BLOQUE]="$(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} + $RESULT" | bc)"
  hdparm -f $DEVICE_INPUT > /dev/null ## Flush the HD buffer.
  ## Read test.
  COMANDO="$({ time -p dd if=$FILE_OUTPUT of=/dev/null bs=$BLOQUE count=$(($FILELEN / $BLOQUE)); } 2>&1)"
  RESULT="$(echo "$COMANDO" | grep -o -m 1 "$TIME_REGEX" | sed "s/real //")"
  READ_FINAL[$BLOQUE]="$(echo "scale=2; ${READ_FINAL[$BLOQUE]} + $RESULT" | bc)"
  hdparm -f $DEVICE_INPUT > /dev/null ## Flush the HD buffer.
  ## Delete test.
  COMANDO="$({ time -p rm $FILE_OUTPUT; } 2>&1)"
  RESULT="$(echo "$COMANDO" | grep -o -m 1 "$TIME_REGEX" | sed "s/real //")"
  DELETE_FINAL[$BLOQUE]="$(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} + $RESULT" | bc)"
  hdparm -f $DEVICE_INPUT > /dev/null ## Flush the HD buffer.
  echo "Finished."
done
## Calculate the average times.
WRITE_FINAL[$BLOQUE]="$(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} / $COUNTER" | bc)"
READ_FINAL[$BLOQUE]="$(echo "scale=2; ${READ_FINAL[$BLOQUE]} / $COUNTER" | bc)"
DELETE_FINAL[$BLOQUE]="$(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} / $COUNTER" | bc)"
## If the number is under 1, the bc command let it in .12, so we add it 0 to the beginning.
if [ $(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} < 1" | bc) -eq 1 ]; then WRITE_FINAL[$BLOQUE]=0${WRITE_FINAL[$BLOQUE]}; fi
if [ $(echo "scale=2; ${READ_FINAL[$BLOQUE]} < 1" | bc) -eq 1 ]; then READ_FINAL[$BLOQUE]=0${READ_FINAL[$BLOQUE]}; fi
if [ $(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} < 1" | bc) -eq 1 ]; then DELETE_FINAL[$BLOQUE]=0${DELETE_FINAL[$BLOQUE]}; fi
## Check if the best times have been exceeded.
if [ $WRITE_BEST -eq 0 ]; then
  WRITE_BEST=$BLOQUE
  READ_BEST=$BLOQUE
  DELETE_BEST=$BLOQUE
else
  if [ "$(echo "scale=2; ${WRITE_FINAL[$WRITE_BEST]} > ${WRITE_FINAL[$BLOQUE]}" | bc)" == "1" ]; then WRITE_BEST=$BLOQUE; fi
  if [ "$(echo "scale=2; ${READ_FINAL[$READ_BEST]} > ${READ_FINAL[$BLOQUE]}" | bc)" == "1" ]; then READ_BEST=$BLOQUE; fi
  if [ "$(echo "scale=2; ${DELETE_FINAL[$DELETE_BEST]} > ${DELETE_FINAL[$BLOQUE]}" | bc)" == "1" ]; then DELETE_BEST=$BLOQUE; fi
fi
done

## Build the best times table in the logfile.
echo "|-------|---------|---------|---------|" >> $LOG_FILE
echo "|-------|  read   |  write  | delete  |" >> $LOG_FILE
echo "|-------|---------|---------|---------|" >> $LOG_FILE
for BLOQUE in $BS_LIST; do
## Put the spaces to fill the column in each value.
case $BLOQUE in
  512) echo -n "|  $BLOQUE  |" >> $LOG_FILE ;;
  1024|2048|4096|8192) echo -n "|  $BLOQUE |" >> $LOG_FILE ;;
  *) echo -n "| $BLOQUE |" >> $LOG_FILE ;;
esac

if [ $(echo "scale=2; ${READ_FINAL[$BLOQUE]} < 10" | bc) -eq 1 ]; then echo -n "   ${READ_FINAL[$BLOQUE]}  |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${READ_FINAL[$BLOQUE]} > 9" | bc) -eq 1 -a $(echo "scale=2; ${READ_FINAL[$BLOQUE]} < 100" | bc) -eq 1 ]; then echo -n "  ${READ_FINAL[$BLOQUE]}  |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${READ_FINAL[$BLOQUE]} > 99" | bc) -eq 1 -a $(echo "scale=2; ${READ_FINAL[$BLOQUE]} < 1000" | bc) -eq 1 ]; then echo -n "  ${READ_FINAL[$BLOQUE]} |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${READ_FINAL[$BLOQUE]} > 999" | bc) -eq 1 ]; then echo -n "${READ_FINAL[$BLOQUE]}|" >> $LOG_FILE; fi

if [ $(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} < 10" | bc) -eq 1 ]; then echo -n "   ${WRITE_FINAL[$BLOQUE]}  |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} > 9" | bc) -eq 1 -a $(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} < 100" | bc) -eq 1 ]; then echo -n "  ${WRITE_FINAL[$BLOQUE]}  |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} > 99" | bc) -eq 1 -a $(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} < 1000" | bc) -eq 1 ]; then echo -n "  ${WRITE_FINAL[$BLOQUE]} |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${WRITE_FINAL[$BLOQUE]} > 999" | bc) -eq 1 ]; then echo -n "${WRITE_FINAL[$BLOQUE]}|" >> $LOG_FILE; fi

if [ $(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} < 10" | bc) -eq 1 ]; then echo "   ${DELETE_FINAL[$BLOQUE]}  |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} > 9" | bc) -eq 1 -a $(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} < 100" | bc) -eq 1 ]; then echo -n "  ${DELETE_FINAL[$BLOQUE]}  |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} > 99" | bc) -eq 1 -a $(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} < 1000" | bc) -eq 1 ]; then echo -n "  ${DELETE_FINAL[$BLOQUE]} |" >> $LOG_FILE; fi
if [ $(echo "scale=2; ${DELETE_FINAL[$BLOQUE]} > 999" | bc) -eq 1 ]; then echo -n "${DELETE_FINAL[$BLOQUE]}|" >> $LOG_FILE; fi

echo "|-------|---------|---------|---------|" >> $LOG_FILE

done

## Show the best times and save it in the log.
echo -e "\n\nBest average write time: ${WRITE_FINAL[$WRITE_BEST]} seconds with blocks of $WRITE_BEST bytes."
echo -e "\n\nBest average write time: ${WRITE_FINAL[$WRITE_BEST]} seconds with blocks of $WRITE_BEST bytes." >> $LOG_FILE
echo "Best average read time: ${READ_FINAL[$READ_BEST]} seconds with blocks of $READ_BEST bytes."
echo "Best average read time: ${READ_FINAL[$READ_BEST]} seconds with blocks of $READ_BEST bytes." >> $LOG_FILE
echo "Best average delete time: ${DELETE_FINAL[$DELETE_BEST]} seconds with blocks of $DELETE_BEST bytes."
echo "Best average delete time: ${DELETE_FINAL[$DELETE_BEST]} seconds with blocks of $DELETE_BEST bytes." >> $LOG_FILE

exit 0



Any help to improve it, to improve the way of the tests, opinions, and in fact anything will be appreciated ^_^

PS: Sorry for the long post, and for my poor english.


Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 1 post ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP