Register
It is currently Wed Dec 13, 2017 10:58 am

monitoring directory


All times are UTC - 6 hours


Post new topic Reply to topic  [ 2 posts ] 
Author Message
 PostPosted: Tue Mar 01, 2016 7:07 am   

Joined: Tue Mar 01, 2016 7:01 am
Posts: 1
Dear All

Before i will post my first question i would like say "hello" for everbody.
I`m not familiar with programming and i`m looking for solution which will give me an option to monitor directory for content.


I have some directory called for example "uploaded" I want to monitor this directory for some updates.
From time to time in this directory will appear new files and directories and files in these directories.

For example

/uploaded/directory/file1.mpg
/uploaded/directory/file2.txt
/uploaded/directory2/file.avi
/uploaded/directory3/file4.mpeg

Then this files should be moved with his own structure to the new location after when all operation on file finished.

/moved/directory/file1.mpg
/moved/directory/file2.txt
/moved/directory2/file.avi
/moved/directory3/file4.mpeg

I`m reading about inotifywatch inotifywait but i havent idea how to force this to do as i wish.
May i ask somebody for some solution ?


Teoritcally i`ve done something like this:
Quote:
[[email protected] /]# inotifywait -m -r -q -e close_write --format %w%f /home/jagrok | while read file; do cp -uvr "$file" /home/ingest/; done
„/home/jagrok/anaconda-ks.cfg” -> „/home/ingest/anaconda-ks.cfg”
„/home/jagrok/test2/debian.iso” -> „/home/ingest/debian.iso”
^C
[[email protected] /]# cd /home/ingest/
[[email protected] ingest]# pwd
/home/ingest
[[email protected] ingest]# ls -l
razem 182020
-rw-------. 1 root root 1139 03-01 14:54 anaconda-ks.cfg
-rw-r--r--. 1 root root 186384384 03-01 14:55 debian.iso
[[email protected] ingest]#


but it seems that i have a problem how to get a directory name


Top
 Profile  
 PostPosted: Tue Mar 01, 2016 8:23 am   

Joined: Mon Oct 20, 2014 9:53 am
Posts: 560
You want some basic IPC (InterProcessCommunication) as no reliable method exists to decide whether a file is already downloaded or just parts are already done.

The idea behind this snippet is to use some three processes to sync the tasks.
One task is to monitor the directory.
The second is to wait until the file ist downloaded.
And the third job is, to mv the new files.

Code:
# we create two fifo pipes
mknod watch_pipe p # the notify process writes to this pipe whenever something happened in given directory
mknod cmd_pipe p  # when the the file is downloaded entirely the pathname is written to the command part

dir_to_watch=/some/dir  # do some testing, if directory given exists and is writable

# we define  a function for each task
notify_process(){
   while true ; do
      inotifywait $dir_to_watch >watch_pipe
      # if an error occurs it is written to syslog
      # redirect STDERR as well, if not wanted
   done
}

# we start this function and send it to background
notify_process &
pid_notify_process=$!   # we need the pid to clean up, when done

# the process to decide when a file is downloaded
download_complete_process(){
    # as we watch each event, we have to check the events occurring
    # we read the notify_process in a loop
    while read line; do
         case $line in   
             CREATE,DIR  )  # directory is created
                             # implement a function to check how many files are created
                             # and check when all of these files are downloaded
                             # then write a message to mother process
                             echo "$path_of_actual_file/$actual_file > cmd_pipe
            CLOSE  )  # writing the file in question has finished.
                             # we do wait some seconds, in case it gets reopened
                              write_remember_array; wait_some_delay_and_test_again_here
                             # the code in line above should run in background too
                             # to keep script reactive too. Implement a mechanism with another pipe here
                             # and finally then write a message to mother process
                             echo "$path_of_actual_file/$actual_file > cmd_pipe
    done < watch_pipe
}

# we start this function in background too
download_complete_process &
pid_download_complete_process=$!   # we need the pid to clean up, when done

# for convenience we use a clean up function
clean_up(){
   kill -SIGTERM $pid_notify_process
   kill -SIGTERM $pid_download_complete_process
   rm -f watch_pipe cmd_pipe
}

# and trap any kind of exiting this script
trap clean_up EXIT

# the script itself just has to do the move of files and dirs
while read line; do  # we read the pathname from pipe
    mv $line  $destination  # set destination porperly! not done here.
done < cmd_pipe

I did not test this. Just typed into forum directly from brain.


EDIT:
Forgot to put a loop around the "inotifywait" call in notify_process function

The problem is that no one can reliable say when a download is finished.
But one can try to get the size of file to be downloaded.
This implies to communicate with the downloading program.
It is pretty easy to curl the download size from an URL.
But it might be as well possible to get this information from the program used for doing the download.
Just relying on the CLOSE event and a certain delay is not overly reliable.


Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 16 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP