Register
It is currently Wed Dec 17, 2014 8:56 pm

clean passwd file based on virtusertable file


All times are UTC - 6 hours


Post new topic Reply to topic  [ 19 posts ] 
Author Message
 PostPosted: Mon Dec 11, 2006 3:57 pm   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
The purpose of this script is to scan the /etc/passwd file one line at a time comparing the usernames to the usernames found in the /etc/mail/virtusertable file. When it comes across a name that is not in the virtusertable file then it will use sed to delete that line from the passwd file. Also, the script will ignore all "system" accounts and other accounts that have no e-mail alias in the virtusertable file. It will be used on a production mail server probably monthly to clean up a mess that gets left behind by other scripts. I really just want some input on this. If anyone sees anything obvious or sees something that could be done easier feel free to chime in. THIS SCRIPT HAS NOT BEEN TESTED!!! USE AT YOUR OWN RISK!! I CLAIM NO RESPONSIBILITY!! Feel free to use it though, if you think it works. I'm a bash scripting newb. I know a little C++ and as you can hopefully tell I've done some homework.


a cleaner version can be found at http://cave.pure.net/~jhite/cleanpasswdfile.sh
the forum doesn't like my tabs and spaces very much

Code:

#!/bin/bash
#cleanpasswdfile.sh
#
#Goal: compare /etc/mail/virtusertable with /etc/passwd; find the entries missing from virtusertable that still exist in #passwd and remove from passwd file
#

ROOT=root      #
E_WRONG_USER=65    # Not root?
BASEUSERS="root
toor
daemon
operator
bin
tty
kmem
games
news
man3
sshd
smmsp
mailnull
bind
proxy
_pflogd
_dhcp
uucp
pop
www
nobody
spamd
jhite
vexira
postm"

#file variables
PASSWORD_FILE=/etc/passwd      #passwd file
PASS_WRK=/etc/passwd.wrk      #
N=1               #counter variable
VIRT_FILE=/etc/mail/virtusertable   #virtusertable file


#root check
#
if [ "$USER" != "$ROOT" ]            #this section works#
then                     #         #
   echo; echo "Only root can run this script."; echo    #         #
   exit $E_WRONG_USER               #this section works#
else                     #         #
echo                     #         #
fi                     #this section works#

 
cp $PASSWORD_FILE $PASS_WRK      #make a copy of passwd (passwd.wrk) file to manipulate and clean up;


for NAME in $(awk 'BEGIN{FS=":"}{print $1}' < "$PASS_WRK" )      #read names in from passwd.wrk file
do
   
   for USER in $BASEUSERS                  #ignore system accounts#
   do                        #             #
                           #             #
      if [ $NAME != $USER ]               #ignore system accounts#
      then
         for VIRTUSERS in $(awk 'BEGIN{FS="   "}{print $2}'  < "$VIRT_FILE" )   
         do
            if [ $NAME != $VIRTUSERS ]      #ignore system accounts#
            then
            sed -n -e "$n"d $PASS_WRK      #delete line $n#
            N=$(($N+1))            #n=$n+1          #
            else
            N=$(($N+1))            #must count even if line not deleted#
            fi
         done
      else
      N=$(($N+1))         #must count when changing lines#
      fi
   done
      
done
   
   
   
#for virtusertable=passwd'OR'systemaccounts do nothing;

#when an entry is found that is not equal then delete that line from the passwd.wrk file using passwd counter variable;


# copy passwd to passwd.old and mv passwd.wrk to passwd
cp $PASSWORD_FILE /etc/passwd.old
mv $PASS_WRK $PASSWORD_FILE

exit 0


Last edited by Buggin on Wed Dec 13, 2006 11:21 am, edited 1 time in total.

Top
 Profile  
 PostPosted: Tue Dec 12, 2006 1:08 pm   

Joined: Wed Sep 06, 2006 12:19 pm
Posts: 54
Location: Covington, WA
Very nice........ :-)

If I may make a suggestion........

For your "BASEUSERS" variable, it might be a good idea to place the list in separate configuration file......The reason is not everyone will have the same system accounts in their particular passwd file......Some may be added by third party programs not included with their distro, or have system users shipped with the distro which is not on the list.......For example, Slackware 11 has over six default system user accounts not included in your base list.....

You can easily do this by cat'ing the list into the BASEUSERS variable:
Code:
BASEUSERS="$(cat /etc/cleanpasswd.conf)"
if [ $? -ne 0 ]; then
    echo "Sorry, but I cannot find or read the configuration file, /etc/cleanpasswd.conf"
    exit 1
fi


This will shift the responsibility on to the sysadmin for making sure it is correct and up-to-date, but that should be okay since the system users accounts rarely get changed, and has the added benefit of you not having to guess all the different possible system accounts..... ;-)

---thegeekster

PS: To make it easy to begin with, you could have a base list of system users already set up in the config file, which can later be modified by the sysadmin.....


Top
 Profile  
 PostPosted: Tue Dec 12, 2006 7:47 pm   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
thanks for your input

i worked out several little bugs today and actually did put the accounts in a seperate file... i knew that list would grow over time... btw, this list with the exception of two or three is a clean FreeBSD 6.1 install with apache, cucipop, vexira, and spamassassin

i did away with some of the loops using grep instead

the script (as it is on my harddrive right now) is working as it should except for one line... the line that actually deletes stuff... i'll get that sorted out tomorow and re-post the official working script


Last edited by Buggin on Tue Dec 12, 2006 10:32 pm, edited 2 times in total.

Top
 Profile  
 PostPosted: Tue Dec 12, 2006 7:54 pm   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
changed my mind... this is today's version after i made my initial changes... which were a lot really

Code:
#!/bin/bash
#cleanpasswdfile.sh
#
#Goal: compare /etc/mail/virtusertable with /etc/passwd; find the entries missing from virtusertable that still exist in #passwd and remove from passwd file
#

ROOT=root      #
E_WRONG_USER=65    # Not root?

#file variables
PASSWORD_FILE=/home/jhite/Desktop/BashScripting/cave/passwd      #passwd file
PASS_WRK=/home/jhite/Desktop/BashScripting/cave/passwd.wrk      #
PASS_OLD=/home/jhite/Desktop/BashScripting/cave/passwd.old      #
PASS_LOG=/home/jhite/Desktop/BashScripting/cave/cleanpasswd.log      #
N=1                           #counter variable
VIRT_FILE=/home/jhite/Desktop/BashScripting/cave/virtusertable      #virtusertable file
BASE_FILE=/home/jhite/Desktop/BashScripting/cave/baseusers      #baseusers file


#root check
#
if [ "$USER" != "$ROOT" ]            #this section works#
then                     #         #
   echo; echo "Only root can run this script."; echo    #         #
   exit $E_WRONG_USER               #this section works#
else                     #         #
echo                     #         #
fi                     #this section works#


cp $PASSWORD_FILE $PASS_WRK      #make a copy of passwd (passwd.wrk) file to manipulate and clean up;
touch $PASS_LOG
cat /dev/null > $PASS_LOG

for NAME in $(awk 'BEGIN{FS=":"}{print $1}' < "$PASSWORD_FILE" )   #read names in from passwd.wrk file
do

grep -q "\<$NAME\>" $BASE_FILE         #grep virtusertable for $NAME


STATUS=$?

   if [ $STATUS -eq 0 ] ; then
   N=$(($N+1))         #counter add

      

   else
   grep -q "\<$NAME\>" $VIRT_FILE   #grep base users for $NAME
      
   STATUS2=$?

      if [ $STATUS2 -eq 0 ] ; then
      N=$(($N+1))      #counter add
         
      else

         sed -n -e "$n"d $PASS_WRK      #delete line $n        #
         echo $NAME >> $PASS_LOG         #log of users deleted#
         N=$(($N+1))            #n=$n+1                #
      fi

   fi

done

# copy passwd to passwd.old and mv passwd.wrk to passwd
cp $PASSWORD_FILE $PASS_OLD
mv $PASS_WRK $PASSWORD_FILE

exit 0




i'll post the finished working copy tomorrow... feel free to figure out that sed line for me


Last edited by Buggin on Wed Dec 13, 2006 11:20 am, edited 1 time in total.

Top
 Profile  
 PostPosted: Wed Dec 13, 2006 1:01 am   

Joined: Wed Sep 06, 2006 12:19 pm
Posts: 54
Location: Covington, WA
Quote:
sed -n -e "$n"d $PASS_WRK #delete line $n #

Looks like $n should actually be $N, sed being case-sensitive, and all ;-) ........Also, nothing is happening with sed, since you're suppressing the output (-n) and not having any way for sed to update the file..........And, the -e switch is not necesary if you enclose the entire "delete" action in quotes.......

Therefore, replace the -n -e switches with -i to tell sed to make the changes directly on the file (in place) and enclose the whole "delete" action in quotes, not just the variable ("${N}d").....You will need to use the curly braces around the variable name to let bash know that 'd' is not part of the name, or simply add a space in between ("$N d")....



Here's a much simpler approach in the for loop, but maybe not as obvious:
Code:
#root check
#
if [ "$USER" != "$ROOT" ]; then      #this section works#
   echo -e "\nOnly root can run this script.\n"
   exit $E_WRONG_USER         #this section works#
fi                        #this section works#

cp $PASSWORD_FILE $PASS_WRK      #make a copy of passwd (passwd.wrk) file to manipulate and clean up;
rm -f $PASS_LOG && touch $PASS_LOG    # remove the old log and create an empty one in it's place

for NAME in $(cut -d: -f1 "$PASSWORD_FILE" )   #read names in from passwd.wrk file
do
    ## If NAME exists in either list, continue on to the next name:
   grep -q "\<$NAME\>" "$BASE_FILE"         #grep virtusertable for $NAME
   if [ $? -eq 0 ]; then      # If NAME exists, increment counter and
      let N++  &&  continue   # skip the rest of the loop
   fi

   grep -q "\<$NAME\>" "$VIRT_FILE"   #grep base users for $NAME
   if [ $? -eq 0 ]; then      # If NAME exists, increment counter and
      let N++  &&  continue   # skip the rest of the loop
   fi

    ## If no name was found, delete it from the passwd file:
   sed -i "${N}d" $PASS_WRK   #delete line $n        #
   echo $NAME >> $PASS_LOG      #log of users deleted#
   let N++            #n=$n+1 #

done


For the root user check, I wasn't sure what you were trying to accomplish with the else keyword in the if statement, but it's not needed unless the 'echo' command was absolutely necessary for something....... :?:

Instead of trying to empty the log file by reading nothing into it (cat /dev/null), why not simply delete it and create an empty log, instead....... :?:

In the loop itself, all those if statements are also unnecessary (and a bit confusing to look at) if all you're doing is checking to see whether the names exist in either file.........If a name is found, you simply increment the counter and then issue the continue command to skip the rest of the loop and start on the next name in the list(s)......

And awk is not necessary and a bit of overkill in this instance..........cut would be more appropriate.......

When incrementing the N counter (N=$(($N+1))), it's much simpler and easier to do it with 'let': let N++.....Notice there is no dollar sign ($) in front of the variable name when using the let command...

And if you wanted to speed things up a bit, you can load both user files (base and virtuser) into memory by reading them into variables, using cat, and grepping from those variables.............Reading from memory is much faster than reading from disk for each and every loop:
Code:
VIRT_NAME="$(cat $VIRT_FILE)"

  .....blah-blah.....

grep -q "\<$NAME\>" <<<"$VIRT_NAME"




HTH..... :-)
---thegeekster


Top
 Profile  
 PostPosted: Wed Dec 13, 2006 8:21 am   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
I think that's wnough to make it work

the only reason i didn't use N++ is because it was giving me an error for some reason (Ubuntu box)

like i said i'm a scripting newb so i'm amazed it worked at all after only a couple days of trying

I'll test this today and make sure it's working ok and let you know.. thanks for all your help


PS I know i changed the variables to a different location than they actually should be... I wasn't about to test this on the actual production files until I knew it was working... now I have a couple hundred user names to scan through and check at random to make sure i didn't delete anything i shouldn't have... if all is well there this script is done.


Top
 Profile  
 PostPosted: Wed Dec 13, 2006 2:40 pm   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
Here it is. As far as I know it works. I've tested it somewhat randomly and no accounts that shouldn't have been removed were. Thanks for your help geekster. I used some of your code (particularly the sed line). I'm still a newb but I think I got one knockout under my belt now. I'm not sure if the && continue is useful the way I did the loop but I never tested it to see if it worked (the way I have it written at least). So there you go, if you use OptiGold or some other program that leaves trash behind in your passwd file here's one way to clear it out quick.



Code:
#!/bin/bash
#cleanpasswdfile.sh
#
#Goal: compare /etc/mail/virtusertable with /etc/passwd; find the entries missing from virtusertable that still exist in
#passwd and remove from passwd file
#

ROOT=root             #
E_WRONG_USER=65    # Not root?

#file variables
PASSWORD_FILE=/etc/passwd       #passwd file
PASS_WRK=/etc/passwd.wrk        #
PASS_OLD=/etc/passwd.old          #
PASS_LOG=/var/log/cleanpasswd.log   #
N=1                              #counter variable
VIRT_FILE=/etc/mail/virtusertable       #virtusertable file
BASE_FILE=/etc/baseusers           #baseusers file


#root check
#
if [ "$USER" != "$ROOT" ]                   #this section works#
then                                          #                         #
   echo; echo "Only root can run this script."; echo    #                #
   exit $E_WRONG_USER                         #                          #
fi                                        #this section works#


cp $PASSWORD_FILE $PASS_WRK      #make a copy of passwd (passwd.wrk) file to manipulate and clean up;
rm $PASS_LOG
touch $PASS_LOG


for NAME in $(awk 'BEGIN{FS=":"}{print $1}' < "$PASSWORD_FILE" )   #read names in from passwd.wrk file
do

grep -q "\<$NAME\>" $BASE_FILE         #grep virtusertable for $NAME


STATUS=$?

   if [ $STATUS -eq 0 ] ; then
   N=$(($N+1))                          #counter add

      

   else
   grep -q "\<$NAME\>" $VIRT_FILE     #grep base users for $NAME
      
   STATUS2=$?

      if [ $STATUS2 -eq 0 ] ; then
      N=$(($N+1))                                   #counter add
         
      else
         sed -i "${N}d" $PASS_WRK        #delete line $n        #
         echo $NAME >> $PASS_LOG         #log of users deleted#
         N=$(($N+1))                     #n=$n+1                #
      fi

   fi

done

# copy passwd to passwd.old and mv passwd.wrk to passwd
cp $PASSWORD_FILE $PASS_OLD
mv $PASS_WRK $PASSWORD_FILE

exit 0


Top
 Profile  
 PostPosted: Wed Dec 13, 2006 4:57 pm   

Joined: Wed Sep 06, 2006 12:19 pm
Posts: 54
Location: Covington, WA
Buggin wrote:
...the only reason i didn't use N++ is because it was giving me an error for some reason (Ubuntu box)...

That's odd..........the 'let' command is a bash builtin and not specific to a distro, unless Ubuntu did something to the bash source which broke it somehow....... :?:

I know Debian is famous for that.....messing with the source, which usually ends up breaking something else, in their quest for the most "stable" distro on the planet........ ;-)


Top
 Profile  
 PostPosted: Thu Dec 14, 2006 8:35 am   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
it may have been a typo on my part... it didn't work the first time so i replaced it with what i have now and it worked so i marked that section as done and kept on going...


i just wrote a test script and unless something else is wrong then let is gone
i just made a counter that echoed it's value and stopped at 10... fairly simple.. and i get this error

counterscript.sh: 10: let: not found
1
counterscript.sh: 10: let: not found
1
counterscript.sh: 10: let: not found


your guess is probably better than mine


Top
 Profile  
 PostPosted: Thu Dec 14, 2006 8:39 am   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
after some more playing around i get this error
counterscript.sh: 10: N++: not found
1
counterscript.sh: 10: N++: not found
1
counterscript.sh: 10: N++: not found

EDIT:
oops nevermind i left the let out on that one.. i put it back and same error again

maybe i should be writing my scripts on one of the FreeBSD boxes around here


Top
 Profile  
 PostPosted: Thu Dec 14, 2006 9:23 am   

Joined: Wed Sep 06, 2006 12:19 pm
Posts: 54
Location: Covington, WA
Hmm.......let is a bash builtin, and the fact it can't be found means something's broke with that bash installation....This is from the bash manpage, under the SHELL BUILTIN COMMANDS section towards the end:
Quote:
let arg [arg ...]
Each arg is an arithmetic expression to be evaluated (see ARITHMETIC EVALUATION). If the last arg evaluates to 0, let returns 1; 0 is returned otherwise.



Look in your bash manpage to see if the above quote is there and, if it is, try reinstalling bash (the same version). If that doesn't work, try using a different version...........If it works with a different version, I suggest submitting a bug report to Ubuntu.........If it doesn't work or there's no reference in the bash manpage, then I suggest compiling it yourself or find a different distro........ :(


Top
 Profile  
 PostPosted: Thu Dec 14, 2006 1:37 pm   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
let is in the builtins manpage

i think i'll go check with ubuntuforums.org and see what i can find regarding this because i would like to have a properly working bash... most any script i write will be user on FreeBSD or FDC6 so it need to be compatible


Top
 Profile  
 PostPosted: Fri Dec 15, 2006 3:07 pm   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
If you use Ubuntu and some of your builtins aren't working correctly try this

Code:
sudo dpkg-reconfigure dash


select no and your builtins such as let should work again
it will require a reboot to take effect


Top
 Profile  
 PostPosted: Fri Dec 15, 2006 7:16 pm   

Joined: Wed Sep 06, 2006 12:19 pm
Posts: 54
Location: Covington, WA
dash....... :?:

You should be using bash, not dash.............dash is nothing more than an ash shell which was ported from NetBSD in '97, and doesn't have all the functionality of bash...............Debian uses dash (Debian ASH) for their installation root floppy since it's smaller and has fewer lib dependencies than bash.........Basically it's a '/bin/sh' replacement, not a 'bin/bash' substitute.........


Top
 Profile  
 PostPosted: Mon Dec 18, 2006 9:23 am   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
yeah... the reconfigure asks if you want to load dash to /bin/sh because it is POSIX compliant and smaller and faster than bash

if you select no it loads bash instead

Code:
Ubuntu Configuration                                                           
                                                                               
                                                                               
                                                                               
                                                                               
  ┌───────────────────────────┤ Configuring dash ├───────────────────────────â”￾ 
  │                                                                          │ 
  │ Bash is the default /bin/sh on a Debian system.  However, since our      │ 
  │ policy requires all shell scripts using /bin/sh to be POSIX compliant,   â”‚ 
  │ any shell that conforms to POSIX can serve as /bin/sh.  Since dash is    │ 
  │ POSIX compliant, it can be used as /bin/sh.  You may wish to do this     â”‚ 
  │ because dash is faster and smaller than bash.                            │ 
  │                                                                          │ 
  │ Install dash as /bin/sh?                                                 â”‚ 
  │                                                                          │ 
  │                    <Yes>                       <No>                      │ 
  │                                                                          │ 
  └──────────────────────────────────────────────────────────────────────────┘ 
                                                                               
                                                                               
                                                                               



i don't have a clue why they would want to load dash as default but if you select no everything will work as it should


Top
 Profile  
 PostPosted: Mon Dec 18, 2006 10:55 am   
Site Admin
User avatar

Joined: Sun May 15, 2005 9:36 pm
Posts: 673
Location: Des Moines, Iowa
ubuntu.... i've never ran it........ so, what your saying is that ubuntu is using dash by default vs bash by default ???? is that correct ?????


Top
 Profile WWW  
 PostPosted: Tue Dec 19, 2006 10:00 am   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
that's the way it seems to me

i happened across a post in ubuntuforums.org that said if you are having trouble with bash try that command

i don't know if it's a bug that happens during install or if every ubuntu machine defaults that way

this was a clean install of edgy 6.10


Top
 Profile  
 PostPosted: Wed Dec 20, 2006 2:18 pm   

Joined: Wed Sep 06, 2006 12:19 pm
Posts: 54
Location: Covington, WA
I'm a bit confused here..........The shebang line in your script clearly says '/bin/bash', not 'bin/sh', so "bash" should be running the script........Is dash also replacing /bin/bash as well?.........I can see using dash, or ash (which I use), for the /bin/sh since it has less dependencies and is useful for a borked system..........

but, since you say everything works after saying 'no' to dash (just say no ;-) ), that leads me to believe dash is also replacing bash....... :?


Top
 Profile  
 PostPosted: Thu Dec 21, 2006 9:52 am   

Joined: Mon Dec 11, 2006 3:26 pm
Posts: 18
i's highly possible... i stopped researching the problem once i fixed it and honestly there isn't much on ubuntuforums.org about it...

i never checked the bug reports to see if it was listed in there...


Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP