Register
It is currently Thu Jul 24, 2014 8:37 am

Script to cut a csv line into fields.


All times are UTC - 6 hours


Post new topic Reply to topic  [ 4 posts ] 
Author Message
 PostPosted: Mon May 09, 2011 7:07 pm   

Joined: Mon May 09, 2011 7:00 pm
Posts: 2
This may be a fairly easy thing to do, but unfortunately I have been searching around and cannot find an answer. Very frustrating.
I have a file (csv) that needs to be split into various fields in bash. An example of the first line is here:

2 ,"Rainbow Warrior : Battle for the Planet","An extra-ordinary adventure","MORRIS, Jill","Greenpeace Foundation. Rainbow Warrior (Ship). Environmental Protection. Nature Conservation.","363.7 MOR","1862913439","Norwood, SA : Omnibus,1998",

Part of my problem is that in the 5th field ("Greenpeace Foundation. Rainbow Warrior (Ship). Environmental Protection. Nature Conservation.") it's split with these large gaps of blank space. Can anyone please help me sort this out???

Thanks in advance,
Kate [-o<


Top
 Profile  
 PostPosted: Mon May 09, 2011 10:42 pm   

Joined: Mon May 09, 2011 7:00 pm
Posts: 2
also another problem here is not splitting the fields when the , is inside a " ". For example "SMITH, Mary", "......


Top
 Profile  
 PostPosted: Tue May 10, 2011 2:17 am   
User avatar

Joined: Tue Apr 27, 2010 2:28 pm
Posts: 172
Location: Czech Republic
If everything is in double quotes, split on double quotes. If double quotes might be escaped or newlines can appear inside double quotes, I'd use perl and Text::CSV.


Top
 Profile  
 PostPosted: Wed May 11, 2011 8:41 am   
Site Admin
User avatar

Joined: Sun May 15, 2005 9:36 pm
Posts: 662
Location: Des Moines, Iowa
Personally, I'd output the csv file with a "|" delimeter and NOT a "," delimiter.
Import the file into a spreadsheet, export it with a "sane" delimiter.

You "could" split on double quotes...until you only have one double quote or no double quote...and I've dealt with data that can't be guaranteed to be sane before. The only logical method for me was to export the data with a delimiter that wasn't in the data as well.

Code:
#!/bin/bash
# 03-18-2010 Crouse
# This script reads in a csv file and does stuff with the data line by line.
# input = /path/to/data/file
input=$1
[ $# -eq 0 ] && { echo "Usage: $0 arg1"; exit 1; }

while read -r line
do
        IFS="|" read -r f1 f2 f3 f4 f5 <<<"$line"
        # You could manipulate the data at this point, rearranging it etc...
        # Reversing it here
        echo "${f5}|${f4}|${f3}|${f2}|${f1}"
done <"$input"


Top
 Profile WWW  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Yahoo [Bot] and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP