Register
It is currently Mon Nov 24, 2014 2:16 pm

bash script for html to csv conversion


All times are UTC - 6 hours


Post new topic Reply to topic  [ 2 posts ] 
Author Message
 PostPosted: Mon May 09, 2011 12:44 am   

Joined: Mon May 09, 2011 12:38 am
Posts: 2
thanks for allowing me to join the forum

i have a html file with three columns ------------Last visit date , URL and link,,,,,,,, how can i convert the same into csv so that i can output into database... the mechine is linux...i made a little googling and got idea that there is ways for conversion using bash and perl./....but dont know how to.........

any ideas please
thanks in advaance
ayyappan

the html file is as follows


<HTML>
<HEAD>
<TITLE> Web Browser Forensics Results File </TITLE>
<!-- Program Created by Manuel Humberto Santander Pelaez <manuel@santander.name> -->
<!-- Generated by root at (none) in Mon May 9 10:52:10 2011
-->
</HEAD>
<BODY style="color: rgb(255, 255, 255); background-color: rgb(0, 0, 0);" alink="#000099" link="#33ff33" vlink="#990099">
<CENTER></CENTER>
<CENTER><H1> Web Browser Forensic Results for Opera </H1>
<H3> Generated by root@(none) on Mon May 9 10:52:10 2011
</H3>
<HR>
<table style="width: 50%; text-align: left;" border="1" cellpadding="2" cellspacing="2">
<tbody>
<tr>
<td style="vertical-align: top; text-align: center;"><span style="font-weight: bold;">Last Visit Date</span>

</td><td style="vertical-align: top; text-align: center;"><span style="font-weight: bold;">URL</span>

</td>
<td style="vertical-align: top; text-align: center;"><span style="font-weight: bold;">Page Title</span>

</td>
</tr><tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:31:53 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

test
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:31:53 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://help.opera.com/updateinfo/?b...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:31:53 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Opera 11.10 is available
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:32:18 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.google.co.in/search?q=ff...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:32:18 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

ff - Google Search
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:32:43 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.google.co.in/search?q=gm...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:32:43 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

gmail.com - Google Search
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:32:56 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.google.co.in/url?sa=t&so...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:32:56 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Gmail: Email from Google
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:37:59 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.google.co.in/accounts/Se...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:37:59 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Gmail
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:16 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Gmail - Access your VMware Player Download - ayyappancheta@gmail.com
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:19 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Gmail - Inbox (9) - ayyappancheta@gmail.com
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:27 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

https://mail.google.com/mail/?logou...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:27 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Redirecting
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:27 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Google Accounts
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:28 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Gmail: Email from Google
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:40 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.google.co.in/search?q=ya...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:40 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

yahoomail - Google Search
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:45 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.google.co.in/url?sa=t&so...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:45 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Sign in to Yahoo! India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:38:58 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

(2 unread) Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:39:01 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Inbox (2) - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:39:03 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Spam (76) - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:39:11 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

You visit illegal websites - Spam - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:39:48 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

You visit illegal websites - Spam - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:39:58 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

You visit illegal websites - Spam - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:40:24 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Spam (75) - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:40:31 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

H9: OH NO YOU DIDN'T - Spam - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:40:40 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Drafts (0) - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:40:46 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

New issue of Hakin9 available: Cloud Security - Spam - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:41:04 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Spam (73) - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:41:06 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Inbox (2) - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:41:10 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Spam (73) - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:41:15 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

May issue of Hakin9 Magazine: Cloud Security & Video trainings from Wayne Burke - Spam - Yahoo! Mail India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:41:29 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://mytalkoot.com/12all/lt.php?c...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:41:29 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Cloud Security 05/2011 | Magazine | IT Security Magazine - Hakin9 http://www.hakin9.org
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:42:15 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Tutorials | IT Security Magazine - Hakin9 http://www.hakin9.org
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:42:37 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://global.ard.yahoo.com/SIG=15o...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:42:37 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

https://login.yahoo.com/config/logi...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:42:38 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.in.yahoo.com/
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:42:38 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Yahoo! India
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:42:50 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.google.co.in/search?q=co...
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:42:50 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

computing net - Google Search
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:42:57 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Computer Tech Support Forum - Windows - Linux - Mac - Computing.Net
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:43:12 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

Computing.Net
</td>
</tr>
<tr>
<td style="width=30%; vertical-align: top;">

Mon May 9 10:44:03 2011

</td>
<td style="width=40%; vertical-align: top;">

Click here to follow
</td>
<td style="width=30%; vertical-align: top;">

http://www.computing.net/cgi-bin/my...
</td>
</tr>
</tbody>
</table>
</CENTER>
<HR>
</BODY>
</HTML>


Top
 Profile  
 PostPosted: Mon May 09, 2011 2:47 am   
User avatar

Joined: Tue Apr 27, 2010 2:28 pm
Posts: 172
Location: Czech Republic
Your HTML is not valid (& should be &amp;). If you correct it, you can use xsh (http://xsh.sourceforge.net/):
Code:
for //tr { for td echo :n xsh:subst(xsh:subst(.,'^\s+',''),'\s+$','\t') ; echo }


Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 14 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP