Killing HTML nodes from shell

Posted by hendry on Stack Overflow See other posts from Stack Overflow or by hendry
Published on 2010-05-03T11:12:13Z Indexed on 2010/05/03 12:08 UTC
Read the original article Hit count: 428

Filed under:

sed

|

html

|

html-parsing

|

linux

Need a solution to kill nodes like <footer>foobar</footer> and <div class="nav"></div> from many several HTML files.

I want to dump a site to disk without the menus and footers and what not. Ideally I would accomplish this task using basic unix tools like sed. Since it's not XML I can't use xmlstarlet.

Could anyone please suggest recipes, so I can ideally have a script running kill-node.sh 'div class="toplinks"' *.html to prune the bits I don't want. Thank you,

© Stack Overflow or respective owner

Related posts about sed

sed + RE , want to ignore remarked line using sed

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to replace the a string with the word string as the following example down , but if a string exist also after comment then a string will replaced too In which way I can add to the sed command the irregular exp that ignore from the #. [root@localhost tmp]# more test a b aa bb #a #b #aa #bb [root@localhost… >>> More
script to search and replace deprecated functions

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am using the following script to search and replace the deprecated functions in a file with the newer ones. 5 for strFile in `ls deprecated_functions_search_and_replace.txt ` 6 do 7 sed "s/ereg_replace[^\(]*(\([^,]*\),/preg_replace\1('#'.\2.'#',/g" $strFile > temp_file 8 … >>> More
Footer not stretching 100% when horizontally scrolled

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I have a footer which is set to 100% width, but if i size the window smaller so a horizontal scrollbar appears, using the scrollbar shows whitespace to the right of the footer ... its not spanned 100% of the page, just the viewport. <!doctype html> <html lang="en" class="no-js"> <head> … >>> More
What does this regex mean and why

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
$ sed "s/\(^[a-z,0-9]*\)\(.*\)\( [a-z,0-9]*$\)/\1\2 \1/g" desired_file_name I apreciate it even if you only explain part of it or at lest structure it with words as in s\alphanumerical_at_start\something\alphanumerical_at_end\something_else\global Could someone explain what that means, why and… >>> More
Sed problem in a Bash script

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
Hello there. I'm having a problem using the sed command . I'm trying to write a bash script that does the following : search for the line that contain :@ then save the line that contained :@ and replace it with new line as in the following: #! /bin/bash echo "Please enter the ip address… >>> More

Related posts about html

Install usblib package - Ubuntu

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I need the package libusb for another package I am installing. I tried the following which seemed to install the package, sudo apt-get install libusb-dev but when I try to install the other package I get, configure: error: Package requirements (libusb-1.0 >= 0.9.1) were not met: No package… >>> More
Prevent malicious vulnerability scan increasing load on a server

as seen on Server Fault - Search for 'Server Fault'
Hi all, this week we have been suffering some malicious vulnerability scans to our servers, increasing the load on them, making them nearly unusable. The attack is easy to defend, just blocking the offending ip, but only after discovering it. Is there any form of prevent it? Is it normal that… >>> More
can't install psycopg2 in my env on mac os x lion

as seen on Server Fault - Search for 'Server Fault'
I tried install psycopg2 via pip in my virtual env, but got this error: ld: library not found for -lpq (full log here: http://pastebin.com/XdmGyJ4u ) I tried install postgres 9.1 from .dmg and via port, (gksks)iMac-Alexander:~ lorddaedra$ locate libpq /Developer/SDKs/MacOSX10.7.sdk/usr/include/libpq /Developer/SDKs/MacOSX10… >>> More
Bitnami redmine error SVN

as seen on Server Fault - Search for 'Server Fault'
I'm installing the Bitnami Redmine stack (redmine + subversion). Firstly I install configure and test it locally (Ubuntu 14.04 LTS). And everything is OK. I install Bitnami stack on server (Red Hat 4.4.7-4) and configure SVN. I commit files into SVN and connect project into Redmine with SVN repository… >>> More
Can the .htaccess file slow down a website to a crawl? If so, are there better ways to solve these problems with different rewrite rules and such?

as seen on Pro Webmasters - Search for 'Pro Webmasters'
here is my htaccess file...... RewriteCond %{REQUEST_URI} ^/patients/billing/FAQ_billing\.html$ [OR] RewriteCond %{REQUEST_URI} ^/patients/billing/getintouch\.html$ RewriteRule ^patients/billing/(.*)\.html$ $1.php [L,NC] RewriteCond %{REQUEST_URI} ^/patients/findadoctor/a\.html$ [OR] RewriteCond… >>> More