regexorcist
New Member
- Joined
- Feb 1, 2010
- Messages
- 178 (0.03/day)
- Location
- ~/
System Name | Slackware Linux |
---|---|
Processor | yes |
Motherboard | yes |
Cooling | currently convection, but considering mineral oil |
Memory | sometimes fails due to too much beer |
Video Card(s) | ATI Radeon HD5570 series |
Storage | IDE |
Display(s) | 32" LCD TV |
Case | sometimes |
Audio Device(s) | huh? what? speak up, I can't hear you |
Power Supply | yes |
Software | Slackware running Open-Source software (it doesn't get any better) |
This thread is for simple bash shell scripts or code snippets.
Here is a simple script I put together using wget, grep, sed
and of course a few regular expressions (regex... see my username )
As you can see, I use wget to retrieve the webpage
and after parsing, I use wget again to spider each link
and display the status.
Here is the page that is retrieved (it has about 30 links or so).
Here I start the script and the data starts streaming.
Here we have scrolled down to the end where the script has finished,
it took about 20 seconds or so to complete.
The script may be small and simple, but for large website maintenance
it's very powerful. My regular expressions are set up for absolute links
but easily modified for relative ones, just as the spider output could be
grepped down to a single line.
Have Fun and lets see some of your scripts
Here is a simple script I put together using wget, grep, sed
and of course a few regular expressions (regex... see my username )
Code:
#!/bin/bash
var_links=`wget -q -L -O - \
http://www.phillylinux.org/ \
| grep -o '<a href="http://[^"]*' \
| sed -e 's/<a href=\"//' -e 's/\"/\n/'`
wget --spider $var_links
As you can see, I use wget to retrieve the webpage
and after parsing, I use wget again to spider each link
and display the status.
Here is the page that is retrieved (it has about 30 links or so).
Here I start the script and the data starts streaming.
Here we have scrolled down to the end where the script has finished,
it took about 20 seconds or so to complete.
The script may be small and simple, but for large website maintenance
it's very powerful. My regular expressions are set up for absolute links
but easily modified for relative ones, just as the spider output could be
grepped down to a single line.
Have Fun and lets see some of your scripts