Spidr for Mac Publisher's description
from Hal Brodigan
Fast and easy to use web spider developed in Ruby
Spidr is a versatile Ruby web spidering library that can spider a website, multiple domains, certain links or infinitely. Spidr is designed to be easy to use and fast.
Here are some key features of "Spidr":
В· Custom User-Agent strings.
В· Custom proxy settings.
В· Black-list or white-list URLs based upon: Hostname, Port number, Full link, File extension
В· Provides call-backs for: Every visited URL, Every visited Page, Every visited URL that matches a specified pattern.
System Requirements:В· Hpricot
Program Release Status:
Program Install Support: