Chilkat Spider ActiveX 

Crawl a website with this free ActiveX spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance.
spider activex component bot robot freeware internet web download intelligence data mining

HttpDetect (EffeTech HTTP Sniffer) 

A HTTP protocol network sniffer, packet analyzer and file rebuilder. Unlike most other sniffers, it is dedicated to capture IP packets containing HTTP protocol, rebuild and save the HTTP communications and files sent through HTTP protocol.
effetech sniffer http sniffer web sniffer web monitor image gif jpg packet sniffer http analyzer web browsing network sniffer snoop