External link spider app

Looking to have an application developed that will spider websites to find external links.

The application should follow all internal links on a site and export all external links to a text file one per line. No duplicate URL's should be added to this external file.

The program should support loading a list of multiple sites to be spidered and running through all of them without additional input.

The app should show a running count for the amount of URL's found.

The program should allow for the use of multiple threads for speed.

I prefer a windows application but am open to exploring web based solutions as well.

Beceriler: .NET, C# Programlama, Perl, PHP, Yazılım Mimarisi

Daha fazlasını gör: found app, spider, follow app, External links, app program, app count, app links, open input file php, php open input file, app follow, php export text file, external websites, link spider, running external program php, open web spider, external internal, app sites, amount app, find app, app find

İşveren Hakkında:
( 66 değerlendirme ) Aurora, United States

Proje NO: #1188517