Need a crawler that will crawl our own web site and output a report with how many internal links each URL has pointing to it. The script should crawl our own domain name The script should crawl our own domain name and figure out all the internal pages we have and how many times each page has been linked to internally.
The final output would be a CSV file with 2 columns:
2) Number of internal links linking to that URL
Would be great if you could also run the script the first time.
Would like job completed and run by Sunday morning (california time).