My project goes like this
1 – Extract data
2 – Process data to use in the next extraction
3 – Extract data
4 – Process data use in the next extraction
5 - Extract data
6 – Save data to SQL Server
7 – Save and rename five related .asp pages related to each record
I need records form The USFed gov’t job site.
[login to view URL]
I'll tell you the search citeria later.
There are about 18,000 records to start but the number goes down. About 1500 URLS need to searched and extracted from for each cycle.
The programming will include processes to avoid data duplication.
Deliverable
I want to own the extraction software and use my PC as its server. I’m familiar with Webscraper and Unitminer. I expect to pay between $150-$400 for this software. I will obtain a web/SQL server account.
I want the script and programs so I can look at the executable code when you are finished. I want the system to run everyday even if I am dead.