We need a script written to query 125 to 160 different websites for data. The script to be written will run these queries twice a day. Most of the sites to be queried have rss feeds and/or basic html interfaces.
We need about twenty one total fields of data to be gathered from these sites, so if some of these sites do not have an rss feed with all of the fields of info that we need, you will need to setup to gather that additional information.
We would prefer you work with Perl/PHP, but am open as long as you can get the job done effectively.
You will have the script write the data to our database via API (HTTP poll of an URL with parameters for each data field).
We also need the script your team writes to query via proxies.
I have the API .sh file available for further viewing as well as the list of sites to be queried - these are attached.
Thank You Very Much for Bidding and Looking forward to working with you!