Gather Data (via Script) from 7 sites to My Website.
Need you to write a script to gather data from these 7 websites (add a .com to the end of each except for Teambuy):
Groupon, LivingSocial, [url removed, login to view], KGBDeals, BuyWithMe, GiltCity, and BloomSpot.
Our developers have an API written for outside script to directly write to our database and it is attached.
A few other requirements:
-Queries to scrape those sites above must be done every 12 hours.
-All cities for all 7 sites above.
-Queries must be passed through proxies so as not to reveal our IPs.
-All fields to be scraped are listed in the api.
-Your queries must only scrape NEW listings that have not been scraped before.
Please go over all 7 sites carefully as they are quite different.
You may write the scraper in whatever language you prefer. The finished script will need to be installed on a vps that I have.
Looking to start immediately and will need fast turnaround on this.