I need a Search Engine (Google / yahoo / Bing) URL scraper:
- I should be able to create projects and save them.
- I should be able to add my footprints or search terms to search for
- I'll add 1 or 100 keywords or 1000+ keywords by pasting them or getting them from a txt or csv file. (Each keyword + footprint will gather the first 10, 20... up to 100 first urls results from each search engine selected).
- To avoid getting banned I should be able to use rotating proxies and random delays.
- After collecting the urls I should be able to save them on a txt or csv file.
- I should be able to access the pregmatch or be able to fix any change of variable in the search strings of each search engine. Sometimes search engines change their source code and that breaks the program/script, I need to have access to change it easily.
- I have a logo that I would like to add.