Hey Guys, I need a fairly simple desktop app that would run each of the keywords as "keyword", allintitle:keyword, and allinanchor:keyword in Google and get the number of competing pages.
SPECIFICS ON HOW IT WORKS:
1) I'll upload a .txt file with keywords. (it'll be about 500k keywords/time, so the bot needs to handle a lot of data at a time/
2) I'll also upload a .txt file with public/private http proxies in it.
3) I'll select the number of threads.
4) I press start and the bot goes out and enters each keyword as: "keyword", allintitle:keyword, and allinanchor:keyword in Google and get the number of competing pages in Google and get the number of competing pages.
For example, lets say I put in "short term loans"
The bot will go out to google at enter- "short term loans"
Google shows "About 1,660,000 results (0.18 seconds)" It takes 1,660,000.
Then it enters - allintitle:"short term loans", the result we get is About 210,000 results (0.63 seconds).. so it takes 210,000 from that.
Then it enters - allinanchor:"short term loans", the result is About 5,530,000 results (0.15 seconds).. and it saves it as 210,000.
It repeats this process for ALL of the keywords that I put in.
If there is an error in some keyword or it was unable to scrape the number for whatever reason, it should say Unknown.
The output file should look something like:
1,660,000 210,000 210,000
Notice how there are 3 spaces between each result and they go by the order of results of "keyword" allintitle:"keyword" allinanchor:"keyword"
It shouldn't include the keywords in the output file, and it should get the results in the order I put the keywords in.
The features I want are:
1) Has to be multithreaded.
2) Be able to use proxies.
3) Handle a lot of data.
4) It should be saving the results to the output file as its scraping them.
5) Blazing fast, I shouldn't see any browser open or anything like that.
That's it. I hope I've been through enough. Please feel free to ask questions.