We are a couple of researchers who need quick help for a small tool which can crawl the wall of Facebook groups, profiles and fan pages and transform this data into a [url removed, login to view] or similar to be used for analysis in Excel etc. We need a crawler that is robust, so it can (in principle) handle several thousand wall posts.
Ideally, the crawler should crawl the pages we need crawled, if we feed it with the URL of the page in question. As an alternative, we can save the page ourselves in Firefox, then the crawler "just" needs to crawl the [url removed, login to view] that Firefox provides of the page.
More specifically, we need the crawler to create a [url removed, login to view] that lists the following data in columns: Name of wall post poster, time and date of the post, textual content of the post, number of likes and number of comments (exact if less than 50 comments, "+50" if more than 50 comments as Facebook now lists theses on an individual page). We would also like the name of all commenters on a post, time and date of each comment to a post, and the textual content of the comment in an affiliated [url removed, login to view] if possible at all.
For further information, please dont hesitate to ask quetions here!
As discussed on PMB and previous project posting.