The website crawler should go through the complete website, collect and download all the available resources of the website like PDF, Document, Excel format files etc. Images and Video format files are not required to be included in the resource dump and it should crawl only web pages with the same root domain. All the other similar and relevant file formats ( Macintosh or Linux compatible as well ) are to be included. The crawler should segregate all the files on the basis of the types of files they are, i.e., pdf, doc etc. The final project should be in the form of an application and should be able to execute without any other requirements other than an internet connection to just crawl the website and download the resources.
Bu iş için 2 freelancer ortalamada ₹5000 teklif veriyor
Is a GUI required or can it just be run on the command line?
I've been developing web applications for the past 2 years and can be develop the application as required.