Showing posts with label web-scraping. Show all posts
Showing posts with label web-scraping. Show all posts

Friday, May 4, 2012

Scrape web pages in real time with Node.js


What's a good was to scrape website content using Node.js. I'd like to build something very, very fast that can execute searches in the style of kayak.com , where one query is dispatched to several different sites, the results scraped, and returned to the client as they become available.

Friday, February 10, 2012

How to extract textual contents from a web page?


I'm developing an application in java which can take textual information from different web pages and will summarize it into one page.For example,suppose I have a news on different web pages like Hindu,Times of India,Statesman,etc.Now my application is supposed to extract important points from each one of these pages and will put them together as a single news.The application is based on concepts of web content mining.As a beginner to this field,I can't understand where to start off.I have gone through research papers which explains noise removal as first step in buiding this application.