What is Screen Scraping?
[wp_lightbox_display_external_page link=”http://www.youtube.com/embed/gVfAr8XSsDM?rel=0″ width=”640″ height=”480″ title=”Extract Data From The Web – Welcome” source=”http://www.iharvest.co.uk/wp-content/uploads/2014/09/Sally_Welcome_To_iHarvest.jpg” autoplay=”1″]
Web Screen Scraping – is a way of extracting data, images, and other documents from a web site.
How can iHarvest help ?
Our Crawler can Screen Scrape data and images, saving your hours of manual effort.
- Get a copy of a web sites images.
- Get data in Word, Excel, CSV, SQL
- Exract just once or every week.
- Get images of the actual web site (screen shots).
A user can easily copy and paste data from a web page. They can even download images and store the file names with the associated product text. For small amounts of data this is fine. But consider many rows of data across 10s, 100s or even 1000s of pages, this can be very time consuming and prone to human error.
Screen Scraping is an automated technique of extracting data and images from a web site. Generally The Screen Scraping Software will work with a Web Crawler.
The Web Crawler will be responsible for automatically navigating a web site. The crawler will follow every link in a methodical way hunting for data to be scraped.
When the crawler (sometimes know has Web Spiders ;Web Robots; or just “bots”) finds a page that the screen scraper is interested in (such as a product page) the crawler then calls upon the Screen Scraper to extract the information that is required.
[wp_lightbox_display_external_page link=”http://www.youtube.com/embed/GUK0JHV560U?rel=0″ width=”640″ height=”480″ title=”Extract Data From The Web – Speed Writing”
source=”http://www.iharvest.co.uk/wp-content/uploads/2013/04/iHarvest_Web_Data_Extraction_SpeedWriting400.jpg” autoplay=”1″]
Why use Screen Scraping?
- Extract data and images from a web site very quickly.
- Analyses a competitors site. “Measure” their product range.
- Identify competitors items In Stock and Out of Stock.
- Identify a competitors brand proportion, how much of one band do they sell, what product types.
- Identify if a competitor is selling products you are not, and vice-versa.
- Extract data and images from a web site very accurately.
- Compile the extracted information into a database, spread sheet to conduct further analysis.
- Screen Scraping data helps with data mining and business intelligence.
More References
Screen Scraping
Save 100’s of hours manually inputting
The extracted data that the screen scraper found is then cleansed, processed, transformed, translated as required and then stored in another place such as a spread sheet or database. Once the data is stored in Excel, CSV (Comma Separated Values) or a Database, it makes life much easier to use the data.
Extract Automatically – no more copy and paste.
iHarvest can save you hours and hours of manual effort.
If you have a Screen Scraping project / idea? Contact iHarvest today, we’ll happily discuss your idea and take a look at the web site you want to extract data from. Initially we’ll help you establish how scrape-able the data is. 100% no obligation.