Apparatus, method and computer program for cloud scraping using pre-scraped big data

Abstract

A cloud scraping system using pre-scraped big data includes an information providing server which, when receiving a scraping request from a user terminal, provides the user terminal with response information to the received scraping request, and a big data storage which stores pre-scraped information, wherein when the scraping request is about static information, the information providing server acquires the response information using the pre-scraped information. According to the above cloud scraping system using pre-scraped big data, there is an advantage that it is possible to quickly respond to a scraping request from the user terminal afterwards by pre-scraping and storing static information in the big data storage. Additionally, it is possible to improve the scraping server operation efficiency by making a proper use of a single or multi-processing scraping server based on policy information of a scraping target external institution.

What is claimed is: 
     
       1. A cloud scraping system using pre-scraped big data, comprising:
 an information providing server which, when receiving a scraping request from a user terminal, provides the user terminal with response information to the received scraping request; and 
 a big data storage which stores pre-scraped information, 
 wherein the information providing server acquires the response information using the pre-scraped information, when the scraping request is about static information, and 
 the cloud scraping system further comprises a scraping error modification server, which in the event that an error occurs while the user terminal directly performs scraping, analyzes an error log from scraping input information inputted in the user terminal when the error occurs, and updates a scraping module in the user terminal based on the analyzed error log. 
 
     
     
       2. The cloud scraping system using pre-scraped big data according to  claim 1 , wherein the static information includes at least one of an address list, real estate price information, real estate actual transaction information, loan products, interest rate information, economic indicators, and closure or cessation of business information. 
     
     
       3. A cloud scraping method using pre-scraped big data performed by a cloud scraping system, comprising:
 receiving a scraping request from a user terminal; 
 determining a property of the scraping request; 
 when the scraping request is about static information, generating response information using pre-scraped information stored in a big data storage; 
 providing the generated response information to the user terminal; and 
 in the event that an error occurs while the user terminal directly performs scraping, analyzing an error log from scraping input information inputted in the user terminal when the error occurs; and 
 updating a scraping module in the user terminal based on the analyzed error log. 
 
     
     
       4. The cloud scraping method using pre-scraped big data according to  claim 3 , wherein the static information includes at least one of an address list, real estate price information, real estate actual transaction information, loan products, interest rate information, economic indicators, and closure or cessation of business information. 
     
     
       5. A computer program stored in a recording medium to perform the cloud scraping method using pre-scraped big data according to  claim 3  in combination with hardware.