M.Ravikumar Reddy, A. Vivekanand, S.Siva Skandha


Fast crawling is necessary to gather the Web documents and keep up the advanced. Forums subsist in numerous different layouts and powered by a selection of forum software packages, other than they always have embedded navigation paths to show the way to the users from access pages to thread pages. Focus which is Forum Crawler under Supervision, a controlled web-scale forum crawler, to trawl appropriate content from forums by means of smallest overhead was introduced. The general idea behind Focus is that index, thread, and page flipping URLs can be noticed on the basis of their layout description and intention pages; and forum pages can be categorised by means of their layouts. Focus is competent in learning ITF regexes and is effectual in discovery of index, thread, page-flipping URL, and forum entry URL.


Crawling; Web documents; Focus; Forum crawler; Thread;


Y. Guo, K. Li, K. Zhang, and G. Zhang. Board Forum Crawling: a Web Crawling Method for Web Forum. In Proc. of 2006 IEEE/WIC/ACM WI, pages 475-478, 2006.

M. Henzinger. Finding near-duplicate Web pages: a largescale evaluation of algorithms. In Proc. of 29th SIGIR, pages 284-291, 2006.

H. S. Koppula, K.P. Leela, A. Agarwal, K.P. Chitrapura, S.Garg and A. Sasturkar. Learning URL Patterns for Webpage De-duplication. In Proc. of 3rd WSDM, pages 381-390, 2010.

K. Li, X.Q. Cheng, Y. Guo, and K. Zhang. Crawling Dynamic Web Pages in WWW Forums. Computer Engineering, 33(6): 80-82, 2007.

G. S. Manku, A. Jain, and A. D. Sarma. Detecting nearduplicates for Web crawling. In Proc. of 16th WWW, pages 141-150, 2007.

U. Schonfeld , N. Shivakumar. Sitemaps: above and beyond the crawl of duty. In Proc. of the 18th WWW, pages 991- 1000, 2009.

X.Y. Song, J. Liu, Y.B. Cao, and C.-Y. Lin. Automatic Extraction of Web Data Records Containing User-Generated Content. In Proc. of 19th CIKM, pages 39-48, 2010.

V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 1995.

Full Text: PDF


  • There are currently no refbacks.

Copyright © 2012 - 2023, All rights reserved.| ijitr.com

Creative Commons License
International Journal of Innovative Technology and Research is licensed under a Creative Commons Attribution 3.0 Unported License.Based on a work at IJITR , Permissions beyond the scope of this license may be available at http://creativecommons.org/licenses/by/3.0/deed.en_GB.