International Journal of Information Technology and Computer Science (IJITCS)
IJITCS Vol. 5, No. 10, 8 Sep. 2013
Cover page and Table of Contents: PDF (size: 566KB)
Web Crawler Based on Mobile Agent and Java Aglets
Full Text (PDF, 566KB), PP.85-91
Views: 0 Downloads: 0
Author(s)
Index Terms
World Wide Web, Search Engine, Mobile Crawler, Aglets, Web Crawler, Mobile Agent
Abstract
With the huge growth of the Internet, many web pages are available online. Search engines use web crawlers to collect these web pages from World Wide Web for the purpose of storage and indexing. Basically Web Crawler is a program, which finds information from the World Wide Web in a systematic and automated manner. This network load farther will be reduced by using mobile agents.
The proposed approach uses mobile agents to crawl the pages. A mobile agent is not bound to the system in which it starts execution. It has the unique ability to transfer itself from one system in a network to another system. The main advantages of web crawler based on Mobile Agents are that the analysis part of the crawling process is done locally rather than remote side. This drastically reduces network load and traffic which can improve the performance and efficiency of the whole crawling process.
Cite This Paper
Md. Abu Kausar, V. S. Dhaka, Sanjeev Kumar Singh, "Web Crawler Based on Mobile Agent and Java Aglets", International Journal of Information Technology and Computer Science(IJITCS), vol.5, no.10, pp.85-91, 2013. DOI:10.5815/ijitcs.2013.10.09
Reference
[1]Web Crawler, “The Web Crawler Search Engine”, Web site, http://www.webcrawler.com.
[2]Berners-Lee, Tim, “The World Wide Web: Past, Present and Future”, available at: http://www.w3.org/People/Berners-Lee/1996/ppf.html. MIT (1996).
[3]Brin S. and Page L., “The Anatomy of a Large Scale Hypertextual Web Search Engine,” Technical Report, Stanford University, Stanford, CA (1997)
[4]Cho J., Garcia-Molina H., and Page L., “Efficient Crawling Through URL Ordering,” Technical Report, Computer Science Department, Stanford University, Stanford, CA, USA (1997)
[5]Lange D. and Oshima M., Programming and Deploying Java Mobile Agents with Aglets, Addison Wesley (1998)
[6]Evens Jean, Tu Jiao, Ali R Hurson, and Thomas E. Potok. “SAS: A Secure Aglet Server”, Computer Security Conference 2007, Myrtle Beach, SC, USA (2007)
[7]Internet World Stats. Worldwide internet users, available at: http://www.internetworldstats.com (accessed on March 5, 2013)
[8]Koster M., “Guidelines for Robot Writers”, A Web Document, http://wsw.nexor.co.uk/mak/doc/robots/guidelines.html
[9]Fiedler J. and Hammer J., “Using Mobile Crawlers to Search the Web Efficiently,” International Journal of Computer and Information Science, vol. 1, no. 1, pp. 36-58 (2000)
[10]Junghoo Cho and Hector Garcia-Molina “Parallel Crawlers”. Proceedings of the 11th international conference on World Wide Web WWW '02”, May 7–11, 2002, Honolulu, Hawaii, USA (2002)
[11]Nath, R.; Bal, S. and Singh, M., “Load Reducing Techniques an the Websites and other Resources: A comparative Study and Future Research Directions,” Computer Journal of Advanced Research in Computer Engineering, vol. 1, no. 1, pp. 39-49 (2007).
[12]B. Kahle, “Archiving the Internet,” Scientific American (1996)
[13]Shkapenyuk, V. and Suel, T., “Design and implementation of a high performance distributed web crawler”, In Proceedings of the 18th International Conference on Data Engineering (ICDE), San Jose, California. IEEE CS Press, pp. 357-368 (2002).
[14]Md. Abu Kausar, V S Dhaka and Sanjeev Kumar Singh. “Web Crawler: A Review.” International Journal of Computer Applications 63(2), pp. 31-36, USA (2013).
[15]Michael S. Greenberg and Jennifer C. Byingfon, Theophany Holding, David G. Harper, Tuffs University, ”Mobile Agent and Security”, IEEE Communications Magazine (1998).