Design of Web Crawler for the Client - Server Technology Article Swipe

PDF

Md. Abu Kausar , Vijaypal Singh Dhaka , Sanjeev Kumar Singh ·

YOU? · · 2015 · Open Access · · DOI: https://doi.org/10.17485/ijst/2015/v8i36/79858

Search engines store information locally with the purpose of deliver quick, accessible search abilities. This information is collected by Web crawler. Web crawling is necessary for the maintenance of complete and latest web document gathering for a web search tool. The web document modifies its content on regular basis hence it becomes necessary to build up a successful framework which could identify these sorts of changes proficiently in the most minimal scanning time to accomplish this modifications. The essential thought behind designing of such a web crawler is to find high quality web documents within limited time frame. The proposed system works on the Client-Server Technology it reduces the overlap problem and downloads high quality web pages. We can add many web crawler in parallel to download web page in parallel way.Keywords: Client-Server Technology, Overlap, Search Engine, Web Crawler, Web Page

Related Topics

Concepts

Web crawler Computer science Focused crawler Static web page World Wide Web Web page Web search engine Web server Information retrieval Web API Web modeling Web development Web design The Internet

Metadata

Type: article
Language: en
Landing Page: https://doi.org/10.17485/ijst/2015/v8i36/79858
PDF: https://sciresol.s3.us-east-2.amazonaws.com/IJST/Articles/2015/Issue-36/Article19.pdf
OA Status: diamond
Cited By: 2
References: 23
Related Works: 10
OpenAlex ID: https://openalex.org/W2277785728

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W2277785728

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.17485/ijst/2015/v8i36/79858

Digital Object Identifier
Title: Design of Web Crawler for the Client - Server Technology

Work title
Type: article

OpenAlex work type
Language: en

Primary language
Publication year: 2015

Year of publication
Publication date: 2015-12-03

Full publication date if available
Authors: Md. Abu Kausar, Vijaypal Singh Dhaka, Sanjeev Kumar Singh

List of authors in order
Landing page: https://doi.org/10.17485/ijst/2015/v8i36/79858

Publisher landing page
PDF URL: https://sciresol.s3.us-east-2.amazonaws.com/IJST/Articles/2015/Issue-36/Article19.pdf

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: diamond

Open access status per OpenAlex
OA URL: https://sciresol.s3.us-east-2.amazonaws.com/IJST/Articles/2015/Issue-36/Article19.pdf

Direct OA link when available
Concepts: Web crawler, Computer science, Focused crawler, Static web page, World Wide Web, Web page, Web search engine, Web server, Information retrieval, Web API, Web modeling, Web development, Web design, The Internet

Top concepts (fields/topics) attached by OpenAlex
Cited by: 2

Total citation count in OpenAlex
Citations by year (recent): 2025: 1, 2023: 1

Per-year citation counts (last 5 years)
References (count): 23

Number of works referenced by this work
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W2277785728
doi	https://doi.org/10.17485/ijst/2015/v8i36/79858
ids.doi	https://doi.org/10.17485/ijst/2015/v8i36/79858
ids.mag	2277785728
ids.openalex	https://openalex.org/W2277785728
fwci	0.0
type	article
title	Design of Web Crawler for the Client - Server Technology
biblio.issue	36
biblio.volume	8
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T12016
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	1.0
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1710
topics[0].subfield.display_name	Information Systems
topics[0].display_name	Web Data Mining and Analysis
topics[1].id	https://openalex.org/T11644
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9919000267982483
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1710
topics[1].subfield.display_name	Information Systems
topics[1].display_name	Spam and Phishing Detection
topics[2].id	https://openalex.org/T11478
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9898999929428101
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1705
topics[2].subfield.display_name	Computer Networks and Communications
topics[2].display_name	Caching and Content Delivery
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C13743948
concepts[0].level	2
concepts[0].score	0.9556949138641357
concepts[0].wikidata	https://www.wikidata.org/wiki/Q45842
concepts[0].display_name	Web crawler
concepts[1].id	https://openalex.org/C41008148
concepts[1].level	0
concepts[1].score	0.8426933288574219
concepts[1].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[1].display_name	Computer science
concepts[2].id	https://openalex.org/C73340581
concepts[2].level	5
concepts[2].score	0.7461680769920349
concepts[2].wikidata	https://www.wikidata.org/wiki/Q5463958
concepts[2].display_name	Focused crawler
concepts[3].id	https://openalex.org/C173576120
concepts[3].level	4
concepts[3].score	0.7387500405311584
concepts[3].wikidata	https://www.wikidata.org/wiki/Q2641220
concepts[3].display_name	Static web page
concepts[4].id	https://openalex.org/C136764020
concepts[4].level	1
concepts[4].score	0.7351977825164795
concepts[4].wikidata	https://www.wikidata.org/wiki/Q466
concepts[4].display_name	World Wide Web
concepts[5].id	https://openalex.org/C21959979
concepts[5].level	2
concepts[5].score	0.7208977937698364
concepts[5].wikidata	https://www.wikidata.org/wiki/Q36774
concepts[5].display_name	Web page
concepts[6].id	https://openalex.org/C521815418
concepts[6].level	4
concepts[6].score	0.6111780405044556
concepts[6].wikidata	https://www.wikidata.org/wiki/Q4182287
concepts[6].display_name	Web search engine
concepts[7].id	https://openalex.org/C11392498
concepts[7].level	3
concepts[7].score	0.6017809510231018
concepts[7].wikidata	https://www.wikidata.org/wiki/Q11288
concepts[7].display_name	Web server
concepts[8].id	https://openalex.org/C23123220
concepts[8].level	1
concepts[8].score	0.43419069051742554
concepts[8].wikidata	https://www.wikidata.org/wiki/Q816826
concepts[8].display_name	Information retrieval
concepts[9].id	https://openalex.org/C127613066
concepts[9].level	4
concepts[9].score	0.4316520392894745
concepts[9].wikidata	https://www.wikidata.org/wiki/Q557770
concepts[9].display_name	Web API
concepts[10].id	https://openalex.org/C130436687
concepts[10].level	3
concepts[10].score	0.42631226778030396
concepts[10].wikidata	https://www.wikidata.org/wiki/Q7978591
concepts[10].display_name	Web modeling
concepts[11].id	https://openalex.org/C79373723
concepts[11].level	3
concepts[11].score	0.4177629351615906
concepts[11].wikidata	https://www.wikidata.org/wiki/Q386275
concepts[11].display_name	Web development
concepts[12].id	https://openalex.org/C521306242
concepts[12].level	3
concepts[12].score	0.41484537720680237
concepts[12].wikidata	https://www.wikidata.org/wiki/Q190637
concepts[12].display_name	Web design
concepts[13].id	https://openalex.org/C110875604
concepts[13].level	2
concepts[13].score	0.1556936502456665
concepts[13].wikidata	https://www.wikidata.org/wiki/Q75
concepts[13].display_name	The Internet
keywords[0].id	https://openalex.org/keywords/web-crawler
keywords[0].score	0.9556949138641357
keywords[0].display_name	Web crawler
keywords[1].id	https://openalex.org/keywords/computer-science
keywords[1].score	0.8426933288574219
keywords[1].display_name	Computer science
keywords[2].id	https://openalex.org/keywords/focused-crawler
keywords[2].score	0.7461680769920349
keywords[2].display_name	Focused crawler
keywords[3].id	https://openalex.org/keywords/static-web-page
keywords[3].score	0.7387500405311584
keywords[3].display_name	Static web page
keywords[4].id	https://openalex.org/keywords/world-wide-web
keywords[4].score	0.7351977825164795
keywords[4].display_name	World Wide Web
keywords[5].id	https://openalex.org/keywords/web-page
keywords[5].score	0.7208977937698364
keywords[5].display_name	Web page
keywords[6].id	https://openalex.org/keywords/web-search-engine
keywords[6].score	0.6111780405044556
keywords[6].display_name	Web search engine
keywords[7].id	https://openalex.org/keywords/web-server
keywords[7].score	0.6017809510231018
keywords[7].display_name	Web server
keywords[8].id	https://openalex.org/keywords/information-retrieval
keywords[8].score	0.43419069051742554
keywords[8].display_name	Information retrieval
keywords[9].id	https://openalex.org/keywords/web-api
keywords[9].score	0.4316520392894745
keywords[9].display_name	Web API
keywords[10].id	https://openalex.org/keywords/web-modeling
keywords[10].score	0.42631226778030396
keywords[10].display_name	Web modeling
keywords[11].id	https://openalex.org/keywords/web-development
keywords[11].score	0.4177629351615906
keywords[11].display_name	Web development
keywords[12].id	https://openalex.org/keywords/web-design
keywords[12].score	0.41484537720680237
keywords[12].display_name	Web design
keywords[13].id	https://openalex.org/keywords/the-internet
keywords[13].score	0.1556936502456665
keywords[13].display_name	The Internet
language	en
locations[0].id	doi:10.17485/ijst/2015/v8i36/79858
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S2764746358
locations[0].source.issn	0974-5645, 0974-6846
locations[0].source.type	journal
locations[0].source.is_oa	True
locations[0].source.issn_l	0974-5645
locations[0].source.is_core	True
locations[0].source.is_in_doaj	False
locations[0].source.display_name	Indian Journal of Science and Technology
locations[0].source.host_organization	https://openalex.org/P4322615647
locations[0].source.host_organization_name	Indian Society for Education and Environment
locations[0].source.host_organization_lineage	https://openalex.org/P4322615647
locations[0].source.host_organization_lineage_names	Indian Society for Education and Environment
locations[0].license	cc-by
locations[0].pdf_url	https://sciresol.s3.us-east-2.amazonaws.com/IJST/Articles/2015/Issue-36/Article19.pdf
locations[0].version	publishedVersion
locations[0].raw_type	journal-article
locations[0].license_id	https://openalex.org/licenses/cc-by
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	Indian Journal of Science and Technology
locations[0].landing_page_url	https://doi.org/10.17485/ijst/2015/v8i36/79858
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5006249339
authorships[0].author.orcid	https://orcid.org/0000-0002-7997-8292
authorships[0].author.display_name	Md. Abu Kausar
authorships[0].countries	IN
authorships[0].affiliations[0].institution_ids	https://openalex.org/I9223067
authorships[0].affiliations[0].raw_affiliation_string	Department of Computer and System Sciences, Jaipur National University, Jaipur -302017, Rajasthan, India;
authorships[0].institutions[0].id	https://openalex.org/I9223067
authorships[0].institutions[0].ror	https://ror.org/01xrazc29
authorships[0].institutions[0].type	education
authorships[0].institutions[0].lineage	https://openalex.org/I9223067
authorships[0].institutions[0].country_code	IN
authorships[0].institutions[0].display_name	Jaipur National University
authorships[0].author_position	first
authorships[0].raw_author_name	Md. Abu Kausar
authorships[0].is_corresponding	False
authorships[0].raw_affiliation_strings	Department of Computer and System Sciences, Jaipur National University, Jaipur -302017, Rajasthan, India;
authorships[1].author.id	https://openalex.org/A5108541054
authorships[1].author.orcid
authorships[1].author.display_name	Vijaypal Singh Dhaka
authorships[1].countries	IN
authorships[1].affiliations[0].institution_ids	https://openalex.org/I9223067
authorships[1].affiliations[0].raw_affiliation_string	Department of Computer and System Sciences, Jaipur National University, Jaipur -302017, Rajasthan, India;
authorships[1].institutions[0].id	https://openalex.org/I9223067
authorships[1].institutions[0].ror	https://ror.org/01xrazc29
authorships[1].institutions[0].type	education
authorships[1].institutions[0].lineage	https://openalex.org/I9223067
authorships[1].institutions[0].country_code	IN
authorships[1].institutions[0].display_name	Jaipur National University
authorships[1].author_position	middle
authorships[1].raw_author_name	V. S. Dhaka
authorships[1].is_corresponding	False
authorships[1].raw_affiliation_strings	Department of Computer and System Sciences, Jaipur National University, Jaipur -302017, Rajasthan, India;
authorships[2].author.id	https://openalex.org/A5031599580
authorships[2].author.orcid	https://orcid.org/0000-0003-1583-9522
authorships[2].author.display_name	Sanjeev Kumar Singh
authorships[2].countries	IN
authorships[2].affiliations[0].institution_ids	https://openalex.org/I155125381
authorships[2].affiliations[0].raw_affiliation_string	Department of Mathematics, Galgotias University, Gr. Noida -201306, Uttar Pradesh, India;
authorships[2].institutions[0].id	https://openalex.org/I155125381
authorships[2].institutions[0].ror	https://ror.org/02w8ba206
authorships[2].institutions[0].type	education
authorships[2].institutions[0].lineage	https://openalex.org/I155125381
authorships[2].institutions[0].country_code	IN
authorships[2].institutions[0].display_name	Galgotias University
authorships[2].author_position	last
authorships[2].raw_author_name	Sanjeev Kumar Singh
authorships[2].is_corresponding	True
authorships[2].raw_affiliation_strings	Department of Mathematics, Galgotias University, Gr. Noida -201306, Uttar Pradesh, India;
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://sciresol.s3.us-east-2.amazonaws.com/IJST/Articles/2015/Issue-36/Article19.pdf
open_access.oa_status	diamond
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Design of Web Crawler for the Client - Server Technology
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T12016
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	1.0
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1710
primary_topic.subfield.display_name	Information Systems
primary_topic.display_name	Web Data Mining and Analysis
related_works	https://openalex.org/W2417265363, https://openalex.org/W2402678663, https://openalex.org/W2377634017, https://openalex.org/W3201526811, https://openalex.org/W2379145789, https://openalex.org/W2092706362, https://openalex.org/W2610451498, https://openalex.org/W4200125296, https://openalex.org/W2277785728, https://openalex.org/W757864652
cited_by_count	2
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	1
counts_by_year[1].year	2023
counts_by_year[1].cited_by_count	1
locations_count	1
best_oa_location.id	doi:10.17485/ijst/2015/v8i36/79858
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S2764746358
best_oa_location.source.issn	0974-5645, 0974-6846
best_oa_location.source.type	journal
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l	0974-5645
best_oa_location.source.is_core	True
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	Indian Journal of Science and Technology
best_oa_location.source.host_organization	https://openalex.org/P4322615647
best_oa_location.source.host_organization_name	Indian Society for Education and Environment
best_oa_location.source.host_organization_lineage	https://openalex.org/P4322615647
best_oa_location.source.host_organization_lineage_names	Indian Society for Education and Environment
best_oa_location.license	cc-by
best_oa_location.pdf_url	https://sciresol.s3.us-east-2.amazonaws.com/IJST/Articles/2015/Issue-36/Article19.pdf
best_oa_location.version	publishedVersion
best_oa_location.raw_type	journal-article
best_oa_location.license_id	https://openalex.org/licenses/cc-by
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	Indian Journal of Science and Technology
best_oa_location.landing_page_url	https://doi.org/10.17485/ijst/2015/v8i36/79858
primary_location.id	doi:10.17485/ijst/2015/v8i36/79858
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S2764746358
primary_location.source.issn	0974-5645, 0974-6846
primary_location.source.type	journal
primary_location.source.is_oa	True
primary_location.source.issn_l	0974-5645
primary_location.source.is_core	True
primary_location.source.is_in_doaj	False
primary_location.source.display_name	Indian Journal of Science and Technology
primary_location.source.host_organization	https://openalex.org/P4322615647
primary_location.source.host_organization_name	Indian Society for Education and Environment
primary_location.source.host_organization_lineage	https://openalex.org/P4322615647
primary_location.source.host_organization_lineage_names	Indian Society for Education and Environment
primary_location.license	cc-by
primary_location.pdf_url	https://sciresol.s3.us-east-2.amazonaws.com/IJST/Articles/2015/Issue-36/Article19.pdf
primary_location.version	publishedVersion
primary_location.raw_type	journal-article
primary_location.license_id	https://openalex.org/licenses/cc-by
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	Indian Journal of Science and Technology
primary_location.landing_page_url	https://doi.org/10.17485/ijst/2015/v8i36/79858
publication_date	2015-12-03
publication_year	2015
referenced_works	https://openalex.org/W1510634602, https://openalex.org/W3041537557, https://openalex.org/W1566984846, https://openalex.org/W2143642112, https://openalex.org/W2103980105, https://openalex.org/W2009888877, https://openalex.org/W1823505524, https://openalex.org/W6652848226, https://openalex.org/W4237126942, https://openalex.org/W2144782353, https://openalex.org/W2169557227, https://openalex.org/W1997438973, https://openalex.org/W2009289247, https://openalex.org/W2029500199, https://openalex.org/W1976232673, https://openalex.org/W4234255778, https://openalex.org/W1613836731, https://openalex.org/W2295141584, https://openalex.org/W2029341294, https://openalex.org/W2066636486, https://openalex.org/W2164542999, https://openalex.org/W2116494892, https://openalex.org/W2147164982
referenced_works_count	23
abstract_inverted_index.a	36, 56, 84
abstract_inverted_index.We	117
abstract_inverted_index.by	18
abstract_inverted_index.in	67, 123, 129
abstract_inverted_index.is	16, 23, 87
abstract_inverted_index.it	50, 106
abstract_inverted_index.of	8, 28, 64, 82
abstract_inverted_index.on	46, 102
abstract_inverted_index.to	53, 73, 88, 125
abstract_inverted_index.up	55
abstract_inverted_index.The	40, 77, 98
abstract_inverted_index.Web	19, 21, 137, 139
abstract_inverted_index.add	119
abstract_inverted_index.and	30, 111
abstract_inverted_index.can	118
abstract_inverted_index.for	25, 35
abstract_inverted_index.its	44
abstract_inverted_index.the	6, 26, 68, 103, 108
abstract_inverted_index.web	32, 37, 41, 85, 92, 115, 121, 127
abstract_inverted_index.Page	140
abstract_inverted_index.This	14
abstract_inverted_index.find	89
abstract_inverted_index.high	90, 113
abstract_inverted_index.many	120
abstract_inverted_index.most	69
abstract_inverted_index.page	128
abstract_inverted_index.such	83
abstract_inverted_index.this	75
abstract_inverted_index.time	72, 96
abstract_inverted_index.with	5
abstract_inverted_index.basis	48
abstract_inverted_index.build	54
abstract_inverted_index.could	60
abstract_inverted_index.hence	49
abstract_inverted_index.sorts	63
abstract_inverted_index.store	2
abstract_inverted_index.these	62
abstract_inverted_index.tool.	39
abstract_inverted_index.which	59
abstract_inverted_index.works	101
abstract_inverted_index.Search	0, 135
abstract_inverted_index.behind	80
abstract_inverted_index.frame.	97
abstract_inverted_index.latest	31
abstract_inverted_index.pages.	116
abstract_inverted_index.quick,	10
abstract_inverted_index.search	12, 38
abstract_inverted_index.system	100
abstract_inverted_index.within	94
abstract_inverted_index.Engine,	136
abstract_inverted_index.becomes	51
abstract_inverted_index.changes	65
abstract_inverted_index.content	45
abstract_inverted_index.crawler	86, 122
abstract_inverted_index.deliver	9
abstract_inverted_index.engines	1
abstract_inverted_index.limited	95
abstract_inverted_index.locally	4
abstract_inverted_index.minimal	70
abstract_inverted_index.overlap	109
abstract_inverted_index.problem	110
abstract_inverted_index.purpose	7
abstract_inverted_index.quality	91, 114
abstract_inverted_index.reduces	107
abstract_inverted_index.regular	47
abstract_inverted_index.thought	79
abstract_inverted_index.Crawler,	138
abstract_inverted_index.Overlap,	134
abstract_inverted_index.complete	29
abstract_inverted_index.crawler.	20
abstract_inverted_index.crawling	22
abstract_inverted_index.document	33, 42
abstract_inverted_index.download	126
abstract_inverted_index.identify	61
abstract_inverted_index.modifies	43
abstract_inverted_index.parallel	124, 130
abstract_inverted_index.proposed	99
abstract_inverted_index.scanning	71
abstract_inverted_index.collected	17
abstract_inverted_index.designing	81
abstract_inverted_index.documents	93
abstract_inverted_index.downloads	112
abstract_inverted_index.essential	78
abstract_inverted_index.framework	58
abstract_inverted_index.gathering	34
abstract_inverted_index.necessary	24, 52
abstract_inverted_index.Technology	105
abstract_inverted_index.abilities.	13
abstract_inverted_index.accessible	11
abstract_inverted_index.accomplish	74
abstract_inverted_index.successful	57
abstract_inverted_index.Technology,	133
abstract_inverted_index.information	3, 15
abstract_inverted_index.maintenance	27
abstract_inverted_index.proficiently	66
abstract_inverted_index.Client-Server	104, 132
abstract_inverted_index.way.Keywords:	131
abstract_inverted_index.modifications.	76
cited_by_percentile_year.max	95
cited_by_percentile_year.min	89
corresponding_author_ids	https://openalex.org/A5031599580
countries_distinct_count	1
institutions_distinct_count	3
corresponding_institution_ids	https://openalex.org/I155125381
citation_normalized_percentile.value	0.17784665
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False