An insight to Search Engine

an insight to search engine
Last modified on Mar 10, 2021

What is search engine?

A search engine can be thought of a block of computer code which continuously crawls the web and index whatever new web content it finds into a directory. The web content can be a new website, a single webpage, updated webpage or any other file. The computer code is generally called a bot or crawler. The crawler searches the web for new or updated webpages if it finds, indexes and go on finding new pages. When user search something on the web it is served with most relevant content which has been indexed in the database.


Googlebot is the official name of Google's web crawler. It is a large computer code which continuously crawls like a spider in world wide web. It is designed to be run simultaneously by thousands of machines to improve performance and scale as the web grows. There are two types of Googlebot crawler. Smartphone crawler and Desktop crawler. Smartphone crawler is the major Googlebot crawler for the web since 2018. This changes has been done seeing the rapid growth of mobile users. Desktop crawler has become secondary Googlebot since then.

How search engine works?

The process of search engine can be broken down into three steps.

  • Crawling
  • Indexing
  • Serving


Crawling is the process of continuously searching for new webpages. When search engine bots or crawlers discovers a new webpage it crawls the complete document, in the process of crawling it may discover some new webpages, then it go on crawling newly find webpages. And this process never stops. So, we can say crawling is the process of discovering and scanning new webpages form the list of already known webpages. Sometimes webmaster submits a new website into search console. This is done by adding a new website property to the search console. The homepage of the new website is crawled and all the pages which are reachable from the home page is crawled at later stage one by one.

Google also periodically checks whether the content of the website has changed, updated or some new webpages are added. It is only upto Google algorithm to decide when to crawl already known pages. But if you want Google to crawl your newly added webpages into the website. Submit a sitemap. Submitting a sitemap is an indication to Google that something has changed in the webiste. The content might have updated or some new pages are added or some old pages has been removed, it can be anything.


After the page is crawled by Google search engine bots. Next step is to analyze the webpage which has been crawled. Analysis of the page starts form the meta tags in the head section of the webpage. Search engine sees the information in the meta tags and understands what the page is all about? After the meta tag is parsed, search engine moved on to the body section of the page. It analyses lots of elements like the page header, title, images, videos, rich content if any. Bots also analyses all the links found in the pages. Depending upon the dofollow and nofollow links are followed or ignored.

Search engine algorithms are very advanced but still its faces difficulty in parsing rich media content. Though it understands the image and videos but text contents are more preferred. Do not forget to add alt attribute in the image tag. The alt attribute helps search engine to understand aboout the image. The alt attribute is a important factor and it helps improve SEO. All the information and media found in pages are stored in a huge databases which spans across many computers with huge storage capacity. This is called indexing.

Serving & Ranking

When user enters a query, database is searched and more than one result which matches user's query can be found. But, the results which are most relevant are presented to the user in the first page of the SERP(Search engine result pages). Relevancy are determined based on lots of factors like quality of the web content, page speed, mobile friendly nature of the webpage etc. Google considers lots of factors to rank the result even on the first page. With number of mobile user growing rapidly, mobile friendly nature of the webpages are an important factor to rank in Google search engine results. Google continuously updates it algorithm to always serve the best relevant result to the user.

Apart from meta tags Google uses another kind of data type to gain information about webpages and that is Structured data. Meta tags are unstructured data which crawlers find it difficult to understand. Structured data presented in the JSON-LD format helps search engine to easily understand the content of the pages. Structured data is one of important ranking factor these days. Structured data assists search engine to generate rich results in the Google search.

Published on Mar 10, 2021

Get in Touch

Rank# SEO

With extensive two year research in Web designing , Development, SEO and Internet Marketing field, we are equipped with every latest web development technologies in our armoury. You can rely on us for making your business next big thing in the internet.

Contact us | About us | Policy Terms & Conditions

: +91 811 644 6262

All Rights reserved     Rank Hash SEO