Meta Robots Tag or RobotsMetaTag – allows a page designer or editor to tell search engines whether or not the page and/or its links should be indexed and/or followed by the visiting search engine spider. While a robots.txt file is often used in the root directory of the site to tell search engine robots what pages and links to index, sometimes webmasters will set this information at the page level instead, using the robots META tag. A RobotsMetaTag is not generally required if a robots.txt file is used.
