Fast guide of robots.txt

Published 23-11-2015

In this fast guide we will see what is and how the robots.txt file is used.

What is?

The robots.txt file basically says to them to the web search engines to that parts of your site can accede and consequent mind to rake.


This file is due to call €œrobots.txt€, and must be in the root directory of your site.

Perhaps there are some pages of your site that you do not want that they are raked if they are not going to be useful for the users when they find them in the results search. If you want to avoid that the web search engines track your pages, then we used this file.


To prevent that my Web leaves in the results and search

There are a few ways more to prevent than your content appears in the results search, as for example adding metaetiqueta €œNOINDEX€, using .htaccess to protect directories with password or to use the tools for webmasters of Google to clear content that already has been tracked.

The USA safer methods for the delicate content

You would not have to only remain calm blocking with robots.txt material delicate or confidential. One of the reasons is because if there are connections to that URL somewhere of Internet the web search engines still could make reference to that URL. In addition, the web search engines that do not follow the norms of Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a peculiar user could study the directories and subdomains in your robots.txt and guess the URL of the content that you do not want that he is seen. There are safer alternatives as encriptar the content or protecting it with a password in .htaccess.


To fight commentaries Spam with €œnofollow€

To establish the value of the attribute €œrel€ of a connection to €œnofollow€ says to him to Google that some connections of your site do not have to be followed nor to pass the reputation from your site to the pages with which you connect. To put €œnofollow€ to a connection is obtained using rel= €œnofollow€ within the anchor label

Why it can serve this? If your site has a blog with the commentaries activated public, the connections of those commentaries can pass your credibility to pages with which you would not feel comfortable connecting you yourself. To use nofollow in these connections added by the users assures you that you are not reporting of your very obtained reputation to a Spam site.


It adds of automatic form €œnofollow€

If you are arranged to give credit to the connections added by third parties (for example, if you trust some user of your site), then it is not necessary to use nofollow in the connections. Of all ways, to connect to sites that Google considers Spam can affect to the reputation of your site.


To make reference to a Web without passing credibility to him

Also you can use nofollow when you are writing contained for your site and you want to make reference to another webpage, but without commenting his credibility. For example, it imagines that you are writing an entrance on the Spam commentaries and want to mention a site that recently put commentaries of Spam in your blog. You want to warn to the others than they do in that site, so you include a connection in your text, but obvious you do not want to give credit him with your connection.

We hope that you it has been of help, until the next one!


(source: google)

Categories: Design Web

It shares this article

It comments this article

© 2020 NonStop-Press - Sitemap