What is Robot.txt?.
The Robot.txt Exclusion Standard is also known as the Robot.txt Exclusion Protocol or robot.txt protocol. You need a robot.txt file only your site includes content that you don't want search engines to index like your site Admin Panel.
Note - Robot.txt is important for SEO. you should know robot.txt is on your website or not?..if not, you have to create Robot.txt because you can restrict your search crawler through robot.txt and you can also restrict through meta tag. if your website is not crawl by the crawler index will not be done. If index is not done, your web page will not show in the search engine.
If you want to disallow your admin panel you can use this code..
0 comments: