A robots.txt file lets you restrict the access of online search engine robots that crawl the web and it might avoid these robots from accessing particular directories and internet pages. Furthermore, it specifies the place the XML sitemap file is located. Your social networking profiles are an extension within your https://anneb896xxd4.shoutmyblog.com/profile