Golgappa.net | Golgappa.org | BagIndia.net | BodyIndia.Com | CabIndia.net | CarsBikes.net | CarsBikes.org | CashIndia.net | ConsumerIndia.net | CookingIndia.net | DataIndia.net | DealIndia.net | EmailIndia.net | FirstTablet.com | FirstTourist.com | ForsaleIndia.net | IndiaBody.Com | IndiaCab.net | IndiaCash.net | IndiaModel.net | KidForum.net | OfficeIndia.net | PaysIndia.com | RestaurantIndia.net | RestaurantsIndia.net | SaleForum.net | SellForum.net | SoldIndia.com | StarIndia.net | TomatoCab.com | TomatoCabs.com | TownIndia.com
Interested to Buy Any Domain ? << Click Here >> for more details...


Is there any way to prevent a spider from grabbing urls that you want to keep off search engines?



Is there any way to prevent a spider from grabbing urls that you want to keep off search engines?..

Answer / Nitin Chauhan

Yes, by using the 'robots.txt' file, you can instruct search engine spiders (or robots) which pages or sections of your website should not be crawled or indexed. You can specify certain directories or URL patterns to be excluded.

Is This Answer Correct ?    0 Yes 0 No

Post New Answer

More Search Engine Optimization SEO Interview Questions

Which of the following options is correct regarding the Keyword Effectiveness Index (KEI) of a particular keyword? ► a. It is directly proportional to the popularity of the keyword ► b. It is inversely proportional to the competition for the keyword ► c. It is directly proportional to the chances of the keyword ranking on the first page of the Google search results

1 Answers  


Explain me what is web hosting?

1 Answers  


Tell me which is the most important area to include your keywords?

1 Answers  


What is Google Autocomplete?

5 Answers  


What is search console? Why do you use search console.

1 Answers  


Are RSS/Atom feeds returned in Googles search results? ► a. Yes ► b. No

1 Answers  


Tell us some seo blogs that you frequently read?

1 Answers  


Explain some major commonly used search engines?

1 Answers  


What is your understanding of YMYL pages?

1 Answers  


A Hallway Page is used to: ► A. Attract visitors from the search engines straight onto the Hallway Page ► B Organizes the Doorway Pages ► C Helps people navigate to different Doorway Pages ► D. Enables search engine bots to index the Doorway Pages

1 Answers  


What is keyword prominence?

1 Answers  


Tell me how do you separate words in url?

1 Answers  


Categories