Yes, by using the 'robots.txt' file, you can instruct search engi

Is there any way to prevent a spider from grabbing urls that you want to keep off search engines?

Question Posted / Nitin Chauhan

1 Answers
983 Views
I also Faced
E-Mail Answers

Answer Posted / Nitin Chauhan

Yes, by using the 'robots.txt' file, you can instruct search engine spiders (or robots) which pages or sections of your website should not be crawled or indexed. You can specify certain directories or URL patterns to be excluded.

Is This Answer Correct ?

0 Yes

0 No

Post New Answer View All Answers

Please Help Members By Posting Answers For Below Questions

What are your good qualities?

857

What all verticals have you handled in your seo career so far?

934

Have you ever dealt with link penalties? How?

965