Softpedia >Windows >  Applications filed under: control spider  (1 item)

Apps filed under: control spider

  RSS Feed for this tag 1 application totalLast updated: May 10th 2009, 03:48 GMT

     
  • 1,320 downloads
  • 80 KB
Robots txt generator

Robots txt generator 1.01

If you want to control the robots or spiders that visit your site, this application is the solution

May 10th 2009, 03:48 GMT
Windows 2003 / Windows Vista / Windows XP / Windows 2K