User-agent: *
list folders robots are not allowed to index 
Disallow: /cgi-bin/folder/
Disallow: /entropybanner/folder/
Disallow: /GeneratedItems/folder/