In my robots.txt I have a disallowing rule like:
User-agent: *
Disallow: /_hcms/
I want to disallow anything placed inside of _hcms and in its nested folders, like /_hcms/a/ or /_hcms/b/.
Should I add a disallowing rule for such subfolders, like
User-agent: *
Disallow: /_hcms/
Disallow: /_hcms/a/
Disallow: /_hcms/b/
Or is the existing rule Disallow: /_hcms/ makes subfolders disallowed? Or should it even be Disallow: /_hcms, without closing slash?
You can use the existing rule
Disallow: /_hcms/to disallow anything placed inside of_hcmsand in its nested folders. The trailing slash is optional and will not affect the behavior of the rule. Therefore, you do not need to add additional rules for subfolders.