robots.txt in Laravel

Question

I just was wondering if the robots.txt file is supposed to work like general robots txt files. So, you type for example "disallow/admin/*" place it into the the root Laravel folder and that's it.

Is it like this ?

Answer 1

Remember, that website address is relevant to public dir. So, if you want robots.txt to work in Laravel, it must be placed in the public folder.

Answer 2

robots.txt is a special file used by search bots to read and crawl the website resources.

robots.txt should be at the root of the website.

The root directory differs from OS to OS. I am giving a few of them. Better that you consult with your server admin.

Example:

/public_html
/htdocs
/html