michaeljjames / silverstripe-robots
Adds a Robots.txt file that is configurable from /admin/settings/.
Installs: 98
Dependents: 0
Suggesters: 0
Security: 0
Stars: 1
Watchers: 3
Forks: 0
Type:silverstripe-module
Requires
- silverstripe/cms: >=3.0
- silverstripe/framework: >=3.0
This package is not auto-updated.
Last update: 2024-04-14 02:08:00 UTC
README
This module will allow you to have a dynamic robots.txt
Installation
$ composer require michaeljjames/silverstripe-robots
You can also install manually by downloading the zip and extracting it into your site root.
You'll also need to run dev/build.
Configuration
It is completely configurable inside /admin/settings
To configure your robots.txt file paste in your robots.txt confiuration into the textarea inside the robots tab in admin/settings/ or use the example below.
User-agent: *
Disallow: /admin
Disallow: /dev
Disallow: /?flush
Disallow: /assets
Allow: /
User-Agent: Googlebot-Image
Disallow: /admin
Disallow: /dev
Disallow: /?flush
Allow: /
Sitemap: https://www.domain.com/sitemap.xml