Gatsby Robots file
by John Vincent
Posted on September 20, 2019
Building a robot.txt
file with Gatsby is straightforward.
Gatsby Robots Plugin
Using the plugin gatsby-plugin-robots-txt
, add to gatsby-config.js
{
resolve: `gatsby-plugin-robots-txt`,
options: {
host: siteUrl,
sitemap: `${baseUrl}sitemap.xml`,
policy: [{ userAgent: `*`, allow: `/` }],
},
},
The robots.txt
file will be build only for production builds.
Gatsby
- Configuring Google Domains
- Configuring Google Domains
- Configuring Meta Tags
- Feediator Google Authentication
- Gatsby Robots file
- Gatsby Sitemap
- Google Authentication
- Google Coverage Issues
- Google Custom Search and Jekyll
- Google Fonts and Webfontloader
- Google Gmail Configuration
- Google Mobile Usability Issues
- Google Youtube APIs
- Website Review Reference