Sitemaps and robots.txt files are important for search engines to crawl and index your website's pages. To ensure your website's optimal performance in search engine results, it is important to have these files at the root of the website.
Now, you can either generate and update these files manually, which is a tedious process, or set up a process that automates these tasks for you. In this article, we'll walk through a few easy steps to generate and update sitemaps automatically in a Next.js application. Let's get started!
Configure ‘next-sitemap’
First, install the next-sitemap package, which will automate the process for us. Next, add the "postbuild": "next-sitemap" script in your package.json file. After that, create an environment variable named NEXT_PUBLIC_HOST_URL containing your website's root domain (e.g., 'https://www.yourwebsite.com').
Finally, create a next-sitemap.config.js file in the root of your website with the following code:
/** @type {import('next-sitemap').IConfig} */
module.exports = {
generateRobotsTxt: true,
siteUrl: process.env.NEXT_PUBLIC_HOST_URL,
};
For additional configuration options and instructions on generating server-side sitemaps, refer to the next-sitemap official documentation.
At this point, most of the configuration tasks are complete. To prevent the sitemap files from being traced in your Git history, include the following lines in your .gitignore file:
/public/robots.txt
/public/sitemap.xml
/public/sitemap-0.xml
Test the configuration
To test the configuration, run the postbuild command (e.g., npm run postbuild), which will generate the sitemaps and robots.txt files and place them in the public directory. Check out the generated files and ensure they contain your website's accurate details.
That's it! With these steps, every time your Next.js application is built, the postbuild script will run and update the sitemaps. To test the configuration in production, commit the changes, build your website in production, and make sure the following pages exist:
- www.yourwebsite.com/robots.txt
- www.yourwebsite.com/sitemap.xml
- www.yourwebsite.com/sitemap-0.xml
Congratulations! Your Next.js website now has a setup to generate and update sitemaps automatically every time it is built. Hope you find this article useful and feel free to share it with others who might benefit from it as well.