Sitemaps & Robots.txt in NextJS

Saroj Subedi
2 min readSep 25, 2021


Photo by Rock'n Roll Monkey on Unsplash

If we want to communicate and provide certain information to search engines & crawlers about your website, then it is good to have sitemaps and robots.txt file. With those on place in site, we can :

  • Define paths and location of sites which are public and private
  • Pin pages as achieve so crawler will know
  • Define priority level, page updated frequency rate
  • Provide search engines info which pages are important in order to index them faster than others

Here is basic example of sitemap:

<urlset xmlns="">

For detailed specification, check here.

Create a Sitemap:

For this example I am assuming our site is a normal blog with lots of posts.

Create a file called pages/api/sitemap.js

import { getBlogPosts } from '../utils/data'
export default async function sitemap(req, res) { const postLists = await getBlogPosts()

const entries = => getSitemapEntry({
pathname: `/post/${post.slug}`
pathname: '/',
priority: 1
const sitemap = `
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="">

res.writeHead(200, {
'Content-Type': 'application/xml'
return res.end(sitemap);
const hostname = ""function getSitemapEntry({pathname, priority = 0.5}) {
return `

Now we need to map /api/sitemap to /sitemap.xml. For this go to next.config.js file

module.exports = {
async rewrites() {
return [
source: '/sitemap.xml',
destination: '/api/sitemap',

We can now access sitemap in http://localhost:3000/sitemap.xml

Lets create a robots.txt file. Go to public folder and create robots.txt file and paste this:

User-agent: *
Disallow: /api

Check here if you want to have more information about robots.txt