---
updatedAt: 2025-05-13T21:39:22.000Z
agentTools:
  projectIndex: https://docs.customgpt.ai/llms.txt
---

# Create a sitemap from website scraping (small websites)

This step-by-step guide explains how to generate a sitemap from a small website and use it to train your CustomGPT agent.

## [\[VIDEO\]](https://www.youtube.com/watch?v=ryN98OsDqFQ\&t=1s)

1. Open the [CustomGPT sitemap tool](https://adorosario-customgpt-st-build-sitemap-from-scrappy-crawl-6na52c.streamlit.app/) and paste your website URL into the URL field.

<Image align="center" className="border" border={true} src="https://files.readme.io/2ea0232-Desktop_screenshot_2.png" />

> 📘 Note:
>
> Make sure the website you want to scrape is publicly accessible. The tool cannot access private or restricted sites.

2. Click **Generate Sitemap** to start the process.

<Image align="center" className="border" border={true} src="https://files.readme.io/eda00ea-Sitemap_Generator_From_Website_Crawling.png" />

3. Check your email inbox for the sitemap link sent by the tool.

<Image align="center" className="border" border={true} src="https://files.readme.io/9198626-Email.png" />

4. Click the link in the email to open the sitemap.

<Image align="center" className="border" border={true} src="https://files.readme.io/e08e897-Desktop_screenshot_3.png" />

5. Copy the sitemap URL.

<Image align="center" className="border" border={true} src="https://files.readme.io/d107826-Sitemap_from_Website_Scraping_1.png" />

To continue with a larger website, see the [Create a sitemap from website crawling guide](https://docs.customgpt.ai/docs/how-to-create-a-sitemap-from-website-crawling-1#/).

You can use the new sitemap to start a [brand new agent](https://docs.customgpt.ai/docs/create-ai-agent-project) or [add it to an existing agent](https://docs.customgpt.ai/docs/manage-ai-agent-data).