How to Create a Sitemap From Website Scraping (Small Websites)

This guide will teach you how to use content from across your site to train your custom AI Agent.


This guide is for smaller websites. If your site is larger, please see the guide on How To Create a Sitemap From Website Crawling

Step 1: Find your target website


Make sure this site is publicly accessible or the tool won't be able to access the information

Step 2: Use CustomGPT tool to generate your sitemap

This tool will generate your sitemap for you.

Paste your website link in the Enter the starting URL field > click Generate Sitemap

Step3: Check your Email Address

Step 4: Add the new sitemap to the Data section of your project

a) When you click on the link, it will appear.

b) Copy your sitemap link

You can use the new sitemap to start a brand new project or add it to an existing project.