How to Create a Sitemap From Website Scraping (Small Websites)

This guide will teach you how to use content from across your site to train your custom AI Agent.

๐Ÿค“

This guide is for smaller websites. If your site is larger, please see the guide on How To Create a Sitemap From Website Crawling

Step 1: Find your target website

๐Ÿšง

Make sure this site is publicly accessible or the tool won't be able to access the information


Step 2: Use CustomGPT tool to generate your sitemap

This tool will generate your sitemap for you.

Paste your website link in the Enter the starting URL field > click Generate Sitemap



Step3: Check your Email Address



Step 4: Add the new sitemap to the Data section of your project

a) When you click on the link, it will appear.

b) Copy your sitemap link

You can use the new sitemap to start a brand new project or add it to an existing project.