Build an AI agent from a slow-loading website

What is Website Slow Mode?

Website Slow Mode allows longer timeouts when indexing web pages. It's specifically designed for websites with heavy JavaScript, where content may take longer to fully load.

When enabled, this mode gives our crawler more time per page, improving the chances of capturing all relevant content accurately, even on slow or complex sites.

Why use Website Slow Mode?

Some websites may not be fully indexed under normal settings due to slow-loading JavaScript. This can lead to issues such as:

• Pages listed in the sitemap are found but not indexed properly due to timeouts or inconsistent loading.
• Sites without a sitemap fail to index beyond the homepage, even though our crawler attempts to discover pages dynamically. (See How CustomGPT builds AI agents from websites.)

These issues result in incomplete or unreliable AI agents.

Solution: Use Website Slow Mode

By selecting the Website Slow Mode option during agent creation, you enable extended page load timeouts. This allows our system to index content more reliably from JavaScript-heavy or slow-loading websites.

📘

Note:

Use this option if your website has complex JavaScript and you notice that some pages are missing after indexing.

How to use Website Slow Mode

1. Click the New Agent button in the top-right corner.

2. Scroll down and select the Website Slow Mode option.

3. Enter your website’s full URL.

4. Click Create Agent to begin the indexing process. This starts the process with longer timeouts per page, which improves indexing accuracy.

How Website Slow Mode affects indexing

When you use Website Slow Mode:

• The Extend allowed time per page setting is turned on by default.
• Indexing may take longer, but more pages are captured correctly.

To learn more about this setting, see: Extend allowed time per page.