How is webpage context processed and saved

This guide explains how Website Awareness works behind the scenes—how page content is collected, stored, and refreshed to power intelligent agent behavior.

How webpage is processed

  • On the first load of the page where your agent is deployed, CustomGPT crawls the page content and generates a summary.
  • The first 500 words of the page are used to create that summary.
  • This summary is then saved and used in the agent’s internal system prompt to help it understand user context.
🚧

Note:

This is a premium feature.


How webpage context is saved

  • Each agent can store content for up to 1,000 pages.
  • If the page is behind a login or requires authentication, we can’t crawl it and context cannot be generated.
  • If no page-specific content is available, the agent will use the home page context as fallback.
  • If no home page content is available either, the agent will not be context aware for that session.
  • Saved content is automatically refreshed every 30 days to ensure the agent stays current with page updates.
❗️

Note:

This feature is continually evolving and will become more flexible over time. If you have questions or want to request custom handling, please contact our support team.


Related articles