The fields of the WebSite class are listed below. For a complete list of WebSite class members, see the WebSite Members topic.
Public Instance Fields
 Authentication | Authentication settings to use to connect to a web site. |
| CrawlDepth | Number of links from the start page to follow. |
| ExcludeFilters | Filename filters indicating which pages should be not indexed. |
| FormAuthentication | Form authentication settings to use to connect to a web site that uses HTTP GET or POST requests for authentication. |
| IgnoreRobotsTxt | If true, the Spider will crawl areas of the site even if robots.txt excludes them. |
| IncludeFilters | Filename filters indicating which pages should be indexed. |
| MaxItemsToIndex | Use this setting to limit the number of pages the Spider should index on this web site. |
| MaxSizeToIndex | Use this setting to limit the maximum size of files that the Spider will attempt to access. |
| PageTimeoutSeconds | Number of seconds to wait before timing out when trying to download a single page. |
| Proxy | Proxy settings to use to connect to a web site. |
| ServerFilters | List of server names other than the starting server that the Spider can visit. |
| SiteTimeoutSeconds | Use this setting to limit the amount of time the Spider will spend crawling pages on this web site. |
| Url | Starting URL for the crawl. |
| UserAgent | Name to use to identify this program to the web server |
| WaitBetweenPagesMillis | Number of milliseconds to wait between page downloads. |
See Also
WebSite Class | dtSearch.Spider Namespace