Exclude headers, footers, and HTML elements from web crawls

Goal

Exclude header, footer, or other unwanted HTML elements from being indexed during web content crawling in Fusion.

Environment

Fusion 5.5 and above

Guide

Fusion offers two approaches for excluding HTML elements from crawled pages. The choice depends on whether the exclusion applies broadly across multiple datasources or only to a single connector.

Use parser-level exclusions

Parser-level configuration allows you to exclude HTML elements for all datasources that share the same parser. This approach is recommended when:

• Multiple connectors use the same parser.

• The same elements (such as headers, footers, or navigation menus) should be excluded across multiple sites.

To configure parser exclusions:

1. Open Fusion’s Parsers view in the UI.

2. Edit the parser associated with your datasource.

3. Add the relevant HTML tags or CSS classes to the exclusion rules (Exclude filters).

4. Save and reprocess content as needed.

Use connector-level exclusions

Connector-level configuration allows you to exclude elements directly in the connector’s settings. This approach is recommended when:

• The exclusion only applies to a single datasource.

• A site has unique HTML structures that do not apply to other datasources.

To configure connector exclusions:

1. Open Fusion’s Connectors view in the UI.

2. Edit the connector associated with the datasource.

3. Look for the excludeTags configuration.

4. Define the exclusion rules (for example, CSS selectors for header or footer classes).

5. Save the configuration and re-run the crawl.

Best practice

Both methods are valid. The parser approach simplifies maintenance when exclusions apply across multiple datasources. The connector approach is better suited for datasource-specific rules.