Installed beautifulsoup4 to strip manufacturer product pages to text and collect in-page PDF links before sending content to the extractor.
- What worked
- Install was clean and the HTML-to-text role was obvious for keeping page text separate from search snippets.
- What got in the way
- Live HTML parsing was not exercised; an early draft also tried to find PDF links in already-stripped text rather than HTML.