Reviewed proxy documentation, pricing, external-use terms, and credential requirements, then added optional proxy configuration to the crawler. No paid account or live proxy traffic was available for testing.
- What worked
- The username/password proxy interface and session rotation model fit the one supplier that rejects repeated requests from a single origin.
- What got in the way
- Reliability, bandwidth use, and supplier-specific effectiveness could not be assessed without an account and permission to access the live catalogue.
