“Rotate every request” is a useful default for some scraping jobs, but it is not a complete strategy. The right rotation pattern depends on what the target expects from a normal session and what your collector needs from a response.

If a job fetches independent public URLs, frequent rotation can be simple and effective. If the job follows pagination, retains a short-lived cookie, or needs to reproduce a localized response, changing the IP too often can make the workflow harder to debug.

The goal is not maximum rotation. The goal is a predictable request pattern that returns complete records at a sensible cost.

Classify the workflow before choosing a proxy mode

Start by placing a job in one of three groups.

1. Independent page collection

Examples include product-detail pages, public listing URLs, and a broad sample of search results. Each request can succeed or fail on its own. These jobs usually work well with rotating sessions because the collector does not need to preserve browser state from one URL to the next.

2. Short stateful sequences

Examples include clicking through a few paginated results, confirming a location-specific page state, or reproducing a parser bug. These jobs can benefit from a sticky session with a fixed lifetime. The same network identity makes the sequence easier to inspect.

3. Long-running browser workflows

Some authorized automation flows need an explicit session plan, careful stop conditions, and a human review process. Do not assume that a residential proxy removes those requirements. The session model should match the approved workflow and the site’s terms.

Use a session budget, not an open-ended session

A sticky session should have a reason and an expiry. Define both before the job starts:

maximum requests in the session;
maximum time in the session;
target country or city;
condition that ends the session;
condition that stops the entire job.

For example, a collector may use one sticky session for ten paginated requests or fifteen minutes, whichever comes first. When that budget is reached, it creates a new session or ends the run. This is easier to audit than keeping an identity indefinitely.

Separate retries from rotation

Rotation is not a retry policy. A good collector distinguishes between errors that may be transient and errors that require a change in the job.

Outcome	First action	Retry?
Timeout	Record the event and apply backoff	Yes, with a small limit
Required field missing	Check parser and page type	Usually no
Wrong region or currency	Check target configuration	No until corrected
Expected page unavailable	Record availability state	No
Authentication or policy boundary	Stop and review scope	No

When a failure is retried, change as little as possible. If the retry also changes the parser, request rate, headers, and proxy location, you will not know what affected the result.

Measure the health of the collector

Monitor outcomes at the record level, not only the network level. Useful measures include:

required-field completion;
empty-page and unexpected-page rate;
timeout rate;
retries per valid record;
median and p95 response time;
requested versus observed location;
cost per valid record.

These metrics show whether rotation is helping the data pipeline or merely producing a different set of errors.

Add location deliberately

For localized SERP monitoring, ecommerce checks, ad verification, and regional research, the requested location belongs in the output record. Store the country, city when relevant, collection time, and parser version. This avoids a common reporting mistake: comparing two values as if they came from the same market when they did not.

Thordata’s residential proxy documentation covers location targeting and both rotating and sticky session modes over HTTP/HTTPS. A sensible rollout is to benchmark the exact countries and session patterns your collector needs, using a fixed sample and one variable at a time. Explore Thordata Residential Proxies when you are ready to test a route.

A simple decision rule

Use rotation when the page requests are independent. Use a short sticky session when the workflow needs continuity. Slow down when a target starts returning incomplete records. Stop when the job moves outside the approved scope or when a response cannot be interpreted confidently.

This approach is less dramatic than “rotate everything,” but it produces logs that are easier to explain, budgets that are easier to control, and data that is more useful downstream.