Home / Blog / Circuit Breaker Pattern for the Amazon Product Data API in Python

Circuit Breaker Pattern for the Amazon Product Data API in Python

A client-side circuit breaker for the SellerMagnet API: stop paying for repeated billed 502/503 attempts during a real upstream outage, with a tested Python implementation that sits above the existing get_with_retry policy instead of replacing it.

November 22, 2026
6 min read
SellerMagnet Team
Share & Bookmark
Terminal showing a Python circuit breaker that stops calling the Amazon API for free after 3 billed 502 errors in a row

A circuit breaker for the SellerMagnet API is a small client-side wrapper that stops calling the API for a while after several SEPARATE lookups have each already exhausted their own retries on a billed 502 or 503 - instead of paying for 50 more lots of retries to relearn the same outage during a batch job. It sits on top of the existing get_with_retry policy, not instead of it: retrying still handles one call's own blip, the breaker handles a dependency that stays down across many calls.

Key Takeaways

  • A circuit breaker reacts ACROSS calls; get_with_retry already reacts INSIDE one call - use both together, not one instead of the other.
  • Only billed 502/503 and connection errors/timeouts count toward the breaker; a 404 (real answer), a free-gate 400/401/403/429, or a lone refunded 500 do not.
  • Once open, every further call raises locally for free - no HTTP request, no credit spent - until reset_timeout passes.
  • The half-open trial after reset_timeout is NOT a free 1-credit probe: it still runs through get_with_retry, so a failing trial can cost up to 4 credits before the circuit reopens.
  • A successful trial closes the circuit and resets the failure count to zero; a failing trial reopens it immediately, without waiting for failure_threshold more failures.

Which SellerMagnet errors should actually open the circuit?

Only the ones that say the dependency itself is unhealthy, not the ones that are a real answer about one ASIN or a policy limit. See "Status codes and billing" in the API rate limits and error handling post for the full table; the breaker only cares about the rows below.

What counts toward the breaker's failure count, and why.
StatusMeaningBilledCounts toward the breaker?
502upstream erroryescounted - signals a real, billed instability
503per-marketplace circuit breaker, Retry-Afteryescounted - the API's own breaker signal
Timeout / ConnectionErrorno response at alln/a (never billed)counted - same as a 502
404ASIN not on this marketplaceyesNOT counted - a real answer, not instability
429per-key concurrency capnoNOT counted - get_with_retry already waits it out
401 / 403 / gate 400key, credits or a bad parameternoNOT counted - a policy answer, not an outage
500 (alone)internal error, refundednoNOT counted - get_with_retry's own 1 retry handles it
Terminal-theme flow of a call through the circuit breaker, get_with_retry and the SellerMagnet API, with the free early exit when the circuit is open
Closed calls go through as normal; open calls never leave the client.

The breaker itself: circuit_breaker.py

CircuitBreaker.call(fn, *args, **kwargs) wraps get_with_retry from the Python quickstart: closed lets every call through as normal; open raises CircuitOpenError immediately, before any HTTP request; half-open (after reset_timeout) lets exactly one trial call through and decides the next state from its result. A real clock is injected so the cool-down can be tested without actually waiting.

circuit_breaker.py - closed / open / half-open above get_with_retry

"""A client-side circuit breaker around the SellerMagnet API's own retry.get_with_retry.

Why this sits ABOVE get_with_retry, not instead of it: get_with_retry already retries a single
call's own 429/500/502/503 up to `attempts` times (see retry.py from the Python quickstart). A
circuit breaker reacts across calls instead: after several SEPARATE calls each exhaust their
retries on a billed 502/503 (the API's own upstream/per-marketplace circuit-breaker signal, documented under
"Status codes and billing" in the API rate limits post), the real dependency is down for a while, and every further attempt
pays another credit for the same answer. Opening the circuit stops those calls locally, for free,
until a cool-down passes.

What counts as a breaker failure, and what does not:
- 502 / 503: billed, and the API itself names 503 as its own circuit breaker signal. These ARE
  counted - a string of them means the dependency is unhealthy, not that one ASIN is odd.
- Timeout / ConnectionError: the API never answered at all. Counted the same way.
- 404: a real, billed answer ("this ASIN is not on this marketplace"). NOT counted - it says
  nothing about the dependency's health, and counting it would open the circuit on a batch of
  genuinely missing ASINs.
- 429 / 401 / 403 / free-gate 400: policy/capacity responses, never billed, and not instability.
  NOT counted - get_with_retry already waits out a 429 itself.
- 500: refunded and already retried once by get_with_retry. NOT counted on its own; a second,
  immediately-following 500 raises past get_with_retry anyway and IS counted (it means the retry
  itself failed, which looks the same as an unhealthy dependency from here).
"""
import time

import requests


class CircuitOpenError(Exception):
    """Raised BEFORE any HTTP call is made - never spends a credit."""

    def __init__(self, opened_at: float, reset_timeout: float, clock=time.monotonic):
        remaining = max(0.0, reset_timeout - (clock() - opened_at))
        super().__init__(f"circuit open, retry in {remaining:.1f}s")
        self.remaining = remaining


CLOSED, OPEN, HALF_OPEN = "closed", "open", "half_open"
# Breaker-relevant failures: the dependency itself looks unhealthy, not a per-item answer.
BREAKER_STATUSES = {502, 503}


class CircuitBreaker:
    """Wraps retry.get_with_retry. One instance per endpoint (or per host) you call repeatedly."""

    def __init__(self, failure_threshold: int = 3, reset_timeout: float = 30.0, clock=time.monotonic):
        self.failure_threshold = failure_threshold
        self.reset_timeout = reset_timeout
        self.clock = clock                          # injectable for tests; defaults to real time
        self.state = CLOSED
        self.failures = 0
        self.opened_at = 0.0
        self._half_open_trial_in_flight = False

    def _counts(self, exc: Exception) -> bool:
        status = getattr(exc, "status", None)
        if status in BREAKER_STATUSES:
            return True
        if status == 500:          # a 500 that survived get_with_retry's own one retry
            return True
        # get_with_retry raises requests' own exception classes, NOT the builtins of the same
        # name - requests.ConnectionError/requests.Timeout do not inherit from the builtin
        # ConnectionError/TimeoutError, so checking the builtins here would silently never match
        # a real network failure.
        if isinstance(exc, (requests.ConnectionError, requests.Timeout)):
            return True
        return False

    def call(self, fn, *args, **kwargs):
        """fn is get_with_retry (or any callable); args/kwargs are forwarded unchanged."""
        if self.state == OPEN:
            if self.clock() - self.opened_at < self.reset_timeout:
                raise CircuitOpenError(self.opened_at, self.reset_timeout, self.clock)
            self.state = HALF_OPEN
            self._half_open_trial_in_flight = False
        if self.state == HALF_OPEN and self._half_open_trial_in_flight:
            # a trial call is already outstanding; do not let a second caller race it
            raise CircuitOpenError(self.opened_at, self.reset_timeout, self.clock)

        trial = self.state == HALF_OPEN
        if trial:
            self._half_open_trial_in_flight = True
        try:
            result = fn(*args, **kwargs)
        except Exception as exc:                    # noqa: BLE001 - re-raised unchanged below
            if trial:
                self._half_open_trial_in_flight = False
                self.state = OPEN
                self.opened_at = self.clock()
                self.failures = self.failure_threshold
                raise
            if self._counts(exc):
                self.failures += 1
                if self.failures >= self.failure_threshold:
                    self.state = OPEN
                    self.opened_at = self.clock()
            raise
        else:
            if trial:
                self._half_open_trial_in_flight = False
            self.state = CLOSED
            self.failures = 0
            return result

Using it in a batch price watch

The script below watches a list of ASINs through the breaker. During a real outage it pays for the failures up to failure_threshold, then skips the rest of the batch for free instead of paying for the same answer on every remaining ASIN.

price_watch_breaker.py - a batch lookup that stops paying once the dependency is down

"""Batch Buy Box price watch with a circuit breaker above get_with_retry.

Without a breaker, a batch of 50 ASINs during a real upstream outage pays for 50 lots of
retries (up to 4 attempts each, see retry.py) before reporting 50 failures - every attempt billed
on a 502/503. The breaker trips after a few of those and skips the rest for free, instead of
paying to relearn the same outage 46 more times.

Helpers: sellermagnet.py (API, ApiError, new_session) and retry.py (get_with_retry) from
https://sellermagnet-api.com/blog/amazon-product-api-python
"""
from circuit_breaker import CircuitBreaker, CircuitOpenError
from retry import get_with_retry
from sellermagnet import ApiError, lookup, new_session

MARKETPLACE = "ATVPDKIKX0DER"
WATCH_LIST = ["B0CL61F39H", "B0SECONDOK", "B0BREAKS01", "B0BREAKS02", "B0BREAKS03", "B0NEVERRUN1"]


def watch(session, breaker: CircuitBreaker) -> dict:
    results = {"ok": 0, "not_found": 0, "skipped_open": 0, "failed": 0}
    for asin in WATCH_LIST:
        try:
            product = breaker.call(get_with_retry, session, "amazon-product-lookup",
                                    asin=asin, marketplaceId=MARKETPLACE)
            price = product["productInfo"]["buyBoxInfo"].get("price")
            print(f"{asin}  OK    price={price}  circuit={breaker.state}")
            results["ok"] += 1
        except CircuitOpenError as open_err:
            print(f"{asin}  SKIP  circuit open, retry in {open_err.remaining:.0f}s - no credit spent")
            results["skipped_open"] += 1
        except ApiError as err:
            if err.status == 404:
                print(f"{asin}  N/A   not on this marketplace (billed, not retried)")
                results["not_found"] += 1
            else:
                print(f"{asin}  FAIL  {err}  circuit={breaker.state}")
                results["failed"] += 1
    return results


if __name__ == "__main__":
    breaker = CircuitBreaker(failure_threshold=3, reset_timeout=30.0)
    summary = watch(new_session(), breaker)
    print(summary)

Output (illustrative data: ASINs starting B0BREAKS simulate a billed 502 outage)

B0CL61F39H  OK    price=19.99  circuit=closed
B0SECONDOK  OK    price=19.99  circuit=closed
B0BREAKS01  FAIL  502: upstream  circuit=closed
B0BREAKS02  FAIL  502: upstream  circuit=closed
B0BREAKS03  FAIL  502: upstream  circuit=open
B0NEVERRUN1  SKIP  circuit open, retry in 30s - no credit spent
{'ok': 2, 'not_found': 0, 'skipped_open': 1, 'failed': 3}
  • B0CL61F39H / B0SECONDOK succeed normally; the circuit stays closed and breaker.failures stays 0.
  • B0BREAKS01-03 are 3 separate billed 502s; the third one trips failure_threshold=3 and opens the circuit.
  • B0NEVERRUN1 never reaches the API at all: CircuitOpenError fires first, so this ASIN costs 0 credits in this run.
  • A real batch would retry the circuit later (after reset_timeout) rather than drop B0NEVERRUN1 forever - the half-open state exists exactly for that.

Why is the half-open trial not a cheap probe?

Because the breaker wraps one LOGICAL call, not one HTTP attempt. When reset_timeout passes and the next call is let through as a trial, that call is still get_with_retry(session, endpoint, ...) underneath - if the dependency is still down, it retries a 502/503 up to its own attempts limit (4 by default) before raising, and only then does the breaker see a failure and reopen. A failing trial can therefore cost up to 4 credits, not 1 - size failure_threshold and reset_timeout with that in mind rather than assuming the probe is free.

Terminal-theme table of the three circuit breaker states, what each one does on a call, and what moves it to the next state
Three states; only OPEN ever skips the API call for free.

What this does not replace

A circuit breaker is not a substitute for respecting Retry-After on a 429 (see the rate limits and error handling post) or for the 120-second client timeout the API itself expects - both of those already work correctly inside get_with_retry. It also does not get you a refund: every call billed before the circuit opened stays billed under the normal rules (see refunds on error), and every call it skips once open was never sent, so there is nothing to refund there either. What it buys is simpler: during a real outage, a 500-ASIN batch stops paying for attempt after attempt on ASINs it has already learned the answer for.

Frequently Asked Questions

What is a circuit breaker, and how is it different from retrying?

Retrying (get_with_retry) reacts INSIDE one call: a few attempts at one ASIN's lookup. A circuit breaker reacts ACROSS calls: once several separate lookups have each exhausted their own retries on a billed 502/503, it stops sending further ones for a cool-down period, so the rest of a batch is not billed to relearn the same outage one ASIN at a time.

Does a 404 (ASIN not on this marketplace) open the circuit?

No. A 404 is a real, billed answer about one ASIN, not a sign the API itself is unhealthy. Counting it would open the circuit on a batch that is simply full of genuinely missing ASINs, stopping lookups that would have worked fine.

Is the half-open trial call free if it fails?

No. The trial still goes through get_with_retry's own policy, which retries a 502/503 up to 4 times before giving up - so a failing trial can cost up to 4 credits, not 1, before the circuit reopens. Budget for that when sizing failure_threshold and reset_timeout.

What should failure_threshold and reset_timeout be?

There is no API-mandated number - both are local policy. A short reset_timeout (10 to 30s) fits a per-marketplace outage, which the API's own 503 Retry-After is usually 30 or 60s anyway; a low failure_threshold (2 to 3) limits how many ASINs get billed again before the breaker reacts.

Does opening the circuit get any credit refunded?

No, and it does not need to: every call that already happened before the circuit opened was billed normally (or refunded under the normal 500 rule), and every call skipped once the circuit is open never reaches the API at all, so no credit is spent on it in the first place.

Bottom line: wrap get_with_retry in a CircuitBreaker keyed per endpoint (or per marketplace, for a very large batch), tune failure_threshold and reset_timeout to your own batch size, and treat a failing half-open trial as a real, billed attempt, not a free one. The API documentation covers every endpoint's own status codes, and a free account includes 150 credits to try this against real traffic.

Ready to Extract Amazon Data at Scale?

Start building with SellerMagnet API today. Real-time product data, competitive pricing, and review analytics at your fingertips.

150 free API credits • No credit card required • Cancel anytime