Build your own proxy checker in Python

A proxy checker answers the questions a pricing page cannot: is this proxy alive, how fast is it, what IP does the site see, whose network is that IP on, and does the proxy leak anything about you. Online proxy testers answer some of that, but they test from their machine, not yours, and you are handing them your credentials.

So build one. This is just under 120 lines of Python that check a list of proxies concurrently and write a CSV. It works on any provider’s proxies, ours included. Run it on us first if you like; the honesty page says what we claim, and this is how you check it.

What a proxy checker should test

When you check a proxy, test five things. Anything less misses the problems that actually cost you money.

• Alive: does a request through it complete at all.
• Latency: the median of several calls, not one. One call tells you about one connection.
• Exit IP and network: the address the site sees, and the ASN it belongs to. The ASN is what decides whether a site treats you as a home user or a server.
• Header leaks: whether the proxy adds headers that name it, or you.
• Rotation: on a rotating gateway, how many different IPs come back across a few calls.

Set up: install aiohttp and write a proxy list

pip install aiohttp aiohttp-socks

aiohttp speaks HTTP proxies itself. It does not speak SOCKS, so aiohttp-socks adds a connector for those. Put one proxy per line in proxies.txt, credentials in the URL:

http://USER:[email protected]:8000
http://USER:PASS@IP:PORT
socks5h://USER:PASS@IP:PORT

The h in socks5h means the proxy resolves the hostname, not your machine, so your DNS lookups do not leave from your own address. The checker hands SOCKS lines to aiohttp-socks, which resolves remotely by default for SOCKS5. If the SOCKS side is new to you, the SOCKS5 glossary entry covers it.

The proxy checker script

Save this as check.py:

import asyncio
import csv
import os
import re
import statistics
import sys
import time

import aiohttp
from aiohttp_socks import ProxyConnector

CALLS = 5
CONCURRENCY = 10
TIMEOUT = aiohttp.ClientTimeout(total=20, sock_connect=10)
LEAKY = {"via", "x-forwarded-for", "forwarded", "x-real-ip", "client-ip", "proxy-connection"}
HOSTING = ("amazon", "microsoft", "digitalocean", "ovh", "hetzner", "linode", "akamai",
           "vultr", "choopa", "leaseweb", "m247", "contabo", "hosting",
           "datacenter", "data center", "cloud", "server")
TOKEN = os.environ.get("IPINFO_TOKEN")
networks = {}


def open_session(proxy):
    if proxy and proxy.startswith("socks5"):
        url = "socks5://" + proxy.split("://", 1)[1]
        return aiohttp.ClientSession(connector=ProxyConnector.from_url(url), timeout=TIMEOUT), None
    return aiohttp.ClientSession(timeout=TIMEOUT), proxy


async def get_json(url, proxy=None):
    session, http_proxy = open_session(proxy)
    async with session:
        started = time.perf_counter()
        async with session.get(url, proxy=http_proxy) as resp:
            resp.raise_for_status()
            data = await resp.json(content_type=None)
        return data, time.perf_counter() - started


async def network_of(ip):
    if ip not in networks:
        if TOKEN:
            data, _ = await get_json(f"https://api.ipinfo.io/lite/{ip}?token={TOKEN}")
            networks[ip] = f"{data.get('asn', '')} {data.get('as_name', '')}".strip()
        else:
            data, _ = await get_json(f"https://ipinfo.io/{ip}/json")
            networks[ip] = data.get("org", "")
    return networks[ip] or "unknown"


def masked(proxy):
    return re.sub(r"//([^:@/]+):[^@]+@", r"//\1:***@", proxy)


async def check(proxy, my_ip):
    row = {"proxy": masked(proxy), "alive": False, "median_ms": "", "exit_ip": "",
           "distinct_ips": "", "network": "", "hosting": "", "leaks": ""}
    ips, times = [], []
    for _ in range(CALLS):
        try:
            data, took = await get_json("https://httpbin.org/ip", proxy)
        except Exception:
            continue
        ips.append(data["origin"].split(",")[-1].strip())
        times.append(took)
    if not ips:
        return row

    row.update(alive=True, exit_ip=ips[0], distinct_ips=len(set(ips)),
               median_ms=round(statistics.median(times) * 1000))
    try:
        row["network"] = await network_of(ips[0])
    except Exception:
        row["network"] = "lookup failed"
    row["hosting"] = "yes" if any(w in row["network"].lower() for w in HOSTING) else ""

    try:
        data, _ = await get_json("http://httpbin.org/headers?show_env=1", proxy)
        seen = data["headers"]
        hops = [h for h in seen.get("X-Forwarded-For", "").split(",") if h.strip()]
        leaks = sorted(k for k in seen if k.lower() in LEAKY - {"x-forwarded-for"})
        if len(hops) > 1:
            leaks.append("X-Forwarded-For")
        if my_ip in str(seen):
            leaks.append("your IP")
        row["leaks"] = " ".join(leaks)
    except Exception:
        row["leaks"] = "check failed"
    return row


async def main(path):
    proxies = [l.strip() for l in open(path) if l.strip() and not l.startswith("#")]
    if not proxies:
        sys.exit(f"no proxies found in {path}")
    me, _ = await get_json("https://httpbin.org/ip")
    my_ip = me["origin"].split(",")[-1].strip()
    gate = asyncio.Semaphore(CONCURRENCY)

    async def guarded(proxy):
        async with gate:
            return await check(proxy, my_ip)

    rows = await asyncio.gather(*(guarded(p) for p in proxies))

    with open("results.csv", "w", newline="") as out:
        writer = csv.DictWriter(out, fieldnames=list(rows[0]))
        writer.writeheader()
        writer.writerows(rows)

    for r in rows:
        state = "up" if r["alive"] else "DOWN"
        print(f"{state:<5}{r['median_ms']:>6} ms  {r['exit_ip']:<16}{r['distinct_ips']:>3} ips  "
              f"{r['network'][:30]:<31}{r['hosting']:<4}{r['leaks'] or '-':<14}{r['proxy']}")


if __name__ == "__main__":
    asyncio.run(main(sys.argv[1]))

python check.py proxies.txt

Every call opens a fresh session, so every call is a new connection. That is deliberate: latency then includes the connection set-up a real scraper pays, and on a rotating gateway each call can come out of a different IP. httpbin’s /ip reports the whole forwarding chain, comma separated, and the script keeps the last entry: that is the address that actually connected to httpbin. Anything before it came from a header the proxy, or anyone upstream, chose to send. The IP lookups go direct from your machine, not through the proxy, so they cost no proxy traffic. The aiohttp setup page covers the client’s proxy options in more depth.

Checking the network behind an IP: ipinfo limits

The script asks ipinfo.io which network an IP belongs to. Without a token, ipinfo.io/<ip>/json still answers and still includes the org field (the ASN and its name, like AS15169 Google LLC), but the response now carries a readme link to their “missing auth” page, which describes this as the legacy free API with a limit of 50,000 requests a month. That was the case when we wrote this; check that page before you rely on it.

For more than a casual run, sign up for their free Lite plan and set IPINFO_TOKEN. The script then uses api.ipinfo.io/lite/<ip>, which returns asn and as_name. Either way the script caches each IP’s answer, so a rotating gateway that returns the same address twice costs one lookup. If you see HTTP 429 from the lookup, you are over their limit: slow down or add the token.

Testing header leaks: why the check uses plain HTTP

The leak test deliberately calls http://httpbin.org/headers?show_env=1, not https://. Over HTTPS the proxy only sees an encrypted tunnel after the CONNECT, so it cannot add a header even if it wanted to. Over plain HTTP it reads and forwards the request itself, and that is where Via, X-Forwarded-For and Forwarded get added. The script also searches the headers for your real IP, which it fetched directly at the start.

Two httpbin details matter here. Without ?show_env=1, httpbin strips Via, X-Forwarded-For and X-Real-Ip before it answers, so a leaky proxy looks clean. And httpbin’s own load balancer always adds one X-Forwarded-For entry for whoever connected to it, which is the proxy. So the script only flags that header when it holds more than one address: the extra one came from the proxy.

A proxy that passes your IP along in X-Forwarded-For is what the old proxy lists called “transparent”. It hides nothing from the site. One that adds Via but not your address hides you but announces itself. A clean result is an empty leaks column.

How to read proxy test results

The network column matters most. Sites that care about automated traffic look up the ASN of every visitor. An address on Amazon, OVH or Hetzner is a server by definition; one on a consumer broadband provider looks like a household. That lookup is most of the difference between residential and datacenter proxies.

Which is why a “residential” IP with a hosting ASN is the red flag to look for. It usually means one of two things: the seller is passing datacenter addresses off as residential, or their pool mixes the two. Either way you are paying the residential price for an address that gets the datacenter treatment. The keyword list in the script is crude on purpose. It leaves out names like Google that run both cloud servers and consumer broadband, so those need a human look. When it flags something, paste the ASN into a search engine and read what the network is.

The reverse is fine. An ISP proxy is a static address hosted in a data centre but announced under a consumer ISP’s ASN, so it should show an ISP’s name. A datacenter proxy should show a hosting network, and showing one is honest.

Checking a rotating proxy: distinct IPs

For our residential gateway, a rotation check should show as many distinct IPs as calls, or close to it. Pools repeat an address now and then; five calls returning four addresses is normal. One address five times means rotation is not happening, most often because the account is on a sticky session, which is a setting in the dashboard. The rotating vs sticky guide explains when you want which.

Raise CALLS to 20 or 50 for a better read on a rotating pool. Each call through a residential proxy is metered traffic, but httpbin’s /ip reply is tiny, so a full run is a few hundred kilobytes at most. Keep CONCURRENCY modest too: the checker is hitting httpbin.org, a free public service, and it deserves the same consideration as any other site.

Where a proxy tester stops

A checker tells you what the proxy is. It does not tell you whether a particular site will accept it. For that, run your real job on a few hundred URLs and watch the status codes. If you see a wall of 403s or 407s, the error field guide sorts out which side of the proxy the problem is on.

It also cannot check how a provider bills you. For that, count your own bytes and diff them against the usage log; the bill audit guide walks through it. A checker plus a byte count is most of what you need to hold any provider to its claims, us included.