Shell · setup guide

curl proxy setup

Before you debug a scraper, debug the proxy. curl is already on your machine, it prints every step of the proxy handshake with -v, and it can count the bytes a request moved, which is the number we bill.
You will need

Residentialresi.proxymonkey.io:8000

ISP or datacenterIP:PORT

CredentialsUSER:PASS

Copy your own from the dashboard, which also lists the host and port for every order. Where it differs from this page, the dashboard is right.

Install

Ships with macOS, most Linux distributions and Windows 10 and later. Check it is there:

terminal
curl --version

Rotating residential

Pass the proxy with -x. Run the same line twice: each run opens a new connection through the residential gateway, so the two addresses that come back should differ.

terminal
curl -x http://USER:[email protected]:8000 https://httpbin.org/ip
curl -x http://USER:[email protected]:8000 https://httpbin.org/ip

A static datacenter or ISP IP

Datacenter goes through its own host. Splitting the credentials out with --proxy-user keeps them away from characters curl might misread in a URL. For an ISP order, use the host and port your dashboard lists against it; the flags are the same.

terminal
curl --proxy http://IP:PORT \
     --proxy-user USER:PASS \
     https://httpbin.org/ip

Keeping one identity

curl has no memory between runs, so every command is a fresh connection and, on residential, a fresh address. If you need several requests to share one residential IP, residential supports sticky sessions; the session setting for your account is in the dashboard. For a quick multi-step check, a static datacenter or ISP address is simpler.

Specific to curl

Things worth knowing

Count bytes the way the meter does

We bill request bytes plus response bytes, headers included. curl reports all four parts, so you can total a job yourself and hold it against your usage log. Expect our figure to be slightly higher: TLS overhead on the tunnel is real traffic and curl does not see it.

terminal
curl -s -o /dev/null -x http://USER:[email protected]:8000 \
     -w '%{size_request} %{size_upload} %{size_header} %{size_download}\n' \
     https://example.com/

Set it once for a whole shell

curl reads https_proxy and http_proxy (lowercase is the safe spelling). Handy when a script calls curl twenty times. NO_PROXY lists hosts that should skip it.

terminal
export https_proxy=http://USER:[email protected]:8000
curl https://httpbin.org/ip

Watch the handshake

With -v, look for the CONNECT line and the reply under it. 200 Connection established means the proxy let you through and anything that goes wrong afterwards is between you and the target.

When it breaks

Common errors and fixes

  • curl: (56) CONNECT tunnel failed, response 407

    Why
    The gateway rejected the credentials. Often it is a password with @, : or # in it, which breaks the URL.
    Fix
    Pass credentials with --proxy-user USER:PASS, or percent-encode the special characters. Copy them fresh from the dashboard.
  • curl: (28) Operation timed out

    Why
    No answer in time. Residential exits are home connections, and some are slow.
    Fix
    Set --connect-timeout 10 --max-time 30 and add --retry 2. A retry through residential leaves from a different address.
  • curl: (60) SSL certificate problem

    Why
    With an HTTP proxy, TLS runs end to end between curl and the target through the tunnel. A certificate error is about the target or your CA bundle.
    Fix
    Update your CA certificates. -k hides the problem for a quick test; do not leave it in a script.
  • curl: (5) Could not resolve proxy

    Why
    A typo in the proxy host, or a shell variable that expanded to nothing.
    Fix
    Echo the proxy URL before using it, and check the host against your dashboard.

A failed connection that moved no data is not billed. Retries you send are billed like any other request, and each shows as its own line in your usage log.

The community layer

curl still misbehaving?

Paste the error and the few lines that set up the proxy into Discord, with the password taken out. Someone there has seen it before.

Join the Discord

4,200+monkeys in the Discord

  • Help from humans

    Post your error, get an answer. Usually in minutes, usually from someone who has hit the same wall.

  • A status bot that tells on us

    Pool health, incidents and maintenance posted automatically. Including the bad days.

  • Deals and free traffic

    Bonus GB drops, early access to new pools, and the occasional giveaway for a good bug report.

Join the Discord4,200+ monkeys, free to lurk