Mobile proxies with Crawlee in 2026

Mobile proxies with Crawlee is the modern Node stack for serious crawling work in 2026. Crawlee, formerly Apify SDK, ships with first-class proxy support, automatic retry handling, fingerprint generation, session pooling, and ready-made crawler classes for plain HTTP, Playwright, Puppeteer, and Cheerio. Combined with SingaporeMobileProxy it gives you Singapore-egress crawls on real SingTel, StarHub, and M1 carrier IPs without writing your own middleware. This guide walks through every pattern you need: ProxyConfiguration with rotation, session-bound proxies, fingerprint integration, retry tuning, and the production gotchas that come with running Crawlee at scale.

The Crawlee documentation is at crawlee.dev. For comparison with other crawlers see /blog/mobile-proxies-with-scrapy-2026 and /blog/mobile-proxy-rotation-nodejs-2026.

why Crawlee with mobile proxies

Crawlee is opinionated where Scrapy and raw fetch are not. It assumes you want retries, you want session persistence, you want fingerprint coherence, and you want metrics, and it gives you all four out of the box. For mobile proxy work, the ProxyConfiguration class handles rotation cleanly, the SessionPool ties cookies to specific proxies, and the fingerprint integrations match browser fingerprints to mobile IPs.

The trade-off is that Crawlee is opinionated. If you want a stripped-down HTTP loop, Crawlee is overkill. For real crawls with depth and structure, the opinions pay back fast.

Mobile proxies in Singapore route Crawlee traffic through real cellular IPs that share carrier-grade NAT with thousands of other phones. The result is traffic that looks like ordinary Singapore consumers to the target, which is exactly what platforms like Shopee, Lazada, TikTok, and Facebook expect from real users.

the simplest Crawlee with mobile proxy

import { CheerioCrawler, ProxyConfiguration } from "crawlee";

const proxyConfiguration = new ProxyConfiguration({
  proxyUrls: Array.from({ length: 20 }, (_, i) =>
    `http://${process.env.SMP_USER}:${process.env.SMP_PASS}@gw.singaporemobileproxy.com:${8001 + i}`
  ),
});

const crawler = new CheerioCrawler({
  proxyConfiguration,
  maxConcurrency: 16,
  maxRequestRetries: 5,
  requestHandlerTimeoutSecs: 30,
  async requestHandler({ request, $, log, proxyInfo }) {
    log.info(`crawling ${request.url} via ${proxyInfo.url.split("@")[1]}`);
    const title = $("title").text();
    log.info(`title: ${title}`);
  },
  failedRequestHandler({ request, log }) {
    log.error(`giving up on ${request.url}`);
  },
});

await crawler.run([
  "https://shopee.sg/",
  "https://lazada.sg/",
]);

Crawlee picks a proxy from the rotation pool for every request automatically. The requestHandler receives the proxyInfo so you can log which IP served which page, which is invaluable for debugging.

per-tier proxy configuration

For larger pools, ProxyConfiguration supports tiered rotation, where Crawlee falls back to higher tiers if requests on lower tiers fail too often.

const proxyConfiguration = new ProxyConfiguration({
  tieredProxyUrls: [
    [
      `http://${user}:${pass}@gw.singaporemobileproxy.com:8001`,
      `http://${user}:${pass}@gw.singaporemobileproxy.com:8002`,
    ],
    [
      `http://${user}:${pass}@gw.singaporemobileproxy.com:8003`,
      `http://${user}:${pass}@gw.singaporemobileproxy.com:8004`,
    ],
  ],
});

Tier 0 is tried first. If a request keeps failing, Crawlee escalates to tier 1, then tier 2. This pattern is great when you want to reserve premium ports for retries and use cheaper ports for the bulk of traffic.

session-bound proxies with SessionPool

Many platforms expect IP and cookie continuity within a session. Crawlee’s SessionPool keys a session to a specific proxy URL, so cookies and IP stay aligned.

import { CheerioCrawler, ProxyConfiguration, Session } from "crawlee";

const crawler = new CheerioCrawler({
  proxyConfiguration,
  useSessionPool: true,
  persistCookiesPerSession: true,
  sessionPoolOptions: {
    maxPoolSize: 50,
    sessionOptions: { maxUsageCount: 50 },
  },
  async requestHandler({ session, request, $, response }) {
    if (response.statusCode === 403) {
      session.markBad();
      throw new Error("blocked");
    }
    session.setCookiesFromResponse(response);
  },
});

session.markBad() removes the session from the pool, which retires the proxy-cookies pair. New requests pick up a fresh session, which means a fresh proxy and fresh cookies. This is the single most useful pattern in Crawlee for production scraping against detection-heavy targets.

sticky sessions through dedicated session tokens

For SingaporeMobileProxy specifically, you can also use the provider-side sticky session feature documented at /blog/mobile-proxy-sticky-sessions-2026. Bake the session token into the proxy username.

import crypto from "node:crypto";

function sessionProxyUrl(accountId) {
  const token = crypto.createHash("sha1").update(accountId).digest("hex").slice(0, 12);
  return `http://${process.env.SMP_USER}-session-${token}:${process.env.SMP_PASS}@gw.singaporemobileproxy.com:8000`;
}

const proxyConfiguration = new ProxyConfiguration({
  newUrlFunction: async (sessionId, options) => {
    const accountId = options?.request?.userData?.accountId ?? sessionId;
    return sessionProxyUrl(accountId);
  },
});

Crawlee’s newUrlFunction lets you compute the proxy URL per request based on session state or per-request user data. This pattern is powerful for multi-account work where each account needs its own sticky IP.

PlaywrightCrawler with mobile proxies

For JavaScript-heavy sites that resist HTTP-only scraping, swap CheerioCrawler for PlaywrightCrawler. The proxy configuration syntax is identical.

import { PlaywrightCrawler, ProxyConfiguration } from "crawlee";

const crawler = new PlaywrightCrawler({
  proxyConfiguration,
  maxConcurrency: 5,
  launchContext: {
    launchOptions: { headless: true },
  },
  async requestHandler({ page, request, log }) {
    await page.goto(request.url, { waitUntil: "networkidle" });
    const title = await page.title();
    log.info(`title via mobile proxy: ${title}`);
  },
  preNavigationHooks: [
    async ({ page }) => {
      await page.setViewportSize({ width: 412, height: 915 });
      await page.setExtraHTTPHeaders({
        "Accept-Language": "en-SG,en;q=0.9",
      });
    },
  ],
});

Combine with the fingerprint generator (next section) and you have headless Chrome traffic that egresses through a Singapore mobile IP with a coherent mobile fingerprint.

fingerprint coherence with FingerprintGenerator

Crawlee integrates with fingerprint-generator to produce consistent, realistic browser fingerprints. Pair it with the mobile proxy and your traffic looks like a real Singapore phone.

import { PlaywrightCrawler, ProxyConfiguration } from "crawlee";

const crawler = new PlaywrightCrawler({
  proxyConfiguration,
  browserPoolOptions: {
    useFingerprints: true,
    fingerprintOptions: {
      fingerprintGeneratorOptions: {
        browsers: [{ name: "chrome", minVersion: 120 }],
        devices: ["mobile"],
        operatingSystems: ["android"],
        locales: ["en-SG", "en-US"],
      },
    },
  },
  async requestHandler({ page, request }) {
    await page.goto(request.url);
  },
});

Each browser context gets a generated fingerprint that matches mobile Android Chrome with en-SG locale, paired with a Singapore mobile IP from the proxy pool. The combination passes the most common detection libraries.

retry tuning

Crawlee has sensible defaults but production workloads benefit from explicit tuning.

const crawler = new CheerioCrawler({
  proxyConfiguration,
  maxRequestRetries: 5,
  requestHandlerTimeoutSecs: 30,
  navigationTimeoutSecs: 30,
  retryOnBlocked: true,
  errorHandler({ request, error, log }, error2) {
    log.warning(`retry ${request.retryCount} on ${request.url}: ${error.message}`);
  },
});

retryOnBlocked: true activates Crawlee’s automatic detection of blocked responses (CAPTCHA pages, 403s, redirects to login). When triggered, it marks the session bad and retries with a fresh one, which is the right behavior on mobile proxies where rotation is cheap.

comparison: Crawlee vs other crawlers

Pick Crawlee when your team is Node-first and you want the framework’s opinions to carry you. Pick Scrapy for Python shops. Pick custom for tiny one-off jobs.

production checklist

Run with Apify or Crawlee storage configured, otherwise crawl state goes to memory and disappears on restart. Use RequestQueue for resumable crawls. Set maxRequestsPerCrawl to bound runs. Monitor stats through the built-in Statistics class. For deployment, Apify Cloud is the path of least resistance. Self-hosted on Kubernetes works well too: one Crawlee process per pod, RequestQueue backed by Redis or PostgreSQL.

Watch memory: PlaywrightCrawler with high concurrency burns RAM fast. Cap maxConcurrency based on memory headroom rather than CPU.

faq

does Crawlee handle proxy auth automatically? Yes. Pass full URLs with userinfo in the proxyUrls array and Crawlee handles auth.

can I rotate proxies per request in Crawlee? Yes through ProxyConfiguration with multiple proxyUrls or with newUrlFunction for dynamic selection.

how does Crawlee differ from Apify SDK? Crawlee is the open-source rebrand of Apify SDK with the cloud-specific bits separated out. Code is mostly compatible.

should I use SessionPool or sticky sessions? Use both. SessionPool ties cookies to a session. Sticky sessions on SingaporeMobileProxy keep the IP stable. Together they give you maximum continuity.

does Crawlee work with mobile proxies at scale? Yes. Crawlee is built for production crawling and the ProxyConfiguration class scales cleanly to hundreds of proxy URLs.

try it on real Singapore IPs

Spin up a SingaporeMobileProxy trial and drop the snippets into a fresh Crawlee project. Swap the gateway hostname for your endpoint. Our /api-docs cover sticky sessions, port-level rotation, and the metrics endpoints you will want for production monitoring.