Building a multi-threaded email verifier with regex and SMTP checks

A multi-threaded email verifier should produce confidence scores rather than pretend every mailbox can be proved. Regex screening detects malformed input, DNS establishes whether a domain accepts mail, and SMTP probing can test recipient acceptance. For Australian marketers, the useful outcomes are valid, invalid, risky, unknown and catch-all, each tied to a specific action.

Speed matters, but so do server etiquette, consent and auditability. Teams in Sydney, Melbourne and Brisbane often combine CRM, ecommerce and event data from platforms such as Klaviyo, HubSpot, Mailchimp and Shopify. A verifier should therefore slot into an existing suppression workflow rather than become another opaque bulk-mailing system.

Separate syntax, routing and mailbox signals

Each validation layer answers a different question. Combining them avoids both false confidence and needless SMTP traffic.

Check	Signal	Main limitation
Regex	Address matches basic mailbox structure	Says nothing about delivery
Domain and MX	Domain publishes mail exchangers	Does not prove a mailbox exists
SMTP RCPT	Recipient is accepted or rejected	Security policies can hide valid mailboxes
Catch-all classification	Domain accepts arbitrary recipients	May reflect policy rather than a real mailbox

A syntax pass makes an address a candidate, not a deliverable address. DNS failure usually justifies rejection, while SMTP rejection can be temporary, policy-driven or definitive. Catch-all and provider-specific blocks belong in a risky or unknown bucket when the evidence is ambiguous. Keep the original address and record the reason for every classification.

Design a bounded concurrent pipeline

Use a queue with a fixed pool of workers rather than launching one thread per row. In Python, ThreadPoolExecutor suits this network-bound work, but submissions must be bounded so a large import cannot exhaust memory or flood one provider. Each task should have connection, command and total deadlines.

Apply a global concurrency limit plus a stricter per-domain limit. A token bucket for each mail domain prevents a shared Gmail or Outlook tenant from receiving bursts, while exponential backoff handles temporary failures. Treat 4xx SMTP responses as retryable and 5xx responses as permanent only after accounting for unusual server behaviour.

Deduplicate by a normalised key, cache DNS results according to TTL, and give each worker its own SMTP session. Store timestamps, MX hostnames, response codes and classifier versions, but redact local parts in routine logs. These controls make failures reproducible without turning the logs into another copy of the mailing list.

Build the regex and DNS stages

Normalise whitespace and convert internationalised domains with IDNA while preserving the user’s original local part. Do not silently remove plus tags, dots or other characters because valid providers can interpret them differently. The regex should reject obvious mistakes, enforce reasonable length limits and avoid assumptions that exclude less common yet legal addresses.

Australian domains need careful handling rather than a country-specific whitelist. Support .au, .com.au, .net.au and .org.au, then perform live DNS checks. Resolve MX records first, account for the null MX convention, and use A/AAAA fallback only where the relevant mail standard permits it. Cache positive and negative answers for their proper TTL.

Keep disposable-domain and role-account checks separate from mailbox validation. They can trigger review, but they do not prove non-delivery. If DNS or SMTP returns an inconclusive result, retain it as unknown and schedule a later retry instead of converting uncertainty into a false invalid result.

Conduct careful SMTP recipient checks

Resolve the chosen MX host, connect on port 25 with a short timeout, greet with an honest fully qualified hostname, issue a null reverse-path envelope sender and test one RCPT address. Do not send message content. Upgrade with STARTTLS when the server advertises it and your security policy permits it, then close the session promptly after the result.

Interpret responses precisely: 250 generally confirms acceptance; 4xx signals a temporary condition such as greylisting; 5xx normally indicates rejection; 252 means the server cannot verify the mailbox. Some servers reject probes or accept every recipient, so a positive or negative response must be combined with DNS and policy evidence.

Outbound port 25 is commonly blocked in Australian offices and cloud networks, while large mailbox providers rate-limit unknown senders. Use a dedicated host with forward-confirmed reverse DNS, cap checks per MX server and back off aggressively. If direct verification is unavailable, record unknown or use a reputable verification service rather than declaring addresses invalid.

Apply privacy, compliance and local operating controls

The Spam Act 2003 generally requires consent before sending commercial electronic messages, along with sender identification and a working unsubscribe mechanism. Address harvesting is also prohibited. Keep consent provenance with each imported record and immediately suppress unsubscribes and hard bounces. ACMA’s Do Not Call Register should not be confused with an email suppression list; it primarily covers telephone and SMS marketing.

Assess whether the Privacy Act 1988 and Australian Privacy Principles apply, as well as relevant state privacy legislation. Email addresses and campaign metadata can be personal information, so restrict access, encrypt stored lists, set retention periods and document any disclosure. A Notifiable Data Breaches response plan should cover accidental exposure of a verification file.

For Australian workflows that also validate postal data, keep state and postcode mappings for NSW, VIC, QLD, WA, SA, TAS, ACT and NT separate from email checks. Batch jobs should use UTC internally and display Australian local time, especially when daylight-saving rules differ between states.

Recommended safeguards for Australian deployments

• Import only addresses with documented consent or a clearly applicable legal exception.
• Use layered verification and preserve unknown results for scheduled rechecks.
• Set global, provider and per-domain concurrency caps with backoff.
• Store minimum necessary data, encrypt it and delete verification artefacts on schedule.
• Keep suppression, bounce and unsubscribe handling inside one controlled workflow.

This design gives Australian teams a defensible, observable verification pipeline with fast syntax screening, accurate routing context, cautious recipient testing and enforceable data controls.