Bot traffic in link analytics: what gets counted and what does not

Three different kinds of machine hit a short link, and conflating them is why people distrust link analytics.

Preview crawlers, which are not counted

Paste a URL into WhatsApp, Slack, Discord, Telegram, a scheduling tool or a document, and that service fetches the link to build a preview card. It happens the moment you paste, before anyone has seen anything.

These are recognised on the redirect path itself, against a maintained list of crawler agents — Facebook's, Twitter's, LinkedIn's, Slack's, Discord's, Telegram's, Apple's, Google's, Bing's and the rest — and the request is never recorded. Not as a click, not as a bot, not at all.

That is a deliberate choice and it costs us a little accuracy in the other direction: a crawler that changes its user agent gets counted as something. We prefer that failure mode to the alternative, where sharing a link in a group chat adds twelve clicks before a human has read the message.

Preview crawlers also get a different response. When a link has custom preview metadata set, a crawler is served a small HTML page carrying the title and image rather than a redirect, so the card renders correctly while the human still gets a clean 302.

Bots, which are counted separately

Everything else automated: scrapers, uptime monitors, SEO crawlers, security scanners, the long tail of scripts.

Two independent signals mark a request as a bot. The user-agent string is checked against a bot-detection library, and where the network edge attaches a bot score, a score below the configured threshold marks the request too. Either one is sufficient.

A request marked as a bot increments the link's bots counter and then stops. It does not add to clicks, it does not add to uniques, and it does not appear in the country, city, referrer, device, operating-system, browser, language, campaign or variant tables. Those tables are human traffic by construction, not by a filter applied at reading time.

The ones nobody can catch

Corporate email security is the hard case. A large employer's mail gateway opens every link in every inbound message to check where it goes. Some of those scanners identify themselves; many deliberately look exactly like a person on a recent browser, because a scanner that announces itself is a scanner that malware can evade.

The practical consequences for an email campaign:

Signal	Likely explanation
Clicks within seconds of delivery	Scanners, not readers
Many clicks, near-zero uniques growth after	One scanner fleet hitting repeatedly
Clicks with no referrer and a desktop browser	Could be either; unresolvable from one request
A steady curve over hours	Humans reading their inbox

The honest position is that no shortener can fully separate these, and any vendor claiming to has not looked hard. Read the shape of the curve rather than the first hour's total.

What to do about it

1. 1

   Compare uniques, not clicks

   Scanner traffic inflates clicks far more than it inflates unique visitors, because a fleet of scanners from one organisation often shares infrastructure and user agents.

2. 2

   Discard the first hour of an email send

   Not because it is all bots, but because you cannot tell which part is not. Compare hour two onwards against the same window in your previous send.

3. 3

   Watch the bot column as a ratio, not a total

   Every link has some. A link that normally sits at five per cent and jumps to sixty has had something happen to it that is worth knowing about.

4. 4

   Check what a preview shows before you post

   Expand the link yourself first. It tells you what a crawler will fetch and confirms the destination is what you think it is, without adding to the count.

Where this happens in the pipeline

Bot detection is not on the redirect. The redirect path captures a small set of raw facts and hands them straight to a queue; user-agent parsing and bot classification run in the background worker that turns those facts into stored events.

The consequence for you is that the separation is permanent and already applied. There is no toggle to re-include bots in your totals, and no way to reclassify past events, because the classification is baked into the stored event rather than computed when you open the page.

Everything above applies to every branded link on every plan. What changes with the plan is how far back you can read: 30 days on Free, a year on Pro, three years on Business and Team.