Bot Traffic Filtering: Clean Up Your Analytics Data

Your analytics data is only as trustworthy as the traffic behind it. Most conversations about data quality jump straight to attribution models or tracking gaps, but there’s a quieter problem that corrupts dashboards long before the first campaign launches: bot and spam traffic mixed into your real visitor numbers.

I’ve opened dashboards for clients who were genuinely proud of their 40% month-over-month traffic growth — and found, on inspection, that about a third of it was crawlers, scrapers, and ghost referrers that had never touched the actual site. The conversion rate wasn’t low because the landing page was bad. The conversion rate was low because the denominator was full of noise.

Bot traffic filtering is the practice of identifying and excluding non-human traffic from your analytics data so that the metrics you act on reflect what real users are doing. It’s not a one-time setup. It’s ongoing maintenance, and most teams don’t do it systematically until something looks obviously wrong.

Why Bot Traffic Damages More Than the Numbers

The obvious damage is inflated page views and sessions. Less obvious: bot traffic warps every derived metric downstream. If your total sessions are 30% non-human, your bounce rate is distorted, your average session duration is distorted, your pages-per-session is distorted, and any conversion rate calculated off that denominator is systematically too low.

That last point is the one that costs teams the most. A 1.2% conversion rate sounds like a landing page problem. It sounds like a CRO opportunity. Teams build hypotheses around it, run tests on it, and ship changes based on it. But if 20% of the sessions in that calculation are crawlers that never had any intent to convert, your “true” conversion rate from real humans might be 1.5% — still not amazing, but the direction of the problem is different.

Attribution is another casualty. When you’re trying to understand which channels drive the most value — as covered in the guide on comparing attribution models — non-human traffic mixes unevenly across channels. Some bots arrive via direct, some via referral, very few via paid search (paid clicks cost money, which bots generally don’t have). So your organic and direct channels look disproportionately inflated versus paid, which skews ROI comparisons.

None of this is dramatic or sudden. It’s slow, structural distortion that compounds over time. That’s what makes it hard to notice.

The Four Main Categories of Junk Traffic

Not all unwanted traffic is the same, and the fix for each is different. Grouping them helps you prioritize.

Known crawlers and search bots

Googlebot, Bingbot, Semrushbot, Ahrefsbot, and hundreds of others. These crawl your pages to index them, audit them, or collect competitive data. They’re doing legitimate work — just not work you want in your analytics.

Most analytics tools try to filter these by checking the user agent string against known bot lists. GA4 filters out “known bots and spiders” by default, and this setting can’t be turned off. For server-side analytics tools or tools that parse server logs, you handle this yourself, usually by checking against the IAB/ABC International Spiders & Bots List.

The catch: new bots appear constantly, and known-bot lists are never complete. Uncommon crawlers, newly registered scrapers, and bots that deliberately spoof user agents (pretending to be Chrome on Windows) slip through.

Ghost referrals

Ghost referrals are traffic that appears in your referral report without ever visiting your site. They work by sending fake hits directly to the analytics tracking endpoint — in the Universal Analytics era, that meant posting directly to the Google Analytics collect endpoint with your measurement ID. No page load, no JavaScript execution, just a fake hit.

You probably still see these if you have any historical UA data. Common culprits were domains like semalt.com, buttons-for-website.com, and a long tail of similar spam. The goal was to appear in referral reports hoping site owners would visit the domain.

In GA4, this vector is much harder to exploit because hits must include a valid session and client ID from the browser SDK. Ghost referrals haven’t disappeared entirely, but they’re less common than in the UA era. If you’re seeing a referral domain with a 100% bounce rate and zero conversions across hundreds of sessions, ghost referral is a reasonable hypothesis.

Internal traffic

Your own team visiting the site. QA testers running through a new flow. A developer refreshing the checkout page to test a deploy. This is the category most teams acknowledge in theory and underestimate in practice.

For small teams this might be 2-3% of sessions. For teams with large QA or customer support departments who spend significant time on the product, it can be much higher. Internal traffic is especially damaging to conversion rate analysis because team members don’t convert the way real customers do — they either convert everything (because they’re testing) or nothing (because they’re browsing as staff).

The standard fix is an IP exclusion filter. Every major analytics platform supports it. The wrinkle is that most modern offices use dynamic IP addresses, and remote workers add more IP ranges constantly. IP filters need maintenance, not just a one-time setup.

Monitoring and performance tools

Uptime monitors like Pingdom, UptimeRobot, and StatusCake ping your pages on a regular schedule — sometimes every minute — to check availability. Some of them execute JavaScript, which means they can fire analytics events. If you use a monitoring service and your analytics is JavaScript-based, check whether those pings are appearing as sessions.

You can usually identify them: they come from a specific IP range (monitoring vendors publish their IP ranges), they hit only the URLs you configured for monitoring, they have no referrer, and their visit duration is milliseconds or zero.

What GA4’s Built-in Filtering Actually Does (and Doesn’t)

Google Analytics 4 filters known bots by default — this is automatic and can’t be disabled. According to Google’s own documentation, this uses the IAB/ABC bot list plus Google’s own signals to exclude hits from known automated agents before they reach your reports.

That sounds reassuring. The reality is more limited.

The IAB list is comprehensive but not exhaustive. A bot that hasn’t made it onto the list yet (a new scraper, a less-common monitoring tool) won’t be caught. Bots that deliberately spoof user agents won’t be caught. And the filter applies to GA4 events only — if you’re running a second analytics tool alongside GA4 (a privacy-friendly alternative, a data warehouse feed), that tool has its own filtering logic, or none.

GA4 also has no automatic filter for internal traffic. You have to configure it manually under Admin → Data Streams → Configure tag settings → Define internal traffic, then create a filter under Admin → Data Filters. Until you do that, every developer, QA tester, and customer support rep using your site is in your reports.

The internal traffic filter in GA4 works by matching against a list of IP addresses you define. Hits matching those IPs can be put in “testing” mode (they appear in DebugView but not regular reports) or excluded entirely. Choose “Exclude” for production data and move on — but remember that this filter is report-level, not collection-level. The raw data still lands in BigQuery if you’ve connected it, which is worth knowing if you run any direct queries.

Referral Exclusions: The Setting Nobody Checks

Referral exclusions are different from spam filtering. They’re for traffic that is real users but is being miscredited as referral traffic when it shouldn’t be.

The classic example: a customer clicks through to a third-party payment processor (Stripe, PayPal, a bank’s 3DS page) and then returns to your confirmation page. GA4 sees the payment processor domain as the referrer on that return visit and credits the conversion to it. The customer came from your Google Ads campaign originally, but the last-touch credit goes to “stripe.com.”

If you’re seeing payment processors, your own CDN domain, or internal subdomains showing up as top referral sources with suspiciously high conversion rates, you almost certainly have referral exclusions missing. In GA4, these are called “Unwanted Referrals” and live under Admin → Data Streams → Configure tag settings.

Referral exclusions don’t remove the traffic — they just prevent a new session from starting when a user returns from an excluded domain. The original session and attribution are preserved. This is a good example of data hygiene that directly improves your UTM attribution accuracy without touching the UTM parameters themselves.

How Privacy-First Analytics Tools Handle Bots

If you’ve moved to a privacy-friendly analytics stack — Plausible, Fathom, Umami, or similar — the bot filtering question gets a different answer.

Plausible, for example, applies its own bot filtering via user agent matching and excludes common bots before logging a pageview. It also won’t track visits from the same IP that bounced within a few seconds, which helps with monitoring pings. Fathom maintains its own bot list and applies it server-side, so bots that don’t execute JavaScript still get caught (unlike GA4, which relies on the JS snippet running).

The trade-off — as covered in the piece on privacy-friendly analytics trade-offs — is that you typically have less granular control. GA4 lets you define custom IP exclusions, set internal traffic filters, and use DebugView to verify what’s getting filtered. Most privacy-first tools offer less configurability in exchange for simpler setup and no consent overhead.

For small sites where the bot volume is low and the audience is clear, privacy-first tools usually have “good enough” bot filtering with no configuration required. For enterprise setups or sites with aggressive scraping issues, the configurability of GA4 (or a full server-side analytics stack) gives you more tools.

Server-Side and Log-Level Filtering

Everything discussed so far is analytics-layer filtering — you’re cleaning up what reaches your reporting tool after the hit has already been sent. Server-side filtering works one step upstream: you filter at the web server or CDN before the analytics payload even leaves the browser.

The most common approach is blocking known bad user agents at the nginx or CDN level. A deny list of scrapers and known malicious bots can be maintained in your nginx config or in Cloudflare’s firewall rules. This prevents the bot from loading your page at all — which saves server resources in addition to keeping analytics clean.

The limits: this doesn’t help with ghost referrals (which never load your page anyway), it doesn’t catch bots that spoof user agents, and maintaining a deny list is an ongoing task rather than a one-time setup.

Log-level filtering is the most reliable source of truth. Your web server logs every request, including requests from bots that don’t execute JavaScript and never fire any analytics events. If you parse server logs (with a tool like GoAccess, or by shipping logs to a data warehouse), you’ll see a picture of total crawl traffic that no analytics tool shows you. That picture is usually illuminating: it’s common for legitimate SEO crawlers to account for 15–25% of total server requests on a content-heavy site.

You don’t necessarily need to do anything about crawl traffic at the server-log level — crawlers are supposed to crawl — but seeing the full picture helps calibrate expectations. The dashboard pyramid has an analogy here: your server logs are the diagnostic layer, your analytics tool is the operational layer. They answer different questions.

How to Spot Suspicious Traffic in Practice

You don’t need a dedicated bot-detection platform to identify most problems. A few diagnostic checks with standard analytics tools surface the majority of issues.

The engagement time signal is particularly reliable in GA4. Real users almost always generate some engagement — scrolling, clicking, hovering — even on a quick page visit. Sessions with exactly zero seconds of engagement time and only a single page view are disproportionately non-human, especially in bulk.

Common Mistakes When Filtering Bot Traffic

Applying filters retroactively. In GA4, data filters apply from the moment you create them. Historical data before the filter was active is not changed. If you’ve been running for a year without an internal traffic filter, that year of data is baked in. The practical implication: when comparing year-over-year, use a consistent date range or acknowledge the filter difference. You can’t go back and clean it.

Filtering at the wrong layer. Blocking bots at the server level doesn’t clean up data in your analytics tool if those bots had already been tracked before the block. And cleaning up your GA4 reports doesn’t fix your CRM if you’re passing GA4 session data downstream. Know where your data flows and filter at every relevant point.

Over-filtering with broad IP exclusions. I’ve seen teams add entire /16 IP ranges to internal traffic filters to catch dynamic office IPs — and then exclude chunks of customer traffic that happened to share a carrier’s IP block. Be specific. Target the ranges you actually control.

Treating filter setup as a one-time task. Office IPs change. Monitoring services change their IP ranges. New bots appear. Your internal traffic filter needs a review every time something in your infrastructure changes, and a general review at least once a year.

Confusing low conversion rate with bot traffic. Bot traffic is one possible explanation for a low conversion rate. It’s also often not the explanation. Before blaming bots, verify the scale. If your total sessions are 10,000/month and your analytics shows 100 clearly-bot sessions, you have a 1% bot problem, not a conversion rate problem. Put the work into finding actual funnel leaks instead.

FAQ

Does GA4 automatically filter all bots?

GA4 filters known bots from the IAB/ABC list by default and this cannot be turned off. It doesn’t filter unknown or spoofed bots, and it doesn’t filter your own team’s internal traffic — that requires manual configuration. The automatic filter applies only to GA4 events, not to raw server logs or other analytics tools.

How do I know if I have a bot traffic problem?

Look for referral sources with 100% new users and zero conversions, spikes in sessions with zero engagement time, and suspiciously large traffic from specific cities or carriers with no corresponding revenue. Comparing your analytics sessions to server log traffic can also reveal the gap between what’s crawling your site versus what real users are doing.

Should I block bots at the server level or analytics level?

Both serve different purposes. Server-level blocking (nginx, Cloudflare) reduces load and stops scrapers from accessing your content. Analytics-level filtering keeps your reports clean. If you only do one, analytics filtering is lower risk — server-level blocks have more potential for false positives if misconfigured.

Do privacy-first analytics tools handle bot filtering differently?

Yes. Tools like Plausible and Fathom apply their own bot lists server-side, which means they can catch some bots that don’t execute JavaScript (and therefore wouldn’t fire a GA4 event anyway). The trade-off is less configurability — you typically can’t define custom IP exclusions or see filtering debug data the way you can in GA4.

Can I clean historical data after setting up a bot filter?

In GA4, no — data filters only apply going forward. You can segment historical reports to approximate the filtered view (for example, creating an exploration that excludes sessions with zero engagement time and known bot referrers), but the raw underlying data isn’t changed. This is a good argument for getting filters set up early rather than retrofitting them later.

What are ghost referrals and are they still a problem?

Ghost referrals are fake hits sent directly to an analytics endpoint, making it appear a domain referred traffic to you when no visit actually occurred. They were widespread in the Universal Analytics era. In GA4, the exploit is harder because hits require a valid browser-generated client ID, so true ghost referrals are less common. High-bounce referral sources with round session numbers are still worth investigating, but they’re usually low-quality real traffic (content farms, coupon aggregators) rather than true ghost spam.

Is internal traffic a significant problem for small teams?

It depends on your conversion rate analysis. If your total traffic is 1,000 sessions/month and your team generates 50 of them via QA and monitoring, that’s 5% contamination — significant when you’re trying to measure a 2% conversion rate. For high-traffic sites the proportion matters less. The setup cost is low, so there’s no good reason to skip it regardless of team size.

Sources and further reading

• GA4: Filter out bots and spiders — Google Analytics Help
• GA4: Define internal traffic — Google Analytics Help
• IAB/ABC International Spiders & Bots List
• How Plausible handles bot and spam traffic — Plausible blog
• Bot filtering in Fathom Analytics — Fathom blog