If you monitor your website’s performance with an Adobe dashboard or report, you may have noticed that the totals for a prior month sometimes change. The volume of visits you reported last month is slightly lower than what is being displayed in Adobe Analytics now. Usually the change is less than 1% - barely noticeable, right? When your most detail-oriented colleague is sure to ask questions, you’re looking to be prepared with an answer so that your stakeholders have no reason to doubt your report. But why is this happening? Is Adobe out to get you?
Rest assured, Adobe isn’t lying to you, but is just trying to give you the most accurate view and sometimes that means a bit of adjusting history.
Websites are constantly being crawled by bots; good ones that index your sites for search engines and bad ones that can cause issues for your site. Adobe Analytics, as well as third-party vendors, maintain bot filters to remove the non-human traffic from your data. If they add a new bot to their naughty list, it will retroactively pull those visits out of last month’s reports.
Whoever administers your Adobe Analytics account may also make changes that influence your reporting. An Admin can update the rules that define your Virtual Report Suite. Your report suite may have included visits from both the US and Canada, but this month the Admin broke the Canadian traffic off into its own report suite. Those Canadian visits that appeared in your data last month will suddenly be gone.
I recently came across a third situation where the reason for the change was much less obvious.
Below shows visit data for the same report suite with the same filter. On the left, the workspace is set to just May, while the right shows both May and June. Both are broken down by the month dimension, but the values for May do not quite match! What’s going on here?
The only difference between the two is the date range on the top right, but the culprit is actually hidden in the segment at the top. This site filter is meant to remove non-customer data from your visit total. To do that, it uses a VISITOR container to exclude all visits associated with the company domain.
While the table in the workspace is broken down by the month dimension, that VISITOR container is being evaluated against whatever data range on the top right. If you've never utilized Adobe Analytics to a large extent, I may have already lost you. Let's take a look at an example.
Example:
Say your coworker, Judd, worked from home all of May. He connected to the internet through his home WiFi and visited your site. In June, he returned to the office and connected his device through the office WiFi.
On the left, Judd’s visits are included in the total for May. None of his visits came through the work ISP in that date range, so his data was not excluded.
On the right, Judd’s visits are excluded from both the May and June rows. The site filter eliminates any traffic from the company ISP within the date range, so Judd’s May visits are removed.
Which one is right? What do you say when that one stakeholder calls you out?
If they are a Star Wars fan, you could try your best Obi-Wan Kenobi voice and say:
So what I told you was true, from a certain point of view.
Both numbers are accurate for the specific data range – though, it is probably not the most satisfying answer.
In this case, we are excluding internal traffic, so the largest date range would be the most accurate total of customer traffic. You could change the date range to a full year or more in order to eliminate as much internal traffic as possible.
The analysts at Campfire understand the nuances of web reporting, including the example above. We frame our reports in the way that is easiest for stakeholders to consume and have the technical expertise that understand the logic behind the numbers. Additionally, we are prepared to make recommendations to help you decide which of these “changing numbers” is the best one to use.
Send us an email and get in contact if you’d like to know more about how Campfire Analytics can help make your company more data-driven.