Journal index

Entry 003 · 4 min

What Ranking Data Can and Cannot Prove

Search exports are useful evidence, but only when the duration, filters, and limits stay attached to the claim.

A ranking export can look definitive while still leaving important questions unanswered.

Take a query export where most of the rows sit on page one and a third of them sit in the top three. Read quickly, that looks like proof a content program worked.

It is not, on its own. A ranking distribution says nothing about total clicks or impressions. It says nothing about a growth rate unless the date range travelled with the file, and exports frequently arrive without one. It also silently excludes everything past the export row limit. Strip those caveats away and a real result quietly becomes a weak claim that a careful reader will take apart.

Keep the limits attached

When I turn project data into a case note, I keep four things together:

  1. the source
  2. the time window
  3. the active filters
  4. the question the data can answer

If one of those is missing, the language should get narrower.

There is a fifth limit that has nothing to do with statistics: whose number it is. Search performance belongs to the client whose property produced it, and most engagements are covered by an agreement that says so. So the case notes on this site describe the work I did and the systems I built, not the results those systems produced. The results exist, and I will talk about them where a client has agreed I can. Publishing someone else’s traffic data to win my own credibility is a trade I am not willing to make.

Credible case studies are not the ones with the largest number. They are the ones where another person can understand exactly what the number means, and where the number was theirs to share.