Are you measuring your social media buttons?

I’d like you to go to a Stack Overflow (or any Stack Exchange) question, and see if you spot a nominally major change.  Such as this one, with a nice answer from Raymond Chen (I think it’s been more than month since I recommended his blog, seriously go read it).

Spot it?

Of course not, what we did is remove our social media sharing buttons.

Left is old, right is new.  A number of other instances were also removed.

The Process

I’m a big advocate of arguing from data and while I freely admit there are many things that are hard to quantify, your gains from social media buttons aren’t one of them.

When discussing sharing buttons, be careful about establishing scope.  Keeping or removing social media buttons isn’t about privacy, personal feelings on Twitter/Facebook/G+, or “what every other site does” (an argument I am particularly annoyed by); it’s about referred traffic and people.  There’s a strong temptation to wander into debate, feeling, and anecdotal territory; stick to the data.

You also need to gather lots of data, at Stack Exchange we prefer to slap a pointless query parameter onto urls we’re tracking.  It’s low tech, transparent, and doesn’t have any of the user facing penalties redirects have.  In particular we slapped ?sfb=1, ?stw=1, and ?sgp=1 onto links shared via the appropriate sharing buttons.  Determining click-throughs is just a matter of querying traffic logs with this approach.  We’ve been gathering data on social media buttons in particular for about four months.

Note that we’re implicitly saying that links that are shared that nobody clicks don’t count.  I don’t think this is contentious, spamming friends and followers (and circlers(?), whatever the G+ equivalent is) is a bit on the evil side. Somebody has to vindicate a share by clicking on it, otherwise we call it a waste.

This sort of nonsense isn’t worth it.

With all this preparation we can actually ask some interesting questions; for Stack Exchange we’re interested in how much traffic is coming from social media (this is an easy one), and how many sharers do so through a button.

Traffic gave us no surprises, we get almost nothing from social media.  Part of this is probably domain specific, my friends really don’t care about my knowledge of the Windows Registry or the Diablo III Auction House.  The sheer quantity of traffic coming from Google also drowns out any other source, it’s hard to get excited about tweaking social media buttons to bring in a few thousand extra views when tiny SEO changes can sling around hundreds of thousands.  To put the difference in scale in perspective, Google Search sends 3 orders of magnitude more traffic our way then the next highest referrer which itself sends more than twice what Twitter does (and Twitter is the highest “social” referrer).

Now that I’ve established a (rather underwhelming) upper-bound for how much our share buttons were getting us, I need to look at what portion can be ascribed to the share buttons versus people just copy/pasting.  This is where the slugs come in, presumably no-one is going to add a random query parameter to urls they’re copy/pasting after all.

You basically want to run a query like this:

FROM TrafficLogs
-- everybody tries to set Referer so you know you got some juice from them
(RefererHost = '' OR RefererHost = '' OR RefererHost = '')
-- Everybody scrapes links that are shared, don't count those they aren't people
UserAgent NOT LIKE '%bot%'
-- Just the share buttons, ma'am
(Query LIKE '%stw=1%' OR Query LIKE '%sfb=1%' OR Query LIKE '%sgp=1%')

Adapt to your own data store, constrain by appropriate dates, etc. etc.  But you get the idea.  Exclude the final OR clause and the same query gives you all social media referred traffic to compare against.

What we found is that about 90% of everyone who does share, does so by copy/pasting.  Less than 10% of users make use of the share buttons even if they’ve already set out to share.  Such a low percentage of an already pretty inconsequential number doesn’t bode well for these buttons.

One final bit of investigation that’s a bit particular to Stack Exchange is figuring out the odds of a user sharing their newly created post.  We did this because while we always show the question sharing links to everyone, the answer sharing links are only shown to their owner and only for a short time after creation.  Same data, a little more complicated queries, and the answer comes out to ~0.4%.  Four out of every thousand new posts* on Stack Overflow get shared via a social media button (the ratio seems to hold constant for the rest of Stack Exchange, but there’s a lot less data to work with so my confidence is lower on them).

Data Shows They’re No Good, Now What?

Change for change’s sake is bad, right?  Benign things shouldn’t be moved around just because, but for social media buttons…

We know our users don’t like them, they’re sleezy (though we were very careful to avoid any of the tracking gotchas of +1 or Likes), and they’re cluttering some of our most important controls (six controls in a column on every question is a bit much, but to be effective [in theory] these buttons have to be prominent).

These buttons start with a lot of downsides, and the data shows that for us they don’t have much in the way of upsides.  So we we could either trudge along telling ourselves “viral marketing” and “new media” (plus we’ve already got them right, why throw them away?), or admit that these buttons aren’t cutting it and remove them.

So, are you measuring the impact of your site’s social media buttons?

You’re probably not in the same space as Stack Exchange, your users may behave differently, but you might be surprised at what you find out when you crunch the numbers.

*By coincidence this number seems to be about the same for questions and answers, it looks like more people seeing question share buttons balances out people being less enthusiastic about questions.

4 Comments on “Are you measuring your social media buttons?”

  1. Great Post. Definitely good to look at the data for this stuff.

    I was just adding some “share” buttons to a site the other day and was astounded by their effects on performance… they didn’t really affect page load times, but if you let the default icons and images be created by their respective parties (fb/twitter/google/etc) then it takes a significant amount of time to load

    the other thing (if you use their share-button scripts rather than hard-coded links) is if you have other asynchronous calls being made after page load, it can delay their execution depending on the browser.

    I’m curious – do you have any data on what the SEO implications are for you guys removing them?

    • We removed them yesterday so it’s too early to look at data about SEO impact.

      However, since in practice almost all SEO impact from sharing to date was coming from people copy/pasting links, and Google has no reason to distinguish between using a share button and not, I doubt there will be any impact.

  2. While I agree with the result in this case, the analysis feels a little flat. The analysis for this feels like it should be similar to offering a deal at a restaurant; the flyer gets them in the door, but the total value is based on their repeat business after that. Interesting data to me here would have been how many new customers does SE get out of the viewers of these social media links, what percentage of your future page hits do you estimate will come from these people, would you have gotten those people to visit the site any other way, etc. The argument that everything pales in comparison to google traffic is weak and is currently a bar so high that anything measured against it will be up for “drop it if we don’t like it” (which is a conclusion I feel confident in assuming based on the questions/answers linked). Even more interesting data to me in your blog post would be where does SE place the threshold for keeping links such as these? Not “this is obviously low” but “this is what we would have expected/needed to see in order for these links to be worth it”.

    As I state however, I agree with the result. In large part, social sharing links are about finding interesting things to read or deals on the internet. I would have never put those links on the SE sites because I already asked my friends/colleagues before I post there. And as you stated they do take up real estate which most SE sites are in short supply of. I love the bots however, and follow links from them regularly, though I wish the links in them were based more on an interesting score than a hotness score.

    • I’d like to be able to offer some statistics about how many new contributing users social media buttons brought it, but when I look at the data I can’t find any evidence of *any*. Now my gut tells there have to have been *some*, but crunching analytics for that sort of data is tricky. Given the likely false positive/negative rate of any analysis, I chose to omit that discussion rather than claim “zero” with low confidence.

      Now, since there’s no evidence that those buttons were pulling in anything but pageviews I felt justified in the Google comparison. It’s apples to apples in practice, if not necessarily in theory.

      Google takes up none of a page, no branding, no nothing. The share buttons naturally take up some non-zero amount of screen real-estate. If the buttons were bringing in 2-3% of our traffic, well yeah we’d probably leave them around. Truth is all of the social media networks are bringing in less than 0.2%; and that’s not just counting traffic from the share buttons.

      I think by the time your justifying dedicating resources to a **fraction** of 0.2% of traffic you’re just deluding yourself. Social media has a strong reality distortion field in my experience, but in the cold light of day the numbers are pretty harsh.