Are you measuring your social media buttons?

I’d like you to go to a Stack Overflow (or any Stack Exchange) question, and see if you spot a nominally major change.  Such as this one, with a nice answer from Raymond Chen (I think it’s been more than month since I recommended his blog, seriously go read it).

Spot it?

Of course not, what we did is remove our social media sharing buttons.

Left is old, right is new.  A number of other instances were also removed.

The Process

I’m a big advocate of arguing from data and while I freely admit there are many things that are hard to quantify, your gains from social media buttons aren’t one of them.

When discussing sharing buttons, be careful about establishing scope.  Keeping or removing social media buttons isn’t about privacy, personal feelings on Twitter/Facebook/G+, or “what every other site does” (an argument I am particularly annoyed by); it’s about referred traffic and people.  There’s a strong temptation to wander into debate, feeling, and anecdotal territory; stick to the data.

You also need to gather lots of data, at Stack Exchange we prefer to slap a pointless query parameter onto urls we’re tracking.  It’s low tech, transparent, and doesn’t have any of the user facing penalties redirects have.  In particular we slapped ?sfb=1, ?stw=1, and ?sgp=1 onto links shared via the appropriate sharing buttons.  Determining click-throughs is just a matter of querying traffic logs with this approach.  We’ve been gathering data on social media buttons in particular for about four months.

Note that we’re implicitly saying that links that are shared that nobody clicks don’t count.  I don’t think this is contentious, spamming friends and followers (and circlers(?), whatever the G+ equivalent is) is a bit on the evil side. Somebody has to vindicate a share by clicking on it, otherwise we call it a waste.

This sort of nonsense isn’t worth it.

With all this preparation we can actually ask some interesting questions; for Stack Exchange we’re interested in how much traffic is coming from social media (this is an easy one), and how many sharers do so through a button.

Traffic gave us no surprises, we get almost nothing from social media.  Part of this is probably domain specific, my friends really don’t care about my knowledge of the Windows Registry or the Diablo III Auction House.  The sheer quantity of traffic coming from Google also drowns out any other source, it’s hard to get excited about tweaking social media buttons to bring in a few thousand extra views when tiny SEO changes can sling around hundreds of thousands.  To put the difference in scale in perspective, Google Search sends 3 orders of magnitude more traffic our way then the next highest referrer which itself sends more than twice what Twitter does (and Twitter is the highest “social” referrer).

Now that I’ve established a (rather underwhelming) upper-bound for how much our share buttons were getting us, I need to look at what portion can be ascribed to the share buttons versus people just copy/pasting.  This is where the slugs come in, presumably no-one is going to add a random query parameter to urls they’re copy/pasting after all.

You basically want to run a query like this:

SELECT *
FROM TrafficLogs
WHERE
-- everybody tries to set Referer so you know you got some juice from them
(RefererHost = 't.co' OR RefererHost = 'plus.url.google.com' OR RefererHost = 'www.facebook.com')
AND
-- Everybody scrapes links that are shared, don't count those they aren't people
UserAgent NOT LIKE '%bot%'
AND
-- Just the share buttons, ma'am
(Query LIKE '%stw=1%' OR Query LIKE '%sfb=1%' OR Query LIKE '%sgp=1%')

Adapt to your own data store, constrain by appropriate dates, etc. etc.  But you get the idea.  Exclude the final OR clause and the same query gives you all social media referred traffic to compare against.

What we found is that about 90% of everyone who does share, does so by copy/pasting.  Less than 10% of users make use of the share buttons even if they’ve already set out to share.  Such a low percentage of an already pretty inconsequential number doesn’t bode well for these buttons.

One final bit of investigation that’s a bit particular to Stack Exchange is figuring out the odds of a user sharing their newly created post.  We did this because while we always show the question sharing links to everyone, the answer sharing links are only shown to their owner and only for a short time after creation.  Same data, a little more complicated queries, and the answer comes out to ~0.4%.  Four out of every thousand new posts* on Stack Overflow get shared via a social media button (the ratio seems to hold constant for the rest of Stack Exchange, but there’s a lot less data to work with so my confidence is lower on them).

Data Shows They’re No Good, Now What?

Change for change’s sake is bad, right?  Benign things shouldn’t be moved around just because, but for social media buttons…

We know our users don’t like them, they’re sleezy (though we were very careful to avoid any of the tracking gotchas of +1 or Likes), and they’re cluttering some of our most important controls (six controls in a column on every question is a bit much, but to be effective [in theory] these buttons have to be prominent).

These buttons start with a lot of downsides, and the data shows that for us they don’t have much in the way of upsides.  So we we could either trudge along telling ourselves “viral marketing” and “new media” (plus we’ve already got them right, why throw them away?), or admit that these buttons aren’t cutting it and remove them.

So, are you measuring the impact of your site’s social media buttons?

You’re probably not in the same space as Stack Exchange, your users may behave differently, but you might be surprised at what you find out when you crunch the numbers.

_{*By coincidence this number seems to be about the same for questions and answers, it looks like more people seeing question share buttons balances out people being less enthusiastic about questions.}