Wednesday, June 13, 2012

The Most Useless Twitter Stat (And How You Can Make it Better)


Arrow, source: Microsoft Clip Art Gallery

As you know, I love my Tweet chats. There's nothing like sitting at home or at your desk, chatting with dozens or even hundreds of tweeps about something that is interesting to all of you. That kind of interaction is really second to none as far as training and development goes.

[Aside: there is a phenomenal list of every Twitter chat known to man that you just HAVE to add to your Google Docs. Check it out. For reals. I'll wait.]

These days, though, the organizers of Twitter chats try to promote their reach and increase their attendance by providing stats along with their chat transcripts. Which is good. It's great. Analytics is a huge part of my job, and I love me some numbers. Numbers of tweets using the hashtag, numbers of members participating in the discussion, number of RTs, etc. All very very good stuff.

And then there's the stat I loathe the most: IMPRESSIONS.

This one is pure garbage. Oh, it can be easily translated into NOT garbage. But the way most people use it, it's COMPLETE GARBAGE. Let me tell you why:

Impressions is calculated as the sum of followers across all people participating in the chat.

The sum is a lie.

In social media, just HAVING followers does NOT guarantee that they will SEE your tweets. Everyone knows this. In fact, it's estimated that depending on the time of day, a very small percentage of your tweeps will be online at any given moment. Just for fun, let's say that number is 10%.

Now let's imagine that these 10% are on Twitter when you are on the chat. How many of them are paying attention to you and your Tweets at that moment? Some are multitasking, some are chatting, some are following lists or hashtags and not checking their Home feed, and some follow too many people to see every tweet that their tweeps send out.

So, how many do you think actually see your tweets, of this 10%? Half? If you're really lucky. Or some sort of celebrity whose tweets are sought out by the general public. But let's assume you're not @ladygaga.

By this math, the correct number of Impressions for the chat would be closer to 5% of the sum of followers across all people participating in the chat. If we say 50 participants with an average of 200 followers apiece, your Impressions just went from (50 x 200) to (50 x 200 x 0.05) or from 10,000 to 500.

Big difference, eh?

Using Meaningful Metrics

So why does ANYONE AT ALL use this metric? Because it looks good. It's impressive. And if there is one thing we know from all the "gurus" and "experts" on Twitter (barf), it's that you can be whatever you want to be in social media.

How about being honest? Here's how you do that:
  • Instead of reporting Impressions, you can report POTENTIAL IMPRESSIONS. Which demonstrates that you realize this is a crap shoot and unlike page views, there is no way to establish how many people really saw ANY of the Tweets in your chat stream.
  • Report REACH: how many people participated in the chat, whether actively (by, you know, chatting)  or passively (by retweeting or responding to a participant, but not directly participating in the chat).
  • Report UNIQUE REFERENCES: the number of times you, your brand, your chat, your chat topic, your discussions were mentioned online during the period of the chat and during the period immediately after the chat. If anything, that is a much better indication of whether your chat and the discussed topics had any traction whatsoever. Kind of like doing unaided recall for TV ads.
  • Report ACTIONS: how many actions did people take as a result of the chat? How many links did they click from the chat Tweets? How many referrers came to your site from Twitter during and after the chat? How many people followed up by email or downloaded something from your site or watched a video or listened to a podcast as a result of the chat? Referral stats from your web analytics + goal completion stats will give you this number.

The Problem: Metrics vs. Analytics

The problem with social media stats is the same as the problem with web metrics: people assume that metrics (data from a system of measurement) is the same as analytics (the science of analysis). But metrics are just data, and you have to interpret (or analyze) it to yield any useful information.

So promise me this: promise me you won't take numbers at face value. Promise me you will question anyone who offers them to you this way. Promise me you will analyze not just measure.

Chances are, your reach isn't as good as it looks on paper. But then again, neither is anyone else's. If you can figure out your true reach, you'll be far ahead of the game; because once you know your true numbers, you will be able to see whether your efforts are having any real impact on your target audiences. And you will be able to course correct where required. In the end, being able to say
"we had # participants, # tweets, # click-throughs on our links resulting in # videos watched/ page views/ downloads/ subscriptions during last night's chat and over the following weeks saw a #% increase in our website traffic"
is a lot more meaningful than:
"we had 10,000 impressions but we don't know for sure and we aren't sure if it had any impact in the weeks following the event."
Which one would get you the raise?

Knowledge is power, my geeks.

3 comments:

Karen said...

So well said. Also, thank you for linking to the twitter chat spreadsheet. I have been trying to find that off and on and I couldn't remember where I'd first seen it.

Annie @ PhD in Parenting said...

And even then, they need to take into account that a portion of those "impressions" may have been people like me crashing the party and raining on their parade. :)

Blaze said...

I love how you saud" Let's say it's 10%". IF you accept the power law in your life, it will always return the favor. ;)