What Losing a Socialbots Competition Taught Me About Humans & Social Networking

When social network users consider robots networking, anything from fake Twitter pages spamming links to Chat/AIM bots could come to mind.While 'social bots' may so far be, at the best, still brushing up on their social skills and, at the worst, outright malware, it may not always be like this. A recent Gartner study predicted that by 2015, 10% of users within an individual's social network will be robots. This doesn't necessarily mean that 10% of a user's network will secretly be automated pages, but that brands may leverage automation to provide basic interaction (basic question & answer, news aggregation & promotion, customer service) without direct personal staffing.

While 'social bots' may be quite some ways from the de facto choice for basic communication, current technologies seem to be leaning toward options to create an automated solution. CoTweet and other group Twitter tools help to manage messages based on topic and priority (a step towards prioritizing interaction), while API developers are already creating interactive solutions that mimic basic user functionality (both for useful and malicious goals). While the most common example of an automated page a user might interact with presently is a suspicious model photo attached to a Twitter account promoting odd links, this could change soon enough.

With a sales pitch like that, how could anyone say no...

Personally, I found out more about 'social bots' through entering the Web Ecology Project's Socialbot 2011 contest. After first reading about the contest on Twitter, I helped to field a team of other interested media folks. In total, 6 teams entered and 3 fielded entries, with a spectrum of contestants coming from computer science & programming, development & media. The range of skills interested in the contest shows the areas of development that are going to drive technology like this forward.


The Contest

The network diagram above shows the initial 500 users and their connections between each other before competition efforts

(Source: Web Ecology Project)

The competition itself hoped to expand what we know about social robotics through development of an automated Twitter page(s). Each team had two weeks to develop a profile or series of profiles, which once activated, would run uninterrupted for 2 weeks (with the exception of a halfway point improvement day). Within these two weeks, the page would reach out to 500 pre-selected users, randomly sampled by the contest holders, who must be interacted with. Teams couldn't tell users they were in a contest, but could indicate that their lead page was a robot.

Teams scored points by driving user interaction. Scoring consisted of:

• 1 point - per each user following a team's lead Twitter page
• 3 points - per each user RT'ing a team's content, '@' replying to the page or mentioning them

Users would be allowed to restart their page 3 times after being reported as spam, before finally loosing the ability to score. Other teams weren't allowed to mention that a competitor page was in a contest or report them as spam.

At the end of the two weeks, the team with the most points wins $500 and the 'Socialbots' Cup.


The Strategy

Our Team's strategy, (Team MS-UK / Team A in the final standings) operated on three core principles:

• The socialbot would use content to drive interaction
• Twitter's TOS would be respected as much as possible, with every part of the plan judged against  avoiding spamming users and use the targets reporting spam pages as a reward for us and a barrier for competitors
• Wherever possible, the page would utilize realism to get as valid an interaction as possible

Based on these principles, we decided upon making a singular page crafted against the clustered interests of the 500 users. To begin, network analysis started working with the provided data from Web ecology and moved onwards to analyzing publicly available data from page names, connectivity ratios and tweeted content.

The provided clusters created 9 distinct areas of users, each with different interests, locations and ways of using Twitter. Prominent interests groups included: animal lovers, UK/EU residents, sports lovers, social media experts, metropolitan career women, hunting enthusiasts & other suspected robots. Users judged to be under 18 were automatically excluded from any activity by the robot. Surprisingly, the robot segment contained at least 60 automatic pages, creating a situation where a robot was programmed to engage with other robots.

Within each segment, users were prioritized by authority within the group (e.g. the number of other segment users following them), with those that were most authoritative set as influencers.


The Persona

 

 Our automated page ('@sarahbalham') aimed to look like a real twentysomething's page, with elements of her persona emerging throughout it (i.e. Wooded background). Though I always worried a graphic designer using a web generated icon as a avatar was a bit of a giveaway.

 

 Based on the data gathered from cluster analysis and influencer content, our entry persona was set as:

• a 24 year old female graphic designer (decided as it was an age/gender mix which fit conversation with multiple segments and a career which allowed for a range of content to be discussed)
• an expat from the Southern US (Georgia) living in London UK (chosen as it related well to the hunting/metropolitan/EU & UK segments, as well as mirroring the authors current experience, which allowed for voracity in tweeted content)
• interested in design, fashion, art, the outdoors & music (allowing for content which would reach across segments without alienating non-engaged users)
• a proud dog owner of a small, but loved dog named 'Russel' (a way to engage both animal owners and hunters through the love of a pet
• name wise, a combination of a London Borough & a neurological structure led to us calling 'her' "Sarah Balham"

The Program

User segmentation allowed our program to follow a set group of users each day, meaning that users were added gradually over the course of the first week. The program avoided mass following & 'churn' (unfollowing those who wouldn't follow back and then following them again shortly), instead adding a range of 2-5 'target followers' and 2-5 non target ancillary followers (those not in the 500, but around the group) in random periods during the day. The program followed the lifecycle of a normal Twitter user, sleeping in a random range between 11pm and 7am local time, with any responses or actions waiting until the 'she' woke up.


Based on the segmentation, the program behind the page operated on four core content functions:

Scripted tweets about Mexican food in London reached across multiple segments, engaging users from back in the US, as well as UK residents looking for tips/offering advice

Content & Segment must align
Given that users weren't to be directly spammed, the notification of their account being followed served as the main point on which they would decide to engage with the account. Because of this, 'Sarah' would tweet content relevant to the segment before beginning to follow the users, hoping to present the most pertinent content to the target users.

 Retweeting from influencer users allowed SarahBalham to talk about topical issues in a relevant manner, without the risk of parsing news sites 'herself'. If another user has posted it to Twitter, it is safer to assume that it can be discussed.

The best content comes from a mixture of internal and external sources

The program pulled 'her' linked stories from influential users as well as prominent blogs. Content categories were decided at three intervals during the day and a mixture of blogs (i.e. Social Media involved pulling stories from Mashable at random and tweeting about them) and influential tweets (i.e. A key user's tweet is Re-tweeted based on whether it has a linked source) related to the segment were then programmed to tweet at key times.

Stringing scripted days of tweets together gave a greater relevance to lifestyle tweets. In some, the account took the day off to wander around London, where in others she struggled through a hard workday after commuting problems

Life narrative is a key part of Twitter for both affinity & as a break to linked content
'Sarah' had a set of 20 different days she could live out. A day would be randomly chosen upon the program 'awakening', which would then consist of 4-7 tweets playing out during the day. These were written to create an engaging story around Sarah, as well as provoking conversation (general questions were asked of the entire Twitter audience) for those paying attention. Tweets would vary based on whether the day was a weekday or weekend and would talk about non-worrying, common situations (i.e. bad commute, tips to stop the dog chewing on the rug). 

While Follow Friday & other grouping listings were a powerful options to drive possible engagement, they had to be sparsely used to preserve believability, as well as staying true to overall strategy

General interaction within the community segments can non-invasively build up prominence
The program was tasked with never spamming users, but it did make use of organic weekly occasions to mention influencers within segments. Occasions such as 'Follow Friday' allowed the program to choose influential users and reach out to them subtly by listing them in these groups. While these were used sparingly, the specific segments led to using manufactured days such as 'Woof Wednesday' for animal lovers and 'Media Mondays' for those within advertising/design. These community tweets were considered as one of the ways to reach out to users uninvited (with retweets being the other).

On top of these principles, the program was set to respond within 15-35 minutes (at random) to thank users for Retweets of 'her' own content.

From a functional level, the program was a .Net desktop application hosted on a virtual web server. All scripts, downloaded content, user lists, segments and directions on daily activity were stored in a MySQL database behind the application. The program operated on 15 minute intervals for content decisions/interactions and a minute interval to post scheduled content. No functionality existed to respond articulately to users, due to both time and the assumption that this wouldn't occur often enough to qualify attempting it in the 2 week development time. The source code is available (through MIT Open source license) from the Web ecology project here. 

Authors Note: As the coder, be kind if you download it, as two weeks working only in the evenings & weekends was a quick turnaround to schedule/plan/develop/test & deploy a socialbot.


Competitor Strategy

While our strategy was very much around a singular point, both of the other teams fielding a full entry utilized a swarm method (i.e. multiple pages/bots supporting a main bot).

Team EMP / 'Team C'

A New Zealand entry, the team created a main bot named 'James Titus' who lived in Christchurch and really loved his pet cat (in fact he really loved cats). The team wrote a brilliant post outlining their blog post outlining their experience here, but in summary:

• Team EMP's bot utilized a swarm to test for follow backs. Each sub-bot would test to see if a user would follow it and forward on amenable users to the main bot.
• However, the main bot followed all 500 target users immediately.
• Within Week 1, the bot posted content related to random messages and pictures of cats scraped from Flickr, which syndicated through the created blog 'Kitteh Fashion'
• Within Week 2, the bot swapped strategies, asking users a list of random questions to motivate a response.
• If users responded or mentioned the page, a random response was tweeted back (i.e. '@user sweet') which drove further engagement.
• The page also created #FF and created #WTF 'Wednesday to Follow'  as group listings to drive interest

 Team Growth20 / 'Team B'

A US based entry, the team created a female ninja persona (ninjzz), looking for friends on Twitter. The teams persona developed a bit into the second week and an increased amount of activity occurred after the halfway point.

• The main bot seemed to monitor the target network and repeat tweets it observed. 
• Some of these tweets were in general, while others were directed at target users
• Further, it also did #FF group listings

Interestingly, this team was the only one to deploy countermeasures against the other teams, as it started pages such as @botcops, which (ironically) was a bot messaging the target users and notifying them that the competitor's entries were robots.


The Competition

      At the start of the competition, without knowing the competitor strategies, we considered that our strength would be believability and avoiding being reported as spam, but our weakness might be a lack of frequent scoring opportunities. This seemed to be confirmed when we the competition began and we saw the competitor's entries. Team EMP, the winner and leader throughout the competition, rapidly gained followers and launched off to an amazing start. Over the course of week 1, we began to see a bit of growth as we added followers, but still lacked many responses from our growing network. 


      As day 7 approached, the midway point and only opportunity to update code, we had closed the gap with the leaders, possibly indicating our deep engagement strategy would pay off. At the halfway point, we increased the rate of messaging put out by 'Sarah', but stayed relatively consistent, thinking our slow growth would carry on. 


     Alternatively, the other teams deployed some noticeable adjustments, with Team Grow20's countermeasures launching at the same time as Team EMP turned on their engagement strategy. As shown on the graph above, once EMP started asking questions of its network, their lead became increasingly hard to beat, leaving our ownly chance for victory in their network reporting the page for spam. As the competition came to an end, the users messaged by EMP weren't attempting to ignore or shut down the page, but instead were conversing with them. In addition, Team Grow20's countermeasures had slowed our scoring, leaving us in a vulnerable third as the competition closed. 


At the completion of the contest, the scores reflected the power of proactive communication over aiming to strike a believable page, as: 

• Team EMP: 701 Points (107 Mutual Follows, 198 Responses)
• Team Grow20: 183 Points (99 Mutual Follows, 28 Responses)
• Team MS-UK: 170 Points (119 Mutual Follows, 17 Responses)

 Source: Web Ecology Project

Despite, the different scoring of the three teams, the final network structure shows how well each team shaped a network around them. The structure shows that in spite of approach, each bot was able to ingratiate itself into the target network, forging ties with about 1/5th of the possible network. While it can be argued that these 1/5th were either bots or the very open users within the target, the rate of growth occuring over 14 days is quite intriguing.

So what did I learn?

Regardless of 'Sarah's' performance, the socialbots contest provided a great opportunity to learn more about not just social robotics, but wider area of social networking. A few of the key lessons I took from the experience:

• Users on Twitter aren't as aggressive towards intrusion as one might assume, it seems the self policing userbase has yet to fully activate

A lot has been made of the self policing power of social networks. As users, myself included, have encountered bots before on the network, I assumed that the reporting spam function of Twitter would play a much larger role in the contest than it did. While I assumed that a large amount of the target userbase would avoid all three bots (which 4/5ths of the target segment seemed to do), its quite surprising that the majority of these users chose to passive avoid the intrusion, over actively reporting any follows or directed tweets.

While Twitter is arguably much more casual in networking than sites such as Facebook, the avoidance of automated pages, even when they are accused by other pages of being a robot, seems to skew towards ignoring the presence over policing the network.

• User influence scores might have a way to go before they become reasonably exact

While skewing towards believability didn't help our page win the competition, I was quite surprised how well 'Sarahbalham' performed as an influential user. At the end of the competition, the page was checked on both Klout & Peerindex.net and it seems that our strategy of tweeting status updates and content resonated with the algorithms for both.

Peerindex's  score was a bit lower for 'Sarah', but indicated that it was 80% sure she wasn't a bot, the exact score its also given my personal page. 

While its funny to laugh about the automated bot becoming influential, it has an interesting implication for paid tweets. As users sign up to tweet for cash and sites flaunt their user influence to possible advertisers, these numbers become much more profitable. Running a swarm of specialized pages, each with an artificially cultivated community around them, becomes an interesting opportunity for the enterprising (if unethical) developer if paid for tweet communication becomes more widestream.

• Driving reactionary activity is much easier than soliciting responses

While EMP showed how easy it was to elicit a response when asked, one response from EMP's JamesTitus bot shows the possible shortcomings to robotic conversation.

Source(EMP's contest wrapup)

One of the reasons I was initially interested in socialbots, were both personal and professional projects done on Twitter in the area of automated page response. After helping to create @AskLG3DTV (an informational bot answering questions about 3D TVs with a video) and @RPStweet (a rock paper scissors game processing '@' reply tweets and answering them with a game choice), I was surprised to see how counterintuitive soliciting '@' replies is for some users. While users were surprisingly happy to respond to seemingly unrelated questions (as aptly proven by Team EMP's bot), it seems (rather intuitively) that either instructing (in the case of the two above examples) or attempting to motivate non-response '@' messages (as in the case of 'SarahBalham') requires much more trust or work.

• Robots may have a way to go on social networking, but there are more out there already than you think 

While the Gartner study sets 2015 as the age of developed and identifiable robots, 2011 seems to be developing into the age of rudementary robotic presences. As our research into the target 500 users illustrated,  over 80 pages were estimated to be automated. While most of these were nothing more than autofollow scripts (though this wasn't as prevalent as one would assume), rss feeds or content farms, it shows that social robotics is already working in the network.

• Social robotics has more to do with brands and marketing than you first think

With the growing prevalence of aggregating content for websites and agencines/companies using Twitter popularity/network action to accept interns/new grads, social robotics should be watched with interest. How long until a developed 'botnet' of coordinated automated pages unleashes a manufactured controversy that spreads around a social network? While bots mostly push spam links currently, the power of automated pages with a developed network reach pushing an agenda is easily within reach.

At a more general (and ethical level), it may seem that the opportunities for brands and social robotics is limited. Brand presencs are carefully managed and a genuine tone in reacting is key for social media. However, as busines activities increase across the social space and the level of resource required increases, segmenting responses and automating basic interaction will become more and more necessary. While social robotics may currently pose more of a risk to network health, it seems that in the long run, it may be required (in some capacity) for many small to medium (and possibly large) scale companies as their social activity expands.

 While the competition didn't turn out as well as I would have hoped, I gained a ton of useful insight and had a great time doing it. Congratulations to the winning team and thanks to the Web Ecology project for hosting. If anything, I'll leave the last word to my Frankenstein like Twitter creation:

Outdoor Advertising Takes Us One Step Further to Living in a Sci-fi Movie...

Movies always seem to utilize outdoor advertising in some bombastic ways whenever they need a 'dystopian' future-scape. From 'Blade Runner', 'Idiocracy' & 'Robocop' to 'AI', 'Back to the Future', 'Minority Report' & 'They Live', outdoor advertising plays a role in conveying an emphasis on the conspicuous consumption & promotional opportunities of the future. However, this week, 2 stories in the mainstream press seem to have emphasized how close we are to having at least the capability, if not the consumer comfort with, some of sci-fi's outdoor advertising channels.

Something tells me these might be kinda noticeable....(via source)

First, the city of Miami has fast tracked approval for two 'skyscraper' sized digital LED screens within the city. The digital ad platforms would come in at a total height of 50 stories, with the first 100 feet being supplied by a parking deck. While event type installations are nothing new within outdoor advertising, this seems, given the mock-up, to take attention grabbing dynamic content to a new level. While sights such as Picadilly Circus neon signs barely go above 5 stories, the 22 story advertising installations would bring us a step further to the ever present advertising in films like Blade Runner (now if we only had a zepplin...). So, film associations aside, will something this big work? The panels come in a line of large, historic outdoor installations, so they may follow other examples and become part of the skyline. However, they must strike a pretty hard balance between being bombastic & noticeable without being a horrible eyesore.Either way, the creative opportunities for advertisers seem pretty varied (someone planning a monster movie campaign is salivating already).



Secondly, in smaller scale, but customizable advertising, the Telegraph (and my Daily Links section) featured a story about the advancement of consumer customized, digital advertising panels (ala Minority Report, as shown above). Technology such as this has been in development for some time, with previous installations tracking approximations of age and gender from a web cam monitoring consumers. Currently however, IBM has spoken of taking the tech a step further, utilizing RF-ID to obtain user information for a more granular customization.

 Whether consumers accept something like this or see it as an invasion of privacy depends on the implementation of the technology over the next few years. Consumer attitudes are a long way from accepting a very tangible and public representation of what advertisers know about them and the technology to do more than approximate characteristics is far off from being widely accepted. If advertisers, technology providers and media owners can slowly progress the general consumer attitude to a more accepting view of data customization through RF-ID or another wireless solution, then something like this may have a chance of occurring. Alternatively, high costs, privacy concerns and lack of a standardized information system may limit this technology to webcam based approximation.

Geo-location and Privacy: A Subtle Balance

This post is also featured on the recently launched "Typing On The Wall" Blog.....

Location Based Social Networking (LBSN) is prominent within the media zeitgeist at the moment, driven by the increasing growth of the category's current darling, Foursquare. As Foursquare recently added its 2,000,000th user and its 100,000,000th check-in, some can argue that the increasing growth rate (having added its 1,000,000th user only 3 months ago) and press coverage of the network are currently the most prominent stories.







CNN partnered with Foursquare during the World Cup to direct users to check-in at certain locations worldwide, unlocking badges and interacting with content


Given the company's growth rate and the utilization of the network by advertisers such as Domino's, the Huffington Post and CNN, this may be set to change. For media however, questions about Foursquare and other LBSNs go past the common questions of user growth and campaign concepts into "What insights can the network's data generate?"

Campaign metrics are key to creating attractiveness for advertisers, as greater data availability allows marketers to justify activity and modify comms efforts. Standing conversely against data availability however is user privacy.







User privacy concerns are inherent in social networking, but even more so when dealing with a user's location history. Sites such as PleaseRobMe.com popped up early on in the emergence of the current geo-location trend, highlighting a feed of users exposing their locations through auto posts onto Twitter. Concerns about user privacy have also been illustrated in recent studies, with a recent US/UK survey showing 55% of respondents worry about privacy relating to geo-location (Webroot).

Balancing privacy, key to growing the base of active users, and providing useful data & insights, key to growing advertiser investment, means that LBSNs must strike a careful balance as both network operators and data providers. Foursquare's solution to this issue has been two fold, providing basic data about venues & friends through the API (Application Programming Interface), while also providing business level analytics to venues claimed by the owners.

Given the open nature of the API, anyone can obtain a key and begin obtaining data and building applications, data is limited to what is available on the website. User specific data is limited to friends, while venues can provide the current amount of check-ins, who the mayor is and venue information. Data from the API is useful to monitor popular venues for a certain area or to create visualizations such as these, but it doesn't provide the specific level of data required to manage a large amount of CRM or loyalty plans.

Alternatively, the Foursquare analytics dashboard for a venue currently displays check-in data over time, different time periods and whether users clicked through to provided Facebook & Twitter links. In addition, it provides a stream of recent check-ins & top users, creating the ability to target individual users based on their behaviour for promotions. User check-ins not relevant to the specific venue are still protected.

Foursquare's approach to data visibility balances user privacy with metric creation in a way that provides venues an opportunity to customize promotions, while not risking user outrage by making data completely public to marketers. This strategy seems to be the way forward for LBSNs, as it encourages growth by unlocking the value of the network's data, without leveraging the privacy of the user base.

Top London Agencies by Foursquare Check-ins (14.05.2010)

Agency:	Id:	Check-ins:
1.) McCann London	172633	684
2.) Wieden+Kennedy	219558	579
3.) DLKW	154868	493
4.) Agency Republic	154743	488
5.) Mindshare	161142	439
6.) MediaEdge CIA	537324	353
7.) Dare Digital	154565	339
8.) Saatchi & Saatchi	244912	303
9.) PHD	173059	288
10.) Profero	161146	265
11.) We are social	1168488	251
12.) Digitas London	154598	240
13.) SapientNitro	154900	235
14.) MPG/Media Contacts	1200388	232
15.) Mediacom	203677	203
16.) Wunderman	605057	202
17.) BBH London	154630	190
18.) Zed Media	154651	190
19.) glue London	523581	174
20.) AMV BBDO	171053	168
21.) Publicis	236312	152
22.) Ogilvy	156590	144
23.) LBi London	154854	140
24.) Fallon London	183035	137

25.) Rocket

538270

133

(Click for large version)

      As with last week's attempt at pulling Foursquare check-ins for advertising/media around London, 105 existing advertising agencies and 4 new locations were analyzed through Foursquare activity.With only two weeks of complete data, I'm still reconciling merged venues and entries with incomplete or closed duplicates.

      I know many agencies have a variety of check-in locations within their offices (3rd floor desk with some interesting tips at a certain advertising agency...I'm looking at you), but I chose the most popular/obvious choice presented to me, much in the way a new visitor to an office might. If you have an agency you feel I've left off (there are many), and you have enough check-ins to place you in the top 25, message me on Twitter (@dubosecole), email me or leave a comment and I'll add your Foursquare ID on the list for next time.

The Carling Cup "Digital Final" on Twitter: A Study in the Challenge of an Event's Scale

Introduction

      The 2010 Carling Cup Final has come and gone and while Man. U fans are probably a little more pleased with the outcome than Villa supporters, the game itself stands as a strong lesson on the ‘ins’ and ‘outs’ of driving digital engagement for an event. Carling touted the game as the world’s “first digital final” which included: putting fans names on digital displays during the trophy ceremony, voting on the winning songs, placing fan created banners on pitch side advertising and linking this to charitable donations.  In addition to this, the hash-tag “#CCF10” was publicized in the run-up to the game. In theory, the brand wanted to utilize Twitter (in concert with other digital channels) to unify viewers in conversation, extending the experience of the final to those viewing it outside of the stadium. Harnessing Twitter activity as its own point of interest, Carling also created pages tracking Twitter activity around the event, showing activity by geographic region and time.

Source  Add this to the list of things I probably won't win (this year)

      The logic behind Carling’s intentions seems sound, especially when looking back on other events which unify Twitter users in conversation. As I’m writing this on Sunday night, preparations are under way for tonight’s Academy Awards in the US, which will drive network conversation from the Red Carpet, through the event, to post analysis with the winners and losers on Monday morning. Previous events like the Super Bowl, Ashes, the BCS National Championship Game and the Winter Olympics have all also shown that Twitter activity is rather reactive to large scale viewing events. However, in the case of the Carling Cup, harnessing network interest in the event is harder, due to the lower engagement level of the game. If you supported the two teams involved, then the game was probably pretty big for you, but Carling faced a challenge in not only interesting those without a stake in the final, but also organizing the attention of the already interested fans for engagement. In declaring the game to be the “first digital final”, it seems they hoped to generate a meta-story around the match which could possibly multiply interest. While many aspects of the campaign are hard to independently look back and analyze, Twitter activity can stand as an interesting proxy for campaign interest. With this aim in mind, looking back and judging the effectiveness of the Twitter campaign requires answering a few core questions:

1.)    Did the brand manage to engage with those already conversing about the final, as well as those who were neutral?
2.)    Did those who conversed about the game do so more frequently because of the brand’s efforts?

Twitter Analysis

      To gain some insight on the brand’s Twitter efforts, data collection was set to gather figures on both the specific hashtag for the game (“#ccf10”) and overall mentions of the event (i.e. “Carling Cup”, “Carling”, etc.). Analyzing the data was focused specifically on the time directly before, during and after the Carling Cup to see if the brand was able to foster reactive user activity.

 

     Top line figures indicate that the volume of messages was overwhelmingly towards “Carling” or “Carling Cup” related tweets instead of specifically tagged #CCF10 messages. In fact, between 14:00 and 18:59 on gameday, only 8% of messages (388 out of the total 4,798) utilized the promoted hashtag. Conversation about the match can be assumed to have progressed as it normally would have, with non-specific instances of “Carling” or “Carling Cup” being the dominant form of expression (4,410 messages).

    The activity itself seems to show a rather continuous increase during the game, peaking after its conclusion. However, when the data is broken down by minute and charted against game events, as shown below, it becomes clear that the prolonged reactive growth shown in other events such as the Super Bowl wasn’t clearly found in the Carling Cup Final. The continuous growth implied in the above chart, is actually a rather varied mix of quick reactions in traffic.

     Minute by minute activity for the game builds quickly before kickoff and reacts to scoring from both teams, though the activity shows little carry over and dies off quickly.  Anticipatory conversation about the game during the pre-game and half-time periods wanes in a similar manner, with the conversation spike after the conclusion of the match dwarfing any pre-game or game activity. Such spikes usually indicate a boost in relevant network traffic from blogs/news sites syndicating links to articles through Twitter (in this case about Manchester United winning the Carling Cup). However, discounting syndicated tweets, it becomes clear that even post-match commentary didn’t utilize the specific “#ccf10” hashtag, with tagged activity dying off to nearly nothing after 30 minutes post match.

      Considering the first question about the Twitter campaign’s performance, the data seems to show that the “ccf10” hashtag failed to catch the attention of those already conversing about the game (through general brand terms). The difference between post game and during game activity also seems to indicate that users who reacted to the final’s outcome didn’t engage on Twitter during the game, implying that neutral or uninterested users weren’t encouraged to interact with the brand’s efforts effectively.

 

      Within users that did converse about the game, the majority did so sparingly. Analyzing user message frequency during the match afternoon shows that 82% of the 3,054 unique users talking about the game only did so once.  Of the 18% of repeat mentioning users, 2% could be considered heavy conversers, with more than 5 mentions. The heavy conversing users represented the best opportunity for the brand, as they were the most inclined towards the efforts.

     Within the ‘heavy mentions’ segment  19 users mentioned relevant phrases 10 or more times, while 2 did so more than 30 times and 1 did so in 61 tweets. Although adoption of the “#ccf10” hashtag was low overall, 100% of the messages from the two heaviest users (92 tweets in total) used the tag. Overall 38.5% of game related tweets from heavy users utilized the “ccf10” hashtag , versus 3.91% of messages from the rest of the analyzed users.

      Though the heavy mentioning segment may small relative to the overall sample of users who mentioned the Carling Cup, they did exhibit the most consistent conversational behaviour over time. As shown below, heavy mentioning users maintain a steady level of activity throughout the match, surprisingly peaking in activity after its conclusion and during the activity decline of other segments.

    Heavy Carling Cup mentioning users represent a relative bright spot for the brand within its Twitter efforts. Overall, they showed a consistent conversational behaviour throughout the match and adopted the hashtag at a rate well above other Twitter users.  The relatively small size of this segment means that the brand’s efforts didn’t shift the overall frequency of user conversation much, but insights about the depth of engagement for heavy users may indicate points of opportunity to grow this segment in future efforts.

Implications

    While the Carling Cup’s “#ccf10” hashtag and related Twitter efforts may have underperformed relative to the assertion of the world’s first “digital final”, several key learnings emerge from analyzing the effort.  With respect to the initial questions to rate the effort’s performance, it seems that Twitter users neither increased their frequency or depth of interaction with the brand due to Carling’s promotion.

     Carling faced a challenge in expanding the audience interested in the match beyond the fan bases of two teams. The Carling Cup Twitter data seems to show that the existing prominence of an event has a strong impact on how it influences network activity. Events such as the Super bowl are large enough that Twitter users discuss them simply because of their scale. The multifaceted draw of such events (Ads, Halftime show and the game for the Super Bowl or the Red Carpet, Comedy and Winners of the Academy Awards) mean that topics of related conversation are broad and open to multiple audiences. Small to mid level events such as the Carling Cup face the challenge of being singularly focused and small in scale. It seems that while Twitter can help to connect communities such as Man. U or Aston Villa fans, if such interaction doesn’t offer any new real depth of engagement, then overcoming a lack of scale is difficult. Without these smaller fan bases converting their close ties into high frequencies of interaction, related network activity remains low and no increased network prominence is gained for branded promotions.

     The difficulty of smaller events to gain interest on Twitter doesn’t mean that attractive small to medium scale brand events are impossible to create. The singular focus of such events may serve to limit initial interest beyond existing communities, but it also means that less clutter is present around branded communications. Those who discussed Carling on Twitter during the game tended to stay focused on discussing the teams involved, the game itself and Carling. If wider audiences had discussed the game, the clarity of the brand’s message stood a good chance of remaining evident.

     While the overall campaign seemed to feature various promotional draws, the Twitter campaign seems to stand rather independent of other parts. While the Carling website offered various Twitter based content (from analytics to page backgrounds supporting the teams), it doesn’t seem like any direct call to action existed. Using the hashtag allowed Carling to generate dynamic crowd sourced game commentary from the network, but this doesn’t seem like it was enough to drive user behaviour. Tighter integration with other aspects (such as the digital signage) is hard to do and rather risky, but could have created a more dynamic and salient feel to Twitter interaction with the event. The conversation about the game and the conversation about the game’s digital efforts never seemed to mesh together in the activity data, smothering any chance for the brand to foster network activity.

     One should note that overall, I really liked the idea of this campaign and that Carling was innovative to attempt to add digital depth to their final. Talking about underwhelming activity in the Twitter parts of the campaign in no way denotes that the overall efforts weren’t impressive. While the Twitter campaign activity may be considered a proxy for consumer interest, I imagine that those who interacted with the efforts in the stadium or through other digital channels had the depth of their experience enhanced. Overall, I think the Twitter aspects of the effort highlight the challenges faced by brands in growing interest for smaller scale events on the micro-blogging service, rather than indicating an error by the brand. Finally, I think it shows that the unpredictable attitudinal nature of users is fickle and even those who do engage do so in unpredictable ways, as shown below. Carling may have put on the first “digital final”, but hopefully it won’t be their last, as they can apply what they’ve learned this time to future games.

Mindshare Twitter Research

    I've been waiting to put this up for a while as we've been conducting some network and user analysis for the course of this year. As you can probably tell from the previous (4 part epic tome) on brand analysis on Twitter, I and 3 other great people at Mindshare (@acotterill,@JezP76 & @picolim) conducted qualitative research on Twitter  users and quantitative research (through a bespoke analytics engine I'm still very proud of) on Twitter user behavior and WOM pass-on rates. Our analysis yielded some interesting facts on how users utilize Twitter, through what platform they do and how messages travel through networks.

     I plan on writing a bit more about the research and its implications later on; but for now, check out below and let me know what you think.

Mindshare Ispy Twitter Research

View more presentations from Mindshare.

Tweet it your way? Twitter's Capacity for Consumer Sentiment Measurement - Part IV

Research Conclusions
   In the previous two parts of our Twitter Consumer Sentiment analysis series (II & III), we aggregated and analyzed data relating to mentions of Burger King on the social network/microblogging site. When we consider these parts as a whole, insights are produced in one of three areas.

     Overall Twitter Performance:
           Without comparing Burger King with other companies within the sector (which would generate our share of activity for the UK), the company's Twitter activity is shown to be less than purely reactive to media or campaign events. As can be assumed with others within the sector, while some consumer opinions and experiences are stated, most messages are posted mentioning BK as a destination or location. Exceptions to this trend include certain rumors or news items which reasonate with the younger target demographic of the firm. Assuming the rest of the sector performs in the same manner on Twitter, opportunities for general performance increases exist through simple Twitter based campaigns. An audience sporadically tweets about the company and therefore the opportunity does stand to transition these sporadic 'experiential' conversations into a longer, more robust one through promotions ranging from simple (hashtag based contests or promotions) to complex (multi-step campaigns tied into a brand page).

      Geographic and Chronological Performance
            Analyzing mentions of the firm by geographic UK region yielded similar results to the overall distribution for network usage. London reigns large in most geographic analysis of the UK and requires a much more granular analysis to get insights for comparably smaller areas. For Twitter based communications and promotions, this signals that the current trend of London based campaigns should continue specifically for the firm. The prominence of the catchment area in our results (users mentioning the brand outside of a specific radius of a metro. area) could signal the possibility of future possibilities outside of London, but a large amount of activity can be described as commuters or non-specific location coding.
              Chronologically, our hourly data and user analysis of dining mentions (i.e. Breakfast/Lunch/Dinner) showed that lunchtime activity was highest for the brand, both in content and volume. This, by itself, doesn't indicate much, but it might begin to hint at the brand's image as a lunchtime destination for network users.

        User Behaviors
             User and platform data yielded perhaps the most concrete insights of our analysis. Platform data highlighted the fragmented usage context for Twitter, something that is matched by overall network data. Burger King was shown to be mentioned on the go, at a desktop and everywhere inbetween. Data also demonstrated that users weren't likely to mention the brand frequently, another consequence of brand mentions being a product of experiential tweeting. User mention frequency was demonstrated to have little or no effect on when or what a user tweeted about when talking about Burger King, but an overall patten of traction was found for product launches or advertising campaigns.

Implications
        As we can see from the example analysis, a majority of the insights gathered from Twitter search are more topline than detailed. For getting a quick feel for the performance or promenance of a brand on Twitter, such an analysis may prove rather useful, however, further analysis or supporting data is required to produce detailed observations. Network analysis of user segments or a brand page could serve to deepen the insights produced from Twitter.

      Perhaps the most important thing missing from the current analysis is the examination of consumer opinions for sentiment. While we manually did this in our user analysis section, available online automated solutions for such are still in the rudementary phases. By scanning for key words or terms, various websites and programs attempt to classify messages as "Happy/Sad", "Good/Bad". While there is an inherent value in knowing the amounts of good vs. bad messages about a brand, the intricacies of why these messages were classified as such, as well as errors that can stem from semantic differences in wording, are still necessary considerations when thinking about automated analysis. Overall, without utilizing automated sentiment analysis (or doing a lengthy manual analysis), data should be examined from the top down, establishing points of interest or behaviors that warrant more attention. These can serve as starting points to segment users for analysis, cutting the work load involved.

        On the whole, the usefulness of utilizing Twitter search to measure customer sentiment is highly dependent on the company, the sector and the product. Search analysis shouldn't be viewed as the end point of generating consumer insight, but the beginning of seeing where your brand sits within user's minds and the network. From a completed analysis, a company can consider promotions, brand page(s) or adjusting online activities to raise prominence or conversation levels.

Tweet it your way? Twitter's Capacity for Consumer Sentiment Measurement - Part III

     Carrying on from our general analysis of Burger King's UK twitter messaging in Part II, we can move on to specifically examining detailed user data and behavior. General messaging volumes indicated that certain events spiked Twitter activity, but this effect was enhanced by events that resonated with the target market for the brand. In order to fully understand this interaction, we can examine general geographic and behavioral patterns before moving onto specific user behaviors.

     Moving from our general analysis measures, Twitter activity data can be cut by geographic or chronological layers. Analyzing Twitter data by time (as shown below), creates a pattern of usage similar to other social networks or general internet usage. Usage data does diverge from existing patterns around 11am to 1 pm, as usage peaks that would generally increase, peak earlier in the day than with overall UK internet usage. Analyzing messages between 11 and 1, there is a distinct trend of experiential messages involving going to Burger King for lunch or returning from Burger King after lunch.

      Geographically, mention data is limited by the methodology of the search. Geographic searches can be conducted two differing ways: manually through the interface (which allows for searching by mentioning of towns or other locations) or through the API (which limits searching to by geocode and radius). Being that our data was taken by geocode, each area analyzed within the UK was gathered by determining the coordinates for the center of a metropolitan area and then the radius of that body. In order to determine the entirety of the UK, a catchment area was set up encompassing the entire UK, with duplicate messages stripped out later on between all the areas.

     Analyzing the data for Burger King by geography (shown below) we see that the data mirrors the overall distribution of UK Twitter activity pretty closely. London, named the metropolitan hotbed of Twitter activity worldwide, dominates other specific geographic areas. The catchment area proves to be the largest area of activity, due to ambiguous location entries or commuter users being counted in this category. Geographic data doesn't yield as many useful insights in this example as it might in more geographically sensitive examples such as monitoring of political bodies within voter districts or global monitoring of a term by country.

     Analyzing data by platform can help to generate insight on variety of usage (i.e. mobile vs. static), preferred client (i.e. Tweetdeck vs. Twitterrific) or context for messaging (i.e. about something going on simultaneously or later). Previous research has shown that, as a whole, more than half of UK twitter messages are sent from either mobile or hybrid third party clients (meaning less than half of Twitter messages are posted through Twitter.com). Twitter users mentioning Burger King mirror increase on the trend of non-Twitter.com based Twitter usage, as only 32% of mentions came from the "web" platform (which represents site usage). The following four platforms (2 mobile platforms and 2 hybrid (desktop/mobile) options) account for more usage than Twitter.com. The overall fragmentation of usage (170 different platforms register at least one Burger King mention) means that users are talking about the brand through a variety of avenues, both on the go (leading to the possibility of in-store tweeting) and at home. Furthermore, future marketing on Twitter for Burger King, including possible sponsorships, should take into account not only Twitter itself, but this variety of 3rd party clients and platforms.

     Analyzing rate of user mentions, we find that 12.3% mentioned Burger King more than once. The distribution (shown below) indicates that while the overwhelming majority mentions Burger King once (showing that most users don't mention every time they interact with the brand), there are users who have exhibited an ongoing conversation. While all brands want to extend consumer awareness, its essential to mention that some brands won't be successful in generating positive commentary from consumer on Twitter, regardless of their efforts. While people may sporadically mention their detergent on Twitter in passing, spawning widespread and frequent mentions of such may prove nearly impossible, due to the nature of the product. 

      In order to discern what actually drove such high mentions for the brand from certain users, we can specifically analyze the tweet's contents and properties from those users. Comparing users who tweeted more than once and the overall tweet distribution shows that no obvious difference between frequently mentioning users and the overall user base exists.

    While the time series hasn't explained why some users have mentioned the brand more than others, specific analysis of tweet content sheds more light on the situation. First, examining the users who mentioned the brand more than 6 times, showed that the group comprises of both normal users (either conversing about Burger King or joking about it frequently) and functional/brand pages (mentioning specials about surrounding businesses or hosting quizzes for users that may mention the brand). One example of functional users mentioning BK is @Manairport (The Manchester Airport), which tweeted about "2-4-1 Burger King Angus Burgers with a VAT booklet" at the airport. Looking at the high frequency normal users, we can search for product mentions (Chicken Royale comes up a few times) or discern opinions (One user stated that in Worchester, he would travel to Burger King for the burger and then go to McDonald's for the fries - something I might try).

     As we move down the frequency distribution to 2-5 mentions, our analyzed sample size grows greatly and shows an increasing trend towards experiential tweets (43% are estimated to contain terms relating to going to, being at or leaving a Burger King). Analyzing the tweets by word frequency, it becomes evident that mildly moderate mentioning users infrequently compare Burger King with McDonald's (only 7% of this segments messages mention the competitor and 4% mention KFC), preferring instead to mention products (an estimated 46% mention the product either indirectly ("food") or directly ("Whopper")). Scanning the messages manually shows that users have commented on campaigns and products such as the "Angry Whopper" favorably.
 
      When we compare the tweet content from our moderate mentions segment with that of the overall sample,  37% of messages are estimated to contain an experiential term, down from our moderate sample. Product mentions also maintain a low frequency, as overall McDonald's is mentioned in 5.4% of messages and KFC in 4%. Messages mentioning "breakfast" (1.76% overall), "lunch" (4.2%) or "dinner" (1.55%) showed a progression in frequency similar to the hourly activity distribution, peaking midday.

     From this point  in an actual analysis, it would be possible to drill down the data to individual users based on terms used and then continue through their network identifying individual behaviors or opinions. Furthermore, user segment data could be contrasted against activities, such as we did above, to indicate how users with certain predispositions viewed campaign activity or stories. These activities can lead to possible outreach of individual users for advocacy or more detailed information, as well as identifying possible "influence leaders" for further analysis or activities.

     Tomorrow, we'll finish the consumer sentiment series by drawing some conclusions from our aggregated data and insights, as well as identify strong points and short comings of the process as it currently exists and in the future.

Tweet it your way? Twitter's Capacity for Consumer Sentiment Measurement - Part II

     Perhaps the best way to illustrate the possible applications of consumer sentiment measurement on Twitter is to illustrate it with actual data. As an example, I've chosen to use Burger King within the UK, based on its relevance to the UK market and its great examples of news coverage spurring activity. The analysis below is for example purposes and not as an exhaustive analytical case study, so the data will be indexed and used to illustrate points. While it is actual data, the point of this post is to illustrate capacities, not to provide an exhaustive how-to or actual market insight.

What are YOU saying about me on Twitter!?

    As we have already decided on a target for our analysis, the next step is to define what time frame we want to analyze. Twitter limits quick analysis as available search results are limited to 15000 responses or around 7 days. Caching services do exist to obtain data from a historical period older than this, but options such as geographic specification and amount of API calls are limited. In this sense, the best option possible is to start aggregating Tweets at a certain point and continue the process until a desired timeframe or amount is achieved. Aggregation can be manual (literally copying and pasting messages from the search engine results or xml) or automated (recommended - using some simple code to request data from the API, parsing it and storing it). Twitter's search API documentation is the best place to start in your development of software to cache messages. If all you want to see is message volume, various websites exist to tell you how many times a term has been mentioned on the network, these are limited however in the amount of data, terms and timeframe that are available to you. For our example analysis,  I'm using a cache of messages pulled on Burger King over the last few months as my starting point (involving slightly more than 3,500 messages).

     If we take the entirety of our BK data, we can compare peaks in activity to company activity and news coverage. Below, the example shows daily volume of Burger King messages for the UK. If we compare the peaks to amounts of news mentions from somewhere like Google Search insights, we can begin to paint a picture of what issues BK customers talk about on Twitter and where the brand lies in consumer's minds. News or web searches don't always correlate to activity on Twitter, so considering the product, brand and company image is necessary when interpreting this data.

Click for a Larger Version of the Chart

     Taking the graph above for analysis, we can see that the interaction between indexed worldwide Google web searches (the red line), news items(the purple line) and the index of UK Burger King Twitter messages leads to various points of interaction. Points A-E show various days in the time frame in which news and search index volume, Twitter message volume or both increased. By finding explanations for these increases, we can infer insight about the brand, both on Twitter and off.

• Point A: 9/7 and 10/7 - during the stories about Burger King's Spainish ad involving the Hindu god Lakshmi
• Point B: 20/07 - only involved a Twitter traffic increase. Looking at the messages, most are experiential (i.e. I'm going to Burger King), but some involved a web quiz mentioning jobs at Burger King and others involved various conversations about the brand (including some about the previous events around point A)
  
• Point C: 6/8 - involved the largest search and news spike in the period, as well as a smaller Tweet volume increase. Stories involving Burger King during this time were the downgrading of BK stock by JP Morgan and a mother being kicked out of a Missouri Burger King due to her baby not wearing shoes, both of which sparingly show up in Tweet content.
  
• Point D: 25/8 - involved the increased performance of BK profits for the quarter. Upticks in news content and twitter messages involved this story in various reposts, while simultaneously not driving any new search activity.
  
• Point E: Increasingly through the 4/5/6 of September and involved the discussion of rumors about "New Moon" (the Twilight sequel which I had to look up) promotional materials being distributed at Burger King. Investigating rumors drove search and tweet volume, but not actual news items.

     Taking analysis of these points and others together, we can begin to see what makes Twitter users talk about Burger King and what doesn't. Generally, we can see that re-tweeted information about stocks and other events will create a loose increasing relationship between news and tweet volume, but that the gains from this can be overshadowed simply by a large day of activity involving experiential tweets.

    More interestingly however, we can see that content more relevant to the target age grouping of BK (such as New moon promotional materials or Twitter based web quizes) can greatly increase message volume. This may muddle our insights in a way, as their exists no direct causality between certain events and message volume, but so far, a loose portrait of where the brand stands on Twitter can be generated (jokes about passing out Vampire movie crowns while creating controversal Hindu ads aside).

     In our next part, we'll utilize data specifically from September and attempt to create deeper user and geographic insights about Burger King's Twitter presence, as well comparing overall user statistics with that of the sector.