A community of technology community managers, leaders, and builders.

Topic: Analytics & metrics for communities and reporting for The Powers That Be


#1

Topic: Analytics & metrics for communities and reporting for The Powers That Be
Propose: Bruce Cronquist
Scheduled: Friday 3pm, space 7
Notes:

Notes from the introductions:
• We define “Community” as both online AND in person
• Many people are from the open source online community
• The-Powers-That-Be may be yourself if you are managing the community, or it could be people “below” you in the organization that are helping you to support your community, or it could be one or more levels above you (middle manager, supervisor, community manager, upper management)

Questions we are trying to answer (came up during Introductions)
• One person would like to offload support costs. Right now they have a ticket-base product support. They would like to figure out the best way(s) to offload tickets to the community or create a public community of Q&A. They would then like to measure the deflection of tickets.
• One person would like Prove the community is necessary and helpful. They would like to get more buy-in from management. They consider their role as “Community messenger.”
• We know that it is hard to measure long term payback. It is not immediately measurable. One person told a story about how they started a community, and three years later one of the participants on the community hosted and ran a great event.
• One person shared that for her company, the more people that are participating in the communities, means need more budget for SWAG, etc
• Success could be people successfully run an event (kitchen table to build an app) (500 persons conference)
• How to prove you are successful if your manager doesn’t agree with what you are doing?
• How do you connect your work to large management goals

Data we can or want to track discussion
• There are always stories, anecdotal evidence that cannot be measured. Including quotes from members of the community in your reports, in addition to numbers, is a best practice. Story telling.
• Trending
If you see a spike, you want to know WHY it happened. How put 2 and 2 together?
• Filter data - remove manages, focus on newer people? This is introducing bias to your data, but that’s OK if you are doing it intentionally.
• HOW DO YOU: You may have to let things go and see what happens in the community. For example, someone makes a majorly negative posting. Let it be and see how the rest of the community responds. They might come in and support you, they might support the poster, and so on. If management is concerned about a negative comment, put it in perspective. Comment could be from 0.01% of the customers How important is the data?
○ Transparency and trust is very important: if you lose them, you lose the community. In general, the “results” cannot be controlled, but you can work with the community to sort the problems out. In fact, the role of the community leader is also to tell the management what they are doing wrong, because the community is perceiving it that way.
○ Story: One person shared that HOW a company deals with a community that doesn’t agree with a company can be a positive OR a negative for that company. When talking with management, he includes two case studies from current events. One shows an external community that went up against a corporation, caused conflict, lawsuits were filed…the company lost credibility and brand happiness. Another example of the same conflict, but where it became a net positive because the company defused the situation by engaging the community. It helped the company and showed usefulness of communities.
• HOW DO YOU: Help prepare newbies in the community?
○ Auto-replier with guidelines, how to ask questions of moderator, who you can email with questions, and so on.
○ Direct folks to a small group (local in the area, this working group on that topic) Direct them to a small subset of the humongous group where they may get lost in or feel too scared to participate.
○ Everyone gets a mentor (requires scale)
§ Mentors of mentor once you have mentored a newbie.
○ Live welcome via some ace-to-face technology such as Lync or GoToMeeting

Things to keep in mind when reporting:
• Many people in management raised up in a world where companies could control the message. They no longer can, they have to realize that.
• Monthly reports - see thread on the forum
• Be sure to include Quotes in addition to numbers and trends
• Provide fewer numbers the higher level up you go in the reporting structure. Since upper managers have to focus on more things, provide them only the few numbers that land your message
• Be sure to report Qualitative and Quantitative
○ For qualitative content, try doing a Success Case Method
• How survey numbers have changed after you made a change in the community
• Number of people conversing is a statistic. But remember, it’s not just about body count. But for management, it is!
• Administrators want to know about what has happen, you have to balance that with the potential, that is a different story.
• Measuring when the community is answering questions for you is a good metric of success.
• Management may be afraid of things the community could do. For example, criticizing the company. You have to confident that is not going to happen, that you can make something better by leveraging on the community. In the end, is how can we engage with this problem, that the community does something against the company, and deal with it.
• Include numbers. What’s changing. Charts, highlighting relationships between charts and actual events. Some tools may help here. For example, Mailchimp, when considering email events.
• It’s important to track the different kinds of engagement.

Example Reports: See this forum posting What does your Community Report look like?

If you could track or report on one thing about your community, what would it be?
• User engagement - what they do. Evidence-based
• Who is in the community on an individual bases and the project(s) they are working n
• How and Why does the community grow? Is it due to a great post, content, forwarded message, etc
• Why do you like to engage in the community, and come back? Tools? Where the community headed?
• Who is in the community: demographics
• Impact of the community. What can we do now that we couldn’t do earlier?
• Who are the contributors, and how can I accelerate their contributions
• Subject matter trends - monthly main topics, how it changes over time
• Evaluations of the community as if it was an acquisition
• Seeing the spikes in the community. Was it based on the meetings being held two months ago?
• Effectiveness of knowledge sharing - how have the communities helped change people knowledge sharing
• As many people on the paid team, answering on the meetings.
• Bhutan model - gross domestic happiness.
• Why do people stop engaging with the community
• Changing roles in the community: listeners, contributors,…how move and why


#2

So there you go, notes from the session. Thanks to everyone who participated, and thanks to Jesus for helping take notes.

If you have some possible solutions to the “If you could track…” items above, please Reply to this topic with your ideas. Brainstorms welcomed.


#3

One easy way to measure this is to take all of the postings in the community and do a Word Map or Word Cloud. It will give you a visual of the terms that are used the most. You can refine the output to remove certain words such as “the” or terms you do not care about. Here is one web site that will create a word map for you: http://www.wordle.net/.


Thanks to for being the kind sponsor for this forum!