Yesterday afternoon at CSL there was a great discussion around engagement, metrics, value, and tools. The session was actually a merger of three separately proposed sessions that had mutually compatible/overlapping contexts. Probably 40-50 people in the room.
For me one of the best takeaways was the idea of looking at engagement as more or less a marketing funnel with metrics around the different levels and activities to move people between the different levels. As with most things you want to measure, you should know what you are trying to achieve. If you don’t you won’t get the value out of the metrics. Metrics drive behavior so you run the risk of driving the wrong behavior (e.g., gamification) if you measure the wrong thing. Of course, if you can use this to your advantage if you know what you are trying to achieve.
The discussion around on-ramps was interesting. Measuring time to first contribution is instructive as it talks about how inviting the community and technology is, how well it is documented, how easy it actually is, … Relating this back to the funnel, it talks about moving people from level 1 (early level) to level 2. Measuring how many people move, how long it takes and defining the activities that we can do to make it easier/faster/more likely for folks to transition.
A number of tools for gathering insights were suggested
- Google analytics
- Metrics inherent in the mechanism/system being used (e.g., github metrics, forum metrics, …)
- Support interaction measurements (number of questions, time to first response, number of interations, time to resolution, number solved, …)
- Adobe Analytics
** Feedback loop to the people answering (let them know that they are helping)
- Stack overflow/stack exchange
- New relic insights
- MetricsGrimoire – http://metricsgrimoire.github.io/ measure oss involvement
- Discourse for forums gives good metrics, gamification
- Sentiment analysis
General sentiment in the room was that this is by no means a solved problem and most people/communities are still trying to figure out how to measure, manage, value engagement. There is lots of different kinds of data, lots of actual data, communities are very different with different goals, … http://communitybuilding.stackexchange.com/ is a forum for community building.