A few months ago, I wrote a blog titled How Online Communities of Practice Shape Our Work. Today I have been thinking a lot about how we evaluate these communities and why we choose to value certain measurements. The most obvious answer is that we have goals. And, while I completely agree that a community needs to have a goal or goals and needs to be prepared to measure steps towards meeting those goals, I also think that every community needs to take the time to examine what we are measuring when we evaluate a community’s success.
Community managers often cite statistics gathered by analytics software about how many people clicked on a certain post, how many times a statement was re-tweeted, or how many users visited the site. These are all key pieces of information in understanding how an online community is working.
But for most education communities of practice there are bigger questions that move us toward thinking about how engagement in a community impacts the way we approach our work, both logistically and intellectually. Wenger, Trayner, and de Laat (2011) created an evaluation framework for communities of practice that suggests ways we can evaluate how a community supports growth in ember work habits and understanding. Others in the field like FeverBee’s Richard Millington offer ideas about how to measure return on investment with an eye to changing the behavior of members.
To evaluate a community by logistics alone or even to include a few simple survey questions does not reveal whether and to what extent that community is promoting knowledge creation and knowledge exchange among members or changing members’ behaviors. And for success in the types of communities West Wind supports members must move beyond clicking on a blog to read something or tweeting a relevant article; members must grow together and share in creating ideas and solving problems.
Having now supported and managed several communities of practice with different goals and members who engage in various ways online, there are several key questions adapted from Wenger, Trayner, and de Laat’s (2011) framework that I always include when thinking about evaluating a community of practice I manage or one that I have joined:
1) What is the value within the activities and interactions themselves?
2) Does the community result in a creation of knowledge capital?
3) Are members able to leverage that capital?
4) What is the impact of knowledge on stakeholder goals if any?
5) How does involvement in the CoP and with the knowledge created cause members to reframe and reconsider their actions or work? And how should managers adapt the CoP to meet changing needs?
References:
Wenger, E., Trayner, B., and de Laat, M. (2011). Promoting and assessing value creation in communities and networks: a conceptual framework. Rapport 18, Ruud de Moor Centrum, Open University of the Netherlands. Retrieved from http://wenger-trayner.com/documents/Wenger_Trayner_DeLaat_Value_creation.pdf.
Millington, R. (2013). How to build an online community: the ultimate list of resources. Retrieved from http://www.feverbee.com/2013/02/how-to-build-an-online-community.html.