If your online community is successful, what will “success” look like in terms of people, interactions, activities, and outcomes? How will you measure success? Often success is defined in terms of effectiveness and health (Bourhis & Dubé, 2010). The effectiveness of a community is the extent to which it has met its objectives and provided value and benefits to its members. The health of the community is typically determined by member satisfaction, level of activity, and quality of interactions. Although there is a range of evaluation techniques that can be used to monitor the effectiveness and health of an online community, it is important to recognize that the purpose and the orientation of the community should drive the evaluation strategies used to examine its effectiveness.
Planning and Implementation Guidelines
During the planning phase, community leaders and stakeholders may want to conduct ongoing needs assessments through informal and formal interviews, online surveys, and focus groups with experts and practitioners from the targeted domain of the community. After implementation, community stewards could use site analytics along with online surveys and focus groups to continually evaluate their members’ activities, interactions, and needs. The following questions are guidelines for leaders and stakeholders to use while considering evaluation approaches:
- What are the ongoing practices and processes that contribute to the “life” of the community and that keep members engaged?
- How is knowledge being shared within the community? Beyond the community?
- Are leaders or roles emerging in the community? In what ways? How are they being cultivated?
- How are members being supported in the community?
- How are members contributing? Posting? Replying? (When? How often?)
- What are the prevalent patterns of members’ interactions?
- How much of members’ online time is spent connecting to others in the community? (e.g., reading or posting in forums, attending webinars)
- Are groups being formed? How active are they? What are they contributing to the community as a whole?
- How is design of the community affecting member use and participation? What are members’ technical issues?
- What are the emerging benefits of the community for members?
- What are the emerging cultural norms or themes of the community?
- What are members’ FAQs?
More about community usability can be found in the Technologies for Online Communities of Practice brief, available on the Connected Online Communities of Practice website at edcocp.org.