For more than 10 years, the National Science Digital Library (NSDL, officially the National Science, Technology, Engineering, and Mathematics Education Digital Library) has been active in the aggregation, contextualization, and dissemination of open digital learning content generated through grants from the National Science Foundation and other federal agencies as well as nonprofits such as museums, research laboratories, and professional societies. As with many new technologies, NSDL’s early efforts as a digital library research and development project emphasized the technology as an end product. As we moved into production, we quickly transitioned to a focus on the content and how best to curate resources for effective educational use and reuse. Along the way, as we iterated improvements on both the technology and the content, we built the most valuable contribution of NSDL to the science, technology, engineering, and mathematics (STEM) education community—the cross-sector network of education, research, technology, and policy partners and the culture of collaboration that NSDL has carefully cultivated. It is through the shared knowledge of this particular connected community that the new concept of learning resource paradata has emerged as a mechanism to make learning networks even smarter.
From its inception, NSDL was tasked to demonstrate the impact of digital libraries on teacher practice or, if at all possible, on student outcomes. This proved to be challenging in the highly heterogeneous, highly distributed, and relatively anonymous usage environment of the open Web. Outside of a few controlled research studies, online resource providers lacked the proper feedback mechanisms to examine the texture of usage at multiple scales. Evaluation methodologies borrowed from conventional library practices or those used to assess the efficacy of formally adopted curricular materials did not effectively represent the authentic use of open learning content by teachers on the ground. Traditional Web metrics also left us wanting: What does it mean that 100,000 users came to our home page? What does it tell us when an individual user clicks on a resource? Was it useful? Did he or she incorporate it into a lesson? Did he or she adapt it for a different context? How easy was it to implement? What do we need to improve about that resource?
We needed to reconceptualize our notions of impact to match the changing realities of teacher practice. In response, we have begun to focus on the online educator communities where new practices are nucleating and where teachers are rapidly expanding their consumption and production of expert-generated, peer-generated, and self-authored content.
The rapid speciation of online communities of practice allows for authentic examination of digital resources as they are being discovered, created, and used by educators. As teachers increasingly interact around digital content in complex collaboration environments online, we have an opportunity to surface this collective knowledge production about what is working and what needs are going unmet with regard to the current generation of digital learning resources—and to meaningfully inform the requirements setting for the next generation of learning tools. Most important, we can now gather that intelligence as it emerges across diverse platforms, aggregate and interpret it, and feed it back into the workflows of user communities as a new layer of data to inform their decision making around digital content.
NSDL launched the STEM Exchange as a means to mobilize content into the hands of educators through online communities and to establish return feedback loops of data created by the activities of communities around that content—a type of data we have defined as paradata, adapting the term from its application in the social sciences. The concept of learning resource paradata is grounded in the premise that teachers trust teachers to understand and inform what might be most useful to a particular pedagogical context. Paradata provides a mechanism to openly exchange data about how resources are discovered, assessed for utility, and integrated into the processes of designing learning experiences. Each of the individual and collective actions of favoriting, foldering, rating, sharing, remixing, embedding, and embellishing that are the hallmarks of today’s teacher workflow around digital content are points of paradata that can serve as indicators about resource utility and emerging teacher practices. This is an idea best served at scale and will realize its potential only by connecting online communities with each other and allowing emergent knowledge from real-world applications to continually accrete as shared content diffuses across networked platforms.
To accelerate that diffusion, and our collective understanding of it, the NSDL STEM Exchange is sharing its underlying concepts and ideas with the U.S. Department of Education and the U.S. Department of Defense to seed the development of the Learning Registry, a new initiative that is building a common technical infrastructure for exchanging descriptive information and usage data about learning resources from federal agencies, nonprofits, and commercial providers alike. Like other smart systems, the Learning Registry’s capacity for meaningful pattern identification will be enhanced as it scales up. The more connected communities that utilize the Learning Registry to discover and share content, and then give back anonymized and aggregated paradata, the greater our collective intelligence will be—and the more our actual needs and demands will drive what digital content becomes. We are no longer constrained to narrowly sourced, one size fits all, or a 10-year adoption cycle.
As we gain the ability to know, not just that a resource exists, but that …
Teachers at the high school level integrated Resources X and Y into a unit tagged as Force & Motion on the American Association of Physics Teachers website.
189 middle-level teachers registered with PBS Learning Media favorited Animation Z in the month of September.
Master teachers in the CTE Online community aligned Podcast A to California State Standard B and Mathematics Common Core standard C.
One Better Lesson community member in the Flipped Classroom group created Slideshow D and assigned it Creative Commons license CC BY-SA 3.0.
75 Florida teachers at the CPALMS portal recommend Educational Gaming Site E for use with English language learners.
… we may gain insights into the nature of online communities themselves and the role of online collaborations in redefining the teaching profession for the digital age.
The Learning Registry is launched this week in beta version. It has already gathered a vibrant open community of collaborators beyond NSDL, including content providers, learning communities, and technical tool developers. I encourage you to recognize these early adopters and join forces with us. Learn how your knowledge network or online community can participate at the Learning Registry website, and let us know what you think as paradata begins appearing in your own online workflows.
Susan Van Gundy is the director of Education and Strategic Partnerships
for the National Science Digital Library, and project lead for the STEM
Exchange (http://nsdlnetwork.org/stemexchange).
This initiative sounds really compelling. Clearly federal agencies have learned some hard lessons about the need for streamlined data-sharing, and it’s great that through this registry, the efforts to achieve these goals within the government will also benefit educators. As Susan points out, this could be great for disseminating innovative teaching resources and methods. A question for Susan: is it expected that the Learning Registry will be used by the private sector as well to see what resources educators find useful, and then to tailor products accordingly?