Empowered Superintendent Blog Series: Measuring the Impact of Technology Initiatives

Using technology in meaningful ways is as essential to today’s teaching and learning as reading, writing, and arithmetic. Stakeholders from around Baltimore County understood this urgency and prioritized graduating every student globally competitive in our Blueprint 2.0 strategic plan. Our theory of action leverages technology in two ways—through equitable access for every student to an effective digital learning environment and second language proficiency. The bold initiatives that are transforming our schools are S.T.A.T., or Students and Teachers Accessing Tomorrow, and the Passport Program for elementary world language instruction.

S.T.A.T. is creating new opportunities for personalization, customization, and engagement guided by digital curriculum more focused on critical thinking and analytical skills. Additional components include the BCPS One digital ecosystem for easy educator and family access to student information and learning resources, 1:1 devices, and wireless and broadband at public schools and libraries. School-embedded instructional coaches known as S.T.A.T. teachers tailor professional development to school needs to facilitate learner-centered environments where students actively choose how to meet objectives and how to demonstrate what they know and can do.

Digital learning is also essential to Passport, which supplements weekly conversational Spanish lessons with an interactive, online program designed for children. Through BCPS One, students have anytime, anywhere access to the program for practicing speaking and listening.

Ongoing investments and support from the community known as Team BCPS are critical to bringing these changes in instruction and infrastructure to nearly 112,000 students. To provide this support, our stakeholders—from families to staff, residents, and decision makers—need to know how these programs are contributing to the future of our students, schools, and county.

For transparency, we opted for independent and objective advice from external evaluators for both initiatives. What was crucial was determining, based on research, how we expected the shifts in teaching and learning associated with S.T.A.T. and Passport to improve student outcomes.

The S.T.A.T. evaluation is framed by a logic model that sets forth inputs, measurable outcomes for years one and two, and goals for years three and four. As shown in the S.T.A.T. evaluation model below, we expect intensive professional development that is tailored to administrators, S.T.A.T. teachers, and classroom teachers to produce changes in five areas: the classroom environment, teacher practice, use of digital content, student engagement, and P21, or 21st century skills. Later effects include student achievement and ultimately, gradation.

S.T.A.T. Evaluation Model

Similarly, the Passport evaluation model below captures the journey from planning and logistics to measurable outcomes for year one and beyond in the areas of program implementation, instruction, teacher perceptions, student engagement, and student satisfaction. These outcomes, in turn, are expected to lead to improvements in language proficiency and readiness for Spanish II in year two.

Passport Evaluation Model

Both evaluation models outline what we expect to see and when, creating the space to get new initiatives into place, and holding us accountable at appropriate times for impacts on the learning environment, staff, and students. They guide us through the change and implementation process so that we can communicate clearly about expectations and timeframes.

Measuring the impacts of our major initiatives in this way is the result of thoughtful, ongoing conversations among our staff with expertise in technology, research, and academics. This team approach builds on the collaboration CoSN recommends for designing and initiating digital transformation. Further, the independent evaluation reports provide the publicly available information needed to inform continuous improvement, identify annual priorities, and justify the resources and supports needed to sustain and grow S.T.A.T. and Passport.

Learn more about CoSN's Empowered Superintendent initiative and download the toolkit.

S. Dallas Dance is the Superintendent of Baltimore County Public Schools. Dr. Dance, whose tenure began in 2012, is responsible for overseeing the instruction of about 111,000 students and leading and managing a $1.76 billion budget, more than 21,000 employees, and 175 schools, centers, and programs.