Comprehensive assessment solutions: Dynamic tools to evaluate learning

Assessing student learning with comprehensive assessment has always been a crucial aspect of education. It provides valuable insights into students' knowledge, skills, and progress, which helps instructors tailor their teaching approaches and improve instructional strategies. In recent years, the concept of comprehensive assessment has gained significant recognition as a more holistic and effective approach to evaluating student learning.

To gain insights into the power of this type of assessment and its impact on students and educators alike, the Turnitin team sat down with seasoned educator and Senior Director of Customer Engagement, Patti West-Smith. She shared her expertise on the significance of assessment, the role of technology, and the benefits of comprehensive assessment tools in fostering academic integrity and higher-order thinking skills.

What is comprehensive assessment?

Patti West-Smith: I think the concept of comprehensive assessment is this idea that you have to create an overall picture of assessment. There is no one way to assess students if you really want to assess the large scope of mastery, right? So, comprehensive assessment is an approach that encompasses multiple facets of assessment to create a more complete and accurate picture of student learning. It goes beyond traditional methods and takes into account various forms of assessment, such as formative and summative assessments, authentic assessments, performance tasks, simulations, and higher-order thinking activities.

Comprehensive assessment acknowledges that there is no single method that can capture the full range of students' mastery in a particular subject or discipline. It aims to assess students in multiple ways to obtain a comprehensive understanding of what they know and can do. It goes beyond relying on a single assessment or evaluation to provide a more accurate and fair representation of students' knowledge and skills. By incorporating various assessment approaches, comprehensive assessment aims to create a complete picture of student mastery.

What are the benefits of a comprehensive approach to assessment?

Patti West-Smith: There are different ways to approach the benefits of comprehensive assessment: there are benefits for students, there are benefits for instructors, and then there are benefits for the program itself. So, I think it's important to talk about the benefits in terms of those three categories.

Benefits of comprehensive assessment to students

Patti West-Smith: Comprehensive assessment provides a more fair and unbiased method of assessing student learning. By utilizing multiple assessment methods, it reduces the impact of situational factors and unconscious bias, promoting a more equitable evaluation process. It also offers a more accurate representation of what students have learned, allowing them to demonstrate their knowledge and skills through various assessment opportunities. Comprehensive assessment supports student growth and development by enabling timely interventions and feedback.

Benefits of comprehensive assessment to instructors

Patti West-Smith: For educators, comprehensive assessment offers an accurate measure of individual student learning outcomes. When I was an administrator, I used to tell my teachers there are three questions you have to ask yourself every single day: “Who got it? Who didn’t? And what are you going to do about it tomorrow?” Comprehensive assessment provides valuable insights into the effectiveness of educators’ instructional approaches, curriculum, and course materials. By identifying areas of strength and weakness, instructors can tailor their teaching strategies to meet individual student needs and improve overall program effectiveness. The feedback loop created by a comprehensive assessment paradigm supports instructional decision-making and continuous improvement.

Benefits of comprehensive assessment to programs of study

Patti West-Smith: Comprehensive assessment ensures that programs of study have a reliable measure of student learning. If you have a nursing program, for example, you want an accurate measure of the learning activities, the curriculum, the course materials, and the instructional methodology. And it's critically important because the stakes are really high for some of these cases; we're talking about licensure that affects people's careers and long term benefits. Assessment data helps validate the effectiveness of the curriculum, course materials, and instructional methodologies while also providing insights that allow for strategic decisions about resource allocation, professional learning needs, and even recruitment needs. By identifying areas where students may struggle or excel, programs can make data-driven decisions to enhance educational offerings and align with desired learning outcomes. With ExamSoft, for example, there is a tool for category tagging to accreditation standards and other learning objectives, which can help a program immensely.

What is the role of technology in changing the comprehensive assessment landscape?

Patti West-Smith: Technology has enabled us to stay true to best practice. For example, Dr. John Hattie’s research shows that personalized, actionable feedback measurably makes an impact on learning. But it’s incredibly challenging because it takes time, effort, all things that educators don’t have.

Over the course of my career, there have been more and more technology applications that actually simplify this work for us. There are assessment platforms that make it easier to deliver traditional forms of assessment. And that saves people time. It makes it more efficient. It makes it more scalable, more sustainable. We also now have AI-assisted feedback and more diverse assessment methods, enabling us to offer accurate assessments and meaningful feedback to larger groups of students in a way that isn’ t labor intensive or logistically challenging. All three of these aspects influence the assessment landscape:

Assessment platforms

Technology-enabled assessment platforms streamline the delivery, grading, and analysis of assessments. They offer features such as automated grading, data analytics, and instant feedback, making assessment more efficient and scalable.

AI-assisted feedback

Artificial Intelligence (AI) can help predict appropriate feedback and scale it across classes large and small. In Gradescope, for example, AI-assisted grading optimizes instructor's workflows by grouping questions and like answers together to dramatically streamline the process and scale actionable feedback across a whole set of exams.

Diverse assessment methods

Technology allows educators to incorporate a wide range of assessment methods, including multimedia projects, online simulations, virtual labs, and interactive quizzes. And with tools like Gradescope, instructors can even digitize their paper assessments. These methods engage students in active learning and provide a more authentic assessment experience.

What's important when developing a comprehensive assessment framework to ensure integrity?

Patti West-Smith: You can't accurately measure what students know, understand, and do, if you have not ensured academic integrity. Because if you don't know if the assessment is fair, nor if it's actually a student’s work, you don't know if you're measuring their learning. When I talk about assessment and integrity, I like to think of it as two sides of a coin: one is the integrity of students in the assessment process, and one is the integrity of the assessment itself.

Technology has enabled us to stay true to best practice, which was once impossible to do at scale. Now, with education technology, we have tools that empower us to do what we know is the best thing to do. Like with ExamSoft, it’s easy to create and administer exams because there is flexibility in delivery (format, location) and offline delivery removes the risk of internet disruptions during testing time. And with complete device lockdown, remote proctoring features, and the ability to randomize questions, instructors using ExamSoft are more confident in the integrity of their exams.

The integrity of an exam is essential in any program, but take a nursing program or a law program: we cannot send nurses and lawyers out into the world, not knowing that they have the appropriate level of knowledge and understanding. In the big picture, it is making sure that people are prepared for whatever application of this knowledge that we expect them to have. And it's going to look different from discipline to discipline and scenario to scenario.

When developing a comprehensive assessment framework to ensure integrity, there are several important factors to consider:

• Multiple assessment methods: It is essential to include both formative and summative assessments. Formative assessments provide ongoing feedback to students and instructors, allowing for adjustments in teaching and learning strategies. Summative assessments, on the other hand, measure student achievement at the end of a unit or course. By utilizing a combination of formative and summative assessments, a more comprehensive understanding of student learning can be achieved.
• Balanced coverage: A comprehensive assessment should cover various aspects of learning, including foundational knowledge and higher-order thinking skills. It is important to assess not only content knowledge but also the ability to apply knowledge in real-world contexts. This can be achieved through authentic assessments, such as performance tasks, projects, or simulations like Objective Structured Clinical Examinations (OSCE). Educators can use ExamSoft to evaluate OSCEs and other performance tasks with customizable rubrics.
• Independence and individual assessment: I think a comprehensive assessment has to allow us as educators to assess at the individual independent level. Within instruction, we might say, “The students did a group project,” or “We had a class discussion,” and those are incredibly important pieces of an assessment picture. But we cannot only do that. Comprehensive assessments should provide opportunities for students to demonstrate their understanding and skills independently. While collaborative activities and discussions are valuable, it is crucial to have individual assessments to identify each student's level of mastery in order to personalize instruction and coordinate intervention when necessary.
• Fairness and accuracy: Comprehensive assessment methods strive to be fair and unbiased. They take into account the diverse backgrounds and circumstances of students, minimizing the influence of external factors on assessment outcomes. By using multiple assessment methods and gathering data from different sources, a more accurate and reliable assessment of student learning can be obtained. There’s also an element of fairness when it comes to grading, which means the anonymous grading feature in Gradescope and the autograding in ExamSoft both ensure that tests are being gradely solely on their content.
• Continuous improvement: Comprehensive assessments offer valuable insights for instructors and programs of study. I really appreciate that in ExamSoft there are reports that summarize how a student performed on an exam, viewable by student and instructor. Instructors and programs can view longitudinal analysis reports that show student outcomes at the individual, course and institutional level, disaggregated by categories (such as accreditation standards or learning objectives) over a select period of time. By analyzing assessment results like these, educators can identify areas of strength and areas that require improvement, leading to continuous enhancement of the teaching and learning process. Gradescope also offers item analysis to inform exam design, which among other things, provides instructors insight into exam quality and therefore accurate measurement of learning. There’s no educator that doesn’t want to know that students are learning and growing.

What are some tips for educators who want to integrate technology into the comprehensive assessment process?

Patti West-Smith: It’s really important that expectations should be clearly communicated to students in terms of what is and is not acceptable. I always say to instructors that the number one mistake that instructors make around academic integrity is assuming that they have the same understanding of it that their students do. It is almost never true. And if you leave a gap, students will create that understanding for themselves—and you can almost guarantee that it will not match up to your framework—so don't make the assumption and make sure that you are communicating it clearly.

From there, educators can integrate technology into the assessment process in various ways, depending on their goals and available resources. Some effective approaches I’ve seen include:

1. Online comprehensive assessment tools, like Gradescope or ExamSoft. These online assessment platforms offer features like automated grading, data tracking, and instant feedback. These tools streamline the assessment process and provide valuable insights into student performance.

1. Digital portfolios, which encourage students to create digital portfolios showcasing their work, projects, and reflections. Digital portfolios allow for ongoing assessment and enable students to demonstrate their growth over time.

1. Collaborative online projects, where students engage in online projects with other students that require research, critical thinking, and creative problem-solving.

In the end, an educator or administrator may ask themselves this question: “How can I integrate technology in a thoughtful way to give myself more time back? Is there a way to automate a process that is still formative and meaningful for the students, but isn't asking me to stay up until midnight every single night, writing in the margins of papers?” I also encourage educators to remember that a career is not an event. You cannot do it overnight. You can't judge yourself and your practice based on a single moment in time, any more than you can judge your students on a single assessment. And you don’t have to do everything all at once; instead, pick an area where it will make a big impact, and dive deeply into that.

In conclusion: Comprehensive assessment solutions to evaluate learning

As our interview with West-Smith highlights, comprehensive assessment solutions have revolutionized the way we measure students' understanding and progress. By leveraging technology and embracing instructional strategies that promote academic integrity and higher-order thinking, educators can create a holistic assessment environment that benefits both students and instructors. These dynamic tools not only provide valuable insights into students' learning to inform exam design and program success but also pave the way for their future careers, equipping them with the skills and knowledge they need to thrive in an increasingly competitive world.

With fast and fair assessment, as well as instructor and program confidence in the integrity and security of an exam, comprehensive assessment continues to shape the educational landscape, enabling educators to adapt their instructional practices and empower students on their journey of lifelong learning.