Learning Technology Guide - DEV
About
Rubric for Learning Technology Evaluation
This rubric has been designed to evaluate online tools that may be used to support learning in higher education. The rubric supports a multi-dimensional evaluation of functional, technical, and social aspects of a resource/tool for supporting learning regardless of delivery method.
Rubric by Section
Functionality
| Category | Sub-category | Works Well (★★★ to ★★☆) | Minor Concerns (★★☆ to ★☆☆) | Serious Concerns (★☆☆ to ☆☆☆) |
|---|---|---|---|---|
| Functionality | Scale | The tool can be scaled to accommodate any size class with the flexibility to create smaller sub-groups or communities of practice. | The tool can be scaled to accommodate any size class but lacks flexibility to create smaller sub-groups or communities of practice. | The tool is restrictive to a limited number of users and cannot be scaled. |
| Ease of Use | The tool has a user-friendly interface that is easy to navigate and personalize for both instructor and learner. Use is intuitive for both instructor and learner. | The tool has an interface that may be confusing to either instructor or learner; there is limited opportunity for personalization. | The interface is not user-friendly for either the instructor or learner; navigation is clunky, personalization is not an option. Unintuitive to use. | |
| Tech Support/Help Availability (for Instructors) | Campus-based technical support and /or help documentation is readily available and aids users in troubleshooting tasks or solving problems experienced . | Technical support and help documentation is available but limited, incomplete, or not user-friendly. | Technical support and help documentation is not available. | |
| Tech Support/Help Availability (for Students) | Campus-based technical support and /or help documentation is readily available and aids users in troubleshooting tasks or solving problems experienced. | Technical support and help documentation is available but limited, incomplete, or not user-friendly. | Technical support and help documentation is not available. | |
| Hypermediality | The tool allows users to communicate through different channels (audio, visual, textual) and allows for non-sequential, flexible/adaptive engagement with material . | The tool allows users to communicate through different channels (audio, visual, textual) but is limited in its ability to provide non-sequential, flexible/adaptive engagement with material. | The tool is restrictive in terms of the communication channels employed (audio, visual, textual) and presents information sequentially in a rigid, inflexible format. |
Accessibility
| Category | Sub-category | Works Well (★★★ to ★★☆) | Minor Concerns (★★☆ to ★☆☆) | Serious Concerns (★☆☆ to ☆☆☆) |
|---|---|---|---|---|
| Accessibility | Accessibility Standards | The tool meets accessibility guidelines (i.e. W3C WCAG 2.0 standards) | The tool has some limited capacity to meet accessibility guidelines (i.e. W3C WCAG 2.0 standards) | The tool fails to meet accessibility guidelines (i.e. W3C WCAG 2.0 standards) or no information of compliance has been made available for the tool. |
| User-focused participation | The tool is designed to address the needs of diverse users, their various literacies, and capabilities, thereby widening opportunities for participation in learning. | The tool has some limited capacity to address the needs of diverse users, their various literacies, and capabilities . | The tool is restrictive in meeting the diversity of needs reflective in the student body. The tool likely restricts some learners from fully participating. | |
| Required Equipment | Proper use of the tool does not require specialized equipment. | Proper use of the tool requires equipment that is common and/or purchased at a low cost (microphone, speakers, cell phone, etc.) | Proper use of the tool requires specialized equipment requiring moderate to significant financial investment |
Technical
| Category | Sub-category | Works Well (★★★ to ★★☆) | Minor Concerns (★★☆ to ★☆☆) | Serious Concerns (★☆☆ to ☆☆☆) |
|---|---|---|---|---|
| Technical | Integration/Embedding within a Learning Management System (LMS) | The tool can be embedded (as an object via HTML code) or fully integrated (e.g. LTI-compliant tools) into an LMS while maintaining full functionality of the tool. | The tool can be embedded within an LMS but with limited functionality. | The tool can only be accessed in an LMS through a hyperlink or static representations of the tool (e.g file export), rather than a functional version of the tool itself. |
| Operating Systems | Users can effectively utilize the tool with any standard, up-to-date operating system. | Users may encounter limited or altered functionality depending on the up-to-date operating system being used. | Users are limited to using the tool with one specific, up-to-date operating system. | |
| Web Browser | Users can effectively utilize the tool with any standard, up-to-date web browser. | Users may encounter limited or altered functionality depending on the up-to-date browser being used. | Users are limited to using the tool through one specific browser. | |
| Additional Technical Requirements | Users do not need to download additional software or browser extensions. Additional accounts or software is not required. | The tool uses a browser extension or software that requires a download and / or user permission to run. An additional account or common software may be required. | The tool requires a past version of a browser extension or software. Additional account and/or uncommon software is required. |
Mobile Design
| Category | Sub-category | Works Well (★★★ to ★★☆) | Minor Concerns (★★☆ to ★☆☆) | Serious Concerns (★☆☆ to ☆☆☆) |
|---|---|---|---|---|
| Mobile Design | Access | The tool can be accessed, either through the download of an app or via a mobile browser, regardless of the mobile operating system and device. | The tool offers an app, but only for a limited set of mobile operating systems. Tool is not accessible through a mobile browser. | Access to the tool is limited or absent on a mobile device. |
| Functionality | There is little to no functional difference between the mobile and the desktop version, regardless of the device used to access it. | Core features of the main tool are functional on the mobile app but advanced features are limited. No difference in functionality between mobile devices. | The mobile app functions poorly such that core features are not reliable or non-existent. Significant difference in functionality depending on the mobile device used to access the tool. | |
| Offline Access | Offers an offline mode: Core features of the tool can be accessed and utilized even when offline, maintaining functionality and content. | Offers a kind of offline mode, where the tool can be used offline but core functionality and content are affected. | The mobile platform cannot be used in any capacity offline. |
Usage and Account Set Up
| Category | Sub-category | Works Well (★★★ to ★★☆) | Minor Concerns (★★☆ to ★☆☆) | Serious Concerns (★☆☆ to ☆☆☆) |
|---|---|---|---|---|
| Usage and Account Set Up | Sign Up/Sign In | Use of the tool does not require the creation of an external account or additional login; no personal user information is collected and shared. | An instructor account is required but learners are not required to create an external account or additional log in. | All users (instructors and learners) must create an external account or additional login, and/or personal information is collected and shared. |
| Cost of Use | All aspects of the tool can be used free of charge. | Limited aspects of the tool can be used for free with other elements requiring payment of a fee, membership, or subscription. | Use of the tool requires a fee, membership, or subscription. | |
| Archiving, Saving, and Exporting Data | Users can archive, save, or import and export content or activity data in a variety of formats (some of which are open standards) | There are limitations to archiving, saving, or importing/exporting content or activity data. | Content and activity data cannot be archived, saved, or imported exported. | |
| Data Privacy and Ownership | Users maintain ownership and copyright of their intellectual property/data; the user can keep data private and decide if / how data is to be shared. | Users maintain ownership and copyright of their intellectual property/data; data is shared publically and cannot be made private. | Users forfeit ownership and copyright of data; data is shared publically and cannot be made private, or no details provided. |
Social Presence
| Category | Sub-category | Works Well (★★★ to ★★☆) | Minor Concerns (★★☆ to ★☆☆) | Serious Concerns (★☆☆ to ☆☆☆) |
|---|---|---|---|---|
| Social Presence | Collaboration | The tool has the capacity to support a community of learning through both asynchronous and synchronous opportunities for communication, interactivity, and transfer of meaning between users. | The tool has the capacity to support a community of learning through asynchronous but not synchronous opportunities for communication, interactivity, and transfer of meaning between users. | Communication, interactivity, and transfer of meaning between users is not supported or significantly limited. |
| User Accountability | Instructors can control learner anonymity; the tool provides technical solutions for holding learners accountable for their actions. | Instructors cannot control learner anonymity but the tool provides some solution for holding learners accountable for their actions. | Instructors cannot control learner anonymity and there is no technical solution for holding users accountable to their actions. | |
| Diffusion | The tool is widely known and popular, it’s likely that most learners are familiar with the tool and have basic technical competence with it. | Learners’ familiarity with the tool is likely mixed, some will lack basic technical competence with its functions. | The tool is not well known/foreign, it is likely that learners are not familiar with the tool and lack basic technical competence with its functions. |
Teaching Presence
| Category | Sub-category | Works Well (★★★ to ★★☆) | Minor Concerns (★★☆ to ★☆☆) | Serious Concerns (★☆☆ to ☆☆☆) |
|---|---|---|---|---|
| Teaching Presence | Facilitation | The tool has easy-to-use features that would significantly improve an instructor’s ability to be present with learners via active management, monitoring, and engagement. | The tool has limited functionality to effectively support an instructor’s ability to be present with learners via active management, monitoring, and engagement. | The tool has not been designed to support an instructor’s ability to be present with learners via active management, monitoring, and engagement. |
| Customization | Tool is adaptable to its environment: easily customized to suit the classroom context and targeted learning outcomes. | Limited aspects of the tool can be customized to suit the classroom context and learning outcomes. | The tool cannot be customized. | |
| Learning Analytics | Instructor can monitor learners’ performance on a variety of responsive measures. These measures can be accessed through a user-friendly dashboard. | Instructor can monitor learners’ performance on limited measures; or data is not presented in a format that is easily interpreted. | The tool does not support the collection of learning analytics. |
Cognitive Presence
| Category | Sub-category | Works Well (★★★ to ★★☆) | Minor Concerns (★★☆ to ★☆☆) | Serious Concerns (★☆☆ to ☆☆☆) |
|---|---|---|---|---|
| Cognitive Presence | Enhancement of Cognitive Task(s) | The tool enhances engagement in targeted cognitive task(s) that were once overly complex or inconceivable through other means. | The tool enables functional improvement to engagement in the targeted cognitive task(s). | The tool acts as a direct tool substitute with no functional change to engagement in the targeted cognitive task(s). |
| Higher Order Thinking | Use of the tool easily facilitates learners to exercise higher order thinking skills (given consideration to design, facilitation, and direction from instructor) | The tool may engage learners in higher order thinking skills (given significant consideration to design, facilitation, and direction from instructor) | The tool likely does not engage learners in higher order thinking skills (despite significant consideration to design, facilitation, and direction from instructor) | |
| Feedback on Learning | Through the tool, learners can regularly receive formative feedback on learning (i.e. they can track their performance, monitor their improvement, test their knowledge, etc.) | Opportunities for receiving formative feedback on learning are available, but infrequent or limited (i.e. poor opportunities for tracking performance, monitoring improvement, testing knowledge on a regular basis) | There are no opportunities for formative feedback on learning (i.e. lacking opportunities for tracking performance, monitoring improvement, testing knowledge on a regular basis) |
Complete Rubric
You can also download a copy of the Rubric (includes all the sections listed above).
License
Rubric for Learning Technology Evaluation by the Centre for Innovation in Teaching and Learning, Memorial University, 2020, is made available under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based upon: Rubric for eLearning Tool Evaluation by Lauren Anstey & Gavan Watson, copyright 2016 Teaching Support Centre, Western University.