Research to back up your best practices
Find executive summaries and full reports here
We’ve got an overview of best practices for performance measures for online programs, an exploration of the use of mobile devices in e-Channel programs, and much more.
Executive Summary
This report provides an overview of best practices for performance measures for online programs. While the initial focus of this report was e-Channel delivery agencies, the pandemic has resulted in many other service delivery agencies across the province delivering service remotely. The report, therefore, may have wider application than originally intended.
Although a funder defined performance management system is in place for literacy and basic skills service delivery agencies, including e-Channel, and the measures reported on provide valuable information for the service delivery agency, they do not cover the breadth of evaluation information that could be available if performance measures specific to online programming are developed.
Different types of performance measures are defined in this report. Important categories of performance measures specific to online learning are expanded with best practices for integration. In addition, steps for developing performance measures are outlined.
Using a variety of different types of assessment in online programs provides additional evidence that a learner is gaining the skills and knowledge required to move along their identified goal path. Organizations offering online courses must be able to develop and use performance measures that take into consideration learners that may not have the digital competence to access additional software or login to separate platforms.
Four broad categories describe the most suitable performance measures for evaluating online programs: engagement measures, learning experience measures, assessment measures, and outcomes. Developing and implementing performance measures across the four categories could help all organizations delivering online programs know if the content being delivered meets the needs of the learners, make decisions on how to revise programming to improve learner engagement and experience, increase learner outcomes, improve internal efficiencies, and provide measured evidence of program value for stakeholders.
Executive Summary
This report summarizes alternative equivalents to popular customized licensed curricula currently being utilized by e-Channel service providers. In order to prepare this report, the researcher reviewed over 200 potential equivalent sources. Eighteen alternative curricula options are summarized.
There are many possible alternative options for curricula online. Both open-source or free alternatives and paid or licensed alternatives have been included in this report. There is not a single source that will meet the needs of all the learners across the varied and culturally-specific programming offered through e-Channel. The next steps will be for service providers to review the options presented and determine feasibility of use within each organization.
Download: e-Channel Alternative Curriculum Sources Report 2020
Executive Summary
The rise of mobile device use is meteoric and adult literacy programs – particularly those that rely on technology as a means for learners to access learning – need to consider the implications for program delivery and design presented by the increased use of mobile devices. This report, commissioned by Contact North | Contact Nord, is a first step towards examining the use of mobile devices within e-Channel programs and the context in which such programs are operating.
The trends in mobile device use that may have the greatest impact on e-Channel programming are ensuring that all forms of content are mobile-optimized – a challenging feat since most online content was not developed with mobile devices in mind. As the use of mobile devices continues to rise, e-Leads may wish to develop a deeper understanding of who is using what (in terms of devices) and why.
Client Perspectives:
This research provided an instrument to interact with the e-Leads (e-Channel lead delivery organizations) to create opportunities to ask learners about their current experiences in accessing learning via mobile devices and to assess learner desires regarding the use of mobile devices in the future. These are some highlights of the information received:
- For Good Learning Anywhere (GLA), 18.2% of survey respondents said they used a mobile device to access their Career Technical Education (CTE) courses.
- There is interest from Good Learning Anywhere clients in using a mobile device as 54.5% of respondents said that if they had a mobile device, they would use it to access e-Channel.
- For the LearningHUB, fewer clients are using mobile devices to access live classes (5%), but there is interest in that 35% of clients would consider using a mobile device. A quarter of respondents (25%) said they could consider using a mobile device to access Independent Study and about 1/3 or 33.64% of HUB short classes survey respondents said they would consider using a mobile device.
- Large number of learners are “not sure” if they would use a mobile device or not, demonstrating that mobile device use, while increasing and popular, may not yet have hit its zenith with adult learners.
Program Perspectives:
Many e-Channel learners access e-Channel programming as part of a blended approach – doing some learning independently with e-Channel and some on site with an adult learning program. Given that many e-Channel learners spend at least some of their time in community programs – literacy, employment services, or Ontario Works – Contact North | Contact Nord launched a survey to ask Employment Ontario (EO) services and Ontario Works organizations about the extent to which they encouraged mobile device use amongst clients. Here are some of the results:
- There is a trend towards agencies providing ways for clients/learners to provide feedback via social media or mobile device, with 42.9% of survey respondents citing that they are currently providing ways for clients/learners to provide feedback via social media or mobile device.
- When asked what methods programs are using to garner client feedback via social media, many survey respondents identified more than one method. Most programs are currently using Facebook with texting being the second most common method of connecting with learners. Twitter and email share the third spot.
- In terms of effectiveness, it appears that the use of social media and mobile devices to gather client/learner feedback, at present, is somewhat limited, with only 12.50% of respondents defining such use as “very effective,” but under half of respondents (42.5%) considering their efforts in this area to be “somewhat effective.”
- Survey respondents suggest that using social media and mobile devices to gather client/learner feedback may be more appropriate for some populations (i.e., youth).
- Respondents who have had positive results with the use of social media/mobile devices to get client feedback have noted several trends. These newer approaches are more client-centred, being about what works for the client and not for the agency or program. Other respondents noted that they receive more feedback and possibly higher-quality feedback because they are using social media/mobile devices to get client feedback.
- There are challenges to using social media/mobile device use. Just under half of respondents (44.3%) noted that they have to change the way they do business in order to maximize the use of social media/mobile device use amongst learners. A slightly lower percentage of respondents (39.3%) cited lack of time to research and investigate the use of these methods as the main challenge.
- Programs are still at various levels when it comes to adopting social media/mobile device use, although several respondents noted that such methods are becoming the “norm,” regardless of the challenges related to implementation. The anticipated learning curve may be preventing some programs from adopting social media/mobile device use as it takes time to learn new programs and use them.
Mobile Device Use and Current Learning Platforms for e-Channel:
At present, the main learning platforms used by e-Channel leads are Saba Meeting, Learnscape, Moodle and Plato. Deaf Learn Now delivers its asynchronous programming for Deaf/Hard of Hearing students using the Desire2Learn (D2L) Brightspace Learning Management System. The compatibility of these learning platforms with mobile device use is a primary concern for some e-Channel lead organizations. All learning platforms are making strides to accommodate mobile device use. Saba Meeting has made a free mobile app available and Contact North | Contact Nord has made video and .PDF resources available to highlight this app. Moodle, Plato and Learnscape are also acknowledging the increased use in mobile device use.
Recommendations for Planning:
The purpose of this research was both to examine the current and potential use of mobile device use among adult learners, but also to assist e-Channel leads in preparing for the increased use of mobile device use. The following are recommendations that e-Leads and Contact North | Contact Nord may wish to consider:
- Encourage e-Leads to not just follow the mobile use of learners but to try to influence it. E-leads could put information on their websites to provide insight to learners about which mobile devices
- work best with their programs. For example, add content to websites along the lines of “Are you considering buying a mobile device? We recommend the use of XXXX for our programs…”
- Are there key messages that all e-Leads may want to develop and incorporate into their websites, perhaps in the form of FAQ, that go beyond recommendations of purchases for learners?
- As a result of blended learning and the fact that many e-Channel learners are participating via a blended learning approach, e-Leads may want to review how they connect with other programs in the province around mobile device usage.
- Analyze e-Channel users – into demographic age groups. Research suggests that some demographics prefer different social media to communicate. Do e-Leads know who is using their services via mobile device use?
- Encourage e-Leads to review the section “Creating Moodle Mobile friendly courses” before they create new Moodle content: https://docs.moodle.org/30/en/Creating_Moodle_Mobile_friendly_courses
- E-leads that use Learnscape as a learning platform may want to look at how Learnscape is being used in Africa to facilitate learner involvement.
- The Desire2Learn (D2L) Brightspace Learning Management System appears to be making strides towards accommodating mobile device use. More information can be found here: https://community.brightspace.com/blogs/product/using_d2l_mobile_applications_on_android_devices
- A collective e-Channel effort could be mounted to identify and review potential feedback programs and perhaps apps (recognizing that the use of apps is on the decline amongst users) that could be made available to all upon the program selection.
Conclusion:
Mobile device use is on the rise – in life in general and in education. Ensuring that e-Channel learning programs are compatible with the most common mobile devices (smartphones and tablets) and that information about using mobile devices to access learning is available and promoted on e-Lead websites is critical. Certainly, the development of new online e-Channel programs should be undertaken with mobile device use in mind.
The use of mobile devices to access e-Channel programming offers opportunity as well as challenges. The opportunities lie in the realm of increasing access for learners, while the challenges lie in the area of programs changing business practices and keeping on top of rapid developments in the technological realm. Employment Ontario programs, including literacy, employment services and Ontario Works, are motivated to further explore the successful implementation of mobile device use, if not for the purposes of learning, then for the critical ability to maintain communication with clients.
Obviously, the use of mobile devices – both to assist clients in accessing learning activities and for programs to explore meaningful ways of staying connected to clients – is paramount. This conversation needs to be embraced and ongoing and e-Lead programs need to be more explicit in assisting learners in understanding the role of and directing the use of mobile device use in programming.
Download: Mobile Device Use Report – March 2016
Executive Summary
This report summarizes alternative equivalents to popular customized licensed curricula currently being utilized by e-Channel service providers. In order to prepare this report, the researcher reviewed over 200 potential equivalent sources. Eighteen alternative curricula options are summarized.
There are many possible alternative options for curricula online. Both open-source or free alternatives and paid or licensed alternatives have been included in this report. There is not a single source that will meet the needs of all the learners across the varied and culturally-specific programming offered through e-Channel. The next steps will be for service providers to review the options presented and determine feasibility of use within each organization.
Download: e-Channel Alternative Curriculum Sources Report 2020
Executive Summary
This paper, commissioned by Contact North | Contact Nord, investigates software, apps and social media that may assist Literacy and Basic Skills (LBS) programs in communicating with their stakeholders more effectively, to make the best possible use of program resources. It will focus on Communication and Delivery Technologies and the role they can play in obtaining client feedback and in program delivery.
The paper reviews two growing trends: the use of mobile devices to facilitate communication and the ever-growing world of social media, as both impact the ways in which programs can and should communicate with learners and stakeholders.
The paper confirms that mobile device usage is currently a factor in successful learning and communication within a program. It emphasizes that it is critical that LBS practitioners and managers recognize that increasing numbers of learners will seek to connect with their program using their mobile devices.
To prepare for the increased desire among learners to use their mobile devices more actively in their literacy programming, the paper recommends that LBS providers review apps that have proven useful in communicating with/delivering follow-up messaging to learners. It can be difficult to know which of the many apps that are on the market is worth investigating. The article 21 Helpful Apps for Teachers and Educators is a good place to begin. LBS organizations with practitioners who have mobile devices may wish to download and try one or more of the listed apps, such as Remind. AlphaPlus delivered a webinar about using Remind in November 2016. Practitioners can review AlphaPlus’ recorded Remind webinar.
The report examines social media and its potential effects on how, when and what is communicated in LBS programs. It reflects the reality that social media is connected to technology, and that social media allows for the creation and sharing of information and ideas via virtual communities and networks.
Most LBS programs do not have the time or the resources to be proficient in the use of all forms of social media. How, then, does a program decide what to invest in? Forum Research’s article, 2015 Canadian Social Media Usage Statistics includes the following helpful information:
| 2015 | Overall Usage | By Sex | Ages | ||||
|---|---|---|---|---|---|---|---|
| Platform | Total % | % Men | % Women | 18-29 | 30-49 | 50-64 | 65+ |
| 72% | 46% | 53% | 82% | 79% | 64% | 48% | |
| 28% | 43% | 56% | 56% | 28% | 11% | 4% | |
| 26% | 51% | 49% | 24% | 35% | 28% | 13% | |
| 31% | 26% | 73% | 32% | 31% | 21% | 14% | |
| Tumblr | 10% | 48% | 52% | 53% | 29% | 13% | 5% |
| 23% | 54% | 46% | 40% | 36% | 16% | 7% | |
Questions around how the use of technology might alleviate the challenges of obtaining client feedback, and exited learners’ follow-up information are considered, together with the issue of survey fatigue. The paper points to the findings of the article In Myths and realities of respondent engagement in online surveys, in which the authors suggest that increasing respondent engagement is based on survey length, topic salience, cognitive burden and frequency of survey requests.
The paper recommends that LBS organizations document the resources they currently invest in client follow up and states that, “if LBS organizations can determine how much client follow up currently costs their organization, especially in terms of human resources, they may find that a new tool is worth investigating, especially if it improves their results and reduces their human resources outlay.”
Tools to Reach Additional Clients
As the need to stay in touch with LBS learners is so critical, this paper highlights three survey companies: Survey Gizmo, mPoll.me, and Client Heartbeat. There are several free survey instruments on the market that LBS agencies can access, but these survey companies may provide some additional benefits.
Survey Gizmo | mPoll.me | Client Heartbeat | |
|---|---|---|---|
Url | https://www.surveygizmo.com/ | https://mpoll.me/ | https://www.clientheartbeat.com/ |
Benefits and Features | · Assists with survey administration across teams (might be useful if you have more than one programming site) · Can also be used on an individual level (within one organization) · Brand customization (add your logo and organization’s colours for consistent branding) · 40+ types of survey questions · Industry leading security · Mobile friendly · Data analysis and reporting tools | · Average 65% survey response rate · All plans come with Custom SenderID (so users know who’s contacting them) and real-time results | · Brand customization (add your logo and organization’s colours) for consistent branding · Sends periodic surveys · Average 60% survey response rate · Testimonial Widget. Easily display positive testimonials on your website. · Log in to your company dashboard to view current and past results for each of your customers or company as a whole. |
On the Downside | · A more sophisticated program – could be a bit overwhelming for small programs | · Only for mobile devices | · No free plan – just a free trial |
Tools to Get Feedback on Website Usability
Since, increasingly, LBS agencies are relying on their websites to attract and inform their customers, the report provides information on potential products that can provide insight into how customers experience the sites. Below, two such products are compared: Qualaroo and UserTesting.com
Qualaroo | UserTesting.com | |
|---|---|---|
Url | https://qualaroo.com/ | https://www.usertesting.com/ |
Benefits and Features | · Qualaroo makes it easy to survey specific groups of website visitors to gain qualitative information. · Get a clear picture of who your customers are, and what they’re looking for on your site. · Qualaroo surveys lets you target questions to visitors anywhere on your website · Offers exit survey options – find out why people are leaving your site | · Can access different levels of support · Get audio, video and written feedback on websites and apps · Get videos of real people speaking their thoughts as they use your website or mobile app. Go beyond analytics and understand why users do what they do. |
Pricing | Start-up Plan: $199/month | Basic: $99 per video (note: website has offer of $49/video for first 10 videos) You can request a free trial For more intensive service, you need to get a pricing quote |
Tools to Improve Use of Social Media
The report also reviews several social media communication tools for LBS organizations’ consideration: Google Not for Profits, Google Alerts, and Audiense.
Google for NonProfits | Google Alerts | Audiense | |
|---|---|---|---|
Url | https://www.google.ca/intl/en/nonprofits/ | https://www.google.ca/alerts#1:1 | https://audiense.com/ |
Benefits and Features | · Free access to Google tools like Gmail, Google Calendar, Google Drive, Google Ad Grants, YouTube for Nonprofits and more · Google Ad Grants alone can increase your organization’s profile substantially | · Find out everything happening related to literacy. Enter “literacy” or whatever word you choose and find out what’s in the news and on social media · Keeps you informed | · Build new audiences · Understand your audiences · Deliver the right experience |
Pricing | Free | Free | · From $31/month |
Summary
The paper concludes that, historically, LBS programs in Ontario have done an excellent job of communicating what they do, how they do it, and the benefits of their work and suggests that as times and technology change, new opportunities for communication develop and require exploration.
Download: Communication and Delivery Technologies Final Report
Executive Summary
This report summarizes alternative equivalents to popular customized licensed curricula currently being utilized by e-Channel service providers. In order to prepare this report, the researcher reviewed over 200 potential equivalent sources. Eighteen alternative curricula options are summarized.
There are many possible alternative options for curricula online. Both open-source or free alternatives and paid or licensed alternatives have been included in this report. There is not a single source that will meet the needs of all the learners across the varied and culturally-specific programming offered through e-Channel. The next steps will be for service providers to review the options presented and determine feasibility of use within each organization.
Download: e-Channel Alternative Curriculum Sources Report 2020
Download: Moodle Unplugged
Executive Summary
Extended Reality (XR), comprising virtual, augmented and mixed reality, is increasingly being adopted
in education and training programs globally and is widely used in various sectors including gaming,
entertainment and education. It has the potential for a wide range of applications in adult literacy,
including technical training, professional development, distance learning, continuing education, soft
skills training, cultural and historical education, stress-relieving, language learning, virtual field trips,
and special needs training.
A survey conducted by Literacy Link South Central (LLSC) and Contact North | Contact Nord in Fall
2022 showed that although over half of the respondents were familiar with Extended Reality (XR),
less than 3% had used it in adult literacy programs, suggesting an opportunity for future adoption as
the technologies become more accessible.
This report identifies various ways XR can be used to support adult literacy and basic skills learners.
It also outlines some of the limitations and best practices for integrating different types of XR into
programming. A basic overview of popular hardware and software is included.
The report concludes with sample case studies and examples of XR projects from across Canada
providing real-life examples of how XR can be integrated into programming.
