Demand for best practice online assessments increased rapidly ever since the education sector had to react to the impacts of a global pandemic.
By the year 2021, it became clear that online training — and online assessments — are here to stay.
- While Covid lock-downs and travel restrictions had a devastating impact on a wide range of industries, the education sector was hit particularly hard.
- Fortunately, industry growth trends are expected to continue despite the initial impacts of the pandemic.
- That noted, the once ‘financially infallible’ Education Sector has, indeed, fallen on extremely difficult times.
But what about the capacity to provide online learning experiences?
While the pandemic prevented an influx of overseas students, online degree programs opened up an even larger market of prospective learners.
It wasn’t an easy transition for a number of education providers.
When University campuses suddenly closed — in an attempt to limit super-spreader environments — Educators had to rapidly adapt to digital delivery formats.
When they did so, they encountered numerous conversion issues.
It seems that many Universities lagged behind other sectors in terms of digital technology investments. We’re talking old systems, inadequate digital training for lecturers/tenured professors, and overcrowded servers/networks (resulting in slow loading speeds).
Converting existing degree programs into digital delivery formats often:
- proved more problematic than anticipated, and
- required more resources and tech-savvy talent than they allotted.
One of many digital course conversion problems experienced by the Education Sector was that the majority of education providers were already behind other industries in terms of technology adoptions. Only a few Institutions were early adopters of technology and automation.
Many Universities and RTOs relied on “old-school” methods of education delivery.
But Universities weren’t known for upskilling their teams in the latest Tech; and failed to invest in modern computer systems and remote connectivity for large groups of personnel.
But the lack of technical ability amongst University staff – along with outdated computer systems and unsupported software – became increasingly problematic when the pandemic forced delivery methods to move to purely digital formats.
The Education industry was significantly behind other sectors in terms of digitisation.
This industry-wide delay in digitising learning resources was described in a Medium.com article published in 2019.
A year before the Covid pandemic spread across the globe like a Tsunami, the Education industry was being ‘called out’ for being painfully slow to ‘digitise’. Particularly when compared to other billion-dollar sectors.
- In one way, the pandemic will have a lasting, positive effect on the education and training industry.
- It will force Education providers and RTOs to finally adapt.
But overall, many entered the digital conversion game from an unenviable position.
They were well behind other sectors and their sector’s more competitive providers. They had failed to modernise their systems.
So their launching point was miles behind the starting line.
Image source: Holon/Media.com
Only a few had adequately digitised their learning resources and enrolment systems prior to 2020.
The second problem with course conversion projects was a lack of adequate resources.
It’s been a year since the first of lockdowns and campus closures began. Financial sustainability remains questionable for once-solid Education Institutes and RTOS.
Digital course conversion projects have also been particularly difficult for Institutes that jettisoned valuable Academic personnel because they could not afford to meet their payrolls.
Education providers are attempting to meet digital course conversion requirements with fewer employees, and limited funds.
Many RTOs and education providers – including renowned Universities like Princeton – have been left with limited resources and financial instability. There have also been unprecedented numbers of redundancies of education professionals across the globe.
Interestingly, these University personnel shortages – and the shortage of students that spawned them – coincides with the need to devote even MORE personnel resources towards providing online learning experiences.
Education industry redundancies are occurring at the same time that education providers need to convert existing classroom courses into digital delivery formats (including online assessments).
These digital course conversion projects require sufficient resources and adequate time frames.
Competition for international and regional student enrollments is also heating up across the globe. But that trend has been in place for decades.
What’s new in the education sector is the rapid shift in education program delivery formats.
Digital conversions and online resources have reached an unprecedented level of importance.For education providers, it’s essentially “a race against time”. They are needing to convert courses into digital formats under ‘panic-driven’ deadlines and drastic budget cuts.
The sudden industry-wide demand for digitally-delivered degree programs and online assessments:
- severely stressed existing resources
- revealed the incompetency levels of some lecturers who were not readily adaptable to digital delivery formats
- created problems in relation to student engagement (enrolments, retention, and completion rates)
Impact of COVID-19 on the Education and Training Sectors (2021)
In early 2020, nearly every education provider across the globe — from pre-schools and RTOs to ivy-league Universities — had to quickly convert their education programs into online learning formats.
But as educators soon learned, it wasn’t as simple as converting classroom facilitated resources into online delivery formats. Nor was it as straightforward as changing a brick-and-mortar classroom setting to a Video Conferencing platform, such as Zoom.
By 2021, most educators had gained a far better understanding of what was required for designing effective online learning experiences – and online assessments – for a larger proportion of their education courses and advanced degree programs.
Converting face-to-face education materials to online delivery formats was time-consuming — and more complex — than Educators presumed.
Moving numerous face-to-face courses into a digital delivery format required significantly more tech-savvy personnel (e.g. Instructional Designers). It also required highly-adaptable lecturers.
Not all lecturers were keen to embrace these pandemic-related changes. And not all readily lecturers adapted to using online formats.
- Data usage plans were also stressed.
- The rapid increase in video-streaming users nearly ‘broke the IOT’.
- Students and lecturers also expressed very mixed views on the effectiveness of digital training formats vs face-to-face lecture settings.
IOT = Internet of Things
Digital courses had initially been plagued by sporadic WiFi signal ‘dropouts’ and similar disruptions.
The education industry — and the students it aimed to serve — suffered during the initial digital course conversion periods of 2021. These online course conversions were hit-and-miss; plagued with everything from inadequate content and poor internet connections, through to a lack of student engagement.
There were also sudden, drastic reductions of Academic personnel in relation to student ‘shortages’ from travel bans.
- However, these digital training issues are starting to resolve in early 2021.
- Fortunately, key education industry influencers are readily sharing their online course delivery learning experiences, from pitfalls to best-practice designs for online assessments, in articles on LinkedIn.
Converting face-to-face course content into digital learning formats was not as straightforward as most education facilities anticipated. But by March 2021, things were looking more positive on the digital-course conversion front.
- By March 2021, course conversion projects appear to be running much smoother than they were a year ago.
- Gaps in course content were becoming more readily recognisable.
- Lecturers were also warming to the idea of delivering training from their home offices and/or working with course designers to deliver ‘self-paced’ learning options rather than facilitated training.
And when it comes to designing online assessments — one of the many challenging aspects of ‘getting the course content right’ — the following top 5 online assessment design tips can be of help.
In this article, as key influencers in the world of GMP training, we’ll share our “Top 5 Tips for Best Practice Online Assessments”.
Before you review online assessment design tips, you’ll want to consider the future of digital education and online learning – and what went wrong with course designs in 2020.
Firstly, if you’re delivering your courses online, it is important to plan ahead in terms of training budget allocations.
Here’s why.
Course conversion projects will continue to be a priority expenditure for numerous Universities and RTOs through 2021-2024.
In 2020, educators had little idea of the time frames and resources required for digital course conversions. Being in ‘panic’ mode didn’t help matters. Course conversion budgets were often tight (or non-existent).
But by 2021, Education providers had a far better understanding of the resources required for successful course conversions.
Having gained this knowledge, a large majority of education providers will continue to convert classroom learning experiences into predominantly digital course delivery formats through 2021, 2022, and beyond.
These Universities and RTOs will make course improvements based upon what they learned during the 2020 and 2021 lockdowns.
But remain aware that course conversions will continue to put pressure on training budgets (and course delivery personnel), from early 2021 through to mid-2023 (or even 2024), particularly as:
- new courses will need to be developed as new education technologies will emerge
- many courses require annual updates pertaining to industry trends and/or legislation changes
- transition pains will continue to be analysed, and improved, as prior experiences inform:
- best practice online course delivery methods and
- best practice online assessments / final assessment designs
These factors will also further increase demand for talented digital training course designers. In particular, there will be a rise in demand for course designers who have existing in-depth knowledge of your training topics and degree plans.
The future of online training course designs and delivery formats
Snapshot: Demand for validated digital courses and online assessments will continue to grow beyond the current intensive [course conversion project period]. The need for increasingly engaging course content will continue to expand through 2021-2024 and beyond.
Because the online training market also increases global competition for students, and for student retention, you’ll need to devote adequate resources to your eLearning training development budgets.
What will happen next is that virtual-reality learning experiences will emerge as a best-practice training delivery method for a variety of training topics; although not all topics will be suitable.
Secondly, course development costs will also increase significantly when virtual reality technologies are added to existing training options.
So as stated above, advanced planning and adequate resource allocations are imperative for successful conversion projects.
Challenges of converting classroom courses into digital delivery formats
Converting education content to purely digital formats is time-consuming and labour-intensive. Conversion projects are often plagued by numerous course design challenges and extended development time frames.
The following are common factors that make conversion projects complex in nature – and time-consuming.
- Most trainers use PowerPoint slides.
- A small ratio of these PPTX files can convert readily into a lecturer-facilitated online delivery format.
- The majority of PowerPoint presentations/lecture aids will have too many content gaps to be usable as “stand-alone” materials.
- These presentations will require intensive rewriting for context and content (a time-consuming exercise).
- They may also fail to suit the variety of instructors that may be tasked with using these materials in their digital classroom environments.
The challenges of online course development increase significantly when training is going to be delivered without an instructor/facilitator; e.g. hosted via an LMS system versus a live-streamed video format. Examples include 24/7 online training formats for Certificate GMP training.
- Converting a Facilitator-led course to a purely online version (self-paced digital training methods) typically requires a platform conversion.
- There is also the need to significantly revise AND increase the content volume:
- to fill in any gaps
- to anticipate and answer common student questions online vs handling questions face-to-face as they arise (and which can vary in depth and scope, depending on the audiences pre-existing knowledge)
- to cover what the facilitator would otherwise explain in a classroom setting or webinar
Once you determine your learning objectives, how should you design the online assessment segment?
Key questions relating to digital course designs/online assessments include:
- Who is the audience and what do they need to know?
- Are there any special considerations (ESL, vision impairment, etc)?
- How many topics should I include in one module?
- Shorter courses are typically better than longer ones.
- Shorter courses also have higher student retention rates and online course completion rates.
- Best practice course design: It is typically best to break up learning modules into individual modules of less than 2 to 3 hours each.
- This may not always be feasible; if not, ensure the learner can stop and start, or completely restart the course, using ‘bookmark’ coding.
- Which content order (topic sequence) will provide the most effective learning experience?
- This is where high-level course planning is imperative.
- Review other similar modules or course agendas in the industry before you design.
- Create an outline and work directly from that outline.
- Then test the module with reviewers (and validate the online assessment) before you publish the course.
- How long should I make the Online Assessment (average number of questions to include)?
- An online assessment should ideally not take the average learner more than 45 minutes to complete.
- But for some topics, this time limit simply isn’t feasible.
- One way to make final assessments easier to take/grade is to split out essay questions from the other types of theory questions (T/F and multiple choice).
- If you’re using essay formats requiring an answer of more than a sentence or two, its typically best practice to split-out those essay questions into a separately graded online assessment; rather than include them with multiple-choice questions and true or false questions.
- You’ll benefit from automated assessment grading (instantaneous) for the ‘radio button’ questions vs the more time-consuming essay grading requirements (which takes considerably longer and is far less reliable).
- What kind of questions should I include on the online assessment?
- Ensure your assessment questions meet the learning objectives (they need to be validated).
- Include a variety of question types.
- Keep wording simple (remember your audience).
- Mix it up a bit for further engagement (scroll down for further online assessment design tips).
- Should the online assessment be timed?
- Typically yes, but not always, especially as most online assessments are taken as ‘open book’.
- But timing online assessments can be fraught with problems.
- Read an example of a major fail for medical degree student assessment.
- Are learners given the opportunity to retake the assessment? If yes, do the online assessment questions need to change? And how soon after their initial unsuccessful attempt can they retake the online assessment?
- This decision will vary depending upon the learning philosophy of the education provider or training organisation (RTO), as well as their student demographics or learner profiles.
- Education delivery regulations also need considerations such as making special allowances for individuals with various ability differences (disabilities) or language barriers.
- Validation of the training program and online assessment are essential.
- Risk management must be performed where continually retaking an important exam indicates a learning issue that could result in serious consequences (such as a medical exam that, if failed repeatedly, indicates potentially serious consequences in relation to future-patient safety).
- How are a person’s actual skills assessed? Or is online training meant just for theory?
- Online training is typically mostly for knowledge assessment (theory) versus skills assessment.
- But there are several exceptions. For example, an online assessment that involves an application coding question may assess a user’s coding skills by requiring the learner to:
- successfully navigate a project in a ‘sandbox’ setting
- accurately complete an actual coding project without exceeding a specified number of steps or keystrokes
- rapidly troubleshoot a coding issue within a pre-set time limit
- How do course designers prevent cheating on online assessments?
- This question is beyond the scope of this article.
- However, cheating on assessments is rampant in the education sector.
- Cheating on online assessments is less rampant when training is one-one (e.g. new employee training) vs one-to-many (higher education).
The latter question regarding cheating on online assessments is more a ‘formal education’ question than an industry one. Because in an industry setting, a lack of training or knowledge becomes readily apparent in the workforce, especially in a GMP environment.
Click here for information on online course completion ratios and eLearning design tips.
But it’s important to remember that online education is only one part of gaining competency; experience counts (a lot).
Online training is primarily used for imparting theory (and assessing knowledge); the foundational aspects of skills training.
But online courses are not typically used as often for skills training and skill assessments.
The one clear current exception is technical skills training (for computer skills or program skills).
- Online training typically cannot typically teach (nor adequately assess) the variety of skills needed in today’s employment market.
- Supervision and/or mentoring are essential.
- Blended learning experiences also offer more effective learning experiences.
In general, then, online courses alone cannot fill the gaps left behind by university campus closures. And many students will crave more direct support from their Lectures and/or PhD Supervisors (some far better than others in terms of academic cultures).
The one clear future exception is the ability of virtual reality to remotely teach — and remotely assess — a person’s skills. But these types of training remain in their early years. Plus, virtual reality is often expensive to implement (on both sides, meaning students and their Course Assessors).
How much does the online assessment count when it comes to course validity?
- The short answer is that it counts a lot.
- Scroll down for the Top 5 Online Assessment Design Tips, or continue reading for the longer answer.
While content matters in terms of online course quality metrics, online assessments either ‘make or break’ the learning experience.
As with all education, the starting point for online assessment design is the ‘Learning Objectives’.
- What are you hoping the learner to take away from the learning resource?
- What are the imperative understandings you hope to convey?
- How detailed does the online assessment need to be, without it becoming an unreasonable length?
Online training course development: the challenges of designing online assessments
One of the biggest challenges when designing online courses relates to the validity of online assessments; particularly for extremely long courses.
When designing an online assessment for a digital course, remember you will need to validate the online courses and assessments before you publish.
Let’s now review some best-practice online assessment strategies, and why these are increasingly important in the current year (2021-2022 and beyond).
- Online course development growth was initially related to the impact of the pandemic.
- The pandemic forced nearly all education providers, including industry-based skills training providers, to adapt.
This meant education providers were suddenly forced into converting face-to-face learning resources into completely digital delivery formats. Not everyone had the required resources for complete digital course conversions, and only a few did this well.
Adding to the everyday challenges of classroom-to-digital course conversion projects were:
- extremely short project time frames (rapid deadlines, e.g. ASAP), and
- significant financial and student retention repercussions for failing to convert courses rapidly enough
The problem was, time was money, and without delaying the graduation plans of students by several years, time was of utmost importance.
This meant course designs were assessed more on turnaround times than on assessment quality and best practice designs.
Permitting Academics and students to remain on campus was considered too great a risk for most education providers. Yet not every University headed the warnings; and some lecturers and students willingly took the risk as they either preferred to be on campus, or didn’t believe the pandemic actually existed. Other educators had inadequate resources to convert to online course delivery formats.
Online assessments, as well as course materials, ended up with such varying degrees of quality as to be a major problem for the education industry for years to come.
Unlike Universities (predominantly ill-prepared for the sudden, mass conversion to online delivery formats), many industry-specific training providers already had sufficient digital training resources.
These industry-specific training companies were accustomed to upskilling personnel across the globe. Particularly in the pharmaceutical industry, where international supply chains have long been complex. And where rigorous training in GMP compliance is required by every contractor, supplier and employee, regardless of location or role.
Decisions to remain on campus ended up contributing to an even greater demand for digital course designs and online assessment designs.
This is related to:
- A reduction in available employees due to illness ramifications, travel restrictions, and University downsizing
- Lack of online course and online assessment design skills amongst Academic staff
- A significant portion of tenured Professors from the baby-boom era nearing their retirement
- Very high demand for digitally-savvy course designers across the globe
Indeed, University campuses rapidly became ‘super spreader’ environments for the virus.
- Most campuses became ghost-towns due to a lack of online lecturers and a lack of students.
- Digital dis-engagement was also rife.
The industry spiralled and recovery is going to be a very long road, according to most economists and Education sector analysts.
To help ease the pain of these digital project conversions, here are the Top 5 tips for designing ‘best practice online assessments’ to maximise engagement.
Best Practice Online Assessments (Top 5 Tips)
#1: Mix up question formats on online assessments
Aim for balance and variety of question types.
Best practice Assessment question types for online assessments include:
- multiple-choice
- true or false
- drag and drop
- scenarios
- practical analytical exercises (formulas, etc) if appropriate to the topic
Practical examples for mixing up question formats in online assessment designs
- Consider that you may have 20 questions on an online assessment
- If so, ensure you use at least 3 different question types
- These should be evenly spread throughout the assessment
- Include at least 1 but ideally 2 to 3 scenario questions
- Follow best practice plain-language wording and consider literacy levels of your student base
Best practice online assessment design:
- The consensus amongst designers is to limit your TRUE or FALSE assessment questions to less than 20% of the total online assessment
- For example, in a 20 question online assessment design, use a maximum of 4 TRUE or FALSE questions and at least 2 scenario questions
#2: Online assessment questions – best practice wording, language and design
Minimise the learner’s reading time when designing online assessments.
- DO keep sentences short and sharp
- The reading of the question shouldn’t consume too much of the available quiz time
- Let them have that time for ‘thinking’ and analysing
- DO, however, maximise their thinking to encourage analytical thought processes and evidence-based decision-making capacity
- DO add in other ‘common error’ answers to test the depth and accuracy of the learner’s understandings of the course materials or required actions
- AVOID long answers andAVOID ambiguous answers in multiple-choice questions
- DO keep sentences short and sharp
Maximise a learner’s retention levels through your wording of assessment questions to solidify the knowledge.
#3: Keep online assessment questions responses to 4 or 5 options versus 3 options (or less)
Avoid ‘false positives’ and answer-guessing issues
Questions should have enough response options so as not to be readily guessable
- Ideally, use at least 4 answers for multiple-choice questions (if using 3, make sure the other 2 questions illuminate common misperceptions so as to seem plausible, but avoid ambiguity as above)
- If you use at least 4, there will be less than a 25% chance of a correct response given when the learner doesn’t actually know the answer to the question
- This is somewhat like a ‘false positive’ and could be detrimental for compliance with regulations if you are talking about a course relating to regulatory compliance behaviours
#4: Split out essay questions or project questions into a separate assessment where possible
- Automate online assessment grading where possible (e.g. for radio-button types of questions such as true/false or multiple choice)
- Write and validate clear marking guidelines and metrics for grading essays
- Balance your assessments so that you cover all learning objectives
#5: Validate the online course assessment
- Follow procedures for validating digital courses and online assessments
- Monitor the completion rates of your course and assessments
- Solicit feedback from a variety of learners (based on your target audience and learning objectives)
- Conduct A/B testing of questions of concern
- Monitor successful vs unsuccessful attempts across all learners for at least 12 months
- Make adjustments as indicated (validating new questions/testing methods)
- Continue to monitor for completion trends and assessment results (formerly called ‘pass/fail’ rates, currently titled ‘successful completion’ vs ‘unsuccessful attempts’ and retaking metrics for course assessments)
- Improve as warranted with at least annual quality review and re-validation of online assessments
Further reading
More “Best Practice Online Course Design Tips for Training and Development professionals”
Top 10 Online Course Completion Tips
FAQs: e-Learning Development Questions
Last updated on October 13th, 2021 at 11:51 am
