In learning, timing matters. Through decades of research into the cognitive science of learning and how memories fade over time, we now understand that spacing out learning into shorter sessions with breaks is more effective than cramming and that the particular spacing schedule is crucial.At Cerego, we leverage cognitive science and machine learning to build an optimized schedule for every user, that adapts as their memory for each concept improves and fades. But having a schedule, and following it, are two very different things. When the timing and frequency of your learning are so important to success, making it easy to fit your learning into everything else in a day becomes critical.
That’s why it is important that Cerego offers fully-featured mobile apps on iOS and Android, that notify users when they need to review material, and allow them to do so conveniently wherever they are. While desktop and laptop computers are accessed during limited time blocks, people generally have their phones available through much of the day — a pattern which better allows the ‘little and often’ strategy of spaced learning. Emails are answered twice as quickly from a mobile device as a desktop or laptop, and mobile learners may further benefit from receiving push notifications — short, visible reminders ideally suited to prompt activities that require timeliness, such as learning on Cerego.
In theory, having access to Cerego on a mobile device should make it easier to review — to learn more, and to study in a more effective way. We set out to measure the differences in how our mobile users distributed their study, and the effect this had on their learning. We found that not only did mobile users learn twice as much material through Cerego, but they did so in a more efficient way, through shorter sessions, sticking closely to their schedule - and showed greater learning gains.
We looked at the anonymized learning activity of 1,739 learners studying Cerego courseware, a higher education product managed through the personalized learning platform on Cerego.com. Courseware students were enrolled in introductory macroeconomic college courses in which materials were offered through Cerego. Of these, 216 learners only studied material through Cerego and did not return to review it; these data were removed from the study and we focused the remaining sample of 1,523 engaged learners.
Of these engaged learners, 427 (28.0%) downloaded one of Cerego’s mobile apps and completed some of their learning activity using the app. We categorized these as mobile users, to compare their data to learners who never used a Cerego mobile app. To start to understand how using a mobile app impacts learning, we ran a series of between-group analyses, examining the practice behavior and learning outcomes of the mobile group vs the non-mobile group.
What we found
Despite being in the same course with similar requirements, mobile learners completed more than twice the number of total reviews (quizzes) than their non-mobile counterparts. This reflected both wider and deeper learning - mobile users learned an average of 255 concepts each, compared to 173 for students who didn’t use mobile, and also completed more reviews for each concept they learned (5.2 vs 3.8).
When and how did this extra practice happen? Interestingly, despite doing twice the total practice, mobile learners tended to review in shorter study sessions.
This pattern of frequent, short practice — little and often — is a healthier and more effective approach to learning. It supports the process of distributed practice overall, helping users to refresh fading memories at the optimal time for them. Short sessions may also help learners to maintain good focus, thereby facilitating effective encoding and retrieval of information.
Another advantage of making shorter but more frequent reviews is that it makes it easier to stick to a schedule and review each memory at close to the optimal time. Since Cerego keeps track of the ideal time for each of a user’s memories to be reviewed, a user logging in to learn will be automatically provided with a review session that prioritizes the memories most in need of review at that moment. Indeed, we found that our group of mobile learners were statistically more “on time” for their personalized learning schedule than non-mobile learners.
Both groups received study reminders at the same rate, although push notifications tend to be responded to faster and more often than emails.
Our mobile learners overall showed a pattern of learning that should improve their outcomes. At Cerego, we have consistently seen that following review time recommendations, and studying in smaller sessions, are both strong predictors of more effective learning and retention on our platform. Therefore, it was no surprise that our mobile group also showed significantly greater learning after they started reviewing in Cerego, gaining more accuracy by later reviews.
To be sure this effect was really due to mobile use, and not because our mobile users were simply a more knowledgeable group to begin with, we used a linear mixed-effects model. This approach accounts for the fact that individual learners can differ greatly in their overall accuracy, and at the same time separately models the accuracy and improvement of each group. We modeled accuracy as a function of time (first v later reviews), mobile use and their interaction (i.e. whether mobile users saw a greater or lesser change in accuracy over time), including the individual user as a random effect.
Both groups gained knowledge using Cerego, showing a roughly 10% improvement in accuracy between first and later reviews [9.8% +/- 0.5%; p < .001]. The two groups showed no significant difference in overall accuracy [p=.280]. This is important since if our mobile users were simply more dedicated and able, we could have less confidence that any differences between the groups were due to mobile use. In fact, our mobile users did not start off any more advanced or knowledgeable than the non-mobile group at the beginning of the exercises (if anything, the opposite). However, after practicing more efficiently, their retention and accuracy in later trials improved by significantly more than the non-mobile group [2.0% +/- 0.9% greater improvement for mobile learners, p=.019].
Though our mobile users didn’t start with any more knowledge than their peers, after learning in Cerego in smaller, more frequent doses, and sticking more closely to their schedules, they showed measurably stronger improvement.
Smartphones have taken society by storm, and we are only beginning to understand their potential roles and applications in learning. We’ve seen here how the study habits that mobile devices allow can lead to more effective learning, and more generally mobile devices may inherently offer greater access to learning opportunities. Data from flashcard company Study Blue revealed that their learners frequently studied on their phones in bed after waking up (52%), or before going to sleep (46%), and while in transit (76%). These transitional moments contrast with traditional ideals of studying having a ‘proper’ time and place. However, our data suggest that encouraging short practice sessions in any context may lead to better learning outcomes overall.
Our primary mission is to make learning measurably more efficient and effective. Mobile may be the ideal way to access the Cerego personalized learning platform, where short, frequent interactions and timeliness are key predictors of better learning.