OpenBack Whitepaper - Reliability in Push Notifications (Delivery & Metrics)

Blog

Last update: December 2019

6 mins to read

Notifications Click-Through Rate Boosted by “Adaptive Scheduling”

It’s old news that, living in the attention economy, we have an overabundance of information. And with an exponentially expanding supply of digital content, our attention has become a precious commodity. Push notifications can be a mobile app’s best way of speaking directly to a smart device user. And a good rule of thumb that we at OpenBack have modeled our platform around, is: Timing is everything. At a basic level, it’s just common sense. If you send a notification at the right moment, when the user doesn’t have anything more urgent going on, you’re more likely to achieve click-through. But has this been proven in an analytical study? How much does your click-through rate (CTR) actually increase?

In August 2019, Kota Tsubouchi from Yahoo! JAPAN, and Tadashi Okochi and Hideyuki Tokuda from Keio University led a study to quantify just how beneficial correct timing can be for push notifications. They assessed the engagement results of over 380,000 smartphone users over the course of 28 days. They published their results in an academic paper for SIGKDD.

The Negative Effects of Interruptive Overload

First of all, the researchers identified the problem: “interruptive overload.” Essentially, the concept of notifications came about as a way to provide users with useful, instantaneous information. However, with the glut of applications running on smart devices simultaneously, many notifications have started to resemble spam:

“users must now face numerous interruptive notifications at random timings, regardless of what they are doing as their primary task. When a notification is perceived and recognized by a user, some amount of his/her limited attention is allocated to the presented information.” (p. 2793)

Prolonged exposure to “interruptive overload,” far from being a slight nuisance, can actually have extended negative effects. The researchers found that a large number of micro-interruptions can result in:

  • poor productivity
  • affected mental/emotional state
  • time wasted on refocusing attention to the original task

When Are Push Notifications Most Effective for Click-Through?

The researchers identify “breakpoints” as the moment users are most likely to engage with notifications. They define a breakpoint as

“the boundary between two adjacent actions of the user. In particular, breakpoints are found to be the times at which interruptions cause the user less frustration and overhead.” (p. 2793)

They found that breakpoints following completion of other tasks on the smartphone – for example, answering a call or text message – tend to be when users are most receptive to reading notifications. It makes sense. The user’s already paying attention to their phone, so that’s half the job done already. Other studies expanded on this discovery to take into account location, emotional state, whether the user is in transit, whether they are alone or in company, and so on.

Researchers used machine learning to devise an “adaptive scheduling” system for sending push notifications to Android phones. Over a series of three challenges, they learned:

  • There was an average of 23.3% increase in click-through rate (maximum 60.7%) with the adaptive scheduling system
  • Adaptive scheduling combined with personalized contextual messages more than doubled CTR
  • Increased gain in CTR on days when unexpected breaking news was conveyed through push notifications

End Results on User Click-Through Rate

Ultimately, the study consisted of a set of “generic” push notifications… mainly updates from Yahoo! about sports and entertainment news. Generic notifications were sent four times a day, at 8 AM, 12 PM, 6 PM, and 9 PM. There was also a set of personalized notifications, sent to targeted user sets. These notifications did not have set times when they were pushed out. But the news sender often sent targeted notifications out at 9 PM, which marketers consider the best time to sent out content.

Image Source: https://www.kdd.org/kdd2019/accepted-papers/view/real-world-product-deployment-of-adaptive-push-notification-scheduling-on-s

As seen in the above images comparing the results of the control group with the group that received “adaptive scheduling” notifications, content that was sent according to optimized timing saw much greater engagement. As stated previously, there was a maximum boost of 67% in click-through.

Following data monitoring the “adaptive scheduling” group, the researchers found a 2.86x increase in the number of users who clicked on the notification within 120 seconds from delivery. They used the data collected to train their machine learning models in the cloud, and the following day send them back out to the device.

Image Source: https://www.kdd.org/kdd2019/accepted-papers/view/real-world-product-deployment-of-adaptive-push-notification-scheduling-on-s

Interestingly, the researchers for Yahoo! Japan had to create separate learning models for weekdays versus weekends, due to users’ behavioral differences.

Ultimately, the results with the adaptive scheduling were so significant that Yahoo! Japan now use it as their base platform for sending notifications to more than 10 million Android users. Overall, it was a very interesting study that was the first to academically quantify the effects of personalization and timing on push notification click-through rate.

How “Adaptive Scheduling” Compares to OpenBack’s Model

This study for Yahoo! Japan is exciting for us at OpenBack because it shows data science is taking a proactive approach to optimizing push notifications. In reality, we have made it our priority from the get-go to assess the best moment to send a push notification, using machine learning and device-side data to gauge users’ schedules. We do this on a user-to-user basis. Our platform also amalgamates data provided so your marketing team can see what sort of timing gets the best results and what might be driving users away.

With regards to the study’s “breakpoints,” the team at OpenBack have designed our platform to identify a number of these situations in which users are most likely to respond to a push notification, such as when:

  • Phone screen is unlocked
  • Headphone jack is in use
  • Phone battery is full

While there is an amount of overlap between OpenBack’s product and the one in Yahoo! Japan’s study, OpenBack ultimately offers a fully functional platform that offers a wealth of capability as well as further data triggers to combine to provide a customized, personal push notification experience for your user. What’s more, OpenBack can be used to send notifications to Android and iOS phones. Unlike the program in the study, which only supports Android.

OpenBack dashboard of user audience segmenting and unique data triggers

Ultimately, both platforms share the same goal: to send notifications that are beneficial to their users, resulting in a boost in interest and engagement. And while OpenBack has been focusing on such user convenience since our launch, it’s gratifying to see the theme finally gaining some traction in the data science world.

To read Okoshi, Tsubouchi, and Tokuda’s paper, click here to download.

Leave a Reply

Your email address will not be published.

ten + 14 =

Submitting...