Overview
This report collates findings from three rounds of CX research conducted as part of the Authentication Uplift project and provides a comparison on models tested.
Round 1 was conducted in September of 2022 and benchmarked the existing ‘Redirect with One Time Password (OTP)’ model. Round 2 research focused on ‘App/Web-to-App with Biometric’ and ran in November of 2022. Round 3 research focused on ‘Decoupled with QR Code’ and ran in March of 2023.
The purpose of the research was to identify consumer experience considerations to support and inform an expanded approach to Consumer Data Right (CDR) authentication. The objective of uplifting authentication in the CDR is to give consumers more choice and freedom when authenticating themselves with DHs (data holders), while maintaining financial grade security.
In total, over 150 consumers participated across the three rounds of research; which involved 90-minute 1:1 interview sessions and 30-minute unmoderated prototype tests. Various prototypes were used to facilitate discussion and generate insights in relation to the authentication models shown, as well as to authentication more generally.
More detail on context can be found in each of the research reports, and in Noting Paper 280 – The CX of Authentication Uplift.
Research goals
Research approach
Findings
Several recurring themes were identified and observed throughout all rounds of research. These recurring themes are significant to the overall research findings and offer valuable insights to the research project as a whole.
- Friction is multifaceted
The research found the principle of friction to be multifaceted, with factors manifesting in various ways; friction can occur both online and offline. Online friction can include extra authentication factors, and offline friction could be the requirement to switch between devices, for example. Friction can be viewed by participants as negatively or positively impacting on an authentication experience, i.e. there are ‘positive’ or ‘negative’ levels of friction in a given flow. One may hypothesise that higher levels of online friction create more frustrating experiences for users, however the research does not support this. While some participants experienced frustrations when accessing devices (such as, to receive one time passwords or access an app), they generally appreciated lengthier processes when accessing sensitive data.
- Users look for, and rely on, visual trust markers to assess risk
Consumer participants across all age demographics were conscious of the risks involved with using the internet and implement practices and habits to ensure their safety online. The research found participants heavily relied upon visual cues to determine whether a platform was trustworthy. Each research round saw an uptick in participant awareness of the potential for data breaches, and an increased understanding of scams. This may be attributed to the increase in highly publicised data breaches. Those who had been impacted by previous security breaches are proactive in their approach to online safety and actively seek out information on how to protect themselves.
- Extra authentication factors are appreciated
Across the board, consumer participants appreciated extra authentication factors even when they were not expected. Although two or more factors were expected for high-risk scenarios such as banking or health related data, participants also appreciated extra factors for actions they deemed as slightly risky. Even when a participant did not expect a second factor, they did not feel negatively toward the increased level of friction. On the contrary, participants perceived the extra layers of security as the brand or corporation’s effort to prioritise consumer privacy and data safety. Implementing extra factors provided participants with a sense of security and comfort. Research indicated that the extra factors or increased friction should be in context and relevant to the use case. A low-risk use case such as social media log in does not warrant multi-factor authentication (MFA).
- Meeting consumer expectations helps build trust
The research highlighted participant opinions on the importance of corporate responsibility in order to build trust. Participants noted they generally only create accounts out of necessity and believe more needs to be done by businesses to protect customer data. Consumer participants expect businesses to treat customer data as securely as possible, remain up to date with cybersecurity best practices to prevent hacking, never share their data with third parties, direct adequate funding to building strong back-end systems and hire talented teams. Interestingly, participants inherently placed more trust in larger and more established brands, though they recognised that their data is not guaranteed safety. Participants cited Optus, Medibank and Latitude as examples of companies whose recent data breaches have shaken consumer trust.
In order to assist consumers feel more in control, DHs should regularly communicate with their customers about data security and methods to keep accounts safe, as well as swiftly advising of any data breaches or any compromises to data.
- Step-up authentication is perceived as the norm
The research found that consumer participants expected authentication to adapt and become more rigorous as the sensitivity of their data increased, as this is what occurs in their present digital experiences and matches their mental models. Participants were familiar with risk-based step-up authentication because it is common in industries such as banking. Participants generally had a decent understanding of the requirements of step-up authentication, and the friction, or “extra layers”, present were considered positive. Step-up authentication aligned with participant expectations of security and demonstrates the importance of security measures that are tailored to meet individual user actions.
- Importance of protecting vulnerable customers
The research reiterated the importance of accessibility and protecting vulnerable customers. All users across the spectrum of human diversity should have access to robust and easy-to-use authentication methods, which match their expectations of security and take measures to protect their privacy. Both permanent and temporary disability impact how users prefer – and are able – to authenticate online. Further findings included people who can not read or write, or those with English as a second language, who may find it hard to comprehend complex information presented to them, reiterating the importance of providing alternative ways to authenticate where possible, which conform to the latest Web Content Accessibility Guidelines (WCAG).
An alarming finding from the research was the risk malicious intent poses to vulnerable users. Cases are varied in nature, but regularly involve Domestic and Family Violence, or elder abuse, particularly in communities where English is a second language. This highlights an issue far greater than the need for secure authentication, and identifies a systemic and widespread societal issue, one which secure authentication can’t solve, but can do its part in reducing potential suffering.
Outcome Summaries
Redirect with One Time Password
The research found One Time Password to be a generally well-performing authentication method. Consumers were typically familiar with the verification requirements, having regularly used the OTP model in various contexts; with banking platforms being the most frequently cited. Consistent exposure to this method meant consumer participants were confident with the flow and aware of what to do at each step, making it a fast and easy process to complete. OTP offers a level of convenience to users by removing the need to recall lengthy and complex passwords, and quickly auto-filling OTPs from SMS text messages on some newer devices. From a security perspective, consumer participants appreciated the OTP expiration window and preferred entering a one time password in place of their actual password; subsequently reducing drop-off rates. However, OTP did not match participant expectations contextually; as most participants were familiar with the model as a second factor of authentication and did not perceive it as strong enough when used as a primary, standalone model.
More detailed findings can be found in the Round 1 report.
App/Web-to-App with Biometric
The research found App/Web-to-App with Biometric to be a generally well-performing authentication method. The majority of participants tested were familiar with biometric methods of authentication and cited using them on a regular basis. The highly automated process of the App/Web-to-App flow and use of biometrics meant participants had very little information to recall or input throughout the flow. Participants appreciated the ease with which they could authenticate with this method, and although they like authenticating with biometric means, they believe it is not always the most appropriate method for sensitive use cases when used as a single factor. While the method was familiar and found to be very easy to use, many participants expected a standardised approach to authentication; with consistent and strong authentication required to access any type of data, no matter the sensitivity. Consumer participants reported greater feelings of control and confidence when more than one factor was required as a ‘confirmation’ of action. The participant expectation is that multi-factor authentication would be in place across all sectors and types of data, regardless of the sensitivity.
More detailed findings can be found in the Round 2 report.
Decoupled with QR Code
The research found that decoupled authentication was accepted in some use cases but not others. There was a strong preference to be taken to an existing, pre-installed app which had been downloaded from a reputable source, as users would have a pre-established level of trust and confidence. Consumer participants were not as comfortable with being redirected to a website in their browser, as they perceived it to carry security risks. When being redirected to a website, it was not immediately clear to participants why they couldn’t simply continue the process on the originating device (desktop in the instances tested), adding to the lack of transparency and trustworthiness. Many consumer participants had their banking provider’s mobile app installed on their phones. This contrasts with less digitally mature sectors, where the use of mobile apps is less common. As such, decoupled experiences that require switching from an originating device to an app may be more successful for the financial sector in the interim, but this may improve over time if app adoption increases in other sectors. Decoupled authentication could be supported with focus on educating users on the process, safety and validity of QR codes, and avoiding device-switching if no DH app is available. There was also a strong desire from consumer participants for extra authentication factors.
More detailed findings can be found in the Round 3 report.
Global Performance: Radial Graph
Global Performance was developed by the research team to define success for various authentication models, made up of five separate measures. Each measure consists of 3 different metrics collected throughout the research sessions. The metrics are then collated to determine a quantifiable outcome for each measure. These 5 measures are then reflected on a five-point radial graph, demonstrating the global performance for the respective authentication model.
Recall &/input | Familiarity & completion | Comfort & control | Purpose & outcome | Expectations |
Information a user needs to recall | Familiarity | User feeling in control | Benefit awareness | User security expectations |
Users’ perception of length of time | Brand influence | Awareness of next step | Sensitivity of value proposition | Perceived security |
Number of user inputs | Current authentication models | Trustworthiness | Level of positive-friction | Sector |
Overall, the authentication model with the highest performing scores was App/Web-to-App. Though, out of all five measures, App/Web-to-App scored highest in only two (Familiarity & Completion and Comfort & Control). One Time Password also performed best in two (Recall & Input and Expectations) and they both tied in one measure (Purpose & Outcome), so it was an even split of winners across the five measures.
One Time Password | Moderated | Unmoderated | Combined |
Recall & input | 3.93 | 4.08 | 4.01 |
Familiarity & completion | 3.73 | 3.39 | 3.56 |
Comfort & control | 3.97 | 3.08 | 3.53 |
Purpose & outcome | 3.97 | 3.08 | 3.53 |
Expectations | 4.10 | 3.25 | 3.68 |
App/Browser-to-App | Moderated | Unmoderated | Combined |
Recall & input | 4.02 | 3.83 | 3.93 |
Familiarity & completion | 3.95 | 3.72 | 3.84 |
Comfort & control | 3.54 | 3.94 | 3.74 |
Purpose & outcome | 3.46 | 3.60 | 3.53 |
Expectations | 3.35 | 3.77 | 3.56 |
Decoupled | Moderated | Unmoderated | Combined |
Recall & input | 3.88 | 3.63 | 3.75 |
Familiarity & completion | 3.52 | 2.98 | 3.25 |
Comfort & control | 3.48 | 3.56 | 3.52 |
Purpose & outcome | 3.39 | 3.33 | 3.36 |
Expectations | 3.62 | 3.58 | 3.60 |
Note: 0.00 to 2.99 is Bad; 3.00 to 3.24 is Poor; 3.25 to 3.74 is Good; 3.75 to 3.99 is Very Good; 4.00 to 5.00 is Excellent. Anything with a 0.40 or greater difference between unmoderated and moderated testing cohorts for a measure is considered significant.
System Usability Scale
App/Web-to-App was the best performing model when it came to System Usability with a score of 82.88, however it only marginally beat Redirect with One Time Password which scored 82.61. This difference of 0.27 points is marginal and there is not much differentiation between the usability performance between these two models. Decoupled scored slightly lower at 74.29, but this is still an above average score.
The average SUS score is 68 for technology in general (while that may indicate 68% of the total maximum score, it’s actually more appropriate to call it 50%). Usability scores of 80.3 or higher are well-performing and bode very well, scores of 68 or thereabouts are average and need some work to improve and anything under 51 is a problem and needs addressing.
All models were well performing when it came to System Usability.
Consumer Behavioural Archetypes
Consumer archetypes help segment and succinctly describe different drivers, behaviours, and needs observed through research. The archetypes have been developed by the DSB to represent common behavioural and attitudinal themes relating to data sharing. There are four identified CDR archetypes; Sceptics, Assurance Seekers, Sense-makers and Enthusiasts. Each archetype has specific needs for how authenticating to share CDR data should work to be trustworthy and comprehensible.
Interestingly, no model tested had any Enthusiast consumer archetypes. Enthusiasts are excited to get the benefits of authenticating to share CDR data and generally value simple experiences once trust is established.
Assurance Seekers were the highest represented group of consumer archetypes across all three models. Characteristically, Assurance Seekers want to read additional information. They generally value familiarity and external reference/support, and are apprehensive to new experiences. They made up just over half of all consumer participants for both App/Web-to-App and Decoupled models.
Redirect with One Time Password saw just under half of all consumer participants fit into the Sceptic archetype. Sceptics are less trusting of organisations and/or technology. They generally value control, and are averse to sharing data based on experience with current practices.
Decoupled had the smallest representation of Sense-makers. Sense-makers need to understand how the process works. They generally value details, and can trust the process if given enough valuable information. This is consistent with the Decoupled qualitative findings and consumer perceptions of authenticating with a QR Code.
Opportunities
The findings from all three rounds of research support the opportunity for a combination of step-up and waterfall authentication frameworks. Many participants were familiar with step-up authentication, and expected corporations to implement 2FA and step-up models regardless of the sensitivity of the data being accessed. This awareness and desire for tighter security may be related to recent high profile data breaches but might also indicate a general increase in data literacy and privacy awareness among consumers. The research found variables such as the authentication platform, sensitivity of data, sector and macro environmental impacts (such as data breaches) all had a bearing on consumer participant perception of the security and trust in authentication models.
No models tested were completely rejected, rather they were accepted with caveats or with areas identified for improvement. There was a clear desire for App/Web-to-App to be supported in the CDR, affording consumers the option to authenticate within their DH app. The research on Redirect with One Time Password identified several key opportunities and improvement areas and could be uplifted to continue being a supported model. Decoupled could also be supported to allow the user to authenticate securely with their known device no matter how they interact with the CDR. This may potentially see a separation of Decoupled from QR Code as the mechanism which connects the two channels.
This suggests all models tested could be supported within the CDR, as part of step-up and waterfall authentication frameworks, with clarity around how they can be implemented and adopted. A waterfall approach to authentication could be considered that supports App/Web-to-App, Redirect with One Time Password, and Decoupled authentication; and facilitate a framework of fall-back options. This approach could give consumers and DHs alike more optionality and flexibility while allowing for consistent authentication experiences.
The step-up framework should also consider Credential Level pairings, recommendations from both the PwC IC Accessibility and Independent Security Review reports, and also uplift the Redirect with One Time Password model with research.
The DSB are now working on a Decision Proposal to consult on the step-up and waterfall authentication frameworks.
Quick links to CX Guidelines: