Positive Change by Design
Back to top

Share This

NSW Department of Finance, Services and Innovation

Using artificial intelligence to better guide users and lighten the load on a call centre under pressure

NSW Department of Finance, Services and Innovation

The NSW Department of Fair Trading is a sub-division of DFSI responsible for the administration of consumer protection laws in NSW. They aim to ensure consumers and business alike are treated fairly and equitably in transacting between themselves. As such, a large component of what they do involves informing consumers of their rights and facilitating dispute resolution between the parties.

The Department was interested to explore the use of artificial intelligence (A.I) e.g. chatbots, to ease the burden on its call-centre and facilitate deeper browsing of its website, more relevant content and resources. After running three years of call transcripts through IBM Watson’s AI Suite, we settled on ‘Rental Bond’ disagreements as a topic area to test the use of such technologies. We set ourselves a goal of rapidly iterating and having a prototype in market within 3-4 weeks.

The volume and quality of work that we’ve collectively delivered, in such a short time, is incredible. Why aren’t we working in this manner all the time?

Peter Kostantakis - DIRECTOR, Better Regulation Division

We began with stakeholder workshops involving both internal and external users groups. We explored a number of key areas, including anticipated use cases, implications these would have on key features of the chatbot and particular areas to watch out for – for example, where the legislation is not clear and the content does not provide clear remediation for a particular situation.

This translated directly into a detailed conversation map, where we clearly curated conversations to lead specific users to useful information. This conversation map fed straight into the widely used Facebook chat framework and we ideated improvements over a three week period. We involved both consistent and new user testing cohorts each week to ensure we weren’t ignoring intuitive design requirements. The resulting chatbot was launched within the four-week deadline and is now in market for full testing and rollout across a number of key department content areas.

Considering Tone of Voice & User Interface Design

Deepend NSW Department of Finance, Services and Innovation

Meet Holly, the Rental Helper

A lot of work went into defining a detailed persona for the Chatbot with us ultimately landing at ‘Holly - The Rental Helper.’ Everything from Holly’s design style to the tone of voice was thoughtfully considered in order to present a more useful and pragmatic Bot.

Colour Palate/Iconography

Colour Palate/Iconography

Developed to signify particular types of content, within the confines of a Bot we needed to rely on a bold colour palette and iconography to communicate core genres of content, reasonably and quickly.

Deepend NSW Department of Finance, Services and Innovation

Framework Determined by User Familiarity

We assessed a number of chatbot frameworks, but ultimately settled on the Facebook Messenger framework (as people already have a certain level of comfort with the way it works).

Leveraging a Familiar Framework

Immersions in 3D

Immersions in 3D

We chose to film the initial stakeholder workshops with a 3D camera, which allowed us the opportunity to revisit, and fully immerse the wider team, in the research.

Lookback User Testing

Lookback User Testing

We used the Lookback tool to capture both test users facial expressions, as well as their mouse movements. This added another level of comprehension to assessing usability, allowing assess the users' expressions at particular difficult moments of interaction.

Inspiring Content Structure

Deepend NSW Department of Finance, Services and Innovation

IBM Watson used to assess 3 years of Call Transcripts

IBM’s Watson was used to process 3 years of call centre data and assisted in choosing rental bond disagreements as an area where AI could drive significant efficiencies, especially when encouraging more people to source information online.

Deepend NSW Department of Finance, Services and Innovation

Content Structure Informed by Data

By analysing breakpoints in our user testing, we realised that the personalised tone of voice we had developed was encouraging users to thank the Bot for its utility. This was in turn, broke content flow and was quickly rectified.

Results Listing

In-market in 3 weeks

It might have been ambitious, but we went from concept to in-market testing of a prototype in just 3 weeks!

We created a more positive experience

Over 85% of test users said that the prototype contributed to a “more positive experience” of the Department of Fair Trading website

Call centre reduction

We are now in extensive conversation mapping across a number of key DFT content areas, with an overarching goal of reducing the load on the call centre