Project: Introducing Machine learning/AI technology

IVA was Bill.com’s automation feature that stands for intelligent virtual assistant. It can do a lot of things, but mainly, it’s used to scan documents to extract data and populate forms for you, such as invoices, bills, receipts, etc.

Really cool, but getting users to understand this technology was one of the challenges at our company. User feedback was consistent in that they didn’t understand what this technology did, and they struggled to understand the concept.

My hypothesis for this is because we were trying to overexplain (I call it TechSplaining) the feature. Here’s how the previous content explained what IVA does:

What are we trying to say here? Does the user understand what’s actually happening?

There’s a few issues with the content here, the first of course is that it’s way too long. But some other questions I would have as a user would be:

• What is IVA, and what does it mean for me?

• What is “auto-save” and how is it related to IVA?

• Do I have to do anything to activate IVA?

• How long is this going to take?

• Is it really efficient if I have to read all that?

When we tested this with users, they often stated they didn’t quite understand what was going on, or what we were trying to tell them. This caused cognitive overload, so the next resulting page confused them when it either returned positive or negative detection results.

Approach

Don’t over explain a feature when a simpler explanation will do. This isn’t the time to “brand” your feature, it’s a time to give the user the simplest, easy-to-digest information while they’re uploading their bill details.

I took a more user centric approach by thinking through the user research, and answering the question of “what does this actually mean for me?”.

Revised content

The revised content is simpler, more concise, and provide just as much value and information as the previous version, without causing too much cognitive overload. The user understands that we’re simply scanning their document to auto fill their data, and that’s all they need to know about how it works.

By giving some reassurance with the time it takes to process, we’re alleviating concerns, giving them confidence that this feature is beneficial for them.

Product feedback and additional requirement

While the loading screens were simplified greatly, the PM still wanted to make sure users were aware of how the automation processing worked, so that users had better awareness of the automation process. Which presented a new problem:

Problem statement: Because the automation had a 95% accuracy rate, users needed to double check the automated fields for accuracy, as the slightest mistake could cause large issues for accounting purposes.

Solution: Provide more contextual, relevant information for FTU, and tooltips in case they need reminders for future use.

Approach: After some further thinking with my designer, we created guidance instructions to provide more contextual information at the time a new bill is created.

With this first-time user tooltip and lightbox, we provided an elegant solution to the problem. The automation feature is now more contextual and relevant where the user expects to use it.

By introducing the feature at the right time, we can create better awareness while avoiding disruption for repeat instances. For returning users, the tooltip is still there to remind users of how the automation feature works.

We also created better help articles and introduced product awareness emails so users understood how they were supposed to use the new product features on the bill creation feature.

Result

The feature went live and testing indicated that users appreciated the assistance, increasing task completion rates by ~5%. While some users struggled to understand the feature at first, once it became more commonly used, their concerns lessened as the technology also improved accuracy rates.