How feedback is shaping the estimator

Project:

Posted on:

2023-12-12

Last updated:

2023-12-12

There are many ways to do usability research and use insights to improve client experience. In our beta phase, one of the ways we’re doing this is by collecting feedback from people who try the Old Age Security Benefits Estimator through a survey.

About 450 visitors have sent responses. This has allowed us to analyze feedback and prioritize changes that are important to our clients. We can see what works and what doesn’t in order to refine the estimator based on their needs.

Reviewing feedback

Every week, our product team meets for “Feedback Friday” to sort through all the new survey responses. We look at the ratings and comments people shared with us about their experience.

We then organize the survey responses by topic (this is called “affinity mapping”). This gives us a wide view of problem areas and their progression over time. If we see the same comment come up a few times, we know we should take a closer look at what we can do to resolve the issue.

Using feedback

The estimator has proven to be user-friendly, but that didn't mean it was serving everyone’s needs. In fact, the initial feedback suggested there were specific things we needed to fix. Below, we show how we’ve used feedback with examples inspired by real survey responses.

I didn’t like having to change my birth year to get an estimate

We now generate results that give future estimates to those who aren’t eligible yet.

It should show the benefit to deferral if I start to receive after 65

We now automatically show a personalized deferred amount to everyone older than 65.

Having access to feedback and being able to make quick updates has allowed us to add features like these and improve where you told us it matters most. We still have a lot of work to do and can’t address every pain point. But by grouping the feedback by topic, we can identify the most common concerns and prioritize solutions.

Here are examples of comments that represent common feedback themes and how we plan to address them:

The income question isn’t clear

Our new question will clarify the types of income to include and calculate how much of your work income is exempted.

I wasn’t given an estimate

We’ve already added $0 amounts to estimates to remove ambiguity. We’re also going to be changing the look of the results to make information easier to find.

Measuring success

After we implement solutions, we’re able to tell if an issue has been resolved through comments and ratings. If we’ve made the right improvements, we stop seeing the issue mentioned, and the ratings start showing positive trends. This allows us to measure the success of our new features and make sure that we’ve improved our clients’ experience.

For example, we can look at our data from before and after the 2 initial fixes mentioned above. If we compare survey ratings from July to those in October, we see that:

20% more people felt that the tool provided the information they needed
15% more people said that the tool made them more aware of the benefits available to them

To make sure the estimator stays user-friendly as it evolves, we’re also tracking its ease of use, which has stayed roughly the same at 80%.

By looking at comments and analytics together, we can see how the changes were received and which pain points are resolved.

What we’re doing next

We’ll keep monitoring our success indicators as we release updated versions of the estimator. In the meantime, keep sending us comments about your experience. We’re listening!