Integrating user research with existing legislative processes

Transcript (and see slides)

We were able to integrate usability testing with the cannabis business permitting project.

But let’s set the stage. In government, usability testing is left to the end, if it’s done at all. We’re too busy building the thing that we have a list of requirements for. And then just hoping it works for the end user.

The problem is, sure we might be building stuff, but are we building the right thing? All of our hard work will be for nothing if our end user can't use it.

We can all do better than crossing our fingers. In this case, that involves usability testing during the development phase.

So, cannabis. It was legalized in the state of California for recreational use in 2016. But the question was, ok, it’s legal, now what? How do we regulate it?

Policymakers in Sacramento and San Francisco are still hashing that out to the tune of almost 200 pages of regulations.

San Francisco’s Office of Cannabis is still creating a few rules, and this is how they do it. They publish a draft of the rule on their website, so people can look at it and email them with any feedback. They call this the “public comment” phase.

In 2 weeks, they stop taking feedback, read all of it, rewrite the rule, and republish the new rule.

Having a public comment period is pretty standard for these kinds of things. And that’s where we saw our chance to do usability testing.

Here’s how it worked. The Office of Cannabis sent us a draft of the rules before they published them. I would write the questions for the form, and we would set up testing sessions with real applicants.

During the 2-week public comment period, we would do usability testing. And then we reported our findings to the Office of Cannabis as a form of public comment. It’s all feedback, right?

Especially when that feedback is video of someone doing a double handed facepalm in reaction to a question they can’t answer!

Now the thing is, I knew people couldn’t answer this question when I wrote it. It’s asking them for detailed information about who is working at their business, which is impossible to answer if you haven’t opened your business yet. And, if you’re an existing business but have more than 100 employees, answering this question would take you hours.

But it was in a rule, that was required information for the application. And so to prove that this rule wasn’t going to work, I had to prove it wasn’t working for real users.

And a reaction like this, is literal gold. This is the kind of stuff you want to see in a usability test, not after launch!

Granted, I'm still pushing whether staff info is actually needed for the permit. But the current compromise is to have them upload the org chart instead of having applicants enter in staff info one at a time.

Instead of an outright blocker, now it's a mere annoyance. An improvement, even if we're still working on it!

To get this going, we did have a few things working for us.

We worked with the Office of Cannabis very closely. Obviously, we had to get a draft of the rules. We checked the questions with them. And we had to tell them they had to change the questions after we tested.

We could prototype the form very quickly. We just switched from Salesforce to Screendoor, which by itself saved days of work building the forms. You want to be able to test and iterate as fast as you can!

We already had a list of applicants to contact for testing. We didn’t need time to recruit, we just could contact people for scheduling. And even if we could talk to 2 people, that’s still better than none!

So it is definitely possible to fit in usability testing in government, if you have the right things in place. Here’s what we did that worked:

1. Lay out the timeline. See where you can work stuff in.
2. Get the content early. See what you're working with.
3. Test and iterate
4. Help write responsive policies!

Thank you.