How regular feedback improves service delivery at ISED

Two years is a long time – a lot has happened. Since publishing the blog Little and often: making critique a daily practice, CDS’s content design critique ritual has evolved and been adopted by other teams in the GC. 

Last summer, CDS ran a critique workshop with teams at Innovation, Science and Economic Development Canada (ISED). The teams at ISED have started their own critique ritual – a structured practice of regularly sharing work with colleagues for feedback, with the goal of service improvements.

To help other GC teams adopt critique practices too, Kivi Shapiro (Senior Analyst, Design and Experimentation Centre of Expertise at ISED) and Christina Valentino (User Experience Designer, Measurement Canada at ISED) are sharing their experiences. 

The work we do at ISED

The Design and Experimentation Centre of Expertise (DECoE)

I’m Kivi Shapiro – a Senior Analyst at ISED’s Design and Experimentation Centre of Expertise (DECoE). My team works with other ISED teams (our clients) who want to improve their existing service offerings and/or are looking for guidance with new ones.

We help in 2 ways:

1. User research and design: We perform research with the people who are or will be affected by a product or service to identify their needs, and we design and test user experiences that meet those needs. By researching and identifying potential or existing obstacles with actual end-users, we can suggest tangible ways to improve.

2. Experimentation support: We work with our clients (public servants running ISED services) to design research experiments on their work. We also run experiments that are brought to us. For example, teams can bring different options to us and we help test them to advise on which will perform better. This can be as high-level as comparing strategy options, or as specific as comparing email versions.

Measurement Canada

My name’s Christina Valentino and I’m a User Experience (UX) Designer at Measurement Canada (part of ISED). I work on internal applications and am the only UX Designer on my team, so it’s very helpful being able to bring our brains together for critiques on my work. 

Measurement Canada plays a vital role in ensuring the accuracy of measured goods in Canada (such as electricity and gasoline). We enforce and develop Canada’s measurement accuracy laws: performing inspections on measured goods, investigating complaints of suspected inaccurate measurement, and approving measurement devices (such as electricity meters and gasoline pumps).

I’m part of Measurement Canada’s Digital Office. We’re responsible for developing, maintaining, and improving applications, such as the Online Reporting Application (ORA). My team works on all things product development – including user research, planning, prototyping, testing, and packaging it all up for development. Our Digital Office is the central point of our application’s users, support team, and development team, so we can support Measurement Canada in improving their services to better meet people in Canada’s needs.

Adapting CDS’s critique ritual for our needs

We found CDS’s content critique ritual to be very applicable to our work, so we organized our own critique sessions for service designers at ISED. Some days we’re looking for answers to general questions, and other days we have work to show, like prototypes.

Setting our schedule

We figured we’d use CDS’s model for critiques to start our own ritual, since we know it works for them. Participants meet for 15 minute critique sessions at 9:45 am, with designated days for presenters. We’re a smaller group, so we decided to start with 2 days a week vs daily (DECoE presents on Mondays and Measurement Canada on Tuesdays). While we have designated days, we’re also flexible with swapping when needed. 

For the DECoE team, we’ve set our critique ritual schedule to work with our other existing team rituals, like Scrum sessions and workshops. For example, if we’re planning on doing usability testing on Wednesday, we’ll bring the mockups to critique on the Monday before. This makes us pause to make sure we’re on the right track and course correct if needed, before sharing with research participants (improving what we show them).

Plans for growth

It’s refreshing to get different perspectives when presenting. When people ask questions, it helps us re-evaluate our work, so we can change for the better. Outside views can really help, especially when you’ve been working on something for so long or by yourself.

Currently, only teams at DECoE and Measurement Canada have committed to presenting at our critique sessions, with other teams participating occasionally. We’d love to grow our critique ritual and add more sessions for designers from other ISED teams to present at (we have 3 more spots to fill)! If your team at ISED is interested in participating or if you’re a public servant wanting to learn more about our critique ritual, please reach out.

How critiques are improving our work

More informed decisions

It’s harder to make assumptions about your work when people are questioning your reasoning for decisions. Critique sessions challenge you to make informed decisions – you need to be able to explain why you chose things, resulting in more intentional work.

Critique discussions benefit everyone

I can’t think of a single critique session I’ve been to where nobody benefitted. Either the presenting team or folks on the call have learned, or both. It’s always a useful session – for its own merits and it’s also a good tool to help us all grow together as designers. There’s always something that we’re discussing.

Example from one of ISED’s critique sessions

Right now at DECoE, we’re working on a project with the people in charge of spectrum management and telecommunications. These are the folks who regulate Canada’s airwaves, including radio signals, TVs, cell phones, etc. We’re helping them design a new, public-facing service that enables individual businesses (like universities or factories) who meet their rules to choose a network that’s just for them (like a 5G network). This removes interference and improves the signal quality of the network.

Since many potential users of this service won’t have technical experience with spectrum, the application form for spectrum licenses needs to be easy to understand and complete. In the form we’re designing, applicants can select one of two power levels for their cellular network: low power or medium power. Low power transmitters can be used in license areas up to 15 km², while medium power transmitters can be used in license areas between 75 km² and 165 km² (any other size of license area is invalid, but people can apply for multiple smaller licenses that cover the desired area).

The step where we ask applicants about the power level is step 3 of 6, and on the following screen, step 4 of 6, we ask them to draw their desired license area on a map. Since the map page has data validation, if the user draws an area that is too large or too small for the power level they chose, then they get an error message. 

When we brought the design to a review session, one of the participants pointed out that we didn’t have any kind of size reminder for the applicant before they started drawing. This was a violation of Jakob Nielsen’s Usability Heuristic #6: “Recognition rather than recall”. They suggested we add a reminder to the page’s instructions, which we did. It’s a small change, but one that has the potential to save people time.

Takeaway tips!

Critique rituals can be really helpful when they’re designed to meet your needs. 

Below are tips from public servants at ISED and CDS to help teams get the most value out of the time:

1. Start with a low-lift. The hardest part of adopting critique rituals is starting. Get into the routine by scheduling just 15 minutes and go from there.

2. Spread the workload. There can be a lot of administrative work involved in planning critique rituals, especially for larger groups with rotating presenters. It reduces the lift when we work together, such as rotating leads in charge of scheduling.

3. Come with curiosity, not assumptions. Don’t assume things have to be done in certain ways. Instead of making statements, ask questions about the service and its design to nurture creativity and explore improvement. 

4. Be prepared to receive or give feedback in psychologically safe ways. Presenters should communicate what they’re looking for feedback on (like on a specific feature or colours), as well as who their audience is and what their desired outcome(s) are. Presenters can either take notes themselves or ask a colleague to (to help with focusing on the presentation and getting valuable insights on the work). Participants should talk about the work and not the person, being constructive, honest, and respectful in all feedback (positive and negative).

5. Respect participants’ time. Pay attention to the clock so you don’t regularly go over the allotted meeting time (occasionally it’ll happen when having valuable critique discussions). If the meeting is going over time, the presenter can let participants know it’s optional to stay and ask remaining questions or give feedback.

Want to start organizing a critique ritual on your team? Reach out to us if you have questions.