Creating equity & cross-organizational connection through talent calibration

One of my most satisfying and impactful career accomplishments is designing and implementing a blind talent calibration process for a 500+ person multi-national team.

Mid-year reviews were approaching, and the Head of Design was keen to introduce a blind calibration talent review into the process to help ensure that the team leaders consistently evaluated their team members regardless of their office location, tenure, background, or personal relationships.

Additionally, he wanted to ensure that we were working with our HRBP (Human Resources Business Partner) to ensure the process was compliant and complementary to all official processes led by our people team.

It is also important to note the short timeline for getting everything in place.

Tools for such processes are available in the marketplace, but we did not have the time to complete the procurement processes. For this inaugural cycle, we leveraged Google Sheets and all the security features available to ensure anonymity and privacy for all team members.

The benefit of this approach was our ability to tailor the tools to meet our organization’s needs.

Career Ladders

What is a career ladder?

A career ladder is a tool for organizations to define job advancement opportunities. By offering a clear path for growth, it can boost employee engagement and retention.

Additionally, career ladders assist individuals in defining their identity and navigating within an organization.

Where does your organization land on the spectrum?

• Inefficient organizations don’t see the value in having career ladders. Without a ladder, they struggle to create equity across their organizations.
• Good organizations have well-defined career ladders that assist individuals in defining their identity and orienting themselves within an organization.
• Great organizations make good use of their career ladders as part of their ongoing practices related to people development.

Talent Calibration

What is talent calibration?

A talent calibration session is a meeting where people managers gather to discuss, evaluate, confirm, and reach a consensus on talent-related decisions.

What is blind talent calibration?

Blind talent calibration includes removing team member names, gender, and other identifiable characteristics from the talent calibration process.

This process helps eliminate bias throughout the reviews and allows people to focus on behaviors, not opinions.

“We are not calibrating people. We are calibrating our understanding of success for a given role.” — Head of Design

Designing a Calibration Process

How do we establish a blind calibration process?

1. Determine how many roles need calibration sessions and whom to invite to each session.
2. Develop a strategy to preserve anonymity.
3. Provide the tools for people managers to prepare for the session(s) and conduct the sessions.
4. Secure all tools before sharing with others and ensure that calibration sessions remain confidential.
5. Partner with your people team (HRBP) throughout.

Sidebar: Our Roles for Today

The roles we’ll be referencing today

To make things less abstract, I would like to introduce the roles we will reference throughout.

While the sample exercise is for the Design Lead role, it is essential to have the roles above and below this level to reference as needed.

Senior Designer (the role below the focus role for the session)

You are concentrating on enhancing your craft skills, assuming ownership of a feature area, or taking on team responsibilities.

You are actively developing your collaboration and communication skills and frequently working closely with a more senior designer.

Design Lead (the role that we are focusing on within the session)

You are investing in developing your craft and collaboration skills and can now impart those skills to other team members.

You are trusted to take ownership of design for your team or feature area with minimal supervision and are beginning to make an impact on the broader design team.

Staff Designer (the role above the focus role for the session)

You are a respected expert in crafts and collaboration, sought out by others for guidance and learning.

You lead the work of a significant product area and manage complex projects across multiple organizational boundaries, making a noticeable impact in the larger organization.

Number of Sessions Needed & Who to Include in Each

How many calibration sessions will you need?

You will need one session per title or role in your organization.

The invitees for a given session should include the people managers for that specific role and the most senior leadership across the larger organization.

A sample org chart for the full organization.

The same org chart with two roles dimmed, including who will not be invited to the session for Design Leads.

Preserving Anonymity

How do you make the sessions anonymous?

Create a unique identifier for each team member. This identifier should not exist in any other system.

Unique identifiers should only be shared with the person’s direct chain of command.

Master Key

The master key is the document that requires the highest level of security. Distribution of the master key should be limited to three individuals:

1. The person orchestrating all of the sessions.
2. The head of the department.
3. The organization’s HRBP.

Tip: If the orchestrator of the sessions requires additional logistical support for coordination, scheduling, etc., provide them with a copy of the file without the IDs.

The master key for the team that is having talent reviews.

Individual People Manager Keys

Which IDs will be shared with each people manager?

Each people manager would only have access to the unique IDs of those reporting to them, the people manager’s people manager would have access to the IDs for all team members in their reporting structure, and so on.

Creating, restricting, and managing all these files can feel daunting in larger organizations with multiple layers of hierarchy.

Do not fret. It is a fair bit of work to get everything set up and distributed, but once that work is complete, you will not need to revisit any of these files for the remainder of this review cycle.

Here are two examples of people manager keys: one would be given to Ethan, and the other to Mirelle.

Preparing Your Calibration Tools

What to include in your calibration tool?

Using that as the foundation, I created a calibration tool (spreadsheet) for each role within our organization with two sections. Our anchor section at the bottom of the tool leveraged content from our career ladder:

1. The role that we were calibrating on in the center column.
2. The column to the left had the role at the career stage below.
3. The column to the right had the role at the career stage above.

By including this information directly in the tool, we ensured that everyone could ensure their assessments aligned with everyone’s understanding of the role.

Tip: To prevent accidental overrides, be sure to lock the career ladder cell and assure your team that if they propose any updates, there will be a post-calibration activity to support their implementation.

Above this anchor section, we had the working section of our tool. In the far right column, we had all the Unique IDs for the given role. To its right were three column headers: ‘Below Expectations,’ ‘Achieves Expectations,’ and ‘Exceeds Expectations.’ ‘Below Expectations’ was above the column for the role at the career stage below the focus of our session. ‘Achieves Expectations’ was above the column for the role that was the focus of our session. ‘Exceeds Expectations’ was above the column for the role at the career stage above the focus of our session.

Tip: While creating your tools, some of you may prefer to use a single spreadsheet with separate tabs for each role. Before sharing the files with others, copy or move each tab into individual files for each role. Having separate files is critical for ensuring privacy through the security features available to you.

A high-level visualization of the calibration tool for a Design Led before the tool being shared.

Securing Your Tools

How do you secure your sessions & tools?

Given the sensitivity of talent calibration, it is essential that you leverage all available tools, features, and functions.

Managing the Sessions

• Document a master schedule and who will be invited to which sessions.
• Leverage all of your security features for virtual sessions.
• Have a partner who can message folks about schedule changes during upcoming sessions.

Managing Your Calibration Tools

• Keep a log of who needs access to what.
• Be diligent about folder structures and file permissions.
• Consider locking down sections of the tool so that only you can edit.
• Keep your master files extra safe.

Who has access to which tools in a session?

To help you and anyone else supporting the process, you should consider creating a visual or separate spreadsheet to ensure shared alignment and a consistent touchstone for any of the operators supporting this task.

The org chart depicts who will be invited to the session for Design Leads, with notes on what will be shared with each person.

• Unique IDs are shared with people managers based on the rules articulated earlier in this piece, each receiving their own file.
• All people managers will receive edit access to the same calibration tool for the Design Lead role.
  Reminder/Tip: Be sure to let everyone with access to the file know that it is a tool accessible by the relevant people managers and that the reference information is non-editable.
• Senior ICs (Individual Contributors) in the sessions will not receive access to any of the files.

Reminder: Ensure that confidential information, including the Unique IDs, does not appear in this documentation.

Activity Before Your Sessions

What happens before the session?

Before each session, more senior leaders with multiple people managers should set up a preparatory alignment conversation for their leadership team. These preparatory sessions will establish a unified perspective for their organization and make the larger sessions more efficient.

Leaders without people managers in their hierarchy should take time for reflection, ensuring that their thoughts are clear and ready to be shared.

Any notes about team members being discussed or reflected on should be kept outside the shared tool.

Conducting Your Sessions

How does the tool adjust for the session?

At the top of the session, you will reveal one hidden column to help capture any notes and two rows between the Unique IDs and career ladder to capture behaviors that are representative of ‘Below Expectations,’ ‘Achieves Expectations,’ and ‘Exceeds Expectations.’

A view of the tool at the top of the session with the notes column and behaviors rows revealed.

How do you set the stage during the session?

• Reinforce which role is being discussed during the session. (senior leaders in multiple sessions will appreciate this)
• Explain who is in the room and why they are there.
• Reestablish that these conversations are confidential.
• Reinforce that we are not calibrating people; we are calibrating our understanding of what success looks like for a given role.
• Remind everyone to focus on behaviors and outcomes.
• Require everyone not to use names or gendered pronouns to preserve anonymity and prevent bias.

How do you begin the session?

Before starting any dialogue about team members, screen share the calibration tool and have each leader move the Unique IDs for their respective team members to the appropriate column.

The tool after people managers have moved their respective Unique IDs to ‘Below Expectations,’ ‘Achieves Expectations,’ and ‘Exceeds Expectations.’

Calibrating on Exceeds Expectations

Starting on a positive note, you open the discussion about the behaviors of folks identified as ‘Exceeds Expectations.’

Who would like to share an example of ‘Exceeds Expectations’ behavior?

As the leader shares about the team member proposed as an ‘Exceeds Expectations,’ document the most salient points in the respective column.

The area of focus during the ‘Exceeds Expectations’portion of the session.

Does this align with everyone’s understanding of an ‘Exceeds Expectations?’

Once they have completed sharing, check in with the rest of the group to see if the behaviors shared about this team member align with ‘Exceeds Expectations.’

A skillfully facilitated conversation should result in a rich dialogue focused on behaviors and outcomes driven by people in the given role and how the leader determined these behaviors warrant an ‘Exceeds Expectations,’ folks adding additional behaviors as needed.

Does anyone want to adjust who is in ‘Exceeds Expectations?’

At this point, you might see leaders moving Unique IDs out of and into the ‘Exceeds Expectations’ column, further discussing the behaviors as needed.

Calibrating on Below Expectations

Once the ‘Exceeds Expectations’ portion of the session has concluded, the conversation moves on to the ‘Below Expectations’ behaviors that team members are exhibiting.

This portion of the conversation follows the same structure:

• Who would like to share an example of ‘below’ behavior?
• Does this align with everyone’s understanding of a ‘below expectations’?
• Does anyone want to adjust who is in ‘below expectations’?

An example of the tool after a person is moved from ‘Below Expectations’ to ‘Achieves Expectations.’

Closing the Session

The final minutes of an encounter can shape how we view the entire experience.

To close the conversation, ask the following questions and create space for dialogue.

The completed calibration tool.

How does everyone feel about where we landed on calibration?

Dialogue during this portion may not be robust, but how can we know the answer to a question we haven’t asked?

Does everyone feel good about how this sets us up for the future?

This is the opportunity to take a final look at the distribution of ratings, determine if we have the correct balance, and identify if an overconcentration of ratings might cause disappointing conversations in the future.

Reality Check: Performance ratings implicitly set expectations for compensation adjustments, year-end bonuses, merit increases, and promotions. As you reflect on where everyone landed, consider how well you can support those expectations across all dimensions.

Any feedback on the career ladder itself?

As a group, review any notes that you may have taken for the career ladder rows, adding new notes as needed.

The tool with notes added to one of the career ladder rows.

With those final questions answered…

• Thank everyone for their engagement and for helping to make the organization better.
• Hand off the continuation of the conversation to your people team partner (HRBP).

Closing the Full Talent Calibration Cycle

What do you need to close everything out?

Once you have conducted sessions for all of the roles on your team:

• Aggregate the results into a single file for the Head of Design and your HRBP.
• Revoke access to most, if not all, files used during the sessions (access can always be re-granted as needed)
• Ensure the career ladder owner has consolidated feedback about the ladder so that they can take action as needed.
• Take a deep breath and give yourself a high five for a job well done.

(un)Expected Outcomes

What did this process achieve?

We accomplished our five objectives through this process and were lucky to achieve a sixth.

1. Consistency in people’s understanding of roles.
2. Visibility into the quality of talent.
3. Greater equity across the global organization.
4. Increased trust from all team members.
5. A solid foundation for all future reviews and related activities.
6. Bonus: Greater camaraderie across a leadership team.

Many of our mid-level leaders entered the calibration guarded. As they spoke about their team members’ behaviors, they encountered curiosity and camaraderie. Where they had expected judgment, they found support and the beginnings of more substantial connections with their peers that deepened long after our sessions wrapped.

This was one of the most incredible shifts I have seen in my professional career. I am so proud of each individual’s bravery in letting go of the past and stepping into such a bright future for themselves and their teams.

Thank you for taking the time to learn about our blind talent calibration process. I hope that it will inspire you to do great things.

Thank you.

This post is based on my presentation for Henry Stuart’s Design Ops New York 2023. Here is the link to the slides for that talk.