Robotics Guide to Judging: Judging Roles & Timeline

Section 3: Judging Roles & Timeline

3.0 Overview

This section describes the roles and responsibilities of judges, judge advisors, and event partners in the judging process to ensure a consistent judging process at all qualifying events. Coordination between the event partner, the judge advisor, and judge volunteers is crucial for the judging process to operate smoothly and effectively. 

An effective judging panel is made up of a judge advisor and judges that bring a wide range of background experiences, knowledge, and perspectives. 

Preparations for judging should start during the initial stages of event planning. The size of the event, the number of awards given out, the event agenda, and volunteer recruitment all impact the judging process. In the case of tournaments, judging should conclude on the last day of competition. In the case of leagues, judging must occur close to the date of league finals.

Exceptions to the volunteer age rules in this section should be rare, and require approval from the RECF RSM.

3.1 Judging Attire

All judging volunteers should dress appropriately for the role, such as wearing comfortable footwear and venue-appropriate attire. Judging volunteers should avoid wearing any team-branded clothing or accessories. Some events provide apparel or accessories to identify judging volunteers.

3.2 Managing Conflicts of Interest 

As described in section 2.5, it’s important to avoid conflicts of interest as much as possible. Due to the sensitive nature of the RECF judging roles, those roles should be filled selectively rather than by a public sign up or walk-ups if possible.

Event partners should recruit judges from a variety of sources in order to  mitigate conflicts of interest. When judges have a wide range of backgrounds, perspectives, and knowledge that they bring to the judging process, the end result is a better judging experience for both volunteers and teams.

Note: Any event that qualifies teams to the RECF World Championship should seek out judge volunteers who do not have any direct conflicts of interest with any team at the event (for example: parents or other family members associated with attending teams or team coaches would be considered to have direct conflicts of interest; this is not an exhaustive list). This is highly recommended for all events.

3.3 Judge Advisor Role Description and Requirements

The judge advisor (JA) is a volunteer who organizes and oversees the overall judging process at an event. They often help the event partner (EP) recruit volunteer judges before the event, and they train volunteer judges before and/or at the event.

• Facilitates deliberations and delivers final award winners to the EP
• Has zero or minimal conflicts of interest with teams at the event
• Is a certified RECF judge advisor for the current season
• Must be at least 21 years old and not part of a team competing at the event
• Assists the event partner with handling Code of Conduct and student-centered issues as necessary (see section 2.12 for more details)
• May serve as the “lead” JA help train a newer JA

3.3.1 Qualities of an effective JA

• Experience with the RECF judging process
• Confident, respectful, and professional leader
• Able to teach processes, then delegate and supervise the process
• Able to set their own opinions and preferences aside during deliberations

3.3.2 Pre-event tasks for JAs

• Become a certified RECF judge advisor for the current season
• Help recruit judges, considering conflicts of interest
• Train judges before and/or at the event
• Work with the EP to create a tentative schedule for judging
• Determine how team notebooks will be collected & managed
• Ensure that the EP will supply printed judging materials
• Coordinate & complete remote notebook judging before the event if applicable
• Coordinate & complete remote initial interviews before the event if applicable

3.3.3 Event-day tasks for JAs

• Train judges before and/or at the event
• Review the judging process and answer any questions
• Receive submitted engineering notebooks for in-person notebook judging
• Coordinate in-person notebook judging if appropriate
• Ensure judges sign in on the Judge Volunteer Check-In Sheet
• Group judges and assign each group a subset of teams to interview, managing potential conflicts of interest and separating judges with existing relationships; the judge advisor should not participate in team interviews unless a personnel shortage requires it
• Monitor team interview statuses and the event agenda to ensure judging is completed in time
• Collect Field Notes to Judge Advisor from event staff before final deliberations
• Guide deliberations for judged awards, including ensuring no team earns more than one judged award
• Record the results of all judged awards and communicate the list of award winners to the event partner and/or tournament software operator
• Have the tournament software operator print the award scripts for the award ceremony if needed
• Maintain confidentiality of judging deliberations and discussions
• Collect all judging materials and destroy them after the event to ensure confidentiality
• Delete copies of digital team notebooks, if applicable
• Return physical notebooks to teams, if applicable

3.4 Judge Role Description and Requirements

A judge is a volunteer who reviews teams’ engineering notebooks and/or interviews teams about their processes, then deliberates with other judging volunteers to identify award candidates and select award winners.

• Evaluates teams and/or notebooks to determine eligibility for judged awards
• Must work with a partner or group when interacting directly with students
• Meets the age requirement for the age level and program at the event:
  • RECF Inspire (collegiate) - Must be at least 21 years old
  • RECF Achieve (MS/HS) - Must be at least 20 years old and not an Achieve team member
  • RECF Engage (ES/MS) - Should be at least 18 years old; may be 16-17 if paired with an adult volunteer and youth protection is considered
  • RECF World Championship - Must be at least 21 years old

3.4.1 Qualities of an effective judge

• Has zero or minimal conflicts of interest with teams at the event
• Background in technology or robotics helps when evaluating technical awards
• Has passed some or all of the RECF judge training
• Able to work respectfully and professionally as part of a team
• Able to work with their peers to create a positive student experience
• Can consider the perspectives of others during discussions and deliberations

3.4.2    Pre-event tasks for judges

• Review the game description or video
• Communicate potential conflicts of interest to the judge advisor
• Review the one-page judging overview, rubrics, and award descriptions
• Complete some or all of the RECF judge training courses if possible

3.4.3 Event-day tasks for judges 

• Participating in judging processes as assigned by the JA, including:
  • Evaluate teams’ notebooks using the Engineering Notebook Rubric
  • Interview teams for judged awards
  • Observe teams in competition
  • Present awards to teams during the award ceremony
• Communicate potential conflicts of interest to the judge advisor
• Work with the JA and other judges to decide on award winners
• Hand in all judging notes and rubrics to the judge advisor at the end of the event
• Delete copies of digital team notebooks if applicable
• Maintain confidentiality of any judging deliberations and discussions

3.5 Event Partner Role Description and Requirements

An event partner (EP) is a volunteer who runs an event using the guidelines in RECF’s policies and game manuals. They oversee the planning and operation of the entire event, including volunteer recruitment and providing support for the judges and judge advisor.

• Is a certified RECF event partner for the current season
• The EP cannot also serve as the JA or a judge at event
• Event partners may not recommend or assign judged awards to any team
• Handles Code of Conduct and student-centered issues as necessary with the support of the JA and head referees (see section 2.12 for more details)
• Must be at least 18 years old

3.5.1 Pre-event judging tasks for EPs

• Recruit a qualified JA
• Recruit enough judges to ensure interviews can be completed before alliance matches end (two judges for every 8-10 teams at the event is a good guideline) 
• Work with the JA to create a tentative schedule for judging
• Work with the RECF RSM to ensure that all required awards are listed on RECFEvents.org
• Make or purchase trophies/certificates for awards
• Identify a secure, private, quiet room with adequate space for the judging panel to discuss and deliberate (the judges’ room) 
• Ensure that the judging panel has appropriate judging materials, including clipboards, pens, highlighters, sticky notes, copies of current judging documents such as rubrics and note-taking sheets, and other needed items

3.5.2 Event-day judging tasks for EPs

• Label the judges’ room to prevent others from entering, and ensure the JA has access
• Ensure the judging panel has all needed materials
• Communicate any team or schedule changes to the JA
• Oversee entering of awards into the tournament software if needed, and ensure that no team will receive more than one judged award
• Work with the judging panel to present awards to teams during the award ceremony

3.6 In-Person Event Timeline Example

The table below is an example of how the in-person judging process might operate in parallel with the rest of the competition schedule during a one-day event. Judges should generally begin to interview teams as soon as possible. Engineering notebook evaluations and follow-up interviews should be concluded by the end of alliance matches at the latest. 
 

Continue to the next section, Guide to Judging: Awards