Drones Guide to Judging Section 10: Remote Judging – RECF Library

Resources

ADC Competition Notebook Rubric
ADC Team Interview Rubric
ADC Pro Competition Notebook and Team Interview Rubric

Note: For ADC teams, competition notebooks and team interviews are evaluated using separate rubrics. For ADC Pro teams, the competition notebook and team interview are evaluated together using the combined Competition Notebook and Team Interview Rubric, with separate scoring columns for the notebook and interview.

Overview

Remote judging occurs when either the competition notebook judging process, initial team interviews, or both are conducted remotely before the event.

Determining the judging format (in-person or remote) that an event offers requires a conversation between the Judge Advisor and Event Partner. Ultimately, the decision on the judging format falls to the Event Partner, but the Judge Advisor should be comfortable working in the chosen format. Remote judging can help better utilize volunteer resources available for the event day, but judge volunteers need to be comfortable with any additional time and/or technology requirements.

All teams being judged for an event must be judged in the same format to ensure consistency in the judging experience and to remove the potential for format-based bias from impacting deliberations. For example, if competition notebooks are submitted for evaluation via links ahead of the event for some teams, then physical notebooks should not be evaluated in person on the day of the event for other teams. For team interviews, either all teams are given an initial remote interview or all teams are initially interviewed in person. Teams that are initially interviewed remotely should not also be initially interviewed in person at the event.

Remote judging should also take place as close to the event as possible. Remote judging follows all guidelines of in-person judging. The following section highlights the key differences in the judging process when some of the judging tasks usually conducted in person are conducted remotely.

Remote judging can occur in the form of remote digital competition notebook judging, remote initial team interviews, or a combination of both, as follows:

Remote Competition Notebook Judging

Digital competition notebooks are judged remotely prior to the event.
Teams will upload links to their competition notebooks via RECFevents.org. It is not permissible for teams to be asked to submit notebooks using a method other than the RECFevents.org link or as a specific file type, nor are additional requirements to be imposed on notebooks that do not appear in this Guide.
Once a digital competition notebook link is uploaded via RECFevents.org, teams may still update their competition notebook on an ongoing basis, even on event day. Competition notebook content is expected to change over time, which is part of the process.
The event partner will provide the links for the digital competition notebooks to the Judge Advisor.
Digital competition notebooks should be freely viewable by the judges by using the link. Teams should ensure that permissions are set to allow judges to view their competition notebook.
The judge advisor will organize judges into groups to review and score competition notebooks using the applicable rubric.
Digital competition notebooks should be evaluated under similar conditions to ensure consistency.
Digital competition notebooks should be assessed by multiple judges to establish a ranking of finalist notebooks.
Some events may want to conduct a variation on this evaluation format. The overriding principle remains that all competition notebook submissions are to be evaluated utilizing the same submission format and in the same timeframe so that no entries are given any real or perceived preference or advantage.

Remote Initial Team Interviews

Initial Team interviews are done remotely before the event, using the applicable rubric.
Team participants can log into the meeting from a single location sharing a webcam or from multiple locations.
The goal of initial remote team interviews is to complete Step 1 of the Deliberation Process.
Judge advisors should establish a system for organizing and collecting judging notes to support effective final deliberations.
Follow-up interviews for final award nominees (Step 2 in the Deliberation Process) must be conducted in person to allow judges to observe both the team and their drone performance during the event.
In-person judges conducting these follow-up interviews should not move teams from one award category to another. Doing so would invalidate the initial deliberations of the remote judges and effectively restart the judging process without giving equal treatment to all teams.

Note: Remote judging does not take the place of in-person follow up interviews and deliberations on the day of the event. It is meant to provide flexibility for event partners and judging volunteers to perform some judging tasks ahead of the event day. Remote judging can allow a smaller group of judges to take advantage of the longer time frame by scheduling judging ahead of the event and can allow for the utilization of judge volunteers who may not be able to attend an event in person.

Remote Judging Protocols

All judging principles and guidelines still apply.
Youth protection must be upheld. While conducting remote interviews, each participating team should have one adult representative (18+ and not a high school student) logged in and visible on camera during the entirety of the interview. This adult representative should join the interview before any students arrive. The adult may be in the same room as the students or logged in separately to the remote call. This adult is not to participate in or contribute to the content of the team interview in any way. Their presence ensures there are multiple adult parties involved in any video meeting.
A judge should never be alone in a remote interview with a team but instead should work as part of a group of two or more judges. With the inclusion of the team’s adult representative, this puts the minimum number of adults in a remote interview at three.
Just as in-person interviews do not allow recording, remote interviews should also never be recorded by any party.
It is acceptable for remote judges to hold separate online deliberation meetings or to share spreadsheets to assist in collating judging information, such as Team Interview or Competition Notebook Rubric scores and Initial Award Candidate Ranking Sheets.
Any meeting notes or data spreadsheets should be under the control of the Judge Advisor, and the information contained in them should be destroyed at the conclusion of the event.

Scheduling Remote Initial Team Interviews

Scheduling the Remote Judging Volunteers

Interview scheduling requires coordination between the Event Partner, Judge Advisor, remote judges, and the teams. It is recommended to first create a schedule of interview times, then ensure that remote judges and the Judge Advisor are available for those times. While the Judge Advisor may not need to participate in an interview, it is highly recommended that they be on hand to help manage any issues that may arise. Additionally, if a remote judge is unable to attend or has a technology issue, the Judge Advisor can step in and serve as a remote judge so teams can be interviewed at their scheduled time.

Scheduling the Teams

Remote initial team interview sign-up times can be manually scheduled by the Event Partner, or an easier method may be for teams to schedule themselves via a first-come, first-served sign-up system. It is recommended that remote interviews be completed a few days ahead of the event in case extra time is needed due to a volunteer or technology issue disrupting the schedule.

If there are enough remote judges to support it, multiple interviews can be conducted in parallel. For example, a single remote judging link can be used with a main room for incoming teams and breakout rooms for each team of remote judges. Teams are then moved from the main room into a breakout room for their interview.

It may be helpful to have two adults (the Judge Advisor and another event staff member) greet teams in the main room as they arrive, ensure they have their adult representative visible on camera, and ensure it is the correct team for the time slot before moving teams into a breakout room to meet their remote judges. Having this “waiting room” also prevents teams from inadvertently interrupting another team’s interview.

Note: Past experience has shown that half-hour interview cycle times work well. Thirty minutes allows ample time for teams to enter the remote judging environment, for remote judges to conduct a 10–15-minute interview, and for remote judges to discuss and score the interview and fill out the Initial Award Candidate Ranking Sheet before the next team arrives.