Overview
Award deliberation is the last step in the Judging Process. In this step judges will work with the judge advisor and one another to select candidates for each award, and create a plan of action for gathering any follow-up information for final decisions.
Award Deliberations involve comparing teams to one another. The integrity of the judging process depends on all judges being able to speak candidly during this process. What transpires during deliberations is particularly sensitive information. Therefore, all judging deliberation notes and conversations need to be kept confidential during and after the event.
The Competition Notebook Rubric and Team Interview Rubric are tools to assist with deliberations. A team’s score, whether a specific line-item on a rubric or the overall score, is a data point that the judges and judge advisor can use as a part of the process. It is not a replacement for qualitative judgements in the deliberation process.
Step 1 – Award Nominations from Each Judge Group
After each group of judges has completed their assigned interviews, they should identify one or two teams from their group as strong candidates for each applicable award. It is not necessary to nominate a team for every award. Once selections are made, judges should return to the judges’ Room to share their nominations with the full judging team and the judge advisor. A common method is to write team numbers on sticky notes and place them under printed award titles on a wall or board, allowing all judge groups to view and discuss the nominations collaboratively.
Award Description sheets can be printed and used to help visually organize judge input/candidate teams during deliberations. Color coding can help keep the nominations from each Judge group organized (see picture below).
The end result will be a shortlist of nominations for each award from all judge groups. When there are many award nominations for each award, the judge advisor may ask judge groups to withdraw weaker candidates from consideration, based on brief arguments for and against each nomination. Or a judge group, upon considering the merits of other candidates, might withdraw their nomination for their initial candidate.
Step 2 – Follow-Up Interviews for Award Nominees
This step should be completed before the end of Alliance Matches. The Judge Advisor will organize Judge groups to gather further information to validate the award nominees. This may take the form of observing Solo Matches, Alliance Matches, and behavior in the pits, as well as conducting follow-up interviews with award nominees. The goal is to determine a final ranking of nominees for each award being presented.
For follow-up interviews, it is recommended that the nominees are interviewed by judges that have not interviewed them previously. If possible, group judges who share an area of expertise to evaluate particular awards. For example, judges who have a background in programming / computer science would likely be best qualified to evaluate the finalist nominees for the Coding Award in ADC and the Coding Systems Award in ADC Pro. This guidance specifically differs from initial interviews, in which judges with similar expertise should be assigned to different judging groups with the intent of giving all teams a more well-rounded initial assessment.
Teams should not be told which awards they are being considered for, as this violates the confidentiality principle outlined in the Guide to Judging.
Step 3 – Final Ranking and Nominations
The next step is the final deliberation for each award at the event. This should be completed shortly after the beginning of Elimination Matches. Quantitative data (Alliance/Teamwork Rankings and Solo Missions Rankings) are needed for deliberations for certain awards and can be obtained from the event management software, Competition Control/C².
After follow-up interviews are conducted, the judges who conducted the follow-up interviews should be allowed to deliberate and create a ranking among those teams. It is a best practice to have first-choice award nominees, plus three or more additional alternate candidates.
If information comes to light that a team may have violated the Code of Conduct or Student-Centered Policy, either by judge observations or from Field Notes to Judges, that team’s consideration for the judged award should be scrutinized by the judge advisor. If there is found to be merit in that information, the award is given to the next alternate team in the award nomination ranking.
In the case of the All-Around Champion Award, the winner should come from the list of Communications Mission Award nominees/finalists for ADC and from the Engineering Design Award nominees/finalists for ADC Pro that meet the criteria for Performance Awards and other Judged Awards. Moving a team from being a Communications or Engineering Design Award finalist to All-Around winner may result in a reshuffling of winners for other awards to ensure that no team earns more than a single judged award at the event. The Judge Advisor should reconcile award winners to ensure that each award winner earns the highest award at the event for which they are eligible. Having three or more ranked candidates for each award is very helpful in this situation and eliminates the need for additional deliberations.
NOTE: Because the Aerial Drone Competition is a developing program, there may be rare cases where judges cannot identify a team that meets all of the key criteria for a specific award. If this occurs, the judge advisor should contact the RSM and will need to complete the REC Foundation Regional Support Request: Event Anomaly Log. judges should then deliberate and use their best judgment to identify the team which best represents the intent of the award.
Step 4 - Entering of Award Winners into Competition Control/C².
After award winners have been determined, the judge advisor should inform the event partner that the process is complete. The winning team numbers are entered in the Award Winner Selection of Competition Control. It is recommended to print the Award Summary Sheet or Award Script Reports, so the judge advisor can double check that all award winners have been entered correctly.
Step 5 – Collection and Treatment of Judging Materials
Prior to the award ceremony, the judge advisor should secure the Judges’ Room, including collecting all notes, rubrics, and ranking sheets, and erasing any whiteboard notes. Judges should not retain copies of any notes that reference individual teams, including rubrics or award ranking sheets. If pictures of teams or robots were taken, judges should delete them.
After the event is over, the judge advisor should destroy all judging materials off-site. These items are not to be given to the event partner for destruction.
Continue to Next Section