Judging Engineering Notebooks

Using the Engineering Notebook for Judged Awards

One of the primary missions of the REC Foundation is to help students acquire real-world life skills that will benefit them in their academic and professional futures. Following the engineering design process and creating an Engineering Notebook helps students practice and develop a variety of real-world life skills including project management, time management, brainstorming, communication, and teamwork. The engineering design process and the Engineering Notebook are used by engineering and design professionals in many different fields.

When teams register with the REC Foundation, they receive a bound Engineering Notebook that includes instructions and sample entry pages. Teams may use the notebook provided or purchase their own from VEX Robotics or most office supply stores.

Teams can also use online collaboration platforms or cloud-based services to create and maintain a digital engineering notebook. Regardless of the format of notebook that a team maintains, or the format of notebook judging (Remote or In-Person), all notebooks are evaluated according to the same criteria and rubrics.

The engineering design process is iterative. Students identify and define a problem, brainstorm design ideas to solve the problem, select an idea to design and build, test their design ideas, and continue to improve their design ideas until a solution is reached. During the engineering design process, students will encounter obstacles, successes, and failures, and learn many lessons. Everything a team does throughout the engineering design process should be documented by the students in their Engineering Notebook.

Elements of All Engineering Notebooks

All Engineering Notebooks should contain these elements.

  • Team number appears on the cover and/or at the beginning of document
  • Errors are crossed out using a single line, but are visible
  • Notebook has not been edited
  • All pages are intact; no pages or parts of pages have been removed even if they contained errors
  • Each page/entry is numbered and dated in chronological order
  • Each page/entry is signed or initialed by a student author
  • Team meeting notes are included as they relate to the design process
  • Pictures, CAD drawings, documents, examples of code, or other material relevant to the design process are permanently merged into the notebook (in the case of physical notebooks, tape is acceptable, but glue is preferred)

Elements of Outstanding Engineering Notebooks

Outstanding Engineering Notebooks should contain all of the elements listed above, as well as these additional elements:

  • Table of contents
  • Student witness and student author signatures on every page
  • Sequential entries for every team meeting
  • Descriptions of brainstorming sessions
  • Descriptions, sketches, and pictures of design concepts and the design process
  • Observations and thoughts of team members about their design and the design process
  • Records of tests, test results, and evaluations of specific designs or design concepts
  • Team organization practices as they relate to the design process
  • Project management practices including their use of personnel, financial, and time resources
  • Notes and observations from competitions to consider in the next design iteration
  • Descriptions of programming concepts, programming improvements, or significant programming modifications
  • A person unfamiliar with the team’s work would be able to recreate the robot design based only on information in the Engineering Notebook

Note: If a team is presenting their engineering notebook to judges in a language that is not common for the region, it is the team’s responsibility to provide a translated copy of their engineering notebook along with the original language version. Adults on a team may assist with the translation of the notebook, but should translate word-for-word to keep the translation as faithful as possible to the original document created by the students.

Digitally Submitted Engineering Notebooks

Digitally submitted Engineering Notebooks may take any form that is accessible to judges by clicking a link without requiring any special software other than a web browser or PDF reader. Notebooks submitted in this format will be referred to as “Digital Engineering Notebooks,” regardless of their original format. A physical notebook that is scanned as a PDF and uploaded electronically meets the definition of a Digital Engineering Notebook, as well as a document that is created and stored in a cloud-based service. These are only two examples of the many acceptable Digital Engineering Notebooks.

In the instance when a team has a Digital Engineering Notebook and is attending an in-person event with physical notebooks being judged, the team must print out their notebook prior to the event and submit a physical copy.

If digital engineering notebooks are being evaluated ahead of the event:

  • Submission deadline is at the discretion of the Event Partner; suggested submission deadline is one to two weeks before the event.
  • The Event Partner must clearly communicate the submission deadline.
  • The Event Partner must clearly communicate the form or method to be used by coaches to submit team information and the Digital Engineering Notebook (e.g., via a Google Form or email).
  • The Event Partner should request the following information from each team during submission:
    • Team number and team name
    • Organization name
    • Primary contact name, email, and phone number
    • Secondary contact name, email, and phone number
    • Event name and event code as shown on the event website at Robotevents.com
    • Link to the Digital Engineering Notebook
  • The Digital Engineering Notebook shall be sent as a link, not as an attachment, by the team’s Primary Contact listed in RobotEvents.com.
  • It is the team’s responsibility to ensure the link is accessible. Additionally, it is the responsibility of the team to ensure their submission is legible.

Judging_a_notebook.jpg

Judging the Design Award: Sorting and Scoring Notebooks

Engineering Notebooks are evaluated by Judges to identify teams with outstanding submissions. These teams will be contenders for the Design Award, and by extension, the Excellence Award. The Judge Advisor may assign one Judge Team as the Design Award Judge Team, with the primary responsibility of determining contenders for the Design Award.

Judges should recognize that Engineering Notebooks will improve during the season, and that early season Engineering Notebooks will include less of the iterative design process. However, even early in the season, notebooks should include documentation of the robot up to its current state in the design process. Judges should also remember that younger students communicate their ideas differently than older students, and that when judging the notebooks they should consider an age-appropriate level of review.

The following process should be used to evaluate all Engineering Notebook submissions.

  1. Perform a quick scan of all the Engineering Notebooks and divide them into two categories. The Engineering Notebook Rubric should not be filled out during this quick sort, but the first three rubric points can be used to guide the sort.
    • Developing Engineering Notebooks contain little detail, have few drawings, and are not a complete record of the design process. These are usually turned in by new teams without a fully developed robotics program. These teams will not be contenders for the Excellence and Design Awards at this event. To save the Judges' time, the Engineering Notebook Rubric will not be completed for these teams. However, their Engineering Notebooks should be retained for consideration of other awards.
    • Fully Developed Engineering Notebooks contain much detail, include detailed drawings, include tests and test results, include solutions to problems the team encountered, and are a complete record of the design process. These are usually turned in by teams with a fully developed robotics program and a strong emphasis on the design process. These teams may be contenders for the Excellence and Design Awards. These should be roughly the top 10 teams or top 30% of teams (whichever is larger).
  2. Complete the first page of the Engineering Notebook Rubric for the Fully Developed Engineering Notebooks and divide them into two categories. The Engineering Notebook Rubric MUST be used for this evaluation, which is intended to narrow the set of notebooks to only the top contenders.
    • Intermediate Engineering Notebooks are a complete record of the design process, but lack some details and are not as complete as Outstanding notebooks. These teams will not likely be contenders for the Design Award at this event.
    • Outstanding Engineering Notebooks  will be a complete record of the team’s engineering design process, and contain all or most of the advanced elements listed earlier in this article. Outstanding notebooks may be the top 5 teams or top 20% of teams (whichever is larger). A Judge Team shall interview the teams with Outstanding Engineering Notebooks, and shall complete the Team Interview Rubric immediately after the interview.

5-Point Bonus

The 5-point bonus for submitting a bound notebook has been removed and replaced with a 5-point bonus for paper, printed, or digital submissions which contain evidence that documentation was done in sequence with the design process. This change to the rubric is meant to allow judges to award the 5-point bonus for submissions that provide appropriate evidence of sequential work irrespective of the notebook submission format.

Examples of this include 1) signed and dated entries in a bound notebook, and 2) time stamps generated by digital collaboration platforms. The Judge Advisor will use reasonable discretion to determine whether evidence is sufficient to warrant the bonus, and will apply that standard consistently to all submissions at the event. When there is doubt or question regarding evidence, the benefit of doubt should be given to the team.