| Table of Contents | The
NASA Academy Independent Assessment Report Executive Summary
The purpose of this independent assessment is to determine what characteristics exist at the four Academies that have produced positive and negative results, and to objectively assess those results to determine what course corrections are needed for a successful Academy program. Several methods were employed in the conduct of this assessment. First-hand reports of the Academy experience were gathered through an extensive 33 question survey of the NASA Academy sent to all 307 alumni of the NASA Academy programs at all four Centers. A total of 164 responses were received from this survey. A total of twenty one-on-one interviews with various Principal Investigators (PIs), Academy staff, academic deans, affiliates of the Academy, and Space Grant Directors were conducted. Most interviews were in person, some via telephone, and a few via e-mail. General documentation from each Center that hosted a NASA Academy was used to gain insight into the differences between each Academy program including variations in content, style, staff, and program execution. Ten of the Academys higher-level processes, features, and program elements were evaluated in this assessment. The unfortunate nature of this type of assessment is that the very positive elements are sometime not manifested or dwelt upon. Specific positive and negative aspects of the Academy program are given in the body of the report along with recommendations to further improve the program. These recommendations are summarized below. Application and Selection The
application and selection process varies from Center to Center. It
was the conclusion of this assessment that the Ames program had the most
effective application and assessment process and should largely be used
as a model for all other Academy implementations. The
specific recommendations of the assessment team include: The
Academy should consider additional ways of getting the word out about the
Academy such as representative organizations, e-mail lists, newsgroups
and communications venues within academia and the aerospace industry. The
NASA Academy should have a presence at the annual Space Grant Directors
meeting and provide a summary of the recent summers program and remind
the directors of the types of students that the Academy desires. The
selection panel should consider how the group they are selecting will interact
as a group in addition to evaluating the applicants individually. The
Academy should accept only students who have completed at least their junior
years and have not progressed beyond their second year of graduate school. The
Academy should select only applicants that have a demonstrated interest
in the aerospace program. The
NASA Academy should only select those students who want the entire Academy
experience. The
NASA Academy should develop a more streamlined web presence. In
addition to an online application, it should include links to quotes, audio,
and even video clips of alumni describing their Academy experience. Consider
modeling the website after some of the more progressive university websites. The
application process should be modified to include interviews of finalists by
the alumni and/or the selection panel. The
NASA Academy Alumni Association (NAAA) should be strongly encouraged to
recruit students that are right for the Academy. Laboratory WorkThe
PIs supervise the RAs research work during the summer. The
best results were reported by students whose interests matched their PIs
research project. PIs requested
more predictable time in the lab for the RAs and they felt they were not
always well informed about the Academy and its expectations. Specific
recommendations include: The
Academy should dedicate at least 60% of RAs time to their research, during
scheduled, predictable hours. The
Center selection panel should identify highest-potential Academy applicants
and distribute only those applications to PIs for their review. The
Academy staff should screen prospective PIs. The
Academy should hold an orientation session for PIs prior to the start of
the summer. A
senior Academy staff member should be assigned to be the interface between
the PIs and the RAs. At least
one progress meeting should be held during the summer with the staff member,
the PI, and the RA. Each
PI should be encouraged to schedule regular meetings with their RA to discuss
progress on the research project. SpeakersThe
Academy features an extensive menu of exclusive speakers that have helped
to make the Academy more than just another internship program. Each
Academy should offer some of its experts in an informal, interactive format. Field TripsField
trips have had a tremendous impact on exposing the RAs to the inner working
of NASA and the aerospace program and they have contribution to the group
bonding process. This assessment determined that field
trips should remain a critical component of every Academy experience. Group ProjectThe
group project is a valuable teambuilding and research effort. In
the past, at many Academies, the group project has been undermined by logistics,
including the size of the group. Therefore,
this assessment recommends that the Academy staff provide some ideas for
the group project to the RAs and then give the RAs a deadline for selecting
their project topic. LeadershipAlthough
the Academy is of a short duration and there is a lot to accomplish, there
has been a continued effort to provide some kind of leadership development
experience. This assessment
concludes that a more dedicated effort should be made to introduce leadership
experiences to the program. Specific
recommendations include: The
Academy should offer speakers versed in effectively implementing new initiatives
in their organizations. The
Academy should host a motivational speaker who links leaders attitudes
with their effectiveness. The
Academy should sponsor a two-hour seminar in which volunteers from the
NAAA address what it means to be a leader in the context of NASA and the
Academy. The
Academy should designate an interactive session with students, to have
them discuss their views of the Academy and leadership. The
Academy staff should interweave and exemplify leadership behaviors throughout
the Academy experience. Program Management and AdministrationThe
individual elements of the Academy and the people putting it together,
significantly impact the smoothness and effectiveness of the Academy. Specific
recommendations include: The
Academy should conduct end of summer wrap up and document the recommendations
for the next summer. The
Academies at the various Centers should share their lessons learned. The
Academy should provide feedback to the selection panel as to which RAs
selected were right for the Academy and which ones were not. The
Academy should consider centralizing the selection process and promotion,
if the funding source changes. The
Academy should conduct a follow up alumni survey (targeting 50 respondents)
and/or continue the exit survey each summer to rank how certain program
elements contribute to the leadership, communication development, team
building, and bonding benefits of the program. The
Academy programs should have a class size of 15 or fewer RAs. The
Academy should work creatively to set out the programs expectations for
and of the students, including pairing accepted students with alumni. The
Academy should consider an early teambuilding exercise. The
Academy should consider adding parents weekend or graduate school visits. The
Academy staff should evaluate the added benefit that each element of the
Academy brings to the overall Academy experience. The
Academy should spend a lot of energy and time on selecting staff. The
Academy should select next years alumni staff by application process during
the summer. The
Academy should solicit staff recommendations from the NAAA and from the
Space Grant Consortia. Program CostAcademy
funding comes from various sources. The
Space Grant Consortia (SGC) pay the students stipends and travel expenses. The
Centers bear the cost of staffing the Academy, providing housing and board,
and field trip expenses. It
was the conclusion of this assessment that the Academy should find a means
of accepting applications from non-funded students (i.e., from states whose
SGCs do not participate) and also seek an objective source of funding. NASA Academy Alumni Association and Participant Retention in the Aerospace ProgramThe
Academy has 307 alumni, of which an active alumni association has been
formed. The NAAA provides
alumni with a communication network for finding jobs, research information,
and more. Specific recommendations
include: The
Academy should use the NAAA as a resource for program design, for finding
and soliciting Academy candidates, and for finding Academy staff. There
should continue to be a strong interaction between the Executive Council
of the NAAA and the Center University Program Offices, and the NASA Academy
Directors, and the staff running the Academy at each Center. Overall Value of the AcademyThe
Academy meets its objectives on the surface, but with tactical changes,
the program could run more smoothly and ultimately be more beneficial to
NASA and the aerospace industry. Specific
recommendations include: The
Academy should commit to defining program objectives. The
Academy should survey Space Grant Consortia for selection methodology and
their measure of satisfaction with the Academy. The
Academy should keep the student rankings confidential. The
Academy should introduce a leadership award voted on by peers. This assessment reviewed the many elements that make up the Academy experience from each experience the RAs have, to the selection of students and PIs, to the Academy staff. Overall, the Academy experience has been described as a once in a lifetime experience that truly meets its objective of developing a communication network for the future leaders of the aerospace industry. This assessment determined that minor tactical changes could make this tremendous program even more effective and beneficial to NASA and to the aerospace industry. |