NASA CUSTOMER SURVEY SUMMARY REPORT Prepared by Science Applications International Corporation (SAIC) Arlington, Virginia August 1998

1
NASA CUSTOMER SURVEYSUMMARY REPORTPrepared by
Science Applications International Corporation
(SAIC)Arlington, VirginiaAugust 1998
2
Organization of Report

• Population
• Approach
• Introductory Questions
• ASTT Program Three Pillars and Ten Goals
• Level of Satisfaction with NASAs ASTT Program
• Facilities and Services
• Technology Transfer
• Program Planning
• Level of Satisfaction with NASA
• Resource Management
• Effectiveness in Meeting National Needs
• Closing Questions Executive Level Only

3
Survey Population Data

• Executive Mid-Level
  Manager
• Survey Population 101 72
• Customer Groups
• Direct Industry 63 65
• Indirect Industry 38 7
• Benchmark Companies 50 59
• Aviation Advisory Committee Members 11 0
• Universities 18 3
• Aviation Associations 8 3

4
Industry Groups

• DIRECT INDUSTRY
• Large Civil Transport Manufacturers
• High Performance Aircraft Manufacturers
• Aircraft Engine Manufacturers
• Rotorcraft Manufacturers
• GA/Commercial/Sport/Business/Other Aircraft
  Manufacturers
• Launch Vehicle Manufacturers
• Launch Vehicle Engine/Propellant Manufacturers
• Aircraft Instrument/Avionics Manufacturers
• Specialized Aerospace Services
• Department of Defense Entities
• Other Federal Agencies

• INDIRECT INDUSTRY
• Aviation Metal/Metal Component Manufacturers
• Non-Metallic Aviation Materials Manufacturers
• Air Carriers
• Satellite Manufacturers and Other Launch Service
  Customers
• Universities (Major ASTT Grantees)
• Aviation Associations
• Aerospace Analysts and Advocacy Groups

5
Survey Approach

• Quantitative Questions
• Statistical Analysis (Mean, Standard Deviation)
• Ratings from 1 to 10
• 1 to 4 - low
• 5 to 7 - medium
• 8 to 10 - high
• Qualitative Questions (open-ended)
• Followed several Quantitative Questions
• YES/NO Questions
• Sorting Items into Categories
• Comparisons with 92 and 95 Surveys

6
Survey Approach Summary

• Executive Mid-Level
  Manager
• Method of Interview
• Personal Visit 86 14
• Telephone 15 58
• Average Interview Duration (Minutes) 56 41
• Level of Understanding (1 to 10 scale) 7.7 7.9
• (Interviewer Assessment)
• Level of Interest (1 to 10 scale) 8.2 7.2
• (Interviewer Assessment)
• ASTT Mission Area to Which Answers Apply
• Aeronautics 63 47
• Space 14 15
• Aeronautics and Space 24 10

7
Overall Ratings by Customer Group

• Category Mean Rank
• Aeronautics Advisory Committee 7.02 1
• Universities 6.85 2
• Government 6.69 3
• Associations/Publications 6.58 4
• Mid-Level Managers 6.55 5
• Benchmark Companies 6.53 6
• Direct Industry 6.50 7
• Executives 6.41 8
• Indirect Industry 6.36 9
• Overall 6.47

8
1. What is your overall level of interest in the
ASTT Program?

• 96 percent of the total respondents said they had
  a high or medium level of interest in the ASTT
  Program. This level was up from the 1995 survey
  when the total was 92 percent.
• 79 percent of the total said they had a high
  level of interest. This compares to 75 percent
  in the 1995 survey.
• NASAs Aviation Advisory Committee (AAC) had the
  highest level of interest at 100 percent in the
  high category.
• University interest dropped from 100 percent in
  1995 to 90 percent in 1998.
• The group that had the lowest level of interest
  in ASTT was Indirect Industry.
• This question had the highest mean value of all
  quantitative questions in the 1998 survey.

Mean
8.9
7.6
8.2
9
1b. Select all the NASA mission areas that
greatly interest you.

• Critical Technology (Base RT) was selected as
  the area of greatest interest by 73 percent of
  the respondents.
• Other areas of high interest were Advanced
  Subsonic Transport (57percent) and High-Speed
  Research (51 percent).
• High Performance Aircraft (45 percent), High
  Performance Computing (45 percent), Space
  Transportation Technology (41 percent) and
  General Aviation (41 percent) all had moderate
  interest.
• Hypersonics (33 percent ) and Rotorcraft (33
  percent) had the lowest level of interest in the
  specifically named categories.

• a. Advanced Subsonic Transport
• b. High Speed Research
• c. High Performance Aircraft
• d. Hypersonics
• e. High Performance Computing
• f. Critical Technology (RT Base)

g. Facilities and Services h. Space
Transportation Technology i. General Aviation j.
Rotorcraft k. Other
10
2. How well do you understand the mission of the
ASTT Program?

• The level of understanding of the ASTT Program
  for all respondents dropped from 55 percent in
  1995 to 44 percent in 1998 in the well
  category. The mean value also dropped from 7.0
  in 1995 to 6.4 in 1998.
• AAC Members had the highest level of
  understanding with a mean value of 8.8.
• Benchmark Customers had the second highest level
  of understanding with a mean value of 6.8.
• The group with the lowest level of understanding
  was Association/Publication with a mean value of
  4.9.
• This questions overall mean score of 6.4 ranked
  14th of 30 quantitative questions.

Mean
11
3. How well are the current mission areas of
NASAs ASTT Program aligned to your
organizations needs?

• Government rated NASAs ASTT Program mission
  areas best aligned to their needs with a mean
  value of 7.0.
• The Benchmark Companies and Direct Industry rated
  the mission areas next well aligned with a mean
  value of 6.1.
• Total scores were close to 1995s scores, only
  dropping one percent from 26 percent to
  25 percent in well aligned, but the poorly
  aligned category grew from 23 percent to
  27 percent.
• Associations/Publications rated the goals least
  aligned with none scoring in the well category.
• This questions overall mean score of 5.8 ranked
  27th of 30 quantitative questions.

Mean
12
3b. What current mission areas of NASAs ASTT
Program are most important to your
organization?

• Organizational interest closely paralleled
  individual interest (Question 1b) but at a level
  approximately 8 to 10 percent less.
• Critical Technology (Base RT) showed the most
  interest with 61 percent of the respondents.
• Another area of significant interest was Advanced
  Subsonic Transport (49 percent).
• High Speed Research (39 percent), High
  Performance Computing (38 percent), High
  Performance Aircraft (34 percent), Space
  Transportation Technology (34 percent),
  Facilities and Services (32 percent), General
  Aviation (30 percent), and Rotorcraft
  (30 percent) all had moderate interest.
• Hypersonics (23 percent ) had the lowest level of
  interest in the specifically named categories.

13
4. How well is NASAs ASTT program accomplishing
its mission?

• 63 percent of the total respondent group rated
  their general satisfaction in the middle range
  and most individual groups followed this pattern.
  
• The mean value for the 98 Survey (6.0) dropped by
  0.3 as compared to the 95 Survey (6.3).
• Government Customers gave the highest rating with
  a mean value of 6.7.
• Mean values for Benchmark Customers was 6.1 which
  is slightly higher than the total population.
• Lower scores for accomplishing mission are
  consistent with lower scores for understanding of
  ASTT mission (Question 2).
• This questions overall mean score of 6.0 ranked
  24th of 30 quantitative questions.

14
4b. What current mission areas is NASAs ASTT
Program accomplishing the best?

• Respondents typically only selected one or two
  items, unlike the previous selection questions
  where they picked several choices.
• Four mission areas were the most common
  selection High Speed Research (31 percent),
  Advanced Subsonic Transport (29 percent),
  Critical Technology (RT Base) (22 percent), and
  Space Transportation Technology (21
  percent).
• High Performance Computing (12 percent) received
  a moderate rating.
• General Aviation (8 percent) High Performance
  Aircraft, Facilities and Services and Rotorcraft
  (all at 7 percent) and Hypersonics (5 percent)
  are accomplished least well according to the
  respondents.

15
5. How familiar were you with the Pillars and
Goals before this survey?

• Over two-thirds of the respondents were somewhat
  to very familiar with the Pillars and Goals
  before the survey.
• Those most familiar were the AAC Members (91
  percent) followed by Benchmark Companies at
  49 percent.
• The groups least familiar were Associations/
  Publications with 62 percent being unfamiliar and
  Indirect Industry with 51 percent.
• This questions overall mean score of 6.1 ranked
  21st of 30 quantitative questions.
• There was no comparable question on the 95 or 92
  surveys.

16
5b. How did you learn about the Pillars and
Goals?

• Most respondents learned about the Pillars and
  Goals through interaction with NASA, either
  through working with NASA or from Committee work.
  These are indicated in the Other column.
• The next most frequent way of learning about the
  Pillars and Goals was from the NASA
  Administrators Speech.
• The least frequent way of learning about the
  Pillars and Goals was from the Internet.

a. NASA Administrators Speech b. Pamphlet
Mailed to You c. Letter Introducing the
Survey d. From the Internet e. Other
17
6. How well are the Goals aligned to your
organizations needs?

• Mean values of ratings for this question were
  closely bunched ranging from a high of 7.1 for
  AAC Members to a low of 6.1 for Associations/
  Publications
• 42 percent of respondents ranked the Goals as
  well-aligned to their organization.
• This questions overall mean score of 6.6 ranked
  10th of 30 quantitative questions.

18
6b. Which goals are best aligned to your
organizations needs?
Many respondents identified specific Pillars or
Goals some identified subsets of Pillars or
Goals.
19
6c. Which important Goal, from your
organizations perspective, is missing?
Respondents provided various ideas regarding
goals that should be/could be added.

• Work with DoD where commonality applies -- need
  Goal to share and integrate cost-effective RD
  with DoD
• Greatly concerned about future of subsonic
  transport RT. Basic is being neglected.
• Development of future rotorcraft concepts...,
  working with FAA to develop infrastructure for
  civil transportation.
• No materials research for airframe technology for
  general aviation.
• Aeronautics and Earth Science not connected --
  NASA ignores internal customers.
• High performance military aircraft.
• Next generation avionics.
• FANS, satellite communications technology,
  composite wing, wet wing.
• Reduce cost to lower Earth orbit applies to only
  part of mission -- need Goals that cover launch
  to orbit mission, transfer orbits and on issues
  with space probe.
• Future engineering education should be in the
  Goals important to next generation.
• Economic viability of the contemplated
  technologies.
• Human factors research.
• Product quality.
• Fundamental flow physics.
• Infrastructure development.
• There isnt anything missing.

20
7. How well are the Goals aligned to national
needs?

• Total respondents overwhelmingly ranked NASAs
  Goals as being very well aligned to national
  needs scoring 66 percent in the well category.
  Total well and moderately well rating was
  97 percent.
• Benchmark Customers and Universities ranked this
  question very high with 70 percent in the well
  category.
• The lowest aligned well rating was Government
  at 54 percent, which also had the second highest
  poorly aligned rating of 15 percent.
• Mean values closely followed the percentile
  ratings.
• This questions overall mean score of 7.9 ranked
  2nd of 30 quantitative questions.

21
7b. Which goals are best aligned to national
needs?
Most respondents identified specific Pillars or
Goals some respondents identified subsets of
Pillars or Goals. These results closely parallel
the results of Question 6b regarding the
alignment to organizational needs.
7c. What important Goal, from a national
perspective, is missing?
Many of the suggestions for National Goals were
similar to those provided in Question 6c in
response to missing Goals from the Customers
organizational needs.
22
8. Overall, how well are NASAs current programs
aligned to the Goals?

• Overall, the respondents thought that NASAs
  current programs were moderately well aligned to
  the Goals. However, within the Customer Groups,
  the rankings for this question were nearly the
  reverse of those observed in most other
  questions.
• The highest rankings were given by the
  Associations/Publications and the Universities
  with a mean value of 6.9.
• The lowest rankings within the customer groups
  were given by the Benchmark Customers and the
  Direct Industry Customers. They each had a mean
  value of 6.3.
• This questions overall mean score of 6.4 ranked
  near the middle at 16th of 30 quantitative
  questions.

Poorly
Moderately
Well
Mean
98 All
58
28
15
98 Assoc/Pub
40
10
50
98 Universities
11
42
47
98 Direct
61
15
24
Industry
98 Benchmark
13
64
23
0
20
40
60
80
100
23
8b. Which current ASTT programs are best aligned
to the goals?
Most respondents provided answers that related to
specific Pillars, Goals, or ASTT Programs.
24
9. What is your level of satisfaction with
NASAs ASTT Facilities and Services?
Mean

• The level of satisfaction with ASTT facilities
  and services dropped significantly as compared to
  the 1995 survey. The mean value in 1995 was 6.8,
  in this survey it dropped to 6.0
• Government Customers showed the greatest
  satisfaction with facilities and services with a
  mean value of 7.0 the AAC had the second highest
  mean value of 6.5.
• Associations and Publications had the lowest mean
  value of 4.9 Indirect Industry had the next
  lowest mean value at 5.4.
• This questions overall mean score of 6.0 ranked
  moderately low at 22nd of 30 quantitative
  questions.

25
9b. Which NASA facility and service issues are
most important to your organization?

• Facility or Service Issue
  Percent
• 1. Quality of the Data 75
• 2. Level of Technical Expertise 68
• 3. Timeliness of the Results 63 High
• 4. Availability of Facility or Service 60
• 5. State-of-the-Art Facilities 58
• 6. Protection of the Data 42
• 7. Ease of Technology Transfer 39
• 8. Familiarity and/or Trust 38 Moderate
• 9. Charge for Using Facility or Service 36
• 10. Ownership by the U.S. Government 35
• 11. Computational Quality 32
• 12. Convenience of Facility or Service 19
• 13. Ownership by U. S. Government 10 Low
• 14. Location of the Facility or Service 6

26
10. Which methods of technology transfer work
best for your organization?
92 Survey
Mean
Rank
4
7.01
1
7.49
6
6.81
7.07
2
6.85
5
7.03
3
5.98
7
These results differ somewhat from results of the
1992 survey. In that survey, customers were
asked to give the various technology transfer
methods a ranking from 1 to 10 with 1 being
unfavorable and 10 being favorable. These
results are shown in the right-hand columns.
27
11. How would you rate NASAs program planning?

• Program planning was rated higher in this survey
  (mean 6.0) than in the 95 survey (mean 5.6).
• Program planning was rated much higher in this
  survey than in the 92 survey when 53 percent of
  the respondents indicated that NASAs program
  planning was inadequate.
• This question was rated higher by Associations
  and Publications (mean 7.3) and Universities
  (mean 6.7). It was rated lower by Benchmark
  Customers (mean 5.9) and lowest by Government
  Customers (5.8)
• 58 percent of respondents answered Yes to the
  question, Does NASA adequately involve your
  organization in its program planning for ASTT?
• 62 percent of respondents answered Yes to the
  question, Does NASA plan research programs to
  provide results in a form, structure and format
  useful to you?
• This questions overall mean score of 6.0 ranked
  quite low at 23nd of 30 quantitative questions.

Mean
28
12. How would you rate NASAs Outreach and
External Communication?

• Outreach and External Communication was rated
  slightly higher in this survey (mean 6.3) than
  in the 95 survey (mean 6.2).
• The AAC (mean 7.4) and University groups (mean
  7.3) rated this question high. Benchmark
  Customers (mean 6.4) and Direct Industry
  Customers (mean 6.3) rated the question lower
  than the overall mean of 6.3.
• 68 percent of respondents answered Yes to the
  question, Does NASA ask for your input on plans
  and issues important to your organization?
• 67 percent of respondents answered Yes to the
  question, Does NASA use your input?
• 65 percent of respondents answered Yes to the
  question, Does NASA provide feedback on
  plans/issues important to your organization?
• This questions overall mean score of 6.0 ranked
  near the middle at 17th of 30 quantitative
  questions.

Mean
29
13. What is your overall level of satisfaction
with NASA?
Mean

• The Customers Overall Level of Satisfaction with
  NASA was rated slightly higher in this survey
  (mean 6.6) than in the 95 survey (mean 6.4).
• There was a significant difference (plt0.05) in
  the mean values of the Executives (mean 6.3)
  and the Mid-Level Managers (mean 7.0) on this
  question.
• This questions overall mean score of 6.6 ranked
  near the top one-third of the questions at 11th
  of 30 quantitative questions.

• 13b. In what areas are you most satisfied with
  NASA?
• Increasing attentiveness to the needs of U. S.
  industry improved communication ... in executing
  contracts.
• Good job getting input from industry good
  facilities and people.
• Quality work, quality reports, results and
  availability.
• Headquarters' interest in having a positive
  national impact through technology development.
• 13c. In what areas are you least satisfied with
  NASA?
• Inability to do meaningful multi-year planning.
• Inability to change with changing needs of
  industry.
• Technology transfer - cost to industry complex
  paperwork nightmare multi-page licenses.
• NASA competition with industry.

30
14. What is your overall level of satisfaction
with NASAs ASTT products?
Mean

• The Customers Overall Level of Satisfaction with
  NASAs ASTT Products was rated moderately high.
  Overall, 85 percent of the respondents rated
  their satisfaction as medium or high.
• University Customers (mean 7.1) and AAC
  Customers (mean 6.9) had a higher rating than
  the overall population. Direct Industry
  Customers (mean 6.3) had a lower rating than
  the overall population.
• This questions overall mean score of 6.4 ranked
  in the middle at 15th of 30 quantitative
  questions.

6.4
7.1
6.9
6.3

• 14b. With what products were you most satisfied?
• Responses
  Responses
• High Speed Research/Travel High Speed Civil
  Transport 23 Technical
  Reports/Symposia 13 Computational Fluid
  Dynamics/Computational Analysis 17
  Use of NASA Facilities 12
• Research and Technology Base 15
  Design and Analysis Tools 10
• Advanced Subsonic Transport/Subsonic Research
  14 Reduce Noise 10
• Experimental Aircraft (X-33, X-34, X-36, Hyper-X,
  Future-X) 13 Rotorcraft/Tiltrotor
  9
• 14c. What products were you least satisfied
  with?
• Those that dont have a cooperative character...
• Full-cost funding resulting in budget cuts and
  diminished expectations.
• Technology transfer between larger corporations
  and NASA and government (as on HSR program).

31
15. How does your organizations overall level
of satisfaction with NASA compare with its
satisfaction 3 to 6 years ago (before the last
surveys)?

• The Customers felt that their satisfaction with
  NASA had improved over the last 3 to 6 years.
  They rated this question slightly higher in the
  current survey (mean 6.5) than in the 95 survey
  (mean 6.3).
• 85 percent of respondents rated this question as
  medium or high. Benchmark Customers (mean
  6.6) provided slightly higher ratings, and
  University Customers (mean 6.1) provided
  slightly lower answers than the overall
  population.
• This questions overall mean score of 6.5 ranked
  slightly above the middle at 13th of 30
  quantitative questions.

6.5
6.6
6.3

• 15b. In what area(s) has your level of
  satisfaction increased?
• Alignment of research to the Pillars six years
  ago there was inefficiency and lack of focus.
• Shift from micro-management to a partnership
  relationship.
• Involvement and visibility of NASA plans and
  goals.
• Just in being part of their activities were in
  partnership with them our relationship has grown
  by leaps and bounds.
• 15c. In what area(s) has your level of
  satisfaction decreased?
• Systems integration emphasis, computational
  resources, and test facility readiness.
• Availability of wind tunnel test facilities
  reduced emphasis on RT base for aeronautics.
• I think the most important is the condition of
  NASAs deteriorating facility support.
• Difficulty in dealing with NASA as a customer.

32
16. NASA provides key laboratory, computing, and
test facilities.
Mean

• The respondents strongly felt that NASA provides
  key facilities.
• 93 percent of respondents rated this question as
  partially agree or agree and 62 percent rated
  it agree. Government Customers (mean 8.2)
  and University Customers (mean 8.1) provided
  high ratings, and Benchmark Customers (mean
  7.5) provided slightly lower answers than the
  overall population.
• This questions overall mean score of 7.6 ranked
  near the top at 3rd of 30 quantitative questions.

7.6
8.2
8.1
7.5

• 16b. What NASA facilities do you view as being
  essential to the conduct of leading edge RD?
• The overwhelming answer to this question was wind
  tunnels, followed by computational resources.
  The comments below characterize these and other
  comments.
• Modern wind tunnel test facilities and world
  class computational resources.
• Space test facilities at Stennis Center.
• Long duration test and space exposure tests.
• Airborne test beds, simulation and flight
  facilities.

33
17. NASA downsizing has affected its ability to
support its customers.

• The respondents were equally divided in response
  to this question.
• 65 percent of respondents rated this question as
  partially agree or agree and 33 percent rated
  it agree. AAC Customers (mean 7.0) and
  University Customers (mean 6.4) provided high
  ratings, while Benchmark Customers (mean 5.9)
  were more inclined to disagree than the overall
  population.
• This questions overall mean score of 5.8 ranked
  near the bottom at 26th of 30 quantitative
  questions indicating significant disagreement
  that downsizing has affected NASAs ability to
  support customers.

Mean
5.8
7.0
6.4
5.9

• 17b. In what ways, if any, has NASA downsizing
  affected your organization?
• Some respondents felt that NASA downsizing had a
  negative effect
• Foundational research is not getting done.
• Primarily, loss of key NASA expertise and missing
  expertise is not being replaced.
• We are starting to reduce our alliance on
  technologies developed by NASA.
• Other respondents felt that NASA downsizing had
  positive effects
• Very positive effect - less micro-management -
  more of a partnership.
• Havent downsized enough.
• Helped since NASA is now less competitive with
  industry.
• Its been positive its made them take a hard
  look at programs and ... it helped in that they
  are more focused.

34

18. NASA is a key contributor to the education
of scientists and engineers.

• The respondents strongly felt that NASA is a key
  contributor to the education of scientists and
  engineers. The respondents of the current survey
  (mean 7.0) gave higher ratings to this question
  than did the respondents of the 95 survey
  (mean 6.7).
• 84 percent of respondents rated this question as
  partially agree or agree and 52 percent rated
  it agree. University Customers (mean 8.0)
  provided high ratings, and Benchmark Customers
  (mean 6.9) provided slightly lower ratings than
  the overall population.
• This questions overall mean score of 7.0 ranked
  near the top at 4th of 30 quantitative questions

Mean
7.0
8.0
6.9
6.7

• 18b. In what ways can NASA improve the education
  and training of our next generation of scientists
  and engineers?
• The main themes were more funding, cooperative
  programs, and more internships. Also NASA
  involvement in university, secondary and
  elementary schools was identified as being
  important.
• Strengthen university relationships that include
  industry.
• Put more money into foundational research in
  universities best equipped to do that type of
  research.
• Re-instituting university programs programs where
  NASA provides seed money to university
  engineering programs.
• Be more involved in recommending curricula of
  engineering schools. Have more open house events
  to stimulate younger students.
• Public relations programs aimed at younger folks
  to kindle their interest early on.
• Target younger children, even in elementary
  grades.

35

19. NASA cooperates with other organizations to
make cost-effective use of limited RD resources,
skills and facilities.

• The respondents were mixed in their rating of
  NASAs cooperation with other organizations. The
  respondents of the current survey (mean 6.3)
  gave slightly higher ratings to this question
  than did the respondents of the 95 survey (mean
  6.1).
• 78 percent of respondents rated this question as
  partially agree or agree and 35 percent rated
  it agree. AAC Customers (mean 7.0) provided
  higher ratings than the overall population and
  Benchmark Customers (mean 6.3) provided ratings
  equal to the overall population.
• This questions overall mean score of 6.3 ranked
  in the lower half at 19th of 30 quantitative
  questions

Mean
6.3
7.0
6.3
6.1

• 19b. In what ways can NASA improve cooperation
  with other organizations?
• More communication and coordination was the
  primary theme of the responses.
• Talk to and coordinate with DoD -- and dont just
  talk -- perform some deeds.
• Expand levels of cooperation with other
  government agencies as well as corporate world
  develop and participate in joint technical
  programs.
• By helping its personnel to understand the
  realities and mechanisms of the commercial
  marketplace.
• Improve communication become more aware of
  industry programs to create cooperative programs.
• Be more pro-active in searching for
  organizations to cooperate with.

36

20. The ASTT program effectively balances
fundamental (RT base) research with its
focused-program research.

• The respondents were mixed in their rating of
  NASAs balance of fundamental versus
  focused-program research. Government Customers
  (mean 7.0) provided much higher ratings than
  the overall population.
• This questions overall mean score of 5.9 ranked
  in the lower half at 25th of 30 quantitative
  questions.
• Respondents of the current survey called for
  significantly more fundamental research than did
  the respondents of the 92 survey.

Mean
5.9
7.0

• 20b. Should NASA pursue more fundamental
  research or focused-program research, and why?
• Some thought balance was good others presented
  arguments for both fundamental and focused
  research.

• Fundamental research if fundamental research is
  done, the focused-program research will be done
  better.
• NASA should be looking at 5 to 10 year
  technology universities should be looking at 10
  to 15 year technology industry should look at 2
  to 5 year technology.
• More fundamental research industry better
  understands market need leave focused-program to
  industry.
• More focused-program research because it is a
  more effective use of resources.
• Focused-program research to become more efficient
  -- pull research rather than push research.
• More focused-program research, because it offers
  more incentive for industry involvement that
  would lead to commercial products.

37

21. The ASTT program effectively balances
analytical, computational, ground-based
experimental and flight research.

• The respondents generally approved of NASAs mix
  of research methods. The overall mean was quite
  high at 6.8.
• There was some variation within the customer
  groups. AAC Customers (mean 7.6) provided
  higher ratings than the overall population
  Benchmark Customers (mean 6.9) provided
  slightly greater than the ratings of the overall
  population Associations and Publications gave a
  low rating.
• This questions overall mean score of 6.8 ranked
  quite high (in the upper quarter) at 6th of 30
  quantitative questions

Mean
6.8
7.6
6.9
5.9

• 21b. Should NASA change its mix of analytical,
  computational, ground-based experimental and
  flight research, and why?
• Most respondents felt the balance was good now,
  but the mix needs to be constantly scrutinized.
  There was some call for more flight testing.
• Yes, not enough flight research for aeronautics
  maintain ground-based experimental facilities.
• More emphasis on flight research will produce
  more rapid progress.
• In general, they are pretty well balanced, but
  caution is necessary on flight research to make
  sure the costs/benefits are scrutinized.
• The mix is fine and doesnt need to be changed.

38

22. The ASTT engineers and scientists play a key
role in the development of innovative or enhanced
technical solutions.

• The respondents generally felt that NASAs
  engineers and scientists play a key role in
  finding technical solutions. The overall mean
  was quite high at 7.0.
• There was only slight variation within the
  customer groups as the mean values ranged from
  6.9 for Benchmark Customers to 7.3 for University
  and Government Customers.
• This questions overall mean score of 7.0 ranked
  quite high (in the upper quarter) at 5th of 30
  quantitative questions

Mean
7.0
7.3
7.3
6.9

• 22b. In what areas have NASA engineers
  scientists played a key role?
• Human factors, structures, propulsion and
  computational fluid dynamics -- Space travel is
  the biggest
• Atmospheric physics earth physical science --
  Development of high performance rocket propulsion
• Lifting-body ceramic and metallic thermal
  protection systems supercritical wing
  coke-bottle design of aircraft
• Composite technology computer fluid analytical
  (CFA) tools and noise research -- Atmospheric
  modeling
• 22c. In what area have they not played a key
  role?
• Theyve tried to play a role in system
  integration, but unsuccessfully -- Solution of
  critical operational problems
• General aviation and airline safety --
  Development of cost effective launch systems
• Have not achieved leadership in information
  technology, for example, redundant/fault-tolerant
  information systems
• Reducing cost of bringing advanced technology to
  market and reducing the time to certification of
  key technologies

39


23. How effective is NASAs ASTT program in
performing fundamental research?


Mean

• As compared with the 95 survey(which contained
  two question relating to performance of
  fundamental research) the 98 survey showed much
  lower mean values. The 98 survey had a mean of
  6.3. The 98 survey had a mean score of 7.1 for
  effectiveness and a 7.5 score for contributions
  to national needs.
• The customer groups presented varied results in
  rating the effectiveness NASAs ASTT Program in
  performing fundamental research. The mean values
  ranged from 6.0 for Benchmark Customers to 7.8
  for Associations and Publications.
• This questions overall mean score of 6.3 ranked
  in the lower half at 18th of 30 quantitative
  questions.

6.3
7.8
6.7
6.0
7.1
7.5
40


24. How effective is NASAs ASTT Program in
improving national security?

• As compared with the 95 survey the 98 survey
  scored much lower mean values. The 98 survey had
  a mean of 5.1. The 98 survey had a mean score of
  6.0 for contribution to national needs.
• The customer groups presented very low results in
  rating the effectiveness NASAs ASTT Program in
  improving national security. The mean values
  ranged from 4.5 for Mid-Level Managers to 5.4 for
  AAC Members.
• The statistical data showed there was a
  significant difference in the mean values of the
  Executive Customers (mean 5.5) and the
  Mid-Level Managers (mean 4.5). For this
  question, there is a high likelihood that the two
  customer groups represent different population
  groups.
• This questions overall mean score of 5.1 ranked
  lowest in the entire survey at 30th of 30
  quantitative questions.

Mean
5.1
5.4
4.8
5.5
4.5
6.0
41

25. How effective is NASAs ASTT Program in
developing/ transferring technologies for
economic competitiveness?



Mean

• As compared with the 95 survey the 98 survey
  scored much lower mean values. The 98 survey had
  a mean of 5.8 for effectiveness. The 95 survey
  had a mean score of 6.9 for effectiveness and 6.6
  for contribution to national needs.
• The customer groups had generally low mean values
  in rating the effectiveness NASAs ASTT Program
  in developing/transferring technologies for
  economic competitiveness. The mean values ranged
  from 5.8 for Benchmark Customers to 6.5 for
  University Customers.
• This questions overall mean score of 5.8 ranked
  in the lowest quarter of the survey at 28th of 30
  quantitative questions.

5.8
6.5
6.4
5.8
6.6
6.9
42

26. How effective is NASAs ASTT Program in
maintaining superiority of US aircraft and
engines?



Mean

• As compared with the 95 survey the 98 survey
  scored slightly higher mean value. The 95 survey
  had a mean of 6.6 for national goals achievement.
  The 98 survey had a mean score of 6.7 for
  effectiveness.
• The customer groups had moderately high mean
  values in rating the effectiveness NASAs ASTT
  Program in maintaining the superiority of U. S.
  aircraft and engines. The mean values ranged
  from 6.1 for Government Customers to 7.4 for
  University Customers.
• This questions overall mean score of 6.7 ranked
  in the highest third of the survey at 9th of 30
  quantitative questions.

6.7
7.4
7.1
6.7
6.1
6.6
43

27. How effective is NASAs ASTT Program in
developing a superior, affordable, global air
transportation system?



Mean

• As compared with the 95 survey the 98 survey
  scored a lower mean value. The 95 survey had a
  mean of 6.2 for national goals achievement. The
  98 survey had a mean score of 5.7 for
  effectiveness.
• The customer groups had very low mean values in
  rating the effectiveness NASAs ASTT Program in
  developing a superior, affordable global
  transportation system. The mean values ranged
  from 5.6 for Benchmark Customers to 6.7 for AAC
  Members.
• This questions overall mean score of 5.7 ranked
  very low in the survey at 29th of 30 quantitative
  questions.

5.7
6.7
6.5
6.4
5.6
6.2
44

28. How effective is NASAs ASTT Program in
developing superior, affordable, space launch
systems?

• As compared with the 95 survey the 98 survey
  scored a slightly lower mean value. The 95
  survey had a mean of 6.4 for contributions to
  national needs. The 98 survey had a mean of 6.1
  for effectiveness.
• The customer groups had very mixed mean values
  rating the effectiveness in developing superior,
  affordable space launch systems. The mean values
  ranged from 5.1 for AAC Members to 6.6 for
  Mid-Level Managers. The AAC Members gave this
  question their lowest rating in the 98 survey.
• The statistical data showed there was a
  significant difference in the mean values of the
  Executive Customers (mean 5.8) and the
  Mid-Level Managers (mean 6.6). For this
  question, there is a high likelihood that the two
  customer groups represent different population
  groups.
• This questions overall mean score of 6.1 ranked
  low in the survey at 20th of 30 quantitative
  questions.

Mean
6.1
6.2
5.1
6.6
5.8
6.4

• There was one question on the 95 survey regarding
  contributions to national needs in space. The 95
  survey question and the 98 survey question are
  slightly related in that regard and a comparison
  is presented herein.

45

29. How effective is NASAs ASTT Program in
ensuring long-term environmental compatibility of
aerospace systems?



Mean

• There were two questions on the 95 survey dealing
  with environmental technology. One related to
  contribution to national needs and the other
  related to national goals achievement.
• The 98 survey scored between the two 95 survey
  questions. The 95 survey had a mean of 6.4 for
  national goals achievement 6.8 for contribution
  to national needs. The 98 survey had a mean
  score of 6.7 for effectiveness.
• The customer groups had generally high mean
  values in rating the effectiveness NASAs ASTT
  Program in ensuring long-term environmental
  compatibility of aerospace systems. The mean
  values ranged from 6.6 for Benchmark Customers to
  7.4 for University Customers.
• This questions overall mean score of 6.7 ranked
  in the upper third of the survey at 8th of 30
  quantitative questions.

6.7
7.4
7.1
6.6
6.4
6.8
46

30. How effective is NASAs ASTT Program in
making partnerships with government, industry,
and universities?



Mean

• There were two questions on the 95 survey dealing
  with partnerships. One related to effectiveness
  and the other related to national goals
  achievement.
• The 98 survey scored between the two 95 survey
  questions. The 95 survey had a mean of 6.3 for
  national goals achievement and 7.0 for
  effectiveness. The 98 survey had a mean score of
  6.8 for effectiveness.
• The customer groups had generally high mean
  values in rating the effectiveness NASAs ASTT
  Program making partnerships with government,
  industry and universities. The mean values were
  tightly bunched and ranged from 6.5 for Indirect
  Industry Customers Customers to 6.9 for several
  customer groups.
• This questions overall mean score of 6.8 ranked
  in the upper quarter of the survey at 7th of 30
  quantitative questions.

6.8
6.9
6.9
6.7
6.3
7.0
47

31. How effective is the overall ASTT
contribution to US RD needs?

• The respondents felt that NASA is an effective
  contributor to U. S. research and development
  needs. However, the respondents of the current
  survey (mean 6.6) gave lower ratings to this
  question than did the respondents of the 95
  survey (mean 7.3).
• 88 percent of respondents rated this question as
  partially agree or agree and 31 percent rated
  it agree. AAC Members (mean 6.9) provided
  slightly higher ratings than the overall
  population. Benchmark Customers (mean 6.6)
  provided ratings consistent with the overall
  population.
• This questions overall mean score of 6.6 ranked
  near the middle at 12th of 30 quantitative
  questions

Mean
6.6
6.9
6.6
7.3

• 31b. What can ASTT do to increase its overall
  effectiveness in contributing to US RD needs?
• The main themes were more funding, more
  fundamental research, and working more closely
  with customers.
• More stable, long term programs more emphasis on
  validation.
• NASA needs to excel in the use of critical
  research and development methods with both
  industry and other branches of the government,
  especially DoD and FAA.
• Maintain a high level of communication in trying
  to understand industrys needs.
• Perform better market research to validate the
  need for the research.
• Continue looking at business-based metrics watch
  U. S. competitive needs.

48

32. Has NASAs planning worsened or improved
over the last few years? (Asked to Executive
Customers only)

• Most respondents (51 percent) felt that NASAs
  planning had stayed the same over the last few
  years. However, more respondents (32 percent)
  felt that planning had improved as compared to
  those who felt that it had worsened (18 percent).
• AAC Members (mean 7.3) provided significantly
  higher ratings than the overall population (mean
  6.4). Benchmark Customers (mean 6.6)
  provided ratings slightly higher than the overall
  population. Indirect Industry Customers (mean
  5.9) rated this question much lower than the
  overall mean.
• This question was not included in the count of 30
  quantitative rating questions because it was
  asked only of Executive Customers. However, the
  overall mean score of 6.4 ranked near the middle
  of the 30 quantitative questions

Mean
6.4
7.3
6.6
5.9

• 32b. What can NASA do to improve its planning?
• Provide a forum that goes beyond workshops and
  actually sets up programs -- smaller group of key
  industry participants to boil down ideas into
  programs do more feasibility studies with
  industry before launching a program.
• Be more responsive to industrys needs -- not so
  much planning as it is attitude.
• More planning in cooperation with DoD elements in
  aeronautics and space.
• Better execution of plans.
• Match planning to budget avoid stop - starts in
  programs due to budget fluctuations.
• Maintain commitments once established.
• Narrow focus -- be better at fewer things.

49
33. Have you participated in a previous NASA
Aero Survey?

• 21 Executive Level respondents (out of a total of
  101) participated in the 1992 survey.
• 35 Executive Level respondents participated in
  the 1995 survey.
• 13 Executive Level respondents participated in
  both prior surveys.

• 33b. If you participated in the 1992 and/or the
  1995 survey, did the results of the survey in any
  way change your organizations relationship with
  NASA? If so, how?
• The responses were mixed concerning changes in
  relationships with NASA.
• Yes, between 1992 and 1995 industry became more
  involved in planning through workshops.
• Yes, helped put more relevancy in NASA which led
  to Pillars and Goals.
• Yes, increased efforts in joint industry/NASA
  planning.
• Created an awareness of the advantages of working
  with NASA.
• No apparent impact -- however, NASA should be
  praised for attempt.
• No, I dont know if I ever saw results of the
  survey.

50

34. Compared to previous NASA Aero Surveys (1992
and 1995), how would you rate this one? (Asked
to Executive Customers only)

• Most respondents (95 percent) rated the 98 survey
  as medium or high.
• Indirect Industry Customers (mean 7.1) provided
  slightly higher ratings than the overall
  population (mean 6.9). Benchmark Customers
  (mean 6.8) provided ratings slightly lower than
  the overall population. Government Customers
  (mean 6.6) also rated this question lower than
  the overall mean.
• This question was not included in the count of 30
  quantitative rating questions because it was
  asked only of Executive Customers. However, the
  overall mean score of 6.9 ranked in the top
  quarter of the 30 quantitative questions

Mean
6.9
7.1
6.8
6.6

• 34b. How can future surveys be improved?
• Offer opportunity to get into specifics in
  different areas of ASTT.
• Have Pillars and Goals ahead of time.
• Need more questions on how NASA measures their
  effectiveness against customer need.
• ... A few weeks before the survey, provide a list
  of fundamental areas that will be discussed.
• Survey well done it cant be improved.
• By dividing them into specific areas, such as
  global civil aviation, space transportation, etc.
• Continue the in-person interviews.
• Emphasize more narrative questions and answers.
  There are techniques to analyze narrative answers.

51
35. For the next survey, what new question(s)
should be asked?

• There are a wide range of suggested questions,
  but respondents were most interested in rating
  NASAs performance and follow through. They also
  wanted to be involved in future NASA planning and
  strategy development processes.
• Specific questions within the mission areas a
  two-level questionnaire.
• What changes would you make to improve NASA
  effectiveness in achieving its mission? What
  diverts or interferes with NASA being effective?
• Break into space and aeronautical segments.
• Are NASA's expenditures properly balanced between
  programs?
• More questions aimed at university programs,
  training, and engineers.
• Rate performance of different NASA levels
  Center, HQ Staff, and Administration
• A lot of the questions from this survey are the
  right ones. Maybe some about how we are doing on
  the implementation of the recommendations would
  improve and smooth the survey effort. Kind of
  the "big stick" approach.
• Pretty complete. Presupposes that interviewee
  knows the breakdown of the budget.
• Ask some fundamental questions on US aeronautics
  policy and what would affect the customer's
  predilection to using technology.
• Questions about individual center's performance,
  especially, performance at headquarters.
• What specific areas do you work with or have
  worked with in the past year with NASA? How
  frequently do you interact with NASA?

52
36. What do you want to say to NASA
administrator about ASTT Program?

• Hold the course, balance is good, good support
  from industry -- don't overreact to public
  statements.
• That the ASTT Program is the most efficient
  dollars NASA spends. The ASTT industry/NASA
  roles are a more appropriate approach for space
  development (not including exploration) than
  currently allocated. For all the criticism, its
  much more efficient when industry contributes
  funds and resources collaboratively.
• Don't sacrifice aeronautics work due to pressure
  of financing space transportation or other space
  activities.
• Strengthen university programs to develop next
  generation scientists and engineers.
• The subsonic section has inadequate advocacy with
  the public and is not on par with the space
  activity.
• Thank you for covering the wide range in ASTT
  from general aviation to high-speed aeronautics.
• We are on a good course. Things have certainly
  been improving.
• NASA should consider and address specific needs
  in industry, especially increased
  competitiveness. Keep up research in propulsion
  area and noise reduction area. Emphasize
  creature comforts and safety in GA.
• It needs a heavier emphasis on space technology.
• Goals are good. NASA needs to open up as going
  through goals. Allows for realistic programs to
  achieve goals.
• I'm very happy to see that the ASTT is stronger
  focused and more dynamic. I like the way the
  program is now and not the way it was before.
• More cooperation with military, there's been an
  attempt but not "tight" enough.
• NASA should focus on space and let the
  aeronautics industries handle aeronautics
  research.
• DoD is interested in being more closely related
  with NASA than in past. Could use a true
  personnel exchange.
• Loss of key personnel, key capability, need to
  recruit the best personnel to stay on top of
  programs.
• Quit using taxpayers money to fund competition
  because you are discouraging private investment
  ...
• Don't let large industry drive high-speed civil.
  Follow the recommendations of National Research
  Council on HSCT. Is it necessary for NASA to be
  so broad? Aeronautics should stick to aeronautics
  ...
• Job well done!!
• Keep providing results to hold confidence of
  Congress to hold the program. But, reach out to
  industry and identify specific programs to which
  those funds should be targeted.