Relatório Especial 14, 1952-1955: como a Força Aérea transformou relatos de discos voadores em cartões perfurados IBM

forgetting the subjectivity of the data at the time that conclusions are drawn from the analysis. It must be emphasized, again and again, that any conclusions contained in this report are based NOT on facts, but on what many observers thought and estimated the true facts to be. Altogether, the data for this study consisted of approximately 4,000 reports of sightings of unidentified aerial objects. The majority were received through military channels or in the form of observer-completed questionnaires; a few were accepted in the form of direct letters from unquestionably reliable sources. Sightings made between June, 1947, and December, 1952, were considered for this study. Sightings alleged to have occurred prior to 1947 were not considered, since they were not reported to official sources until after public interest in "flying saucers" had been stimulated by the popular press. REDUCTION OF DATA TO MECHANIZED COMPUTATION FORM As received by the Air Technical Intelligence Center, the sighting reports were not in a form suitable for even a quasi-scientific study. A preliminary review of the data indicated the need for standardized interrogation procedures and supplemental forms for the reduction of currently held and subsequently acquired data to a form amenable to scientific appraisal. The plan for reduction of the data to usable form consisted of a program of development comprising four major steps: (1) a systematic listing of the factors necessary to evaluate the observer and his report, and to identify the unknown object observed; (2) a standard scheme for the transfer of data to a mechanized computation system; (3) an orderly means of relating the original data to all subsequent forms; and (4) a consistent procedure for the identification of the phenomenon described by the original data. Questionnaire The first reports received by ATIC varied widely in completeness and quality. Air Force Letter 200-5(2) and Air Force Form 112(1) were attempts to fix responsibility for and improve the quality of the reports of sightings. To coordinate past efforts and to provide standardization for the (1) A modified Air Force Form 112 lists pertinent questions to be answered in regard to an unidentified-object sighting. (2) Air Force Letter 200-5 places responsibility with the Air Force for the investigation, reporting, and analysis of unidentified aerial objects. This letter is dated 29 April 1952.

future, it was imperative to develop a questionnaire form listing the factors necessary for evaluation of the observer and his report, and identification of the unknown objects. In addition, it was decided that such a questionnaire should be designed to serve as an interrogator's guide, and as a form for the observer himself to complete when personal interrogation was not possible or practicable. Ideally, a questionnaire for the purposes required should contain questions pertaining to all technical details considered to be essential for the statistical approach, and should serve to obtain a maximum of information from the average individual who had made a sighting in the past or would be likely to be reporting sightings in the future. Besides these discrete facts, an integrated written description of a sighting would be required, thus enabling the reported facts of the sighting to be corroborated. Also, a narrative description might allow subtle questions to be answered concerning the observer's ability, such as indirect questions that would reveal his reasoning ability, suggestibility, and general mental attitude. As a whole, then, the information contained in a questionnaire should make possible the classification and evaluation of the sighting, the rating of the observer, the probability of accuracy of reported facts, and the identification of what was reported by the observer as unidentified. During the course of this project, three questionnaire forms were developed, each intended to be an improved revision of the one preceding. The improvements were suggested and confirmed by members of the panel of consultants connected with this project. The original form was evolved by the panel of consultants as their first work on this project. It was intended to allow the start of the reduction of reports to discrete data, and was immediately subjected to extensive review and revision by the panel. The revised (second) form was subjected to a trial test before adoption. ATIC sent a copy to observers reporting sightings, with the request that the form be completed and returned. Of the first 300 questionnaires returned during July and August, 1952, 168 were analyzed by a consulting psychologist. On the basis of this analysis, plus the experience gained in working with past reports, the final form of the questionnaire - the U. S. Air Force Technical Information Sheet - was evolved. Copies of the three forms of the questionnaire, in the order of their development, are shown as Exhibits B1, B2, and B3 in Appendix B. In order to implement the transcription of data from past sighting reports, each succeeding form was put to use as soon as it was developed and approved. Accordingly, experience was obtained with each form in relation to past data, an important factor in the improvement of the quality and completeness of the later reports included in this study.

Coding System and Work Sheet The reduction of non-numerical data to numerical form is mandatory in the machine handling of data. Thus, the selection of the IBM punched-card system for analysis of data forced the adoption of a master coding plan. Since it was impracticable to transfer detailed data of an exact nature from the questionnaire to the IBM card, an intermediate transfer form, coordinated with the master code, was necessary. The master coding plan was evolved during the early stages of the preliminary analysis of data, and was reviewed by the panel of consultants before use. It was recognized that this system of coding would be the heart of the analysis, that is, the completeness of the facility for translation of data could make or break the study. Accordingly, every conceivable factor that might influence the identification of unidentified aerial objects was included, together with a wide range of variations within each factor. The original coding system (with minor corrections) was used throughout the translation of the original data with marked success. A copy of this system, called CODES, is enclosed as Exhibit B4, Appendix B. To facilitate the preparation of the punched-card abstract, an intermediate form called the WORK SHEET (later, the CARD BIBLE) was developed. Referenced to both the data from the questionnaire and the system of report identification, the WORK SHEET permitted an orderly transcription of data simultaneously by several people. In conjunction with the CODES, the WORK SHEET was used during the reduction of the original data to code form necessary for transfer to punched cards. A sample is included as Exhibit B5, Appendix B. After the analysis was under way, it became apparent that the mechanics of machine processing could be improved by incorporating in the IBM card system group classifications of certain factors requiring more than one column for discrete expression. In addition, the inclusion of certain data relating to the evaluation and bearing of the sun with respect to the observer was considered necessary. Finally, a critical examination of certain segments of the data indicated the need for the definition of a new factor relating to the maneuvers of the object or objects sighted. Prior to the start of the analytical study, it had been assumed that a combination of stated factors would, by inference, define the maneuver pattern. All these additions have been incorporated in a revised set of CODES and CARD BIBLE that are illustrated as Exhibits B6 and B7, Appendix B. However, at the time that the maneuver factor was determined to be critical, it was physically impracticable to make the required definitions and re-evaluate the original data. Therefore, no code for maneuverability has been included in the CODES, CARD BIBLE, or IBM cards.

Identification of Working Papers The actual reduction of data to IBM punched-card form presented a problem of mass transfer of figures by several workers. Recognizing that an orderly system of relating the original data to the questionnaire, the WORK SHEET, and the IBM card was imperative, a scheme of SERIAL NUMBERS was developed to answer this need. The first data consisted of a series of letter-file folders identified by the year and location of the sighting or sightings they contained. The number of reports of sightings in a single folder varied from 1 to over 20. Under these conditions, there was a great possibility for incorrect transcription of data, duplication of transcription, or misplacement of intermediate forms. Further, it was considered desirable to relate all sightings of the same object or objects to one another. The concept of a four-digit serial number (major), followed by a two-digit subserial number (minor), was adequate to fulfill these requirements. To expedite handling of the data, temporary serial numbers were assigned until each report had been evaluated and the phenomenon had been placed in a category of identification. The use of temporary serial numbers permitted the consolidation of duplicate reports from apparently diverse sources, such as a teletype message and an Air Force Form 112. However, this consolidation was made ONLY when it could be proved conclusively that the sources of the two documents were one and the same. Factors of the observer's location, date and time of observation, description of the phenomenon, and finally, the name of the observer were considered. In this manner, the assignment of major serial and minor subserial numbers in continuous series was made only to the reports accepted for the statistical study. It is believed that the reports accepted represent unique and unduplicated instances of sightings. In the establishment of the serial-number system, it was necessary to define certain terms, so that a standard interpretation could be achieved. The terms and corresponding definitions were: OBSERVER - Any witness reporting to a proper authority that he had seen unidentified aerial objects. SIGHTING - The report or group of reports of the same observed phenomenon that remained unidentified to the observer or observers, at least until reported.

SINGLE OBSERVATION - A SIGHTING consisting of a single report from (1) one OBSERVER with no knowledge of additional OBSERVERS of the same phenomenon, or (2) a group of witnesses of the same phenomenon, each cognizant of the others. The witness who made the report is called a SINGLE OBSERVER. MULTIPLE OBSERVATION - A SIGHTING consisting of several reports from OBSERVERS of the same phenomenon who were cognizant of each other. The witnesses who made reports are called MULTIPLE OBSERVERS. ALL SIGHTINGS - (1) The group of reports consisting of one report for each OBSERVER, including both SINGLE and MULTIPLE OBSERVERS. (2) The questionnaire, work sheet, and IBM card representing the report from each OBSERVER - in other words, the representation of each report accepted for the statistical study. UNIT SIGHTINGS - (1) The group of reports consisting of one report for each SIGHTING, including all the reports of SINGLE OBSERVATIONS and the one most representative report from each MULTIPLE OBSERVATION. (2) The questionnaire, work sheet, and IBM card representing the report for each SIGHTING accepted for the statistical study. A major serial number (four digits) was assigned to each sighting, segregating the year of occurrence by selection of limits for each year, as follows: 0001 to 0500 reserved for 1947 0501 to 1000 reserved for 1948 1001 to 1500 reserved for 1949 1501 to 2000 reserved for 1950 2001 to 2500 reserved for 1951 2501 to 4900 reserved for 1952 While this scheme would serve to identify any individual sighting, identification of each report and its subsequent forms was necessary. The minor subserial numbers (two digits) fulfilled this requirement. For all SINGLE OBSERVATIONS, a major serial number followed by two (2) zeros, for example, 2759.00, was sufficient identification. For MULTIPLE OBSERVATIONS, the major serial number followed by a series of two-digit numbers ranging from 00 to 99 was used to identify the individual reports. In general, the most complete report from the most reliable observer of that

MULTIPLE OBSERVATION was identified with the .00 subserial number. As an example, a MULTIPLE OBSERVATION consisting of six sighting reports would have the following serial numbers: 1132.00 representing the best report and observer 1132.01 representing an additional observer 1132.02 representing an additional observer 1132.03 representing an additional observer 1132.04 representing an additional observer 1132.05 representing an additional observer During the course of the transcription of the data to machine card form, it became obvious that certain reports could have been independent observations of the same phenomenon. So, if the presentation of an analysis based on one report for each sighting was valid (the concept of UNIT SIGHTINGS), a presentation of an analysis based on one report for each phenomenon should be valid also. Further, the examination of data relating to the actual number of phenomena was considered to be the proper basis for assessing the probability of technological developments outside the range of present-day scientific knowledge. Therefore, a designation of OBJECT SIGHTINGS was established, with the following definition: OBJECT SIGHTING - (1) The group of reports consisting of one report for each phenomenon. (2) The questionnaire, work sheet, and IBM card representing a report for each phenomenon accepted for the statistical study. In brief review, ALL SIGHTINGS refer to all reports, UNIT SIGHTINGS refer to actual sightings, and OBJECT SIGHTINGS refer to the assumed number of phenomena. It must be recognized that the process of identifying OBJECT SIGHTINGS was deductive, while that for UNIT SIGHTINGS was definitive. A conservative approach was adopted in the determination of OBJECT SIGHTINGS, using the factors of date and time of observations, location of observers, duration of observations, and range, bearing, track direction, and identification of the phenomena. Any error of selection of OBJECT SIGHTINGS will tend to be in the direction of reducing the actual number of phenomena observed (several instances of UNIT SIGHTINGS that might be one OBJECT SIGHTING were noted, but the evidence was not conclusive enough to justify consolidation of the reports). Following the determination of OBJECT SIGHTINGS, a series of serial numbers, called the INCIDENT SERIAL NUMBERS, was established to facilitate any future study of a specific object sighting. Each reported sighting that relates to an OBJECT SIGHTING received the same incident serial number, a four-digit code paralleling the major serial number series.

For machine manipulation, it was desirable to be able to select the sample of cards (all reports, all sightings, or all phenomena) to be included in a particular study. The concept of a SIGHTING IDENTIFICATION NUMBER was evolved to fill this desire. Using one column of the IBM card, and the correlated working papers, the code for this function was developed. Multiple punching eliminated the need to use several columns for discrete expression of the variations. Selection of the proper number in this column thus permitted selection of the desired sample of cards. Evaluation of Individual Reports Evaluation of sighting reports was recognized as a crucial step in the preparation of data for statistical treatment; inconsistent evaluations would have invalidated any conclusions to be derived from this study. A method of evaluation was, therefore, determined simultaneously with the development of the questionnaire, the coding system, and the work sheet. It is emphasized that all phases of evaluation, even including the tedious preparation of the original data for statistical treatment, were entrusted only to selected, specially qualified scientists and engineers. Evaluation consisted of a standardized procedure to be followed for: (1) the deduction of discrete facts from data which depended on human impressions rather than scientific measurements, (2) the rating of the observer and his report as determined from available information, and (3) the determination of the probable identification of the phenomenon observed. Categories of identification, established upon the basis of previous experience, were as follows: Balloon Astronomical Aircraft Light phenomenon Birds Clouds, dust, etc. Insufficient information Psychological manifestations Unknown Other The first step in evaluation, the deduction of discrete facts from subjective data, required certain calculations based on the information available in the sighting report. An example was the finding of the approximate angular velocity and acceleration of the object or objects sighted. Care was taken during this phase of the work to insure against the deduction of discrete facts not warranted by the original data. Thus, even though there was a complete lack of any valid evidence consisting of

physical matter in any case of a reported unidentified aerial object, this was not assumed to be prima facie evidence that "flying saucers" did not exist. In those cases in which an attempt to reduce the information to a factual level failed completely, the report was eliminated from further consideration, and thus not included in the statistical analysis. About 800 reports of sightings were eliminated or rejected in this manner. Most of these reports were rejected because they were extremely nebulous; the rest were rejected because they contained highly conflicting statements. The second step in evaluation, the rating of the observer and his report, logically followed the first step, the reduction of the data to usable form. Ratings were assigned on the basis of the following factors of information, considered in relation to one another: (1) The experience of the observer, deduced from his occupation, age, and training; (2) The consistency among the separate portions of the description of the sighting; (3) The general quality and completeness of the report; (4) Consideration of the observer's fact-reporting ability and attitude, as disclosed by his manner of describing the sighting. In cases in which insufficient information was available to make a judgment of the observer or report, none was made, but the report was accepted for the statistical study. The third step in the process of evaluation, the attempted identification of the object or objects sighted, was done twice, first by the individual who made the transcription of the data (the preliminary identification), and later (the final identification) by a conference of four persons, two representatives from ATIC and two from the panel of consultants. Although representatives of ATIC participated in making the final identifications, it must be emphasized that any previous identification of a sighting made by ATIC was not introduced or referred to in any way. In the coding system, the choices provided for final identifications were based on ATIC's previous experience in analysis of the data. They had found that the majority of sightings could be classified as misinterpretations of common objects or natural phenomena. Accordingly, categories for objects most frequently present in the air were provided. Balloons, aircraft, astronomical bodies (such as meteors), birds, and clouds or dust were recognized as major categories. The less frequent, but common objects, such as kites, fireworks, flares, rockets, contrails, and

meteorological phenomena like small tornadoes, were collected into a category called OTHER. A separate category for the uncommon natural phenomena associated with light reflections or refractions, such as mirages, sun dogs, inversion-layer images, and distortions caused by airborne ice, was established with the title of LIGHT PHENOMENON. Categories for INSUFFICIENT INFORMATION, PSYCHOLOGICAL MANIFESTATIONS, and UNKNOWN were provided for the sightings that could not be fitted into the preceding identifications. An explanation of their use follows: INSUFFICIENT INFORMATION - This identification category was assigned to a report when, upon final consideration, there was some essential item of information missing, or there was enough doubt about what data were available to disallow identification as a common object or some natural phenomenon. It is emphasized that this category of identification was not used as a convenient way to dispose of what might be called "poor unknowns", but as a category for reports that, perhaps, could have been one of several known objects or natural phenomena. No reports identified as INSUFFICIENT INFORMATION contain authenticated facts or impressions concerning the sighting that would prevent its being identified as a known object or phenomenon; PSYCHOLOGICAL MANIFESTATIONS - This identification category was assigned to a report when, although it was well established that the observer had seen something, it was also obvious that the description of the sighting had been overdrawn. Religious fanaticism, a desire for publicity, or an over-active imagination were the most common mental aberrations causing this type of report; UNKNOWN - This designation in the identification code was assigned to those reports of sightings wherein the description of the object and its maneuvers could not be fitted to the pattern of any known object or phenomenon. For the purposes of this study, two groups of identifications were recognized, the KNOWNS (including all identification categories except the UNKNOWNS) and the UNKNOWNS. All possible identifications provided in the code system, except INSUFFICIENT INFORMATION and UNKNOWN, could be assigned according to two degrees of certainty, designated "Certain" and "Doubtful".

A "Certain" identification indicated a minimum amount of doubt regarding the validity of the evaluation. By "rule-of-thumb" reasoning, the probability of the identification being correct was better than 95 per cent. A "Doubtful" identification indicated that the choice was less positive, but that there was a better than even chance of being correct. It is emphasized again that, as was true for other phases of evaluation, preliminary and final identification was entrusted only to scientists and engineers who, in addition to their broad scientific background, had received instruction, where necessary, in specialized subjects. The panel of consultants provided background information for this instruction. Many of the cases representing unusual features or maneuvers were submitted to and discussed with various members of the panel of consultants prior to the final identification. Consistency in the application of the knowledge necessary for making identifications was maintained by frequent collaboration among the personnel involved, and systematic spot checks of the work. In addition to the general fund of knowledge required to identify satisfactorily a reported unidentified aerial object, an attempt was made to correlate specific data such as flight plans of aircraft, records of balloon releases, weather conditions, and an astronomical almanac with the reported sighting. The procedure followed in making final identifications deserves explanation because of the importance assumed by the identification as a basis for statistical treatment. As was mentioned, a conference of four qualified persons, two from ATIC and two from the panel of consultants, decided upon the final identification for each sighting report. This work was done at ATIC, periodically, as reports became ready. During an identification conference, each sighting report was first studied, from the original data, by one person. If that person arrived at a decision, it was checked against the preliminary identification; if the two identifications were the same, the report was appropriately marked and considered finished. If the two identifications did not agree, the report was considered later by everyone participating in the conference until a group decision could be made. If an evaluator was unable to categorize the report as one of the common objects or as a natural phenomenon, and his opinion was that the sighting should be recorded as UNKNOWN, a group decision was also required on that report before it was considered finished. A group decision was necessary on all reports finally recorded as UNKNOWN, regardless of what the preliminary identification had been. In cases where a group decision was not made within a reasonable time, the report was put aside and later submitted to certain members of the panel of consultants for their opinions. If, after this, disagreement continued to exist, the report of the sighting was identified as UNKNOWN.

Upon completion of final identifications, all data were transferred to IBM cards, preparatory to analysis. ANALYSIS OF THE DATA Broadly stated, the problem at this point consisted of the judicious application of scientific methods of categorizing and analyzing the subjective data in reports of sightings of unidentified aerial objects. It was recognized that an approach to this problem could best be made by a systematic sorting and tabulation program to give frequency and percentage distributions of the important characteristics of sightings. A suggestion that an attempt be made to anticipate all questions that might be asked in the future about a sighting or a group of sightings, and to provide answers, was rejected. The systematic approach also made it possible to develop a detailed reference manual of the attributes of the sightings included in this study. Thus, at the beginning of the analysis, a detailed plan was developed for sorting, counting, and tabulating the information from the punched-card abstracts of reports of sightings. It was believed at the time, and later substantiated, that the results of the program for sorting and tabulating would serve as a guide for the more sophisticated treatment involving statistical methods. Also, it was anticipated that any patterns or trends that might be found could be subjected to concentrated study in the hope of discovering significant information relating to the characteristics of "flying saucers". Further, it was believed that these trends could serve as certain of the criteria of validity for any concepts (models) developed in the attempt to discover a class of "flying saucers". The three parts of this study (1) the sorting and tabulation program, (2) the advanced study of the results of that program, and (3) the investigation of the possibility of conceiving a model of a "flying saucer" from descriptions reported, are discussed in sections entitled "Frequency and Percentage Distributions by Characteristics", "Advanced Study of the Data", and "The 'Flying Saucer' Model". Frequency and Percentage Distributions by Characteristics The original conception of this study assumed the availability of sufficient data to describe adequately the physical appearance, maneuver characteristics, range, direction, and probable path of the object or objects observed. However, familiarity with the data, acquired during the

INDEX OF FORMS. Exhibit B1. Tentative Observers Data Sheet, page 259. Exhibit B2. Tentative Observers Questionnaire, 267. Exhibit B3. U. S. Air Force Technical Information Sheet, 277. Exhibit B4. Codes for Work Sheet, 289. Exhibit B5. Work Sheet, 297. Exhibit B6. Codes for Card Bible, 301. Exhibit B7. Card Bible, 309. Exhibit B8. Example of an IBM Card, 313. [page number: 257 and 258]

32. In the following sketch, imagine that you are at the point shown. Place an "A" on the curved line to show how high the object was above the horizon (skyline) when you first saw it. Place a "B" on the same curved line to show how high the object was above the horizon (skyline) when you last saw it. 33. In the following larger sketch place an "A" at the position the object was when you first saw it, and a "B" at its position when you last saw it. Refer to smaller sketch as an example of how to complete the larger sketch. [page 284]

EXHIBIT B8. EXAMPLE OF AN IBM CARD. IBM-SERVICE BUREAU. Left labels: Blank - Unused; Sighting Reliability; Observer Reliability; Maneuvers; Final Elevation; Disappearance Bearing; Angular Acceleration; Light Brightness; Shape; Number; Light Color; Sound; Duration of Observation (Group, Duration, Time Units); Angle of Elevation of the Sun (Group, Hundredths, Degrees); Strategic Area; Regional Area; Latitude (Hundredths, Degrees); Observed G.C. Time (Minutes, Hours), Year, Month, Day; Subserial Number; Serial Number. Right labels: Blank - Unused; Final Identification; Report Reliability; Observer Occupation; Orientation; Initial Elevation; Initial Bearing; Angular Velocity; Size; Speed; Color Group; Object Color; How Observed; Sun Bearing Angle (Hundredths, Degrees); Hour Angle (Hundredths, Degrees); Longitude (Hundredths, Degrees); Local Sun Time (Minutes, Hours, Key); Incident Serial Number; Sighting Identification. [page 315 and 316]