RETURN
to History Center HomePage or RETURN
to the Publications List
SECTION ONE: SUMMARY OF PROJECT ACTIVITIES AND FINDINGS
SECTION TWO: PROJECT RECOMMENDATIONS
EXECUTIVE SUMMARY
[Back to Top]
Although the multi-institutional collaboration is increasingly the organizational framework for scientific research, it has received only incidental attention from scholars. Without a dedicated effort to understand the process of collaborative research, the records necessary for efficient administration, for historical and management studies, and for posterity, will be largely scattered or destroyed. The Center for History of Physics of the American Institute of Physics (AIP) has worked to redress this situation with a multi-stage investigation. The aim of the long-term study was to identify patterns of collaboration, define the scope of the documentation problems, field-test possible solutions, and recommend future actions. The first phase of the study addressed collaborations in high-energy physics; the second phase addressed space science and geophysics (including oceanography). In this, the third and last phase of the study, we studied briefly five disciplinary fields: ground-based astronomy (divided into observatory builders and observatory users), materials science, heavy-ion and nuclear physics, and medical physics, plus one area we named computer-mediated collaborations.
In general, we continued to find that most scientists, like other groups, only keep documents when they think they are useful to them. Good records-keeping may be acknowledged by all as necessary while the experimental process is alive, but when the experiment is over, records can easily be neglected, forgotten, or destroyed. We also have accumulated evidence that a major obstacle in documenting multi-institutional collaborations is the lack of archival programs at some critical institutions. Most administrators fail to consider the documentation of collaborations, no matter how significant, as their responsibility.
In our archival analysis, we coupled the organizational patterns of multi-institutional collaborations to the patterns of records creation, retention and destruction, and likely locations of records. Our reports include appraisal guidelines--to assist archivists and others with responsibilities for selecting records for long-term preservation--and the identification of a small set of "core" records that should be permanently preserved for all collaborations in a given disciplinary field, as well as the more extensive array of documentation that scientist-administrators, historians, and others will need in order to understand collaborations of outstanding significance. We also developed a set of recommendations directed to both scientific and archival institutions to promote preservation of valuable documentation. These products were based on a number of sources.
We found that in ground-based astronomy, collaborations that build observatories have similar structures with a Board of Directors overseeing an individual in charge of dealing with observatory construction and committees of scientists providing advice and building scientific instruments. Within this structure, these collaborations varied in how much they relied on external contracts and professional managers. As a class, observatory-building collaborations involve unique documentation difficulties that arise from the circumstances of their financial support. Academic observatories are built with funds primarily from a variety of non-Federal sources with less stringent records requirements. The national observatories, supported by the NSF as contract laboratories, do not create Federal records and are not required by law to maintain records management programs or secure records of archival value. Our project recommendations include actions to address these obstacles to documenting the building of observatories.
By contrast, ground-based astronomy collaborations that used multiple radio observatories for interferometry were of two types. One was the minimally formalized collection of astronomers who shared an understanding of what was needed to take data successfully and who sought to expand the wavelengths at which interferometry was possible. The other was the formal consortium of observatories that helped smaller groups of astronomers to coordinate the use of several telescopes for observations at wavelengths that were comfortably within the state-of-the-art. The difficulties of documenting these collaborations are even more complex. They leave a scanty paper trail (except for observational data) because they require little or no dedicated funding and only minimal organizational structure; furthermore, they neither design nor build the instrumentation they use. Finally, our principal hope for documenting observatory-using collaborations must rely on changing the documentary habits of individual scientists.
Materials science collaborations were divided between those that built beamlines to use accelerators at Department of Energy National Laboratories and those that coordinated the research performed in the laboratories of member institutions. The organization of the accelerator users varied widely depending on how many uses and users the collaboration wanted the beamline to serve. However, they consistently created ample documentation because of the records requirements of these laboratories and the records management programs of DOE. The organization of the collaborations that coordinated the laboratories of its members was more homogeneous, with a committee of leading researchers from each institution invariably being the most important decision-making body within the collaboration. Preservation of their valuable records will depend largely on the ability of academic archives to include these new organizations among their responsibilities, because the terms on which they were funded, especially those funded as NSF centers (the Science and Technology Centers and the Materials Research Science and Engineering Centers), enabled these collaborations to solicit and fund proposals from scientists at their member institutions.
Nuclear and heavy-ion physics recapitulated the organizational forms with which we became familiar in our earlier study of high-energy physics. These collaborations built multi-component detectors to take data at accelerator laboratories. The collaboration's scientific leader served as the "spokesperson" and presided over highly participatory discussions of the collaboration's strategy and tactics. An engineer or a technically minded scientist tracked construction of the detector's components at the several member institutions and oversaw their integration at the accelerator laboratory. These collaborations should be adequately documented under the auspices of the records programs at DOE and other major accelerator laboratories.
The three collaborations we studied in medical physics varied widely in their organizations. One was rigidly organized to insure that the participants adhered to a detailed research protocol; one was loosely organized to enable participants to explore and compare various possibilities for developing a new diagnostic procedure, and one was moderately organized to balance the need to integrate different forms of expertise with the need to grant autonomy to different experts. We were unable to reach generalizations about which organizational form was typical or to determine whether we had encountered the full range of medical physics collaborations. In addition, the AIP Center's experience in documenting the contributions of individual practitioners and its knowledge of the field's research centers and the key funding agency--the National Institutes of Health--was much more limited in comparison to the other fields. At best, our appraisal guidelines and project recommendations are suggestive./P>
We examined computer-mediated collaborations to explore how computation capabilities and the computation problems in physical research are being addressed. We found programs that support collaborations of computer scientists and physical scientists who were using computation problems in physical research to generate computer techniques with broad applicability. These new kinds of projects should continue and thrive over the near future. Like medical physics collaborations, these were diversely organized, but the diversity resulted from different responses to the same problem: how best to identify, foster, and satisfy the shared interests between computer and natural scientists. In all but one case, the focus on computer science and computer techniques did not create new documentation challenges. The exception was the testbed for a national collaboratory, which generated a plethora of records by creating electronic venues for scientific discussion and debate. Because the purpose of this area of the study was to provide the AIP with a glimpse of the likely structures of future multi-institutional collaborations, it did not lead to records appraisal guidelines or project recommendations.
Our reports include detailed recommendations to promote preservation of valuable documentation for future use by science administrators, policy-makers, and historians and other scholars. The single most important recommendation urges Federal science agencies to employ professional archivists or records advocates (i.e., someone who can argue on behalf of the historical value of records) as part of their records management staff. It has been seen how effective such professionals have been at scientific settings, such as some of the laboratories of the Department of Energy. This addition would help the National Archives understand the unique records creation process at each of the science agencies while increasing the effectiveness of the agencies' records management programs.
The Phase III Study of Multi-Institutional Collaborations was guided by a working group of distinguished scientists, science administrators, archivists, historians, and sociologists. It was supported by the AIP, the Andrew W. Mellon Foundation, the National Historical Publications and Records Commission, and the National Science Foundation.
SECTION ONE: SUMMARY OF PROJECT ACTIVITIES AND FINDINGS [Back to Top]
I. PROJECT GOALS, METHODOLOGY, AND ACTIVITIES [Back to Top]
A. Purpose and Methodology of the Long-Term Study of Collaborations.
Since World War II, the organizational framework for scientific research is increasingly the multi-institutional collaboration. However, this form of research has received slight attention from historians, sociologists, and other scholars. Without a dedicated effort to understand such collaborations, policy makers and administrators will continue to have only hearsay and their personal memories to guide their management; even the records necessary for efficient administration, for historical and management studies, and for posterity, will be largely scattered or destroyed.
The Center for History of Physics of the American Institute of Physics (AIP), in keeping with its mission to preserve and make known the record of modern physics and allied sciences, is working to redress this situation with a multi-stage investigation into areas of physics and allied sciences where large collaborations are prominent. In order to locate and preserve historical documentation, we must first get some idea of the process of collaborative research and how the records are generated and used. Hence, we made a broad preliminary survey, the first of its kind, into the functioning of recent research collaborations that include three or more institutions. Our study was designed to identify patterns of collaborations since the mid 1970s and define the scope of the documentation problems. Along the way, we built an archives of oral history interviews and other resources for scholarly use. We use our findings to recommend future actions and promote systems to document significant collaborative research.
The goal of the study is to make it possible for scholars and others to understand these transient "institutions." As collaborative research becomes ever more pervasive in our world, archivists and records officers cannot avoid addressing documentary issues they raise. These reports are designed to help responsible parties develop appropriate goals and set priorities to save the records of greatest historical value.
The long-term study began in 1989. Phase I, which focused on high-energy physics, was completed in 1992. Phase II, which addressed collaborative research in space science and geophysics, was completed in 1995. This report completes Phase III's study of five new disciplinary areas (ground-based astronomy observatory builders, ground-based astronomy observatory users, materials science, heavy-ion physics, and medical physics), and a category we named computer-mediated collaborations.
Whereas earlier phases of the AIP Center's long-term study had focused on one or two disciplinary areas, in our third and last phase, we examined more briefly five areas in which multi-institutional collaborations were well-established (or, in one area, just emerging) as vehicles for research. In choosing this approach, we were aiming to round out the coverage of physics and allied fields, to investigate the feasibility of reaching reliable conclusions with less intensive collection of data, and to look toward the future: What directions would multi-institutional collaborations take? What new documentation problems might they present a decade from now? This last objective resulted in two decisions: (1) to include more recent projects among our case studies, e.g., projects that were not yet completed and (2) to include a category we named computer-mediated collaborations, a category of collaborations that made use of brand-new and dynamic computer techniques that were becoming widespread.
Phase III of the AIP Study of Multi-Institutional Collaborations has been guided by a Working Group of distinguished scientists, science administrators, and archivists (see Working Group list) who joined in reviewing its findings and recommendations. In addition, sociologists Wesley Shrum and Ivan Chompalov assisted us in designing the project's methodology and research instruments. The project was directed by Joan Warnow-Blewett with the assistance of Spencer R. Weart. Joel Genuth served as project historian, and Anthony Capitos as project archivist (until 1997).
B. The Phase III Study of Collaborations
For our third, and last, phase of work 23 projects were chosen to serve as case studies: seven for ground-based astronomy, eight for materials science, two for heavy-ion and nuclear physics, three for medical physics, and three for computer-mediated collaborations. For a list of the case studies selected, see Report No. 2, Section One: "Selected Case Studies." Considerable time and research on the part of project staff and consulting sociologists was devoted to the design and construction of a question set that would make sense to interview subjects and, at the same time, meet project needs for historical, sociological, and archival data. The AIP staff selected collaboration scientists who could serve as reliable informants, and conducted a total of 78 interviews.
During previous phases of the AIP Study, scores of site visits were made to university archives, government laboratories, FFRDCs (Federally Funded Research and Development Centers), and corporate laboratories. During this third phase, site visits made by project staff focused on Federal science funding agencies and the National Archives, where we discussed archival issues and records policies.
The analyses of project interviews by the project historian and consulting sociologists provided discrete images of the institutional structures and functions that had the greatest impact on the project formation, organization and management, data analysis, and dissemination of projects. These findings, combined with the project's archival analysis, site visits, and our previous knowledge of archival institutions, provide the most reliable available guide to identifying areas of documentation problems and potential solutions.
Support for this phase of the project was provided by the American Institute of Physics, the Andrew W. Mellon Foundation, the National Historical Publications and Records Commission of the National Archives and Records Administration, and the National Science Foundation.
II. HISTORICAL ANALYSIS [Back to Top]
As in the previous phases of the AIP Study, a major empirical basis for project recommendations is the historical qualitative analysis of interviews with participants in multi-institutional collaborations. An understanding of the social processes that require collaborations to generate and use records is essential to conceptualizing what is worth saving and what is feasible to save. Secondarily, such an understanding is the basis for observations about where the organizational framework for multi-institutional collaborations may affect the social relations and careers that are necessary for scientific research.
For Phase III, we studied 23 collaborations in five areas of physics and allied sciences: ground-based astronomy observatory builders, ground-based astronomy observatory users, materials science, heavy-ion and nuclear physics, and medical physics. In addition, we included computer-mediated collaborations, which are highly important for the near future. This breadth of coverage required us to conduct fewer interviews per collaboration than in the previous phases. To facilitate quantitative-sociological analysis, we asked more closed-ended questions than in the previous phases. Though less numerous and intensive, these interviews proved more than adequate for generating pictures of how collaborations have functioned in these research specialties.
Each scientific specialty is discussed in turn. For each specialty, we strive to characterize those aspects of multi-institutional collaborations that are most important for archival policies and practices.
A. Ground-Based Astronomy: Observatory Builders
We conducted interviews on four collaborations that built astronomical observatories. Our focus was the design and construction of the observatories, not their use. Design and construction were subject to collaboration management, but their use, in general, was determined by the individual institutions. Our selection of collaborations did not include any involving national observatories or the Association of Universities for Research in Astronomy, which manages the national observatories.
These university-dominated collaborations were motivated by astronomers' frustrations with the quality of their universities' facilities and with the difficulty of obtaining observing time at national observatories. To obtain enough capital needed to make a university-based observatory that could match or outperform national observatories in some research area, university astronomers needed to combine their intra-university funds and jointly raise additional funds. Federal funding was, at most, an important supplement to other sources. Finding partners was an awkward exercise for would-be instigators, because astronomy departments that could raise funds usually had their own plans for capital improvements. Personal connections between astronomers were usually necessary for their department to learn of their common interests. Once instigators had two of three institutions lined up to be major investors in an observatory, other, often foreign institutions were welcome to provide smaller parts of the remaining funds.
These collaborations were formalized through signed, legal agreements among the institutional members. The basic principle behind all the agreements was that collaboration members received observing time in proportion to their contributions. The agreements specified a governing structure that in all cases vested intra-collaboration authority in a Board of Directors comprised of representatives from the member institutions. However, one individual was most responsible for the physical construction of each observatory (though the individual's title varied from collaboration to collaboration). Usually, these collaborations organized advisory committees of scientists from the member institutions to deliberate on science-engineering trade-offs, to set specifications for scientific instruments to be used with the observatory, and to plan commissioning measurements. Also, these collaborations usually organized external panels to perform design reviews of major observatory components.
Within this common structure these collaborations varied mostly by the degree to which they professionalized the development and construction of their observatories. The professionally managed collaborations empowered a project manager to let and oversee contracts for the development, construction, and integration of the major observatory components. The moderately professionalized selected a scientist from one of the member institutions to oversee contracts that the members' universities awarded for the major observatory components. The self-managed did much of the work in-house, using the labor of graduate students and postdocs to conserve cash. These varying arrangements reflect the ambivalence of astronomers about the trade-off between achieving efficiency by centralizing project management and maintaining their individual institutions' prerogatives and traditions.
The major power retained by the member institutions was to determine how the observatory would be used. Usually each institution has its own "Time Allocation Committee" to consider proposals from its own scientists, though the self-managed collaboration centralized consideration of observing proposals. Additionally, most of these collaborations commissioned member institutions to design and build scientific instruments to be used at the observatory. Once the collaboration set broad specifications for the instruments, the instrument builders were able to proceed in near total autonomy.
Because use of all of these observatories has been determined by judgement of proposals, none of the collaborations managed the topics addressed through the use of its facilities. In most cases, the collaborations have left data processing and analysis almost entirely in the hands of individual observers. Observatory operations--e.g., making possible remote operation of the observatory from member institutions--have been the primary computational problem addressed in the context of the collaboration.
None of these collaborations ever lost institutional members, which is hardly surprising given the institutions's investment of their own funds. Each has or apparently will succeed in building and operating its observatory, but only a professionally managed collaboration finished on time and budget. All have been or will be used for a wide variety of studies. Author lists for publications based on use of these observatories include only individuals involved in performing the observations, not individuals who developed the observatory and its scientific instruments.
B. Ground-Based Astronomy: Users of Observatories
We conducted interviews on three collaborations that used existing observatories. Our focus was mostly on data collection, analysis, and dissemination, because these collaborations relied on the observatories or other scientific programs to generate the instrumentation they needed. None of the collaborations we studied performed sky surveys or interferometry with optical telescopes.
These collaborations were mostly comprised of university-based radio-astronomy observatories, whose affiliated scientists wished to perform very long baseline interferometry (VLBI). Circa 1970, informal collaborations succeeded in obtaining interference fringes by "correlating" their independently recorded data tapes. Success spawned imitation and competition. By the mid-1970s, competitive astronomers did not want to continue relying on one another for the operations of each other's observatories, and observatory directors wanted formal rules to govern the coordinated use of their observatories. Astronomers resorted to forming two types of collaborations, which dominate our sample. One involved formal arrangements among the radio observatories for scheduling and supporting VLBI at conventional wavelengths; the other was the continued use of informal collaborations to attempt to expand the wavelength regime in which interferometry was possible. The former required a formal agreement and designated itself a consortium; the latter just required that the interested astronomers propose the observation to their respective observatories. Our sample did not include collaborations that performed sky surveys, optical interferometry, or VLBI observations at conventional wavelengths.
Neither type of collaboration had much organizational structure. Issues of fiscal accountability did not inspire these collaborations to develop an organizational structure, because their costs were covered within the budgets of the observatories. Furthermore, the consortium of observatories required little organization because it did not have to manage the work it made possible. Once its chairman had negotiated with observatory directors for observing time and resources for VLBI observations, and once its secretary had collected reviews of proposals for VLBI observations and scheduled the best proposals, the consortium's work was done and individual observers took over. An annual meeting, in conjunction with the American Astronomical Society meeting, sufficed for the community of VLBI researchers to elect officers and make known their views on opportunities for and obstacles to VLBI observations. As for the informal collaborations, their members's mutual understanding of what VLBI observations required obviated the need for organizational structure. The scientist acknowledged as having the deepest personal investment in an observation moderated the collaboration-wide discussions needed to produce an observing plan, dealt with the observatory directors, and saw to obtaining any additional equipment the collaboration needed. No further organization of tasks was required; individual participants who best knew a particular observatory took responsibility for the technically tricky task of configuring the observatory for VLBI observations.
Internationalism, though common in VLBI to increase the lengths of the baselines, also did not lead to any elaborate organization. Because there were no funds to account for or technical developments to coordinate, international collaboration was only marginally more onerous than trans-continental collaboration within the United States.
Neither type of collaboration designed or built the instrumentation it used. The radio observatories themselves research and develop electronics for radio-wave detection and amplification. Support for the development of instrumentation peculiar to VLBI comes from NASA to support geodetic measurements of continental drift.
The formal consortium played no role in data analysis, interpretation, and the dissemination of scientific findings. These were entirely the responsibility of the individual observers who gained access to the observatories under the consortium's auspices. The informal collaborations had to correlate the individual data sets from the participating observatories in order to have a hope for scientific success. (Correlation generates the interference patterns that would have been produced had the observatories been hard-wired together to form a physical interferometer.) Correlation has been thus the central drama of VLBI observations, as participants struggled to find a synchronized playback that would yield interference fringes and justify the time and effort spent in acquiring the data. Once correlated, a data set still required considerable processing before it could be the basis for a scientific interpretation. Only the participants most interested in the objects being observed worked on processing correlated data and drafting manuscripts for publication in scientific journals. Participants who were more concerned with data acquisition and correlation than with the objects being observed reviewed manuscripts before their submission, and had the right to have their names removed from the author list. All participating scientists and engineers from observatories whose data successfully correlated with others' were entitled to be authors.
These two types of collaborations reflect a commitment to maintaining the ability of individual astronomers to claim credit for observations of astronomical objects. Ongoing, informal collaboration was the province only of the technically ambitious few, who were more intent in their collaborative research on expanding the wavelength regime in which VLBI could be used than in observing particular objects. Once significant numbers of astronomers became convinced of the feasibility and fertility of performing VLBI at a particular wavelength, the purpose of collaboration shifted to formalizing relations among the observatories in the interest of freeing individual astronomers from the need to cooperate with competitors. When the consortium organized centimeter wavelength VLBI, the technically ambitious began exploring prospects for millimeter VLBI. It remains to be seen whether the process will repeat itself.
C. Materials Science
We conducted interviews on eight collaborations in materials science. The scope of their collaborative activities varied considerably even within the two broad categories of collaborations that worked at accelerator laboratories and collaborations that coordinated the research of laboratories within their member institutions.
These collaborations formed for one of two purposes: to focus materials scientists at several institutions on investigating a particular set of materials or to build a beamline for materials science research at a national accelerator laboratory. They were all multi-sectoral with universities and corporation being major participants in most. In some, competing corporations jointly participated. Internationalism was rare, and in cases related to national defense or economic interests, it was banned outright. Government agencies were frequently a direct source of funds, but the legal arrangements they made with the collaborations varied; corporations shared costs when they participated.
All the collaborations that did not use accelerators owed their existence partly to changes in the organization or authority of funding agencies. This is not to say that the government has foisted collaborations on an unwilling community, but rather that the significance of studying particular materials and the prospects for acquiring significant government funds were together so alluring that the participating institutions bent their customary operations to accommodate each other.
Accelerator-using collaborations in materials science mostly formed in response to the opportunity to develop customized, novel beams and complementary detectors for examining classes of materials. They varied widely in their scientific breadth from studying a couple of related materials to supporting as many disciplines as their instigators found feasible.
Geographic proximity was a significant factor in the formation of the non-accelerator collaborations, but they rarely ended up regional in scope. They either expanded geographically in order to include expertise that was essential to win funding, or important individual members changed employers and brought their new institution into the collaboration. Geographic proximity was far less important to the formation of accelerator-using collaborations--except insofar as having a participating institution close to the accelerator laboratory was obviously convenient.
As with most other collaborations, drafting a proposal was the central challenge to the formation of all materials science collaborations. Defining a "complete" or "excellent" proposal was more ambiguous for the non-accelerator collaborations because few of their specific tasks had to be done within a collaborative framework. To defuse the criticism that they were administrative fictions created to obtain funding that their members could not have individually obtained in national competition, some proposal authors claimed their collaborations to be "device-oriented"--meaning they needed a collaboration to cover all the facets of using novel materials for a technological purpose. Others claimed their collaborations would overcome the organizational obstacles that made it hard to mobilize the quantity and quality of resources that research into the novel materials merited. Device-oriented collaborations included corporate competitors, and formalizing these collaborations required the instigators to write an intellectual property agreement that their institutions found acceptable. In all the cases we studied, these negotiations were onerous.
Ultimate authority for the non-accelerator collaborations was usually vested in an inter-institutional Board of Directors that typically included the research administrators overseeing each institution's participation in the collaboration and other representatives from the participating institutions. Once this board set the broadest fiscal and personnel policies within which the collaboration was to operate, it had no role unless called on to decide internal disagreements.
Substantive policies, like the collaboration's division of labor and allocation of resources, were set by a committee below the Board of Directors. This committee, whose name varied from collaboration to collaboration, usually included the overseeing research administrators and scientists representing research areas. Daily management of affairs, especially relations with the funding agency, were the responsibility of a collaboration director, who was usually also the overseeing research administrator for the institution that was fiscally responsible for the collaboration's funding.
By contrast, the accelerator-using collaborations usually did not create much of an authority structure. They did not need a Board of Directors, because their participants' mutual understanding of what constituted a workable system for generating data insured they could reach timely decisions on design and construction of a beamline. They did not need a policy-making committee because they designed their instrumentation so that each team could use it independently. Routine daily affairs were handled by a participating scientist at or near the accelerator laboratory, and a collaboration spokesperson dealt with the accelerator laboratory's administration.
Determining (and in some cases redetermining) an internal structure and allocating resources across the divisions it created were the central collective tasks for the non-accelerator collaborations. The collaborations that proposed to overcome organizational obstacles treated their internal organization as malleable objects of experimentation, and the balance between the research areas and the participating institutions on the policy-making committee was critical to dealing with the potential conflicts of reorganization. Annual or semi-annual collaboration-wide workshops were the primary formal occasions for assessing the wisdom and efficacy of the collaboration's organization in light of the latest results. Meetings of external advisory committees also stimulated ferment.
The device-oriented collaborations did not pursue internal reorganizations. They divided labor along institutional lines to make it easy for their corporate scientists to participate without releasing information their corporations wished to keep proprietary. Although funding-agency reviews did force participants to look critically at their arrangements, the difficulties in negotiating the initial intellectual property agreement inhibited any efforts to reform the collaboration's internal structure, even when participants were dissatisfied with the quality of intra-collaboration technical discussions. The central collaborative task of the accelerator-using collaborations was to design, build, and operate the beamline (including detectors) that would serve the needs of the members. When corporations were involved, the collaborations had to make arrangements for the corporations to perform proprietary research. They did so in varying ways, but in no case were the arrangements nearly so difficult to design and maintain as the intellectual property agreements that were negotiated in non-accelerator collaborations. One accelerator-using collaboration we studied pursued integrated studies of specified materials using extant beamlines and detection instrumentation. It needed no significant management. The number of participants was small enough, their roles so self-evidently clear, and the needed facilities so easily tapped that nobody even had to organize a meeting of all the participants.
"Teams" in the non-accelerator materials science collaborations referred to a multi-institutional group of researchers concentrating on a substantive problem. All data in these collaborations were taken as part of team activities with corporations sharing the data they took under the requirements and protections provided by the collaboration's intellectual property agreement. None of these collaborations collectively built instrumentation with which to take data streams for the use of everyone in the collaboration. None of the teams and individual scientists in these collaborations have had to build up their instrumentation from scratch, because their researches within the collaboration have involved using the techniques they employed in their pre-collaboration researches. Though development of instrumentation was not a principal activity of the non-accelerator collaborations, they did support the acquisition of new instrumentation by member institutions. The new instrumentation was invariably purchased--sometimes by contract in which the purchasing institution specified novel features for the maker to incorporate. The teams operated at diverse levels of autonomy in these collaborations. The device-oriented tended towards the extremes; the teams operated either with high autonomy to make protection of proprietary information easy or with high coordination to make integration of novel materials into a new device easy. The teams in collaborations oriented towards overcoming organizational obstacles operated with more intermediate autonomy. The participants were all building on their individual prior research, which as a rule they had pursued autonomously, but the participants also knew that collaboration administrators and funding-agency officials would judge the collaboration on whether participants together performed research that would not have been done had the collaboration not existed.
The meaning of "teams" in accelerator-using collaborations was idiosyncratic for almost each collaboration. It could mean: a sub-set of institutional members responsible for a particular set of instrumentation; the member institutions and how each determined how it used its time to operate the collective instrumentation; the member institutions and how they used the expertise in which each was strongest; groups of similarly specialized scientists from all member institutions interested in pursuing a particular form of experimentation; or nothing at all. This condition probably reflects the novelty of this form of research for materials scientists; norms and precedents are sufficiently few that each collaboration invents its sub-structure from scratch.
None of the non-accelerator materials science collaborations have needed collaboration-wide policies on the acquisition and processing of raw data, because none of them took data on a collaboration-wide basis. Data sharing was often encouraged, though the interdependent researchers were left to their own devices for making arrangements to share data and to analyze them jointly. Data sharing was only banned when a corporation in an accelerator-using collaboration took proprietary data. In device-oriented collaborations, the intellectual property agreements regulated data sharing. In general, these agreements obliged participants to share data about the characteristics of the materials and the performance of the components they were experimenting with, but not to share data about the processes by which they were making the materials and components. Data archiving and long-term ownership of data are subjects with which materials scientists are just beginning to grapple.
Almost none of these collaborations maintained ongoing collaboration-wide policies for determining the substance and quality of research results. Teams or individual data-takers largely decided when and where to disseminate what. In device-oriented collaborations, manuscripts were internally reviewed for compliance with the collaboration's intellectual property agreement; nobody ever reported experiencing a problem with the reviews. Accelerator-using collaborations rarely published scientific results with a collaboration-wide author list (except for papers reporting on beamline design and performance).
The non-accelerator materials science collaborations we studied have been granted funding commitments for as short as two years and potentially as long as 11 (subject to renewals). The accelerator-using collaborations were open-ended in time; they existed (or will exist) for as long as their participants successfully pursued funding. Once formed, all materials-science collaborations have had stable institutional memberships with few institutions dropping out or joining. (The individual investigators in the collaboration have changed significantly over the life of the collaboration). However, the accelerator-using and non-accelerator collaborations have differed in their likelihood to reorganize. The former have been far more likely to stick with their original organizations than the latter. When the non-accelerator collaborations did not reorganize themselves, they were short; and when they reorganized, they were long.
Success for the accelerator-using collaborations meant creating conditions that enabled its members to take data and publish papers. Participants were presumed to know what their scientific interests were and to be capable of independently satisfying them once the collaboration provided the necessary instrumentation. The premise among the non-accelerator collaborations, by contrast, was that their participants could not independently satisfy their scientific interests and perhaps were constrained by their institutional arrangements from even realizing what their best interests were. Success for the device-oriented was development of a prototype device--or at least the knowledge needed for designing and building a prototype device; success has led to dissolution as members have preferred not to reinvent the collaboration and recast the intellectual property agreement to take into account the shifts in interests among their members. Success for the collaborations that proposed to eliminate organizational obstacles was the creation of an internal structure that its members would continue to want to work within; they endure in order to demonstrate that their findings and their organization together generate compelling lines of research.
As a group, these collaborations have been most significant as attempts to find satisfying working relationships among institutions from different sectors. The accelerator-using collaborations functioned smoothly because they left the participating institutions with the latitude to decide what to examine and who to involve in its examination. The non-accelerator collaborations attempted a more organic integration of university, industrial, and government science, and elicited more conflicts. No single publication from any of these collaborations was said to have significantly affected scientific or technological practice. Nevertheless, interviewees have all expressed satisfaction with the intellectual quality of their participation. All are credited with wisely bringing together experts with different perspectives but common interests in a class of materials.
D. Heavy-Ion and Nuclear Physics
We interviewed participants in two collaborations in heavy-ion and nuclear physics. We expected our earlier study of high-energy physics to be applicable to this field, and the Working Group confirmed our expectation. These collaborations fit readily into a pattern we found in our earlier in-depth study of high-energy physics. They were comprised predominately of American universities and national laboratories with foreign institutions being integral to meet the collaborations' needs for manpower and expertise in particular forms of instrumentation. They formed when the construction of a new accelerator inspired professional friends and their circles of colleagues to draft a proposal, and they considered themselves formal entities once the proposal was accepted. They focused a beam of particles on a metal target to generate interactions that were detected by elaborate combinations of instruments arrayed behind the target. And they had dedicated, centralized funds for instrumentation; the participating institutions covered personnel and travel costs. The organization of these collaborations also followed a high-energy physics model. They designated a "spokesperson," who was the scientific leader, to represent the collaboration to the accelerator laboratory and to lead intra-collaboration discussions. They divided labor for building detector components along institutional lines and designated an individual to track the development of the components and deal with systems engineering problems. They held collaboration-wide meetings three to four times a year to discuss research strategy and to review results. They assumed that discussions would result in consensus with the spokesperson making decisions only as a last resort.
"Team" in these collaborations usually referred to the institutional member(s) responsible for a particular component. Each team had to design its component to fit the geometry of the overall detector and to perform at a level that fit the capabilities of the other components. Most components were built in the laboratories and shops of the participating institutions. Each team had to provide the software for reading out and processing the raw data from its component, and the software had to conform to collaboration-wide standards so that data analyses could be readily performed on multiple data streams. All data streams from every detector component were deemed the collaboration's collective property. In various ways, they all struggled with the trade-off between maximizing individuals' freedom in methods of analysis and topics of analysis while insuring that results could be compared and that graduate students had well differentiated dissertation topics.
Publication was a collective effort. Drafts of papers were circulated among all participating scientists, and manuscripts were not sent to a journal for consideration until all had indicated their approval.
Like high-energy physics collaborations performing "strings" of experiments, these collaborations had a central core of personnel and institutions plus groups that joined and left depending on the collaborations' needs and the other groups' interests. Success for these collaborations meant producing scientific publications. Nobel Prize caliber discoveries were hoped for, but participants were satisfied with numerous publications and good opportunities for graduate students.
E. Medical Physics
We interviewed participants in three medical physics collaborations. These collaborations formed either to develop new medical procedures or to test state-of-the-art procedures. Radiologists at medical schools and their affiliated hospitals participated in all the collaborations we studied. Collaborations developing procedures included physical scientists, who often worked for university science departments, corporate laboratories, or national laboratories. Collaborations that tested procedures included statisticians from public health departments and medical professional societies. Internationalism was rare, probably because of the importance of national standards for medical practice.
The instigation of these collaborations varied widely. At one extreme, when physicists and physicians hatched ideas for using physics instrumentation for diagnostic purposes, word-of-mouth and geographic proximity were essential for collaborators to find each other and build the intellectual intimacy needed for the various specialists to understand each other. At the other extreme, when policy-makers wanted a collaboration to form that would address the paucity of information for assessing diagnostic modalities, the funding agency selected the participants on the basis of their individual proposals in the hopes that the collaboration would produce an impersonal consensus on the effectiveness of various diagnostic tools. The collaborations at the extremes and in intermediate positions had difficulties formalizing their arrangements, indicating that the institutions that support medical physics do not form multi-institutional collaborations often enough to have smooth procedures for managing collaborations when they do form.
The organization of these three collaborations ran the full range from rigidly organized in response to external pressures, to self-organized in response to perceived needs, to barely organized in response to perceived lack of need. Procedure-testing collaborations were most rigidly organized to insure the collection of comparable data streams for statistical analysis. The procedure-developing collaborations required less organizational formality because the obvious differences in the expertise of their participants led to a well-understood division of labor and responsibilities. When a procedure-developing collaboration did not need to coordinate data acquisition around a central facility, its organizational structure consisted of nothing more than a "project director" who did little beyond organizing collaboration-wide meetings.
The very name "medical physics" evokes cross-disciplinary exchanges and the corresponding possibility of conflict based on different scientific orientations as well as different financial expectations, cultural expectation, and institutional affiliations. However, the only collaboration-threatening conflict among any of the cases we studied involved confusion over which members were entitled to apply for funding from which funding program, given that there was no single program for the collaboration to apply to as a whole.
Procedure-developing and procedure-testing collaborations defined "teams" in different ways. In the former, teams were comprised of functionally differentiated groups that each covered one of the range of skills and specialties needed to create an effective diagnostic system. Each team was responsible for the research and development of the instrumentation, algorithms, or clinical evaluations required for its task. In the latter, teams were comprised of functionally equivalent groups in order to standardize diagnostic procedures over a statistically significant portion of the population. Each collected data by performing the specified diagnostic procedures to the collaboration's standards; tinkering with instrumentation or procedures was expressly forbidden. Medical physics collaborations, unlike all others AIP has studied, had human subjects and were obliged to archive data on patients and to keep those data confidential. In procedure-testing collaborations, information was carefully compartmentalized and teams kept ignorant of each other's findings in the interest of eliminating the possibility of bias in ongoing data acquisition. Data-sharing was desirable or necessary in procedure-developing collaborations, which also faced computational challenges in processing data because of the technical novelties of their instrumentation.
Publication of results from procedure-testing collaborations had to await the completion of data acquisition and statistical analysis; manuscripts were then subject to a collaboration-wide review with all participants listed as authors. Dissemination policies varied across procedure-developing collaborations, depending on the extent to which the teams were independently able to produce publishable results.
The costs of medical physics collaborations are difficult to assess because of their occasionally disjointed funding, cost-sharing with corporate participants, and the participation of physicists without dedicated funding. However, the costs of performing clinical trials to assess the efficacy of diagnostic systems dwarfed the cost of developing the systems.
In the collaborations we studied, those developing diagnostic systems have been more stable than the collaboration assessing the efficacy of diagnostic techniques. As the latter has switched its focus from organ to organ, it has made close to wholesale changes in institutional participants. Internally, however, all of these collaborations have been organizationally stable.
The small number of cases examined plus AIP's lack of familiarity with the field make generalization difficult. Also, new policies at NIH to attract experts in computation to bio-medical research have the potential to alter the environment for multi-institutional collaborations in medical physics. However, one clear theme was that all of these collaborations struggled with the trade-off between standardizing practices in the interest of testing practices clinically and tinkering with the components of diagnostic systems in the interest of discovering obviously superior practices. Conflicts were most significant in the collaboration that explicitly and only tested practices. When the collaboration could not collect data and publish results before equipment manufacturers produced their next generation of systems for acquiring data, participants considered the results obsolete upon publication and questioned each other's practices.
F. Computer-Mediated Collaborations
We decided to assume the responsibility of investigating an area we named computer-mediated collaborations because collaborating around the new capabilities made possible by advances in computation seems likely to increase in the future. Most of the participants in the three collaborations we studied were university faculty, but non-university participants were essential to each of these collaborations. International participation was minimal. None of these collaborations was funded from within the traditional programs of the government funding agencies, and all included both computer scientists and physical scientists. All had difficulty forming because of their unconventional funding arrangements or composition.
The central managerial task of all these collaborations was coping with the diversity of interests they included. None entirely succeeded in finding a comfortable and appropriate organizational structure. One avoided the problem by increasing the autonomy of participants and decreasing the role of collaboration leader. Another eliminated the problem by reducing the number of participants and interests in the collaboration and increasing the number of collaboration managers to make sure that all the remaining interests were well represented in collaboration governance. The third sharply distinguished software writers from software users and accepted the toll on collaboration morale from misunderstandings and poor communication among participants who were not confident in each other. They all sometime precariously balanced how strongly the collaboration should integrate its members' ongoing research, how broadly the collaboration should reach across the possible topics it could investigate, and how centered the collaboration should be within one of its institutional members and participating disciplines. Their survival and success demonstrate the importance of computational sophistication for progress in physical science, and empirical challenges for progress in computer science.
The individual teams within these collaborations usually published their research autonomously. None set an explicit policy for whom to include in an author list. The utility of computation in so many sciences poses obvious organizational conundrums. Scientists making their careers as computer scientists tend to focus on topics of general significance to their disciplinary colleagues and to downplay the importance of topics of significance to other disciplines. Physicists who become sophisticated in computation tend to use their computational skills to advance their fields and to downplay the general significance of their computational work. These collaborations represent an attempt to have it both ways: to create intellectual intimacy between computer and natural scientists without losing the intellectual power that comes from specializing within a disciplinary tradition. The fact that these collaborations hung together despite the tensions they internalized suggests that "grass-roots" support for multi-disciplinary collaborations between computer scientists and physical scientists is developing.
III. ARCHIVAL ANALYSIS AND APPRAISAL GUIDELINES [Back to Top]
The historical analysis summarized above described the patterns of organization of multi-institutional collaborations and the activities they employed to carry out those functions. The following archival analysis is organized in the same way as the historical analysis, i.e., by the five disciplinary fields covered in our Phase III work followed by the category we named computer-mediated collaborations.
The archival analysis couples the organizational patterns of multi-institutional collaborations to the patterns of records creation, retention and destruction, and likely locations of records. In Report No. 2, we offer appraisal guidelines to assist archivists and others with responsibilities for selecting records for long-term preservation. In particular, we identify a small set of "core" records that should be permanently preserved for all collaborations in a given disciplinary field, as well as the more extensive array of documentation that scientist-administrators, historians, and others will need to understand collaborations of outstanding significance. For multi-institutional collaborations in all scientific disciplines, additional documentation should be provided by saving professional papers of distinguished practitioners of these disciplines. We also developed a set of recommendations to promote preservation of valuable documentation (see Project Recommendations, Section Two of this report).
Our report on archival analysis and appraisal guidelines is based on a number of sources: (1) the archival assessment of 78 interviews conducted on the 23 selected case studies; (2) the patterns uncovered through the historical and, in part, the sociological analysis of these interviews; (3) numerous site visits to Federal science agencies and to the National Archives and Records Administration; (4) site visits to archival repositories, especially during previous phases of the AIP Study; and (5) the AIP Center's general knowledge of archival institutions in various settings.
Throughout the AIP's long-term study we have found that--in all multi-institutional collaborations--some types of records are created by necessity: proposals, designs of instruments, purchase requisitions, logbooks of data acquisition, data analysis records, and progress and final narrative and financial reports. In addition to these operational records, collaborations usually create minutes and reports of committees and sub-committees. Our interviews with individual scientists show that decisions to create (as well as to retain) these records to a large extent reflect the style and personal inclinations of individuals. This is especially the case for their own correspondence, notebooks, and other files. Certain circumstances affect the creation or retention of valuable documentation. These include the degree of centralization of the collaboration, the role of engineers, and the increasing impact of fax, electronic mail, and the World Wide Web.
We have accumulated evidence that a major obstacle in documenting multi-institutional collaborations is the lack of archival programs at some critical institutions. Even where archival programs exist, administrators at most universities, at most FFRDCs (Federally Funded Research and Development Centers), and those at virtually all research institutes, corporate or government laboratories, fail to consider as their responsibility the documentation of collaborations, no matter how significant.
A. Ground-Based Astronomy: Observatory Builders
The difficulties of documenting the work of telescope-building collaborations are distinctive among the disciplines covered by the long-term AIP Study. The unique difficulties arise from the circumstances of their financial support. Academic observatories are built with funds primarily from a variety of non-Federal sources and, when these are private foundations or universities, records requirements are less stringent. In the case of national observatories, the National Science Foundation is virtually the sole supporter. These observatories are operated in a fashion similar to the Department of Energy's national laboratories, but--unlike the DOE--the NSF contract laboratories and observatories do not create Federal records. Accordingly, these national observatories are not required by law to maintain records management programs or secure records of archival value. To address these obstacles, our recommendations urge universities to take responsibility for documenting their academic observatories and, for national observatories, we ask that the NSF take positive steps to secure relevant records at its headquarters and to support archival programs at the national observatories.
Despite these important differences, we have found that the patterns of organization and management of all telescope-building collaborations are quite similar. All four collaborations included in our case studies vested authority in a Board of Directors, and made one individual most responsible for the physical construction, usually with the title of project manager. In most cases they organized Science Advisory/Science Steering Committees of scientists from the member institutions to cope with the trade-offs between scientific capabilities, and engineering and financial burdens. Individuals in these offices create substantial documentation in carrying out their responsibilities. The creation of records does not, of course, equate with saving those records. It is fortunate that virtually all of the individuals holding these positions in observatory-building collaborations are on university faculties where archival repositories are well-established.
Because of the expense and uniqueness of observatories and because few are built in any one decade, the AIP Study has taken the position that each telescope-building collaboration should be categorized as significant with substantial documentation permanently preserved for future use by scientist/administrators and historians and other scholars. In addition to NSF grant award jackets and NSF cooperative agreement jackets for research facilities, the following records should be saved for all telescope-building collaborations: documents of incorporation (sometimes called MOUs), Board of Directors' minutes of meetings, records of project managers, records of Science Advisory/Science Steering Committees, records of Design Review Panels, records of Science Project Teams, contracts and associated records, and technical reports (sometimes called memoranda series or technical memoranda).
B. Ground-Based Astronomy: Users of Observatories
If it is difficult to document the building of observatories, it seems virtually impossible to document collaborations of observatory users--at least radio telescope users. The reason is straightforward. They leave a scanty paper trail (except for observational data)--because they require little or no dedicated funding and only minimal organizational structures--and they neither design nor build the instrumentation they use.
The best core record of a given collaboration is the lead scientist's proposal for use of a participating observatory's telescope and his/her collaboration-wide correspondence. To secure that documentation, we need radio observatories to have policies to preserve their proposal and evaluation records. For a richer record, we are dependent upon lead scientists to save their papers and their employing institutions to accession them for their archives (once again, we are fortunate that lead scientists are on academic faculties or staffs). For collaborations of high distinction, the records of observatory consortium secretaries should also be preserved.
C. Materials Science
Our historical analysis of collaborations in materials science makes distinctions between those that make use of accelerator and reactor facilities at DOE National Laboratories and those that do not. Our archival analysis is strikingly different for these two categories.
Collaborations that do not use national laboratory facilities present documentation challenges whether managed by universities or corporations. NSF centers (the Science and Technology Centers and the Materials Research Science and Engineering Centers) have emerged in recent decades on university campuses; most, if not all, of the centers are responsible for judging proposals from researchers employed by member institutions. This practice diminishes the detail of documentation at NSF Headquarters. All three of our case studies of university-managed collaborations lacked a physical location beyond their offices at the fiscally accountable university. In a field with strong participation of corporate organizations, it is not surprising that our case studies included an instance in which the collaboration was managed out of a corporate institutional member which no longer exists because it was merged into another corporation; such mergers endanger records.
The characteristics of those collaborations that did make use of accelerators or reactors at DOE National Laboratories (half of our case studies) are quite different. For one thing, they had some characteristics similar to those we were familiar with from other studies involving DOE National Laboratories: they were all required to submit both technical and managerial plans to the Facility Advisory Committees (our generic term for a variety of titles) of the laboratory facility, and they all had a liaison with the DOE Laboratory facility (whether called spokesperson, staff director, or an untitled member who played the role). These characteristics assure preservation on the part of the DOE National Laboratories of some core records and help us locate documentation for significant collaborations.
The core records for materials science collaborations are proposals to Federal funding agencies and/or to corporate management and/or to Executive (Program) Committees of NSF centers-- and--for those using synchrotron radiation facilities--records of Facility Advisory Committees at DOE National Laboratories. Additional records to be preserved for collaborations of high significance are: records of the Executive Board (or Governing Board, Program Committee, or Technical Representatives Committee), records of external advisory committees, records of annual meetings of the collaboration, records of spokespersons/staff directors, newsletters and sector descriptions, and collaboration records on compact disk.
D. Heavy-Ion and Nuclear Physics
We are confident of our findings in this category even though we have only two case studies. Our earlier in-depth study of high-energy physics experiments at national accelerator laboratories provided substantial understanding of experiments in the related fields of nuclear and heavy-ion physics at these laboratories. We find the familiar roles of laboratory PACs (Physics Advisory Committees) governing access to particle accelerator beamtime, the MOUs (Memoranda of Understanding) detailing institutional commitments, spokespersons providing liaison, and collaboration-wide meetings. We also find management structures more familiar to us from collaborations in other disciplines--structures that may indicate emerging complexities in the various areas of particle physics collaborations that archivists should be on the lookout for. Finally, we are in the world of Web sites with areas accessible only to those with proper passwords and no guarantee of permanence.
In addition to proposals to Federal funding agencies, the core records for heavy-ion and nuclear physics collaborations are the records of Physics Advisory Committees at DOE National Laboratories and other accelerator laboratories. Additional records to be preserved for the most significant collaborations are records of spokespersons, collaboration group leaders, project managers, and project engineers, in addition to Intra-Collaboration Technical Committee records and collaboration Web site records.
E. Medical Physics
For several reasons, it is virtually impossible for us to assess with any certainty the archival situation in the area of medical physics. For one thing, the AIP Center has had little experience in documenting the research activities of medical schools or other medical research centers, in saving papers of individual practitioners, or in dealing with the key funding agency--the National Institutes of Health (or its constituent parts such as the National Cancer Institute). In addition, we had difficulties in persuading individuals in the discipline to participate in our AIP Study. Consequently, our appraisal guidelines (see Report No. 2) and our Project Recommendations to funding agencies and research institutions in the field are--for the most part--merely suggestive.
We classify the following as core records for collaborations in the area of medical physics: proposal jackets to private foundations and Federal funding agencies (including referee and panel reports and annual and final progress reports) and--for those using DOE synchrotron radiation facilities--records of DOE Facility Advisory Committees. Additional records to be preserved for highly significant collaborations are minutes of collaboration meetings, records of group leaders for statistical analysis, and protocols and samples of data collection forms.
F. Computer-Mediated Collaborations
In this third and last phase of the long-term study, the AIP determined that it should deliberately examine a new category of collaborations that might well become more dominant in future collaborative research. The principal characteristic our three case studies in this category have in common is the central role of computer science and technology--hence the name for this group, Computer-Mediated Collaborations. Our sample of NSF-funded collaborations included a Science and Technology Center (STC), a collaboration in a program devoted to using computation for theoretical problems, and another collaboration in a category referred to as testbeds for a National Collaboratory which focuses on access to remote instrumentation and improved communications of researchers. We sought to learn of the relative health of these new kinds of projects. From our site visits to NSF and DOE and the meeting of our Working Group, we were convinced that they would continue and thrive over the near future.
Another purpose of studying these collaborations was to obtain a clearer picture of the ways, if any, the focus on computer science and computer techniques would affect a collaboration's organizational structure and the records the collaboration generated, as well as which records should be preserved. We found that the impact on organization structure and on records creation is not apparent in the case of the NSF STC and the collaboration using computation for theoretical problems in our sample. But the impact on our testbed for a national collaboratory project was a different matter. There are typically two purposes for collaboratories: to operate scientific instruments by remote control and to provide researchers a venue for discussion and debate. We could not see in our case study that the introduction of remote control of instruments had a distinctive impact on organizational structure and related records creation. But the electronic venues for discussion and debate generated a plethora of records--far more than can be saved, even for significant projects. At least until the design of these discussion chatrooms is better understood, the records generated also require analysis by social scientists; this in itself has an impact on the collaboration's organizational structure and management as well as the records created.
It seems inappropriate to specify records to document this category of our AIP Study. We can generalize that funding agencies should preserve grant and cooperative agreement award files as core records. We can also recommend that data generated by chatrooms should only be saved for significant collaborations and that, even in these cases, a selection of the data be made based on a key aspect of the research program.
IV. SOCIOLOGICAL ANALYSIS [Back to Top]
The sociological study of Phase III projects has focused on two questions that confront scholars of multi-institutional collaborations:
(1) What types of collaborations are there?
(2) How, if at all, are these types related to important outcomes?
Significant variation among collaborations is recognized. Its mere demonstration is no longer as important as the question of whether there are identifiable patterns of social organization. Our goal is to characterize multiple types in a systematic fashion--that is, can a robust classification scheme be developed? Accordingly, the first problem is to determine the general dimensions that characterize multi-institutional collaborations in science, to operationalize these dimensions and examine the extent to which they allow us to distinguish empirical clusters to form a classification. But such a classification scheme is of limited value in the abstract. It becomes significant insofar as the types defined are related to important sociological outcomes. The second problem is to identify and develop indicators of these outcomes and to determine whether they bear any relationship to the types identified in the classification.
A. Outcomes of Multi-Institutional Collaborations
Contemporary scholars have distinguished trust, stress, documentary practice, conflict, and perceived success as critical dimensions in knowledge production.
The role of trust in interorganizational relations has been well documented. It is not an exaggeration to claim that trust is required for all systems of knowledge production and especially when scientific institutions and individual researchers have to coordinate their efforts toward a common goal, as is typically the case for multi-institutional collaborations.
Scientists engaged in multi-institutional collaborations are often exposed to high levels of stress for a variety of reasons: complex technological demands, unclear or changing social arrangements, the need to coordinate geographically dispersed groups, the clash of interests, ambiguity in the distribution of authority, and the pressure to perform according to time constraints and the expectations of funding agencies. The factor of time constraints is especially important, since time is a critical resource in working together. In many cases the degree of stress induced by schedules and deadlines is higher than in typical academic settings. This is mainly due to pressure from funding agencies and participating institutions to perform within tight budgets as well as under time constraints.
Documentary practice--the generation and use of records--is essential for the work organization of multi-institutional collaborations. Beyond that, the role of historical accounts has long been recognized--as has the fact that the reconstruction of such accounts depends heavily on the preservation of written documents. The ultimate intent of the AIP Study is to assist archivists and others in identifying and locating the kinds of records most valuable in documenting the organizational structures and functions of multi-institutional collaborations. Data on two variables that measure documentary practices of collaborations--dispersion of core records and quality of records--were analyzed to help meet this archival goal. Project historians and archivists identified the core records (i.e., the small set of records that should be saved for all collaborations of a given scientific discipline); they also identified the likely locations of the core records for each of the collaborations selected for study. Project sociologists included these data in their database. We posit that the extent to which core records are dispersed is an indicator of project decentralization as well as of the degree of difficulty involved in reconstructing accounts of the collaboration (Warnow-Blewett 1997).
All social formations that involve ongoing use of resources, even those that involve only prestige, have the potential for conflict. Multi-institutional collaborations are not devoid of disagreements, contentions, and conflict. From a sociological point of view, conflicts are especially interesting because they provide insight into the dynamics of social cohesion in the collaboration, as well as what this might be due to.
Of course, "performance" is the most valued outcome of science, the criterion by which projects are evaluated. Multi-institutional collaborations may be defined as successful or unsuccessful in terms of many dimensions: the extent to which they accomplish objectives, are completed on time or within budget, produce results that are used by others within and outside the field, and so forth. Yet there is often a general sense in which projects (especially those that require substantial commitments of resources and personnel) are evaluated positively or negatively by the scientists who work on the collaborations. It is in this sense that we speak of the "success" of a collaboration.
B. Structural Dimensions of Multi-Institutional Collaborations
The AIP Study of high-energy physics (Phase I) and space science and geophysics (Phase II) enabled us to identify primary dimensions that were important in multi-institutional collaborations. All of the interviews from the first phases had been previously assessed and categorized in terms of major topics or themes. These major themes, along with other factors identified in the historical analysis, led to the recognition of general and specific properties of collaborations. We identified seven major structural dimensions of multi-institutional collaborations: project formation, magnitude, interdependence, communication, bureaucracy, participation, and technological practice. These dimensions and some of their constituents may be summarized briefly:
(1) Project Formation and Composition. Collaborations have a variety of origins. Some encompass academic, governmental, and private sectors. In others one sector is dominant, both in origin and constitution. The role of pre-existing relationships among researchers varies, as well as supervision and funding agency involvement.
(2) Magnitude. Some collaborations are larger than others, in terms of the number of organizations, teams, individual participants, graduate students, and subcontracts. Costs for personnel and instruments differ a great deal, as does the length of the project.
(3) Interdependence. Data-sharing, the analysis of joint data, and the autonomy of organizational teams with respect to instrumentation distinguish collaborations in terms of the interdependence of their constituent social formations.
(4) Communication. Relations with the public are sometimes managed by a designated public relations officer. Results may or may not be popularized and restrictions may be placed on publications. Internal to the project itself, a communications center is sometimes utilized, and projects may depend more or less on formal communication modes.
(5) Bureaucracy. The degree of bureaucracy is a fundamental aspect of organizational structure and has been conceptualized in a wide variety of ways. Phases I and II showed that collaborations could be distinguished according to the presence of a lead center, designated scientific and administrative leaders, and the division of authority between them. In the current Phase III, our examination also included the presence of written contracts and coordination of schedules as well as the presence of outside formal evaluation in assessing the degree of formalization. We also found that projects vary in terms of levels of authority, style of decision-making, and presence of external advisory committees.
(6) Participation. Graduate students are involved more in some collaborations than others. Principal scientists may be more or less interested in and devoted to a project. International involvement is sometimes crucial for a project, but in others it is not present at all.
(7) Technological practice. Multi-institutional collaborations vary in the ways they acquire and use instrumentation. Characteristics of acquisition and use allow us to distinguish a broad array of factors that may be designated the "technological practice" of the collaboration. Some collaborations design and build their own equipment, some advance the state-of-the-art in instrumentation, and some modify their instruments during the course of the project. Technological practice is not merely instrumentation but includes the management of research topics and the checking of results.
C. Results
1. Findings from Bivariate Analysis The most important results from cross tabular analysis may be summarized in three points:
First, surprisingly, in the Phase III sample, field of research was not related to many variables at the bivariate level, and--more importantly--it was not significantly associated with the organizational indicators. Therefore, certain organizational features of collaborations persist regardless of the broad specialty area. Nevertheless, there were several relationships where field mattered. For example, in our sample of collaborations (all primarily American), field affects instigating sector. Ground-based astronomy and medical physics are more likely than heavy-ion physics and especially materials science to have been instigated only by the university sector. Sectoral composition of institutional members reflects the same pattern. Field of research is also significantly related to scrutiny from outside authorities. Medical physics was the only field in which most collaborations received scrutiny from Congressional committees or White House offices--two thirds of these projects were scrutinized. This seems natural, since there is a great social and political interest in research on medical diagnostics and treatment. The only other field to receive some attention from the same authorities was ground-based astronomy.
Second, the magnitude of multi-institutional collaborations was, as anticipated, positively related at the bivariate level to their formal organization and management. Thus, size (number of participants) and the existence of external advisory committees are positively associated. Large (83.3%) and medium (75%) collaborations are more likely to have such a committee than small collaborations (11.1%). This finding is within reasonable expectations, since normally we would expect greater oversight for bigger projects, where more people are involved. Size of the project is also significantly related to the presence of an administrative leader. The direction of the association is in accordance with previous findings in the organizational literature--that larger organizations tend to be more centralized and formalized. For our sample, large (100%) and medium (87.5%) collaborations are more likely to have a designated administrative leader than small collaborations (33.3%). Furthermore, larger collaborations are more prone to have a division between intellectual and administrative authority. Collaborations with a large number of participants have division of authority in 83.3% of the cases, those with a medium number of participants in 75%, and those with a small number of participants in 22.2%. Finally, duration is usually related to greater formalization. For example, we found a significant covariation between levels of authority and length (from the formulation of the original idea to funding). Collaborative projects with a shorter period prior to funding are likely to have fewer levels of authority than longer projects.
Third, a number of structural characteristics of multi-institutional scientific collaborations were related to two important outcomes of these collaborations--conflict (disagreements) and trust. The general conclusion is that greater magnitude and greater formalization lead to more problems and less trust. For example, style of decision-making is significantly related to problematic results due to time pressure (although, strictly speaking, this is not necessarily a causal relationship). This relationship is in the predicted direction--that more "hierarchical" collaborations would tend more often to have problematic results caused by time pressure. Next, length from funding to first publication of results from the collaboration is significantly related to conflict between teams. Longer collaborations (up to first publication) tend to have more disagreements between teams. Division of authority significantly contributed to conflict between researchers and the project management. Multi-institutional collaborations in which there was a division between intellectual and administrative authority had conflict between scientists and the project management in 38.5% of the cases vs. none of the projects where there was no such division.
Overall, the degree of trust was fairly high. However, there was some covariation with the organization and magnitude variables. Style of decision-making is associated with trust. In all collaborative projects with a consensual style of decision-making, the degree of trust towards other researchers was high; by contrast, trust was high in only one-third of collaborations with a hierarchical manner of decision-making. Size and trust towards the project management are negatively related. In over three-quarters of the small collaborations, and all medium projects, the degree of trust towards the management was high, as compared to only one-third of the large collaborations. Finally, hierarchy (levels of authority) is also negatively associated with trust.
Collaborations with fewer levels of authority than a typical academic science department are characterized by high degree of trust towards project management in 93% of the cases, in contrast to only 44% of multi-institutional collaborations with the same hierarchical complexity as a university department.
2. Technological Practice as a Basis for Classifying Collaborations Our principal analytical question is whether collaborations may be classified into types based on structural dimensions that are related to important outcome variables. Cluster analysis provides a useful tool for categorization, while analysis of variance is appropriate for determining the relationship between types of collaboration and outcome dimensions. Cluster analysis was performed for each of the seven major structural dimensions described above. Each analysis produced groups of collaborations based on different distinguishing criteria. The solutions utilized ranged from two factors for interdependence to five factors for magnitude and participation.
Which clustering solution is best? Clearly, different solutions may be preferred for different purposes. In light of the fact that the present state of our knowledge of collaborations in science does not allow us to make a decision based on prior research or on theoretical grounds, we concluded that this issue should be resolved empirically.
The results showed that the clustering based on technological practice is superior to other structural dimensions in providing a classification that relates to outcomes. Our five outcome indicators are success, trust, conflict, dispersion of core records, and stress. Only the clustering based on technological practice is related to all of these outcomes. Clusterings by magnitude and bureaucratic organization are related to reported conflict (as the bivariate analysis already suggested) and also to documentation, while clusterings by project formation, participation, interdependence, and communication are unrelated to any outcome. We conclude that technological practice is the most promising dimension for framing a classification of multi-institutional collaborations.
D. Discussion
Cluster analysis of technological practice identified four types of projects: managerial, decentralized, noninstrumental, and routine:
(1) Managerial Type. The most distinctive feature of the seven multi-institutional collaborations that constitute the first type was the combination of management of data analysis and planned development of instrumentation. We propose to designate these collaborations as managerial--not to imply high levels of bureaucracy, but because there are relatively high levels of control over instrumentation and data analysis.
The managerial group is the only type in which most of the collaborations actively managed the topics to be analyzed by individual members. Topical management does not imply imposition of research themes on the participants, but rather the coordination of data analysis by the collaboration team. For example, the observation of the Galactic Center Sagittarius A at 3 mm frequencies had to be done at four observatories according to a maser time standard.
The technological configuration associated with type one, centered on management of data analysis, is associated with lower levels of trust between project teams and relatively high levels of stress and disagreements. These collaborations are also perceived as less successful than all but the routine type (type four). Apparently, the relatively standardized, planned development of instruments is not associated with lack of conflict. Rather, attempts to maintain high levels of control may themselves generate difficulties. Managerial multi-institutional collaborations are in this respect the opposite of those in type two (decentralized), which have the lowest levels of management of data analysis and the highest perceived success.
(2) Decentralized Type. The most significant difference in our cluster analysis is the one that sets type two apart from the other types. In none of these projects was there central management of data analysis. Topics for analysis were controlled by independent teams. For this reason we term type two decentralized. The characteristics of decentralized collaborations are otherwise in some respects quite similar to those of the managerial type in terms of a focus on technological instrumentation and cross-checking of results among teams.
One difference between managerial and decentralized collaborations is reflected in their documentary practices. The dispersion of core records is much greater in the latter. This state of affairs is a result of their characteristic of team control. Decentralized collaborations tend not to exert control over data analysis, while managerial collaborations exercise a great deal. Management of data analysis and lack of changes in the instrumentation seems to have contributed to the greater centralization of records in the managerial type. It is worth emphasizing here that decentralized collaborations view themselves as extremely successful, perhaps because they are patterned on the traditional, academic organization of science.
(3) Noninstrumental Type. The third type can be designated as noninstrumental because its primary distinction is that these collaborations neither design nor build their own equipment, nor do they subcontract the construction of such equipment. All of them performed sophisticated experiments or observations by making use of already existing facilities. Thus, for example, the project on Crystal Structure brought together materials scientists, solid state chemists, and solid state physicists from Dupont, BNL, and SUNY-Stony Brook. These researchers sought to determine the structure of certain materials, using an already existing beamline at the National Synchrotron Light Source in Brookhaven.
(4) Routine Type. What distinguishes collaborations that belong to the last type is relatively low innovation and high coordination of results. Typically, high coordination of results is the product of the division of labor within a collaboration--separate research teams tackling specific topics that have to be integrated. Like the noninstrumental type, these projects had relatively large overlaps in the topics addressed. But unlike the noninstrumental type, teams in routine collaborations never checked the accuracy of each other's results. For example, there were three separate teams in the Advanced Light Source Beamline Collaboration, each responsible for an end station. Checking and coordination occurred within the individual research teams, but not between them. Another distinctive feature of these routine projects was that, while several designed and built instruments, they were less likely than other types to push the state-of-the-art in their respective scientific fields and there was not much time pressure.
The nature of the relationships between the technological practices of collaboration and the dependent measures is complex, and the emerging patterns are not always intuitive. Nevertheless, at least one fairly clear-cut contrast in terms of collaboration outcomes appears to be the division between managerial and routine projects. Where managerial and routine projects differ is in such interpersonal relations as trust, stress, and conflict. Managerial collaborations have a lower degree of trust toward other researchers, higher levels of reported stress, and more serious disagreements between teams, while informants from routine collaborations report higher trust toward their colleagues, lower degree of stress due to time pressure, and relatively few disagreements.
Neither of these types define themselves as particularly successful compared to the other types. The most successful projects belong to the decentralized type, which is also characterized by comparatively high degree of stress and between-team conflicts. Thus, it looks like success in multi-institutional collaborations comes at a price. Although management of data analysis is associated with higher conflict, our data do not allow us to determine whether these management practices generated the conflicts, or were implemented to reduce them. A closer examination shows that management of data analysis in and of itself may or may not be positively associated with conflict and stress within the collaboration. Thus, the collaborations that comprise the decentralized type are not highly managed, yet exhibit higher levels of stress and conflict than routine collaborative projects. It seems that this is due to the combination of lack of management of topics to be analyzed and frequent modification to the instrumentation. Managerial collaborations, which also experienced high degrees of conflict and stress induced by deadlines, did not have any changes in instrumentation, but like decentralized projects, engaged in results checking. Thus, regular checking of the accuracy of each other's results could be the common denominator of high levels of conflict and stress.
SECTION TWO: PROJECT RECOMMENDATIONS [Back to Top]
PROJECT RECOMMENDATIONS
The following recommendations pertain to actions needed to document collaborative research in physics and allied sciences, particularly in those fields studied by the AIP Study of Multi-Institutional Collaborations during its third and final phase, namely: ground-based astronomy (divided into observatory builders and observatory users), materials science, heavy-ion and nuclear physics, and medical physics and an area we named computer-mediated collaborations.(1) The recommendations are justified in more detail in the second volume of this report. Many, if not most, of the documents referred to are currently on paper, but our recommendations also apply to information in electronic format.
The AIP Center has encountered a wide range of complexities facing the documentation of experiments in modern physics and allied fields. On the most basic level, good records-keeping may be acknowledged by all as necessary while the experimental process is alive, but when the project is over, records can easily be neglected, forgotten, or destroyed. As a result, the most important recommendation (Recommendation #10.c.) urges a new approach to securing the documentation for future collaboration projects. We suggest that, once a project has been approved by a research laboratory (observatory, NSF center, etc.), the collaboration be required to designate a member to be responsible for its collaboration-wide records. In addition--where historical significance warrants--individuals should be named to be responsible for group- (or team-) level documentation of innovative components or techniques. This information should be incorporated into any contractual agreement with the collaboration. Use of this simple mechanism would assist archivists by assuring that records will be available for appraisal and by providing information on their location.(2)
Multi-institutional collaborations are virtually all funded by Federal science agencies and much of the research and development is carried out at agency facilities. Most of our recommendations are addressed to these agencies, as well as the National Archives and Records Administration (NARA), because successful documentation relies heavily on the effectiveness of their records management programs.
The recommendations are grouped in the following order:
Recommendations--Policies and Procedures
1. General
2. National Archives
3. Federal Science Agencies
4. Specific Federal Agencies
5. Other Institutional Settings
Recommendations--What and How to Save:
1. Core Records by Scientific Discipline
2. Significant Collaborations
RECOMMENDATIONS--POLICY AND PROCEDURES
CATEGORY ONE--GENERAL
General Recommendation
Recommendation #1: Professional files of key scientific faculty/staff members should be permanently preserved by their institutional archives.
Explanation:
Virtually all of our recommendations are focused on securing records of collaborations; accordingly, we must make clear at the outset the importance of preserving papers of individual scientists.
For some decades now, it has been traditional--especially in English-speaking countries--for professional files of academic scientists to be permanently preserved in their institutional archives. Those papers most frequently sought are of individuals who have made major contributions to science or science policy on a national or international level or to their university.
There are two principal targets for this recommendation. First, university archives in all countries should have policies to permanently secure files documenting the professional careers of their distinguished scientists. Second, similar policies are sorely lacking at virtually all research laboratories and other nonacademic institutions; they should be initiated and supported by directors of laboratories, whether they be in the corporate or government sector.
CATEGORY TWO--NATIONAL ARCHIVES
National Archives and Records Administration (NARA)
Recommendation #2: NARA should receive increased input from subject matter experts so that it can make more informed decisions on records appraisal; NARA should work with agencies to monitor and promote agency records management practices to insure that legal regulatory responsibilities are met, including the identification and maintenance of records of permanent value; NARA should identify and promote best practices for records management programs that agencies should utilize, including the development of R&D records criteria. The R&D records schedule of the Department of Energy (DOE) could serve as a model for other scientific agencies; and, NARA should consider, on a case by case basis, accessioning non-Federal records essential to documenting Federal support of science.
Explanation:
2.a. NARA should receive increased input from subject matter experts so that it can make more informed decisions on records appraisal.
Although the National Archives has responsibility for the final appraisal of Federal records, we are heartened that it has become increasingly aware of the importance of obtaining input from subject matter experts when appraising records of science and technology. Our particular concern is for the policy and planning records as well as the R&D records themselves. In these cases, it is urgent that the appraisal process be initiated with those who best understand the value of the documentation--the onsite records-creating scientists. Specifically, NARA should seek out subject matter specialists for the review of R&D records schedules of scientific agencies; it should also encourage records officers at science agencies to include subject matter specialists in the assessment of the importance of particular research projects; other opportunities for including subject matter specialists should be pursued.
2.b. NARA should work with agencies to monitor and promote agency records management practices to insure that legal regulatory responsibilities are met, including the identification and maintenance of records of permanent value.
NARA holds to its traditional position of discouraging the placement of professional archivists at external agencies. In its experience, the placement of an agency archivist equates directly to the assembly of an institutional archives rather than conformance to the legal requirement to transfer Federal records to the National Archives. For this reason, when these recommendations discuss Federal records we refer to "records advocates" (i.e., someone who can argue on behalf of the historical value of records) rather than "archivists."
Accountability should be the cornerstone of a records management program. While we propose some ways to improve existing agency records schedules (see, e.g., our Recommendation #2.c., below), the most serious problems we see are the failures to implement records programs by the agencies themselves. All too often, those responsible for records programs are ill-informed about their own institution and its science and technology, passive about gathering records, and passive about suggesting to NARA the additions or adjustments to their records schedules needed to protect valuable records series. Typically, scientists, administrators, and other staff at the agencies are uninformed about record-keeping programs.
Consequently, it is critical that NARA work with agencies to monitor and promote agency records management practices. They should see to it that the responsibility for records management has been clearly assigned and defined and that staff are appropriately trained and experienced.
Records officers must be grounded in records management principles and should be expected to serve as "records advocates." Competencies for records advocates would include skills in dealing with non-current records and archival, historical, or records management training and experience. The National Archives has seen that records advocates have been effective at such scientific settings as some of the accelerator laboratories of the Department of Energy; these have offered the National Archives a far better selection of records. The selection is better because a proactive program is in place to review records at the place where they are created--consulting those who created them--for the purpose of providing adequate documentation of the entire facility. The records advocates we have worked with most closely have been professional archivists, but trained historians or records managers skilled in dealing with noncurrent records could work equally well as part of a records management team. Records advocates should be expected to be knowledgeable about the scientific institution and the research programs it carries out. They should argue for the historical value of records in the context of agency records schedules and help NARA understand the unique records creation process at each of the science agencies. For all these reasons, we recommend that records advocates (e.g., trained archivists, historians, or records managers skilled in noncurrent records) should be made part of the records management programs--both at agency headquarters and at the key facilities and laboratories.
2.c. NARA should identify and promote best practices for records management programs that agencies should utilize, including the development of R&D records criteria. The R&D records schedule of the Department of Energy (DOE) could serve as a model for other scientific agencies.
As part of a program to monitor records management practices at Federal science agencies, NARA should consider conducting a survey of science agencies about their basic records management practices to determine the kinds of infrastructure now in place. This--along with the our suggestions for implementation and for training and use of "records advocates" in Recommendation #2.b., above--should help NARA identify Best Practices for agencies records management programs. A set of Best Practices is sorely needed and should be widely promulgated through the Wide World Web, other publication vehicles, and discussions at sessions of professional meetings of records managers.
For science agencies, it is critical that NARA develop Best Practices for developing criteria for the appraisal of R&D records, including procedures for ranking the importance of specific scientific research projects. Since NARA rescinded the part of its General Records Schedule covering research and development records, it became necessary for each science agency to schedule these records according to the unique practices of their individual agencies. A number of Federal science agencies have already done so. Among these, DOE, NASA, NIST, and NOAA have gone further to include sets of criteria that help their agencies identify significant R&D records. We believe all Federal science agencies should include such sets of criteria in their records schedules. The schedules of the DOE, NIST, and NOAA could serve as models.
The new DOE R&D Records Schedule, approved in August 1998 by NARA, is by far the best schedule we have studied. We are particularly impressed with its guidelines for procedures to rank scientific research projects as "significant," "important," and "other" and to involve the science records creators in this ranking. We also want to point out the importance placed on the proper evaluation of scientific policy and planning records in the DOE records schedules.
Our main purpose in this recommendation is to ask NARA to include the development of criteria for the appraisal of R&D records in its Best Practices. In addition, because National Archives appraisal archivists play a key role in developing agency records schedules, we ask NARA to urge them to encourage their assigned science agencies to have sets of criteria that provide effective procedures for identifying significant research and development records for permanent retention. This may require additional resources for the National Archives' Life Cycle Management Division.
2.d. NARA should consider, on a case by case basis, accessioning non-Federal records essential to documenting Federal support of science.
Many important Federally funded research organizations do not legally produce Federal records, yet some of the records they produce provide valuable evidence of the government's support of science. Accordingly, we ask NARA to consider--on a case by case basis--serving as a repository of last resort for selected records of organizations not formally affiliated with the Federal government that have no appropriate repository for their records. Prime examples are contractor institutions that oversee FFRDCs (Federally Funded Research and Development Centers) and free-standing research institutions.
See also Recommendation #6.b. to academic archives and #8 to NSF National Observatories.
CATEGORY THREE--FEDERAL SCIENCE AGENCIES
Federal Science Agencies
Recommendation #3: Federal science agencies should employ records advocates as part of their records management staff; Federal science agencies' records management programs should increase educational programs within the agency in order to stress the importance and benefits of records management and the criteria for saving scientific records; Federal funding agencies should save records documenting interagency funding of collaborative research projects; Federal agencies whose research centers/laboratories are operated under contract should permanently secure their headquarters' records relating to the contractor organizations; Federal agencies should permanently secure proposals and other documentation related to major research facilities at their centers/laboratories and other sites; and, Federal funding agencies should save controversial--albeit unsuccessful--collaborative research proposals in addition to successful ones.
Explanation:
3.a. Federal science agencies should employ records advocates as part of their records management staff.
Each science agency should examine the effectiveness of its existing records management program and seriously consider the benefits of adding records advocates--e.g., trained archivists, historians, or records managers skilled in noncurrent records--to its staff, both at headquarters and at major laboratories, flight centers, etc. that carry out national scientific programs. Such advocates should be expected to work proactively with scientists and administrators to become knowledgeable about their organization and the science and technology it is dedicated to.
See also Recommendation #2.b. for additional arguments.
3.b. Federal science agencies' records management programs should increase educational programs within the agency in order to stress the importance and benefits of records management and the criteria for saving scientific records.
During our interviews with agency scientists and administrators, it became clear that many individuals creating important science policy records or scientific research records were unaware of the record-keeping program of their agency. This was the case in varying degrees at each of the agencies involved in our selected projects throughout our long-term study: DOD, DOE, NASA, NIH, NOAA, NSF, and USGS. Education programs to stress the importance of working with the scientists who create the records and of following records retention policies in order to document their projects would increase the survival of significant records. Agency records management staff should take advantage of workshops offered by the National Archives. They should, in turn, be expected to offer workshops for their agency employees, both at headquarters and in the field. One very effective means is to hold periodic workshops for secretaries and other files administrators (including those responsible for maintaining central files) so that they understand agency records schedules and are knowledgeable about identifying which records should be destroyed, which saved, and how and why.
3.c. Federal funding agencies should save records documenting interagency funding of collaborative research projects.
Individual Federal agencies are usually the sole funder of collaborative research projects. In the instances where their funding responsibilities are shared with other agencies, the agency that takes the lead role should preserve on a permanent basis its records of interagency meetings, correspondence, agreements, and so forth.
3.d. Federal agencies whose research centers/laboratories are operated under contract should permanently secure their headquarters' records relating to the contractor organizations.
In some important instances Federal agencies (notably DOE and NSF) do not operate their research centers/sites directly but rather through contracting organizations. Some contractors are universities, corporations, or other longstanding institutions; other contractors are set up for the very purpose of operating FFRDCs (Federally Funded Research and Development Centers). Examples of the latter category are AUI (Associated Universities, Inc.), AURA (Association of Universities for Research in Astronomy, Inc.), and URA (University Research Association, Inc.). The role exercised by these contractor organizations over the research directions and policies of their centers/laboratories is considerable and, therefore, the importance of documenting their activities is clear. Records at the relevant agency headquarters would include correspondence between the agency and contractor, minutes of contractor board meetings, annual fiscal and progress reports, and copies of committee reports--with names like Users Committee and Visiting Committee--of the centers/laboratories under contract.
See also Recommendation #5 to NSF and #8 to NSF National Observatories.
3.e. Federal agencies should permanently secure proposals and other documentation related to major research facilities at their centers/laboratories and other sites.
When laboratories request support for the new research facilities (such as accelerators, particle "factories," telescopes, reactors, and supercomputers) and for other major instrumentation, Federal agencies should permanently secure the proposals--whether accepted and rejected-- along with relevant correspondence. Files for successful facility proposals should also include financial and narrative progress reports, final reports, records of agency site visits, correspondence with site officials, and any other materials that provide valuable documentation.
N.B.: This Recommendation pertains to proposals from centers/laboratories/observatories for building major research facilities; Recommendation #3.f. pertains to proposals for experimental research projects.
3.f. Federal funding agencies should save controversial--albeit unsuccessful--collaborative research proposals in addition to successful ones.
Federal funding agencies are currently required to save records on successful research proposals (contracts, cooperative agreements). We recommend that--for multi-institutional research collaborations--the agencies also preserve the records for the (relatively few) unsuccessful proposals that stimulate significant debates or controversies. The files typically would include proposals, referee reports, minutes of panel meetings, and--in some cases--records of agency site visits.
N.B.: This recommendation pertains to proposals for collaborative research projects; Recommendation #3.e. pertains to proposals from laboratories for building major research facilities.
CATEGORY FOUR--SPECIFIC AGENCIES
Department of Energy (DOE)
Recommendation #4: DOE should be commended for its new R&D records schedule; it should make certain the implementation of the schedule is fully supported.
Explanation:
The DOE and its Records Management staff, as well as the NARA liaison archivist, deserve congratulations on the development of its excellent, new schedule for R&D records--no modest task. We now ask DOE to provide the fiscal and moral support needed for the implementation of these important schedules.
We believe that the DOE's new R&D Records schedules support these AIP Project Recommendations as well as our appraisal guidelines (see Report No. 2, Section Three, Archival Findings and Appraisal Guidelines). We ask that DOE Records Officers contact us regarding any discrepancies.
See also Recommendation #2.c. to NARA and #3.b. to Federal Agencies, above.
National Science Foundation (NSF)
Recommendation #5: NSF should include archival arrangements in the requirements for cooperative agreements to support its research facilities and its centers.
Explanation:
These NSF-supported research facilities (e.g., National Observatories) and centers (both its Materials Research Science and Engineering Centers [MRSECs] and its Science and Technology Centers [STCs]) do not create Federal records. Special arrangements should be made to permanently secure the essential documentation of their research programs. Specifically, NSF should fully fund the archival programs at its national facilities and recommend archival care of core records of its centers.
NSF Facilities. Because of their long-standing importance and because they lack affiliations with established archival repositories, we are especially concerned about the NSF National Observatories (e.g., National Radio Astronomical Observatory). To our knowledge these observatories lack strong records management programs. The NSF should encourage them through fiscal as well as moral support to initiate archival programs to permanently secure at least their most important documentation.
NSF Centers. MRSECs and STCs are relatively new and rapidly growing phenomena at academic settings. NSF funds its centers for a period of years to function as multi-institutional collaborations and foster research in particular areas of materials science or science and technology. Although the centers are at academic settings, academic archivists will need to be persuaded to consider the documentation of NSF centers to be part of their responsibility. The fact that the NSF centers are impermanent institutions presents another danger to the records.
NSF should stipulate appropriate arrangements for records in its cooperative agreements / contracts. A very small fraction of the amount awarded to the National Observatories would pay for the proper organization of records permitting greater efficiencies of operations as well as the archival maintenance or orderly transfer of records. Special NSF funding may not be required to secure the small set of core archival records of NSF centers.
See also Recommendations #6.b. to Academic Archives and #8 to NSF National Observatories, below.
CATEGORY FIVE--OTHER INSTITUTIONAL SETTINGS
Academic Institutions
Recommendation #6: Professional files of collaboration principal investigators and other key academic scientists should be retained by their home institutions according to their individual careers; and, Academic archives should enlarge as necessary the scope of collecting policies in order to accession non-Federal records of NSF centers.
Explanation:
6.a. Professional files of collaboration principal investigators and other key academic scientists should be retained by their home institutions according to their individual careers.
Theprofessional papers of PIs (principal investigators) are a prime location for information concerning the development of an experiment or an experiment team. A substantial fraction of the principal investigators in the collaborations we studied are employed by academia. The papers of those who have regularly led or participated in important collaborative research are well worth saving. In other cases, collaboration-related records kept by a faculty member should be accessioned, especially if the collaboration was deemed significant.
N.B.: This is a rewording of Recommendation #1, above. Our point here is to emphasize the essential role academic archives play in documenting collaborative research by preserving the papers of individual scientists who played leadership roles in the projects.
6.b. Academic archives should enlarge, as necessary, the scope of collecting policies in order to accession non-Federal records of NSF centers.
The NSF centers (both its Materials Research Science and Engineering Centers and its Science and Technology Centers) are funded for a period of years; although renewals are possible, they are not permanent. The NSF centers are organized to function as multi-institutional collaborations; most, if not all, make the final decisions on which researchers at member institutions get funded. Like other NSF-funded organizations, the centers do not produce Federal records.
The academic institutions within which they operate should hold themselves responsible for accessioning core records of the center. If such arrangements are not possible, the records should be offered as a gift to the Archivist of the United States and the National Archives and Records Administration.
See also Recommendation #2.d. to NARA and #5 to the NSF, above.
Nonacademic Research Laboratories
Recommendation #7: Non-academic research laboratories lacking programs to identify and permanently secure records of historical value should initiate them.
Explanation:
The nonacademic laboratories/observatories in our long-term study have included all major categories of research laboratories--primarily those in the U.S., but also some major laboratories abroad. (During our Phase III work, corporate laboratories and FFRDCs supported by DOE and NSF have been institutional members of our selected collaborations.) With the exception of DOE laboratories, virtually all nonacademic laboratories--however important their contributions to science may be--lack programs to protect their valuable records.
Our experience shows it is possible to permanently preserve an adequate record of scientific research where laboratories have records advocates (i.e. archivists, historians, or records managers trained in noncurrent records), and impossible where laboratories lack them. Records advocates are needed to work with scientists to identify and permanently secure those records of interest to future scientist-administrators, historians, and other users. From our experience it seems clear that the chief responsibility for initiating these programs lies with the individual laboratory directors. Once programs are in place, records advocates develop relationships of trust and provide an array of invaluable services to laboratory staff and management. The records they preserve are the best means to achieve the all-important institutional memory.
National Science Foundation National Observatories
Recommendation #8: NSF National Observatories that lack archival programs should initiate them.
Explanation:
As already stated, these NSF facilities consist of some of the most important observatories in the country, if not the world. There is no doubt that future historians and other scholars will need to draw on their historically valuable records.
NSF National Observatories should consider maintaining their collections of archival records on site. Where this is not feasible, the essential records may be offered to a nearby university or state historical society; they may also be offered to the National Archives because they provide important evidence of Federal support of science.
See also Recommendation #2.d. to NARA, #3.d. to Federal Agencies, and #5 to NSF.
RECOMMENDATIONS--WHAT AND HOW TO SAVE
CATEGORY ONE--CORE RECORDS BY SCIENTIFIC DISCIPLINE
Core Records by Scientific Discipline
Recommendation #9: A core set of records should be saved at appropriate repositories to document multi-institutional collaborations.
Explanation:
There is a short list of records that, taken together, provide adequate documentation of most collaborative projects. Prime examples of core records are proposal files at Federal funding agencies (including referee and panel reports and annual and final progress reports). Core records for collaborations in the disciplinary fields studied in Phase III are listed in this report (Section One, Part III). For further information on these records, see the Archival Analysis and Appraisal Guidelines, Section Three of Report No. 2.
CATEGORY TWO--SIGNIFICANT COLLABORATIONS
Significant Collaborations
Recommendation #10: Fuller documentation should be saved for significant collaborations; Scientists and others must take special care to identify past collaborations that have made significant contributions; and, Research laboratories and other centers should set up a mechanism to permanently secure records of future significant experiments.
Explanation:
10.a. FulleR documentation should be saved for significant collaborations
A wider array of substantial documentation should be preserved for highly important collaborations to meet the needs of scientist/administrators as well as historians and other scholars. The early identification of current experiments of outstanding significance should initiate actions to permanently secure fuller documentation for subsequent appraisal. For example, laboratory research directors should make sure that chief scientists take steps to safeguard records of potential historical value. This documentation would include those categories of records specified in the appraisal guidelines prepared by the AIP Study and other records found to contain valuable evidence of the collaboration's organizational structure and research process. Records to be saved for significant collaborations in the disciplinary fields studied in Phase III are listed in this report (Section One, Section III) and described in detail in Report No. 2, Section Three: Archival Analysis and Appraisal Guidelines.
N.B.: We make note that, for the largest and most controversial multi-institutional collaborations, significant documentation will also be found at higher administrative levels, such as offices of presidents and provosts of universities, top administrators at agencies and laboratories, and other key policy boards. We do not address recommendations to offices at such higher levels on the assumption that their records are already secured.
10.b. Scientists and others must take special care to identify past collaborations that have made significant contributions.
Future scholars, as well as science administrators and policy makers, will need considerably more documentation in order to study in more detail those multi-institutional scientific collaborations that can be considered most significant in their contributions to advances in scientific knowledge, including theory and experimental techniques.
There exist general guidelines for identifying significant research projects. The best we have found thus far are in the 1998 DOE Research and Development Records Schedule.(3) Other parameters for identifying significant projects can obviously be made to meet the needs of particular research laboratories, say in the corporate sector, or by disciplines outside those covered by DOE research. Our first concern must be the identification of past collaborative research projects, since the documentation becomes endangered as soon as the project has ended and scientists turn their attention to other matters. The participation of all knowledgeable parties is needed:
1. Individual scientists could bring the contributions of a research project they consider to be significant to the attention of their research director, institutional archivist, etc.
2. Academic departments or research laboratories could set up an ad hoc history committee from time to time to identify their most significant research projects and bring them to the attention of their provost, archival program, etc.
3. Policy and planning bodies, such as DOE's High Energy Physics Advisory Panel, could compile lists of most significant research collaborations and broadcast them to their disciplines.
4. History committees of AIP Member Societies could either compile lists or survey their members for contributions and then broadcast the lists to their members.
The AIP Center for History of Physics will also contribute to the identification of recent significant research collaborations by working proactively with Boards of the National Academy of Sciences and other policy and planning bodies.
10.c. Research laboratories and other centers should set up a mechanism to permanently secure records of future significant experiments.
The scientists and research directors--at laboratories/observatories and other research centers--are best informed to identify those experiments/projects that are likely to be considered significant by future judgements. We are aware that efforts to document events from earlier decades will be frustrated by frailties of records-keeping practices. Therefore, we urge the laboratories themselves to identify as early as possible experiments/projects of potential significance. While doing so, the research directors should bear in mind the recent emergence of subcontractors for major research and development collaborations and identify experiments/ projects in which significant subcontracts should be documented--either by the laboratory, the subcontractor, or a combination of both.
Laboratories and other research centers can easily reduce the complexity of locating the additional records needed to document the more significant experiments by setting up a mechanism to identify and permanently secure records during or prior to their creation. Once a proposal for an experiment is approved, the research center should require a collaboration to include in their next write-up a statement as to: (1) which individual collaboration member should be responsible for collaboration-wide records and (2) which, if any, records on the team level should be retained on a long-term basis because of scientific significance.(4) A collaboration's chief scientist knows at the outset when a particular component of the instrument or technique is revolutionary or innovative; appropriate identification and assignment of records responsibilities for these should be included. When assigning responsibility for collaboration-wide records to an individual, the chief scientist should select a collaboration member at a permanent institution; in many cases, this will be an academic institution or the research center itself. A collaboration's statement about records-keeping responsibilities should be incorporated in its MOU (Memorandum of Understanding) or other contractual agreement with the research center.
The purpose of this recommendation is to secure the records that may be needed to document significant experiments. Later, when an experiment has been identified as significant, archivists will be in an excellent position to contact the individuals assigned responsibility for the records and make arrangements to permanently preserve those of enduring value.
The laboratories and research directors should also consider employing technologies that would assist in the capture, retention, and access to valuable evidence. For example, the research centers could retain certain files, such as collaboration e-mail, Web sites, and other relevant electronic records, on the their computer systems.
Recommendation #11: Institutional archives should share information on their relevant holdings with each other and with AIP/RLIN.
Explanation:
Knowledge of institutional records and professional papers of individuals is essential to foster use by historians and other scholars. For example, papers documenting a particular experiment are likely to be physically located in various repositories; shared catalogs will bring them together intellectually for the user. Archivists should include sufficient facts--such as laboratory name and experiment/project number or title--to identify the experiments documented in their collections when they prepare inventories, scope and content notes (or any other descriptions), and indexes.
One means for archivists to broadcast information on their holdings is to send descriptions of collections or records series to the AIP where they will be added to the International Catalog of Sources for History of Physics and Allied Sciences, maintained by the AIP Center for History of Physics (https://www.aip.org/history-programs/physics-history). In cases where the archives itself does not report its holdings to the American database RLIN-AMC (the Research Libraries Information Network-Archives and Manuscript Control) of the Research Libraries Group, the AIP can provide this service.
THE ROLE OF THE AIP CENTER
The AIP Center can play a facilitating role in a number of these recommendations. It can work with laboratories and other research institutes by: (1) providing advice to those that decide to establish or upgrade archival programs, (2) aiding in the process of identifying significant experiments, and (3) assisting laboratory advisory committees in such areas as identifying appropriate repositories for papers and records documenting significant experiments. The AIP Center will continue its work with corporate, academic, and other institutional archivists to preserve significant papers and records and to provide advice on records appraisal. In addition to its International Catalog of Sources, the Center offers, upon request, such cataloging tools as topical indexing terms and authorized names of thousands of individuals and institutions. Contact information is available on our Web site (https://www.aip.org/history-programs/physics-history).
AIP
Working Group for Documenting Multi-Institutional
Collaborations in Ground-Based Astronomy, Materials Science, Heavy-Ion and
Nuclear Physics, Medical Physics and Computer-Mediated Collaborations
Dr. Gabriel Aeppli
NEC Research Institute
Dr. Alan Berson
Program Manager
National Heart, Lung, and Blood Institute
National Institutes of Health
Dr. Robert L Dickman
Program Manager
Division of Astronomical Sciences
National Science Foundation
Prof. Paul A Fleury
Dean of Engineering
University of New Mexico
Prof. Phillip Geil
Dept. of Materials Science & Engineering
University of Illinois at Urbana-Champaign
Dr. Russell H. Huebner
Research Program Manager
Advanced Photon Source
Argonne National Laboratory
Prof. Thomas John Humanic
Department of Physics
Ohio State University
Dr. Goetz K. Oertel, President
Association of Universities for Research in Astronomy
Prof. Thomas F. Rosenbaum, Director
James Franck Institute
University of Chicago
Dr. Ulrich Strom, Program Director
Materials Research Science and
Engineering Centers
National Science Foundation
Paul A. Vanden Bout, Director
National Radio Astronomical Observatory
Dr. John Watson, Deputy Director
National Heart, Lung, and Blood Institute
National Institutes of Health
Dr. Sidney Wolff, Director
National Optical Astronomy Observatories
AIP Project Consultant
Prof. Wesley Shrum
Department of Sociology
Louisiana State University
AIP Project Staff
Ms. Joan Warnow-Blewett
Project Director, AIP Study
and
Archivist Emeritus, AIP Center
Dr. Spencer R. Weart
Chair, AIP Study Working Group
and
Director, AIP Center
Dr. Joel Genuth
Project Historian, AIP Study
Dr. Ivan Chompalov
Project Sociologist, AIP Study
Mr. R. Joseph Anderson
Assistant Director, AIP Center
1. The AIP project recommendations for Phase I (high-energy physics) and for Phase II (space science and geophysics) are available on the AIP Center's web site (https://www.aip.org/history-programs/study-multi-institutional-collaborations) or upon request to the AIP Center (here's how to contact us or send e-mail.
2. The recommendation is well-suited to projects conducted at, or--in the case of NSF centers--approved for funding at a central research site. Unfortunately, fields like VLBI observations and medical physics lack a central site; in such cases, the most appropriate person to recommend the identification of "records keepers" would be the program officer at the funding agency.
3. See the DOE Website (http://www-it.hr.doe.gov/) for this schedule; of particular interest is the Introduction which includes a review of the guidelines and an R&D evaluation checklist. See also Recommendation #2.c. to the National Archives, above.
4. See also footnote 2.
posted 6/22/01
RETURN
to History Center HomePage or RETURN
to the Publications List
RETURN
to the top