When one looks at a computer system that is doing a very complex job, he sees on the surface a machine that can execute some extremely sophisticated processes. If he is a layman, his concept of what provides this sophisticated capability may endow the machine with a mysterious power to sweep information through perceptive and intelligent synthetic thinking devices. Actually, this sophisticated capability results from a very clever organizational hierarchy so that pursuit of the source of intelligence within this system would take one down through layers of functional and physical organization that become successively more primitive.2c1a

To be more specific, we can begin at the top and list the major levels down through which we would pass if we successively decomposed the functional elements of each level in search of the "source of intelligence." A programmer could take us down through perhaps three levels (depending upon the sophistication of the total process being executed by the computer) perhaps depicting the organization at each level with a flow chart. The first level down would organize functions corresponding to statements in a problem-oriented language (e.g., ALGOL or COBOL), to achieve the desired over-all process. The second level down would organize lesser functions into the processes represented by first-level statements. The third level would perhaps show how the basic machine commands (or rather the processes which they represent) were organized to achieve each of the functions of the second level.2c1b

Then a machine designer could take over, and with a block diagram of the computer's organization he could show us (Level 4) how the different hardware units (e.g., random-access storage, arithmetic registers, adder, arithmetic control) are organized to provide the capability of executing sequences of the commands used in Level 3. The logic designer could then give us a tour of Level 5, also using block diagrams, to show us how such hardware elements as pulse gates, flip-flops' and AND, OR, and NOT circuits can be organized into networks giving the functions utilized at Level 4. For Level 6 a circuit engineer could show us diagrams revealing how components such as transistors, resistors, capacitors, and diodes can be organized into modular networks that provide the functions needed for the elements of Level 5.2c1c

Device engineers and physicists of different kinds could take us down through more layers. But rather soon we have crossed the boundary between what is man-organized and what is nature-organized, and are ultimately discussing the way in which a given physical phenomenon is derived from the intrinsic organization of sub-atomic particles, with our ability to explain succeeding layers blocked by the exhaustion of our present human comprehension.2c1d

If we then ask ourselves where that intelligence is embodied, we are forced to concede that it is elusively distributed throughout a hierarchy of functional processes--a hierarchy whose foundation extends down into natural processes below the depth of our comprehension. If there is any one thing upon which this 'intelligence depends' it would seem to be organization. The biologists and physiologists use a term "synergism" to designate (from Webster's Unabridged Dictionary, Second Edition) the "...cooperative action of discrete agencies such that the total effect is greater than the sum of the two effects taken independently..." This term seems directly applicable here, where we could say that synergism is our most likely candidate for representing the actual source of intelligence2c1e

Actually, each of the social, life, or physical phenomena we observe about us would seem to derive from a supporting hierarchy of organized functions (or processes), in which the synergistic principle gives increased phenomenological sophistication to each succeedingly higher level of organization. In particular, the intelligence of a human being, derived ultimately from the characteristics of individual nerve cells, undoubtedly results from synergism.2c1f

It has been jokingly suggested several times during the course of this study that what we are seeking is an "intelligence amplifier." (The term is attributed originally to W. Ross Ashby (2, 3). At first this term was rejected on the grounds that in our view one's only hope was to make a better match between existing human intelligence and the problems to be tackled, rather than in making man more intelligent. But deriving the concepts brought out in the preceding section has shown us that indeed this term does seem applicable to our objective.2c2a

Accepting the term "intelligence amplification" does not imply any attempt to increase native human intelligence. The term "intelligence amplification" seems applicable to our goal of augmenting the human intellect in that the entity to be produced will exhibit more of what can be called intelligence than an unaided human could; we will have amplified the intelligence of the human by organizing his intellectual capabilities into higher levels of synergistic structuring. What possesses the amplified intelligence is the resulting H-LAM/T system, in which the LAM/T augmentation means represent the amplifier of the human's intelligence.2c2b

In amplifying our intelligence, we are applying the principle of synergistic structuring that was followed by natural evolution in developing the basic human capabilities. What we have done in the development of our augmentation means is to construct a superstructure that is a synthetic extension of the natural structure upon which it is built. In a very real sense, as represented by the steady evolution of our augmentation means, the development of "artificial intelligence" has been going on for centuries.2c2c

The human and the artifacts are the only physical components in the H-LAM/T system. It is upon their capabilities that the ultimate capability of the system will depend. This was implied in the earlier statement that every composite process of the system decomposes ultimately into explicit-human and explicit-artifact processes. There are thus two separate domains of activity within the H-LAM/T system: that represented by the human, in which all explicit-human processes occur; and that represented by the artifacts, in which all explicit-artifact processes occur. In any composite process, there is cooperative interaction between the two domains, requiring interchange of energy (much of it for information exchange purposes only). Figure 1 depicts this two domain concept and embodies other concepts discussed below.2c3a

Where a complex machine represents the principal artifact with which a human being cooperates, the term "man-machine interface" has been used for some years to represent the boundary across which energy is exchanged between the two domains. However, the "man-artifact interface" has existed for centuries, ever since humans began using artifacts and executing composite processes.2c3b

Exchange across this "interface" occurs when an explicit-human process is coupled to an explicit-artifact process. Quite often these coupled processes are designed for just this exchange purpose, to provide a functional match between other explicit-human and explicit-artifact processes buried within their respective domains that do the more significant things. For instance, the finger and hand motions (explicit human processes) activate key-linkage motions in the typewriter (couple to explicit-artifact processes). But these are only part of the matching processes between the deeper human processes that direct a given word to be typed and the deeper artifact processes that actually imprint the ink marks on the paper.2c3c

The outside world interacts with our H-LAM/T system by the exchange of energy with either the individual or his artifact. Again, special processes are often designed to accommodate this exchange. However, the direct concern of our present study lies within the system, with the internal processes that are and can be significantly involved in the effectiveness of the system in developing the human's comprehension and pursuing the human's goals.2c3d

Before we pursue further direct discussion of the H-LAM/T system, let us examine some background material. Consider the following historical progression in the development of our intellectual capabilities:2c4a

2c4b

It is undoubtedly true that concepts which people found useful ended up being symbolized in their language, and hence that the evolution of language was affected by the concepts the people developed and used. However, Korzybski 4 and Whorf 5 (among others) have argued that the language we use affects our thinking to a considerable extent. They say that a lack of words for some types of concepts makes it hard to express those concepts, and thus decreases the likelihood that we will learn much about them. If this is so, then once a language has begun to grow and be used, it would seem reasonable to suspect that the language also affects the evolution of the new concepts to be expressed in that language.2c4c

Apparently there are counter-arguments to this; e.g., if a concept needs to be used often but its expression is difficult, then the language will evolve to ease the situation. However, the studies of the past decade into what are called "self-organizing" systems seem to be revealing that subtle relationships among its interacting elements can significantly influence the course of evolution of such a system. If this is true, and if language is (as it seems to be) a part of a selforganizing system, then it seems probable that the state of a language at a given time strongly affects its own evolution to a succeeding state.2c4d

For our conceptual framework, we tend to favor the view that a language does exert a force in its own evolution. We observe that the shift over the last few centuries in matters that are of daily concern to the individual has necessarily been forced into the framework of the language existing at the time, with alterations generally limited to new uses for old words, or the coining of new words. The English language since Shakespeare has undergone no alteration comparable to the alteration in the cultural environment; if it had, Shakespeare would no longer be accessible to us. Under such evolutionary conditions, it would seem unlikely that the language we now use provides the best possible service to our minds in pursuing comprehension and solving problems. It seems very likely that a more useful language form can be devised.2c4e

The Whorfian hypothesis states that the world view of a culture is limited by the structure of the language which that culture uses. But there seems to be another factor to consider in the evolution of language and human reasoning ability. We offer the following hypothesis, which is related to the Whorfian hypothesis: Both the language used by a culture, and the capability for effective intellectual activity are directly affected during their evolution by the means by which individuals control the external manipulation of symbols. (For identification, we will refer to this as the Neo-Whorfian hypothesis.) 2c4f

If the Neo-Whorfian hypothesis could be proved readily, and if we could see how our means of externally manipulating symbols influence both our language and our way of thinking, then we would have a valuable instrument for studying human-augmentation possibilities. For the sake of discussion, let us assume the Neo-Whorfian hypothesis to be true, and see what relevant deductions can be made.2c4g

If the means evolved for an individual's external manipulation of his thinking-aid symbols indeed directly affect the way in which he thinks, then the original Whorfian hypothesis would offer an added effect. The direct effect of the external-symbol-manipulation means upon language would produce an indirect effect upon the way of thinking via the Whorfian-hypothesis linkage. There would then be two ways for the manner in which our external symbol manipulation was done to affect our thinking.2c4h

One way of viewing the H-LAM/T system changes that we contemplate--specifically, integrating the capabilities of a digital computer into the intellectual activity of individual humans--is that we are introducing new and extremely advanced means for externally manipulating symbols. We then want to determine the useful modifications in the language and in the way of thinking that could result. This suggests a fourth stage to the evolution of our individual-human intellectual capability: 2c4i

2c4j

These hypotheses imply great richness in the new evolutionary spaces opened by progressing from Stage 3 to Stage 4. We would like to study the hypotheses further, examining their possible manifestations in our experience, ways of demonstrating their validity, and possible deductions relative to going to Stage 4.2c4k

In search of some simple ways to determine what the Neo-Whorfian hypothesis might imply, we could imagine some relatively straightforward means of increasing our external symbol-manipulation capability and try to picture the consequent changes that could evolve in our language and methods of thinking. Actually, it turned out to be simpler to invert the problem and consider a change that would reduce our capability for external symbol manipulation. This allowed an empirical approach which proved both simple and effective. We thus performed the following experiment.2c4l

Brains of power equal to ours could have evolved in an environment where the combination of artifact materials and muscle strengths were so scaled that the neatest scribing tool (equivalent to a pencil, possible had a shape and mass as manageable as a brick would be to us-assuming that our muscles were not specially conditioned to deal with it. We fastened a pencil to a brick and experimented. Figure 2 shows the results, compared with typewriting and ordinary pencil writing. With the brick pencil, we are slower and less precise. If we want to hurry the writing, we have to make it larger. Also, writing the passage twice with the brick-pencil tires the untrained hand and arm.2c4m

How would our civilization have matured if this had been the only manual means for us to use in graphical manipulation of symbols? For one thing, the record keeping that enables the organization of commerce and government would probably have taken a form so different from what we know that our social structure would undoubtedly have evolved differently. Also, the effort in doing calculations and writing down extensive and carefully reasoned argument would dampen individualexperimentation with sophisticated new concepts, to lower the rate of learning and the rate of useful output, and perhaps to discourage a good many people from even working at extending understanding. The concepts that would evolve within our culture would thus be different, and very likely the symbology to represent them would be different--much more economical of motion in their writing It thus seems very likely that our thoughts and our language would be rather directly affected by the particular means used by our culture for externally manipulating symbols, which gives little intuitive substantiation to our Neo-Whorfian hypothesis.2c4n

To reflect further upon the implications of this hypothesis, the following hypothetical artifact development can be considered, representing a diiferent type of external symbol manipulation that could have had considerable effect. Suppose that our young technology of a few generations ago had developed an artifact that was essentially a high speed, semi-automatic table-lookup device--cheap enough for almost everyone to afford and small and light enough to be carried on the person. Assume that the individual cartridges sold by manufacturers (publishers) contained the look-up information, that one cartridge could hold the equivalent of an unabridged dictionary, and that a one-paragraph definition could always be located and displayed on the face of the device by the average practised individual in less than three seconds. The fortunes of technological invention, commercial interest, and public acceptance just might have evolved something like this.2c4o

If it were so very easy to look things up, how would our vocabulary develop, how would our habits of exploring the intellectual domains of others shift, how might the sophistication of practical organization mature (if each person can so quickly and easily look up applicable rules), how would our education system change to take advantage of this new external symbol-manipulation capability of students and teachers (and administrators)?2c4p

The significance to our study of the discussion in this section lies in the perspective it gives to the ways in which human intellectual effectiveness can be affected by the particular means used by individuals for their external symbol manipulation. It seems reasonable to consider the development of automated external symbol manipulation means as a next stage in the evolution of our intellectual power.2c4q

The concept of our H-LAM/T system possessing a repertoire of capabilities that is structured in the form of a hierarchy is most useful in our study. We shall use it in the following to tie together a number of considerations and concepts. There are two points of focus in considering the design of new repertoire hierarchies: the materials with which we have to work, and the principles by which new capability is constructed from these basic materials.2c5a

a. Basic Capabilities2c5b

b. Structure Types2c5c

c. Roles and Levels2c5d

d. Model of Executive Superstructure2c5e

e. Flexibility in the Executive Role2c5f

f. Compound Effects2c5g

He wrote as World War II was coming to an end, and his principal purpose seemed to be to offer new professional objectives to those scientists who were soon to be freed from war-motivated research and development. It would seem that he also wished to induce a general recognition of a growing problem--storage, retrieval, and manipulation of information for and by intellectual workers--and to show the possibilities he foresaw for scientific development of equipment which could significantly aid such workers in facing this problem. He summarized the situation: "...There is a growing mountain of research...The investigator is staggered by the findings and conclusions of thousands of other workers. Professionally our methods of transmitting and reviewing the results of research are generations old...truly significant attainments become lost in the mass of the inconsequential...The summation of human experience is being expanded at a prodigious rate, and the means we use for threading through the consequent maze to the momentarily important item is the same as was used in the days of square-rigged ships."3a5a

Then he brought out some general considerations for hope: ".. But there are signs of a change as new and powerful instrumentalities come into use...Photocells...advanced photography...thermionic tubes... cathode ray tubes...relay combinations...there are plenty of mechanical aids with which to effect a transformation in scientific records." And he points out that devices which we commonly use today--e.g., a calculating machine or an automobile--would have been impossibly expensive to produce in earlier eras of our technological development. "...The world has arrived at an age of cheap complex devices of great reliability and something is bound to come of it."3a5b

In six and a half pages crammed full of well-based speculations, Bush proceeds to outline enough plausible artifact and methodology developments to make a very convincing case for the augmentation of the individual intellectual worker. Extension of existing photographic techniques to give each individual a continuously available miniature camera for recording anything in view and of interest, and to realize a high-quality 100:1 linear reduction ratio for micro-record files for these photographs and published material; voice-recognition equipment (perhaps requiring a special language) to ease the process of entering new self-generated material into the written record--these are to provide the individual with information-generating aid.3a5c

For the detailed manipulation of mathematical and logical expressions, Bush projects computing aids (which have been surpassed by subsequent development) that allow the individual to exercise a greater proportion of his time and talents in the tasks of selecting data and the appropriate transformations and processes which are to be executed, leaving to the machinery the subsequent execution. He suggests that new notation for our verbal symbols (perhaps binary) could allow character recognition devices to help even further in the information-manipulation area, and also points out that poor symbolism ("...the exceedingly crude way in which mathematicians express their relationships. They employ a symbolism which grew like Topsy and has little consistency; a strange fact in that most logical field.") stands in the way of full realization of machine help for the manipulations associated with the human's real time process of mathematical work. And "...Then, on beyond the strict logic of the mathematician, lies the application of logic in everyday affairs. We may some day click off arguments on a machine with the same assurance that we now enter sales on a cash register."3a5d

Then " ..So much for the manipulation of ideas and their insertion into the record. Thus far we seem to be worse off than before--for we can enormously extend the record; yet even in its present bulk we can hardly consult it. This is a much larger matter than merely the extraction of data for the purposes of scientific research; it involves the entire process by which man profits by his inheritance of acquired knowledge The prime action of use is selection, and here we are halting indeed. There may be millions of fine thoughts, and the account of the experience on which they are based, all encased within stone walls of acceptable architectural form; but if the scholar can get at only one a week by diligent search, his syntheses are not likely to keep up with the current scene." He goes on to discuss possible developments that could allow very rapid (in the human's time frame) selection of unit records from a very large file--where the records could be dry-process photographic micro-images upon which the user could add lnformation at will.3a5e

Bush goes on to say, "The real heart of the matter of selection, however, goes deeper than a lag in the adoption of mechanisms...Our ineptitude in getting at the record is largely caused by the artificiality of systems of indexing." He observes the power of the associative recall which human memory exhibits, and proposes that a mechanization of selection by association could be realized to considerable advantage. He spends the last two pages (a quarter of his article) describing a device embodying this capability, and points out some features of its use and of its likely effect. This material is so relevant and so well put that I quote it in its entirety:3a5f

3a5g

There are many significant items in the article, but the main ones upon which we shall comment here will be those relative to the use and implications of his Memex. The associative trails whose establishment and use within the files he describes at some length provide a beautiful example of a new capsbility in symbol structuring that derives from new artifact-process capability, and that provides new ways to develop and portray concept structures. Any file is a symbol structure whose purpose is to represent a variety of concepts and concept structures in a way that makes them maximally available and useful to the needs of the human's mental-structure development -- within the limits imposed by the capability of the artifacts and human for jointly executing processes of symbol-structure manipulation. The Memex allows a human user to do more conveniently (less energy, more quickly) what he could have done with relatively ordinary photographic equipment and filing systems, but he would have had to spend so much time in the lower-level processes of manipulation that his mental time constants of memory and patience would have rendered the system unusable in the detailed and intimate sense which Bush illustrates.3a6a

The Memex adds a factor of speed and convenience to ordinary filing-system (symbol-structuring) processes that would encourage new methods of work by the user, and it also adds speed and convenience for processes not generally used before. Making it easy to establish and follow the associative trails makes practical a new symbol-structuring process whose use can make a significant difference in the concept structuring and bssic methods of work. It is also probable that clever usage of associative-trail manipulation can augment the human's process structuring and executing capacilities so that he could successfully make use of even more powerful symbol-structure manlpulation processes utilizing the Memex capabilities. An example of this general sort of thing was given by Bush where he points out that the file index can be called to view at the push of a button, which implicitly provides greater capability to work within more sophisticated and complex indexing systems3a6b

Note, too, the implications extending from Bush's mention of one user duplicating a trail (a portion of his structure) and giving it to a friend who can put it into his Memex and integrate it into his own trail (structure). Also note the "wholly new forms of encyclopedia", the profession of "trail blazers," and the inheritance from a master including "the entire scaffolding" by which such additions to the world's record were erected. These illustrate the types of changes in the ways in which people can cooperate intellectually that can emerge from the augmentation of the individuals. This type of change represents a very significant part of the potential value in pursuing research directly on the means for making individuals intellectually more effective.3a6c

A number of useful new structuring processes can be made available to an individual through development and use of relatively simple equipment that is mostly electromechanical in nature and relatively cheap. We can begin developing examples of this by describing the hand operated, edge-notched card system that I developed and used over the past eight years.3a7a

a. An Existing Note and File System3a7b

b. Comments on the System3a7c

c. Associative-Linking Possibilities3a7d

d. An Experiment Illustrating Usage and Further System Possibilities3a7e

This section may be of value both to readers who are already familiar with computers, and to those who are not. A little familiarity with computer technology, enough to help considerably in understanding the augmentation possibilities discussed in thls report, can be gained by the uninitiated. For those already familiar with the technology, the following discussion can perhaps help them gain more understanding of our concepts of process and symbol structuring.3a8a

A computer is directly capable of performing any of a basic repertoire of very primitive symbol-manipulation processes (such as "move the symbol in location A to location 12417," or "compare the symbol in location A with that in location B, and if they are the same, set switch S to ON"). There may be from ten to over a hundred different primitive processes which a particular machine can execute, and all of the computer's more sophisticated processes are structured from these primitive processes. It takes a repertoire of surprisingly few such primitive processes to enable the construction of any symbol-manipulation process that can be explicitly described in any language.3a8b

Somewhat the same situation exists relative to symbol structures i.e., there are only a very few primitive symbols with which the machine can actually work, and any new and different symbol has to be defined to the machine as a particular structure (or organization) of its primitive symbols. Actually, in every commercial digital computer, there are only two primitive symbols. Usually these are dealt with in standard-sized packets (called "words") of from eighteen to forty-eight primitive symbols, but arbitrary use can be made of individual primitives or of subgroups of the word.3a8c

To have the computer perform a non-trivial task or process, a structure of the primitive processes is organized (a computer program) and stored within the computer as a corresponding symbol structure. The computer successively examines the symbol substructure representing each primitive process in the program and executes that process--which usually alters the total internal symbol structure of the machine in some way. Lt makes no difference to the computer whether the symbols involved in the re-structuring represent part of the computer program or part of the lnformation upon which the program is operating. The ability to have the computer modify its own process structure (program) hss been a very im portant factor in the development of its power.3a8d

Thus, some very sophisticated techniques for process and symbol structuring have evolved in the computer field, as evidenced by the very sophisticated processes (e.g., predicting election returns, calculating orbits, translating natural languages) that can be structured to manipulate very complex structures of symbols. Among the more interesting computer-process structures that have evolved are those that can automatically develop a structure of primitive computer processes to accomplish symbol manipulation tasks that are specified on a relatively high level of abstraction. Special languages have been evolved in several fields (e.g., ALGOL and FORTRAN for scientific calculations, COBOL for business processing) that enable explicit prescription of complex manipulation processes in a rapid and concise manner by a human, thinking about the processes in a rather natural manner, so that special computer programs or process structures (called Translators, Compilers, or sometimes in a slightly different sense, Interpreters) can construct the necessary structures of primitive processes and symbols that would enable the computer to execute the prescribed processes. This development has extended immensely our capability for making use of computers--otherwise the specification of a complex process would often occupy a formidable number of man hours, and be subject to a great many errors which would be very costly to find and correct.3a8e

Computers have been used to simulate dynamic systems for which we humans had none but descriptive models, from which we otherwise could gain little feel for the way the system behaves. A very notable instance of this, for our consideration, has been in the area of the human thought processes. Newell, Shaw, and Simon initiated this approach) from which there has derived a number of features of interest to us. For one, they discovered that the symbol structures and the process structures required for such simulation became exceedingly complex, and the burden of organizing these was a terrific impediment to their simulation research. They devised a structurlng technique for their symbols that is basically simple but from which stem results that are very elegant. Their baslc symbol structure is what they call a 'list," a string of substructures that are linked serially in exactly the manner proposed by Bush for the associative trails in his Memex--i.e., each substructure contains the necessary information for locating the next substructure on the list. Here, though, each substructure could also be a list of substructures, and each of these could also, etc. Their standard manner for organizing the data which the computer was to operate upon is thus what they term "list structuring."3a8f

They also developed special languages to descrlbe different basic processes involved in list-structure manipulation. The most widely used of their languages, IPL-V (the fifth version of their Information Processing Languages), is described in a recent book edited by Newell.(7) In these languages, both the data to be worked upon and the symbols which designate the processes to be executed upon that data are developed in list-structure form.3a8g

Other languages and techniques for the manipulation of list structures have been described by McCarthy,(8) by Gelernter, Hansen, and Gerberich, (9) by Yngve, (10, 11) by Perlis and Thornton, (12) by Carr, (13) and by Weizenbaum.(14) The application of these techniques has been mainly of two types--one of modelling complex processes and systems (e.g., the human thought processes), where the emphasis is upon the model and its behavior, and the other of trying to get computer behavior that is intelligent whether or not the processes and behavior resemble those of a human. The languages and techniques used in both types of application promise to be of considerable value to the development of radical new augmentation systems for human problem solvers, and we shall deal later with them in more detail.3a8h

Computers have various means for storing symbols so that they are accessible to it for manipulation. Assuming that the human might want to have a repertoire of sixty-four basic symbols (letters, numbers, special symbols), we can discuss various forms of storage in terms of their capacity for storing these kinds of symbols (each oi which would be structured, in the computer and storage devices, as a group of six primitive computer symbols). Fast access to an arbitrary choice of a few neighboring symbols (of the human's repertoire) can be had to perhaps 100,000 such symbols within the period in which the computer can execute one of its primitive processes (from two to ten millionths of a second, depending upon the computer involved). This is the so-called high-speed, random-access working store, where space for the human's symbols might cost between sixty cents and $1.50 per symbol.3a8i

Cheaper, larger-capacity backup storage is usually provided by devices to which access takes considerably longer (in the computer's time reference). A continuously rotating magnetic drum can hold perhaps a million of these symbols, for which access to a random storage position may average a thirtieth of a second (waiting for the drum to come around to bring that storage position under the magnetic reading head). This is short in the human's time scale, but a reasonably fast computer could execute about 3,000 of its primitive processes during that time. Generally, information transfer between a drum and fast-access working storage takes place in blocks of data which are stored in successive positions around the drum. Such block-transfer is accomplished by a relatively small structure of primitive computer processes that cyclically executes the transfer of one word at a time until the designated block has been trans ferred. Drum storage costs about 5 cents per each of the basic symbols used by the human in our example.3a8j

Another type of backup storage uses a number of large, thin discs (about three feet in diameter), with magnetic coating on the sur faces. The discs are stacked with enough space between each so that a moveable read-record head can be positioned radially to line up over a specific circular track of symbol storage space. A commercially available disc storage system could hold over a hundred million of the human's basic symbols, to which random access would average about a tenth of a second, and where the cost per symbol-space would be about one seventh of a cent.3a8k

Magnetic tapes are commonly used for backup storage, too. For these, the random access time for storage blocks are of the order of a minute or two. Here, however, the actual storage units (the tape reels) can be taken off and shelf stored, so the total storage capacity may be very large--however, the time to locate a reel and exchange reels on the tape transport adds to the above-quoted access time--and this locating and reel changing are not generally automatic processes (i.e., a human has to do them). A transport unit, connected to the computer, might cost $30,000, with tape reels at $50 each holding about five million of the human's basic symbols. For one reel, storage space for each such symbol cost about two-thirds of a cent, but for twenty full reels in a "library" the cost comes down to about one-thirtieth of a cent per symbol space.3a8l

Other types of buffer storage for computer symbol structures are becoming available, and there is considerable economic demand spurring continuing research toward storage means that give high capacity at low cost, and with as short an access time as possible. Within the next ten years there would seem to be a very high probability of significant ad vances to this end.3a8m

For presenting computer-stored information to the human, techniques have been developed by which a cathode-ray-tube (of which the television picture tube is a familiar example) can be made to present symbols on their screens of quite good brightness, clarity, and with considerable freedom as to the form of the symbol. Under computer control an arbitrary collection of symbols may be arranged on the screen, with considerable freedom as to relative location, size, and brightness. Similarly, line drawings, curves, and graphs may be presented, with any of the other symbols intermixed. It is possible to describe to the computer, and thereafter use, new symbols of arbitrary shape and size. On displays of this sort, a light pen (a pen-shaped tool with a flexible wire to the electronic console) can be pointed by the human at any symbol or line on the display, and the computer can automatically determine what the pen is pointing at.3a8n

A cathode-ray-tube display of this sort is currently limited in resolution to about 800 lines across the face of the tube (in either direction). The detail with which a symbol may be formed, and the preciseness with which the recurrent images of it may be located, are both affected by this figure so that no matter how large the screen of such a tube, the maximum number of symbols that can be put on with usable clearness remains the same.3a8o

The amount of usable information on such a screen, in the form of letters, numbers, and diagrams, would be limited to about what a normal human eye could make out at the normal reading distance of fourteen inches on a surface 3-1/2 inches square, or to what one could discern on an ordinary 8-1/2-by-11-inch sheet of paper at about three feet. This means that one couldnot have a single-tube display giving him an 8-1/2-by-11-inch frame to view that would have as much on it as he might be used to seeing, say on the page of a journal article.3a8p

The costs of such displays are now quite high--ranging from $20,000 to $60,000, depending upon the symbol repertoire, symbol-structure display capacity, and the quality of the symbol forms. One sbould expect these prices to be lowered quite drastically as our technology improves and the market for these displays increases.3a8q

Much cheaper devices can "draw" arbitrary symbol shapes and diagrams on paper, at a speed for symbols that is perhaps a quarter of the rate that a typewriter can produce them. Also, special typewriters (at $3,000 to $4,000 apiece) can type out information on a sheet of paper, as well as allow the human to send information to the computer via the keyboard. But these two types of devices do not allow fast and flexible rearrangement of the symbols being displayed, which proves to be an important drawback in our current view of future possibilities for augmentation.3a8r

For communicating to the computer, considerable freedom exists in arranging pushbuttons, switches, and keysets for use by the human. The "interpretation" or response to be made by the computer to the actuation of any button, switch, or key (or to any combination thereof) can be established in any manner that is describable as a structure of primitive computer processes--which means essentially any manner that is explicitly describable. The limitation on the flexibility and power of any expllcit "shorthand" system with which the human may wish to utilize these input devices is the human's ability to learn and to use them.3a8s

There are also computer-input devices that can sense enough data from handwriting to allow a computer to recognize a limited number of handwritten symbols--both as they are being written and afterwards. Means for recognizing typescript are rather well developed and are already being designed into some large documentation and language translation systems. Also, a little progress has been made toward developing equipment that can recognize a limited spoken vocabulary. There is considerable economic pressure toward developing useful and cheap devices of this type, and we can expect relatively sophisticated capabilities to become available within the next ten years. Such equipment may play an important role in the individual-augmentation systems of the future (but our feeling is that a very powerful augmentation system can be developed without them).3a8t

An important type of development for our consideration of providing individual humans with close-coupled computer services is what is known as time sharing. Suppose a number of individual users connect to the same computer The computer can be programmed to serve them under any of a wide variety of rules. One such could be similar to the way the telephone system gives you attention and service when you ask for it--i.e., if too many other demands are not being made for service at that time, you get instant attention; otherwise, you wait until some service capacity is free to attend to you.3a8u

Our view of the interaction of human and computer in the future augmented system sees a large number of relatlvely simple processes (human scale of large and simple) being performed by the computer for the human--processes which often will require only a few thousandths of a second of actual computer manipulation. Such a fast and agile helper as a computer can run around between a number of masters and seldom keep any of them waiting (at least, not long enough that they would notice it or be inconvenienced appreciably). Occasionally, of course, much larger periods of computer time will be needed by an individual, and then the other users might get their periodic milliseconds of service slipped in during these longer processes.3a8v

When we began our search, we found a great deal of literature which put forth thought and work of general significance to our objective-- frankly, too much. Without having a conceptual framework, we could not efficiently filter out the significant kernels of fact and concept from the huge mass which we initially collected as a "natural first step" in our search. We feel rather unscholarly not to buttress our conceptual framework with plentiful reference to supporting work, but in truth it was too difficult to do. Developing the conceptual structure represented a sweeping synthesis job full of personal constructs from smatterings picked up in many places. Under these conditions, giving reference to a backup source would usually entail qualifying footnotes reflecting an unusual interpretation or exonerating the other author from the implications we derived from his work. We look forward to a stronger, more comprehensive, and more scholarly presentation evolving out of future work.3a9a

However, we do want to acknowledge thoughts and work we have come across that bear most directly upon the possibilities of using a computer in real-time working association with a human to improve his working effectiveness. These fall into two categories. The first category, which would include this report, presents speculations and possibilities but does not include reporting of significant experimental results. Of these, Bush (6) is the earliest and one of the most directly stimulating. Licklider (15) provided the most general clear case for the modern computer, and coined the expression, "man-computer symbiosis" to refer to the close interaction relationship between the man and computer in mutually beneficial cooperation. Ulam (16) has specifically recommended close man-computer interaction in a chapter entitled, "synergesis," where he points out in considerable detail the types of mathematical work which could be aided. Good (17) includes some conjecture about possibilities of intellectual aid to the human by close cooperation with a computer in a rather general way, and also presents a few interesting thoughts about a network model for structuring the conceptual kernels of information to facilitate a sort of self-organizing retrieval system. Ramo has given a number of talks dealing with the future possibilities of computers for "extending man's intellect," and wrote several articles (18, 19) His projections seem slanted more toward larger bodies of humans interacting with computers, in less of an intimate personal sense than the above papers or than our initial goal. Fein (20) in making a comprehensive projection of the growth and dynamic inter-relatedness of "computer-related sciences," includes specific mention of the enhancement of human intellect by cooperative activity of men, mechanisms, and automata. He coined the term "synnoetics" as applicable generally to the cooperative interaction of people, mechanisms, plant or animal organisms, and automata into a system whose mental power is greater than that of its components, and presented a good picture of the integrated way in which many currently separate disciplines should be developed and taught in the future to do justice to their mutual roles in the important metadiscipline defined as "synnoetics."3a9b

In the second category, there have been a few papers published recently describing actual work that bears directly upon our topic. Licklider and Clark, (21) and Culler and Huff, (22) in the 1962 Spring Joint Computer Conference, gave what are essentially progress reports of work going on now in exactly this sort of thing--a human with a computer backed display getting minute-by-minute help in solving problems. Teager (23, 24) reports on the plans and current development of a large time-sharing system at MIT, which is planned to provide direct computer access for a number of outlying stations located in scientists' offices, giving each of these users a chance for real-time utilization of the computer.3a9c

There are several efforts that we have heard about, but for which there are either no publications or for which none have been discovered by us. Mr. Douglas Ross, of the Electronic Systems Laboratory at MIT has, we learned by direct conversation, been thinking and working on real-time man-machine interaction problems for some years. We have recently learned that a graduate student at MIT, Glenn Randa, (25) has developed the design of a remote display console under Ross for his graduate thesis project. We understand that another graduate student there, Ivan Sutherland, is currently using the display-computer facility on the TX-2 computer at Lincoln Lab to develop cooperative techniques for engineering-design problems. And at RAND, we have learned by personal discussion that Cliff Shaw, Tom Ellis, and Keith Uncapher have been involved in implementing a multi-station time-sharing system built around their JOHNNIAC computer. Termed the JOHNNIAC Open-Shop System (JOSS for short), it apparently is near completion, and will use remote type writer stations.3a9d

Undoubtedly, there are efforts of others falling into either or both categories that have been overlooked. Such oversight has not been intentional, and it is hoped that these researchers will make their pertinent work known to us.3a9e

To try to give you (the reader) a specific sort of feel for our thesis in spite of this situation, we shall present the following picture of computer-based augmentation possibilities by describing what might happen if you were being given a personal discussion-demonstration by a friendly fellow (named Joe) who is a trained and experienced user of such an augmentation system within an experimental research program which is several years beyond our present stage. We assume that you approach this demonstration-interview with a background similar to what the previous portion of this report provides--that is, you will have heard or read a set of generalizations and a few rather primitive examples, but you will not yet have been given much of a feel for how a computer-based augmentation system can really help a person.3b3a

Joe understands this and explains that he will do his best to give you the valid conceptual feel that you want--trying to tread the narrow line between being too detailed and losing your over-all view anc being too general and not providing you with a solid feel for what goes on. He suggests that you sit and watch him for a while as he pursues some typical work, after which he will do some explaining. You are not particularly flattered by this, since you know that he is just going to be exercising new language and methodology developments on his new artifacts--and after all, the artifacts don't look a bit different from what you expected--so why should he keep you sitting there as if you were a complete stranger to this stuff? It will just be a matter of "having the computer do some of his symbol-manipulating processes for him so that he can use more powerful concepts and concept-manipulation techniques," as you have so often been told.3b3b

Joe has two display screens side by side, but one of them he doesn't seem to use as much as the other. And the screens are almost horizontal, more like the surface of a drafting table than the near-vertical picture displays you had somehow imagined. But you see the reason easily, for he is working on the display surface as intently as a draftsman works on his drawings, and it would be awkward to reach out to a vertical surface for this kind of work. Some of the time Joe is using both hands on the keys, obviously feeding information into the computer at a great rate.3b3c

Another slight surprise, though--you see that each hand operates on a set of keys on its own side of the display frames, so that the hands are almost two feet apart. But it is plain that this arrangement allows him to remain positioned over the frames in a rather natural position, so that when he picks the light pen out of the air (which is its rest position, thanks to a system of jointed supporting arms and a controlled tension and rewind system for the attached cord) his hand is still on the way from the keyset to the display frame. When he is through with the pen at the display frame, he lets go of it, the cord rewinds, and the pen is again in position. There is thus a minimum of effort, movement, and time involved in turning to work on the frame. That is, he could easily shift back and forth from using keyset to using light pen, with either hand (one pen is positioned for each hand), without moving his head, turning, or leaning.3b3d

A good deal of Joe's time, though, seems to be spent with one hand on a keyset and the other using a light pen on the display surface. It is in this type of working mode that the images on the display frames changed most dynamically. You receive another real surprise as you realize how much activity there is on the face of these display tubes. You ask yourself why you weren't prepared for this, and you are forced to admit that the generalizations you had heard hadn't really sunk in--"new methods for manipulating symbols" had been an oft-repeated term, but it just hadn't included for you the images of the free and rapid way in which Joe could make changes in the display, and of meaningful and flexible "shaping" of ideas and work status which could take place so rapidly.3b3e

Then you realized that you couldn't make any sense at all out of the specific things he was doing, nor of the major part of what you saw on the displays. You could recognize many words, but there were a good number that were obviously special abbreviations of some sort. During the times when a given image or portion of an image remained un changed long enough for you to study it a bit, you rarely saw anything that looked llke a sentence as you were used to seeing one. You were beginning to gather that there were other symbols mixed with the words that might be part of a sentence, and that the different parts of what made a full-thought statement (your feeling about what a sentence is) were not just laid out end to end as you expected. But Joe suddenly cleared the displays and turned to you with a grin that signalled the end of the passive observation period, and also that somehow told you that he knew very well that you now knew that you had needed such a period to shake out some of your limited images and to really realize that a "capability hierarchy" was a rich and vital thing.3b3f

"I guess you noticed that I was using unfamiliar notions, symbols, and processes to go about doing things that were even more unfamiliar to you?" You made a non-committal nod--you saw no reason to admit to him that you hadn't even been able to tell which of the things he had been doing were to cooperate with which other things--and he continued. "To give you a feel for what goes on, I'm going to start discussing and demonstrating some of the very basic operations and notions I've been using. You've read the stuff about process and process-capability hierarchies, I'm sure. I know from past experience in explaining radical augmentation systems to people that the new and powerful higher-level capabilities that they are interested in--because basically those are what we are all anxious to improve--can't really be explained to them without first giving them some understanding of the new and powerful capabilities upon which they are built. This holds true right on down the line to the type of low-level capability that is new and different to them all right, but that they just wouldn't ordinarily see as being 'powerful.' And yet our systems wouldn't be anywhere near as powerful without them, and a person's comprehension of the system would be rather shallow if he didn't have some understanding of these basic capabilities and of the hierarchical structure built up from them to provide the highest-level capabilities."3b3g

"For explanation purposes here, let's say that the lowest level at which the computer system comes into direct play in my capability hierarchy is in the task of what I'll call 'single-frame composition.' We'll stick to working with prose text in our examples--most people can grasp easily enough what we are doing there without having to have special backgrounds in mathematics or science as they would to gain equal comprehension for some of the similar sorts of things we do with diagrams and mathematical equations. This low-level composition task is just what you normally do with a pen or pencil or typewriter on a piece of paper-- that is, assemble a bunch of symbols before your eyes in order to portray something which you have in mind."3b4a

You listened and watched as Joe showed you some of the different ways in which the composition of straightforward text was made easier for him in this system. With either hand, Joe could "type" (the keysets didn't look at all like typewriter keyboards) individual letters and numbers, and if he had directed it to do so, the computer would put each successive symbol next to its predecessor just as a typewriter does--only here there was completely automatic "carriage return" service. This didn't impress you very much, since an automatic carriage-return feature was sort of a trivial return on the investment behind all of this equipment--but then you reflected that, as long as the computer was there anyway, to help do all the flashy things you had witnessed earlier, one might as well use it in all of the little helpful ways he could.3b4b

But there were other ways in which help was derived for this composition task. He showed you how he could call up the dictionary definition to any word he had typed in, with but a few quick flicks on the keyset. Synonyms or antonyms could just as easily be brought forth. This also seemed sort of trivially obvious, and Joe seemed to know that you would feel so. "It turns out that this simple capability makes it feasible to do some pretty rough tasks in the upper levels of the capability hierarchy--where precise use of special terms really pays off, where the human just couldn't be that precise by depending upon his unaided memory for definitions and 'standards,' and where using dictionary and reference-book lookup in the normal fashion would be so distracting and time-consuming that the task execution would break down. We've tried taking this feature away in some of these processes up there, and believe me, the result was a mess.3b4c

You could get some dim feeling for what he meant, having watched him working for a while, but you were nevertheless much relieved to find the next thing he showed you to be more directly impressive. He showed you how he could single out a group of words (called the "object symbol string," or simply "object string") and define an abbreviation term, composed of any string of symbols he might choose, that became associated with the object string in computer storage. At any later time (until he chose to discard that particular abbreviation from his working voca bulary) the typing of the abbreviation term would call forth automatically the "printing" on the display of the entire object string. Joe showed you another way in which this abbreviation feature might work. He "arranged" for the computer to print the abbreviation on the display, just the way he typed it in. At a subsequent reading, if he had for gotten what the abbreviation stood for, he could call for substitution of the full object string to refresh his memory.3b4d

Then he showed you how this sort of facility had been extended, in a refined way, to provide a rather powerful sort of shorthand. He could hit a great many combinations of keys on his keyset--i.e., any one stroke of his hand could depress a number of keys, which gave him over a thousand unique single-stroke signals to the computer with either hand. Some of these signals were used as abbreviations for entire words. It seems that, for instance, the 150 most commonly used words in a natural language made up about half of any normal text in that language. Joe said that it was thus quite feasible to learn and use the single-stroke abbreviations for about half of the words he used, but beyond that each added percent began to require him to have too many abbreviations under his command. But he said that there were a lot of word endings, letter pairs (diagrams), and letter triplets (trigrams) that were so common as to make it pay to abbreviate them to a single stroke. A whole word so abbreviated saved typing all the letters as well as the spaces at either side of the word, and a word-ending abbreviated by a single stroke saved typing the letters and the end-of-word space. He claimed that he could comfortably rattle off about 180 words a minute--faster than he could comfortably talk. You believed him after he transcribed your talking for a minute or so, and it gave you an eerie feeling to see the near instantaneous appearance of your words and sentences in neat printed form.3b4e

Joe said that there were other miscellaneous simple features, and some quite sophisticated features to help the composition process. He made some brief references to statistical predictions that the com puter could make regarding what you were golng to type next, and that if you got reasonably skillful you could "steer through the extrapolated prediction field" as you entered your information and often save energy and time. You gathered that he thought you would saturate about there on this particular subject, because he went on to the next.3b4f

"Even if I couldn't actually specify new symbols here any faster than with a typewriter, the extreme flexibility that this computer system provides for making changes in what is presented on the display screen would make me very much more effective in creating finished text than I could ever be on a typewriter." With this statement, Joe proceeded to show you what he meant. The frame full of your transcribed speech was still showing, and it represented the clumsy phrasing and illogical progression of thought so typical of extemporaneous speech. Joe took the light gun in his right hand, and with a deft flick of it, coordinated with a stroke of his left hand on its keyset, caused the silent and instantaneous deletion of a superfluous word. The word disappeared from the frame, and the rest of the text simultaneously readjusted to present the neat, no-gap, full-line appearance it had had. 3b5a

With but slightly more motion of his light pen, he could similarly delete any string of words or letters. He demonstrated this by cutting out what I thought to be some relevant prose, and then he showed how the system allowed for second thoughts about such human-directed processes--those words were automatically saved for a brief period in case he wanted to call them back. Leaving his light pen pointed at the space where a deleted symbol string used to be, Joe could reinstate it instantaneously with one stroke of his left hand.3b5b

Adding one more light-pen pointing to what it took to delete an arbitrary string of symbols, Joe could direct the computer to move that string from where it was to insert it at a new point which his light pen designated. Again it would disappear instantaneously from where it had been, but now the modified display would show the old text to have been spread apart just enough at the indicated point to hold this string. The text would all still look as neat as if freshly retyped. With similar types of keyset and light-pen operations, Joe could change paragraph break points, transpose two arbitrary symbol strings (words, sentences, paragraphs, etc., or fragments thereof), readjust margins of arbitrary sections of text--essentially being able to affect immediately any of the changes that a proofreader might want to designate with his special marks, only here the proofreader is always looking at clean text as if it had been instantaneously retyped after each designation had been made.3b5c

Joe also demonstrated how he could request that each instance of the use of a given term be changed to a newly designated term, and this would again be instantaneously accomplished. Also, he could arbitrarily set the margins between which any section of text must appear, and its line lengths and number of lines would automatically be adjusted. He showed how this was useful in displaying parallel or counter arguments-- although he said that actual use of this feature was a bit more sophisticated--by squeezing each into half width and putting them side by side (with a vertical line suddenly separating them). One of the sections of text was about a third longer than the other--but two quick strokes with Joe's left hand caused the computer to adjust the display automatically. The middle separator line was moved toward the shorter piece of text, and the line lengths of the two sections were adjusted so that they occupied the same length along the dlsplay frame. Yes, you were beginning to get a feel for what the expression "flexible new methods for manipulating symbol structures" might really imply, at least on this basic-capability level.3b5d

"If we want to go on to a higher-level capability to give you a feeling for how our rebuilt capability hierarchy works, it will speed us along to look at how we might organize these more primitive capabilities which I have demonstrated into some new and better ways to set up what we can call an 'argument.' This refers loosely to any set of statements (we'll call them 'product statements') that represents the product of a period of work toward a given objective. Confused? Well, take the simple case where an argument leads to a single product statement. For instance, you come to a particular point in your work where you have to decide what to do for the next step. You go through some reasoning process--usually involving statements--and come up with a statement specifying that next step. That final statement is the product statement, and it represents the product of the argument or reasoning process which led to it.3b6a

"You usually think of an argument as a serial sequence of steps of reason, beginning with known facts, assumptions, etc., and progressing toward a conclusion. Well, we do have to think through these steps serially, and we usually do list the steps serially when we write them out because that is pretty much the way our papers and books have to present them--they are pretty limiting in the symbol structuring they enable us to use. Have you even seen a 'scrambled-text' programmed instruction book? That is an interesting example of a deviation from straight serial presentation of steps.3b6b

"Conceptually speaking, however, an argument is not a serial affair. It is sequential, I grant you, because some statements have to follow others, but this doesn't imply that its nature is necessarily serial. We usually string Statement B after Statement A, with Statements C, D, E, F, and so on following in that order--this is a serial structuring of our symbols. Perhaps each statement logically followed from all those which preceded it on the serial list, and if so, then the conceptual structuring would also be serial in nature, and it would be nicely matched for us by the symbol structuring.3b6c

"But a more typical case might find A to be an independent statement, B dependent upon A, C and D independent, E depending upon D and B, E dependent upon C, and F dependent upon A, D, and E. See, sequential but not serial? A conceptual network but not a conceptual chain. The old paper and pencil methods of manipulating symbols just weren't very adaptable to making and using symbol structures to match the ways we make and use conceptual structures. With the new symbol-manipulating methods here, we have terrific flexibility for matching the two, and boy, it really pays off in the way you can tie into your work.3b6d

This makes you recall dimly the generalizations you had heard previously about process structuring limiting symbol structuring, symbol structuring limiting concept structuring, and concept structuring limiting mental structuring. You nod cautiously, in hopes that he will proceed in some way that will tie this kind of talk to something from which you can get the "feel" of what it is all about. As it turns out, that is just what he intends to do.3b6e

"Let's actually work some examples. You help me." And you become involved in a truly fascinating game. Joe tells you that you are to develop an argument leading to statements summarizing the augmentation means so far revealed to you for doing the kind of straight-text work usually done with a pencil and eraser on a single sheet of paper. You unconsciously look for a scratch pad before you realize that he is telling you that you are going to do this the "augmented way" by using him and his system--with artful coaching from him. Under a bit of urging from him, you begin self-consciously to mumble some inane statements about what you have seen, what they imply, what your doubts and reservations are, etc. He mercilessly ignores your obvious discomfort and gives you no cue to stop, until he drops his hands to his lap after he has filled five frames with these statements (the surplus filled frames disappeared to somewhere--you assume Joe knows where they went and how to get them back).3b6f

"You notice how you wandered down different short paths, and criss-crossed yourself a few times?" You nod--depressed, no defense. But he isn't needling you. "Very natural development, just the way we humans always seem to start out on a task for which we aren't all primed with knowledge, method, experience, and confidence--which is to include essentially every problem of any consequence to us. So let's see how we can accommodate the human's way of developing his comprehension and his final problem solution.3b6g

"Perhaps I should have stopped sooner--I am supposed to be coaching you instead of teasing you--but I had a reason. You haven't been making use of the simple symbol-manipulation means that I showed you--other than the shorthand for getting the stuff on the screens. You started out pretty much the way you might with your typewriter or pencil. I'll show you how you could have been doing otherwise, but I want you to notice first how hard it is for a person to realize how really unques tioning he is about the way he does things. Somehow we implicitly view most all of our methods as just sort of 'the way things are done, that's all.' You knew that some exotic techniques were going to be applied, and you'll have to admit that you were passively waiting for them to be handed to you."3b6h

With a non-committal nod, you suggest getting on with it. Joe begins, "You're probably waiting for something impressive. What I'm trying to prime you for, though, is the realization that the impressive new tricks all are based upon lots of changes in the little things you do. This computerized system is used over and over and over again to help me do little things--where my methods and ways of handling little things are changed until, lo, they've added up and suddenly I can do impressive new things."3b6i

You don't know. He's a nice enough guy, but he sure gets preachy. But the good side of your character shows through, and you realize that everything so far has been about little things--this is probably an important point. You'll stick with him. Okay, so what could you have been doing to use the simple tricks he had shown you in a useful way? Joe picks up the light pen, poises his other hand over the keyset, and looks at you. You didn't need the hint, but thanks anyway, and let's start rearranging and cleaning up the work space instead of just dumping more raw material on it.3b6j

With closer coaching now from Joe, you start through the list of statements you've made and begin to edit, re-word, compile, and delete. It's fun--"put that sentence back up here between these two"--and blink, it's done. "Group these four statements, indented two spaces, under the heading "shorthand," and blinko, it's done. "Insert what I say next there, after that sentence." You dictate a sentence to extend a thought that is developing, and Joe effortlessly converts it into an inserted new sentence. Your ideas begin to take shape, and you can continually re-work the existing set of statements to keep representing the state of your "concept structure."3b6k

You are quite elated by this freedom to juggle the record of your thoughts, and by the way this freedom allows you to work them into shape. You reflected that this flexible cut-and-try process really did appear to match the way you seemed to develop your thoughts. Golly, you could be writing math expressions, ad copy, or a poem, with the same type of benefit. You were ready to tell Joe that now you saw what he had been trying to tell you about matching symbol structuring to concept structuring--when he moved on to show you a succession of other tech niques that made you realize you hadn't yet gotten the full significance of his pitch.3b6l

So far the structure that you have built with your symbols looks just like what you might build with pencil-and-paper techniques-- only here the building is so much easier when you can trim, extend, insert, and rearrange so freely and rapidly. But the same computer here that gives us these freedoms with so trivial an application of its power, can just as easily give us other simple capabilities which we can apply to the development and use of different types of structure from what we used to use. But let me unfold these little computer tricks as we come to them.3b6m

"When you look at a given statement in the middle of your argument structure, there are a number of things you want to know. Let's simplify the sltuation by saying that you might ask three questions, 'What's this?', 'How come?', and 'So what?' Let's take these questions one at a time and see how some changes in structuring might help a per son answer them better.3b6n

You look at a statement and you want to understand its meaning. You are used to seeing a statement portrayed in just the manner you might hear it--as a serial succession of words. But, just as with the statements within an argument, the conceptual relationship among the words of a sentence is not generally serial, and we can benefit in matching better to the conceptual structure if we can conveniently work with cer tain non-serial symbol-structuring forms within sentences.3b6o

"Most of the structuring forms I'll show you stem from the simple capability of being able to establish arbitrary linkages between different substructures, and of directing the computer subsequently to display a set of linked substructures with any relative positioning we might designate among the different substructures. You can designate as many different kinds of links as you wish, so that you can specify different display or manipulative treatment for the different types."3b6p

Joe picked out one of your sentences, and pushed the rest of the text a few lines up and down from it to isolate it. He then showed you how he could make a few strokes on the keyset to designate the type of link he wanted established, and pick the two symbol structures that were to be linked by means of the light pen. He said that most links possessed a direction, i.e., they were like an arrow pointing from one substructure to another, so that in setting up a link he must specify the two substructures in a given order.3b6q

He went to work for a moment, rapidly setting up links within your sentence. Then he showed you how you could get some help in looking at a statement and understanding it. "Here is one standard portrayal, for which I have established a computer process to do the structuring automatically on the basis of the interword links." A few strokes on the keyset and suddenly the sentence fell to pieces--different parts of it being positioned here and there, with some lines connecting them. "Remember diagramming sentences when you were studying grammar? Some good methods, plus a bit of practice, and you'd be surprised how much a diagrammatic breakdown can help you to scan a complex statement and untangle it quickly.3b6r

"We have developed quite a few more little schemes to help at the statement level. I don't want to tangle you up with too much detail, though. You can see, probably, that quick dictionary-lookup helps." He aimed at a term with the light pen and hit a few strokes on the keyset, and the old text jumped farther out of the way and the definition appeared above the diagram, with the defined term brighter than the rest of the diagram. And he showed you also how you could link secondary phrases (or sentences) to parts of the statement for more detailed description. These secondary substructures wouldn't appear when you normally viewed the statement, but could be brought in by simple request if you wanted closer study.3b6s

"It proves to be terrifically useful to be able to work easily with statements that represent more sophisticated and complex concepts. Sort of like being able to use structural members that are lighter and stronger--it gives you new freedom in building structures. But let's move on--we'll come back to this area later, if we have time.3b6t

"When you look at a statement and ask, 'How come?', you are used to scanning back over a serial array of previously made statements in search of an understanding of the basis upon which this statement was made. But some of these previous statements are much more significant than others to this search for understanding. Let us use what we call 'antecedent links' to point to these, and I'll give you a basic idea of how we structure an argument so that we can quickly track down the essential basis upon which a given statement rests."3b6u

You helped him pick out the primary antecedents of the statement you had been studying, and he established links to them. These statements were scattered back through the serial list of statements that you had assembled, and Joe showed you how you could either brighten or underline them to make them stand out to your eye--just by requesting the computer to do this for all direct antecedents of the designated statement. He told you, though, that you soon get so you aren't very much interested in seeing the serial listing of all of the statements, and he made another request of the computer (via the keyset) that eliminated all the prior statements, except the direct antecedents, from the screen. The subject statement went to the bottom of the frame, and the antecedent statements were neatly listed above it.3b6v

Joe then had you designate an order of "importance to comprehension" among these statements, and he rearranged them accordingly as fast as you could choose them. (This choosing was remarkably helped by having only the remainder statements to study for each new choice--another little contribution to effectiveness, you thought.) He mentioned that you could designate orderings under several different criteria, and later have the display show whichever ordering you wished. This, he implied, could be used very effectively when you were building or studying an argument structure in which from time to time you wanted to strengthen your comprehension relative to different aspects of the situation.3b6w

"Each primary antecedent can similarly be linked to its primary antecedents, and so on, until you arrive at the statements representing the premises, the accepted facts, and the objectives upon which this argument had been established. When we had established the antecedent links for all the statements in the argument, the question 'So what?' that you might ask when looking at a given statement would be answered by looking for the statements for which the given statement was an antecedent. We already have links to these consequents--just turn around the arrows on the antecedent links and we have consequent links. So we can easily call forth an uncluttered display of consequent statements to help us see why we needed this given statement in the argument.3b6x

"To help us get better comprehension of the structure of an argument, we can also call forth a schematic or graphical display. Once the antecedent-consequent links have been established, the computer can automatically construct such a display for us." So, Joe spent a few minutes (with your help) establishing a reasonable set of links among the statements you had originally listed. Then another keyed-in request to the computer, and almost instantaneously there appeared a network of lines and dots that looked something like a tree--except that sometimes branches would fuse together. "Each node or dot represents one of the statements of your argument, and the lines are antecedent-consequent links. The antecedents of one statement always lie above that statement-- or rather, their nodes lie above its node. When you get used to using a network representation like this, it really becomes a great help in getting the feel for the way all the different ideas and reasoning fit together-- that is, for the conceptual structuring."3b6y

Joe demonstrated some ways in which you could make use of the diagram to study the argument structure. Point to any node, give a couple of strokes on the keyset, and the corresponding statement would appear on the other screen--and that node would become brighter. Call the antecedents forth on the second screen, and select one of special interest--deleting the others. Follow back down the antecedent trail a little further, using one screen to look at the detail at any time, and the other to show you the larger view, with automatic node-brightening indication of where these detailed items fit in the larger view.3b6z

"For a little embellishment here, and to show off another little capability in my repertoire, let me label the nodes so that you can develop more association between the nodes and the statements in the argument. I can do this several ways. For one thing, I can tell the computer to number the statements in the order in which you originally had them listed, and have the labelling done automatically." This took him a total of five strokes on the keyset, and suddenly each node was made into a circle with a number in it. The statements that were on the second screen now each had its respective serial number sitting next to it in the left margin. "This helps you remember what the different nodes on the network display contain. We have also evolved some handy techniques for constructing abbreviation labels that help your memory quite a bit.3b6aa

"Also, we can display extra fine-structure and labelling detail within the network in the specific local area we happen to be concentrating upon. This finer detail is washed out as we move to another spot with our close attention, and the coarser remaining structure is compressed, so that there is room for our new spot to be blown up. It is a lot like using zones of variable magnification as you scan the structure--higher magnification where you are inspecting detail, lower magnification in the surrounding field so that your feel for the whole structure and where you are in it can stay with you."3b6ab

"If you are tangling with a problem of any size--whether it involves you for half an hour or two years--the entire collection of statements, sketches, computations, literature sources, and source extracts that is associated with your work would in our minds constitute a single symbol structure. There may be many levels of substructuring between the level of individual symbols and that represented by the entire collection. You and I have been worxing with some of the lower-ordered substructures--the individual statements and the multistatement argu ments--and have skimmed through some of the ways to build and manipulate them. The results of small arguments are usually integrated in a higher level network of argument or concept development, and these into still higher-level networks, and so on. But at any such level, the manner in which the interrelationship between the kernels of argument can be tagged, portrayed, studied and manipulated is much the same as those which we have just been through.3b7a

"Substructures that might represent mathematical or formal-logic arguments may be linked right in with substructures composed of the more informal statements. Substructures that represent graphs, curves, engineering drawings, and other graphical forms can likewise be integrated. One can also append special substructures, of any size, to particular other substructures. A frequent use of this is to append descriptive material--something like footnotes, only much more flexible. Or, special messages can be hung on that offer ideas such as simplifying an argument or circumventing a blocked path--to be uncovered and considered at some later date. These different appended substructures can remain invisible to the worker until such time as he wants to flush them into view. He can ask for the cue symbols that indicate their presence (identifying where they are linked and what their respective types are) to be shown on the network display any time he wishes, and then call up whichever of them he wishes If he is interested in only one type of appended substructure, he can request that only the cues associated with that type be displayed.3b7b

"You should also realize that a substructure doesn't have to be a hunk of data sitting neatly distinct within the normal form of the larger structure. One can choose from a symbol structure (or substructure, generally) any arbitrary collection of its substructures, designate any arbitrary structuring among these and any new substructures he wants to add, and thus define a new substructure which the computer can untangle from the larger structure and present to him at any time. The associative trails that Bush suggested represent a primitive example of this. A good deal of this type of activity is involved during the early, shifting development of some phase of work, as you saw when you were collecting tentative argument chains. But here again, we find ever more delightful ways to make use of the straightforward-seeming capabilities in developing new higher-level capabilities--which, of course, seem sort of straight forward by then, too.3b7c

"I found, when I learned to work with the structures and manipulation processes such as we have outlined, that I got rather impatient if I had to go back to dealing with the serial-statement structuring in books and journals, or other ordinary means of communicating with other workers. It is rather like having to project three-dimensional images onto two-dimensional frames and to work with them there instead of in their natural form. Actually, it is much closer to the truth to say that it is like trying to project n-dimensional forms (the concept structures, which we have seen can be related with many many nonintersecting links) onto a one-dimensional form (the serial string of symbols), where the human memory and visualization has to hold and picture the links and relationships. I guess that's a natural feeling, though. One gets impatlent any tlme he is forced into a restrlcted or primitive mode of operation--except perhaps for recreatlonal purposes.3b7d

"I'm sure that you've had the experience of working over a journal article to get comprehension and perhaps some special-purpose conclusions that you can integrate into your own work. Well, when you ever get handy at roaming over the type of symbol structure which we have been showing here, and you turn for this purpose to another person's work that is structured in this way, you will find a terrific difference there in the ease of gaining comprehension as to what he has done and why he has done it, and of isolating what you want to use and making sure of the conditions under which you can use it. This is true even if you find his structure left in the condition in which he has been working on it--that is, with no special provisions for helping an outsider find his way around. But we have learned quite a few simple tricks for leaving appended road signs, supplementary information, questions, and auxiliary links on our working structures--in such a manner that they never get in our way as we work--so that the visitor to our structure can gain his comprehension and isolate what he wants in marvelously short order. Some of these techniques are quite closely related to those used in automated-instruction programming--perhaps you know about 'teaching machines?'3b7e

"What we found ourselves doing, when having to do any extensive digesting of journal articles, was to type large batches of the text verbatim into computer store. It is so nice to be able to tear it apart, establish our own definitions and substitute, restructure, append notes, and so forth, in pursuit of comprehension, that it was generally well worth the trouble. The keyset shorthand made this reasonably practical. But the project now has an optical character reader that will convert our external references into machine code for us. The references are available for study in original serial form on our screens, but any structuring and tagging done by a previous reader, or ourselves, can also be utilized.3b7f

"A number of us here are using the augmented systems for our project research, and we find that after a few passes through a reference, we very rarely go back to it in its original form. It sits in the archives like an orange rind, with most of the real juice squeezed out. The contributions from these references form sturdy members of our structure, and are duly tagged as to source so that acknowledgment is always implicitly noted. The analysis and digestion that any of us makes on such a reference is fully available to the others. It is rather amazing how much superfluous verbiage is contained in those papers merely to try to make up for the pitifully sparse possibilities available for symbol structuring in printed text."3b7g

There was a slight pause while Joe apparently was reflecting upon something. He started to speak, thought differently of it, and turned to flash something on a screen. You looked quickly, anticipating that now you would comprehend. Well, more of the display looked meaningful to you than when you had first watched him going about his work, but you realized that you were still a bit uneducated. I've developed a sequence for presenting the different basic features of our augmentation system that seems to work pretty well, and I just wanted to be sure I was still following it reasonably closely."3b8a

He noticed you wrinkle your face as you looked at the display. "It's time to shift the topic a bit, and some of the things on the screen that are probably puzzling you can make a starting point for a new discussion phase. See, when I outlined a delivery for giving a feel for these techniques to the uninitiated, I could have sketched out the subject matter in a skeletal argument structure. From what we've been through so far, you might expect it to be like that. What I did, though, was to treat the matter as a process that I was going to execute the process of giving you a lecture demonstration. It is a rather trivial exercise of the techniques we have for developing and manipulating processes, but anyway that's the form I chose for making the notes.3b8b

"A process is something that is designed, built, and used--as is any tool. In the general sense in which we consider processes to be a part of our augmentatlon system, it is absolutely necessary that there be effective capability for designing and building processes as well as for using them. For one thing, the laying out of objectives and a method of approach for a problem represent a form of process design and building, to our way of looking at it. And an independent problem solver certainly has to have this capability. Indeed, we find that designing and coordinating one's sequence of steps, in high levels or in low levels of such process structuring, is an extremely important part of the total activity.3b8c

"One of our research guys in the early phases of our augmentation development was considered (then) to be a bug on this topic. He maintained that about ten percent of the little steps we took all day accounted for ninety percent of the progress toward the goals we claimed to pursue--that is, that ninety percent of our actions and thoughts were coupled to our net progress in only a very feeble way. Well, we can't analyze the old ways of doing things very accurately to check his estimated figures, but we certainly have come to be in general sympathy with his stand. We have developed quite a few concepts and methods for using the computer system to help us plan and supervise sophisticated courses of action, to monitor and evaluate what we do, and to use this information as direct feedback for modifying our planning techniques in the future.3b8d

"There are, of course, the explicit computer processes which we use, and which our philosophy requires the augmented man to be able to design and build for himself. A number of people, outside our research group here, maintain stoutly that a practical augmentation system should not require the human to have to do any computer programming--they feel that this is too specialized a capability to burden people with. Well, what that means in our eyes, if translated to a home workshop, would be like saying that you can't require the operating human to know how to adjust his tools, or set up jigs, or change drill sizes, and the like. You can see there that these skllls are easy to learn in the context of what the human has to learn anyway about using the tools, and that they provide for much greater flexibility in finding convenient ways to use the tools to help shape materials.3b8e

"It won't take too much time to give you a feel for the helpful methods we have for working on computer-process structures -- or programs -- because there is quite a bit of similarity in concept to what you have seen in the symbol-structuring techniques. No matter what language you use -- whether machine language, list language, or ALGOL, for instance--you build up the required process structure by organizing statements in that language. Each statement specifies a given process to your computer. Well, you have already seen how you can get help in developing precise and powerful statements, or in gaining quick comprehension of state ments, by charting or diagramming them and using special links between the different parts. "Look here.'" And he went after what he said was a typical process structure, to give you an example of what he was talking about. In several brief, successive frame displays, before he got to the one he wanted, you got glimpses of network schematics that reminded you of those used in symbol structuring. But, what he finally had on the display frame was quite different from the argument statements you had seen.3b8f

"In explaining symbol-structuring to you, I used the likely questions, 'What's this?' 'How come?' and 'So what?' to point out the usefulness of some of our structuring methods. Here, in process structuring, corresponding questions about a statement might be: 'What does it say to do?' 'What effect will that have?' and 'Why do we want that done?' Let's take a quick look at some of the ways you can get help in answering them.3b8g

"The language used to compose these process-description stateme for the computer is considerably more compact and precise than is a natural language, such as English, and there is correspondingly less advantage to be gained by appending special links and tags for giving us humans a better grasp of their meaning. However, as you see in this left-hand section of the statement portrayal, geometrical grouping, linking, and positioning of the statement components are used in the blown-up statement display. But this portrayal doesn't stem from special appended information, it can be laid out like this automatically by the computer, just from the cues it gets from the necessary symbol components of the statement. The different significant relationships are more perceptible to a human in this way of laying it out, and an experienced human thus gets quite a bit of help in answering the first question: 'What does it say to do?'3b8h

For the second question, relative to what effect the specified action will have, some of these symbols to the right give you a quick story about the very detailed and immediate effect on the state of the symbol structure which this process structure is manipulating. Other symbols here provide keys which a light-pen selection can activate to bring to you displays of that symbol structure, usually a choice of several relevant views at different levels of the structure. Then I can use the keyset to ask for the preceding statement, if I'm a little puzzled about the detailed manipulation--or, I can request a specific higher-level view of the process structure by light-pen selection on one of these remaining symbols here.3b8i

So saying, Joe selected one of these symbols with his pen, and a new and different display popped into view. "This is the next level up in the process structure. It consists of lists of compactly abbreviated statements, and some condensed notes about their effects. If we want, we can blow up one at a time as we study over the list. In this context, one can get some answer to the larger picture of what effect will a given statement have, and also some answer to the question about why we want a given effect produced. But this is a sort of a holdover from old pro gramming habits, and most of us nowadays are making considerably more use of the schematic techniques that evolved out of the program flow-charting techniques and out of our symbol-structuring techniques.3b8j

"I know that you have less prevlous familiarity with the nature of programs than you do wlth the nature of arguments, so I'll just give you a few quick views of what these process-structure schematic portrayals look like, and not try to explain them in any detail. He flashed a few on the screen, and indicated how some of the different features could give the human a quick appreclation of how different component processes were cooperating to produce a more sophisticated process. You could appreciate some of the tricks of linking in explanatory and descriptive substructure and the general means of using all the different symbol-structuring tricks for representing to the human the considerations, critical features, and lnterdependencies involved in the process structure.3b8k

"Most of this portrayal technique actually represents special structuring of what we previously defined in a loose way as arguments. The human who wants to approach an established process structure in order to modify it, needs to gain comprehension of the relevant features both of the functioning and of the design of the structure. You saw how this could be facilitated by our symbol-structuring techniques. And if he is building a new process structure or changing an existing one, he needs to structure the argument or reasoning behind the design. We have developed a number of special symbol-structuring techniques that allow us to match especially well to the concepts involved in designing processes.3b8l

"But there is a very significant feature involved in this particular type of process structuring that I should tell you about. It is based upon the fact that the process-description language for the computer is formal and precise. Because of this fact, we can establish explicit rules for treating statements in this language, and for treating symbol structures composed of these statements, such that computer pro cesses based upon these rules can be said to extract meaning from these statements and to do operations based upon this meaning. The result is that the computer is able to find answers to a much wider range of questions about a specified process structure than it could if only the structural characteristics were discernible to it.3b8m

"In our studying and designing process structures, we have found many ways to capitalize upon this more sophisticated question-answering capability now possessed by the computer. We are learning, for instance, how to get the computer to decide whether or not some types of design specificatlons are met, and if not, where the limitation exists. Or, perhaps we approach an already designed process structure which we think we can modify, or from which we can extract some useful sub-process that we contemplate incorporating into another process we are designing. We are getting terrific help in this type of instance, since we can now ask the computer direct questions about types of capability and limitation in this structure. The computer can even lead us directly to the particular design features from which these capabilities or limitations stem, and it is simple then to examine the descriptive and explanatory arguments linked thereto in order to see why these features were designed into the structure.3b8n

"But I don't want to spend a disproportionate amount of time on the computer processes. The augmented man is engaged more often in structuring what we call composite processes than he is in structuring computer processes. For instance, planning a research project, or a day's work, are examples of structuring composite processes. A composite process, remember, is organized from both human processes and computer processes--which includes, of course, the possible inclusion of lower-order composite processes. The structuring here differs from that of a computer process mainly in the sophistication of the sub-processes which can be specified for the human to do. Some of these specifications have to be given in a language which matches the human's rich working framework of concepts--and we have been demonstrating here with English for that purpose--but quite a few human-executed processes can be specified in the high-level computer-processing language even though we don't know how to describe them in that language. This means that there few composite-process structures about which the computer can answer very useful questions for us.3b8o

"But to be more specific--we find that setting up objectives, deslgning a method of approach, and then implementing that method are of course our fundamental operating sequence--done over and over again in the many levels of our activity. We mentioned above what the characteristic structural difference was between computer processes and composite processes. But perhaps more important to us is the difference in the way we work with composite-process structures. Here is a crude but succinct way to put this. With the human contributing to a process, we find more and more as the process becomes complex that the value of the human's contribution depends upon how much freedom he is given to be disorderly in his course of action. For instance, we provide him as much help as possible in making a plan of action. Then we give him as much help as we can in carrying it out. But we also have to allow him to change his mind at almost any point, and to want to modify his plans. So, we provide augmentation help to him for keeping track of his plans where he is in them, what has been happening in carrying them out to date--and for evaluating possibilities that might occur to him for changing the plans. In fact, we are even learning how the computer can be made to watch for some kinds of plan-change possibilities, and to point them out to the human when they arise.3b8p

"Here's a simple example of this sort of help for the human. Last winter, we designed a computer process that can automatically monitor the occurrence of specified types of computer usage over a specified period of time, and which, from the resulting data, can deduce a surprising amount of information regarding how the human made use of that time. This was quite helpful to us for evaluating our ways of doing things. Then we added more features to the program, in which the computer occasionally interrupts the human's activity and displays some questions to be answered. From these answers, together with its normal monitoring data, the program can provide evaluative data regarding the relative success of his different work methods. Our augmentation researchers became intrigued by this angle and bore down a little on it. They came up with a package process which gives the human many different types of feedback about his progress and way of doing things. Now, as part of my regular practice, I spend about five minutes out of each hour exercising with this package. This almost always reveals things to me that change at least the slant of my approach during the next hour, and often stimulates a relatlvely significant change in my short-range plans3b8q

"You appreciate, of course, that I accomplish many more meaningful steps in an hour now than I used to, or than would be your norm now. This once-an-hour review for me now might compare with a once-a-day review for you, as far as the distance travelled between reviews is concerned.3b8r

"Our way of structuring the statement of our objectives, the arguments which lead to the design of our plans, and the working statements of our plans, has been influenced by this review process. We found special types of tags and descriptive codes which we could append to these respective planning structures as we developed them which later facilitated our man-computer cooperative review of them. Also, our methods of developing these structures have evolved to facilitate their later modification. For instance, every basic consideration upon which a given planning statement is based is linked to that statement as a matter of standard argument structuring. But we have taken to linking special tagging codes into these argument structures involving our planning, to identify for the computer some of the different types of dependency relationships in the antecedent linkages. Later, if we consider changing the plan, these special tags often enable us to make use of some special computer processes that automatically isolate the considerations relevant to a particular type of change we have in mind.3b8s

"Maybe an example will help here. There is a plan I am currently using for the way I go about entering miscellaneous scraps of infcrmation lnto my total symbol structure. It is designed so that there will be a good chance for these scraps later to be usefully integrated. It turns out that this plan is closely coupled in its design argument to the general plan for reviewing process structures--and symbol-structures, too, for that matter. Recently, I got an idea as to how I might add a little feature to that process that specially suited my particular way of wanting to deal with miscellaneous thoughts that I get. By various means, I very quickly learned that this would be easy to do if I could but reverse the order ln whlch I execute the sub-process Steps A and B, when I enter a piece of lnformation. I had to find out if I could safely reverse their order without getting into trouble someplace in my system.3b8t

"This I could do relatively rapidly, by your standards, by snooping down the antecedent trails, looking for statements relevant to this timing question. There is, in fact, a semi-automatic processes available to me for speeding just such searches. The computer keeps track of where I have looked, where I've marked things as yes, or no, or possible, and does the bookkeeping and calculating necessary to guide me through an optimum search strategy. But the special tagging we do when we make a process structure lets this search be fully automatic when certain kinds of relationships are involved--and relative timing happens to be one of these relationships.3b8u

"So I phrased a question which essentially asked for considerations relevant to the order in which these two steps were executed, and turned the computer loose. It took about three seconds for the results to be forthcoming--you haven't yet seen me request a task that took a noticeable period of machine time, have you? But anyway, the computer discovered a relevance trail that ended up showing that reversing the order of Steps A and B during the information-scrap entry process would cripple a certain feature in the planning-review process, where miscellaneous thoughts and possibilities are gleaned from this store to be considered relative to the planning.3b8v

"But let's try to back away from details for a bit, now, and see if we can get a feeling for the significance of the things we've been talking about. Comparison with other working domains would be helpful, perhaps. If you were an inventor of useful mechanisms, you would like to have a wide range of materials-processing and shaping techniques available to you. This would give you more freedom and more interesting possibilities in the way you worked and designed. But many of these techniques are very specialized; they require special equipment, special skills to execute the processing and shaping, and special knowledge about applicability and possibillties for the techniques.3b8w

"Suppose you were told that you could subscribe to a community- owned installation of special equipment--containing all sorts of wonderful instruments tools and machines for measuring and processing with such as chemical, optical, mechanical, electronic, pneumatic, vacuum, metallurgy, and human factors. But this wasn't all that was included in the subscription. There would be a specialist assigned to you, instantly available for consultation and help whenever you requested it. He wouldn't have high-level theoretical trainlng. His specialty would be familiarity with the special manuals compiled from what the theoreticians, equipment builders, and technicians know, and being able to pinpoint relevant data and apply complex rules and specifications.3b8x

"A lot of questions you might ask he couldn't answer directly, but in such a case he could often lead you quickly to some relevant pages in his books. You discovered that usually a succession of well-chosen questions of the sort he could answer, interspersed with your occasional study of succinct and relevant material he'd dig up for you, could very rapidly develop answers to conceptually sophisticated questions. His help in your minute-by-minute designing work could be extremely valuable-- availing you of quick and realistic consideration of a great many new design possibilities.3b8y

"Similarly, when it came to carrying out a planned set of operations, it turned out that he couldn't carry out all of the processes for you--he could manage complex rules and procedures beautifully, but he would break down when it came to steps that required what you might call a larger view of the situation. But this wasn't so bad. The set of routine processes which he could manage all alone still provided you with a great deal of help--in fact, you got to developing ways to build things so as to capitalize upon his efficiency at these tasks. Then the processes which were too much for him would be done by the two of you together. He filled in all the routine stuff and you took care of the steps that were beyond his capability Often the steps you had to take care of were buried in the middle of a complex routine whose over-all nature didn't have to be understood by either of you for proper execution. Your helper would keep track of the complex procedure and execute all the steps he could. When he came to a step that was too big for him, he would hand you enough information to allow you to take that step, whereupon he would take over again until he met another such step.3b8z

"As an inventor and builder of devices that solve needs, you could become a great deal more versatile and productive, applying your imagination, intuition, judgment, and intelligence very effectively over a much wider range of possibilities. You could tackle much more complex and sophisticated projects, you could come up with very much better results--neater, cheaper, more reliable, more versatile, higher-quality performance--and you could work faster. Your effectiveness in this domain of activity would be considerably increased.3b8aa

"So let's turn back to the working domain which we are considering here. It is an intellectual one, where the processing and shaping done is of conceptual material rather than physical material. But between these two types of working domains we nonetheless find closely analogous conditions relative to the variety and sophistication of the processes and techniques applicable to what nonroutine workers do. Consider the intellectual domain of a creative problem solver, and listen to me rattle off the names of some specialized disciplines that come to mind. These esoteric disciplines could very possibly contribute specialized processes and techniques to a general worker in the intellectual domain: Formal logic--mathematics of many varieties, including statistics-- decision theory--game theory--time and motion analysis--operations research--classification theory--documentation theory--cost accounting, for time, energy, or money--dynamic programming--computer programming. These are only a few of the total, I'm sure.3b8ab

"This implies the range of potentially applicable processes. Realize that there is also a correspondingly large list of specialized materials potentially usable in the fabrications of the intellectual worker. I speak, of course, about the conceptual material in the many different flelds of human interest. The things that I have been de monstrating to you this afternoon were designed to increase significantlY the range of both processes and materials over which a human can practically operate within this intellectual domain. You might say that we do this by providing him with a very fast, agile vehicle, equipped with all sorts of high-performance sensory equipment and navigational aids, and carrying very flexible, powerful, semi-automatic devices for operating upon the materials of this domain. Not only that, but to provide an accurate analogy, we have to give him a computer to help him organize and monitor his activity and assess his results. We get direct help on many levels of activity in our system, you see.3b8ac

"But back to the topic of tools, and the analogy of the inventor who was given the equipment and the helper. Our augmented intellectual worker gets essentially this same kind of service, only more so--a compounding of this kind of service. Structuring our processes with care and precision enables the computer to answer limited questions, to guide you to relevant descriptions and specifications within its structure, to execute complex but limited-grasp processes on its own, and to take care of complex rule and procedure-following bookkeeping in guiding the execution of sophisticated composite processes. This actually makes it practical to use many specialized processes and techniques from very esoteric fields--to assess their applicability and limitations quickly, to incorporate them intelligently into the design and analysis of possible courses of action, and to execute them efficiently.3b8ad

"Our specialized processes represent a beautiful collection of special tools. These tools are designed by specialists, and they come equipped with operating instructions, trouble-shooting hints, and complete design data. Furthermore, we are provided with other tools that help us determine the applicability of these tools by automatically operating upon the instruction manual for us. Further, if something goes wrong with one of these tools, if we want to design a new tool of our own and make use of one of its modular components, or if we want to rearrange some of its adjustable features, we get considerable help in learning what we have to know about its design, and in making adjustments or coupling a part of it to another tool. Our shop contains an efficient tool-making section, where we can design and build our own tools from scratch, or by incorporating parts or all of any other tools we have.3b8ae

"Let me tell you of an interesting feature stemming from my using such improved Process-structuring techniques. An effective job of breaking down a complex problem into humanly manageable steps--and this is essentially what we seek in our process structuring--will provide the human with something to do at every turn. This may be to ponder or go searching, true enough--we aren't saying that the steps are necessarily straightforward. But the point I want to make is that no longer am I ever at a loss as to what to do next. I get stuck at times, to be sure, but when I do I have clean and direct ways to satisfy myself that I should just beat away at that roadblock for the time being.3b8af

"And then, for beating away at the roadblock, my bookkeeping regarding what I've tried, what possibilities I've collected, and what my assumptions and objectives are, is good enough to help tremendously in keeping me from getting into loops and quandaries,in carefully ex hausting possibilities, and in really analyzing my assumptions and objectives. What's more, I'm not generating reams of cyclic arguments, lists, calculations, or the like--either I'm checking the validity of what I've already structured, or I am correcting or expanding the structure. In other words, it seems that the growth of my comprehension is sure and steady up to the point at which I succeed or give up. If I give up, I leave a structure which is very well organized to accommodate a subsequent revisit with new data, possibilities, assumptions, objectives, or tools. Also, I set up a sentinel process that will operate in the future to help alert me to concepts which may clear the block.3b8ag

"This feature, of always having satisfying actions to perform, and having a good feeling that they are what I should be doing at that time, gives a surprisingly contented, eager, and absorbing flavor to my work. I guess it's an adult instance of the sort of change observed in students when they were given teaching machines that provided continuous participation and reinforcement.3b8ah

"Anyway, with the quick flexibility available to me for structuring arguments, and semi-automatic application of special tagging and linking rules, I find it really quite easy to construct, use, or modify sophisticated process structuring. And I can turn right around and apply this toward improving my abllity for structuring argumentg and processes. The initial, straightforward capabilities for manipulating symbol structures, that were more or less obviously availed me by the computer have given to me a power to participate in more sophisticated processes that capitalize more fully upon the computer's capability--processes which are very significant to my net effectiveness, and yet which weren't particularly apparent to us as either possible or useful in the days before we started harnessing computers to the human's workaday activities in this direct way.'3b8ai

"Let me mention another bonus feature that wasn't easily fore seen. We have experimented with having several people work together from working stations that can provide inter-communication via their computer or computers. That is, each person is equipped as I am here, with free access to the common working structures. There proves to be a really phenomenal boost in group effectiveness over any previous form of cooperation we have experienced. They can all work on the same symbol structure, wherever they might wish. If any two want to work simultaneously on the same material, they simply duplicate and each starts reshaping his version--and later it is easy to merge their contributions. The whole team can join forces at a moment's notice to 'pull together' on some stubborn little problem, or to make a group decision. Most points of contention are resolved quite naturally, over a period of time, as the developing structure of argument bears out one, or the other, or neither stand.3b9a

"No one can dominate the show, since seldom do you have to 'listen' to the person concurrent to the developments he is pursuing-- and yet at any time another person can tune in on what he has done and is doing. One can either take immediate personal issue with another about some feature, anywhere in the structure where he might find something done by the other to which he wants to take issue, or he can append his objection and the associated argument there where the disagreement lies, and tag this with a special cue that signals a point of contention that must ultimately be resolved. Any idea of the moment by any member can easily be linked to where it can do some good. It gets to be like a real whing-ding free-for-all--tremendously stimulating and satisfying, and things really get done. You find yourself 'playing over your head' almost all of the time.3b9b

"We have been experimenting with multi-disciplinary teams and are becoming especially excited over the results. For instance, there is a great reduction of the barrier that their different terminologies used to represent, where one specialist couldn't really apply his experien ce, intuition, or conceptual feel very well unless the situation could be stated and framed in his accustomed manner, and yet the others couldn't work with his terminology. Here, they meet at their concept and terminology interface and work out little shifts in meaning and use which each can find digestible in his system, and which permit quite precise definitions in each system of the terms and concepts in the others. In studying the other's structuring then, either of them can have his own definitions automatically substituted for the other's special terms. Reduce this language barrier, and provide the feature of their being able to work in parallel independence on the joint structure, and what seems to result is amplification of their different capabilities.3b9c

"Remember the term, synergesis, that has been associated in the literature with general structuring theory? Well, here is something of an example. Three people working together in this augmented mode seem to be more than three times as effective in solving a complex problem as is one augmented person working alone--and perhaps ten times as effective as three similar men working together without this computer-based augmentation. It is a new and exhiliarating experience to be working in this independent-parallel fashion with some good men. We feel that the effect of these augmentation developments upon group methods and group capability is actually going to be more pronounced than the effect upon individuals methods and capabilities, and we are very eager to increase our research effort in that direction."3b9d

"I have dragged you through a lot of different concepts and methods so far. I haven't been complete because we won't have the time. But I have selected the sample features to present to you with an eye toward giving you a maximum chance to identify these as being something significant to your own type of work. I avoided discussing techniques applicable to esoteric problem-solving processes--although some of them display especially stimulating possibilities to those with appropriate backgrounds. The ability to structure arguments organized in English-language statements, and to make use of the linking and tagging capabilities at all levels of the structure, can be seen to lead to many interesting and promising new capabilities for organizing your thoughts and actions. I think you could picture learning these tricks and using them in your own work.3b10a

"What I hoped to avoid by presenting the system in this way, was losing your identification with these possibilities by letting you get the mistaken impression that an individual couldn't harness these techniques usefully unless he first learned a lot of very sophisticated new language, logic and math. It is true that the more of the sophisticated tricks you learn, the more computer power you can harness and the more powerful you become--but very significant and personally thrilling practical problem-solving capabilities have been developed by quite a few subjects who were given only fifteen hours of training at one of these stations. The training, incidentally, was all provided by the computer without the presence of a human instructor. And the people were of such diverse fields as sociology, biology, engineering management, applied mathematics, and law. These were all relatively high-level people, and they were completely and unreservedly unanimous in their faith that their increased capability would easily justify the capital and operating outlay that we predicted for work stations of this sort in five years, if the computer industry really were to take this type of potential market seriously.3b10b

"What these people became capable of was somewhat less than the range of capabilities that we have discussed so far--but they would find it very natural to develop further techniques on their own, and new teaching programs could be provided them so that they could continue learning the improved techniques turned out by a research group such as ours here.3b10c

"But let me give you a brief view of some of the more advanced concepts and techniques that have evolved here, compatible with, but beyond, what I have so far shown you. And evolved is a good word to use here, because our appreciation for the potential worth of possibilities to be developed had to evolve too, and only came with the experience and perspective gained in our earlier work.3b10d

< class="level4"p>"For instance, we initially felt that defining categories and relationships, and making a plan for action, were things to be done as quickly as possible so that we could get on with the work. But, as our means developed for dealing with definitions and plans more precisely, easily, and flexibly, we began to realize that they in reality might be the most significant part of that work. With our immenseiy increased capability for complex bookkeeping relative to our interlaced hierarchies of objectives, plans, and arguments, we found that defining a new cate gory, searching for members or instances of it, or applying its selection criteria were becoming ever conscious and specific tasks.3b10e

"For instance, we began to find it more and more useful to distinguish different categories or types of process, different types of arguments, different types of relationships, and different types of descriptions. For a specific example, Ranganathan1 once cited five specific relationships that could obtain between two terms, where one modifies the other. He called these phase relations, and named how one term could relate to the other as either biasing it, being a tool used to study it, being an aspect of it, being in comparison with it, or influencing it. Vickery gave more examples, saying one could also have an effect on the other, be a cause of it, be a use for it, be a substitute for it, a source for it, an implication of it, be an explanation of it, or be a representation of it. There are even more categories mentioned in the literature.3b10f

"It was easy to form tags and links, and we experimented with the gains to be made by consciously specifying and indicating categories. It turned out to be a very invigorating innovation, and we began to take more pains with our structuring. It took longer to set up links and nodes in our structures, to be sure, but we found on the one hand that the structures became much cleaner and required fewer members, and on the other hand that we could get considerably more sophisticated help from the computer in doing significant chores for us.3b10g

"We began to work up processes that would help us establish categories, give them good definitions, check their relationship with other established categories, decide whether something fit a given category or not, search for all possible members of it within a given substructure, and so forth. The very fact of using this careful classifi cation within our structures allowed us to get more powerful help from the computer in these classification processes. I should mention that the relationships among the terms in a sentence--the syntax if you wish--had been given further specification tags than those I showed you earlier, to remove ambiguities that hindered the computer from going back to a statement and resolving the syntactical structure. Also, ambiguities in the meaning of the terms began to limit us, and we developed methods for removing a good deal of this semantic ambiguity. This slowed us down, as I've mentioned, but not as much as you'd think.3b10h

"Let me demonstrate one of the advanced processes which has evolved. It is heavily dependent upon the very care in building structures that it so nicely facilitates, and also upon several other developments. One of these other developments stems from the concepts and techniques of the semantic differential, as first introduced by Osgood, Suci, and Tannenbaum 2 back in 1957, and from some subsequent work by Mayer and Bagley 3 on what they called semantic models. These offered useful possibilities for establishing quite precisely what meaning a concept has to an individual, relative to his general conceptual framework, and for representing this meaning in a specific way that was amenable to computer manipulation.3b10i

"The other development upon which this process to be exhibited is based, was stimulated by our realizing that flexible cooperation with the computer was calling for lots of little interactions. Our working repertoire of small-task requests for computer service was getting quite large, and it was proving to be extremely valuable to use them and to be able to remember automatically their procedures and designation codes. One of our research psychologists had worked on human-memory phenomena before he came with us, and had interested himself in mnemonic aids of all sorts. He has developed some useful techniques for us to use in connection with this, and other problems. Now let me demonstrate this example of an advanced process for helping work with categories.3b10j

"Suppose that I want to establish a new category. Let's say that I have developed its description in what you and I have been calling an argument structure. I want to give it a name--a short and meaningful one--and I want a good definition. In fact, I want a definition that the computer can later work with. Look, I'll dig up a description that is awaiting such a definition, and you can watch what happens." So saying, Joe drummed on his keysets for a moment, with one interruption when the computer flashed something on the screen that was apparently a question about what he was asking the computer to find for him. He finally had a network display on one screen and a set of "exploded" statements on the upper half of the other.3b10k

"I'm initiating the naming and defining process now, and de signating to it the argument structure represented by this network as what I want named and defined. Watch what happens." A few more strokes on the keyset, and he picked up his light pen in anticipation and waited a few moments. A statement appeared in the lower half of the second frame. He studied it a moment, then looked at the statements above, picked out a node on the network with the pen, and hit the keyset a few strokes. Another statement flashed on almost immediately, with two familiar adjectives placed below and a graduated line between them. Joe studied this, referred to the statements above, flipped through several levels of network portrayals, through a few statements representing a couple of low-level nodes, reflected a moment, and then pointed his light pen at a point on the graduated line, part way between the adjectives, and pressed its button.3b10l

"Actually, right now I'm demonstrating a cooperative process execution technique. This process is applying some very sophisticated criteria and using some very sophisticated analytical techniques, and it is set up so that it is actually the computer that is now in the executive seat. I called for the process, but its execution essentially involves the computer's asking me questions, and feeding me successive questions according to how I've answered the previous ones. It also is doing a lot of work on the symbol structure that represents my description. It, with some small help from me, is proceeding through a quite complex analysis of the meaning that this incipient concept has to me, and of certain types of mental associations that I may have with it. I don't have to remember the special rules and forms of analysis involved-- nevertheless, a very sophisticated little capability is mine to use at will, taxing neither me nor the computer."3b10m

After a little over a minute of these question-answer interactions, the process apparently terminated, with four lines of special terms remaining on the screen. "This first line gives me two suggested names for this category or concept. The first term is a newly coined formal name, while the remaining three terms represent a compound expression, involving established concepts, that can be used also as a designation of the new category. The second line furnishes me with an association chain to use for a mnemonic aid in remembering the new name-- linking the name to several characteristics of the concept. The name itself was selected under mnemonic criteria, as well as to have a structure that goes with its syntactic and semantic categories. The third line lists the names of some previously defined categories or concepts that are the closest to this in meanlng--these before the break were found to overlap, and the rest are just close.3b10n

"The fourth line you recognize as a statement form, perhaps. This is the definition, as developed by the computer. It's in a special language and I won't try to explain. I'll just mention that I can now study it, take it apart, check its references, so to speak, and perhaps even see if the computer and I might work out any changes or improvementS But this process has been worked on pretty hard, and we're getting definitions that are hard to improve.3b10o

"This special language, in which I said the definition was stated, is a recent development. We had found that the types of structuring we were developing had a lot of extra tags and links that were traceable to the complexity of the rules and combinatorial possibilities of the English language with which the statements were construct We finally got a clear enough picture of the requirements we place upon a language in our use here that we could consider designing our own special language. It turned out to be a straightforward and rather simple language compared with English, but much more precise and powerful. It proves rather inflexible and awkward to use for speaking, but it provides plenty of flexibility and power for expressing things in the visual-symbol forms that we use. Its precision leaves no syntactic ambiguity in a well-formed statement, and makes it much easier to reduCe semantic ambiguity to the point where the computer can deal with our statements much as it can with mathematical or formal-logic expressionc3b10p

"It is worth mentioning, too, that we are experimenting with standard ways of structuring arguments at levels higher than the state ments--sort of a super grammar or syntax, with rules for assembling argument modules of different function lnto what becomes a well-formed higher-level argument module. There are some mixed feelings around here about this possibility, but I myself have become very much excited by it.3b10q

"Also we have been introducing formal methods for manipulating what you might call reasonable statements--as opposed to absolute true false statements which the more familiar formal logic can manipulate. This finds approval and faith in all of us here, but it is going a bit slowly.3b10r

"Let's run over some of the results we've seen to date, stemming from this new language and the new semantic awareness thus given the computer. If it can get hold of and manipulate important aspects of the meaning that is contained in our structures, it can develop answers to some questions for which there existed only conceptually implicit data. With practice and good strategy, asking questions like this proves to be a tremendously effective way to gain comprehension about a structure. We even have special processes and symbol-structuring methods to help organize the questioning and the answers. Some of the answers are a bit costly, however--in computer time and charges--and we have to watch the way we ask questions. Some of our researchers are studying the language and structuring techniques relative to this problem, and they think they see ways to change them to make question answering generally more efficient. But this sort of thing will likely always have its cost problems, as far as we can see now."3b10s

He went on to say that the computer now represents such an intelligent helper--although much less so than any human helper they would hire--that they refer to it as the Clerk. They can make a tentative new statement in the development of a structure, and have the clerk look over the structure to detect inconsistency or redundancy. The Clerk can also point out some of the weaknesses in the statement, as well as some of the effects of the statement upon the rest of the structure. They find that they need to give less and less human concern for the details of structure building--in-fact, the roles have reversed a little. Where the human used to set up tags and links so the computer could find its way around the structure as it ran errands for him, they now have the computer studiously installing similar things that are for the benefit of the human when he is studying the structure.3b10t

He also mentioned a recently developed computer process that could go back over a record of the human actions involved in establishing a given argument structure and do a creditable job of picking out the steps which contributed the most to the final picture--and also some of those that contributed least. This process, and some of the past data collected by its use, were becoming an important addition to the planning review sessions, as well as to the continuing development of improved methods. And apparently, it had a surprisingly positive psychological effect upon members of a cooperating team, where an objective means of relative scoring was thus available.3b10u

Figure-3. Initial Augmentation-Research Program.

4e3a

Figure-4. Regeneration.

4f5a

Figure-5. A Total Program. Suggested relationship among the major activities in achieving the stated objective (essentially, of significantly boosting human power in A 4 and U 1). Solid lines represent subject information or artifacts used or generated within an activity, and dashed lines represent special tools and techniques for doing the activity in the box to which they connect. Subject product of an activity (output solid) can be used as working material (input solid) or as tools and techniques (input dashed). Tools and techniques as used or needed in an activity (output dashed) can be used either to work on (input solid) or as tools and techniques to work with (input dashed).

4g8a