1964 04_#25 04 #25

(for n-l:::::'i). Then all qi are single points. Next consider the point V (i>n-l) with ordinate y==l on one of the lines Qi, and its projection on . The directed line from qi to Ii is parallel to grad Qi, and : grad Qi : == 1/ ! qi_/i I. The n coordinates of == (f ~ ••• f ID are found by solving simultaneously the n linear equations, Ek, corresponding to the n Tk in the expression for Qi and setting y == 1. Setting y == 1 is arbitrary, and insures only that y =1= o. The n coordinates of qi == (ql . .. qi.) are found by solving the same system with y == o. Next consider some arbitrary point V (OLi < n-l) with ordinate y == 1 on one of the linear subspaces pi of dimension n - i, and its projection fi on . Let qi (for OLi < n-l) be the point on pi closest to !i. The directed line from qi to f i is then, again, parallel to grad pi, and also I grad pi I == 1/ I qi Since V can be chosen arbitrarily from all the points on pi with y == 1, we set It == 0 (for k == i + 2, ... , n), and find the first i + 1 coordinates of t == f~ ... lii+ h N+2 ... NJ by solving simultaneously the i + 1 linear equations, Ek, corresponding to the i + 1 Tk in the expression for pi, again also setting y == 1. The n coordinates of qi == (q; ... qD are determined by requiring that the square of the distance from Ii to qi be a minimum, i.e. d:! == (I; _ q~) 2 + (tJ - q~):! + ... + (ll~ - qD2 == min. Furthermore we note that the q~ must satisfy the constraint that qi is to be on pi, which is the same as saying that the q~ must also satisfy the i + 1 linear equations, Ek (k == 0 ... i), corresponding to the i + 1 Tk in the expression for Pi, but now setting y == o. The Lagrangian method of undetermined multipliers leads us then to the unconstrained minimization of the function g (q: ... q)il' Au ... Ai) == d:! J\.Eo AiEi. Equating to zero the partial derivatives of g with respect to the n q~ and the i 1 Aj, we obtain again a system of linear algebraic equations, here of order n i 1. Note that this way of finding qi applies only to the initial cycles of the search for which i < n - 1 ; hence the largest linear system to be solved here will have order n + (n-2) + 1 == 2n-1. r r r . j + .. + + + + + 197 The linear equations Ek (OLk) correspond to our hyperplanes Tk which are tangent to the hyper-surface S at the locations Xk. The analytic expression for one of the Ek can readily be obtained from a Taylor expansion of F about Xk: F (x) == F (Xk) (X_Xk) . grad F (Xk) higher-order terms. If we replace F(x) by 'Y}k and x == (Xl ... x n ) by ~k == a~ ~~), delete the higher-order terms, collect all constant terms, and rewrite this expansion in terms of the individual running coordinates on the tangent plane, we obtain with grad F(Xk) == -(a~ ... a~): a~ ~r+ a~ ~~ a~ ~~ 7Jk == b k. This can be simplified considerably for computational purposes by always setting bk == O. This is permissible because we are only interested in the slopes of the Tk, or in the slopes of subspaces common to several Tk. The Ek are then ,~­ plicitly: Ek == a~ ~~ a~ ~~ 7Jk== 0, and they describe hyperplanes which are parallel to the tangent planes, and contain the origin of the space G (Sill) < G (s,,). The procedure is described below in several steps. STEP (1) 81 is equated to SJ, and G == G (SI) is evaluated. This defines the point Pi == (S1, GJ for i == 1, and with Gi == G (sJ. 198 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Question: Is G1 :::::".. Go ? If yes, a parabola is passed through Po with the given slope Go, and through Pl. This is done in the following Step (2). If no, we go directly to Step (3) to check whether PI is above or below the line through Po with slope G-o. STEP (2) The minimum location S2 is found for the parabola through Po and PI with slope Go at Po. S2 will satisfy: Isol < IS21 L. IS11/2. G2 == G (S2) is evaluated. Question: Is G2 < Go ? If yes, we have solved our problem, i.e. we have obtained a bracketed 'approximation to the minimum, with Su == so, Sm == S2, Sb == Sl. However, our experience showed that the ex· penditure of one more function evaluation often allowed a considerable improvement of the value for Sm. This is particularly desirable toward the end of our search for the minimum of F (Xl ... xn). The improvement of Sm is obtained by cubic interpolation, 'and is described in Step (4). Before going there, however, the indices 1 and 2 are interchanged on the quantities Ph Gh Sl and P 2, G2, S2 so as to number the points Po, Ph P 2 in the order of increasing lsi-values. If no, we substitute P 2 for Ph and repeat Step (2) until either G2 < Go, or IS21 < 18 11/ 100. In the latter case we replace Si by -8h Go by the slope of the line (Po, P 2), and return to Step (1). This starts a search for tbe minimum of G in the range of'negative s-values (assuming that Go was inaccurate). In case our search should then again lead us back to the vicinity of the same s-value, s == so, we set Sm == So with brackets s" == -Is] 1/100 and Sb == 1"811/100. In this case our starting point and our approximation to the minimum are one and the same point. That situation will always happen at the final minimum of F (Xl ... Xu), but may also happen occasionally sooner. The termination procedure, described further below, then is triggered. STEP (3) Question: Is PI above the line through Po with slope Go ? If yes, the minimum location S2 is found for the parabola through Po and PI with slope Go at Po. It satisfies IS21 > IS11/2. If IS21 < !s11, the indices 1 and 2 are interchanged. If no, the minimum locatIon S2 is found for the cubic with clope Go at Po and with its point of inflection at Pl. It satisfies IS21 > 21st!. Question: Is G1 < G2 == G (S2) ? If yes, we found a bracketed approximation to Smiu, and proceed with Step (4). If no, we re-define Go to be the slope of the parabola through Po, Ph P 2 at point Ph then subtract 1 from all index values occurring in Pi == (Si, Gi ) (i == 0, 1, 2), and return to Step (3). STEP (4) A cubic is passed through Po with slope Go, and through PI and P 2. ~ll is the location of the minimum of this cubic. Note that the existence of brackets for Sill guarantees the existence of the minimum. STEP (5) We test next the three lowest values of G, found so far, whether they agree with each other within a given tolerance. If yes we set Sm equal to that si-value for which G (Si) is smallest. (Sm usually is, but need not be, that sj-value.) If no, a parabola is passed through the lowest three points, and the minimum value of this parabola is compared with the two lowest available values of G. If these three values agree with each other within the given tolerance we have found Sm; if they do not agree, we evaluate G for that s-value for which the parabola had its minmium, and go to Step (6) . STEP (6) We test the two lowest values of G with the minimum value of the last parabola. If these three values agree with each other within the given tolerance we have found Sm; if they do not agree, a parabola is passed through the lowest three points of G and we compare the minimum values of the last two parabolas with the lowest value of G. If these three values agree with each other within the given tolerance, we have found Sill; if they do not agree, we evaluate G for that TWO NEW DIRECT MINIMUM SEARCH PROCEDURES FOR FUNCTIONS OF VARIABLES 199 s-value for which the last parabola had its minimum, and repeat Step (6). (1.5) Detection of the Minimum The iterated execution of the procedures for Parts I and II of each cycle may be discontinued whenever one of the following conditions is detected: Condition A: The partial derivatives of F == (x~ ... xft) satisfy laF /ax~1 LEfor all i == ... n, and for a given small value of E, say E - 10-6 • As explained at the end of section (1.3, these partial derivatives are evaluated to obtain the coefficients a k in the equations Ek. CondiUon B: The starting point and the approximation to the minimum point on the baseline are one and the same point. How this can happen was explained in Step (2) of section (1.4). Condition C: The matrix of the linear equations Ek, discussed in section (1.3), is very illconditioned or singular. This may occasionally happen before either Condition (A) or Condition (B) is satisfied. Condition D: The numerical values of the coordinates of the minimum location and/or the minimum of the function have stabilized to a given number of significant figures. Whenever one, or perhaps two of the above conditions are satisfied, there is an excellent chance that the minimum has been found. One may then do one of two things: (i) terminate the execution of the program, or (ii) begin a somewhate more extensive testing procedure in the course of which the vicinity of the suspected minimum location is explored, and it is thereby made even more likely that the solution has been found. This is what was programmed at G.D'; Astronautics. Preference was given to method (ii) because it also provides an answer to the question how sensitive the function is to small deviations from the minimum location. The knowledge of this sensitivity is usually desired together with the minimum location itself. (2) CIRCULAR ARC SEARCH METHOD FOR 2 VARIABLES (2.1) Summary of the Method The iterative circular search is preceded by a starting procedure, which finds five points near the spine of the valley of the function to be minimized. Thereafter, each iteration begins with an attempt to approximate by ap.ortion of a circle the course of the projection of the valley onto the space of the independent variables. The first circle is fit to three of the five valley points found. All succeeding circles are fit to four or more valley points. As the iteration progresses toward the minimum these circles approach the osculating circle at the minimum location. During each iteration the determination of a circular arc approximation to the spine of the valley is followed by an attempt to predict a point on this arc which is closer to the desired minimum than any other point obtained so far. Again during each iteration, the prediction of such a point is followed by a steepest descent from this point into the valley, which amounts to a minor, yet important correction. The correction is important since it insures that deviations between the courses of the circular arcs and the portions of the spine of the valley approximated by the arcs will not accumulate as the iteration progresses. The new valley point then replaces the highest of the four lowest valley points found before, and this predictor-corrector type method is repeated in the next iteration. Empirical results show that the sequence of valley points obtained converges quite rapidly to the desired minimum. In the following sections we emphasize again the geometric aspects of the more important steps summarized above, rather than to give the algebraic details. This is done since some of the details are still subject to experimentation and analysis. (2.2) The Starting Procedure We assume that a first guess is available as a starting location Xl. We find its valley point VI. A valley point Vi will always mean the location of the minimum of the given function F along a line through Xi in the direction of -grad F (Xi). We find the distance d 1 == Ivl - xli and a unit vector e l in the direction of the projection of -grad F (VI) on a line normal to -grad F (Xl). We then find four more valley points Vi (i == 2 ... 5) by taking a step d i == (3/4)i-2 d l in the direction of ei'-l from Vi - 1 to Xi, and then by minimizing from Xi to Vi. The directional minimizations are done by using the 200 PROCEEDINGS---, where Vi and i were explained above, and Fi = F (Vi). It is helpful to visualize now a cylinder in 4> parallel to the y-axis and intersecting cJ> along the circle of the preceding section. This cylinder and the above plane E;,! intersect in an ellipse. The location u of the point on this ellipse with the smallest y-value is our predicted point. The distance of the point u from the spine of the valley decreases rapidly as the iteration p~ogresses, and appears to become insignificantly small long before the minimum is reached. Nevertheless, it is unknown at the present time how damaging the cumulative effect woud be. \Ve are employing, therefore, during all iterations a straight steepest descent correction which leads from u to the next valley point. The correction is done in the same way as was explained for the five starting points in (2.2), i.e. by using the method described in (1.4). The initial step taken in minimizing from the low point on the circle to the valley is set equal to one-half the length of the gradient vector at the low point divided by the coefficient of the second-degree term of the parabola approximating the shape of the function along the previous line of minimization. We expect that further studies of this particular correction will lead to considerable sa vings in the number of function evaluations. For example, it appears that the correction usually need be made only in a radial direction rather than the negative gradient direction, while the latter is needed only occasionally. Moreover, it is doubtful whether any such correction is needed at all in each iteration. (2.5) The Switch to Pa.rabolas The above predictor-corrector step is taken at least four times. If thereafter the function value of the lowest valley point is within t/:! 7< of the function value of the second-lowest valley point, the method of predicting a new point u on the circle is modified somewhat. Instead of finding the above described plane E;,!, we find the "best-fitting" line L to the same four points Vi defining our circular arc and in the same· plane with this circular arc. The lowest three Vi are then projected on L to give the three points Wi on L. These and the Fi -values associated with the Vi define three points Wi = (Wi, Fi) in the plane through L and parallel to the y-axis. The location Will on L of the minimum of the parabola through the three points Wi is then projected back onto our circular arc to give our predicted point u. The steepest descent correction is gain applied to u, and produces the next valley point. (2.6) The Tail Correction After successive approximations to the minimum value of the function have stabilized to five figures, the search terminates with a simple tail correction, which consists in minimizing the function along the line through the two lowest valley points, and which usually adds several significant figures to the answers. It is interesting to observe the behavior of the radii of the circular arcs after the iteration has begun to converge. As was said before in Section (2.1) the circles approach the osculating circle of the spine of the valley at the minimum location. However, once the sequence of valley points has converged to the minimum, all subsequent points fall very close to each other; they then no longer define the course of the valley nor do they define its osculating circle. Instead, the radii of the "best-fitting" circles suddenly shrink by several orders or magnitude to dimensions comparable to the inaccuracies in the coordinates of the minimum location. When this shrinkage occurs the search should be terminated. Usually, however, this does not happen before the search had already been terminated with the above described tail correction. (2.7) An Extension to n Variables: The Spherical Shells Method The good results obtained with the two- 201 TWO NEW DIRECT MINIMUM SEARCH PROCEDURES FOR FUNCTIONS OF VARIABLES dimensional Circular Arcs Method suggest its extension to n independent variables as follows: (-1) ST ART. A starting point Xi is given in n space as a first approximation. Find its valley point VI, the distance d l == !Vl - xli, the n - 1 space El normal to grad F (Xl), and the direction e 1 in EI as the projection of -grad F (VI) on E I. Find n + 3 more initial valley points Vi (i ==2 ... n + 4) by taking steps d i == (n + 1) (n + 2» i~2dl from Vi- 1 in the direction e i- l to Xi, and then by minimizing F in the direction of -grad F(Xi) from Xi to vi, for all values of i == 2 ... n + 4. I ( - 2) FLA TS. Fit an m fiat (Le. a linear m space) to the lowest m + 2 valley points in n space, such that its dimension m is as small as feasible. (The feasibility could be judged by observing successive standard deviations CT IlI of the points from the m fiats for the sequence of decreasing m - values: m == n, n - 1, ... , 1. A sudden large increase of CTk - lover CTk would indicate that m == k is feasible while m == k-I is not feasible.) Skip to ( - 4) if m == 1. ( - 3) SPHE RE S. Fit the surface of an mdimensional sphere to the projections of the lowest m + 2 valley points on the above m fiat. (3.1) (3.2) (3.3) (3.4) (3.5) (3.6) Some shell of this sphere will approximate the course of the (hyper - ) valley. Pin-point next an optimum surface point on this shell to which to go next, find its valley point, and return to (-2). ( - 4) PARABOLAS. Find the parabola through the projections of the lowest three valley points on the plane spanned by the function axis and the regression line found in (-2). Go to the minimum of this parabola,find its valley point, and return to ( - 2) unless the minimum value of the function has begun to stabilize. ( - 5) T A I L. Minimize the function along a line through the lowest two valley points "in n space. Find the valley point corresponding to this directional minimum. Repeat ( - 5) till the function minimum has stabilized to the desired number of figures. The above described Spherical Shells Method will be programmed at General DynamicsAstronautics in the near future, and results will be published when available. (3) EXAMPLES AND COMPARISONS The following functions of x and y were minimized. (r == (X2 + y2) % and () == tan- 1 (y Ix).) (1- X)2 + (I-x):.! + 100(1X)2 + (I-x):.! + O.I( 2-x):.! + + y:.! (x - IO)-l IOO(y-x:.!):.! ROSIE (y - x:.!):.! SHALOW == (y - x:.!):.! STRAIT 100 (y - x:{):.! CUBE DOUBLE 100 (r - 1)2 (r - 2)2 HEART ---: 1000 ( 1 + cos () - r II O):.! All were programmed in double-precision for an J .B.l\1. 7094 computer. Results are listed in the following tables, and the results for ROSIE are discussed in detail and compared with other methods. (3.1) ROSIE Minimum (F == 0) at (1, 1), with a steep valley along y == x:.!, and a side valley along the negative y-axis. Table (3.IA) summarizes the end results of various investigators. Table (3.IB) summarizes the rapidity of convergence of some of these methods. Table (3.1e) shows the rate of convergence of the circular arcs method for various starting points. Table (3.ID) gives minimization details for the circular arcs method. Table (3.1A) End results of various methods of minimizing the function ROSIE when starting from the point (x, y) == (-1.2, 1) k == number of valley points found. N == number of function evaluations Note: The value of the function given by ref. (5) in the table is not consistent with the values given for x and y. The correct value of F equals 1 ( - 6) for (x, y) == (1.0001, 1.0001). Discussion: Parentheses in column N indicate that the number of function evaluations was not published in the indicated papers, but instead 202 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 was estimated by us to correspond to each individual method, assuming an average number of 8 function evaluations for minimizing the function in a given direction. The method of row 5, ref. (7), requires the programming of the analytic expression of the gradient of F; none of the other methods do. The numbers in parentheses in the (min F) column give exponents of the powers-of-ten factors. It is obvious that the method of ref. (10) is the least efficient, while the circular arcs method is the most efficient for the function ROSIE. Table (3.1A) N 146 200 33 (264) 85 (765) 18 (144) 12 103 k 17 minF 6(-8) 2(-5) 8(-9) 6(-5) 1(- 8) 3(- 9) y 0.999938 0.991 1.0001 1.002077 x 0.999957 0.995 1.0001 1.000653 ? ? 1.000008 1.000010 Method "Lin. Search", this paper, 1953 Rosenbrock, ref. (4), 1960 Powell, ref. (5), 1962 Baer,ref. (10),1962 Flethcer,ref. (7),1963 "Cir. Arcs", this paper, 1964 Table (3.1B) References: (E) Ref. (5) k 0 3 6 9 12 15 18 21 24 27 30 33 2.4+1 3.6 0 2.9 0 2.2 0 1.4 0 8.3-1 4.3-1 1.8-1 5.2-2 4 -3 5 -5 8 -9 (G) "Circ. Arcs H Ref. (7) 2.4+1 3.7 0 1.6 0 7.5-1 2.0-1 1.2-2 1 -8 Table (3.1B) Rate of minimization of function ROSIE for four different methods, all starting from ( - 1.2, 1). All function values are in the powers-of-ten notation, e.g. 2.4 + 1 = 24 or 3.4 - 9 = 0.0000000034. k "Linear Search" = number of valley points found The table clearly shows the better rate of con- 2.4+1 3.5 0 1.9 0 8.3-1 3.4-1 1.1-1 6.0-8 2.4+1 3.5 0 7.3-2 8.9-3 3.4-9 k 0 3 6 9 12 15 18 21 24 27 30 33 vergence of the circular arcs method for the function ROSIE. Table (3.1C) Rate of minimization of function ROSIE with the circular arcs method for five different starting points (x, y). Bottom row gives the total number, N, of function evaluations. The location of the minimum is at x = 1 and y = 1. TWO NEW DIRECT MINIMUM SEARCH PROCEDURES FOR FUNCTIONS OF VARIABLES 203 Table (3.1e) x y 5.621 -3.635 -1.200 1.000 -0.221 0.639 -2.547 1.489 -2.000 -2.000 x y k 0 2.4+1 1.2+5 3.6+1 2.5+3 3.6+3 k 0 1 2 3 4 5 4.1 3.8 3.5 3.3 3.2 9.8+2 2.6+1 1.5 0 2.4+2 1.1-4 2.0 0 8.9-1 5.5-1 2.7-1 1.4-1 5.4 0 3.2 0 1.4 0 8.6-1 2.2-1 2.2+2 2.6 0 1.4 0 2.1-2 5.2-2 1 2 3 4 5 5.8-2 3.5-2 1.2-2 2.6-3 4.1-4 1.2-5 2.6-7 6.6-10 1.9-1 1.8-1 7.4-2 1.9-2 5.1-3 2.7-3 6.4-5 3.4-6 4.3-8 2.1-11 1.7-3 1.6-3 1.4-4 7.5-6 6.1-8 3.7-11 - - 6 7 8 9 10 11 12 13 14 15 16 93 118 101 N 0 0 0 0 0 6 7 8 9 10 11 12 13 14 15 16 7.3-2 3.3-3 8.4-3 8.9-3 7.0-7 4.4-6 3.4-9 - 7.5-1 4.7-1 5.3+2 4.7-1 4.9-1 3.2-2 4.2-3 2.6-5 1.3-7 5.3-10 8.9-14 N 103 195 - - - - Table (3.1D) k N F 0 1 24.2 1 2 3 4 5 9 17 25 33 41 4.1 3.8 3.5 3.3 3.2 6 7 8 9 I;9 69 75 82 10 11 12 t x Tl n. grad I I total correction -1.2000 1.0000 1-2 1-2 1-2 1-2 1-2 -1.0298 -0.9464 -0.8800 -0.8266 -0.7852 1.0694 0.9043 0.7829 0.6924 0.6256 7-2 3-3 8-3 9-3 1-2 1-2 4-3 4-3 1.2705 0.9427 1.0918 1.0940 1.6151 0.8885 1.1923 1.1972 4.1 1.2 1.3 1.4 5-0 3-1 5-2 3-2 2-0 8-1 3-1 3-1 89 95 7-7 4--.:6 3-3 3-6 1.0008 1.0021 1.0017 1.0042 7.1 5.8 4-4 6-6 1-1 3-3 103 3-9 7-7 1.000008 1.000010 Table (3.1D) Details of minimization progress with function ROSIE using circular arcs method and starting at (-1.2, 1). = number of valley point N = number of function evaluations k y 2-1 2-2 2-2 1-2 2-2 2-3 = function value: for k = 0 ... 5 in decimal notation, for k = 6 ... 12 in powers-of-ten notation, e.g. "7 - 2" = 0.07. t = accuracy achieved by F (k), = tolerance to be satisfied by F (k + 1). F 204 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 is a circle through points 3, 4, and 5; its predicted low point was at x == 6.4 and y == 12.2. It was corrected to give the indicated valley point (k == 6). The next valley point (k == 7) was obtained from a circle fitted to points 3, 4, 5, 6. Likewise (k == 8) was obtained from 4, 5, 6, 7; and (k == 9) from 5, 6, 7, 8. Valley point (k == 10) was obtained from a circle which was no longer fitted to any of the five starting points, but instead only to the much more accurate points 6, 7, 8, 9; hence, we had a sudden speed-up in the convergence from point 9 to point 10, as evidenced also by the sudden decrease in the ratio of the grad correction to the total correction. Also, the radius R began to approximate more accurately the radius of the osculating circle (R == 5.59) of the valley y == X2 at the minimum point (1, 1). Also, the program had automatically switched at this time from the regression plane to the regression line when predicting point 10 on the circular arc (sec. 2.5). The last valley point (k == 12) was the result of applying the tail correction (2.6); note the sizeable increase in accuracy. The t-values are calculated from (F (k) F (m» / (F (m) C), where F (m) is the lowest available valley point prior to F (k), and where C is a suitably chosen constant (here C == 1,23456) to avoid the possibility of the divisor tending to zero. These t-values are then used as a tolerance when making the next gradient correction to find F (k + 1). This automatically tightens the tolerance as the minimum is approached. + x, y== coordinates of starting points and all valley points. R== radius of circle fitted to lowest three or four valley points. grad correction = distance from predicted point on circular arc to gradient corrected point in valley. total correction == distance from previous lowest valley point to new valley point. The first five valley points (k == 1 ... 5) were found by the starting procedure described in (2.2). The first circle (with radius R == 4.1) (3.2) SHALOW Minimum (F == 0) at (1, 1) with valleys along y == X2 and x == 1. SHALOW is similar to the function ROSIE, but has a shallow valley compared with the steep valley of ROSIE. Table (3.2) x -2.000 1.184 0.803 0.211 0.820 x y -2.000 0.574 -0.251 3.505 4.690 y k 0 4.5+1 7.2-1 8.4-1 1.3+1 1.6+1 k 0 1 2 3 4 5 3.0 0 2.2-1 5.9-2 9.0-2 1.6-2 2.1-2 3.6-6 4.3-3 1.1-3 6.1-4 3.3-1 1.1-] 4.7-2 1.3-2 3.0-3 1.1-1 2.7-1 4.6-2 5.6-2 1.6-2 9.5-1 4.0-1 1.0-1 2.1-3 3.3-2 1 2 3 4 5 6 7 8 9 10 11 4.9-4 4.6-4 3.3-4 4.5-6 7.1-6 5.4-7 7.7-6 8.1-6 5.2-8 2.7-10 4.7-13 9.9-5 1.1-4 9.9-6 6.0-8 1.1-10 2.8-16 2.8-4 1.9-4 1.7-4 2.5-8 1.2-7 8.4-13 3.4-4 4.4-4 2.4-5 5.8-5 6.7-9 1.1-10 6 7 8 9 10 11 86 94 96 N N 93 92 TWO NEW DIRECT MINIMUM SEARCH PROCEDURES FOR FUNCTIONS OF VARIABLES 205 Table (3.2) Rate of minimization of function SHALOW with the circular arcs method for five different starting points (x, y). Bottom row gives the total number N, of function evaluations. (3.3) STRAIT Minimum (F = 0) at (1, 1) with a steep valley along x = 1. Table (3.3 x 2.000 2.000 2.019 1.992 1.986 x y -2.000 -2.322 -1.505 -3.222 5.227 y k 0 k 0 1.4+2 1.4+2 1.3+2 1.5+2 9.9+1 1 2 3 4 5 8.4 0 3.4 0 1.3 0 3.0-1 1.7-2 1.0+1 4.6 0 2.1 0 7.3-1 1.9-1 5.8 0 1.8 0 3.9-1 1.6-3 1.6-1 1.7+1 9.0 0 5.4 0 2.9 0 1.7 0 1.7+1 1.0+1 7.3 0 4.8 0 3.5 0 1 2 3 6 7 8 9 10 11 12 13 5.5-8 3.4-5 9.6-9 3.9-5 0.0 0 6.2-6 4.6-3 3.6-6 1.8-8 2.7-14 6.4-15 9.0---.:11 8.9-9 1.6-3 1.9-1 2.2-4 1.9-5 1.6-12 0.0 0 3.7-2 3.8-1 1.2-2 5.1-5 2.7-5 5.7-8 2;5-11 0.0 0 6 7 8 9 10 11 N 78 83 58 83 95 4 5 12 13 N Table (3.3) Rate of minimization of function STRAIT with the circular arcs method for five different starting points (x, y). Bottom row gives the total number, N, of function evaluations. 206 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 (304) CUBE Minimum (F = 0) at (1, 1) with a steep valley along y = x 3 • Table (3.4) x 1.200 1.391 1.243 0.284 -1.200 :x y -2.000 -2.606 -1.974 -3.082 -1.000 y 1.4+3 2.8+3 1.5+3 9.6+2 5.8+1 k 0 1 2 3 4 5 4.6 0 4.8-1 5.2 0 1.8-1 1.3-3 1.7+2 1.8 0 2.4-3 3.5-1 4.8-3 8.2+1 1.2 0 1.2-1 7.2-1 2.3-2 2.1 0 5.4 0 1.0+1 1.7-1 3.8-1 4.1 3.8 3.6 3.4 3.3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1.9-2 2.4-2 6.3-4 1.2-6 6.7-7 7.5-10 1.3-5 1.6-5 8.5-6 2.5-9 8.8-11 - - 4.5-2 3.7-2 2.5-2 1.7-2 2.0-3 5.5-4 5.9-5 1.8-6 3.6-8 5.6-11 1.5+2 3.4 0 6.7-4 4.9-3 1.9-3 6.2-5 2.6-6 1.9-8 4.2-11 - 1.5-1 6.0-2 7.4-2 1.0-1 7.9-2 1.6-2 6.4-3 2.4-3 2.5-4 2.3-5 8.5-7 7.2-9 4.7-12 - 6 7 8 9 10 11 12 13 14 15 16 17 18 N 99 113 135 162 N k 0 - - - 98 - - - - - 0 0 0 0 0 Table (3.4) Rate of minimization of function CUBE with the circular arcs method for five different starting points (x, y). Bottom row gives the total number, N, of function evaluations. TWO NEW DIRECT MINIMUM SEARCH PROCEDURES FOR FUNCTIONS OF VARIABLES 207 (3.5) DOUBLE One minimum (F == 0.099899899) at (x == 1.001005; y == 0), another minimum (F == 0) at (x 2; y == 0) ; with one steep valley along r == 1,and another steep valley along r == 2; saddle points around (-2,0) and (-1,0). == Table (3.5) x -4.000 -3.709 -3.935 -5.392 -7.633 x Y 0.000 -0.957 0.042 -1.708 -3.785 Y k k o 3.6+3 2.7+3 3.2+3 2.9+4 2.4+5 o 1 1.6 0 7.3-1 5.1-1 1.8-1 7.5-1 1.5 0 6.5-1 1.1 0 1.5-1 4.4-1 1.6 0 7.3-1 7.3-1 1.8-1 6.2-1 1.5 0 5.8-1 7.2-1 2.2-5 7.9-2 1.4 1.7 2.5 1.0 2.4 1 2.4-1 1.6-1 1.5-1 1.2-1 1.3-1 1.0-1 1.0-1 1.0-1 1.0-1 2.2-1 4.4-4 1.0 0 1.1 0 1.4-1 1.1-1 1.2-8 5.7-6 2.3-6 1.2-8 2 3 4 5 6 7 8 9 10 11 12 13 14 15 N 1.3-1 1.1-1 1.0-1 1.0-1 1.0-1 1.0-1 1.0-1 115 113 o <) "1 "'.0-.1. 1.8-1 1.1-1 1.0-1 1.0-1 1.0-1 1.0-1 1.0-1 1.0-1 2.9-5 1.8-6 1.5-6 1.2-6 118 106 0 0 0 0 0 -14 2 3 4 5 6 7 8 9 10 11 12 13 14 15 139 N Table (3 ..5) Rate of minimization of function DOUBLE with the circular arcs method for five different starting points (x, y). Bottom row gives the total number, N, of function evaluations. 208 PROCEEDINGS---.,SPRING JOINT COMPUTER CONFERENCE, 1964 (3.6) HEART Four minima (all with F == 0) at left (0,0), at right (20,0), at top (10, +12.72), and at bottom (10, -12.72). Three valleys: along the x-axis, along x == 10, and along the cardioid r == 10 (1 + cos 0). Table (3.6) x y -1.000 0.000 -9.870 >-2.159 -11.43 -7.344 -6.255 -21.13 -7.539 -27.04 x y k 0 1.0+3 7.3+5 1.1+7 3.1 +7 6.9 +7 0 1 2 3 4 5 9.5+1 9.9+2 4.8+2 1.6+1 8.3-2 8.8+2 5.2+2 3.1+2 1.9+2 1.1+2 4.1+2 7.6+2 4.2+2 1.2+2 5.8-2 5.3 1.0 1.9 3.9 2.5 8.6 1.4 6.2 7.4 3.6 +2 +3 +1 +2 +0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1.8-2 1.4-4 1.3--3 9.1-4 3.1-4 1.7-4 7.7-5 4.8-4 9.7-5 7.3-5 7.1-5 (top) 149 1.4+1 2.2 0 2.3+1 8.7 0 6.2 0 2.1 0 5.6-3 8.3-6 1.7-7 1.7-7 (left) 3.4 0 1.8-3 2.2--3 3.7-5 3.5-9 6.7-7 8.2-12 (bottom) 3.0 +1 4.5 +0 2.4-8 1.2 -2 3.3 -10 6.9 -4 5.7 -11 (right) 3.6 +0 1.8 -1 4.3 -6 2.7 -3 4.1 -3 4.1 -4 3.3 -5 6.8 -6 2.0 -6 (bottom) 6 7 8 9 10 11 12 13 14 15 16 127 119 N +1 +3 +1 +2 +2 - - - - - - - 121 150 N Table (3.6) Rate of minimization of function HEART with the circular arcs method for five different starting points (x, y). Bottom row gives the total number, N, of function evaluations. CONCLUSION We feel that the foregoing results conclusively show the gain in efficiency obtainable for a direct minimum search procedure when an attempt is made to approximate by suitable curves the valleys of the function to be minimized. The circular arcs method which strives to do so can only be a beginning. Various steps and details of its procedure can and will doubtlessly be improved in the future; most interesting should be the results of its generalization to n variables. It is possible that the gain in efficiency is larger when the circular arcs method is applied to functions with steep-sided valleys than when it is applied to functions with more clearly defined minimum points and less clearly defined valleys. On the other hand, when penalty terms are added to an objective function, in order to also observe inequality constraints, functions with steep-sided valleys will automatically be generated; thus, the methed may become useful in the field of non-linear programming. The authors wish to acknowledge the constant support and encouragement they received from Dr. Wm. J. Schart, Chief of Mathematical Analysis at General Dynamics/ Astronautics, and Mr. Carl E. Diesen, Manager of Scientific Programming and Analysis at General Dynamics/Astronautics. We also wish to thank Mrs. Marilee Culver for the efficient and neat typing of the manuscript. TWO NEW DIRECT MINIMUM SEARCH PROCEDURES FOR FUNCTIONS OF VARIABLES LITERATURE REFERENCES 1. BROWN, R. R.-"A Generalized Computer Procedure for the Design of Optimum Systems." Presented at the Winter General Meeting of the American Institute of Electrical Engineers, New York, N.Y., Feb. 2-7, 1958. 2. JOHNSON, S. M.-"Best Exploration for Maximum is Fibonaccian." RAND Corporation, Santa Monica, Calif., Report No. P-856 (1959). 3. G;ELFAND, I. M., and TSETLIN, M. L.-"The Principle of Non-local Search in Automatic Optimization Systems." Soviet PhysicsDoklady, Vol. 6, No.3, Sept. 1961, pp. 192194. 4. ROSENBROCK, H. H.-"An Automatic Method for Finding the Greatest or Least Value of a Function." Computer Journal, October 1960, Vol. 3, pp. 175-184. 5. POWELL, M. J. D.-"An Iterative Method for Finding Stationary Values of a Function of Several Variables." Computer J ournal, July 1962, Vol. 5, No.2, pp. 147-151. 209 6. FORSYTHE, G. E., and MOTZKIN, T. S."Acceleration of the Optimum Gradient Method, Preliminary Report" (Abstract). Bull. Amer. Math, Soc., Vol. 57, 1951, pp. 304-305. 7. FLETCHER, R., and POWELL, M. J. D.-"A Rapidly Convergent Descent Method for Minimization." Computer Journal, JUly 1963, Vol. 6, No.2, pp. 163-168. 8. DAVIDSON, W. C.-"Variable Metric Method for Minimization/' A. E. C. Research and Development Report, ANL-5990 (Rev.). 9. SHAH, B. V., BUEHLER, R. J., and KEMPTHORNE, O.-"The Method of Parallel Tangents (Partan) for Finding an Optimum." Office of Naval Research, Report NR-042207 (No.2). 10. BAER, R. M.-"Note on an Extremum Locating Algorithm." Computer Journal, Oct. 1962, Vol. 5, No.3, p. 193. 11. KROLAK, P., and COOPER, L.-"An Extension of Fibonaccian Search to Several Variables." Communications of the A.C.M., Oct. 1963, Vol. 6, No. 10, pp. 639-641. ON THE EVALUATION OF THE COST-EFFECTIVENESS OF COMMAND AND CONTROL SYSTEMS N. P. Edwards Weapons Systems Evalwtion Group Washington, D. C. INTRODUCTION One of these also states " ... C&C system is a communications system ... "1 Under the' aegis of C&C, Electronic News reports that Mitre has developed ". . . a new approach, whereby computers will be provided with a broad, flexible program which contains nothing regarding the using commanders' specific problems. The program-which will be a 'learning program' based on artificial intelligence-will enable the COffi_ •• L __ L_ 1 ____ 1.. ________ ! ______ L1.. _ ____ _ _ _ 1. some time now I have been previleged to be associated with a project which has been taking a critical, and hopefully, constructive look at the workings of the National Military Command system. Working in the Pentagon one regularly hears the expression "Cost-effectiveness." Sooner or later someone had to combine cO!!lmand 'and control with cost-effectiveness. This paper then addresses the cost-effectiveness of military command and control systems. ~ pUL~I- LV I~CiJ.-l1 uy ~Av~.n~uc~ (:LO:S Ll1~ ~_ O:SyO:SLt::1U CVIU- mander feeds his specific problems into it."2 3. From this small sample of the current literature, it ean be seen that the expression "military command and control systems." has a wide range of meaning to different users, and covers a broad scope of systems and activities. If a meaningful attempt is to he made to discuss cost-effectiveness of command and control systems, it is essential to find out what we mean by command and control. So let us begin by defining the individual terms and attempting to break the problem of cost-effectiveness into manageable segments for examination. 2. Command and control is a broad and very active field as can be seen from the many articles in military and trade publications and in the ,popular press. For instance, the Armed Forces Management magazine devoted its July 1963 issue to command and control. ! would like to review some of the titles of the various articles: -"NMCS: The Command Backup to Counterforce" -"Total Command Control Through Computers?" -"Taylor's Flexible Response Strategy Shifts C&C Perspective" -"Electronic C&C Will Aid Army Commander, Not Replace Him" -"We Have More Than 800 C&C Systems" -"Resource Management: A New Slant on C&C" -"Real Time C&C Aids USAF, NASA" 4. "Command and control" as defined in the Dictionary of the United States Military Terms for Joint Usage (JCS Pub. 1) is "An arrangement of personnel, facilities and the means for information acquisition, processing and dissemination employed by a commander in planning, directing and controlling operations." Note that the word "system" is not used. 5. A "command and control system" as the term is normally used, includes many different 211 212 ~ROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 functions. In order to begin to seperate these so that they can be handled individually, let us differentiate between the comm'and function and the control function. We will first define and describe the characteristics of control systems: These are a class of systems sometimes included in the general definition of command and control, but which we propose to exclude from the subsequent discussion, except for use as a basis for analyzing the command subsystems we are really interested in. CONTROL SYSTEMS 6. As normally used in the technical sense, 'a control system regulates the action or activity of a person or device in order to perform a prescribed function. The key point here is that the mission to be accomplished is understood exactly and the means of accomplishing this end are also well understood. A control mechanism operates in accordance with precisely defined laws. It senses the performance of the controlled device and corrects deviations from the desired course in accordance with mathematical formulae. Control systems frequently have people in the control loop. An aircraft controller is an example of a human link in a control system. While the controller does not operate in a completely predictable mode, the majority of his actions are predictable in any except an emergency situation. In a weapons control system, of which SAGE is probably the largest single example, the majority of the functions can be described mathematically. Those functions which cannot be described mathematically are referred to human judgment. However, the ultimate goal is to eliminate the requirement for human judgment from the control system once the command to execute has been issued. Some systems, such as missile weapons systems' and antimissile systems, operate at such a high speed that it is necessary to eliminate the human operators from the control system loop. A distinguishing characteristic of control systems (as defined in common technical usage) is "feedback." This is true regardless of whether the system controls things or people. "Feedback" is the process by which a control system constantly senses what is occurring and adjusts the system to achieve the desired result. An aircraft controller directing the landing ap- proach of an ai.rplane uses feedback. He continually follows the actions of the plane and gives correcting instructions in case the plane is not following the desired course. 7. The diagram in Figure 1 is an oversimplification but it is intended to illustrate the dependence on sensors and feedback, and the prescribed nature of the action. COMMAND GO-STOP ANALYSIS CONTROL FEEDBACK SENSOR ACTION CONTROL SUBSYSTEM Figure 1. COST AND EFFECTIVENESS OF CONTROL SYSTEMS 8. Various control systems which are completely (or fo.r the most part) mechanized, have been in production for many years. Such systems include aircraft autopilots, instrument landing systems, missile guidance systems, tank and naval gun stabilizing and aiming systems, and many industrial process control systems. In the case of an industrial control system, such as a system controlling the operation of an oil refinery, cost and effectiveness trade-offs have been very carefully examined and are well understood. This is possible because the processes, procedures and desired end products are well defined. This is also the case in many small military control systems. Although the value of increments of improvement in the performance of a military control system may be difficult or impossible to measure, the value of an improvement in the performance of a public utility generating plant can be measured with great precision. An increase in operating efficiency of 0.1 % may represent a significant increase in the profit margin. ON THE EVALUATION OF THE COST-EFFECTIVENESS OF COMMAND 9. There are three factors which predominate in their effect upon the accuracy of a costeffectiveness estimate for a new control system. These factors are: value of performance, how well the function to be performed has been defined and the performance level desired. a. Value of Performance. Can a value be assigned to an incremental improvement in the performance of the system? In many instances it is possible to identify :a significant im·provement in performance and assign a value to it. Two performance factors of interest in the control subsystem of a weapon system are: (1) Reliability-If the reliability of the weapons control system can be improved, this improvement can be reflected in the overall reliability of the weapon. If the control is the least reliable subsystem, then the improvement will probably be quite significant. If the control subsystem is already the most reliable part of the system, then the improved reliability may have little effect on the weapon's ·performance. However, it might still be well worthwhile because of a reduced neid maintenance problem. The total system cost and increased efficiency must be taken into account. (2) AccuracY-An improvement in the accuracy of the control system may significantly reduce the number of weapons required if the target location is known to a higher accuracy than the accuracy of the weapon. Suppose the target location is known "exactly." If the accu.racy of delivery of the weapon is such that the target will always be within the radius of destruction, only one weapon (shell, iron bomb, or other) is required. If the error in aim and delivery is greater than the kill radius, we have a more expensive and probably less efficient situation (see Figure 2). The number of weapons required goes up logarithmically as the .probable dispersion gets significantly larger than the radius of destruction of the weapon. 213 IMPACT POINTS WEAPON EFFECT RADIUS AIM POINT TARGET CIRCLE OF ERROR PROBABILITY Figure 2. Formulae and nomographs exist for calculating the weapons required for a desired probability of destruction of the target. 3 If you cannot assign a meaningful value to incremental improvements in the system, a costeffectiveness projection cannot be made. However, analysis can still be used, and it still may be possible to make a relatively accurate comparison of the projected cost of different apmake a proper (or best possible) decision using the facts nlade available by careful analysis. b. Definition of Function. Are the functions to be performed by the control system defined and will they remain constant during the design, test, and production of the system? If so, one of the necessary conditions for an accurate cost projection has been met; If not, it is certain that the initial cost estimates will be significantly in error. The more poorly defined are the functions to be finally performed, the greater the probable error in the cost estimate. It would appear that it should be relatively easy to determine the functions required.of a control system. However, if the device to be controlled is being developed concurrently, and by a different engineering group or company, as is frequently the case, it is probable that control requirements will occur which were not anticipated. It is impossible to describe this risk numerically in the abstract, as each case will be different. c. Performance Levels. Is the system's performance specified? Will this performance re- 214 P-ROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 main essentially constant during the design, test, and production of the systems? If so, another necessary condition for a good cost estimate has been met. If not, the estimate has a high probability of being wrong. Is the performance required comfortably within the present state of art? If so, a major condition necessary to permit an accurate cost estimate has been met. If not, R&D is required and the estimate may be in error by factors of tens or hundreds. 10. Even where a new hardware/software system is of a kind which has been designed and produced repeatedly, accurate cost-effectiveness measurements or predictions cannot be made unless the value of what is to be accomplished is known, how it will be accomplished, and whether it C'an be done without R&D. This discussion has been included to illustrate the minimum conditions which must be met before reasonable cost-effectiveness projections can be made of any system or equipment. COST-EFFECTIVENESS OF "COMMAND AND CONTROL" SYSTEMS 11. For this discussion, three parts of the "command and control" function will be considered. These are more accurately called "Command and Support Systems." Figure 3 is a very simplified version of a very complex and personal activity, It is complex because it is an activity of sophisticated people in a complicated and changing environment, reacting to ambiguous situations which are inaccurately and incom,pletely reported. It is personal because each CO....ANDER COMMAND SUBSYSTEM Figure 3. commander has his own particular likes, dislikes, prejudices and means of operating. The commander will augment any formal "subsystem" we may build in any way he sees fit. The augmentation may be temporary or permanent (for the tenure of the commander) but it is not part of the subsystem discussed here. Realizing that we are discussing command support subsystems, let us consider each of the three selected which are: a. Data-Gathering Subsystem. A subsystem which gathers information necessary to support a command decision. b. Analysis Subsystem. A subsystem which provides or supports the analysis of the information. (The command function is not considered a subsystem.) c. Transmission Subsystem. A subsystem for the ,promulgation of the decision to the appropriate subordinates. 12. The function of command, i.e., evaluation of the information and the comm'and decision, is not addressed. This simplified diagram would emphasize the facts that: a. Commanders use many inputs, including feedback. b. The commander knows and relies on the intelligence and experience of the person responsible for executing the order. 13. Webster's New World Dictionary uses the phrase "Command . . . when it refers to giving orders, implies the formal exercise of absolute authority ... order sometimes suggesting an arbitrary exercise of authority ... " This is a clear recognition of the fact that it is not possible to describe rigorously the transfer function involved in the command 'process. This is true not only in general; it is difficult to think of a single specific command situation in which a series of logical conditions can be defined which describe the specific performance required of the commander. However, it is possihle to describe a few selected situations in which the correct performance can be readily predicted and completely defined. A doctrinal response is such a case. However, it is easier to describe one of the infinite number of situations in which the correct performance ON THE EVALUATION OF THE COST-EFFECTIVENESS OF COMMAND 215 cannot be completely defined in advance. Also, it will probably not be possible to determine if the decision was correct for days, or even months or years. can be made concerning timeliness, accuracy or reliability of the system. These are functions of the specific situation. Consider these major performance factors: 14. Since the com'mand -process itself cannot be described precisely, or for that matter with any degree of precision whatever, it is meaningless to talk about cost-effectivenesspredictions for the command function. Specific arbitrary situations can be defined (as is done in a war game), decisions postulated and the predicted results evaluated. This is useful to contribute to the learning process of commanders. It may also be useful to assist in evaluating a commander's thinking in the absence of a real war (certainly one would not want to leave the fate of a real military decision exclusively in the hands of a computer war game). While industry has for some years used a husiness game as a means of training and evaluating executive management, they do not leave real business decisions in the hands of this business game. a. Timeliness. How much is it worth to have the data a day, hour, five minutes, or ten seconds sooner? This is completely dependent upon the nature of the system, the nature of the situation, and the nature of the data. Given a specific data requirement, it is probably possible for an experienced military commander to put an arbitrary (approximate) value on the timeliness of the data. It is the commander's responsibility to make this decision since he is the one person who is held responsible for the consequences of his decisions. DATA-GATHERING SUBSYSTEM 15. Any cOfiifiiander (or business executive) needs information on which to base his decision. Military data-gathering systems have existed since before man first picked up a club. General Custer had a data-gathering system which provided him with adequate information to have prevented the massacre. His scouts told him that the Sioux were as numerous as blades of grass on the -prairie. It was in the data evaluation that Custer's command system was at fault. 16. Since some data-gathering systems are well understood, it should be possible to establish a cost-effectiveness comparison ,between various systems designed to achieve a specific performance. However, remember that this comparison is subject to all the conditions which limit the accuracy of cost-effectiveness estimates described above for control systems. It should also he possible to predict (with varying accuracy) the cost and performance of a given data-gathering system. However, it is certainly not always possible to accurately assess the value of increments of performance of a datagathering system. No generalized observations b. Accuracy. How much is accuracy worth in a data-collection system? This again is dependent upon all the factors listed above and also on the accuracy of the raw data and the quantity of the data. Given a specific requirement for data, arbitrary and approximate values can be assigned by the commander. It is not possible to do this in the abstract. (The accuracy of the system could be defined as the percentage of the data entered into the system \vhich arrives UDchanged at the output of the data-collection system.) c. Reliability. This could be d~fined as the percentage of the time that the system is performing in its normal manner. What effect will a temporary or permanent failure ha ve ? Here again, the finai answers are dependent on the specific system and can be assessed by experienced judgment only. 17. To reiterate, it is possible to compare the relative cost and performance of various datacollection systems designed to do a specific job. If the perform'ance is equal, or assumed to be equal, the relative cost can be compared. The problem is, proposed systems are, likely to be only approximately equal and then only where a system has been designed in response to a rigid set of specifications such as may be th~ case when contractors respond to a request for proposal. Even here a data-collection system is usually rather complex and the variations of system performance implicit in the proposals 216 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 are very likely to be of significance to the military commander. 18. Certain types of command situations permit a relatively accurate and profitable assessment of the value of timeliness, accuracy and reliability. Consider the case of a moving target with a known top speed. Knowledge of its exact present location is limited by the speed, accuracy and reliability of the reporting system. If we don't know of any restraints on its direction of travel, we must assume the target has a certain probability of being within a circle whose radius is determined by its speed and the age and quality of our knowledge of its last position. If we assume, for simplicity, that we have an accurate, reliable delivery system and a certain radius of kill, we can calculate the number of weapons which must be applied to the target area to give a desired probability of destroying the target. A method giving minimum overlap of destructive areas is the inscribed hexagon model. 4 The target area is covered by hexagons inscribed into radius of damage circles. If the radius of the target and the radius of desired damage are equal, one wea:pon is required. If the radius of the target is 5 times the radius of destruction, 37 weapons are required. (See Figure 4.) The number of weapons goes up as a function greater than, but asymptotic to, the square of the linear uncertainty as to the location of the target. This uncertainty includes, when you are estimating the number of weapons to stock: a. Reporting accuracy. b. (Speed of target) x (Probable reporting time loss). c. A safety factor to allow for the fact that the information you have may be older than you think (reliability of the reporting system). This analysis still will not give you the total cost of being effective, because it does not include such things as the 'probability of being able to deliver more than one weapon, time loss if weapons are delivered sequentially, or the reaction of the target if you miss the first time and get it excited, etc., etc. This simple example is virtually a control problem, but illustrates the point. 19. Assignment of absolute values to the total aspects of the performance of these systems requires the highest knowledge of the situation, the military strategy, policy and innumerable other factors relating to the national welfare. Hence, it is necessary that judgment be exercised in making the decision as to which system is selected. This judgment can never be made with full possession of all pertinent facts. It will always be subject to human error and can always be challenged. The military commander is held responsible for the consequences of this decision and, therefore, must have final responsiblity for making the decision. This does not belittle the contributions that can be made by proper application of analytical techniques. Careful analyses and cost projections of competitive systems can highlight areas of uncertainty as well as 'provide evaluations of known facts, and should greatly enhance the probability of the commander making the correct decision. ANALYSIS LAST TARGET REPORT POSITION Figure 4. 20. While the function of analysis has occurred in the commander's mind, with the aid of his staff, since the beginning of military action, it is possible to describe precisely only very limited aspects of the total field of information analysis for military command. In spite of this, much of the analysis requires the rapid and orderly retrieval of data from a large body of reference and current operational information. Facts relating to force status, logistics of supply and movement, geography, climate and installations are required, to name some of the more important. These must be stored in an ON THE EVALUATION OF THE COST-EFFECTIVENESS OF COMMAND orderly fashion. However, to support analysis, the facts must be related and sorted into many different orders or categories. Various manual or mechanized systems to aid this activity can be compared for cost, convenience, speed and other factors. Statements can be made regarding cost and relative performance. In some cases it may he possible to put a specific value on a difference in performance, but usually it requires command ex'perience and judgment to make the decision. And the decision as to the worth of a particular fact, array of facts in a category, or the worth of specific analysis may change drastically when the commander changes. Here, perhaps more than in any other command support subsystem, the effectiveness can be assessed only in the mind of the commander. What does he think he needs? SYSTEM FOR THE PROMULGATION OF ORDERS 21. Here again we have a function which has been performed by the military since the beginning of organized warfare. Given a specific situation, or a general requirement, it is possible to descrihe many different approaches to disseminating command information. These have included word of mouth, runner, carrier pigeons, bugles, drums, flags and rockets. Again, it is possible to com'pare the relative cost of various systems only if they have equal performance. Again, it is a matter of military judgment as to the value of improved performance in such a system. Accuracy is extremely important. Reliability, Le., probability that the command will be delivered, is of great value. The value of speed may be dependent in part upon the response time of the force commanded. A command transmission which takes seconds may be of no greater value than one which takes minutes where the response time of a military unit is a week. However, minutes or hours may be important even though the response time of a unit is days or weeks. It is possible to miss even a slow boat. To establish values of various system performance factors usually requires military judgment and cannot be generalized. Each planned application can be studied and in many cases definite, but not definitive, values can be attached to improvements in performance of some part of the sys- 217 tem. Other factors which cannot be assigned specific values, such as giving the order a greater sense of urgency or authenticity, may be more important. 22. It is worthwhile to single out the specific case of a nuclear exchange. This is certainly the most thoroughly studied of all the possible military activities which are not yet a matter of history. The nuclear exchange, 'once it is initiated, is usually considered to be more nearly a control problem than a straight command 'problem. It is bounded by doctrine,. a finite number of weapons and target choices. Command support subsystems with specific tasks can be evaluated for specific cost and relative effectiveness. A missile detection system can be evaluated-with existing missiles-and its performance against hypothetical new missiles estimated. An analyst can even assess many of the advantages of having a few minutes more warning time. But an evaluation of the dollar worth of this time is a task for the top military policy makers. Values can be assigned to degrees of reliability and accuracy. These can contribute to a sound decision, as they would in any other weaPQn of destruction system. In general these are like or are really control subsystem problems and can be ad~ dressed hy an analyst with considerable hope of obtaining some meaningful results. 23. The" command problem in the transattack and postattack period is mnre complex and probably less understood than conventional warfare. We don't have any relevant experience since fortunately we have never had an exchange. There are several questions we can ask about any proposed subsystem for command support: a. Can the system survive to do its job? b. Will there be anyone to use it? c. Will there be any data to feed it? d. Will it help? e. What will it cost? 24. Prior to this point an assumption has been made that hasn't been stated. It has been assumed that the subsystem or system component does something useful-it contributes to the better solution of the problem. This is the first question: Is this system relevant? The 218 PlROCEEDINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 answer to this may save a lot of analysis time and many procurement dollars. If the answer is "Yes, the subsystem is relevant and useful," the following general observations can be made: a. Two or more approaches to the solution of the problem can be compared as to cost and performance factors. b. With luck, one of the alternatives m!ay be both better and cheaper. c. If additiona) performance is offered by the more expensive alternative, the problem arises of assessing the value of the increment of performance. Even if dollar values can be assigned, there are always other less tangible operational factors which must enter into the decision. d. A choice may exist as to whether or not to have a control system (or a particular weapon or device), but no choice is offered in the case of a command system. The command systems exist now. So we are always offered a relative choice. We can use present methods or add support subsystems. These can always be compared with the existing way of doing business. In either event we ha ve the basic command structure. Command systems are not a question of hardware. e. In all system component choices we may be faced with the cost estimator's standard dilemma. System A may use a part of an existing system, while System B may have growth potential or reduce the cost of future systems. f. The analyses and quantifications which can be done using the techniques provided by the engineering and scientific communities are valuable tools which can assist the commander in making the best <;>f an extremely difficult situation. g. The final decision as to the cost-effectiveness of a military command and control system should rest with the commander who is responsible for using the system and for the effect that it has on the success of his mission and the national security, whether he is a component commander or the Com- mander-In-Chief of the U.S. Armed Forces. He should have all 'available help, utilizing the best analytic. techniques, in order to make the best decision. 25. The commanders of the Unified and Specified Commands have been, in fact, given the responsibility for their command and control systems by the Secretary of Defense on 26 October 1963, according to Armed Forces Management, January, 1964. The Commanders will be looking eagerly for help in solving their extremely knotty and complex command support problems. I believe that virtually everyone here considers himself a professional, and many of you will be called upon to help solve these problems. Let us take a responsible attitude, and recommend hardware and software only when it is relevant to the problem, and can contribute to the solution better than an improved manual system, and only then after we are certain that the improvement is worth the cost; and that the user wants the kind of solution we propose. And I mean the user, the man who will have to make the system work as a part of his daily job. Let us have the courage to 'say, when necessary, "Glamorous hardware and elegant software alone won't solve your problem." And then work with the people who have the problem, and together find out what the problem is. And perha'ps find the solution, which may even make use of good hardware and software. Our professional ethics and pride require it. REFERENCES 1. ROBERT G. PFEFFERKORN, "Taylor's Flexible Response Strategy Shifts C&C Perspective," Armed Forces Management, Vol. 9, No. 10, Page 35, July 1963. 2. RAY CONNOLLY, "System Evaluation is Underway at ESD," Electronic News, Vol. 8, Page 22, March 18, 1963. 3. JOSEPH S. TOMA, USAF, "Probability Applications to Weapons Systems Analysis," AFSWC-TDR-62-59. 4. Ibid., page 29. FRACTIONIZATION OF THE MILITARY CONTEXT Dr. Frederick B. T h01npson TEMPO, General Electric Company Santa Barbara, California mand and control are the problems of maintaining adequate, relevant contexts under changing conditions and problems, and of insuring consistency across the varied and changing contexts of the members of the organizational hierarchy. It is on these problems of contextual dynamics that I would like to/focus your attention. INTRODUCTION Every subject is said to have both an inside and an outside. Its outside has to do with the relationships between the subject and matters external to it; while its inside is concerned with the subject's own internal dynamics. My talk concerns the dynamics of information in today's institutions with specific reference to the problems of command and control. Thus I shall leave questions concerned with the relation of COillmand to strategy and the nature of future wars to my colleagues who are well prepared, as I am not, to discuss them. THE SELF-ACCELERATION OF CONTEXT Our contexts tend to grow. The more we know, the greater our curiosity, the more the remaining unknowns provoke us. Each new discovery seems to create more problems than it solves, problems which beckon us to more complex, more extensive theories and understanding. This quest for knowledge is not only for facts concerning the external world, but also for a better understanding of ourselves, of our wants and objectives, and of the relative values to us of the alternatives we perceive. Context is ever-expanding. Throughout my remarks I shall make a great deal of use of the word "context." And so I shall first clarify just what I mean to convey by that word. I shall mean by context the total view a man, or a group of men, has of his situation at a given time. Thus context includes knowledge of environment, historic perspective, plans for action, motives, goals, all aspects of our understanding of the what, how and whys of our current situation. The context of a military headquarters is rather well-delineated. It includes the personnel records of the command, intelligence, operational information such as status of forces, logistic re ~ords and plans, operational plans, communication procedures, and certainly a clear understanding of the command's mission. If our contexts were limited to those which could be contained in the head of a single individual, our knowledge would be limited indeed. We have learned to build communities, communities whose principal function is to build and maintain a group context which exceeds in information that of any of its constituents by many orders of magnitude. Specialization can be viewed as a fractionization of the group context in such a way as to maximize the total information iri a community that can be brought In its broader meaning, command and control are the means by which an org--d.nization maintains adequate and interconsistent contexts for its constituents. The inside problems of com- 219 220 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 to bear on its various concerns. And the essence of organization is to insure intraconsistency across the group context while maximizing its informativeness. Our motivation for this insatiable quest for ever greater knowledge is clear. The more we know, the more efficiently we can modify our environment to meet our desires. Certainly our current standard of living is ample evidence of what the expanding human context can mean to us. But if expanding context brings greater control over our environment, it also extends our ability to gain knowledge itself. The great extensions of context made by Copernicus and Galileo and the observations of Tycho Brahe, opened the door for the kinematics of Kepler. In turn these advances showed the way for the dynamics of N ewton, and so on through the discovery of the Van Allen belt and solar winds. This same effect is equally applicable in the small as in the large. When a boy learns to ride a bike, this new knowledge vastly increases the world he can explore. Contextual growth is self-accelerating. Look at .this fact very carefully: Context is self-accelerating. If context were limited to a single individual, there would be a ceiling on contextual growth and this acceleration would be controlled. But by building larger and larger communities, we have removed this bound on contextual growth. If the community context were to grow steadily, one could conceive of an ever-enlarging culture. But the self-acceleration does not permit steady growth. As the rate of growth of knowledge increases, as well as its overall extent, self-consistency of the total construct becomes strained. Communication across the context cannot keep up with the new discoveries and conceptual extensions taking place on all sides. The feeling of being out of touch rapidly deteriorates to a fractionization of community identification. The basis of communication, the commonality of context, disintegrates. Castastrophic fractionization of context ensues, and the result is the plummeting of the amount of information remaining in the community. The symptoms of contextual fractionization are many. Among them are, for example, a feverish preoccupation with the problems of military command and control, attempts to centralize control of departmental or corporate informational activities, acceleration of organizational change, greatly exaggerated evaluation of communication activities and increase in "paperwork" of all kinds, and standardization by fiat, rather than by community recognition of common practice. And the final desperate stage to secure the shattering informational community is dictatorship. Is such catastrophic fractionization of our communities inevitable? I do not know. Certainly the rea,ding of history gives us pause to wonder. But there are ways of dealing with the problem of contextual explosion. Context must fractionate, because the strains on communication do become excessive, strains which build up faster than technological advance can relieve them. But we can achieve orderly fractionization. The constituent communities can be organized to retain through the organizational hierarchy the means to bring to bear a total body of know-how and understanding not achievable by anyone element of the organization. Gentlemen, the Department of Defense is tending toward catastrophic fractionization. The almost explosive rate of contextual expansion is all too clear. Intercontinental missiles are the result of global concerns and scientific advance, not vice versa. And the rate of expansion of the total defense context, indeed, of every facet of that context, has become explosive. There is no reason here to recite examples of the disintegration of communications, of inconsistencies of policy, or of the increasing difficulties of initiating definitive actions. We are all too aware, each in our limited sphere, of this fractionization. And even the newspapers and trade journals confirm our fear that it may be widespread. Those of us gathered here represent a sizable part of the community that has both means and mission to reverse this rapid decay in overall information in our monolithic organizations. We can assist in the achievement of ordered fractionization that can in turn increase the potential for information of today's institutions, and thus our potential for decisive, efficient action. But it is not in the increase in commu- FRACTIONIZATION OF THE MILITARY CONTEXT nication, the centralized data centers or the paternal agencies established in the name of efficiency, that this reversal will be achieved. Efficiency in the informational activities of a community come from entirely different sources. Before discussing specific approaches that our industry might undertake to give effective service at this critical time, I would like to take an~ overview of the kinds of measures a community can take to accommodate its expanding context-measures I believe the Defense Department must take if it is to maintain an adequate defense posture for our nation. STEPS TOWARD PREVENTING CATASTROPHIC FRACTIONIZATION Organize to Minimize Communication Requirements The task is not that of maintaining a single, all inclusive context where the centralized Secretary's Office has all the details of its far-flung operations at its fingertips. No single context can accommodate it. It is not just that no one man can take it all in a single thought. It is simply impossible to put such a context together no matter what resources are brought to bear. The means of communication are not adequate, nor can they be made adequate, to maintain the consistency of such a context whose rate of .change in every facet is as high as it is today. The essence of traditional command doctrine is the delegation of command prerogatives to the responsible officer on the scene. Let us examine this doctrine for a moment. What do we mean "on the scene?" In a task force operation the man on the scene is the task force commander. He is directly in touch with his subordinate command elements which he controls and maneuvers in· the accomplishment of the task force objective. However, if we consider one of these subordinate units, say a combat ship or combat wing, the commander of that unit is on the scene relative to its control and the coordination of its operation. In turn, for the operation of the ship's power plant, the engine room officer is on the scene; for the operation of a particular aircraft, the aircraft commander is on the scene. 221 A military operation - any operation - directly deals with reality; not with hypothetical plans, nor with far-flung systems which can never be fully sensed or directly experienced, but with individual men and individual items of equipment, and individual reports and individual points of view. This reality is strangely peculiar to the particular operation. It may be an operation at the highest level, such as dealing with Rusk and McNamara, Khrushchev and De Gaulle. It may be an operation at a middle level, such as the execution of a blockade of Cuba dealing with particular ships, the personalities of well-known officers and with weather conditions you have personally tasted on many a watch. Or it may be an operation at the lowest level such as knowing instinctively the coverage pattern for a radar and the kind of returns to expect, and the sense to be able to distinguish the moon from incoming missiles. Traditional military command doctrine gives the responsible officer, who is in the position to directly sense the reality, the means and the prerogatives to deal with that reality "on the scene." The essence of this doctrine is twofold. First, the responsible officer on the scene is in the best position to have the relevant information. Second, the delegation of prerogatives assunles responsible reporting of the relevant facts upward. As a result, the 1'equi1'ements for communication are reduced to a minimum and the relevance of communication is maximized. To attempt to obtain at a far-away headquarters the essential details, the peculiarities of an operation, and to exercise detail control from a distance puts impossible strains on communications. I do not mean by "communication" solely the transmission and reception of signals, but all that is implied when we say two men are communicating, one to the other. Those who think that this problem can be solved by great automatic data systems-and there are many people who think this in our industry-have no appreciation for the complexities of military operations, though they would decry the difficulties of debugging a simple computer program by telephone. The result of such systems is inevitably the inundation of the higher headquarters with noise and the rapid deterioration of the effectiveness of the subordinate. 222 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 The notion is also popular that one can centralize a portion of an operational task in a central agency, far removed from the realities that give meaning to the subtask in the first place. Thus, for example, major functions of military intelligence and military communication are centralized for the sake of efficiency. But the sources of feedback and the measures of relevance germane to the operational realities and requirements are missing in such centralized agencies. The relevant measures, heard only indistinctly, are all too easily replaced by feedback from the local environment which are all too often at cross purposes with the needs of the field. One of the control centers of a unified command wished to move a telephone fJ;:om one side of an office to the other. I t took two years for the paperwork to be processed through Washington before official permission was given. Of course, the phone was moved long before that. But it is precisely the inconsistency as well as communication strains which result from such foul organization and failure of delegation with which we are concerned. An equally ridiculous example is that a major military command now is in the process of putting all. their personnel records for the last ten years on punch cards and sending them to Washington. Washington may be able to process punch cards very fast, but any manager who has had to match man to job and stay on top of the personnel matters of his employees can imagine the reduction to primitive levels that managers on the scene will be forced to employ and the stratagems that will be used to keep Washington permanently misinformed. To delegate operational tasks to men on the scene calls for attention to the kind of tasks which are likely to occur, the kinds of control which must be maintained at higher headquarters and strategic considerations. These are the outside aspects of command and control. But once these are properly assessed, the inside principle must be followed: organize and delegate to minimize the volume and maximize the relev!Lnce of communication. And the key to this principle is to assign responsibility and prerogaUves to the man on the scene. To Stay Consistent, Stay Close to Reality One of the greatest deteriorating factors in any organization is incipient conviction on the part of a constituent that 'others with whom they must deal are out of touch with the realities of the situation. If the higher military headquarters appears to be functioning on the basis of assumptions that the field knows by its direct observation to be false, or even if the field only believes them to be false, the field will inevitably respond by subverting the information structure to its own ends. If the higher headquarters loses confidence in the meaningfulness of the reports it receives from the field, it will ignore those reports in making its decisions. The result is rapid collapse of communication, a collapse that cannot be detected in bit rates. The problem of maintaining touch with reality is greater now than ever before, for we have so few bench marks on which to lay our conceptions of military operations. We had not had operational experience under full scale war time conditions with weapon systems and unit organizations with which we are now equipped. There is no underlying body of experience that, among other things, assures the consistency of view that is so necessary to maintain the cohesion of the military organization. Thus we must insist with uncommon concern and tenacity that the operational realities of which we are cognizant are kept firmly in mind, let the chips fall where they may. In many cases, the knowledge of our senior military officers concerning what is actually going on in their commands is simply inaccurate or misinformed. It is generally accepted that the SAGE command and control system led the way in the development of modern automated command control techniques; that SAGE was the great prototype of automated command and control. The indications are, however, that many SAGE sites are not exercising their equipments from one inspection to the next, that some have even put cellophane overlays over their scopes and returned to essentially manual operation. What confidence can we put in the development of future command and control systems when SAGE is named as the great prototype? But it is not only that our higher headquarters are net looking. Of much greater concern FRACTIONIZATION OF THE MILITARY CONTEXT is that we are ignoring at all levels realities all too apparent to the analyst who will but unflinchingly layout the known facts. How many systems, organizations, bureaus and military commands live on when both they and their missions are obsolete? What is the degree of collusion in the conduct of military exercises? What studies have been buried? Who has not felt too often the pressure to justify rather than establish, and witnessed too many contrived demonsrations? We are not looking; we are not probing below the surface. But we have found a substitute: the operations analysis that purports to "simulate" military operations in a computer or establish formulas for determining the cost effectiveness of weapon systems. Operations analysis can be useful, but only if soundly established on ~dequately developed experimental evidence from the field. It is important that those who are applying the results of such analyses at the strategic level understand our technical deficiencies in the basic and crucial assumptions of these studies. A few specific examples of such deficiencies are: • there is no adequate method of discounting over the long haul in cost/effectiveness studies; • economic comparisons or economic resource cost estimates involving the Soviet Union depend on an adequate analysis of the ruble/dollar ratio, which has proved intractable to date; • the "soft" effects of nuclear weapons on population, communications and organization appear to dominate the "hard" effects, yet we have no adequate means beyond intuition to project these "soft" effects; • there is no adequate method to assess the effectiveness or the effects of deterioration of combat communications of any of the military or diplomatic services. Yet these deficiencies lie across the very roots of every major operations analysis today. Eight years ago I participated in a study to determine whether the results of a certain class of effectiveness studies were sensitive to the 223 underlying parameters of the studies. We found that the degree of sensitivity, relative to our knowledge of the parameters, reduced the significance of the results to a point where their usefulness was in grave question. I recently examined this same area to see what improvements had been made. I not only found the same practices, but the same reports which I had presumably established as inadequate were quoted as the basis for current studies. Is it not the case that our studies grow from roots that penetrate only as far as the intuition of the analyst? One of the best known and respected operations analysts once showed me his notebook where he wrote down prior to undertaking a study what results he expected to prove. He was very proud that he had never failed to predict his results correctly. If I were to be asked to choose between the expertise of a Ph.D. mathematician and a flag officer of our armed forces in matters of mathematics, I would pick the mathematician. As a mathematician, I do not have the arrogance to hold that the mathematician's "guesstimates" and intuition should hold sway in the arena of military operations. I am not here decrying lack of objectivity as such, or pointing in alarm at malpractice. What I am saying is that if we are to reverse the catastrophic fractionization of context which is occurring in the Department of Defense, we must insure by all available means that consistency which can legitimately be found. The best source of consistency is reality itself. We must get out of our nonprofit ivory towers, administrative labyrinths, and self-mesmerizing brochures. \Ve must insure that our context matches the reality that is there, if we bother to look. Experience must again be valued, the man on the scene respected, and the hunch heeded that tells us we had better drop. in unexpectedly to get a first-hand look. The Importance of Purpose Modern communication theory teaches us that information can be measured in terms of our relative uncertainty concerning a set of alternatives. What it does not tell us is where these alternatives come from in the first place. Once the alternatives are clearly defined, then that part of a communication which does not 224 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 properly reduce our uncertainty is identified as noise. However, if we were to change our selection of alternatives, what was once noise may become highly informative communication. Thus the selection of alternatives is a crucial aspect of obtaining information. However, it is in the selection of alternatives that we often go astray. It is far too easy, when we come across a new facet of a situation, to either make simplifying assumptions related to the techniques of the analysis rather than the problem, or to simply ramify the alternatives, establishing new requirements for data. In either case, communication theory gives us neither reason nor warning for the resulting dilution of relevance. We pay homage to systematic methods, and, like the physicist who ignores viscosity in favor of Newton's laws, forget about the sticky aspects. We must be sure to get all the hard facts; to distrust-indeed fear-the man who is outspokenly biased. We dare not make judgements of relevance, and the only limitations on alternatives are those made in favor of allowing more precise analysis. And as a result, we are faced with the paradox that since there is nothing we dare call noise, it is all noise. I know of one office of the Defense Department that has stored away over 7000 magnetic tapes, each containing a million and a half words. They never have and never will have the capability to look at them, yet they are reluctant to throw them away. The selection of alternatives is where our purpose is revealed, where our measures of relevance are established. It is only when there is clear purpose that there exists the means for establishing relevance. And it is only through a sense of relevance that we can winnow out the essential elements of information from the swirling, encompassing chaff of data and reports. When we have our purpose clearly in mind, then the one or two alternatives that we actually have stand out clearly in front of us. What is purpose? Purpose is not something set down in writing under the subhead, objectives; nor is it the first few paragraphs of a justification paper. I imagine that each of us has on occasion sat with tablet and pencil before us trying to put into a position paper or proposal a meaningful statement of purpose, and finding our words empty and unconvincing. Purpose is not expressed that way. Purpose is only truly meaningful to the man on the scene who has at his command the means of accomplishing something and is caught up, engrossed, in getting a task done. Such a man is not objective; indeed, he is vehemently biased. But this is the kind of bias we so desperately need. He ignores most of the facts, and in so doing has time to attend to his job. He is too preoccupied to fill in all the data forms; but when he communicates, the message both demands attention and clarifies by its clear relevance. Purpose is the man on the scene, struggling with a task, who can get direct feedback from his actions, who has at hand the measure of his accomplishment or failure. I do not think you can separate the two. Separate the man from the means for accomplishing his task, and purpose goes also. Remove him from the scene, from the source of direct feedback, and you remove purpose as well. In designing information systems for command and control there seems to be no attention given to conveying purpose. How is purpose conveyed? Purpose is conveyed by saying what the alternatives are, not by evaluating their cost effectiveness. Purpose is conveyed by saying what the measures of effectiveness will be. Purpose is conveyed by asking questions, not for the sake of the answer, but to be sure the man questioned got the gist of your order. Purpose is conveyed by leadership and follow through. The systemization of human knowledge has grown to such an extent that we find ourselves entrapped in a great web in which it seems that any strand we pull ties in with all the others tugging us in ways we did not anticipate. We tend to replace the question as to which way we wish to go by the question as to which way we can go. And we assume this latter question can be solved by simply getting our facts untangled, tracing through the scientific web of cause and effect, and thus clarifying the old alternatives and their cost effectiveness. The realities with which the military must deal today a're beyond such scientific explana- FRACTIONIZATION OF THE MILITARY CONTEXT tion. I am not talking about the "intangibles," nor about the role of "judgement," nor any of the other cliches used to justify intuitive behavior. The self-acceleration of context, in particular of our scienti~c/military context, has reached the point where changes are taking place in the system faster than our ability to properly take them into account. I have stressed above that fractionization is occurring, and the essence of fractionization is that overall rationalization of the total system becomes more and more impossible. If science is inadequate for the direction of our military programs, by what should we supplement science? Under these conditions of fractionating context and decay of overall informativeness we must find some means to reestablish relevance and spur and inspire purposeful action with a unity of direction. Our current attempt to gather all possible information into one place and then use scientific methods to reduce it to a point where deCisions can be made is resulting in just the opposite of what is req,uired. Indeed, the very cry of our operations analysts for clearer statement of requirements points up the almost total loss of measures of relevance and sense of purpose. Let us recognize that some men have a better sense than others for where we are going and what can be done about it; and that some men have the capability to inspire and lead. Just as we look for the finest intellects to become scientists, ,\-ve should similarly look for and esteem our men of judgement and our men of leadership. How much money are we spending today to encourage those who rate high on the college board examination compared to the incentive we are giving to courage of conviction, personal initiative, and the qualities of leadership? We do not breed leadership by centralization of decision making, by defiling the image of those who have achieved prominence as leaders and men of judgement by casually reversing their decisions, by giving precedence to the mathematically encapsulated intuition of men whose judgement has never been tested in operational situations, by occupying the time of our field officers with routine paperwork and the exclusion of honest tests of their effective- 225 ness in successful operation and sustained operational preparedness. It is in the practices of command and control that we determine the quantity and the quality of judgement and leadership that we shall be able to call upon in the future. It is the practices of command and control that allow the quantity and quality of judgement and leadership we now have to be exercised effectively. And it is the quantity and the quality of judgement and leadership that will determine whether the exploding context will fractionate in an orderly fashion, giving us the defensive capability that will sustain our great country, or will hasten the catastrophic fractionization that is now rotting our defensive capability from within. To summarize: • the requirements for communication must be minimized and the relevance of communication must be maximized by placing the prerogatives of command and the responsibility for reporting upward in the hands of the responsible official on the scene; • the interconsistency of context must be maintained by keeping close to reality, by delegation to the point where feedback from the real world can be felt, by follow through in insuring that things are as they are reported to be, and by facing up to realities that we know but are now ignoring; • relevance and purpose must be stimulated by attention and esteem for judgement and leadership and by the stimulation of these qualities by allowing them to be exercised at all levels of command. These three principles are the central issues of command and control today. Our attention to them will mark the success or failure of the establishment of effective command and control across our military organizations. Do not be misled that the only problems that remain are how to automate decision making, store and retrieve documents or harden communication centers. Let us turn now to these central issues and what our industry can do to resolve them. 226 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 IMPLICATIONS FOR THE COMPUTER INDUSTRY other measures in deciding the appropriate way to apply computers. Exploitation of the Adaptability of the Co'mputer Further, when the problem that the computer programmers and computer engineers see are so very large and complex and so distant from the underlying uncertainties giving rise to the computation, the idea that the data are suspect and the answer only an indication of what new question to ask, immediately requiring a new program, or second guess at inputs, is foreign, indeed. Thus the adaptability of the computer, which makes it such an ideal tool of conceptual exploration, has been downgraded. Little has been done until recently toward exploiting the other half of the computer's great capabilities. Indeed, relative to tot?-l expenditures or total population in our industry, little is being done today. The development of the digital computer has provided us with a machine whose combination of adaptability and computational power is way beyond anything previously available. As a means of manipulating great q,uantities of data and carrying out extremely complex data processing, the computer opens whole new avenues by which we can bring to bear distant observations on current concerns. Furthermore, the high speed, stored program computer has the potential of placing these capabilities at our fingertips. However, the exploitation of these two great virtues of the computer cannot be carried out independently. There is a trade-off between the computer as an adaptable instrument and the computer as a data processing instrument. To make the most efficient use of the computer as a data processing instrument requires efficient programming, minimizing of housekeeping and input-output functions, and careful scheduling of the computer's utilization. To make the most efficient use of the adaptability of the computer requires, in contrast, that the computer respond directly to the sometimes ill thought out instruction of the user, with much time spent on input and output to deal with the elaborate redundancies and conventions by which we normally communicate, and that the computer essentially wait ·for the user rather than the other way around. Now the principal uses of the computer today are in applications where the maximization of its data processing capabilities are required. It is indicative to recall that the biggest sources of funds by far for the development of computers have been the Atomic Energy Commission and the data handling aspects of military intelligence. In both cases the emphasis is almost entirely on speed of processing. Even the output required is in the concise and sophisticated language of mathematics. No wonder the current generation of programmers and computer engineers place efficiency of computation and efficient use of the computer ahead of all In particular, our command and control systems are not responsive to the rapidly changing environment, capabilities and objectives of the user. In fact, the principal result of all the push in command and control to date is the proliferation of organizations inserted between the ultimate user and his computer. There is the EDP office of the using command, his counterpart in the procuring command, the system monitors, the contractor's project office and finally the programming and design people. I t is in the interest of each of these offices to seal the system requirements years before its operational date and see to it that the most efficient-and thus necessarily the most immutable-configuration of programs and equipment are provided. For example, soon after the initiation of one of the Air Force L systems, it became evident that the requirements stated in the request for proposal (by then 15 months old) did not take cognizance of the user's trends and plans for reorganization. The system contractor duly recommended a rather complete reorientation in his effort. Not only was this recommendation ignored, but 10 months later, 6 months after a reorganization of the user that made the work statement obsolete, the system contractor was required to deliver a detailed systems design and establish by detailed analysis that it filled, but did not go beyond, obsolete requirements of a nonexistent organization. FRACTIONIZA TION OF THE MILITARY CONTEXT In the first part of this talk, I stress the growing problem of catastrophic fractionization of the context of the Department of Defense. I am convinced that a by no means insignificant contribution to this impending catastrophe is beingmade by past and current efforts in command and control systems. The huge, inflexible data systems with their insatiable demands on the field for irrelevant, often unused data, the cumbersome automation of decision processes whose development time cannot keep pace with changing requirements are contributing not only to the inflexibility of our defense posture, but immeasurably contributing to the inconsistencies they are supposedly resolving. There is nothing easier to corrupt than a complex, highly formalized data system. And there is nothing for which the coercions to corruption are stronger. I shall never forget the conversation I had some years ago with one of the leading contributors in our industry. At the time he was at a Naval shipyard, in charge of their computer installation. He described the problem. there as having nothing to do with efficient methods of inventory control or production scheduling. Rather their then current data system had been so completely corrupted at the grass roots level that one could place no reliability on any available data, and as a consequence, the yard management was in a very real sense helpless in face of the working level control of shop foremen. How much farther has this gone today? The real damage, though, is not being done by the command and control systems that are working; there are too few of these. It is the great promises and strange delusions that in a centralized data system we have the panacea of all our command and control problems. Mr. Gilpatric, when Assistant Secretary of Defense, said, "The machinery for gathering, analyzing, and presenting the data necessary for decisionmaking has, due largely to the extensive and imaginative use of automatic data processing by the military, advanced to a point where centralized decision-making is both efficient and effectiv.e." Mr. Esterly Page, Director, National Military Command System (Technical Support), repeating the above quote from Gilpatric, went on to elaborate on the choice that has been made: "During the past two years this nation 227 has undergone a basic change· in the philosophy of command and control which has resulted in new enlphasis on the requirement for large quantities of finite and precise information to form the basis for consideration at the highest level of problems that previously were considered and unfortunately decided at a much lower one." Like the wooden walls of Athens, computers may be the key to our survival. But what happens when those high up in the lonely citadel find they chose the wrong wooden walls, and the ones they chose have been rotted a way by the proper corruption which seeps in when the traditional coercions for responsibility have been taken a way? However, there is a bright spot in this otherwise disturbing picture. The Advanced Research Projects Agency is supporting a number of well-chosen programs for the development and exploitation of the adaptive capabilities of computers. Certainly one of the most ambitious of these programs is Project MAC, undertaken by the Massachusetts Institute of Technology. The Air Force is also turning toward developments along somewhat similar lines, as was stated by General Terhune 3 at the Fall Joint Computer Conference in Las Vegas. * There are, as well, programs sponsored by other agencies, both public and private. Forexample, the commercially supported project undertaken by the Control Data Corporation concerning computer .supported engineering design. These projects have as their general objective making the computer directly available to the individual researcher, analyst, and staff officer, as an adjunct to his creative thinking. The technical objectives are threefold. First, by means of remote consoles, interrupt features, and associated software, to provide a number of users direct and parallel access to the computer. Second, by means of problem oriented languages, including graphic and natural languages, to provide access to and response from the computer in a form that is natural and effi* This point of view was anticipated by "A Concept for the Navy Operational Control Complex of the Future," by R. W. Callan and F. B. Thompson,! and "Fundamentals Underlying Military Information Syst3ms Design," by F. B. Thompson, a paper presented at the First Congress on the Information Systems· Sciences, November 20,1962. 2 228 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 cient for the user. Third, by organization and advanced concepts of software logic, to provide more and more intelligent responses to an increasing variety of possible requests. These capabilities will take the computer from down the hall and behind the counter, and place it as a direct adj unct of the staff officer. Command and Control Implications of Direct Access Systems The general nature of these technological developments is probably known to you. What I should like to bring into sharp focus here is the implications these technological developments hold for the central issues of command and control. A computer as a direct adj unct of the staff officer is a very different thing than a computer as the central instrument of a great data system. Those who speak of direct access systems often describe how they will allow the staff officer to make use of some central data base, and the implication seems to be that the staff officer will have direct access to the great assemblage of fact that now stands assembled but inaccessible by the great data system. This is a false view. Acces~ to a base of fact can be meaningfully accomplished only if the user has a feel for the relevance, the source, and the units of those facts. In a very real sense, they must be his facts, his data. We too often assume that the number of missiles in Cuba, or the number of atomic weapons we could deliver to enemy targets are sufficiently well defined when in fact their uncritical application by the uninitiated is fraught with danger. This point needs to be stressed. Meaningful data is a fragile, subjective thing. The intelligent analyst knows this when he insists on knowing the use that is going to be made of his estimate before he is able to produce it. The contracting officer knows this when he refuses to allow even a unilateral passage of information during deliberations on a source selection. The combat commander knows this when he gets out among the troops to sense the situation on the spot. One can reasonably and responsibly use data only if it is one's own data. There is a second aspect of this matter of the data base underlying these direct access systems. The amount of data that could be ac- cumulated in, say, a major command headquarters is astronomical. The choice of just what data is to be found in the system thus becomes an important command decision in itself. That choice embodies the sense of relevance of the headquarters. Further, the implementation of that choice by directives on subordinates and requests to lateral and higher headquarters, is a principal means of conveying purpose and establishing direction. The headquarters whose reporting requirements are static and allinclusive gives up one of its principal tools of control and opens the door to subversion of its leadership. It is the direct access system that can give new vitality to data as a sensitive instrument of command and control. Therefore, the implication of the technological development of direct access systems is that there will be closely knit staffs addressing relatively small but dynamic bases of highly relevant data as they deal intensively and purposely with problems local to their scene. Such direct access systems cannot be reasonably justified on any other grounds. Thus the technological development of the computer as an adaptable instrument, responding directly to the requirements and local context of the user marks the beginning of the counter trend. Not only are the continuing failures of the automated information systems becoming more and more obvious, but the seeds of a new.type of system suited to the needs of the purposeful headquarters on the scene are being sown. The catastrophic fractionization of the Defense Department is being accelerated by the obsolete centralized information systems. It is the latter, the system that can be used responsively by the purposeful staff, that can bring the necessary vitality and responsiveness to orderly fractionization. The Promise and the Needs of the Highly Responsive Staff Throughout this paper, the underlying problem has been how to deal with the exploding military context. We have emphasized (1) the need to place the prerogatives of command and responsibility for reporting upward in the hands of the responsible officer on the scene; (2) the need to maintain a direct hold of FRACTIONIZATION OF THE MILITARY CONTEXT reality; (3) the need to establish purpose and measures of relevance through leadership. How can this be done, say in the environment of military command? The essence of orderly fractionization is that the units of command must be in a position to maintain a viable context, a cohesive and consistent view -that integrates their capabilities, environment and objectives in a way that is responsive to the reaHties with which they must deal. The commander and his staff must know what he wants to do, what it will take to do it and what he has to do it with, and they must have a sense of readiness that his objectives, his requirements and his capabilities add up with possibly a margin of capability in reserve. Noone of these three ingredients of his context-objectives, situation and capabilities -is given to him immutable. He must be able to modify his detailed objectives, reassess his situation, reorganize and redeploy his capabilities. That is, he must maintain the consistency of his context of command. But more than that, he must make sure of this intelligence, test and exercise his forces, prod and react to his superiors; he must maintain the validity of his con~ text of command. This maintenance of context is the essence of staff work and the foundation of command and control. The orderly fractionization of command is tantamount to the establishment and reestablishment of a hierarchy of command in which each headquarters can maintain such a context peculiar and appropriate to its position in the hierarchy. As the rate of expansion of overall context increases, this hierarchy of command must expand and deepen. Each element in turn must increase the efficiency and maintain the relevancy of its context. If this is' done there will indeed be fractionization, for the contexts of forward elements, in touch with the rapidly shifting local situations, will responsively shift and move in ways the contexts of higher elements cannot moment by moment respond. These higher elements, however, operating at higher levels of abstraction, will follow at their appropriate rate, their appropriate realities, of their appropriate scene. Fractionization of the overall context inevitably will occur. 229 Let me restate that in the bluntest possible terms: In the Department of Defense today, inconsistencies between the contexts of the various elements of the organization are inevitable. It is the character and source of these inconsistencies that must be controlled. If that fractionization is realized in terms of the responsiveness of the disparate elements of the command hierarchy, it will accommodate greater and greater responsiveness to change, and permit an orderly control of a vast and vastly informed organization. Consistency of overall context cannot be maintained in the face of change. But orderly fractionization of context that places the prerogatives of control at the point where the sense of change is most meaningful, that insures the degrees of change will be felt at the appropriate level and that inserts purpose into change, such orderly fractionization can permit our organizations to live a'nd grow and prosper. In Surn1nary Thus the crucial informational problem is the maintenance of relevant, consistent, valid context in each separate, disparate organization or military headquarters. This is the central task of staff work, the essence of command and control. It is in the perspective of this task that our information systems must be planned and developed. The appropriate information system must be a direct tool of staff, as natural and direct to use as the te~phone. The data held by the system must be put there by the staff, must be the intimate changing product of the staff, the factual base of their context whose relevance is their most crucial task. And that system must be the staff's system, not a universal system, not a standard system, but one that is recognized as what it is, inconsistent with all others, to a degree irrelevant to all others, and thus free to be responsive to the sense of relevance of its staff, the purpose of its commander or administrator. From such a context, communication will be relevant. And this communication, by its very shifts and changing emphasis, will convey purpose and direction as well as fact. This is a very different kind of system than the great data systems and automated control systems. It is a system that allows direct access in natural language, it is a system whose files 230 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, and formats, whose inputs and vocabulary are always changing, continuously changing. The Department of Defense is tending toward catastrophic fractionization. The reversal of the process of this explosion may still be possible. It rests in the hands of two groups: First, those in control of d~fense policY, in particular those who are determining the policies for military command and control. Second, it rests on us, for it is our industry that can turn to the development of systems that will exploit the adaptability of computers and will place them as the tools of staff in maintaining relevant, responsive contexts of command. The challenge to realize orderly fractionization is very real and urgent today, a challenge to which we must respond. 1964 REFERENCES 1. CALLAN, R. W., and F. B. THOMPSON, "A Concept for the Navy Operational Control Complex of the Future" (General Electric, TEMPO, RM62 TMP-49, July 1962). 2. THOMPSON, F. B., "Fundamentals Underlying Military Information Systems Design" (General Electric, TEMPO, SP-183, November 1962), to be published in Military Infor1nation Systems, edited by Edward Bennett (Praeger, New York, in press). 3. TERHUNE, T. H. An Address to the American Federation of Information Processing Societies, Fall Joint. Computer Conference, Las Vegas, Nevada, November 12,1963. SOME OBSERVATIONS CONCERNING LARGE PROGRAMMING EFFORTS Almon E. Daniels Weapons Systems Evaluation Division Institute for Defense Analyses INTRODUCTION The requirement to construct large systems involving both men and computers to assist in the evaluational and decision making processes of command and control leads very quickly to the development of large and complex computer programs. While my own efforts have been largely devoted to the development of detailed computer simulations of strategic air warfare;1 I have had an opportunity to follow in more or less detail the progress on a few of the command and control systems. The observations which are made here relate to a broad class of computer data systems and certainly apply to the ones evolving in the command and control area. One may think of a computer program as a progression of smaller programs or routines. Among these routines are ones which alter data presented to them, perform calculations, make new arrangements or combinations of the data, provide for employing alternate paths in the program, etc. Of considerable importance also are the control routines which bring together at the proper time the correct routine with the data on which it is to operate. In more complex programs, the sequence of use of the routines and data is subject to modification by external events. The tougher problems in the design of computer programs begin to arise when the in- ternal memory of the computer is too small to contain all of the routines and data involved. For the large man-machine systems the internal memory might be likened in size to a teacup used to dip a mixture of program and data from a barrel. Much larger memories will simplify some of the problems, but they will not eliminate nearly all of them. These remarks have been made to clarify, for the present purposes, the meaning of the expression "large computer program." The development of a large system goes through several phases. First, there is a formulation of system requirements. Then follows a period of system analysis, problem definition, and design. The computer hardware is selected, a detailed design is made and implementation is undertaken with numerous changes and delays. Experimental applications are made, redesign occurs, and modifications of the computer program are made. Further trials are made, and the cycle is repeated. The elapsed time for the accomplishment of these phases is measured in years. The design and implementation of most of the Air Force "Big L" systems have extended for three to five years. The other services have faced similar schedules. Under very favorable circumstances, a substantial application can be brought to a productive state in about two years. 232 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 '\ \ DESIGNERS' DILEMMA Designers of large data systems begin at condition A and envision a solution B, hurdling a number of intermediate steps. Because of the anticipated elapsed time of three to five years, embarking on the task of attaining solution B without intermediate steps requires the acceptance of a fairly bold assumption. It has to be assumed that solution B, which has been arrived at in terms of present day problems, will in fact be suitable for the intended task when the system is operational. In other words, ,the assumption must be made that the problem and the solution will remain relatively constant. Experience has shown that this happy state does not exist. The more distant the goal, the less certain the solution becomes. Furthermore, because the problem exists at condition A, there is pressure to arrive at a useable solution as soon as practicable. Consequently, as work progresses, several compromises are made. Call these less elegant solutions in succession Bh B2 , and B3 • As time passes, additional knowledge is acquired, and the actual requirements of the problem change to B' while the compromise solution is still Ba• A person not troubled by the esthetics of pure design could set out toward the not-crystalclear goal B by making the step from A to A' (which may be rather close to B 3 ) with much greater chance of early accomplishment of the lesser goal. Our experience with computer applications verifies the comment of one keen observer in the profession: "When we cease to change a program, we cease to use it." If there is any ~onclusion to be drawn from considering these two ideas together, it is the following: Make as complete a system study as possible, but design the implementation in such a way that a portion of the system can be exercised usefully at an early date. One must attempt to build the system and its modifications so that the user can operate the system while its usefulness is being extended incrementally. It is almost inevitable that exhibiting sample products of the system will lead to changing the design of the system; and the sooner this restatement of the requirement occurs, the sooner the first round of modifications to the system can be undertaken. Changes will be required on a continuing basis as the users of the system better understand what the system can do for them. USER PARTICIPATION It is next to impossible for the system analysis team to discover the full extent of the system being studied or to find all the boundary conditions unless the cooperating representatives of the user are sufficiently experienced in both the operational and computer aspects of the application to exhibit all the pertinent facets of the problem. The distinction is here made between the ultimate user of the system and his representatives who are assigned to assist with the design and incremental implementation of the system. The backgrounds and perspectives of the representatives may result in the incorporation of some undesirable features in the system, if they are not able to project their thinking to the needs of the ultimate user. If the ultimate user can be involved in the early exercising of the incomplete system, clarification as to emphasis and detail should occur before the whole design is implemented in an unsatisfactory fashion. Despite the possibility that guidance from representatives of the ultimate user may introduce some ideas into the design which will have to be modified at a later date, the participation of carefully selected representatives of the user throughout the design, implementation, and testing of the data system is vital. Without such participation, the user's organization will be ill-prepared to understand the series of compromises which have been made to bring the system to an operational state, and will not understand that asking for too much in a hurry is likely to delay the completion of SOME OBSERVATIONS CONCERNING LARGE PROGRAMMING EFFORTS the initial portion of the system and its incremental extensions. The user's personnel must be able to undertake the maintenance and modification of the system after the team from outside the organization has completed its work. In order to supply this capability, the user organization must provide sufficient personnel of suitable talent to participate in the implementation of. the system. These participants must be deeply involved in translating the design to language acceptable to the computer, debugging the routines created, developing test problems of proper' sophistication, producing the documentation, etc. As the key personnel assigned this duty may require two years or more to become proficient in substantial portions of the system, care must be taken to assure continuity in their assignments. Until the problem of rapid turnover among .computer programmers becomes less acute, it is likely that most of the more stable personnel involved will be among those in the employ of the using organization. TESTING For computer programs of considerable size, it is useful to distinguish three levels of testing. The programmer devises a test or tests for each subroutine he builds in order to assure that the computer reacts as anticipated to all the circumstances he considered in designing the subroutine. As these building blocks are completed, they are, ready for testing in combination with other subroutines and other aspects of their intended environment. Their input and output conventions are checked for compatibility. Larger test procedures are devised to exercise the joint functions of the program complex. Appropriate adjustments are made in' the program until all anticipated results are produced. The third stage of program testing2 consists of putting the program into limited operation to observe its behavior when dealing with the many variations of data which arise in practice. I t is very difficult to devise test problems which employ all the possible paths 233 in a program. Even though the program responds correctly in a limited test, it is possible after' several weeks of successful operation for the program to encounter some new combination of inputs against which it was not adequately protected. DOCUMENTATION Documentation of the system created in one of these three-to-five-year projects is an important though time-consuming endeavor. The description of the program in the language accepted by the computer is a key part of this record and must be kept up to date as program changes are made. This level of documentation is the "court of last resort" in isolating precisely what some routine does, but the technical remarks or diary prepared by the programmer while he has all the facts in mind will reduce the amount of detective work which must be done some months later when an unanticipated change must be made. Two or three other descriptions of the system may also be required at decreasing levels of detail. Adequate documentation is important to the person who created the computer program, as even he soon finds the maze of detail confusing without rather complete records. The person who creates one program is never free of it unless he makes it possible for others to modify his program without performing major research. The documentation must be generated so that a given programmer may be assigned to a different project. This documentation is essential, for the protection of the user, as the programmer may at some critical time no longer be available for consultation. SIMULATION Simple exercising of a large man-machine data system with the use of a single bank of information in a fixed pattern will bring into focus the utility of such a system to aid military decision makers only under rather limited circumstances. A simulation technique should be superimposed to alter the data flow according to various assumed alliances, postures, strategies, or the like. Such simulation exercises of the system in the operational environment by users of appropriate echelons will give 234 PROCEEDINGS-,sPRING JOINT COMPUTER CONFERENCE, 1964 them synthetic experience about situations which have not been encountered in sufficient detail in any other medium. Only by exercising the system with data flowing at an accelerated pace will many of its weaknesses be discovered. The simulation must, of course, incorporate uncertainty, conflicting information, degraded communications, etc., if realism is to be obtained. of truth, every new group which embarks upon a new project is dominated by persons who believe that they can avoid the mistakes made on other projects and shorten their own schedule. To some extent they do profit from the experience of others, but they usually make a few new mistakes of their own. Considerable effort has already been devoted to the development of detailed computer simulations of the air interactions in a nuclear exchange. 3 It would be hoped that these simulations might be used to generate under various assumptions much of the information required to alter the behavior of the man-machine data system. It is important to understand that a small team of four or five experienced analysts can accomplish about as much in the first four months as a team of twenty. Toward the end of the initial phase, the small team should be supplemented by specialists in various areas as required. There will be no advantage in gathering a team of 100 technicians at the beginning of the effort. As a matter of fact, such an action will probably delay the project. It should not be assumed that providing the simulated environment to influence the large data system is a small undertaking. Depending upon the detail introduced, the project may be even larger than the original one. A healthy growth in the effort applied to the project is important. A modest beginning with thorough indoctrination in the early months of the user-contractor relationship will payoff in the end. Thus far this discussion has dealt with the notions of system design, incremental implementation, inevitability of modifications, user participation throughout the project, testing at three levels, integrating documentation with the development of the program, and the usefulness of providing a simulation environment for the data system. Wide fluctuations in the support given the contractor will cause serious difficulties. If enough money is provided to acquire too large an increment of personnel, most of the old hands will be devoting too much time to training the new arrivals. If too little support is provided, progress lags and morale suffers. As new arrivals on the project will not contribute much for several months, the project is relatively insensitive, in the short run, to small personnel increases. It also seems appropriate to record some additional remarks based on participation in or observation of several large programming efforts. These remarks will range over the topics: level of effort, estimation of time required, hardware considerations, reaction to frustration, and programming systems. While these topics are not clearly related, the comments are intended to provide at least a partial check list for those who are brave enough to undertake the development of a large computer program. LEVEL OF EFFORT It requires great skill and unusual circumstances for a large system of programs to arrive at a satisfactory operational state in less than three years. Even though the period since 1955 has provided several instances of this bit ESTIMATION OF TIME REQUIRED Aside from the scarcity of experienced personnel capable of performing the scientific and engineering tasks involved in the development of a large computer data system, there are some administrative problems which lead to fixing the minimum time at two years regardless of the complexity of the system. Among the administrative problems are the following: budget justification and funding release practices, site construction schedules, and equipment delivery schedules. I t has been customary in some quarters to estimate the productivity of skilled programmers at 3000 instructions per man year. When SOME OBSERVATIONS CONCERNING LARGE PROGRAMMING EFFORTS a great deal of on-the-job training is involved, the estimate should be set at about 1000 instructions per man year. When smaller teams are given the responsibility for well defined sub-projects, instances of 12,000 to 15,000 instructions per man year have been observed. In estimating costs, it appears necessary to allow for each programmer two other persons classed as analysts, administrators, or engineering support personnel. The more complex data systems fall in the 100,000 to 200,000 instruction class. 4 Using the factors just given, one can begin to estimate the manpower required and the cost of implementing such systems. By imposing limits on the size of the team to be involved and taking into consideration the funding schedule, it is then possible to obtain a first approximation for a completion date. After the design is relatively complete. the users and designers must agree that the design meets the requirements of the project. After the detailed programming effort begins, a major re-orientation of emphasis which requires a redesign of a major segment of the system will normally delay considerably the completion of the project. Once agreement has been reached on the design of a specific large computer program, the user must exercise a great deal of patience and forbearance. No amount of exhortation will materially affect the progress of the project and may even cause delays. The user who schedules other events very closely on the basis of the promised completion date of a large programming effort is almost certain to have major disappointments. Some attempts have been made to apply PERT techniques to programming efforts. Until estimating techniques in the programming field are more reliable and the addition of extra manpower on lagging subroutines has a more predictable effect in the short run, the development of a PERT network will serve only to help the managerial echelons to understand the relationship of the various pieces of the program being constructed and to make the initial assignment of effort more intelligently. 235 HARDWARE CONSIDERATIONS Rapid strides are still being made in computer hardware development, and significant improvements as to speed, reliability, capacity, and flexibility are available every two or three years. It has been pointed out that the design and programming effort for large problems takes a minimum of two years. It is hard to imagine the confusion which would arise if a computer, of not too well defined characteristics, were being constructed for a large problem which was being programmed at the same time. First, it would require most of the hardware construction period to create the sophisticated programming system required to permit the large problem to be accepted by the computer. Further, about half way through the period, there would be a requirement to debug some of the routines of the large problem. As the problem has probably changed materially during the programming period and most certainly will change as soon as the program has been used a few times, most persons who have considered these facts have agreed that nothing is to be gained by trying to specify the characteristics of a large-scale special purpose computer. If it is built to satisfy the requirements of a problem as defined now, there is a good chance that it will not be adequate for the problem as it is understood shortly after the computer is completed three years from now. A useful competition among the computer hardware manufacturers is in progress. For reasons which need not be considered here, different . types of computers are being selected for installation at various locations on th basis of current requirements with well-documented justifications. Much of what is to be done at location A on computer X may also at some time have to be done at location B on computer Y, etc. If a programming system implemented on computers X and Y would, in fact, accept the programs first written for the other computer with only modest changes, considerable savings would result. 236" PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 REACTION TO FRUSTRATION Any person who has the tenacity to spend several weeks trying independently different approaches to the solution of a puzzle before success can appreciate in some small measure the frustrations of the programming activity and the final feeling of personal triumph when the last important programming bug is corrected. As the process of preparing a large program often extends over more than a year and the mistakes of many people are involved, the frustrations are of a higher order. In the normal course of events, nerves become frayed, fingers are pointed, neuroses appear, and other psychological and physiological reactions are not unknown. A few persons on the project who accept the ultimate technical responsibility for the completion of the project will find the pressures unabated for a period of months. It is small wonder that persons, who have experienced this trauma a time or two, have fairly firm ideas as to the circumstances under which they will accept such a responsibility again. The completion of the project will lag unless this concentrated effort does occur, and the managerial chain on the project cannot, at the beginning, predict which members of the team will punish themselves in this fashion. One of the important contributions being made by the more powerful programming systems is the elimination of many of the frequent recording and cross-referencing errors so that the more important logical blunders do not remain concealed for long in the debugging process. This alone materially reduces the length of the effort and permits more concentration on the solution of the important problems. PROGRAMMING SYSTEMS Because of the general confusion in connection with the use of the words "programming system," it may be helpful to distinguish (1) programming languages, (2) compilers, and (3) operating systems. 5, 6 A programming language is the set of symbols and conventions which are designed for the convenience of the programmer to express his detailed solution to the logical, manipulative, and computational problems encountered. A compiler translates the expressions of the programming language into sequences of instructions in the basic language of the computer hardware. An operating system augments the capabilities of the computer in such a fashion as to relieve the programmer of the requirement to provide in detail for input/output scheduling and assignment, memory allocation, etc., so that the programmer need not resolve these recurring problems in numerous variations and can devote his efforts to implementing the data system design. The JOVIAL system includes all three of these components and has been implemented on several large-scale computers, but the versions of the language are not identica1. 7 NELIAC has been widely used and consists of a language and a compiler. The data routines are incorporated during the compilation. This technique has permitted the larger computers to compile programs for computers with memories too small to accommodate a compiler. s The CL-II programming system consists only of a compiler and an operating system. The compiler is described as "syntax-directed" and can accept any of the present algebraic languages for which it is provided the necessary input tables. It has been demonstrated that careful preparation of these tables enables the compiler to generate as compact code as an experienced programmer. The sophisticated operating system of CL-II is based on the concept of the "extended machine" first expressed by Holt and Turanski. 9, 10 A good programming system should (a) Encourage modular construction of the computer programs. (b) Provide for data descriptions which are independent of the program until compilation occurs. (c) Relieve the "programmers of the necessity of constructing substantial debugging environments. (d) Eliminate the need for developing elaborate control programs. SOME OBSERVATIONS CONCERNING LARGE PROGRAMMING EFFORTS There seems to be no way to a void all the human errors which are made in the course of developing a large computer program, but a good programming system can relieve the programmer of many of the housekeeping problems so that his thoughts can be directed to the peculiarities of the data system being automated. 237 of the troublesome housekeeping details. (j) Standardizing on a single algebraic programming language is apparently no longer required. If there is to be reasonable progress in the production of a large computer data system, the programmer's work should be reduced as much as possible, and the initial system should be constructed with the anticipation that changes and extensions of the system will be required on a continuing basis. No claim is made that these observations form an exhaustive list of all matters to be considered by a group embarking upon the development of a large computer data system to assist in the evaluational and decision making processes of command and control. Each person who has been involved in such an effort will have a few items to add. Perhaps even this partial list will help some group to avoid a few of the mistakes others have made. SUMMARY REFERENCES In closing, it seems worthwhile to summarize the main points of this discussion: (a) The users of the data system must collaborate with the technicians throughout the design, construction, and testing of the system. (b) A full design is desirable, but a path should be chosen through this design so that a useful sub-system is available as soon as possible. (c) Anticipate changes and extensions to the system. (d) Do not stint on the documentation. (e) Exercising the system fully will probably require building a simulated environment. (f) Putting extra manpower on a project at the wrong time to accelerate the effort will usually accomplish little except to heighten the disappointment. (g) Large-scale special purpose computers still seem inadvisable because of changing requirements. (h) The programmer who takes his profession seriously does not find it to be an easy life. (i) Programming systems may not have reached perfection, but they offer the programmer some relief from many 1. DANIELS, A. E., "The User's Viewpoint on the Creation and Application of a Large Simulation Model," Proceedings, First War Gaming Symposium, Washington Operations Research Council, 30 November 1961. 2. SAMS, BURNETT H., "Some Observations on the Development of Large Programs," Sciences, Air Force Electronic Systems Division .and The :rvHTRE Corporation, November 1962. 3. PENNINGTO~, A. W., "A Description of the STAGE System," Proceedings, 11th Military Operations Research Symposia, April 1963. 4. HEINZE, K., N. CLAUSSEN, and V. LABoLLE, "Management of Computer Programming for Command and Control Systems," System Development Corporation, 8 May 1963. 5. KROGER, MARLIN G., et al., "Computers in Command and Control," Institute for Defense Analyses, Resarch and Engineering Support Division, Technical Report Number 61-12, November 1961. 6. HAVERTY, J. P., and R. L. PATRICK, "Programming Languages and Standardization in Command and Control," RM~447-PR, The RAND Corporation, January 1963. 7. SHAW, C. J., "The JOVIAL Manual, Part 3: The JOVIAL Primer," TM-555/003/00, 238 PROGEEDING~SPRING JOINT COMPUTER CONFERENCE, 1964 System Development Corporation, 26 December 1961. Code Translation," Proceedings of the Western Joint Computer Conference, 1960. 8. HALSTEAD, M. H., "Machine Independent Computer Programming," Spartan Books, 1962. 10. CHEATHAM, T. E., JR., and GENE F. LEONARD, "An Introduction to the CL-II Programming System," CA-63-7-SD, Computer Associates, Inc., Wakefield, Mass., November 1963. 9. HOLT, A. W., and W. J. TURANSKI, "Man to Machine Communication and Automatic SOME COST CONTRIBUTORS TO LARGE-SCALE PROGRAMS Burt N anus and Leonard Farr System Development Corporation Santa Monica, California INTRODUCTION uled and controlled are man months and computer hours. Together, these resources may be considered the cost of producing the program. Historically, these costs have been very poorly estimated; there are abundant examples of actual costs that exceeded estimated costs by 100 per cent or more. In the early days of computer technology, only a small handful of highly competent, scientifically-oriented researchers were familiar with the programming arts. In those days, the management of programming effort was only slightly different from the management of other types of research activities. Each project was unique and its probability of success uncertain; experience was severely limited; tools and techniques were custom-built for each job. In the c·omputer field today, despite many thousands of man years of experience in program development, we still tend to plan and to manage as if each program were a unique research project. This is partly due to the immaturity of the field, and partly because we have not yet fully recognized that the similarities between computer programs and their development are far more extensive than the differences between them. We have seen the development of many new tools to make the programmer more effective in his work; we must now ask whether we can develop new tools to make the program manager more effective in planning and organizing his scarce resources of talented manpower and expensive computer time. Because better cost estimation is an important step toward more effective programming management, because the costs of programs may be a significant portion of the total costs of large management or command information systems and because the estimates have been little better than guesswork to date, the Advanced Research Projects Agency of the Office of the Director of Defense Research & Engineering sponsored some research in this area at the System Development Corporation (SDC). Early efforts were aimed at data collection and analysis of several large-scale command and control programming efforts representing a total of more than two million instructions and 1500 man years of work. 2 , 3, 6, 7 This paper is a summary of a subsequent effort to (a) identify the common factors that influenced the cost of developing programs and (b) perform a preliminary analysis on some of the data. * These are viewed as necessary first steps toward the development of a more accurate cost esti- One of the most important requirements for management planning is an accurate estimate of the resources required for the completion of tpe project. In programming management, the two principal resources to be estimated, sched- * The full reseach report upon which this paper is based will be published as TM-1447, Factors that Affect the Cost of Computer P'rogramming, System Development Corporation. 239 240 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 mating ·procedure. Further progress depends upon more systematic data collection, and one of the purposes of this p'aper is to recommend the types of data that should be collected by programming managers for estimation purposes. PROBLEMS ENCOUNTERED IN DETERMINING COST FACTORS In the process of identifying and analyzing cost factors, one soon encounters a number of limitations in the 'programming field th~t may not exist in other, more mature disciplines. Some of these problems include the following: . 1. Lack of Agreement on Terminology- There are no universally acknowledged definitions of m'any of the terms used in the computer programming process. For example, the words "debugging," "parameter test" and "program validation" may all describe the same process; a "programmer" in one organization may be another and a "system called a "coder" analyst" in a third. Although we attempted to keep within the more limited context of command and control programs in our research, we found, even in this narrower field of programming, a widespread lack of agreement on terminology. in 2. Poor Definition of Product Quality-Apparently there has been little success in defining those attributes that characterize the nature or the quality of a computer program. For example, one hears programmers .talking in terms of flexibility, economy of memory, and maintainability, but there seems to be no generally agreed upon criteria for comparing programs on the basis of these attributes. 3. Poor Quality of Cost Data-Present cost collection methods seem to be designed primarily for accounting purposes and not for planning or control. For example, the collecti~ns of costs are usually grouped by organizational units rather than by product or function to be performed. 4. Dynamic Nature of the Field-Although computer programming is maturing as a discipline, there is still a wtde diversity of techniques and approaches being developed and used. As a result, any study of cost factors must consider the history and likely future trends of programming technology . 5. Nonquantitative Nature of Some Factors -Experience has shown that many of the factors that affect the cost of computer programs are qualitative in nature. In some cases, it is ·possible to predict at least the direction that cost will be affected by an increase in a given factor. For example, one would expect that the more experience one had with the particular type of program or computer involved in a given task, the less it would cost to perform that task. In other cases, qualitative factors appear to have a nonmonotonic effect as when an increase in a given factor (e.g., management planning) first decreases and then increases total cost. Of course, determination of the magnitude of the effect on cost of qualitative factors is even more difficult than determining the direction of the effect. Although these problems combine to make an analysis of computer 'programming cost factors somewhat difficult, a start must be made if program development efforts are to be more effectively planned and managed. COST FACTORS It is possible to identify hundreds of factors that contribute to the cost of computer programs, if such a level of detail is desired. In this paper, we will present a list of approximately 50 such factors, consolidated from a much larger list. Obviously, some classification scheme is necessary for discussion or analysis purposes. Factors might be grouped by work phase, such as program design or test; by management activity, such as planning or evaluation; by general categories such as resources, requirements, or environment; by units of cost measurement, such as man months or dollars; or by the classic accounting method of direct and indirect costs. However, these schemes seemed to cause difficulties because of ambiguities and overlap; as a result, a new classifi- SOME COST CONTRIGUTORS TO LARGE-SCALE PROGRAMS cation scheme was developed and is illustrated in Table I, in which factors are divided into these categories: The Job to be Done, The Resources that are Available, and The Nature of the Working Environment. 241 Since a full discussion of all of the factors is not possible here, only the most important one or two in each category will be briefly presented; the complete list is included as an appendix. TABLE I-COST FACTOR CLASSIFICATION SCHEME Category Name Logical Grouping THE JOB TO BE DONE Category Definition 1. Operational Require- ments and Design 2. Program Design and Production THE RESOURCES THAT ARE AVAILABLE Includes cost factors associated with the operating characteristics of the system for which the program is being written. Includes cost factors associated with both support and operational programs as determined by the constraints imposed by personnel, hardware and operational requirements. 3. Data Processing Equip- Includes cost factors associated with the data processing equipment required to ment produce and test a program, including all input, output and peripheral equipment. 14. Programming Personne~ velop a program. THE NATURE OF THE WORKING ENVIRONMENT 5. Management Procedures Includes cost factors associated with the plans, policies, practices and review techniques used in the administration of all phases of program development. 6. Development Environment Includes cost factors resulting from relationships of the programming staff with other organizations, such as customers and other contractors. 7. Facilities, Services and Includes cost factors related to supplies, Supplies physical plant, indirect labor and overhead. THE JOB TO BE DONE 1. Operational Requirements and Design The factors in this category tend to center around the question, "How well are the operational requirements of the information system known?" or, "How well is the problem defined?" Unfortunately, it is virtually impossible at the current state-of-the-art to evaluate the cost-contributing effect of many of the factors which relate to this question. 242 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 The primary cost factor in this category would appear to be the extent of innovation in the system, in its components, and especially in the automatic data-processing function. The extent of similarity of the new system to older systems may be a clue to estimating how clearly or easily the requirements of the new system can be stated. If the requirements are well known to the program designers, the programming job is more straightforward and less costly. Somewhat related to this factor is the extent to which the programming organization will participate in the formulation of requirements; the less active its role in determining requirements, the more likely misunderstandings will develop, resulting in costly errors, omissions and ambiguities. 8,000 INCOMPLEY 7,000 6,000 5,000 V 4,000 3,000 2,000 1,000 /. ",- 100 ~~ . / / Vt ~ 10--_. . ,..,.,.,. ~_-C" -". • • 200 300 400 500 600 700 No. Machine In5truction5 (Thou50nd5) Figure 1. Man Months Versus Program Size for Eleven Large-Scale Programs. 2. Program Design and Production In a large-scale program system, program design involves the determination of the broad logical subdivisions of the computer system, the design of an executive program to control the sequencing of programs, the design of the data base structure, the allocation of computer storage, and specifications for utility and support programs. As in the first category, these factors center around the question, "How clearly understood are the program requirements?" The factors also are concerned with the size and complexity of the job; the resources and tools available; and the plans for documenting, verifying and testing the product. Undoubtedly, the most important cost factor in this category is the number of computer program instructions and the types of programs that must be produced. In current techniques for estimating cost, the size of the program is often used as an intermediate measure to estimate the number of men who will be assigned and the number of computer hours required. Despite this reliance on size as the key to cost estimation, it appears that little research has been done to develop systematic and reliable ways to predict the number of instructions.*l So far, the experience of m'any program managers is that the number of instructions is often * R. Bleier of SDC has reported on our attempt to relate total program length to the frequency of certain decision-class instructions, TM-1603. grossly underestimated, except when very similar programs can be used for com'parison. **4,8 The conversion of the estimate of number of instructions to programming man months for large-scale systems is frequently done by allowing one man month for each 200 instructions. To test this rule of thumb, empirical data on the number of machine language instructions in eleven large command and control systems were compared with the 200- instruction-perman-month guideline (Figure 1). The programs represent a variety of command and control systems using several different computers and languages. The number of man months includes program design, testing and coding. A further analysis of the data revealed that the production rate for operational programs averaged 225 instructions perman month while the rate for utility programs was 311 instructions per man month. The explanation for the higher rate of utility program production was that the ** For example, a report of the Controller's Institute stated, " . . . almost all EDP groups have at one time or another seriously underestimated the number of steps required. . . . Every company we visited added a substantial safety factor varying from 20% for a company which claimed, due to experience, a reasonable accuracy in its estimating procedurES, to 400% for a company which had found itself that far out on a previous estimate." (Business Experience with Electronic Computers, New York Controller's Institute Research Foundation). "Measuring the Profitability of a Computer System." See also J. D. W. JameS. SOME COST CONTRIGUTORS TO LARGE-SCALE PROGRAMS program developer is his own customer for the program system and, therefore, can write his own requirements with little external coordination. For smaller programs not shown on the figure (Le., less than 10,000 instructions), rates of as much as 400 to 1000 instructions per man month for individual programs were reported. The conversion of the estimate of number of instructions to computer hours is also ~ubject to various rules of thumb. Figure 2 is based on experience with eight large ,programs. The three points falling below the line represent efforts in which a procedure-oriented language was used, indicating that such use may reduce the amount of machine time required for program development. As shown in the figure, one 16,000 • 14,000 12,000 5 10,000 V o J: ! 8,000 ~ .v ::~b¥1 I a. V• 6,000 100 200 300 .V V IIII 400 500 600 700 No. Mochine Instructions (Thousonds) computer hour is required for approximately every 53 instructions. Another hypothesis resulting from the same data is illustrated in Figure 3. This shows a near-linear relationship between the two resources of manpower and computer time. It is likely that better information for 'predicting the number of computer hours required for the production of programs of various sizes and complexities is available in most programming organizations although it is often buried in accounting data. A second important factor in this category is the extent of support program availability, reliability and documentation, including utility programs, debugging programs and library routines. Clearly, the more support ,programs that have to be produceq "from scratch," the more manpower and computer time will be required for the total program development. One other factor that may have a considerable influence on cost is the number and types of documentation produced for various types of programs. The graph in Figure 4 is based upon an analysis of five large-scale -programming efforts. It suggests that there is a linear relationship between the number of pages of documentation actually produced to satisfy contract requirements and the number of instructions in the program. These data represent Figure 2. Computer Hours Used as a Function of Program Size for Eight Large-Scale Programs. 7,000 6,000 I· 7,000 ~ ~ c: E / III U 0 Q '0 // 3,000 1,000 .->' ,..,.,- . ./ 4 3,000 Qj V ./ 2,000 4,000 ::I / 4,000 5,000 E J 5,000 .£ c: .~ I( 6,000 0) 0 a.. 2,000 "., 1,000 10 12 14 243 16 Computer Houn (Thousands) Figure 3. Man Months Versus Computer Hours for Seven Large-Scale Programs. ~ ~ ~ 0 100 200 ~ 300 ./ 400 500 No. Machines Instructions (Thousands) Figure 4. Pages of Documentation Versus Program Length for Five Large-Scale Programs. 244 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964. documents delivered to customers. There may be many times this number of pages actually written in a large programming effort. Some other guidelines for estimating the costs of documentation have been suggested. A drafting rate of 3 to 5 pages per day (750-1250 words) is a good rule of thumb for various types of programming documentation. A technical review rate of 20 p'ages ·per day seems average, but reviews that do not require extensive rewrites may run to 50 or 100 pages per day. Estimates on typing rates, illustrating rates and duplicating rates are usually fairly easy to obtain in most organizations. RESOURCES THAT ARE AVAILABLE 3. Data Processing Equipment The development of computer hardware is proceeding at such a rapid rate that it is difficult to make long-range estimates (beyond a few years) concerning its effect upon programming. Such improvements as faster add times, greatly increased memory capacities and speeds, multi-processors and new input/output devices may profoundly affect the accuracy of programming:cost estimation. With respect to equipment, a critical factor is the number of hours per day that the computer is available to the programming staff. A commonly held intuitive notion is that the more hours per day the computer is available to programmers, the lower the over-all cost of the programming effort. Among the considerations in determining the number of hours per day of availability are the number of shifts per day, the time required for preventive maintenance and the number of other computer users sharing the equipment. Of course, another strong influence is computer capability; for example, large memory capacities seem. to make programming easier. Further considerations are the power of the order code, the speed of access to primary and secondary memories, operating time and speed of input/output gear. 4. Programming Personnel The most important factor in this category rela tes to the experience of the programmers assigned to the job. There are three particularly important types of experience: (a) Experience with the particular computer-Clearly the more experience a man has with the particular machine for which the program system must be designed, the more apt he is to be familiar with its capabilities and therefore the less time it should take him to do the programming and testing. (b) Experience with the 'particular language -Programming languages differ in their suitability for various types of programming efforts. Familiarity with an appropriate special language, such as NELIAC or JOVIAL, certainly makes the programmer more efficient and requires a smaller number of man months. (c) Experience with the particular application-If the programmers have eXlperience with the particular type of system application and/ or design being programmed, then less time should be needed for the early analysis and definition phases of the job. In addition to experience, cost is affected by the number of man months of programmer training required for the project. A possible hypothesis is that the cost of programmer training may take the form of aU-shaped curve. That is, for any given task there seems to be an optimum amount of training that the programmer should have. More training presumably would not produce a commensurate return, while less would lead to errors and confusion in the programming. The number of people to assign to a given function or task also has an important impact on cost. There is ,probably an optimum number, for each type of programming effort, although research has not been conducted to determine what this is. In the field of research and development, which has some similarities with programming, the optimum number in a work group seems to be between 4 to 7. 5 There are also costs associated with obtaining personnel-either in hiring or in transferring from other contracts and in relocating them, if necessary. This is a function of both SOME COST CONTRIGUTORS TO LARGE-SCALE PROGRAMS employee turnover and the size and type of the project. A study is currently being sponsored by the Navy at the University of Southern California to analyze the job of the computer programmer, develop criterion measures of performance and determine optimal personnel selection and classification procedures. 9 NATURE OF THE WORKING ENVIRONMENT 5. Management Procedures The design and institution of clear-cut management procedures may seem costly at the outset, but their true value must be determined by comparing the cost of the plan with the cost of not having the plan. Both these costs are extremely difficult to determine but experience in many programming efforts indicates that well planned projects enjoy higher productivity rates. 245 staff must coordinate. In addition to the user or customer, there may be separate agencies responsible for the contracting and for the other aspects of the system, such as hardware development. Problems of coordination and concurrence multiply as a function of the number of groups with which the programming staff must deal. Often, mutual education between these various staffs becomes necessary and the cost of briefings and meetings for such purposes may make the cost per instruction higher for large-scale program systems than for smaller ones. 7. Facilities, Services, and Supplies The most im·portant effect upon total cost in this category is the use, maintenance, and monitoring of a management plan that includes at least communications and decision-m'aking procedures, mechanisms for handling changes in the program, delineations of responsibility and authority, and schedules for major milestones and products. The plans should also specify standards for flow charts and documentation 'and quality,control procedures. There is probably some optimum per cent of time that should be spent in planning for ,program development. In most accounting systems, nor~al overhead and miscellaneous supplies are covered by a percentage addition to the estimated direct labor and materials. However, unusual expenditures associated with large-scale programming efforts may not be adequately covered by the average overhead burden rate. For example, various types of technical and administrative support are needed to assist the programmer. Computer operators and EAM personnel save the programmer considerable time during testing. Effective and experienced management and administrative personnel assure that the work is efficiently organized and fre~ the programmer to cQ,Ilcentrate on technical matters. Technical editors help to ensure that documentation is adequate and understandable. There is some optimum mix of such support personnel that will ensure minimum cost. Another important cost factor is the number of computer runs permitted to each programmer each day. One of the reasons for the current activity in the development of time-sharing procedures is the belief that a larger number of computer runs per day for each programmer will shorten lead times 'and decrease costs. An equally reasonable hypothesis is that a cost trade-off exists between desk checking and computer testing such that some optimum number of computer runs will minimiz'e total cost, the optimum depending upon the relative cost of computer time and programmer time. Another important factor in this category is travel and communication cost. Trips may be required for briefings and conferences with user organizations and associated developmental agencies; for data gathering; for training and familiarization; and for concurrence on requirements or design. There is a strong tendency to underestimate these costs or to curtail them as an economic or political measure; this often results in delays in getting data or concurring on programming details, and these delays may be very costly in terms of time and manpower. 6. Development Environment In some cases an important consideration is the cost of special facilities, e.g., simulation devices, special office equipment, or computer A particularly important problem is the number of agencies with which the programming 246 PROCEEDINGS~SPRING JOINT COMPUTER CONFERENCE, 1964 room facilities. Special wiring and air conditioning, false flooring, space for storage and movements of parts and equipment, maintenance and test hardware, etc., all contribute to the cost. SUMMARY AND CONCLUSION In this paper we have listed some cost factors that represent a consolidation and skimming of a larger list of factors that contribute to the cost of computer programming efforts. In many cases, the factors are very difficult to measure or quantify; further, their effect upon other factors and upon the total costs of the programming effort is often difficult to determine. For many of the factors, some data exist in current accounting records, but these data have not been collected, compiled and analyzed. The mere listing of cost factors in programming is only a first step, albeit an important one, toward the development of a more scientific and, hopefully, a more precise method of estimating the cost of programming efforts. Ultimately, one would hope to discover by analysis of data some predictors that would enable a more accurate estimate of the number of instructions, man months and machine hours. Among the cost data-both estimated and real -that we would hope programming managers would begin to accumulate for the various types of products and activities in their projects are the following: 1. The number of machine instructions and the programming' language used. Also, the percentage of the finished program consisting of library routines and subprograms from previous programs. 2. The number of man months of programmer effort, including the first level of supervision, and the experience level of the programmers. 3. The number of hours of machine time required for testing and debugging purposes and the types of machines used. Also, the pattern of machine usage in terms of runs per day and hours per run. 4. The number, types, and timing of important program changes and, in at least a qualitative sense, the effects of these changes on the final product. 5. The types and number of pages of documentation required, including a notation as to whether they are single or double spaced. In addition, it would be useful if a log could be kept by a project "historian" describing certain qualitative attributes such as those described earlier in this paper. This section should describe the data-processing functions of the program system and its relationship to other systems. It should also identify all interim and end products, such as types, listings and descriptive documents. A series of experiments to study those factors that can be analyzed only in controlled environments ideally should be conducted simultaneously with the collection of new and existing data on program costs. For example, well designed, statistical experiments would be useful for determining the,effects of different types of programming languages upon total cost, the effects of greater or lesser machine availability upon costs, the optimum size of 'programming staff for different types of programs, and the best mix of programming talent (experienced versus inexperienced) for given types of jobs. Obviously, it may be quite some time before a valid predictive set of equations can be developed for program cost estimation. N evertheless, it is to be hoped that each small step in this direction will represent a useful experience in itself in terms of increased insight into the programming process. Certainly, resea~rch into techniques for improved programming management should be encouraged if the industrY'is to keep pace with the increasing demands for its services. APPENDIX I LIST OF COMPUTER PROGRAMMING COST FACTORS Summarized below for the convenience of the reader is the complete list of cost factors discussed in this paper. Operational Requirements and Design 1. Extent of innovation in the system, its components, and especially the automatic data-processing function. SOME COST CONTRIGUTORS TO LARGE-SCALE PROGRAMS 2. Extent to which the programming contractor will ,participate in a determination of the information processing needs (i.e., the system and operations analysis, and the system and operational design). 3. Number, size, frequency, and timing of system design changes. 4. Extent of command and control system decentralization and number of interfaces. 5. Number of other components and subsystems being developed concurrently as part of the command and control system (e.g., intelligence, sensor, etc.). Program Design and Production 1. Number of computer program instructions and the types of programs that must be produced. 2. Number, types, and frequency of inputs and out'puts to the computer (s). 3. Extent of innovation required in the program system; that is, the degree to which programs are similar in nature to those previously written. 4. Number, types, and quality of publications and documentation for both 'customer and internal use. 5. Extent of complexity of the data-processing functions. 6. Degree to which the following program design characteristics are recognized and must be incorporated. (a) Maintainability-the ease with which program errors can be detected and corrected. (b) Changeability-the ease with which new functions can be incorporated in the program. (c) Usability-the ease with which personnel other than designers can use the program. (d) Flexibility-the ease with which the program can be used for other purposes with only slight modification (e.g., SAGE programs for air traffic control) . (e) Generality-the ease with which a program can accept a wide range of inputs. 247 7. Extent of the constraints on program design (e.g., real-time requiremnets). 8. Number, size, frequency, and timing of program design changes. 9. Extent to which data for data base is available, or data collection is required. 10. Number of entries (total size) for the data base, the number of different types of data needed for it, and the extent to which each can serve many programs or subprograms. 11. Efficiency of the programming language and the com'piler or assembler. 12. Extent of the. completeness and clarity of the system test and acceptance test requirements. Data Processing Equipment 1. N umber of hours 'per day of computer availability. 2. Extent of capability of the computer and its suitability for the job to be done. 3. Extent to which the operation of the computer and peripheral equipment is reliable, well tested, and well documented. 4. Number of equipment components being developed concurrently with the program. 5. Number of different computers for which programs are being prepared. 6. N umber and types of displays used. 7. Extent to which adequate EAM support will be available. 8. Extent to which routine preventive and emergency maintenance will be available. Programming Personnel 1. Types and quality of programmers. 2. N umber of man months of programmer training required. 3. Number of programmers to be assigned to a given function or task. 4. Policy of obtaining and phasing of person~ nel to staff a new contract. 5. Rate of turnover. Management Procedures 1. Extent of use, maintenance, and monitoring of effective management plans within 248 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964: both the customer's and program developer's organizations. 6. Cost of punched cards, magnetic tape and other special supplies or equipment. 2. Extent of form,alized procedures to use the computer facility. 7. Cost of special security requirements (e.g., Top Secret vault). 3. Extent to which there is a well defined and controlled system change procedure. 4. Extent of an error-reporting and -correcting procedure. 5. Extent of contingency plans in the event the computer is overloaded or otherwise unavailable. 6. Extent of quality control that is exercised during testing (e.g., reliability requirements). BIBLIOGRAPHY 1. BLEIER, R. E. Frequency Analysis of Ma- 2. 3. Development Environment 1. Number of agencies the programmer con- tractor must deal with and their level of experience with system development. ~_ 2. Average number of days and effort required for concurrence. 3. Travel requirements. 4. Extent to which delivery dates for required 'programming tools are reliable, and correspondingly, the amount of pressure caused by a tight schedule. 5. Extent to which the computer is operated by another agency. 5. 6. Facilities, Services, and Supplies 1. N umber of computer operators and EAM personnel required. 2. Number and experience of technical management personnel, administrative personnel, and technical editors. 3. Cost of special simulation facilities, computer room facilities or special office equi'pment. 4. N umber of square feet of new office s'pace or building required. 5. Exceptional costs of graphic arts and reproduction. 7. 8. 9. chine Instructions in Computer Program Systems, TM-1603, System Development Corporation, 19 November 1963. FARR, L. A Description of the Computer Program Implementation Process, TM1021/002/00, System Development Corporation, 25 February 1963. HEINZE, K. P., N. CLAUSSEN and V. LABOLLE. Management of Computer Programming for Command and Control Systems, A Survey, TM-903/000/02, System Dt\felopment Corporation, 8 May 1963. JAMES, J. D. W. "Measuring the Profitability of a Computer System," The Computer Journal, 5 :4, January 1963, pp. 284293. JOHNSON, ELLIS A. "A Proposal for Strengthening U.S. Technology," in Burton V. Dean (editorL Ope'rations Research in Research and Development, New York, John Wiley and Sons, Inc., p. 34. LABoLLE, V. Management Aspects of Computer Programming for Command and Control Systems, SP-1000/000/02, System Development Corporation, 5 February 1963. NANUS, B. Cos,t Estimation Bibliography, TM-1430, System Development Corporation, 20 August 1963. New York Controller's Institute Research Foundation. Business Experience with Electronic Computers, 1959, p. 48. RIGNEY, J. W., R. M. BERGER and A. GERSHON. Computer Personnel Selection and Criterion Development: I. The Research Plans, Los Angeles, Department of Psychology of the University of Southern California, February 1963. HYBRID COMPUTATION ... WHAT IS IT? ... WHO NEEDS IT? ... Thos. D. Truitt Electronic Associates, Inc. Research and Computation Division Princeton, N ew Jersey INTRODUCTION flown" as the effects upon the dynamic behavior of the model, wHh changes in design parameters, were examined. The characteristics and performance of the simulation model often provided input data for a digital computer program that would evaluate, optimize, and calculate final design specifications. The digital computer was also useful as a simulation tool in calculating orbits and ballistic trajectories and in designing guidance and control systems for such aerospace flights. Some Historical Notes During the nineteen-fifties the capabilities of electronic computers expanded so fast as to stay well ahead of the needs of the average computer user. Such was the case in both the analog computer and scientific digital computer fields. One effect of this situation was the formation of two schools of experts with opposite views on the choice of the "best" general purpose scientific computer. Differences of training, experience, and semantics led to a serious barrier to communications between :these two groups. At best, the fashionable, middle-of-the-road position was to admit that each computing technique "had its place," which did little to break down the barrier. Only with the appearance of a computational task that could not be accomplished satisfactorily by either type of general purpose computer-only then was the barrier cracked and a small opening made. While conventional computer simulation techniques were quite adequate for most aerospace design problems, the exceptional costs and hazards involved in missile and manned vehicle missions required computer simulation of complete missions. Some full mission simulations required inclusion of actual control mechanisms and of a manned cockpit, thereby dictating simulation on a real-time scale. It was apparent that such total system simulations exceeded the capabilities of any single type of computer. Only by the combined application of analog and digital computers could adequate simulation be accomplished. It was hoped that the special features of the two computer types could be combined to form a hybrid simulator of superior performance. In the aircraft and the budding aerospace industries lthe analog computer had long been the primary, if not the only, means employed for simulation of new or proposed engineering designs. In fact the analog computer grew up as the aircraft simulation tool. A mathematical model of each new design was formulated and an electronic analog model formed on the computer. Performance data obtained from wind tunnel tests were added to complete the model, and large numbers of experimental "flights were Thus the earliest attempts :to combine the computation of analog and digital computers took place in about 1958 at the Convair Astronautics plant in San Diego and at the Space 249 250 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Technology Laboratories in Los Angles.19,21 ,::19 In both cases the job at hand was the complete mission simulation of the trajectory of a long range missile. The speed of the analog computer was a necessary element in the study to permit a "real time" simulation of the rapid motion of the vehicle and of control surfaces. However, the dynamic range required of the simulation was in excess of that of the best analog computers. That is, the ratio of the total range of the traj ectory to the required terminal phase resolution (a dynamic range of 105 to 107) was greater than 104, the upper limit of analog computer dynamic range for small programs. Hence the digital computer was used to calcula~those variables for which such dynamic range was necessary. The most important of these were the navigational coordinates-the digital computer performed the open integration of velocities to determine the vehicle's position, plus the dynamic pressure, a function that is very sensitive to altitude and velocity. It is fortunate that in such long range aerospace trajectory simulations the variables with wide dynamic range requiring precise calculation are not, at the same time, rapidly changing. Moreover the "high speed" variables do not require precise calculation. The early combined computer. systems employed the largest and fastest digital computers available at the timeUnivac 1103A and IBM 704-together with 300 to 400 amplifiers of general purpose PACE analog equipment. In both cases even these fast digital computers were only just fast enough to perform the required repetitive calculations for the slowly changing variables of the simulation in real time. Since the installations of the first combined computers at least a dozen computer laboratories have employed general purpose analog and digital computers together to solve simulation problems, and a number of attempts have been made to devise spacial purpose systems of analog and digital devices. Among the latter are the CADDA of the National Bureau of Standards and the "pulsed analog computer" of the MIT Electronic Systems Laboratory.l0,ll,12, 13,35,42 Hybrid computers of a unique type are the combinations of a general purpose digital computer and a digital diff-erentjal analyzer (dda), illustrated by the Bendix G15 with the DA-1 attachment and the Packard Bell PB250 with the Trice dda. 27 The former system consists of a small, slow computer with an even smaller serial dda ($50,000· and $10,000 respectively). In contrast to this the Packard Bell system combines a small, m'edium speed computer ($40,000) with a large serial-parallel dda ($500,000). Among the systems of general purpose computers, generally large analog computers have been combined with both large (IBM 7090) and small digital computers (PB 250, LGP 30) .2,3,7,17,19,21 A brief analysis of the applications to which existing installations of combined systems have been applied leads to these generalizations. For the most part the analog computers in these systems have been employed in a normal manner to simulate the dynamic behavior of physical systems by solving sets of non-linear, ordinary differential equations, while the digital computer has performed one or more of the following three functions: complex control logic, storage of arbitrary functions or sampled analog functions, and high precision arithmetic primarily for numerical integration. Examples of the applications are: a. Analog computer plus digital control logic. A system that in itself contains discrete control functions of continuous dynamic variables is appropriately simulated by a hybrid computer. The kinetics of a chemical process are simulated by continuous analog means while its digital control system is represented by a digital program. 29 ,41 Similarly an aerospace vehicle with an on-board digital computer, control system, or autopilot is simulated by hybrid techniques. h. Analog computer plus digital memory. A very common difficulty in the simulation of a chemical or nuclear reactor is providing an adequate representation of the transport of fluid in pipes from one point to another - from reactor to heat exchanger. The simulation of this transport delay of a dynamic variable, such as the time variation of the fluid temperature, is very nicely accomplished by the use of a digital computer for storage of the temperature function for a fixed, or variable, length of time. HYBRID COMPUTATION . . . WHAT IS IT? . . . WHO NEEDS IT? . . . Digital computer memory has also been used effectively to store multivariable arbitrary functions - an operation which is seriously limited in the analog computer. c. Analog computer plus digital arithmetic. This type of application is the "classic" one where the digital computer is used to perform precise, numerical integration of space vehicle velocities to keep track of the exact position of the vehicle over a very long range flight. It should be noted that a significant difference is apparent in the applications of computer systems employing a very small digital computer and those with large, very fast computers. In general the small machines are limited to execution of control logic programs, one or two channels of transport delay simulation, or limited function generation programs. Since numerical integration and complex function generation by digital programs require considerable time for each calculation, for each discrete step in time, only the fastest digital machines can be used effectively for these tasks. SOlVIE COMMENTS ON THE EVOLUTION OF HYBRID SIJVIULATION The term "computer simulation" appears in so many contexts it is important to emphasize that its use in these pages is limited to the modelling of complex physical systems for the study of their dynamic behavior. These systems are represented by sets of differential equations, algebraic equations, and logic equations. As in most simulation studies the objectives of hybrid simulation may be experimental design, prediction and control, and design evaluation, verification, or optimization. It is not expected that the important applications of hybrid simulation will include: data processing system simulation; information handling simulation; business system simulations; siInulation of television coding, character reading machines, communications coding systems, (;tc . . . . These are all digital computer simulatiJn applications. Similarly hybrid computers are not warranted for the simulation of circuits, devices, and systems for which the analog computer is quite adequate. It is in the simulation of total systems that bring together components, some of which are suited for digital and some for analog simulation, that the newer hybrid techniques are re- 251 'quired. There probably are few, if any, simple hybrid computer applications. If hybrid computation can be said to be a field of endeavor it must be considered to be in the formative stage. Developments to date have led to equipment configurations and programming techniques that were dictated by specific problems and limited objectives. The growth rate of the field will be determined by the exteI1lt to which a broader view is taken of hybrid computer programming techniques and applications. The greatest advances in computers have been made when the experiences of users, programmers, have been brought to bear on design of equipm'ent. For the most part, hybrid computers of today consist of general purpose analog and digital machines, which are not aesigned for hybrid operation, tied together by "linkage equipment" designed only to solve the communication problem. This has been a necessary first step. Newer hybrid systems will be I designed not just for· communications but for efficient solution of hybrid problems and for convenient programming. The purpose of the following discussions is to bring attention to the essentials of hybrid computation and their relationship to problem solutions. Out of this should come some indication of how present day computers can be most effectively employed. The next generation of hybrid computers will undoubtedly achieve a greater degree of integration of parallel computing elements with the sequential stored program principle. Eventually patch panel programming of parallel elements will be replaced by an automatic system, thus affording a fully automatic method for computer set-up and check-out. Even today a secondary activity of the digital part of the hybrid machine is the partial automation of setupaiId check-out of tbe analog computer. This feature becomes increasingly important as the computer system grows in size and the programs grow in complexity, for the attention of the programmer and problem analyst needs to be directed to the simulation itself rather than the simulator. THE ELEMENTS OF HYBRID COMPUTERS Digital Computers Many conflicting factors have influenced the choice of digital computers used in hybrid sys- 252 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 terns. Computer speed and economics have probably been dominant. Since there are so many computers on the market today that have sufficient speed and that span the complete range of prices, it is more instructive to examine the features that are essential for hybrid computation. Speed. The speed of execution of arithmetic operations is most important, and this is a function of memory access time and multiplier speed. The access speed of drum and delay line memories is too slow. Magnetic core access times of 2 to 5 microseconds are currently popular. This means the time for addition of two numbers is 4 to 10 microseconds. Multiplication and division take longer-times of 15 to 40 microseconds are generally available and quite satisfactory. Overall program speep can be increased by the use of index registers-three registers is desirable; more are useful. Special instructions for subroutine entry, for executing commands out of sequence, and for testing and skipping can help increase computing speeds. Word Structure. The basic requirement is for a fixed point, binary word of at least 24 bits. Since round-off errors affect the last several bits a smaller word size would result in a dynamic range limitation of less than 106 • A longer word may be useful in a few applications where fixed point scaling may be difficult. Floating point computations may make things easier for the programmer but should not be depended upon at the expense of computational speed. It may be noted that the equivalent of fixed point scaling is a necessary part of the analog program, and hence floating point operations may not prove as advantageous as for some alldigital programs. Decimal format and character oriented machines do not offer any advantages for hybrid computation, and usually they are slower than equivalent sized binary computers. Input/Output. High data rates in and out of core memory and any feature that minimizes loss of computing time for input/output operations are highly desirable. In addition a fast, flexible means for communicating control signals to and from the analog section of the hybrid system is necessary. Three kinds of control signals are usually provided: interrupt and sense lines, and output control signals. It is by means of these controls that the sequential operations of the digital machine are made compatible with the parallel, simultaneous operations of the ana10g machine. Since communications must be made with many points in the analog computer, a number of these control signals are needed. Interrupt signals, from outside the computer, stop the current sequence of calculations and force transfer to another sequence. Sense lines simply indicate to the digital program the current state of devices outside the computer. They may be sensed by specific programmed instruction. Other programmed instructions will send control signals outside the computer on the output control lines. Memory. As noted above the digital computer main memory should be a high-speed magnetic core. Since most hybrid applications do not require a large memory for either program instructions or data, four, eight or twelve thousands words of core memory should suffice. Larger memories may be desired for special digital programs and larger hybrid problems when more experience has been gained in this field, thus expandability of a memory to 16K words is a good feature. N ewer computers are being introduced with small, very high speed "scratch pad" memories. Such memories may ha ve cycle times less than a microsecond and are used to store intermediate arithmetic results. This feature increases the overall computation speed of the computer. The normal manner of operating an analog computer involves a fair amount of non-computing time when the computer remains in the Hold or Reset mode. These intervals may range from seconds to minutes while adjustments are made, pots are set, or recorders changed, or while the programmer analyzes results. It is not possible for the analog computer to operate on other programs at these times, however, with a hybrid system, where such waiting periods are likely to occur also, it is reasonable to consider having the digital computer work on a different program during the intervals, whatever their length. Appropriate "interrupt" and "memory lock-out" features are possible to permit time sharing of the digital machine without affecting the hybrid program and without the danger of one program interfering with HYBRID COMPUTATION . . . WHAT IS IT? . . . WHO NEEDS IT? . the other. The secondary program (a strictly digital problem) is simply stored in a "protected" part of the core memory and utilizes all the bits of time not required by the hybrid program. Peripheral Equipment. In many digital computer installations the investment in peripheral equipment rivals that in the central computer. Current hybrid computer applications require only a minimum of digital peripheral equipment. The graphic output equipment associated with the analog computer is sufficient for computational results. Punched paper tape reader and punch and typewriter may be all that is required for programming. Larger systems in the future win employ punched cards and magnetic tape, primarily for rapid change-over of problem and automatic check-out. Large offline data storage does not appear necessary for most applications. In summary the digital computer must be characterized as a sequential machine. For effective use within a hybrid system the machine (a) must have sufficiently high internal speed for it to appear as though a number of calculations were taking place simultaneously; (b) must be organized for maximum speed in executing mathematical calculations; and (c) must have efficient means for input and output of data during calculation (Figure 1). OC·~---' INT---.. SENSE - - - - - ' Figure 1. The digital computer must be characterized as a sequential machine. For effective use within a hybrid system the machine must have sufficiently high internal speed for it to appear as though a number of calculations were taking place simultaneously. It must be organized for maximum speed in executing mathematical calculations; and it must have efficient means for input and output of data during calculation. 253 Analog Computers In contrast to the above the analog computer is a parallel machine, with many computing components and I/O devices operating in concert. There are few, if any, features of the modern analog computer that are not appropriate to a hybrid system. However, only the largest analog machines have been used for general purpose hybrid simulation. The common measure of a large computer is that it has 100 to 200 operational amplifiers. Since two or more computers may be "slaved" together, larger systems are possible when required. Analog computer features that are important for hybrid systems can be simply listed as: -Integrators with multiple time scales -Amplifiers for tracking and storing voltages -Very fast control of the modes of individual amplifiers -Automatic, remote control of the setting of potentiometers -Fast, accurate multipliers and trigonometric resolvers -High speed comparators with logic signal outputs -Electronic switches (logic signal gating of analog signals) In the early days logic equations or switching functions were programmed with relays and stepping switches, which were connected to the patch board by various means. Present day technology employs electronic switching of integrator modes and voltage signals at high speeds, and the delays inherent in relay devices can no longer be tolerated for logic operations. The logic building blocks common to the digital computer designer (fiip-fiops,gates, inverters, monostable multivibrators, . shift registers, and counters) are ideally suited to these operations. Thus with electronic switches replacing relay contacts, logic modules have become an integral part of all new, large, analog systems. These modules are programmed like the other analog components oy interconnections at a patch panel. Many signals occur simultaneously but they are logic signals-two values, Zero and One, that change as functions of time. Input signals to logic programs come from comparators, manual switches, and external control sig- 254 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 nals. Logic program outputs go to integrator mode controls, storage amplifier controls, electronic switches(DA switches) to gate analog signals. As will be shown later, it is essential for a hybrid system to have a very significant complement of digital logic components. A few gates and flip-flops are not sufficient. The potentialities for use of logic components in· an analog computer for hybrid operation are so great that the EAI HYDAC Digital Operations System is an entire computer console with its own patching system used entirely for the programming of digital components for parallel computation. This console is really a complete logic computer. It is used together with a conventional analog computer to form what is truly an all parallel hybrid computer. 1, 9,23,24,29,40,41 In summary· the modern analog computer must be characterized as a parallel machine. It is not solely a computer for continuous variables. I t is a parallel assemblage of building blocks: integrators, multipliers, etc., for continuous variables; and flip-flops, gates, counters, etc., and "digital" circuits for discrete variables. It is organized for convenient representation of an "analogous" physical system by means of a computer model constructed of these building block (Figure 2) . Figure 2. The modern analog cOplputer must be characterized as a parallel machine. Not solely a computer for continuous variables, it is a parallel assemblage of building blocks: integrators, multipliers, etc, for continuous variaw.s; and flip-flops, gates, counters, etc., and "digital" eircuits for discrete variables. It is organized for convenient simulation of an "analogous" physical system by a computer model that is constructed of these building blocks. Conversion Devices In providing data communication between a sequential computer and a parallel computer three kinds of devices are commonly used: the multiplexer, the analog-to-digital converter (ADC), and the digital-to-analog converter (DAC). In addition, all early systems have employed a timing and control unit which performs a relatively fixed set of operations, with manual switches to select options such as sampling frequency, and number of channels. Such "linkage systems" thus consisted of a timer unit plus a group of linkage building blocks prewired to perform a specific task. With the integration of digital logic components into the parallel computer, however, greater programming flexibility is possible by use of these logic units for timing control of the data conversions. Furthermore, the converters and multiplexer can act very naturally as additional building olocks in the parallel computer. Thus it is likely that future hybrid systems will simply incorporate the "linkage system" within the parallel computer. Usually several or many analog signals in a hybrid program will be sampled, converted and transmitted periodically to the sequential digital program. The numbers, of the several sequences of numbers to be entered into the core memory, can be accepted only one at a time. Since this is so, the conversions from voltage to number form can be performed one at a time-first from one analog variable and then from another. The multiplexer is used to select one from many analog signals, to step through a sequence of signals, and thus to furnish voltage input signals to the ADC. The output of commonly used ADC's is a binary number of 10 to 14 bits. A 13 bit binary output probably is the best compromise; it represents a resolution of one part in eight thousand, and resolution of analog voltage signals is at best one part in ten thousand. Conversion times range from 50 to 300 microseconds. A typical time of 100 microseconds would allow the converter to be shared by 16 analog signals each with a frequency spectrum extending to 20 or 30 cycles per second. This will be explained later. DAC units should have the same binary word size as the ADC, except for special low accuracy HYBRID COMPUTATION . . . WHAT IS IT? . . . WHO NEEDS IT? . . • uses. Conversion times are not determined by the converter but rather by the bandwidth of the analog amplifier following the converter. Data from the sequential computer appear only one word at a time, and some means for retaining the latest value, of each sequence of numbers, for each output function is needed. The sequence of numbers coming from the computer may first be converted to voltage values by a single DAC, and then distributed to storage amplifiers for each channel. It is more customary, however, to hold each of the latest words for each channel in a digital register which is an integral part of the DAC assigned to each channel (Figure 3). Special Forms As a passing thought it may be noted that while the primary emphasis here is being placed on the distinction between parallel and sequential operation, the term "hybrid," his;.. tori cally, has been used to imply the combination of continuous and discrete calculations, and that therefore consideration might be given to two special kinds of hybrid computers: The Parallel Hybrid Computer; which is a proper term for the EAI HYDAC 2000 machine. This system combines an analog computer with a general purpose system of programmable logic building blocks, multiplexer, ADC, DAC's, Figure 3. ADC's and DAC's are format converters, changing voltage to numbers and vice versa. As components of the parallel machine the converters together with logic components must act to "match impedances," i.e., resolve the incompatibility between the parallel and sequential programs. 255 digital memory units for storage· of sampled analog functions, and several digital numerical adders and subtractors. The application of this system encompasses an intermediate range of hybrid problems, such as: a. Transport delay simulation 24 b. Single and multi-variable function generation c. Logic control systems23 d. Automation of the analog computer for parameter searches and optimization studies 41 e. Simulation of numerical and sampled data control systems 1,9 The Sequential Hybrid Computer, which is exemplified bytthe experimental "pulsed analog computer" techniques developed at M.LT. for use in an ai~craft flight trainer.1° This system employs a [:sequential digital computer which controls a small number of analog functional components--one multiplier, one reciprocal generator, one integrator, and several adding units. These units are interconnected and receive inputs by digital program control. They form, in effect, "analog subroutines" for the sequential computer. THE SEQUENTIAL/PARALLEL HYBRID COMPUTER The term "hybrid" is most appropriately used to indicate the combined use of sequential and parallel computing techniques, first because the future growth in hybrid simulation will be predominantly in this direction. Second~ and more important, is that from the standpoint of the programmer who must bring the two types of computers together to find a useful solution to a problem, the only really significant disparity lies. between sequential and parallel operations. The difference between continuous voltage and discrete number is simply one of format. It would make little difference if the analog signals were frequency modulated, pulse code modulated, hydraulic, or pneumaticappropriate format converters could be found. The feature of hybrid computation that is of importance is: in part of the machine many operations are taking place simultaneously, and many time-varying problem variables exist in parallel; while elsewhere a number of operations take place, one at a time in a repetitive 256 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 manner, so as to effect the generation of several problem variables, as if they occurred simultaneously. Furthermore, the parallel computation is tied to a real-time base: the very passage of time itself accounts for the changing of the basic independent computer variable. The sequential program is asynchronous-not controlled by a clock. Operations are executed in sequence at whatever rate is possible, and for any reference to be made to the actual elapsed time, external communication is necessary. This basic incompatihility requires that the interface between the two types of operation embody more than the simple format conversions performed by the ADC and DAC's. It is necessary for data and control information in the parallel machine to be available to the sequential machine and conversely that the latter be able to send data and control signals to many points in the former. Coincidence or simultaneity of events communicated to and controlled from the sequential program are particularly difficult to handle. The logic and data control of the interface equipment must resolve these differences in timing and operation. What might be termed an "impedance matching" device is needed between the parallel and sequential program in order to make most efficient use of both machines. The exact manner in which this is done will vary from problem to problem (Figure 4). A Simple Example An example will illustrate some basic considerations in defining a general purpose hybrid system. First, the operation of the simplest of linkage systems: an ADC and multiplexer, a Figure 4. A hybrid computer is a compatible. system of parallel computing components, both digital and analog, and a stored program sequential machine. The hybrid computer programmer must constantly be aware of the relative timing of events in the parallel and sequential parts of the program. "Matching impedances" between these parts is accomplished by programming of the interface components to suit the simulation. number of DAC's, and an "interrupt clock." The flexibility of the stored program digital computer is relied on for control of these units. Assume 10 analog signals are to be converted one at a time. These. words are placed in memory (average program time: 40 p.sec. per word and then about 7.5 or 8.0 milliseconds of sequential, digital calculations takes place, followed by output from memory of ten words (20 p.sec. per word) to ten DAC's. The entire cycle requires 7500 + 10 X 40 (input) + 10 X 20 (output) = 8100 microseconds of digital program time. If it is assumed the conversion of the data (A to D) requires 150 microseconds per word then 1.5 additional milliseconds, or 9 ms are needed if everything proceeds sequentially. Assume further that because of the frequencies, or the analog signals, it is necessary to sample at least some, and therefore all, of the channels at 100 samples per second. A "real4ime interrupt clock" is set at 100 cycles per second. This timer unit is an adjustable oscillator that sends an interrupt pulse to the digital computer. The latter then activates the ADC, waits 150 p'sec for completion of a "Ready" signal, steps the multiplexer to the next channel, stores the converted word in memory, and then repeats this cycle ten times. With the tenth step the multiplexer resets to the first channel. The program then proceeds with the 7.5 milliseconds of calculation, outputs ten words, one at a time, to ten DAC's, and then waits for the next Figure 5. An early hybrid system configuration. The sequential program controls the timing of the conversion cycle. The cycle is initiated by the "interrupt clock." For a tyical problem the clock might be set for 100 cps; and 7 to 8 milliseconds per cycle would be available for calculation. HY,BRID COMPUTATION WHAT IS IT? . . . WHO NEEDS IT? . . . 257 and control devices external to the sequential computer. In other problems these percentages may be higher. The other weaknesses in this system lie in the fact that it was not designed to be a general purpose system. It is restricted in application to a class of problems for which the periodic "input/calculate/output" cycle is useful. Figure 6. The sequential program flow diagram, for the example hybrid system of Figure 5, shows the steps required for control of the converters. IN~~~~~PT .........1....-_ _ _ _ _ _ _---(,1-'_ ___"_ _ PULSE IIII ADC OPERATING TIME DIGITAL COMPUTER TIME iIIuu I-- CALCULATE 7.4MS..j ~ 'OH'OH'O ~ ~~ I t, tl LI 1, TnT COMPLETION STEf' IIILTIPUXER --"':J 1NDEX II TltANSFER '"'-~j--START "DC L ~40~SEC--1 Figure 7. Typical times are shown for the steps in the conversion cycle of the example system of Figure 5. All ten channels are converted and stored in memory as fast as possible before proceeding with calculationsat the expense of intervals of "waiting time." interrupt pulse. (See Figures 5, 6, and 7.) Manual controls are provided for selecting the interrupt clock frequency and the number of channels in the multiplexer stepping cycle. This is certainly a simple system and it appears to satisfy the basic requirements for communications. Some of the shortcomings of the system are apparent: sampling and outputting of each channel do not take place simultaneously, 15 % of the sequential program time is "waiting" time, and 3 to 5 % is used to select System Improvements or Variatio~ By programming the parallel digital COMponents of the parallel computer to perform timing and control functions for the system the following changes to the above system are suggested: a. Simultaneous sampling. If the sequential program operates on two or more of the input numbers together to calculate an output, then errors may occur since the input numbers were sampled at different times and correspond to different values of the independent variable. A similar effect may occur at the output since the numbers in a group of ten appear at the ten DAC's at different times. It is certainly possible by numerical means to compensate for the errors, at the penalty of additional program time. 18, 28 The common soiution is to add memory to each of the ten input and ten output channels. Ten Track/Store amplifiers are added before the multiplexer and a control signal causes them all to sample, by storing the voltages, simultaneously. At the output, 13 bit registers are added in front of the DAC's. When all ten registers ha ve been loaded, a transfer pulse causes all DAC values to be updated at the same time. b. External timing of ADC and multiplexer. Sequential program time can be saved by permitting the control of the ADC, multiplexer, track store cycle to be controlled externally. A simple clock, counter, flip-flop, and group of logic gates will permit the input conversion cycle to run at its own rate-interrupting the sequential program only at the completion of a conversion. Thus the conversion time can overlap the calculation time, eliminating the waiting time. Upon interrupt only 10 to 20 p.sec. may be required per sample; many control steps are eliminated. Similarly on output, the addressing and selection of output channels can be done by simple circuits rather than using sequential program time (Figure 8). 258 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Figure 8. Several improvements to the interface system of the example of Figure 5 include: simultaneous sampling, external ADC and multiplexer timing by a parallel logic program, use of sense lines to reduce conversion channels, detection of random events, multiple sampling frequencies, and/or asynchronous sampling. c. Real-time clock to establish sampling frequency. If the sequential calculation involves numerical integration over a long term, the accuracy of the sampling interval is just as important as the round-off and truncation error in the numerical calculation. Although numerical means may be resorted to for very accurate integration, in a hybrid program the calculations still need to be referred to a real-time base. This is done by using an accurately calibrated source for setting the sampling interval, or frequency. A good high-frequency crystal stabilized oscillator is an important part of the parallel digital subsystem. Sampling frequencies lower than the oscillator frequency are selected by use of preset counters. d. Use of sense lines to reduce number of conversions. In the simple example problem only whole number data are transmitted to the sequential program. Thus, if the relative magnitude (greater or less than) of two analog signals is needed in the digital calculation, the two numbers must be converted, stored, and then compared. This can be accomplished more simply by use of an analog comparator the output.of which is sent to the digital computer by a sense line--saving time and equipment. The state of any parallel logic component may be monitored conveniently by sense lines. These are tested in one memory cycle (2-5 fLsec.). If many such communications are needed the savings will be significant. Sense lines should also be added to allow the sequential program to determine the mod~s of the analog computer, the relative sizes and signs of error quantities, and the states of recording devices. e. Detection of random events. With fixed, periodic sampling the sequential program cannot tell exactly when events take place in the parallel machine. With comparators and parallel logic, complex functions of analog variables can be monitored. For example, it might be required to determine when the overshoot in Xl exceeds X2 after the third cycle, but only when X3 is negative and X4 is less than X5. After determination the sequential program can be interrupted to perform specific conversions and calculations-asynchronously with respect to the primary conversion cycle. In this manner the parallel logic avoids the delays in the sequential program and uses the latter only when required. The parallel logic program analyzes the data, interrupts the sequential program, and sets up the proper channels for conversion in and out of the digital computer ( Figure 9). f. Multiple sampling frequencie.s. In the example problem all channels are sampled at a frequency determined by the highest frequency present in anyone channel. It may often be the case that there are two or more groups of variables with different ranges of variable frequencies. It may then be appropriate to sample each group at different frequencies. Another ap- -XI -x.. x. Figure 9. A parallel logic program is used to detect random events in the parallel computer. The "desired control signal" interrupts the sequential program when the overshoot in Xl exceeds X2 after the third oscillation; but only when Xs is negative and X4 is less than X5. Xl through X3 are analog voltages. HY·BRID COMPUTATION . . . WHAT IS IT? . . . WHO NEEDS IT? . . • proach using different sampling rates is to use several eight channel multiplexers in cascade so that the output of two of them feed two channels of a third which feeds the ADC. On each cycle of the third unit the first two are stepped, yielding different variables for those two channels on each cycle. Alternatively, each time the third steps to the two special inputs the corresponding multiplexer makes a full cycle. Timing control of these operations is performed by parallel logic components (Figure 10). g. Asynchronous sampling. A completely asynchronous conversion system has been designed by one computer laboratory, in which the sequential program is interrupted only by comparators. Twenty analog problem variables are compared to reference values that are adjusted by the digital computer when necessary. Each comparator calls for conversion of some group of variables (the same variables may be called for by different comparators). When two or more comparator signals occur simultaneously or during a conversion operation, two levels of priorities are set up by logic elements to determine what interrupts are to be made. While the system appears complex, it is accomplished in a simple fashion in the paraHei computer and makes good use of the sequential computer time. A longer list of useful variations in the control and timing of sequential/parallel communications can be compiled. For the most part, however, they should be explained in terms of the particular problem applications. Operating Times for Typical Mathematical Functions The repeated emphasis upon the efficient utilization of the sequential program time, high arithmetic speeds, and programming tricks to gain speed can be seen to oe warranted when one examines the sequential operating times for several typical mathematical functions, which, on the analog computer, would be executed continuously and in parallel. a. The sum: a + b + c + d 40 microsec. b. The expression: ax + by + cz 160 microsec. c. Sinwt or coswt 215 microsec. d. Square root of X2 + y2 432 microsec. e. Generate z == f (x, y), where two dimensional interpolation is required between functional values evenly placed in x and y: 0.5 to 1.5 millisec. f. Rotate a vector through three coordinate angles: 2 to 6 millisec. g. Perform one integration of a single derivative for a single time step: 0.1 to 1.3 millisec. A program of three first-order differential equations, where the derivatives are calculated from the functions above (items a through f) would not be a large program; and yet for a single step in time, the calculation time would be about 11.2 milliseconds. dx dt - x+y + z + f (x,y) dy dt == ax - by 2 2 dz dt == x + y ANALOG SIGNALS Figure 10. Cascaded multiplexers, in this or other configurations, can aid in establishing mUltiple sampling rates for different groups of analog signals. 259 + cz • t + Slnz Allowing another millisecond or two for control and input/output instructions one can estimate' the real-time speed performance of this program. The speed is best expressed in terms of the useful upper frequency (at full scale), in a problem variable, that can be represented by the computer. Although the example equations have no real meaning, the frequency limit for such a program is about 1 :1f2 cps. This does not seem like very fast performance for so few, simple equations. On the other hand, it is fast compared to frequencies of some of the vari- 260 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 abIes in an aerospace simulation program for which digital precision is required. In any event, it should be clear for best utilization of the sequential computer, care should be taken to reduce operating time whenever possible. Sampling Rates When parallel and sequential machines are connected in a closed loop it is assumed that at least part of the task of the sequential program is to accept sequences of sampled values of continuous inputs, calculate functions of these inputs, which are then transferred out as sequences of numbers to be smoothed into continuous signals. The digital computer attempts to appear as if it were a parallel computer, and as in a movie projection the effectiveness of this approximation is determined by the ratio of the frame or cycle rate to the rates of change of the signals communicated, and the time resolution of the person or computer receiving the information. Thirty frames per second will not catch the information in the trajectory of a humming bird. A higher rate is needed for an accurate recording of the flight. The human eye, however, cannot resolve time intervals less than 1/30 of a second. Thus an accurate recording can be transmitted to the eye only by a time scale change to slow motion. Fortunately, the parallel computer has a time resolving power sufficient to detect the shortest practical cycle time on the sequential computer, so the limiting factor in determining useful cycle frequencies is the rates of change of the variables that pass between the two computer sections. It is customary to speak of the bandwidth or spectrum of these signals-or more particularly the highest useful frequency that must range over the full magnitude scale. The sampling rate or cycle rate must be selected in terms of the number of discrete numbers or voltage samples necessary to represent this highest frequency at the desired accuracy. In sampled data theory the "sampling theorem" states that the sampling frequency must exceed two times the highest signal frequency if all the information in the original signal is to be retained. 36 That is, some number greater than two samples per cycle is necessary. Another important point comes from the theory: in sampling voltages at the input to the digital computer the rate must exceed twice the frequency of any signal present. If noise signals are present that are higher in frequency than the desired signal and exceeding 112 the sampling frequency it is possible for this noise to be reflected Into the signal spectrum, thereby destroying useful information. This can be avoided with appropriate noise filters. If this were the only limitation it would be fortunate. However, too few samples per cycle makes it difficult for the sequential program to extrapolate and predict what takes place in between samples. I t is possible to do this, of course, by numerical means, but at cost in program time. Furthermore, numerical algorithms for integration are sensitive to the ratio of sampling interval to the rates of changes of the variables, and the calculations may become unstable if too few samples per cycle are used (Figure 11). The most important criterion for determining sampling rates appears in the conversion of the discrete data to continuous analog functions. Two sources of error affect the accuracy of the resultant continuous function. The first is the delay due to conversion and the sequential program itself. The output numbers are functions of input numbers that were sampled at an earlier time. Since the delay is unavoidable, but is predictable, numerical means are used to extrapolate the data to the time of actual digitalto-analog conversion. 18, 28 The second error source is in the mechanism for conversion from Sampling Oemultiplexing a a Multiplexing Smoothing Figure 11. A Sequential-Parallel Hybrid Computer: The noise and delays due to sampling and multiplexing on the one side and the magnitude and phase errors due to demultiplexing and smoothing on the other are dominant factors in determining the proper sampling frequency. 261 HYBRID COMPUTATION . . . WHAT IS IT? . . . WHO NEEDS IT? . discrete to continuous form. A sequence of discrete values fed to a DAC results in a "staircase" analog function. The desired output is a smooth curve passing through each data point (the left corner of each step). If the staircase is smoothed with an analog circuit the result is a smooth curve shifted in time 1hat, passing through the center of each step. The amplitude of this curve is attenuated from what it should be. This technique of smoothing is called "zero order" filtering. (See Figure 12.) The size of the errors is a function of the number of samples per cycle. At ten samples per cycle the magnitude attenuation is about 1.1 % and the phase shift is 18 degrees. At 30 samples per cycle the errors are 0.7% and 6.5 degrees. A first order filter may be applied to the DAC output to reduce the errors. For special purposes higher order extrapolating filters are feasible. These filters are programmed from analog components. The first order filter extrapolates from the last two discrete values to generate intermediate values until the output voltage is reset to the next discrete value from the DAC. The first order filter has a much improved phase characteristic but at low sample rates the magnitude is erroneously accentuated. The IDEAL OUTPUT Figure 12. The discrete to continuous signal conversion requires smoothing and hence yields only an approximation to the ideal output from the sequential computer. The zero order conversion simply holds the output voltage at the last sampled value until the next arrives. When the "steps" are filtered out the result is shifted in time by the width of one half step. ThE:. first o;der scheme uses the past two converted values to predict the value between points, before "resetting" to the next value. 0.1 % ,'- " \ ... ----- ~ Zero Order " Sample I Cycle 10 ,, ~ 20 " " 30 40 -- ---- 50 Figure 13. Zero and first order conversion methods are compared, after filtering. The errors are functions of the number of samples per cycle of full scale signal frequency. A good rule of thumb for the zero order filter is that 50 samples per cycle, the errors are less than 0.1 % and 3.6 degrees. error characteristics of the two filters are shown in Figure 13. A good rule of thumb for the simple zero order filter is that for 50 samples per cycle the errors are less than 0.1 % and 3.6 degrees. The above rule is convenient for estimating the required sample rate and hence the time available for the sequential program. If the variables converted from digital vary at a maximum frequency of 2 cps, then 100 samples per second are needed, and 10 milliseconds is the cycle time for the sequential program, for primary calculations and input/output operations. Dynamic Range of Dependent and Independent Variables The time resolving power of the analog computer was mentioned above in connection with an analogy to the resolution of the human eye. It is instructive to pursue this concept further. The time resolving power of a computer is measured by the shortest time interval that can be accounted for in a calculation. For all signals in an analog computation, the resolution is directly related to the bandwidth of the components; however, the computer's ability torespond to on-off signals and very short pulses, or 262 PROCE.EDINGS-,SPRING JOINT COMPUTER CONFERENCE, 1964 to discriminate between two closely spaced events is a closer description of time resolution. In a digital computer the absolute minimum resolution might be taken as the time to execute three instructions; however, within a hybrid system the resolution of the sequential program is either the sample interval discussed above, or at best, for asynchronous operation, the time for a complete interrupt program to respond to an event. In the parallel computer the time resolution is, of course, much greater because computing elements need not be time-shared. For relay circuits the resolution is about 1 millisecond, for electronic switching of analog signals from 10 to 100 useconds, and for parallel digital operations from 0.1 to 10 useconds. If these numbers are compared to the total length of a typical computer run, say 1.5 minutes, computer time resolution can be measured by a nondimensional number: Parallel digital logic operations .. 1: 109 Parallel digital arithmetic 1: 101 operations Parallel analog, electronic switching 1: 5 X 106 Sequential program, minimum useful program 1: 3 X 105 Parallel analog, relay switching 1: 105 Sequential program, typical sampling 1: 5 X 103 The digital computer is employed in a simulation where the dynamic range of dependent variables requires a wide dynamic range (reciprocal of resolution) in the computation. It is seen that resolution of the independent variable is traded for that of the dependent variables when a particular calculation is moved from analog to digital computer. Figure 14 shows these functions plotted against each other for different computers. The flat part, or "operating range," of the sequential computer plot is seen to be limited on one end by truncation error and the 'other by round-off error. This is to be interpreted as meaning that for a given set of mathematics short cuts and approximations may be used to improve the time resolution up to a point where the truncation error becomes serious. On the other end, special techniques may be used to reduce round-off error, including double precision operations, at the expense of time resolution. REAL TItt£ FREQ. (CPS) z 2 ~ § 3I iIII II: ~ ~ ~ 10 10 1 f"1O •Z 10 10 ~ 10 z 10 I!: 10 ~ 7 & .... 10 I I ! I 7C ~ I i I " ... .... , r--_ DDP 24 ........... 1 r--- . PlIRAW LDIGI ~AL .,r\ " 2 I \ r\ "AU AillL 'ANAL PG 1 100 1000 I \ ................ , 10 Z ~ .' 0.1 ~ 1'0.... ................ ....... I I I I 10 10 J MIN. 2 I SEC. I 10 1 I 10" I 10' 1m. SEC. ........ I I 10& 10 I 7 10" 10' I" SEC. INDEPENDENT YARIA8l.£ OWE) RESOLUTION Figure 14. Certain performance characteristics of computers can be deduced from this comparison of the available resolution in the computer representation of dependent and independent variables. Notes: (1) The limits on the "Parallel Digital" curve are given as those of the EAI Hydac, 0.5 ,usec. in time and 16 bits in magnitude; (2) the attenuation on the right hand end of the DDP-24 and 7094 curves corresponds to truncation errors; (3) the slope at the left of these curves corresponds to reduction of round-;off errors at the expense of speed including use of double precision methods (broken lines); (4) The "Real Time Frequency" scale refers to signal frequencies passed by the sequential program (DDP-24, 7094) assuming a maximum sampling rate of 100 samples/cycle. When the digital computer curves are related to the frequency scale, at the top of Figure 14, a particular size program must be considered. For example, at the 2 cps point, six second order differential equations for a trajectory simulation could be calculated, for throt point corresponds to a 10 millisecond (bottom scale) sequential program time interval. At the 20 cps point, either very crude integration algorithms and approximations are used for the same problem, or one would be considering a much smaller calculation. Moving to the left on the curve, more time is available either for more accurate calculation or for computation of more functions. The useful operating ranges for the different techniques are evident from this figure, and this point of view should be useful to the problem analyst in considering hybrid simulation. FORMULATION OF THE SIMULATION MODEL AND PROGRAMMING Mathematical analysis of the behavior of physical processes and systems is a basic tool HYBRID COMPUTATION . . . WHAT IS IT? . . . WHO NEEDS IT? . . . for the design engineer, and the frequency of its use has been growing for many years at an increasing rate. At first, analysis was restricted to the smaller elements in a system, to linearized approximations, or to phenomena that can be isolated from interaction with its environment. For example, there have been many studies of noninteracting servo control loops, heat diffusion in devi,ces of simple geometry, the "small signal behavior" of aircraft and their control systems, and batch and continuous chemical reactors of simplified geometry. Analysis starts from a consideration of the basic laws of physics as applied to the process at hand and proceeds to develop a mathematical model. The solution to the equations of this model for a range of the independent variable (s) constitutes a simulation of the process. The nature of the designer's task and the very fact that the analysis has been limited to an element of a more complex system, requires that many such solutions be calculated. The simulation is performed numerous times over to determine the variations in the process behavior with changes in (a) internal design parameter~ of the process, and (b) environmental conditions. Electronic computers, of both types; have aided immeasurably in reducing this task to a manageable one. The facility in obtaining simulation results ~tp~t compu·ters have afforded the designer and analyst has accelerated the general acceptance of the analytical approach to difficult design problems. In addition to the wider use of simula.tion the successful correlation of experimental results with analytical predictions has built confidence in these techniques, which has led to the undertaking of simulations of more complex systems. It was once felt that a simulation model could be made so large that the analyst would have difficulty coping with the variables. Indeed this can happen, when poor engineering judgment leads to a model with many more variables than known conditions and assumptions. However, a complex model carefully built up from verified models of subsystems may lead to valuable results attainable by no other means. Thus as analysis and simulation have yielded understanding of the behavior of small systems, a natural process of escalation has led to simulation of systems of greater and greater complexity. The increased sophistication of simulation models has made the analyst even more 263 dependent upon computers for effective control of the simulation and for interpretation of results. The rapid growth of analytical methods and the exploitation of computer technology have paralleled an even faster expansion in complexity of engineering systems. Aerospace systems, moon missions, space satellite laboratories, nuclear reactor power systems, and automated chemical plants are examples of engineering systems that are so expensive and/or potentially hazardous that the design cannot be undertaken without computer simulation to predict ultimate performance. It is no longer feasible to restrict analysis to linearized or isolated subsystems in development of such systems. As technology continues to expand reliable methods must be found to predict, by simulation, the performance of total systems. For only with such analysis and prediction can decisions, involving capital investment as well as design features, be made. It is in this context that hybrid computation can be seen to fulfill a growing need. One might well ask what are the implications of t.his escalation of complexity. If simulation models must necessarily grow larger, just ho\v does this affect the procedures of analysis, computer progr~"'TIming, computation, and inteL-vretation of results? How are the hazards, which were o~ earlier concern, of becoming overwhelmed with useless data and meaningless computation be avoided? There is a pernicious theory about programming for very large digital simUlation, that says if two men can do the job in six months, four men will take twelve, and eight men would never complete it. How can the step from mathematical model to the first computer simulation run be held within bounds--to avoid inordinate investment in programming that may never work or may have to be scrapped for a better approach? How can the analyst or design engineer stay in touch with his model? Surely there are no answers that yield guaranteed results. But these are serious questions and some direction is needed in order to evaluate properly the true potential of advanced computer techniques. The implications in the field of hybrid simulation may be divided into three categories: 264 PROCEEDINGS-SPRING JOINT COM·PUTER CONFERENC.E, 1964 (a) Model building in programming (b) Software (c) Automation Model Building in Programming The analyst, design engineer, and programmer of a large hybrid simulation must all (if they are more than one person) become involved in all phases of the simulation process. Responsibility cannot be divided up, as it often is at the digital "closed-shop" facility, between analyst and computer programmer. The hybrid computer laboratory must be operated, as many analog laboratories, on an open-shop basis with expert programming support available from the laboratory. The design engineer must have a genuine understanding of the computers to be used, even though he may not do the actual computer programming. Since the' computer actually becomes the model of his system, he must know the limitations imposed by the machine as well as by the mathematics, and he must be able to communicate effectively with the computer. Moreover, during the construction of the mathematical model the analyst must keep in mind the features of the parallel and sequential parts of the hybrid computer in order to achieve a proper partitioning of the mathematical model to suit the computer. Much attention has been given here to the relative speeds of computation inherent in the different computing techniques. It may be evident at this point that the presence of a very wide range of signal frequencies in a system to be simulated is the one characteristic that most clearly indicates the need for a hybrid computer. As an example, consider the simulation of a long range flight of a space capsule. In "real time" the position coordinates probably vary at 0.01 cps over most of the range, and, at most, at 3 cps during launch and re-entry. At the same time, pitch, roll, a!ld yaw rates and thrust forces may reach 10 cps or more. Adaptive control functions and control surface forces may have transient frequencies as high as 50 cps; and a simulation of reaction jet control forces may require tO'rque pulses as narrow as one millisecond. Since there is little or no damping in an orbital flight,these pulses have a long term eifect, and accuracy in their representation is important. If an on-board predictive computer is included in the simulation, iterative calculations on the analog computer may involve signal frequencies of 100 to 1000 cps. Thus, this simulation spans a frequency range of 105 as well as a dynamic range in some dependent variables of 105 or 106 (Figures 14, 15, and 16). The following observations may be fairly apparent, but in considering division of a problem between computers it is well to note the types of mathematics for which each is best suited. The forte of the digital computer is the solution of algebraic equations. If the equations are explicit, the calculation time is easily determined. Implicit equations often require a variable length of time, and if there are not too many of them they may be readily solved continuously on the analog computer. Numerical integration comes as a by-product of the computer's power in solving algebraic problems. Time is the only penalty; If the high precision is not needed, the integration is better done by the analog computer, for the solution of ordinary differential equations is its strong feature. Evaluation of arbitrary functions is performed with ease by both computers, as well as by parallel digital components; however, if Figure 15. The relative speeds of computation inherent in the different computing techniques are important considerations in the planning of a computer simulation of a large complex system. The presence of a wide range of signal frequencies in a system to be simulated is the one characteristic that most clearly indicates the need for a hybrid computer. 265 HYBRID COMPUTATION . . . WHAT IS IT? . . . WHO NEEDS IT? . . . Logic equations that must be evaluated continually with respect to their relation to analog variables clearly must be programmed with parallel logic elements. On the other hand, decision and control functions that have to do with the occasional evaluation of states of the computer and errors signals and with sequencing various sections of the total system through different modes and states of operation, may require both parallel and sequential oPerations. Hybrid simulation requiring solution of partial differential equations opens a whole new subject for discussion. Let it just be said that although there is very little practical experience in this field, it appears to offer one of the most promising areas of growth for hybrid simulation. The digital computer approach to the solution of PDE is often limited by available computer time-particularly in simulation problems where it is desired to solve the problem many times for various conditions. The analog computer can solve some PDE problems very efficiently but it is often seriously limited by the necessity of large amounts of equipment for complex problems. Moreover, only with memory to store complete functions (either in a parallel L!_1 _ _ _ _ _ .. Figure 16. (Note: place figure horizontal with the word "time" on the left). A wide range of signal frequencies is suggested here: although not to scale, for a dynamic range of 106 could not be illustrated conveniently. Consider the simulation of a long range flight of a space capsule. The curves might represent (1. to r.): position coordinates varying at 0.01 cps; deviation from a desired path, 0.1 cps; pitch, roll, or yaw rate, 1-5 cps; reaction jet control pulses, 1 ms. pulses at several hundred pulses per sec.; thrust forces or control surface transients, 1-50 cps; iterative calculations for trajectory predictions, 100-1000 cps. speed is important, and if the data are functions of two or more variables, a digital program is the best choice. Simulation of nonanalytic non-linear functions, such as limits, back-lash, dead-zone, stiction, and hysteresis again are amenable to both techniques but analog is probably more economical. Evaluation of trigonometric and hyperbolic functions also can be done both ways and the choice seems to depend on the particular problem. In this case there is a third choice, for there are techniques and equipments for executing these functions by parallel digital components. 37 _l! __ ! L _ ' _____ L ___ ____ ulglLal SyS('~UI or a __ ~ .. ___ S~Ut:U('lC:U L _ _ '\ CUIUPULt:::l-, __ _ can certain kinds of boundary value problems be approached. The hybrid computer has the abiiity to store boundary conditions as well as complete sets of intermediate solutions so that parallel computer programs can be used for speed, but then be time-shared over again for different parts of the space domain. Some very challenging problems and some promising possibilities face the experimenter in this field. Returning now to the implications of the growth of complexity in simulation, the important point in programming is that whoever prepares the computer program must himself be a model builder and be familiar with the system to be simulated. The program should be designed, constructed, and checked out in much the same way that any other kind of model of a complex system would be built. The computer model should be put together from working models of subsystems. Each subsystem, or group thereof, is verified for ;correct performance in a linear or simplified mode before connecting it to other parts of the model. At each point in the expansion of the model, 266 HY;BRID COMPUTATION . . . WHAT IS IT? . . . WHO NEEDS IT? . . . including the final one, at least one test case of a linear mode of operation should be checked against known or precalculated behavior. Computers can extend the analytical powers of man, but they cannot work magic. The computer is a storehouse of answers, but the burden is upon the analyst to ask meaningful questions if he is to receive useful answers. Only with a step by step process, asking questions of the computer at each step, can a valid and useful computer model be built for a large, complex system. Software Standard programs and routines for digital .computers, of general utility to programmers, known as "software," are in such wide use that the production of software is virtually an industry in its own right. "Automatic programming systems," which make computer programming easy for the non-computer expert, are responsible for the almost universal acceptance of digital computers in scientific research and development. The development of sophisticated software has made it possible to increase computer utilization, to gain wider use of computers with minimum training of personnel, and to reduce duplication of programming effort for programs of general utility. A total dependence upon automatic programming has the disadvantage of isolating the problem analyst, and ·even the programmer himself, from the computer. The analyst is restricted from communicating with his computer model while computation takes place. The programmer is limited in taking full advantage of the computer's special features. On the other hand, in the analog computer field the reverse situation exists. No comparable "software" usage has shown up. There is little in the way of automatic programming and preserving of standard programs by analog computer programmers. However, another characteristic of simulation by analog computer is that the problem anayst is involved in building the computer model and he maintains rapport with his model during the simulation. The role that software must play in hybrid simulation of large complex systems is evident. "Hybrid software" must ease the programmer's buraen, as it does for the digital programmer, and at the same time it must bring the analyst closer to his model rather than isolating him from it. Hybrid software must include not only coding for the sequential computer but also interconnection diagrams and prewired patch panels for the parallel machine. The following types of software are needed to support growth of hybrid computation to meet the simulation needs of today. a. Compilers and Assemblers. Conventional compilers and intepreters have their useful place in hybrid computation, to aid in processing data prior to computation and processing results for interpretation. Automatic programming systems for hybrid computation may differ from conventional systems only in three ways: running time of the objeCt program is minimized at the expense of compiling time; actual running time for each program statement is precalculated or estimated to aid the programmer with timing of the parallel/sequential interface; and while the programmer's language is "problem oriented," it is required to be machinedependent. The programmer must be able to utilize special machine features and to program for control of all interface operations. b. Utility LibranJ. In addition to the conventional utility routines for mathematical functions, format conversions, and input/output operations, the hybrid simulation library should expand with routines for specific transfer functions of useful subsystem models that have become standard and are used in larger models, e.g., a typical servo controller. Another type of example is a function generator program to any number of aerodynamic functions. A standard program for a complete aerodynamic vehicle simulation is also feasible. c. Input/Output Routines. Direct on-line control of the computer model by the analyst is needed. In a convenient language it must be possible to experiment with time scales, parameter values, and even to make substitution of mathematical algorithms (particularly integration algorithms), without any penalty in running time. This means that a complete symbol table and all definitions of parameters must be available to an executive routine that will accept operator instructions to modify a particular problem variable and proceed to calculate HYBRID COMPUTATION . . . WHAT IS IT? .•. WHO NEEDS IT? . . . changes in all the "machine va·riables" that are affected. The executor also permits interrogation of the state (and time history) of any problem variable, in engineering units. Upon operator command the model can be modified by changing the linkages between submodels or subroutines. It may, for instance, be desired to linearize part or all of the model for checking purposes, or to isolate or "freeze" certain parts of the model. In monitoring and adjusting the model the executor must not be limited to the sequential computer but should have full access to the parallel elements in the computer system. Automation One important aspect of hybrid computation, which is barely mentioned above, is the opportunity for automating much of the routine parts of programming and check-out of the analog computer. It is evident that a necessary feature of any hybrid computer system is the mechanization of, and sequential program control of, as many of the manual operations on the analog computer as possible. This includes setting of potentiometers, switches, modes, time scales, recorders, and the selector and readout system. This kind of control is import....ant for some of the software functions mentioned above. Also it makes possible automatic set-up, testing, and diagnosis of machine and program faults. Some interesting diagnostic programming for such a hybrid computer was developed in 1959-60 at General Electric MSVD in PhHadelphia. 15,17,32 A different type of programming automation is offered by the Apache system developed at Euratom, Ispra, Italy, for the IBM 7090 and P ACE analog computers. 14 This is a digital program that translates a mathematical statement into detailed programming instructions for the analog computer. While Apache is not intended for hybrid computing the appropriateness of such a program should be apparent. One last important characteristic of hybrid simulation concerns the automation of the model building process itself. Simulation inherently involves trial and error experimentation. The elements of a model are verified; sensitivity to environment is explored; and variation of performance due to parameter changes are evaluated. When a criterion for optimality can be 267 specified, experiments are made to obtain optimum performance. The sequential computer is perfectly suited to the automation of these procedures. Between calculations the digital computer can evaluate the results, decide upon changes to the model or the data, and implement the changes. At the same time the analyst can monitor the progress of the simulation and interrupt the automatic process whenever human judgment is required. CONCLUSIONS The main points developed in this paper may be listed simply. a. Hybrid computation is built upon the technology of analog and of digital computers and is equally dependent upon the programming methods, software, and procedures of problem analysis that have been developed for each. b. A hybrid computer is a compatible system of parallel computing components, both digital and analog, and a stored program sequential machine. The hybrid programmer must be constantly aware of the relative timing of computationa! events in the para!!e! and sequential parts of the system. c. There is an ever growing need for simulation of very complex engineering systems. The process of analysis and building of a computer model for evaluation and prediction of behavior are a required step in many large development programs. The hybrid computer offers a means for many such simulations that would- be impractical by other means. Hybrid computation is inherently a tool for very complex simulations rather than simple studies. BIBLIOGRAPHY 1. BARNETT, R. M., "NASA Ames Hybrid Computer Facilities and Their Application to Problems in Aeronautics," Internati01zal Symposium on Analogue and Digital Technique Applied to Aeronautics, Liege, Belgium (September 1963). 2. BAUER, W. F., and WEST, G. P., "A System for General Purpose Analog-Digital Computation," JACM, Vol. 4, No.1 (January 1957), p. 12. 268 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 3. BAXTER, D. C., and MILSUM, J. H., "Requirements for a Hybrid Analog-Digital Computer," National Research Council of Canada, Mechanical Engineering Report MK-7, Ottawa (October 1959), and American Society of Mechanical Engineers, Paper No. 59-A-304. 4. BERTRAM, J. E., "Effect of Quantization in Sampled-Feedback Systems," U.S. Government Research Reports, Vol. 31 (January 16, 1959), p. 19 (A), PB 133341. 5. BIRKEL, G., JR., "Hybrid Computers for Process Control," Communication and Electronics, No. 52 (January 1961), pp. 726-734; discussion, p. 734. 6. BLANYER, C. G., and MORI, H., "Analog, Digital, and Combined Analog-Digital Computers for Real-Time Simulation," Proc. EJCC (December 1957), p. 104. 7. BURNS, A. J., and Kopp, R. E., "A Communication Link Between an Analog and a Digital Computer (DATA-LINK) ," Grumman Aircraft Engineering Corp., Research Department, Research Report RE-142 (October 1960), ASTIA, No. AD 244 913. 8. , "Combined Analog-Digital Simulation," Proc. EJCC, Vol. 20 (December 1961), pp. 114-123. 9. CAMERON, W. D., "Determination of Probability Distribution Using Hybrid Computer Techniques," International Symposium on Analogue and Digital Techniques Applied to Aeronautics, Liege, Belgium (September 1963). 10. CONNELLY, M. E., "Simulation of Aircraft," Servomechanisms Lab. Report 7591R-1, M.LT. (February 15, 1959). 11. , "Analog-Digital Computers for Real Time Simulation,". M.LT. Report DSR8215, Final Report, ESL-FR-110 (June 1961). "Real-Time Analog-Digital Computation, IRETEC, Col. EC-11, No.1 (February 1962), p. 31. 13. Cox, F. B., and LEE, R. C., "A High-Speed Analog-Digital Computer for Simulation," IRETEC, Vol. EC-8, No.2 (June 1959), pp. 186-196= 12. 1964 14. DEBROUX, A., and GREEN, C., and D'Hoop, H., "Apache--A Breakthrough in Analog Computing," IRETEC, Vol. EC-11, No.5 (October 1962). 15. FEUCHT, K.; "Diagnostic Programs for a Combined Analog-Digital System," Proceedings of the Combined Analog Digital Computer Systems Symposium, Philadelphia (December 16-17, 1960). 16. GAINES, W. M., and FISCHER, P. P., "Terminology for Functional Characteristics of Analog-to-Digital Converters," Control Engineering, Vol. 8, No.2 (February 1961), pp. 97,8. 17. GELMAN, H. D., "Evaluation of an Intercept Trajectory Problem Solved on a Combined Analog-Digital System," Proceedings of the Combined Analog Digital Computer Systems Symposium, Philadelphia (December 16-17, 1960). 18. GELMAN, R., "Corrected Inputs-A Method for Improving Hybrid Simulation," Proc. FJCC, Vol. 24 (November 1963). 19. GREENSTEIN, J. L., "Application of ADDA Verter System in Combined Analog-Digital Computer Op2ration," Pacific General Meeting, AlEE (June 1956). 20. HALBERT, P. W., "Hybrid Simulation of an Aircraft Adaptive Control System," Proc. FJCC, Vol. 24 (1963). 21. HARTSFIELD, E., "Timing Considerations in a Combined Simulation System Employing a Serial Digital Computer," Proceedings of the Combined Analog Digital Computer Systems Symposium, Philadelphia (December 16-17,1960). 22. KORN, G. A., "The Impact of the Hybrid Analog-Digital Techniques on the Analog Computer Art," Proc. IRE, Vol. 50, No. 5 (May 1962), pp. 1077-1086. 23. LANDAUER, J. P., "Simulation of Space Vehicle with Reaction Jet Control System" (1962), EAI Bulletin No. ALHC 62515. 24. , "The Simulation of Transport Relay with the Hydac Computing System," EAI Bulletin No. ALHC 63011. 25. McLoED, J. H., and LEGER, R. M., "Combined Analog and Digital Systems-Why, When, and How," Instrument and Automation, Vol. 30 (June 1957), pp. 1126-1130. HYBRID COMPUTATION ... WHAT IS IT? ... WHO NEEDS IT? . . . 26. MITCHELL, B. A., "A Hybrid Analog-Digital Parameter Optimizer for ASTRAC-II," Proc. SJCC, Vol. 25 (April 1964). 27. MITCHELL, J. M., and R UHMAN, S., "The Trice-A High Speed Incremented Computer," IRE Nat. Conv. Record (1958), Pt. 4, pp. 206-216. 28. MIURA, T., IWATA, J., "Effects of Digital Execution Time in a Hybrid Computer," Proc. FJCC, Vol. 24, (November 1963). 29. NORONHA, L., "An Integrated General Purpose Hybrid Computing System," International Symposium on Analogue and Digital Techniques Applied to Aeronautics, Liege, Belgium (September 1963). 30. PALEVSKY, M., "Hybrid Analog-Digital Computing Systems," Instruments and Automation, Vol. 30 (October 1957), pp 18771880. 31. , "The Digital Differential Analyzer," Computer Handbook, Edited by G. A. Korn and H. O. Huskey, New York: McGraw-Hill (1961), Chapt. 19. 32. PASKMAN, M., and HElD, J., "Combined Analog-Digital Computer System," Proceedings of the Cmnbi'ii£d Analog D-ig-ital Computer Systems Symposium, Philadelphia (December 16-17, 1960). 33. SHAPmo, S., and LAPIDUS, L., "A Combined Analog-Digital Computer for Simulation of Chemical Processes," Proceedings of the Combined Analog Digital Computer Systems Symposium, Philadelphia (December 16-17,1960). 34. SHILEIKO, A. V., "A Method for Selecting the Optimum Structure of a Digital Analog Computer," Automation and Remote Crmtrol (Avtomatika i Telemekhanika) , Vol. 22, No.1 (August 1961), (originally published January 1961), pp. 76-81. 35. 269 SKRAMSTAD~ H. K., ERNST, A. A., and NIGRO, J. P., "An Analog-Digital Simulator for Design and Improvement of ManMachine Systems," Proc. EJCC (December 1957), p. 90. 36. SUSSKIND, A. K., "Notes on Analog-Digital Conversion Techniques," M.LT., Technology Press (1957). 37. VOLDER, J. E., "The Cordie Trigonometric Computing Technique," IRETEC (September 1959), pp. 330-334. 38. WEST, G. P., "Computer Control Experience Gained from Operation of a Large Combined Analog-Digital Computation System," Proc. of Computers in Control Systems Conference, Atlantic City (October 1957), p. 95. 39. WILSON, A., "Use of Combined AnalogDigital System for Re-entry Vehicle Flight Simulation," Proc. EJCC, Vol. 20 (Decembe-r 1961), pp. 105-113. 40. WITSENHAUSEN, H., "Hybrid Simulation of a Tubular Reactor" (1962), EAI Bulletin, No ALHC 6252-25. o 41. , "Hybrid Techniques Applied to Optimization Problems," Proc. SJCC, Vol. 21 (May 1962). 42. WORTZMAN, D., "Use of a Digital! Analog Arithmetic Unit within a Digital Computer," Proc. EJCC (December 1960), p.269. 43. ZETKOW, G., and FLEISIG, R. (Grummen Aircraft Corp.), "Dynamic Analysis of OAO Spacecraft Motion by Analog-Digital Simulation," IRE Convention, Space Electronic Session (March 1962). A HYBRID ANALOG-DIGITAL PARAMETER OPTIMIZER FOR ASTRAC II Baker A. Mitchell, Jr. Deparbnent of Electrical Engineering UniveTsity of Arizona Tucson, Arizona INTRODUCTION This paper describes an optimizer designed to find system parameter combinations which optimize a functional, F, such as '1' F (a" ... ,an) = f [y2 (t) + u2(t)] dt o where y(t) = y(t, at, ••• , an) u(t) = u(t, at, ••• , Simple all-digital logic permits implementation of different sequential optimization strategies, including correlation between randomperturbation vectors and step-size changes depending upon past successes and failures. The analog integrator/multipliers commonly used to set system parameters have bEfen replaced by simple, reversible binary counters driving D / A converters 3 for simplified design and improved reliability.4 The principle' of the optimizer is shown in the block diagram of Fig. 1. an) are state and control variables depending on the unknown parameters at, ••. , all in accordance with the system equations ~Q, ._- y, '.., ~an ·· · ~ System Equations --.: rF F(t,uj,Yj} ~ , .... I Success Detector ~ I The new optimizer is designed to work with a fast all-solid-state iterative differential 'analyzer (ASTRAC II) which is capable of producing complete solutions Yi (t) and the corresponding values of the performance measure F (aI, ••• , all) for up to 1000 new parameter combinations per second.! To simplify optimizer logic and memory requirements in problems involving many parameters, we simultaneously implement random perturbations 2 on all parameters ak and step to the perturbed point whenever the perturbation yields an improvement in the performance measure F (at, ... ,all)' OpfimiHr iLoliJ1c --- S Figure 1. Optimizer Block Diagram. 271 ~ I 272 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 The most frequently used method employs n trial steps to compute approximate gradient components LF / La in each parameter direction; these gradient components are then stored and used to compute the optimal correction Lai for a working step, or for a series of working steps in the same direction. 8 Figure 2a. Optimum Gradient Method. Figure 2b. Pure Random. Perturbations. Figure 2c. Correlated Random Perturbations. Figure 2. Typical Optimization Paths. Figure 2a shows the parameter-space path over which a conventional deterministic system would optimize a simple two parameter system. Starting with a trial set of parameters (lai the conventional optimization logic employs the results of successive differential analyzer runs to obtain succeeding parameter values IOal = 1'-1 + I'LaI \vhich successively improve rF= F (raj, ... , ran). Such deterministic methods require complex logic and storage. Although they may converge well for favorable performance functions F (ab ... , an), they may "hang up" on ridges or in canyons of the multidimensional landscape of the performance measure domain. 5 Furthermore, if the performance measure contains discontinuities, nonlinearities, or large higherorder derivatives with respect to the parameters, our information of past performances will be of little value in determining succeeding -steps; thus the step-size may have to be reduced to such a degree that convergence to the optimum is extremely time-consuming. Figure 2b shows how a pure randomperturbation scheme might optimize the same function. Here, Lai may be positive, zero, or negative with equal probability; and the nominal parameter point is moved as soon as the first improvement in the performance function occurs. No attempt is made to affect perturbations by past results or gradient methods. On the other hand, if perturbations are to be correlated with past successes or failures, then the optimization path might appear as shown in Fig. 2c. Such a scheme causes future increments Lai to favor the direction in which past improvements in the performance function were made. Notice, however, that we still do not require computation of individual gradient components, as in deterministic gradient optimization schemes. Hence, logic and memory requirements are reduced. Motivation for Random Search The basis for all direct computer methods of parameter optimization is the same: using a mathematical model or simulated system, we set the parameters to some trial values and compute the performance criterion. Then according to some rule, we reset the parameters to new values and again compute the performance criterion. This procedure is repeated until some desired degree of improvement is obtained. A HYBRID ANALOG-DIGITAL PARAMETER OPTIMIZER FOR ASTRAC II Naturally it is hoped that the rule for adjusting the parameters will take maximum advantage of the knowledge gained from observing previous trials, and by so doing achieve the optimum set of values for the parameters in the shortest possible time. Usually, however, this rule depends solely upon the knowledge gained from recent past trials and this is thought to be equivalent to using this knowledge to maximum advantage. If, however, the performance criterion contains discontinuities, nonlinearities, or large higher-order partial derivatives with respect to the parameters, our information of recent past behavior (actually a total or partial first derivative) may be of little value for the determination of successive steps; and if the step size is too large, this information from the preceding step (or from a short forward trial step) will be totally misleading. Thus, with conventional, deterministic perturbation such as the gradient method, one may be .(orced to reduce the step size to such a degree that convergence to the absolute optimum is excessively time-consuming. For this reason it has been suggested that randomness be introduced into the search rule-perhaps in proportion to the expected severity of the discontinuities, nonlinec:rities, etc., present in the cost function domain. In reality, it may be difficult to make any reliable prediction concerning the behavior of the performance function. Even with reasonably w~ll~behaved performance functions, it can be quite c;lifficult to foresee "ridges," "temporary plateaus," "saddle-points," and other features which render deterministic rules far from foolproof. 273 will be used to indicate that the incremented set, 6a, has yielded a more favorable value for the performance measure than was obtained with the unincremented set, ai' A failure will mean that the incremented set yielded a less favorable value for the performance measure, in which case the failing increment is subtracted from the parameter before a new increment is added. Successive fail"ures and successes can be counted and used to decide when an increase or decrease in the step size might be ai + Fi~ure 3. The function defined represents a system compo;ed by particular percentage~ of each of the four compounds shown at the corners. Contour lines are drawn in order to indicate values of a property on the system. The heavy lines are lines of discontinuity in slope. S Reference 5 goes further into such motivation for random-search methods (Figs. 3 and 4). PRINCIPLES OF OPERATION Most of the optimization strategies proposed here can be based on the flow chart of Fig. 5. The operation common to all the various strategies consists of incrementing all parameters simultaneously by individual random increments +6a, -6a, or zero; the common magnitude 6a (step size) of these increments is subject to. a separate decision. The term success Figure 4. This sketch of a dynamic vibration absorber shows the amplitude of vibration F plotted over the frequency range w for values of the parameter al. The effect of only one of the three parameters, aI, a2, a3, of the actual system could be drawn. A possible criterion for an optimum absorber is to require that the maximum amplitude of vibration yielded by a particular set of parameter values, aI, a2, a3, be minimum over the frequency range of interest. s 274 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 integrator RESET pulses, and sampling pulses. The digital section has its own patchbay with removable patchboards for implementing various logic functions. Patchable gates and flipflops are used in conjunction with the prewired timing and RESET circuits. In view of the amount of logic involved in the optimizer, however, it was thought best to build it as a separate digital section with its own removable patchboards, devoted to this purpose. The complete ASTRAC II optimizer will not only implement the sequential random search optimization described here, but will permit comparison with deterministic optimization schemes. Figure 5. Flow Diagram for a Typical Optimization Routine. advantageous. A binary noise generator 7 together with digital correlation logic decides what th~ sign of each new increment will be. Thus, the overall effect of the perturbation scheme is an n-dimensional random walk. THE ASTRAC II SYSTEM Optimizer Logic The digital logic of the optimizer is subdivided into basic functional units whose inputs, outputs, and control points are wired to its patchbay (Fig. 6). With a different prepatch panel, these components are also available for other uses besides optimization. In particular, the parameter-setting circuits will also serve for experiments with deterministic optimization schemes. • The dynamic system and the performance criterion are to be simulated on the Arizona Statistical Repetitive Computer, ASTRAC II, although ASTRAC I served for preliminary studies. ASTRAC II is a ± 10 vol,t, all-solidstate iterative differential analyzer capable of iteration rates of 1 Kc. as well as real time computation. The first 20-amplifier section of ASTRAC II is to be completed in the fall of '. ,:, !". - !"!,.- ~; • .!,-~ 1964. The analog section has a large conventionally appearing patchbay. 20 Mc. transistorized amplifiers mounted in shielded cans plug directly into the rear of the patchbay without any intervening wiring. The analog section will comprise sample-hold memory pairs, comparators, analog switches, swi~ched integrators, diode quarter-square multiplifiers, and diode function generators. Timing and logical control is furnished by the digital section, which provides timing pulses, Figure 6. Photograph of the Optimizer. A D I A multiplier card is shown at bottom left. A HYBRID ANALOG-DIGITAL PARAMETER OPTIMIZER FOR ASTRAC II The functional units are built of commercial plug-in logic cards interconnected on racks with wire-wrap terminations for ease of modification and expansion. The resistor networks and switches comprising the D / A multipliers are mounted in shielded plug-in cans adjacent to the operational amplifiers behind the analog patchbay. Shielded digital control lines connect each D/ A multiplier to the optimizer patchbay. Hybrid Analog-Digital Noise Generator ASTRAC II will employ a new noise generator10 producing pseudo-random maximumlength shift-register sequences at any desired clock rate up to 4 Mc. We may obtain either a single pseudo-random sequence repeating after 33 million bits, or four uncorrelated sequences one-fourth as long. Success-Failure I ndica.tor Essentially, the function of this circuit (Fig. 7) is to compare the value of the best performance measure, LF, obtained to date, with the performance measure just yielded by the last com- 275 puter run, rF. If the last run was a fail'/!-re, (e.g., rF < LF for maximization), then U a and ~ S remain zero; and LF is still held as being the best value. If, however,-· the last computer run was a success (rF > LF), then U a becomes "I" which causes the second sample-hold to take on the value just obtained, rF, as now being the new optimum. The pulse U a also causes flip-flop 7 to assume the "I" state (s-o) which indicates to the digital optimization logic that a success was obtained with the last set of incremented parameters. If it is known that F will always be monotonically increasing during the latter part of the COMPUTE period, Switch 3 and U 2 need not be used. Master Clock The Master Clock provides all timing pulses throughout the digital optimization routine. It consists of a four bit Gray-code counter driven by a 1 MC. pulse, Ch from the differentialanalyzer digital control unit. C1 is gated to the counter during the differential-analyzer RESET period. With the exception of the SuccessFailure Indicator circuit, \vhere timing is under a., ___ ~ c, _____---' ~--------' ~ ~ ~~ ~ rlL..~------------ rlL.._____ _____________ rlL..________________ -=~ NN ____________~ R U ______________________~n~ I ~ U3 S PI L-...J M. _ _ _ _ _ _ _ _ _ ~rI~ Pz ________---- L-.J Mz ____________________ ____________________----JnL.._________ n] ____________ If Success occur. r---L- Figure 7. Success-Failure Indicator: P3 M3 c ~rlL.. ________ L-J n~ ____ ~ Figure 8. Pulse Sequence from Master Clock. 276 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 control of the differential-analyzer, no optimization logic is performed during the COMPUTE period of the differential analyzer. Parameter 5 ettin9 ~~~,..--- From~ -x CIICIIyz.r ..... The sequence of the tjming pulses from the Master Clock is shown in Fig. 8. S', as shown, occurs only after failures, i.e., S == 1; if S == 0, S' == 1 throughout the RESET period. A success or failure also causes nH or nl<' to occur (Fig. 15). +x PARAMETER SETTING O\A convert ... I muIIpIIer The flexibility needed to implement a variety of different optimization-logic schemes while maintaining simplicity, reliability, and low cost is achieved by using a unique method of parameter setting. Binary-Counter Operation Referring to Fig. 9, the binary up-down counter increments whenever a pulse appears on the "I" line. The right-left shift register contains zeros in all except one of its stages, and the position of this "I" selects the stage of the binary counter which is to receive the "I" pulse. By controlling the D / A multipliers, the counter has then increased or decreased Ui by an Figure 9. Parameter Setting. S'_-__--.· USi uC', uo Mj uti US;, UP-DOWN LOCK LC LS; LC i Lc Lei LSi 1 S BC C Figure lOa. U -L-D Selector Circuit. Figure lOb. U-L-D Memory. A HYBRID ANALOG-DIGITAL PARAMETER OPTIMIZER FOR ASTRAC II ory associated with the binary counter for each parameter (Fig. lOb). increment 6ai; the magnitude 6 is determined by the position of the "I" in the shift register, and the sign (+ or -) is determined by the logic level on the UD line (1 == count up, 0 == count down). The selector circuit accepts two uncorrelated random bits IN}, IN:! obtained from the ASTRAC II noise generator. Depending on the interconnections of the selector-circuit gates G}, G2 , G:{, G4 to gates U, L, D, the gate outputs Un' Lo, Do will have different joint probability distributions as shown in Table 1. U-L-D Digital Logic The U-L-D (UP-LOCK-DOWN) Logic is composed of two sections: a central U-L-D Selector Circuit (Fig. lOa), and a U-L-D Mem- TABLE I P(U == 0) P(L == 0) P(D == 0) Connect 1 0 0 Gh G2 , G3 , G4 U % ~ 0 G}, G2 , G3 G4 U L % 0 1,4 G h G2 , G3 G4 U D 112 112 0 G b G2 G3 , G4 U L o G· U L ~ . <" • 277 G G G:' 2 to Gates U L " I ~_____________________G_4__________________ .~~ G1 U ~'~ ~ 1,4 1;2 1,4 1;2 0 112 U D 0 1 0 L ~ 1,4 1;2 0 3,4 1,4 0 112 112 1,4 0 3,4 0 1,4 3,4 0 0 1 Effectively, these distributions can be obtained by placing a logical "I" on the RC line. I I 278 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 The gate outputs, Uo, Ln, Do, determine the states of the dc set and reset level controls on the UP-DOWN flip-flops and the LOCK flipflops of all U-L-D Memories (one for each parameter) simultaneously. The pulse Ml from the Master Clock now first sets the U-L-D flipflops of the first parameter to their proper states as determined by the first set of random bits IN h IN 2 • Next, the noise generators are pulsed again by the Master Clock (pulses NN) producing two new random bits, 2Nh 2N 2, which determine a new set of states for the dc set and reset level controls of the U-L-D Memory flip-flops. Now, the pulse M2 to the U-L-D Memory of the second parameter sets its UP-DOWN and LOCK flipflops in accordance with 2N h 2N 2. The process repeats for the remaining parameters. U-L-D Memor'lj Note that we shall require two decisions for each parameter. The LOCK circuitry decides whether ak is to be incremented or not incremented ("locked"). The UP-DOWN circuits decide the direction (up or down) of the increments Dak, if any. We shall consider the UPDOWN circuits first. failure or a success, if a particular parameter had been locked (Le., its last increment was zero) then a probability function giving equal weight to all three states might be desirable for the next trial. Referring to Fig. 11 the correlation circuit is patched as desired and senses the sign of the last increment in each U-L-D Memory, starting with the first parameter. If Dak is positive, the RC line changes the selector gating so as to- increase the chance of a positive Dak for the next run if the last run was successful. The reverse can take place if the last run was a failure, depending upon the correlation control "CC". If, however, Dak was zero, as indicated by the state of the LOCK flip-flop, then the correlation line RC is turned on or off with equal probability by a random-noise input. This entire process is repeated for each parameter in turn, as they are sequenced by the pulses Pi from the Master Clock. S5 CC If an UP-DOWN flip-flop is set (reset), a logical 1 (0) will appear on the "UD" line to the binary counter, if S' == 1. We now come to the LOCK decision. If the LOCK flip-flop is set or reset the incrementing commands "C" and "BC" from the l\1:aster Clock will or will not carry through the gates on "I" to increment the counters in the direction dictated by "UD" RC Correlation Circuit This circuit can be patched so as to introduce correlation between successive random perturbations according to some strategy selected to speed optimization. Suppose that the last set of increments succeeded in improving the performance function. Assuming that the performance function is fairly well-behaved, the greatest chance for another success lies in weighting each parameter so that it, will probably increment in the same direction as in the previous trial (strong positive correlation). By the same reasoning, after a failure a strong negative correlation might be introduced. After either a cc- I I I I I I I I t nevCltive correlation after toilure pOlitive correlation after nevotlve nO and pOliti". lucce .. canelatlon correlation Figure 11. Correlation Circuit. A HYBRID ANALOG-DIGITAL PARAMETER OPTIMIZER FOR ASTRAC II OPTIMIZATION STRATEGIES 1. Pure Random Perturbations Pure random perturbations are achieved if we preset the "1" in the shift register to gate· the count pulse into the binary counter stage yielding the desired step distance, e.g., V2, 14, ... , 1/128. The noise generator is connected directly to the SET and RESET level controls of the Up:'Down flip-flop of each parameter. During the analog RESET period, the noise generator is pulsed once for each parameter shortly before the sequenced pulse to the proper Up-Down flip-flop arrives. Thus, the Up-Down flip-flops sequentially assume the successive states of the noise source. The count pulse, C, is then fed to all parameters simultaneously. Hence, all parameters increment by the same magnitude but with random signs during each analog RESET period, executing a random walk in the perturbation scheme. With pulses S, S' and BC added to the U-L-D Memory, the set of increments used in the preceding run can be subtracted from the counters prior to the addition of increments for the next run. Thus, after a failure in an optimization run, the counters can be returned to the state which yielded the last success, before making another search. 2. Random Walk with Reflecting Barriers This scheme is exactly the same as the pure random walk with one additional operation. Prior to assigning a new set of signs to the Up-Down flip-flops, a pulse to signify that a parameter has reached a barrier-possibly from a comparator in the analog system-can be gated to the U-L-D logic of the parameter to be reflected, causing its Up-Down flip-flop ~o complement and, later, to ignore the sign that is assigned to it by the noise source. In this manner, the parameter will have been reflected to its old position held two analog COMPUTE periods previously. 3. Random Walk with Varied Step Size This scheme also contains only one addition to the pure random walk. When it is desired to increase or decrease the step size-possibly after a certain number of successive failures 279 have been counted-one has only to insert the increase or decrease command, e.g., the counter output, into the shift-right or shift-left input of the shift register. This gates the count pulse either to the next higher or next lower parameter counter stage. The noise generator then assigns polarities to the Up-Down flip-flops, and the parameters are incremented by the new step size. For this purpose, the optimizer contains two success-failure counters. 4. Correlated or Biased Random Walk Since two independent noise sources are available, one can arrange P(N 1 N 2) == 14, P(N 1N2) == 14, pd~lN2) == 14, P(N 1N 2) == 14. These can combine to give P (X) == lh, or ~ P (Y) == 34 , where X == N IN2 + N IN2, Y == NIN2+NIN2+NIN2. There are three possible states for the U~L-D Memory: 1. Up; 2. Down; 3. Lock. If a failure occurred in the Up state, after subtraction of the failing set of increments, it would be desirable to assign weighted probabilities which would be more likely to result in the next state being Down. The probability distributions which can be' prepatched are listed in Table I. Likewise, if a success occurs in the Down state, the next assignment of states could be weighted on the same basis. ~ ~ This strategy can be combined with step size variations implemented with the aid of the success/failure counters. TESTS The following tests were carried out using the optimizer in conjunction with ASTRAC I, ,a ±100 volt, 100 run/sec. iterative differential analyzer. In order to generate performance measures with the precise characteristics desired, only algebraic functions F (aI, a2) were used for quantitative evaluation of the various optimization strategies. For practical use in optimizing dynamic systems, the optimizer setup would be entirely unchanged, except that the function F (ah a2) would be generated as samples at the end of a differential-analyzer run. 280 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Also, the relatively slow repetition rate of 10 runs/ sec. was employed only for recording purposes. were allowed to vary over ±100 volts. Convergence was considered to be satisfactory when F was within 0.5 volt of the maximum. Functions Optimized Initial tests were made with two different types of performance functions. Strategies Employed Parameters were allowed to step only after successful runs, i.e., unsuccessful parameter changes were subtracted out. Minimization of functions of the general type (6) Various types and degrees of correlation between successive parameter runs were tried. (See Correlated and Biased Random Walk.) was carrIed out. The parameters al and a:.! were allowed to vary over ± 100 volts, and convergence of al, a:.! was considered to be satisfactory when they were within 0.78 volts of their values which optimized F. The step size, 6, was increased by a factor of 2 after N R consecutive successes and decreased after Nl<' consecutive failures; Ns and Nl<' were varied. (al-kd 2 + (a!!-k:J2 a b The next experiment involved maximization of a function F (al, a:.!) exhibiting sharp ridges formed by the intersection of three planes as shown in Fig. 12a. The equation for this function is . Max a:!-al -a:,! +2al -a:,! 1.5al = SOK ~501( 1M 100 + -3- (7) cfJ c = (optimum) 66.6; al (optimum) 44.4 (8) Extremely sharp ridges or intersections were simulated by using accurate high-speed selector circuits (Fig. 12b). The parameters al and a:! a:! ISOK +300 - oi, /501( SOl< -dO 0 - aLl. + F c.. ." 1:[': I ..,.., I: " ,,:;, ... .:.,,: Ii',:. I , ..'J ,., ~ I: 1--' , I' I .. I:il,' ;,11 "IT lh :f :1 :;, ""'"! " !J:I !!I! III 1'1 : I' :i j,L,' ":' 1:; .,,1',·' c· .!,,"" "., j"L",i .. :- Figure l3b. Optimization of Ridge. "! '1 282 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 ginning of both trials No. 1 and No.2. Thus, it was not necessary to climb the longest ridge. The paths converged more quickly up the shorter ridge (path B in Fig. 12c). This favorable possibility is not present in ordinary gradient techniques. Path A in Fig. 12c is most nearly like a path resulting from conventional gradient techniques. Fig. 14a shows a correlated random search with step-size variations employed in the strategies. The optimizer is not confined to merelyzigzagging slowly up the ridge but can traverse great distances while searching for improvements. Correlation im-proved convergence time somewhat, but only when weak positive correlation after successes was used. For 45 trials using this correlation, the average time to converge was 73 runs if the step-size was decreased after 2 failures. Other types of correlation slowed convergence by 10--.:20 per cent. For comparison note that an optimizer going directly in a straight line to the optimum point would require 106 steps if a fixed step size of 1.56 volts were used. Stopping Conditions When the optimum value of F is unknown, defining a stopping condition is subject to several factors. One such factor arises when we are not certain that only one peak exists in the domain of F. In this case, we would want to cycle the step size through its decrease-increase scheme several times before stopping. Then a rescaling of the simulation might be desirable in case the initial ranges of the parameters was chosen too large. Conclusions While experience with this optimizer is still quite limited, it appears that its performance can compare quite favorably with conventional techniques. Better conclusions can be made when this system is expanded to accommodate four parameters and the logic is enlarged to permit implementation of the conventional deterministic schemes-permitting a direct comparison between random and deterministic methods optimizing the same function. Presently, however, the random techniques seem well able to handle cases in which the performance measure is not well-behaved (Fig. 3 and 4). Figure 14a. Optimization of Ridge. Figure 14b. Employing Step Size Variations. ACKNOWLEDGEMENT The project described in this report is part of a hybrid analog-digital computer study directed by Professor G. A. Korn. The writer is grateful to the Office of Aerospace Research, Information Research Division, Air Force Office of A HYBRID ANALOG-DIGITAL PARAMETER OPTIMIZER FOR ASTRAC Scientific Research and to the Office of Space Sciences, National Aeronautics and Space Administration for their continuing support of this study under joint grant AF -AFOSR-89-63; and to Professors L. W. Matsch, Dean of Engineering, and H. E. Stewart, Head, Department of Electrical Engineering, for their encouragement and contribution of University facilities. REFERENCES 1. KORN, G. A., and T. M. KORN, Electronic Analog and Hybrid Computers, McGraw- Hill, N. Y., 1964 (in print) . 2. MUNSON, J. K., and A. I. RUBIN, "Optimization by Random Search on the Analog Computer," IRE Trans. PGEC, June, 1959. 3. WAIT, J. V., and B. A. MITCHEL~, "A Simple Solid-State Digital-to-Analog Converter for Hybrid Systems," ACL Memorandum No. 61, University of Arizona, 1963. 4. MITCHELL, B. A., "A Hybrid Analog-Digital One Parameter Optimizer," Ann. AlGA, January, 1964. 5. KARNOPP, R., Ph.D. Thesis, Massachusetts Inst. of Technology, 1963. 6. BROOKS, S. H., "A Comparison of Maximum-seeking Methods," Operations Research, JulY-August, 1959. 1 II 283 7. HAMPTON, R., G. A. KORN, and B. A. MITCHELL, "Hybrid Analog-Digital Random-Noise Generation," IEEE Trans. PGEC, August, 1963. 8. WITSENHAUSEN, H. S., "Hybrid Techniques Applied to Optimization Problems," Proc. SJCG, 1962. 9. HOWELL, M., "Automatic Parameter Optimization as Applied to Transducer Design," Proc. SJCG, 1963. 10. HAMPTON, R., "Hybrid Analog-digital Pseudo-random Noise Generator," Proc. SJCC, 1964. APPENDIX I The following units are contained in the optimizer digital and/or are wired to the optimizer patchbay. On the drawings, a small circle on the end of a wire indicates a patchbay termination. 1. 3 4-bit Gray code counters. One of these counters has a free-running multi vibrator for the input. This counter is used exclusively for sequencing operations throughout the logic scheme subroutine, e.g., the analog RESET period. The remaining two Gray code counters have their flip-flop outputs adjacent to two gates on the 1 n Figure 15. Gray Code Counter Circuit. 284 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 patchboard. These may be patched to yield outputs after any preset number of input pulses. These counters may reset themselves or may be reset externally (Fig. 15.) 2. 1 8-bit right-left shift register. The DC set and reset lines, along with the set and reset outputs, appear on the' patchboard for parallel drop-in on command. Also the set and reset level controls for the first and last flip-flops are on the patchboard thus permitting operation as a ring counter. Four inputs are provided for shifting in either direction. (Fig. 17b.) 3. 4 8-bit up-down binary counters. The DC set and reset lines, along with the set and reset outputs, appear on the patchboard for parallel loading. A gated complement input for each flip-flop is brought out, allowing the counter to be stepped at any stage. A flip-flop with associated gates permits both pulse and level control of the up-down lines. (Fig. 17a.) w RIGHT-LEFT SHIFT REGISTER x ____ w' ---- x' y - - - - y' Shift "'vhf Shift Lei'. Figure 17a. UP-DOWN Binary Counter. •• -x z. PsludoRandOll! Nai,. G.n.r-'or ~. Ma ...r Clock ~ Gro, Cod. lola Count.,. NI ...-~SlIift t.' R.... '!. " 51 .5% ..L Tolerance I ~o Re.lator •• It . .. . ... 'K "I "2 G· , 10 • ProbOblllty Oiatributlan aX Selector Count.r 1K ,. ,. 4K RC 10. ZO· I. 10. GI Noi •• Corr.lotlon Circuit HI Count.r PI I. Gj Figure 16. D I A Multiplier and Switch. Figure 17b. Right-Left Shift Register. OIA A HYBRID ANALOG-DIGITAL PARAMETER OPTIMIZER FOR ASTRAC II 4. 4 8-bit D / A multipliers. The digital control lines for the D/A multipliers (mounted in shielded cans behind the analog patchbay) are wired to the digital patchbay. Each set of lines is associated with one of the up-down binary counters (Fig 16.) 5. 2 pseudo-random discrete-interval, binary noise generators. The noise generator for ASTRAC II is a 25-stage shift register with modulo 2 adders in feedback, generating a maximum length of 3XI0 7 random bits. This can be divided into two noise generators, each having a 1.5XI07 random bit sequence. The noise generator shifts out a random bit when a pulse is applied to the shift "input. These inputs, outputs, and complement outputs appear on the optimizer patch bay as' well as on the noise generator patchbay. Figure 18. Digital Logic Block Diagram. 285 A HYBRID ANALOG-DIGITAL PSEUDO-RANDOM NOISE GENERATOR R. L. T. Hampton College of Engineering Department of Electrical Engineering Analog Hybrid Computer Laboratory The University of Arizona Tucson, Arizona from stationary mechanisms such as a radioactive source, still create problems and expense.! For example, the 80 Kc random-telegraph wave generator developed at The University of Arizona's Hybrid Computer Laboratory and described in Ref. 2 required a iairly sophisticated and not completely satisfactory countrate control loop. I. INTRODUCTION Analog and hybrid analog-digital computers used for random-process and statistical studies require noise sources capable of generating random signals whose amplitude distribution, d-c unbalance, spectrum, and RMS level are specified within the computer accuracy limits. Noise samples must not be correlated for time delays exceeding one-ten-thousandth to onethousandth of a computer run. In the design of the University of Arizona's new ASTRAC II iterative differential analyzer, which is to be capable of taking statistics over 1,000 random-input computer runs per second . ' It was decided to abandon analog noise generation completely. Instead, the machine win em.. ploy a digital shift-register sequence generator that can produce binary pseudo-random noise sequences at any clock rate between zero and 4 Mc. This permits exact time scale changes or intermittent operation. The noise generator produces digital computer random numbers as well as analog noise. The digital numbers are easily stored or transmitted and may be used to produce binomially or normally distributed random analog coefficients. Digital multiplexing yields multiple uncorrelated noise signals from a single shift-register. These noise signals are independent of any physical quantity except for the output clamping levels. Also the flat spectrum binary output permits direct logicalor analog multiplication. The length of the Noise derived from thyratrons in magnetic fields, noisy diodes and photomultiplier tubes coated with radioactive paint changes with various environmental conditions. Thus, elaborate gain-control, sampling and/or filter circuitry is required in order to meet computer specifications. By using the random signal from such sources to trigger a flip-flop or thresholdsensing comparator, flat-spectrum binary noise with ±A volt output levels is obtained. Such binary noise can easily be filtered to produce Gaussian signals and is very useful in its own right: it can be used to drive analog or digital switching systems used to simulate random events, machine failures, etc., and lends itself to direct correlation with other signals without the use of analog multipliers. By precision clamping, the RMS level of binary noise can be closely controlled, but the non-stationarity of the circuits used to obtain electrical noise, even 287 288 :PROCEEDINGS---SPRING JOINT COMPUTER CONFERENCE, 1964 pseudo-random output sequence is 33, 554, 431 bits which is equivalent to several thousand computer runs. generates the sum (A + B)Mod: 2 of any two binary inputs A and B according to the following table: Shift Pulses _ _ _ _-. II. BINARY PSEUDO-RANDOM NOISE GENERATION WITH SHIFT-REGISTERS 2.1 Pseudo-Random Binary Sequences Binary pseudo-random noise, as the term is used in this paper, differs in two important respects from purely random binary signals: 1. A truly random binary signal is nonperiodic, while a pseudo-random sequence repeats itself after some suitably long sequence. 2. In many random binary processes (e.g., a random telegraph wave) the transition from the "I" state to the "0" state (or conversely) can occur at any. time, and the state at any instant of time is independent of the state at any other instant of time. In the pseudo-random binary process the binary level transitions can occur only at specific clock pulse times, separated by intervals during which the binary state is fixed. In this case the state during the fixed time interval is independent of the state in neighboring time intervals. A periodic binary sequence will be classified as a pseudo-random sequence if it satisfies the following conditions: 1. In each period the number of "l's" must not exceed the number of "O's" by more than one (or conversely). 2. In each period there must be twice as many sequences of "1's" or "O's" of length n as those of length n + 1. 3. The autocorrelation function must have the form shown in Fig. 3b, Le., peaked in the middle (T == 0) and tapering off rapidly at both ends. s Such a binary sequence with a sufficiently long period-longer than a desired series of computer runs-can be used essentially like true random noise for computing purposes. Periodic binary sequences may be obtained from a digital shift-register with modulo-2 adder feedback (Fig. 1). Module-2 addition x (t) Output + Modulo - 2 adder Figure 1. + Bhlod: 2 A B 1 0 1 0 1 1 1 1 0 0 0 0 (A As can be seen from the above table, mcdulc-2addition can be implemented logically witp. an EXCLUSIVE-OR circuit. The shift-register consists of cascaded flip-flops driven at the desired rate by external clock pulses (shift pulses). The outputs of certain flip-flops are added modulo-2 and their sum is then fed back to the first stage of the shift-register. Figure 2 illustrates three shift-register periodic sequence generators and their corresponding sequences. Each column of "1's" and "O's" corresponds to the successive states of each stage of the register. It should be noted that No. 1 and No. 2 outputs are periodic every 23 - 1 == 7 bits, while No. 3 is periodic every A HYBRID ANALOG-DIGITAL PSEUDO-RANDOM NOISE GENERATOR 289 and the integers s are 1, 2, 3, 4, 5, 6. Therefore, fJ(2 3 Inltlol Condition I I I 0 I 0 I I o I I o 0 0 I 0 I 0 Initial Condition o I 0 I 0 I I I I o I I 0 I 0 0 I I o I I 0 I I I 0 o o I I 000 000 000 : : : 000 ; : : 000 000 : 1st o ~ ~} I I I 0 I o o 0 0 0 0 lat PeriOd Period o 0 0 Figure 2. 3-Stage Periodic Binary Sequence Generators. 3 bits. In each case the periodic series is completely determined by the initial state of the flip-flops and by the feedback connect~ons. The resulting sequences for an all-zero initial condition are also shown. 2.2 llfax'i'inw;n Length Seque'nces It is easy to show that the maximum length of any sequence produced by a shift-register is 2n-1, where n is the number of flip-flop stages. For an n-stage generator, there are 2n possible states. The all-zero state can be ruled out as an admissible condition, since with modulo-2 addition, each succeeding state wQuld also be all zero as illustrated in Fig. 2. Therefore, the sequence is periodic with a maximum of 2n - 1 bits. Golomb has proven that every maximum length shift-register sequence satisfies the three conditions required for a pseudo-random sequence. 3 The number of different maximum length series obtainable from an n-state shift-register n is given by 0(2 - 1) , where 0 (m) is Euler's n Phi Function, defined as the number of integers s such that 0 < s < m and s is prime to m. For a three-stage generator n == 3, m == 2n - 1 == 7, - 3 1) == 2, so a three stage shift-register can produce two different maximum length sequences, each corresponding to a unique feedback arrangement. Generators No.1 and No.2 in Fig. 2 show these two feedback arrangements. Generator No. 3 in the same figure is an example of a feedback arrangement that produces a non-maximum length sequence. N onmaximum length sequences do not in general satisfy the three conditions required for pseudorandom sequences, and thus they are not useful for statistical studies. Consequently, to design a practical pseudo-random noise generato'r of n-stages, it is necessary to determine the feedback connections which produce a sequence of 2n - 1 bits in length. Another important characteristic of maximum length sequences is the so-called "shift and add" property. That is, when any maximum length sequence is delayed by an integral number of clock periods and then added mod·, ulo-2 with the original sequence, a third identical sequence delayed with respect to the first two is formed. 6 This property has a useful application, as will be pointed out in Section HI where the actual hardware design of a binar.{ pseudo-random noise generator is considered. The "shift and add" property is also used to derive the autocorrelation function for maximum length sequences (see Appendix). 2.3 Obtaining the Maximal Period A mathematical technique for obtaining maximum length sequences is rigorously presented in Refs. 3 and 5. The method consists of viewing each state of the n-stage shift-register as an n-dimensional vector and the shiftregister /modulo-2 adder system as a linear operator, producing the successive states of the n-dimensional vector. Such an operation may be represented by an n x n matrix X. The first row of this matrix corresponds to the first stage of the register, the second row to the second stage, etc. The same is true for the n columns, 290 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 i.e., the first column represents the first stage, etc. Each element of the matrix is either a "1" or "0". A "1" in any position indicates that the flip-flop stage denoted by the column drives the flip-flop stage denoted by the particular row. Otherwise, a "0" elements exists. Considering generator No.1 in Fig. 2, the X matrix may be constructed by observing that stage 1 is fed by stages 2 and 3; therefore in the first row, a "1" is entered in columns two and three. Stage 2 is fed by stage 1, so in row two a "1" is placed in column one. Again stage 3 is driven only by stage 2, hence in row three a "1" is placed in column two. This completes the X matrix for generator No.1, viz., Similarly, for generator No. 2 the defining matrix is o o 1 It should be noted that the diagonal below the main diagonal will always consist of a series of "1's", while the first row of the matrix represents the feedback coefficients. Generalizing, the X matrix for n stages may be written: Ci · o o 1 o o 1 o o o 1 ·0 1 0 where the C's represent the feedback coefficients ("0" or "1"). The characteristic polynomial of the X matrix is thus, det [X-AI] == Ci C1-A C2 -A 1 1 0 0 -A' 1. 0 0 0 1 -A (_ A)n-l(C 1-A)-C2(-A)n-2+C a(-A)n-a ... (_A)n-n Cn == (_\)n(l_~_ C2 A A2 I\, _ _ Ca _ Aa . .. Cn) An 1 where () ==A n L C, f1. ] = 0 is == 1 defined as the characteristic equation of the shift-registel" generator. A necessary, but not sufficient, condition for the period of the shiftregister to be of maximum length is that the The bracketed term [ 1 - . 1 characteristic equation be irreducible (unfactorable). In Fig. 2 both generators No. 1 and No. 2 have irreducible characteristic equations, but that for No.3 can be factored. Using a technique known as the "sieve method," the irreducible polynomials which produce maximum length sequence have been tabulated, e.g., see Ref. 4. A HYBRID ANALOG-DIGITAL PSEUDO-RANDOM NOISE GENERATOR 291 2.4 The Autocorrelation Function 2.5 The Power Density Spectrum One of the most important properties of a maximum length sequence generator output is its time autocorrelation function. Since the sequence is periodic every 2 n - 1 bits, the autocorrelation function will also be periodic every 2n - 1 bits. Each bit represents one clock period, 6. t seconds in duration. In general, the autocorrelation function for the first period of an n-stage shift-register can be shown to be From the Wiener--Khintchine theorem, it is known that the power density spectrum Gxx(f) of a function x(t) is simply the Fourier transform of the autocorrelation function for x(t). As derived in the Appendix, the power density spectrum for a maximum length pseudo-random binary sequence x (t) is j 1- Rxx (T) == - ,T 2n I 2 n2n 1 for , T -=- 1 iL 00 a=-oo 1 bit a=,F [Si::/,£Z for , T , > 1 bit Rxx (T), for T equal to an integral number of bit intervals, is derived in the Appendix. Figure 3 0 r Il ( f _ ;f,) where Z == 2R_l = number of bits in one period of x(t) f~ = 1/ 6. t = clock frequency From the above expression, it should be noted that (1) Gxx(f) is a discrete spectrum with a xCt) -J=u harmonic separation of Ca) (2) Nulls occur at integral multiples of the clock irequency Ie. (3) The bandwidth of Gxx(f) =0.32 f(" 2"-1 Clock Periods ~~-r~====~~~~~~~~==:'T L r ~ c.p.s. 1 Clock Period. At Therefore, for a 25-stage shift-register (Z = 225 _ 1) and a 4 Mc. clock rate, Gxx(f) has a harmonic frequency separation of 0.123 c.p.s. and a bandwidth of 1.28 Mc. 1 2"-1 (b) III. DESIGN OF A PSEUDO-RANDOM NOISE GENERATOR Figure 3. 3.1 Overall Design shows the autocorrelation function of a maximum length sequence x (t). It can be seen that for time delays equal to or greater than one clock period, the correlation is essentially zero for large values of n. As opposed to the one-peak-per-period autocorrelation function of maximum length sequences, the autocorrelation function of nonmaximum length sequences has several positive and negative peaks per period. 1II This results because, as stated before, non-maximum length sequences do not satisfy the three conditions for randomness. The noise generator employs the same plug-in digital module cards (Computer Control Co. S-Pacs) standard in the new ASTRAC II computer. The block diagram of the entire noise generator system is shown in Fig. 4. This system will operate at clock rates as high as 4 Me and as low as desired. Multiplexing yields 4 uncorrelated pseudo-random noise outputs from a single 25-stage shift-register. The clamping circuitry produces ±6 volt analog outputs which are easily filtered to generate Gaussian signals; diode function generators can be used to produce other types of analog noise with 292 P,ROCEEDIN"GS--SPRING JOINT COMPUTER CONFERENCE, 1964 specified amplitude distribution and power spectra. 3.2 The Shift-Register and Modulo-2 Adders The shift-register consists of 25 general purpose flip-flops. The SET output of each flip-flop is connected to the Reset Level Control of the next stage, and the RESET output is connected to the Set Level Control (Fig. 5). The shiftregister is normally driven from the main 4 Mc ASTRAC II Clock Oscillator. The maximum length period for n == 25 is 225 - 1 == 33, 554, 431 bits. Since there are usually many feedback arrangements which will produce a maximal sequence, the one requiring the least hardware will be selected. Using the table in Ref. 4, it was found that if the outputs of stages 3 and 25 are fed back, only one modulo-2 adder is needed. Using this same table, the feedback arrangements, for n == 10 through n == 33, requiring only one modulo-2 adder to produce a maximum length were calculated and are listed in Table 4~.Io'.d HoI . . Outputs 1 of the Appen.dix. In terms of C.C.C. logic modules, a modulo-2 adder requires 4 NAND gates. The first 3 gates form an EXCLUSIVEOR, and the fourth gate provides the complement of the EXCLUSIVE-OR circuit (Fig. 5) . The output of the modulo-2 adder is fed back to the Reset Level Control of the first shift-register stage, and its complement is fed back to the Set Level Control. The output of each flip-flop and its complement are wired to a small patchbay (with a removable patchboard) provided for the noise generator. Since the binary sequence for any stage of the shift-register is the same as that of the preceding stage delayed by one clock interval, the patchbay allows access to 25 seQuences--each delayed by one clock interval from the previous one. The DC SET and DC RESET input terminals of each flip-flop are also brought out to the patchbay so that any desired initial state may be patched into the shiftregister from the computer NOISE GENERATOR RESET LINE at the start of each computation. This resetting feature is of considerable interest for rechecking a series of Monte Carlo computations with different pseudo-random noise inputs. If the NOISE GENERATOR RESET patching is omitted, the noise generator will run free and produce periodic sequences at a repetition rate not in general commensurable with the analog computer iteration rate. To provide the option of having two completely independent noise generators, the connections between shift-register stages 11 and 12 are patched rather than wired internally. The first 11 flip-flops require only one modulo-2 adder to produce a maximum length sequence, but the 14 stage shift-register requires three adders to produce a maximal period. One arrangement used to obtain a maximum length sequence from the 14-stage generator is to add the outputs of flip-flops 3, 4, 5, and 14 modulo-2 and to feed the sum back to stage 1. Figure 4. ASTRAC II Noise Generator. F.F.I F.F.2 NOISE F.F. 3 I GENERATOR RESET F.F.2! LINE Figure 5. Pseudo-Random Noise Generator Connections. As one of many alternative patching combinations, two identical II-stage maximum sequence noise generators, employing only one modul~2 adder each, may be constructed by using the same feedback connections for the 14-stage shift-register as are used for the 11- A HYBRID ANALOG-DIGITAL PSEUDO-RANDOM NOISE GENERATOR stage register (the outputs of flip-flops 2 and 11 are modulo-2 added and fed back to flip-flop 1). The sequence output of the 14-stage shiftregister is then taken from the eleventh stage, leaving 3 stages unused. 3.3 Multiplex Logic The purpose of the multiplex logic is to provide several (in this case four) uncorrelated pseudo-random noise sources from a single shift-register. There are at least two ways of accomplishing this. The first method consists of sampling the shift-register output once every fourth clock interval. This method is used in the noise generator for ASTRAC II, because it requires less digital equipment. The second method utilizes the "shift and add" property of maximum length sequences. A c B 293 o First period of the Shift-register o A B c Second period of the shift-register c o A B Third period of the shift-register B c o A Fourth period of the shift-register 3.3.1 Method I-Output Sampling The output sampling technique uses four staggered 1 Mc pulse trains to sample the 4 Mc pseudo-random noise output. Each pulse train samples the shift-register noise output every fourth clock interval, e.g., the first sampling pulse train senses the state of the noise generator output during the first, fifth, ninth, etc., clock intervals. The final result is four essentially un correlated 1 Mc pseudo-random noise outputs, each of ~ which is a maximum length sequence (33, 554, 431 bits if the 25-stage shiftregister is used) . Since the 4 Mc noise sequence is sampled only once every fourth clock interval by anyone of the 1 Mc pulse trains, it takes 4 periods of the 4 Mc sequence to produce the 33, 554, 431 bit period of anyone of the 1 Mc sequences. The 1 Mc sequences do not become periodic after the first 4 Mc period because the number of bits in the 4 Mc period is not evenly divisible by four. In fact, by designating the four 1 Mc outputs as A (t), B (t), C (t), and D (t) respectively, it can be shown (see Fig. 6) that during the second 4 Mc shift-register period A (t) takes the same sequence B (t) had during the first period of the shift-register etc., until the fifth period. Then A(t) begins to repeat the sequence it had during the first shift-register cycle. A similar process holds for the other three sequences B(t), C(t), and D(t). Actually, all four A B c o Fifth period of the shift- register Figure 6. outputs have the same binary sequence, but each of them is delayed from the previous one by approximately 8, 388, 608 bits. That is, for the outputs A(t), B(t), C(t) and D(t), A (t) A (t) A (t) A (t) == B (t + 8,388,608 bits) == C (t + 16,777,216 bits) == D (t + 25, 165,824 bits) == A (t + 33,554,431 bits) It follows that the cross-correlation function of A (t) and B (t) will peak only after 8, 388, 608 bits. For the sampling rate of 1 Mc, 1 bit is equivalent to 1 microsecond in time. Therefore, the correlation peak occurs approximately 8.4 seconds away from the time origin. The same is true for B (t) and C (t), or C (t) and D (t). This is long enough for, say, 500 computer runs using 15,000 bits per run from four essentiaUy uncorrelated noise generators. The digital circuitry required to implement the above operations is shown in the Appendix. Using the same approach, other combinations are possible. For example, the ASTRAC II 294 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 pseudo-random noise generator is designed so two un correlated 2 Mc maximum length sequences may be generated. Denoting them E (t) and F (t), then E (t) == F (t 16, 777, 216 bits). One digital bit in this case is equivalent to 1/2 of a microsecond. This added feature re",,quires little additional circuitry, as can be seen in the Appendix. + As with the shift-register, the flip-flops of the multiplex logic may be reset as desired by patching between the flip-flop DC SET or DC RESET terminal and the computer NOISE GENERATOR RESET LINE. 3.3.2 Method II-HShift and Add" Property In effect, the sampling method described above merely produces four identical sequences delayed with respect to one another. In view of the "shift and add" property of maximum length sequences, 6 similar results can be achieved by suitable modulo-2 addition of the various flip-flop outputs. By selecting the arrangement which produces four sequences delayed by approximately 8 million bits from one another, four essentially uncorrelated noise outputs are obtained. The main external difference between this method and the sampling method is that here the noise sequences are still 4 Mc, whereas before they were reduced to 1 Mc. Removeable Patchboard ON ~ OFF Figure 7. Front View of Noise Generator Panel. 3.5 Level-Shifting and Clamping Circuitry To obtain precision-limited binary sequences of ±6 volts, it is necessary to pass the digitalnoise sequences through level shifting and output clamping circuits. The voltage level shift is required since the standard outputs of the C.C.C. digital modules are -6 and 0 volts. Shown in Fig. 8 is the circuit used to accomplish the level shifting and clamping operations. Transistors Ql and ·Q2 form a Schmitt trigger. Q" provides a current source for the Schmitt trigger, and Q4 provides a clamped emitter fol. lower type output. The rise time is less than 50 nanoseconds as is the delay through the circuit. +15V +6 3.4 Digital Equipment Requirements Using commercially produced (Computer Control Co.) 5 Mcplug-in cards, the entire digital system (25-stage shift-register, 3 modulo-2 adders, and the multiplex logic) costs approximately $2,100. The following components are required: 1. Eight MF-35 flip-flop cards (4 flip-flops per card). 2. Four DI-35 NAND gate cards (8 NAND gates per card). 3. One PN-35 power amplifier card (4 power amplifiers per card). 4. One mounting rack (19 card capacity). 5. Vector patchboard and frame (300 contact&) . The design of the front panel and associated patchboard of the ASTRAC II Noise Generator is shown in Fig. 7. -I5V Figure 8. Level-Shifting and Clamping Circuitry. 3.6 Control of the Mean and Mean Square Value 11 3.6.1 Mean Value At high digital-clock frequencies (10 Kc to 4 Mc) used in typical repetitive computer applications (10 cps to 1 Kc repetition rates), the DC level E [x] of the noise generator is best A HYBRID ANALOG-DIGITAL PSEUDO-RANDOM NOISE GENERATOR controlled by a DC blocking filter with the transfer function TS H(S) == 1 TS (1) + where T == RC (Fig. 9) is an appropriately long time constant. For "slow" analog computation, symmetrical precision clipping of the binary noise output is usually sufficient to control E [x], or the active filter of Fig. 10 may be used to generate the transfer function (1). x (f ) O~---U-":""""--____- - - - O XO ( 295 during anyone computation, the peak-to-peak voltage will be monitored during computation by oscilloscope comparison with a DC reference voltage read on the computer digital voltmeter. At low clock freq.uencies, simple analog precision limiters may be used to clip the binary waveform. with an accuracy of approximately 0.1 per cent. Still, at high clock frequencies improved transistor clamping circuits, as opposed to more cumbersome AGC-type control, would be useful in regulating the mean square value. t) IV. EXPERIMENTAL TESTS R The following experiments are now being conducted with the aid of the ASTRAC I repetitive statistics computer. 4.1 Time Average and Mean Square Value Figure 9. Passive DC Blocking Filter. xCt) o_ _~_.......;..11 ~ __- -..........--_..oo xo(f) I ..... --~----O Xo ( t) The time average of the pseudo-random noise may be measured by filtering it with a low pass filter. The filter output is then sampled and converted into the appropriate number for a digital counter. The direction of the count is determined by the polarity of the sample. When the preset run counter stops, say, at m computer runs, the number indicated in the counter will be m Figure 10. Active DC Blocking Filter. L Xk==mx k==l 3.6.2 Mean Square Value Finite time averages of the squared binary noise output X2 are not subject to fast random variations, and equal one-fourth of the squared peak-to-peak output when a DC blocking filter is used. However, even with a constant load (operational-amplifier input), the.peak-to-peak output of the clamped emitter follower in Fig. 8 is subject to slow drift due to temperature variations of the break points of the clamping diodes. For example, a 10 0 F temperature variation could change the peak-to-peak output as much as 30 millivolts, or 0.25 per cent of the nominal 12 volt value. This corresponds to a 0.5 per cent change in the mean square output E [X2]. Although drift should be negligible Alternately, the time average may be found by using an analog integrator with a long time constant. By using a diode squaring circuit and the same "sample and count" method used to find the time average, the mean square value is obtained. After m runs the counter will readS m Lx==mx 2 k==1 4.2 Amplitude Distribution This measurement is made by using the Amplitude - distribution Analyzer built into ASTRAC 1. The analyzer uses a slicer-circuit 296 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 which enables its output counter to register one pulse per computer run if and only if the analyzer input voltage x(t) lies between the preset values ( X - 6;) and ( X + 6;) . The result- ing count estimates the Probability that 6X 6X] [ X - """2 < x (t 1 ) L X + ""2 . If 6X is made sufficiently small, an approximation of the density function is obtained.9 4.3 Correlation Experiments A unique method for measuring the autocorrelation function of pseudo-random noise consists of wiring two pseudo-random generators to produce identical maximum length sequences. One of the generators is then driven at a clock frequency of fl and the other at a clock frequency of fl + 6f, where 6f is very small compared to f 1 ; thus the noise generator outputs appear with a continuously varying delay with respect to one another. The generator outputs are then multiplied together and low-pass filtered to give the approximate autocorrelation function for direct oscilloscope or recorder presentation. 10 A more accurate estimation of the autocorrelation function may be obtained by using the ASTRAC I computer to generate the following functions 8 : kX(t 1 ) + kX(t l + T) and kX(t 1 ) kX (t 1 + T), k is the kth computer run and the samples at tl and tl + T are taken during each computer run. The two functions are then square to give (i[x(t 1 ) + kX(t l + T»)2 and [kX(t 1 ) - kX(t l + d]2 After m runs, the final result stored in the counters is m/2 %. L {[tX(t1) + 'x(t1 + .)], k==l _[kX(tl) - 'x(t, m/2 == L kX(tt} kX(t + k==l l T) + .)]'} For a large number of runs, , m/2 estimated Ru (T) ==! L kX(t 1) kX(t l + T) k==l 4.4 Probability Distributions of the PseudoRandom Sequence Assuming the sequence period is large, binomially distributed random numbers q are obtained by counting the noise generator "l's" occurring during Q clock periods. p (q) == ( ~) (~ )Q q == 0,1,2, ... E {q} == Q/2 Var {q} == Q/4 Note that, the random variable q is asymptotically normal as Q ~ 00 • Thus, the random variable u, where u == q-Q/2 2q-Q Q/4 == Q converges in probability to a standardized normal random variable. V. CONCLUSION In conclusion, it should be pointed out that such a hybrid noise generator has several striking features not generally shared by existing analog noise generators: "1. The analog noise output is independent of any physical quantity other than the output clamping levels. 2. The shift-register can be reset at any time to repeat a sequence of random events exactly. 3. The binary noise bandwidth is proportional to the shift register clock rate, which can be changed (and modulated) at will. This permits exact time scale changes or intermittent operation. 4. The noise generator produces digital computer random numbers as well as analog noise. Random digital words are easily stored and transmitted and also may be used to produce binomially distributed random analog coefficients with the aid of D / A-converter multipliers. A HYBRID ANALOG-DIGITAL PSEUDO-RANDOM NOISE GENERATOR 5. Digital multiplexing yields multiple uncorrelated noise signals from a single shift-register. 6. The binary output permits direct logical or analog multiplication. In addition, the group-alphabet property of shift-register sequences yields delayed sequences, with the aid of digital logic; which is useful for correlation experiments." 1 7. The flat spectrum binary noise yields Gaussian noise with any reasonable spectrum by filtering; diode function generators may then be used to produce different amplitude distributions. 8. One noise generator may also be broken into two or more completely independent generators if desired. In view of the versatility of pseudo-random noise generators and the reliability and decreasing prices of digital-logic modules, the conventional analog noise generator may well become obsolete. REFERENCES 1. 2. 3. 4. 5. 6. R., G. A. KORN, alld B. 1YIITCHELL, "Hybrid Analog-digital Random-noise Generation," I.E.E.E. Transactions on Electronic Computers, August 1963. MANELIS, J. B., "Generating Random Noise with Radioactive Sources," Electronics, September 8, 1961. GOLOMB, S. W., Sequences with Randomness PrlYperties (Internal Report) , Glenn L. Martin Co., Baltimore, Md., June 14, 1955. I{EPCKE, J. J., R. M. TAYLOR, and W. S. WYROSTEK, Geese Techniques for Pseudo Noise Generation, Defense Systems Dept., General Electric, Syracuse, New York, August 1962. BIRDSALL, T. G., and M. P. RISTENBATT, Introduction to Linear Shift-Register Generated Sequences, Technical Report Number 90, University of Michigan Research Institute, Ann Arbor, Michigan, October 1958. PETERSON, W. W., Error Correcting Codes, The MIT Press and John Wiley and Sons, Inc., New York, 1961. HAMPTON, 297 7. Instruction Manual for 5-Mc S-Pac Digital Modules, Computer Control Company, Inc., Framingham, Mass., July 1962. 8. CONANT, B. K., "A Hybrid Analog-Digital Statistics Computer," ACL Memo Number 45, The University of Arizona, Tucson, Arizona. 9. BRUBAKER, T. A., G. A. KORN, "Accurate Amplitude-Distribution Analyzer Combines Analog and Digital Logic," Rev. Sci. Instruments, March 1961. 10. STERLING, J. T., Introduction to PseudoNoise Codes and Correlators, Defense Systems Dept., General Electric, Syracuse, New York, May 1962. 11. KORN, G. A., unpublished lecture notes, University of Arizona, 1963. ACKNOWLEDGEMENT' The project described in this report is part of a hybrid analog-digital computer study directed by Professor G. A. Korn, who suggested the project and contributed a number of ideas. The author is grateful to the Office of Aerospace Research~ Information Research Division, Air Force Office of Scientific Research and to the Office of Space Sciences, National Aeronautics and Space Administration for their support of this study under joint grant AF-AFOSR-89-63 ; and to Professors L. Matsch, Dean of Engineering and Harry Stewart, Head, Electrical Engineering Department, for their encouragement and contribution of University facilities. APPENDIX Multiplex Circ1-titry To obtain the four staggered 1 Mc sampling signals, a Gray-Code logic circuit driven by the 4 Mc ASTRAC II clock is used (Fig. A). The Gray-Code effectively divides the clock input by four. The required sampling pulses exist at the outputs of the four NAND gates. They are denoted by Q), ®, ®, and 0 (Fig. B). The multiplexing is accomplished by inverting one of the sampling pulses, for example 0, and applying it to the AC input of a flip-flop. The inversion is necessary for correct triggering. The 4 Mc pseudo-random noise sequence and its 298 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 NOISE GENERATOR I RESET Derivation of the Autocorrelation Function R:r.:r. ( T) of Maximum Length Sequences lO LINE I (For T equal to an integral number of bits or clock periods) Previously, the pseudo-random noise from a shift-register has been considered to be a seSet t>utput 0 A Reset Output _~l.J1JlI1J B 4Mc. Clock Pulses -6 -t*~.1- A_:n Aj B B B i -:n L °LJ -6 n °-:1 Figure A. Gray-Code Circuit. complement are applied to the flip-flop Set and Reset Level Controls as shown in Fig. C. The flip-flop output is then the desired 1 Mc pseudorandom noise. The other three 1 Mc noise sources are similarly derived. Two staggered 2 Mc sampling pulses are easily obtained by NAND-gating pulse trains CD and ® to give the first 2 Mc sampling signal and NAND-gating ® and ® to give the second. They are then applied to the AC input of a flipflop, as was done with the 1 Mc sampling pulses, to obtain two un correlated 2 Mc maximum length sequences. A Gray-Code circuit is used instead of two cascaded flip-flops to provide the "divide by four" operation, because only one of the flipflops shown in Fig. B changes states per clock pulse. On the other hand, with two cascaded flip-flops, both go through a transition of states every second clock pulse. This would create undesirable coincidence spikes when the cascaded flip-flop outputs were NAND-gated to obtain the four staggered 1 Me sampling trains. ~-:I ---Jn. . __ L..--._ _ n '----------~~~-------- ~o -6 @ 0 -6 Figure B. Four Staggered 1 Me Sampling Pulses. NOISE RESET LINE I s.t yo-------r Out ut > IMC. PauedoRandom Nol•• y 0--------1 Reset Output y= 4Mc. Pseudo- Random Y=Complement Noise of 4 Mc. Pseudo - Random Noise Figure C. A HYBRID ANALOG-DIGITAL PSEUDO-RANDOM NOISE GENERATOR quence of "O's" and "1's", representing the two stable states of the flip-flop stages. If instead, the two stable flip-flop states are designated as "I" and "-I" respectively, it can be shown that modulo-2 addition with "0" and "I" is equivalent to arithmetic multiplication with "I" and Since 1 bit is equivalent to 1 clock interval in time, the period T == 2 n - 1 bits. Now substituting equation (1) into equation (3) and expressing the limits in terms of n, we have 2n-1 2 "-I". Rxx ( 7"1) - 2 n B A.B A B (A+B)MOD 2 A 1 0 1 -1 1 -1 0 1 1 1 -1 -1 0 0 0 1 1 1 1 1 0 -1 -1 1 f -=- 1 (2 n x (t + 7"2) dt where 7"] and 7"2 are equal to some integral number of clock periods. It follows that for maximum length sequences of "-l's" and "l's", x (t) x (t 7"1) == X (t 7"2) . (1 ) + + The time autocorrelation function for a func- 2n - 1 times Since the all "I" state (or equivalently, the all "0" state in terms of modulo-2 addition) is a forbidden condition, a maximum length sequence always has one more "-I" state than it does "I" state. Consequently, Rxx ( 7"1) == - - x(t)x(t+7"])dt.(2) 7"1 =1= 0 (2 n -1) 2n-l 2 f 2n -=-1 1 dt - (2n-l) 2 ==1 T/2 x(t) x(t + 7"1) dt. (3) -T/2 Derivation of the Power Density Spectrum Gn·(f) of Maximum Length Sequences 00 Gxx(f) == F (Rxx(T» / 2 -T/2 When x (t) is periodic, as is the case for a maximum length shift-register seq,uence, no limiting process is needed. Therefore, the autocorrelation equation becomes i f -=- 1 n T~oo R xx (T1) == 2n == fRxx(T)e-27rfT d7" -00 (6) If 7"1 == 0, the autocorrelation function may be written, 2n-l 2 R {0 \ 1 ~"xx,"",_ == 2 - 1 J{v2 ..... It ,"',\ tlt ..... T/2 if (5) Number of "l's" in] the sequence minus the number of "-l's" [ in the sequence 1 tion x (t) is by definition Rxx(7"])==lim 7"1 =1= 0 1) - 2 Thus, the "shift and add" property for "I" and "0" sequences becomes the "shift and multiply" property for "-I" and "I" sequences. Since for maximum length sequences of "1's" and "O's", it is true that x (t) + x (t + 7"1) == X (t + 7"2) , 299 PROCEEDINGS-SP~RING JOINT COMPUTER CONFERENCE, 1964 300 using a Fourier series representation for Rxx ( T ) T/2 00 = L raja Rxx( T) 2Tr fo T i f R(T)e- ,ra 00 = 2Tr foT dT, fo - i -T/2 0.=-00 Gxx(f) ja 00 f L rae -00 e - j 2TrfT dT ja 2Tr fo T 0.=-00 since the Fourier series for Rxx( T) converges uniformly, the order of integration and summation can be reversed, giving 00 00 fe \ ' ra Gxx (f) == L 0.=-00 j e- j a 2 Tr f 0 T 211"fT -1200.001 FREQUENCY IN CYCLES Figure 3. Demodulating Filter Response. +1 0 i= c a:: !d o· 1.6 PHASE -I 0 1.4 ~ ~ C -2 1&1 -3 .J > A. en c :I: A. 1.2 -4 ~ 5 bandwidth). Filter cutoff at 50 kc seems to be about optimum for a 2 mc bit rate; values above this admit excessive noise into the output. The demodulator has objectional phase shift above 1 kc (see Fig. 3 for filter response, which was improved, at the expense of some amplitude response, by lead networks at the input, Fig 6.) The overall system response is shown in Fig. 4. The uncompensated system is flat out to 40 kc. Fig. 5 shows the static input output characteristic. This curve was obtained with a digital voltmeter and is seen to be quite linear. 100,000 10,000 1000 100 10 305 AMPLITUDE -5 The output pulse widths of the CCC Delay Multivibrator Cards were narrowed to about half their normal minimum value. It was found necessary to add several additional logic cards to what could be considered a minimum component delta sigma system in order to prevent cross coupling effects from causing pulse-train jitter. Such jitter makes pulse reshaping more difficult, but has no noticeable effect on other aspects of system performance . Operation of the modular circuit, shown in Fig. 6, is straightforward. The 2 mc clock trig- 1.0 !50 500 20.000 5000 FREQUENCY IN CYCLES Figure 4. System Response of Complete b.~ System. GAIN OF DEMODULATOR AMPLIFIER WAS LESS THAN UNITY MODULATOR -----------"1 DEMODULATOR I +2~---4----~----*-----~--~----~-- -3 -2 -I INPUT o 2 VOLTAGE Figure 5. Static System Response. 3 OUTPUT .0044 ILt I 3MHY I T /LILt Figure 6. b.~ Modulator Demodulator. 306 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 gers a delay multivibrator, whose complementary output pulses are NAND gated so that one or the other is allowed to reach flip-flop FFl. The flip-flop output is added to the analog input, and the sum is integrated. The integrated output is fed into a Schmitt trigger which in turn controls the NAND gates in such a manner that feedback of the flip-flop output always tends to return the integrator output to zero. As a result, the running average of the output pulse F (t) is proportional to the analog input to the integrator. SYSTEM RESPONSE Although it is difficult to determine the modulator response or pulse code pattern for an arbitrary input, it is instructive to consider the system response to selected inputs. As Fig. 2 indicates, the flip-flop output for a zero input signal consists of a square wave with the transitions occurring at the clock intervals. Consider an input step voltage equal to one half the flip-flop voltage. The flip-flop pulse pattern which would result from this input is displayed in Fig. 7 along with various other waveforms in the system. The average value of the flipflop voltage F (t) is E./2, one half its peak value where Eo is the amplitude of the flip-flop output. The fundamental frequency of the waveform for this input is 1/4T where T is the clock period. If this waveform is averaged by a low pass filter, the steady state output of the filter would be E/2 volts or the input signal. Input S (tl CUIJVS ZJ\ / V ClocItO 0 0 0 0 X (tl 0 FIt) 0 -.-lr==tR V n 0 0 0 0 0 R .!-. Figure 7. Modulator Waveforms for Half Full Scale Input. o fc Frequency ~ Figure 8. Ideal Low-pass Filter. If the system is to be responsive to step voltages in a reasonable amount of time, it is necessary to keep the bandpass of the averaging filter as large as possible. On the other hand, too large a bandpass would permit an unacceptable amount of ripple to appear at the output. If one assumes the existence of an ideal low pass filter as shown in Fig. 8 and if the cutoff frequency fe is set below 1/4T cycles per second, no ripple components will appear in the steady state output. Unfortunately, the ripple frequency is a function of the input level and for inputs close to ± Ell volts, the fundamental frequency of the flip-flop waveform becomes arbitrarily small. In fact, for input levels equal to ± Eo the system reaches the limits of its dynamic range and the flip-flop remains permanently in one state. For an averaging filter with a bandpass fe cycles per second the static range of the input must be limited totO (feT) Eo 100 :: ~:f7'~ < 100 I...J > 200t ~.::j~ 100 '" 311 100 10 15 10 0 o hc 2.5 KC MODULAnON 5 FREQUENCY 10 15 IN 0 5 10 .5 INPVT VOLTAGE .75 1.0 RATIO Figure 14a. Spectral Line Amplitudes Versus M. MODULATION ~IlLWJ 0 25 15 300 32 KC BI T RATE 1.5 KC MODULATION 200 15 .01-1- = = = - - - t - - - - - + - - - - - - + - - - - - - t - 1 15 ~::~ > KC 200 Figure 13. Spectrum Study of Flip-flop Output. 1 :UJ 100 o::;) I- ::; 0 Q. It was noted that spectra containing exclu- sively harmonjcs of the fundamental occur only when the bit rate is an integral multiple of the modulating frequency. The converse of this is not always true, however, as can be seen in Fig. 13b. Spectrum analysis was also performed for the 2 me bit rate and 20 kc filter cutoff up to 16 kc. Under these conditions only harmonic frequencies of the fundamental were present. Typical spectra are shown in Figs. 13d, e, and f. For bit rates several orders of magnitude higher than the modulating frequency, the question of even division depends on the fourth and higher digits in the clock frequency, and these are subject to random drift. Under these conditions only harmonics of the fundamental were observed. These amplitudes were much l'ess than that of the fundamental. Spectrums for low bit-rates are shown in Figs. 13g through 1. This allows the spectrum analyzer to cover more of the spectrum and gives a better insight into system operation. Figs. 13g, h, and i are for a 16 kc bit-rate in which the entire spectrum may be studied. In Fig. 13g there is no input and the main component is at one half the clock frequency as has been previously noted in Fig. 2. There is only ~ ~~:!~ o .25 .5 INPUT VOLTAGE .75 1.0 RATIO Figure 14b. a small clock component and no measurable noise (note logarithmic millivolt scale). In Figs. 13h and i the fundamental frequency evenly divides the clock frequency and all signal harmonic frequencies are present. Note that the amplitude of some harmonics are greater than the fundamental. A bit-rate of 32 kc is used for Figs. 13j, k, and I allowing half of the spectrum to be observed. In this case nonfractional modulating frequencies are shown. Fig. 13j has many of the harmonic frequencies missing and shows no pattern of recognizable order as compared 8with the two spectrums to be described below. Fig. 13k has all signal harmonics present plus addi~tional frequencies four tenths of the way in between successive harmonics. Fig. 13-1 also has all harmonics present with the intervening space equally divided by two more spectral lines. Figs. 13k and I also show some spectrum lines with amplitudes above that of the fundamental. 312 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 CONCLUSION Providing time delay and memory in an analog computer opens up new classes of problems which need to be solved. Simulations involving transport lags, correlation studies and communications problems require the fo.rmation of time delays. Iterative techniques for solving integral eqijations require analog function storage. The use of a magnetostrictive delay line for the storage medium is attractive because of the success this technique has had in digital computers. However, information must be introduced into this line in a digital manner. Although PCM is the most efficient method of digitizing, its equipment complexity and hi~!'h cost make it unattractive to small computer installations. Delta sigma modulation, which encodes an analog signal into a digital code, is an inexpensive technique which has excellent linearity and good noise characteristics. Using this method of modulation in conjunction with a magnetostrictive delay line, 5 to 10 milliseconds delays may be obtained at 2 mc bit rates. The device will accommodate analog signals of up to 8 kc with phase shift below 2 degrees. The total dynamic error is within .2 per cent of half scale up to 1 kc. Magnetostrictive delay lines are limited at present to pulse rates of 5 mc. However, advances which will be made in high speed digital storage techniques can be utilized for analog storage since the analog signal is encoded into digital form. This system, due to its fixed spacing of narrow pulses, also offers the possibility of multiplexing. That is, one delay line with its read-write circuitry could be used to store several analog functions. This would, however, bring back the problem of the use of precise timing circuits in order to separate the pulses belonging to e·ach analog function. where S (t) is the input signal and -F (t) is the flip-flop output. Consider Equation A-I evaluated at t == nT where T denotes the clock period and n an arbitrary integer. :t E(nT) == S (nT) - F (nT) (A-2) dt E(nT) may be approximated by dt d[ (n + I)T] - dnT) T hence, E[ (n + 1)~] - dnT) == SenT) _ F(nT) (A-3) Now F( T) -{ +Eo if (nT) ::::"'0 n - -Eo if E(nT) <0 so F(nT) == E(nT) Eo I E(nT) , provided that E(nT) , dnT) I == 1 for dnT) == 0 Rewriting Equation A-3, we find E[(n+l)T] -dnT) E(nT)T == T S(nT)-Eu 'dnT), (A-5) l]=TS(nT) (A-6) A-6 is the basic equation describing the behavior of the modified delta modula'tion system. Unfortunately, the presence of the absolute value term (which has as its counterpart the comparator in the actual system) does not allow an analysis of the modulator on a linear system basis. However, Equation A-6 may be solved for specific inputs either on a digital computer,14 or by a step by step solution. APPENDIX Delta Sigma Modulator System Equations Consider the basic delta sigma modulator as shown ~n Fig. 1. The integrator output, (t) is given by d" (A-I) dt E (t) == S (t) - F (t) BIBLIOGRAPHY 1. KORN, G. A., New High-Speed Analog and Analog-Digital Computing Techniques: The ASTRAC System, Electronic Industries, July 1962. A DELTA-SIGMA MODULATION SYSTEM FOR TIME DELAY / ANALOG STORAGE 2. ECKES, H. R., and G. A. KORN, Digital Program Control for Iterative Differential Analyzers, ACL Memo No. 86, University of Arizona. 3. KORN, G. A., High-Speed Iterative AnalogDigital Computation: The ASTRAC II System, ACL Memo No. 93, University of Arizona. 4. KORN, G. A., Analog/Hybrid Storage and Pulse Modulation, IEEE Transactions;, on Electronic Computers, September 1963. 5. KENNEDY, J. D., Representation of Time Delays in Huskey, H. D., and G. A. Korn, Computer Handbook, McGraw Hfll, 1962. 6. JURY, S. H., Memory and Function Generation in Analog Computers, Military Systems Desi[ln, January 1962. 7. DUNDON, T., Magnetosti-ictive Delay Lines for Digital Applications, Computer Design, January 1963.. 8. Hybrid Compute'r Course Notes, Electronics. Associates, Inc., Princeton' Computation Center, Princeton, New Jersey. 9. DE JAGER, Delta Modulation, A Method of PCM Transmission Using the One Unit Code, Phillips Res. Repts., Vol. 7, 1952. 10. INOSE, H. Y., YASUDA and J. l\1URAKAMI, A Telemetering System by Code Modulation, IRE Transactions on Space Electronics and Telemetry, September 1962. 313 11. STORER, J. E., Pass~ve Network Synthesis, McGraw Hill, 1957. 12. HANDLER, H., Measurement of Phase Shift, IEEE TTansactions on Electronic Compute1'S, June 1963. 13. INOSE, H., and Y. YASUDA, A Unity Bit Coding Method by Negative Feedback, IEEE Special Inte1'national Issue, November 1963. 14. TRIPP, J. S., A Deterministic Study of Delta Modul~tion, Kansas State University Bulletin, Special Report Number 17. ACKNOWLEDGEMENT The project described in this report is part of a hybrid analog-digital computer study directed by Professor G. A. Korn. The writers are very grateful to the Office of Aerospace Research, Information Research Division, Air Force Office of Scientific Research and to the Office of Space Sciences, National Aeronautics and Space Administration for their continuing support of this study under joint grant AFAFOSR-89-63; and to Professors L. Matsch, Dean of Engineering and Harry Stewart, Head, Electrical Engineering Department, for their encouragement and contribution of University facilities. A COMPUTER-SIMULATED ON-LINE EXPERIMENT IN LEARNING CONTROL SYSTEMS J. D. Hill, G. J. McMurtry,.and K. S. Fu Control and Information Systems Laboratory School of Electrical Engineering Purdue University Lafayette, Indiana INTRODUCTION including hill-climbing, methods of classification for pattern recognition, reinforcement learning, problem solving machines and various other aspects of artificial intelligence. Nearly all of ,these systems utilize some form of memory and do improve their performance with time. However, they require an off-line training period involving a human operator to judge whether or not their response to a given stimulus is satisfactory, i.e., they lack a bunt-in index of performance. The structure of a learning control system combines digital memory and logic circuitry with an adaptive system. Data obtained through adaptation is stored (remembered) and later utilized to improve the system performance. The learning control system will operate satisfactorily under changing environmental conditions in which an adaptive system fails to improve the performance. The general learning system operation is outlined in this paper and a simple experimental system which illustrates the improved performance is discussed. The example sys,tem is a second order plant in which the damping ratio and undamped natural frequency are considered to be affected by the environment in a piecewise constant manner. The system was simulated on a GEDA analog computer and the memory and logic functions were supplied by an IBM 1620 through IBM 1711 and 1712 analog .to digital and digital to analog equipment. A typical example of this is given by Gabor, Wilby and Woodcock 3 who have constructed a universal nonlinear filter which will organize its internal parameters in such a way as to act as an optimum filter, prediotor, or simulator of an unknown mechanism, depending upon how it is trained. The important point is, however, that it must be trained by a human operator before it can function in the desired manner. This is typical of the general class of so-called learning or self-organizing pattern recognition machines. "Pandemoni urn," 4 "Perceptron," 5 and "Adaline"6 are other pattern recognition devices with self-organizing capabilities. All, however, require off-line training by human operators. LITURATURE SURVEY The words "learning system" and "selforganizing" appear frequently in the literature in the area of automata studies and artificial intelligence. Hawkins! has given an excellent review on self-organizing systems as well as an extensive bibliography, while Minsky2 has reviewed various aspects of artificial intelligence Thus, though some of the techniques which have been developed for pattern recognition and automata theory may be useful when applied to the learning control problem, there has as yet 315 316 PROCEEDINGS SPRING JOINT COMPUTER CONFERENCE, 1964 been very little direct application to control systems. In the hierarchy of automatic control systems, adaptive systems are more sophisticated than standard control systems. TruxaF gives a good general review of adaptive control systems while Mishkin and Brauns discuss several specific examples. The next advancement beyond adaptive systems appears to be in the area of learning control systems. Krug and Letskii 9 have suggested the use of a learning process for optimum control. In their paper they suggest that the optimum control of slow but complex processes, such as chemical processes, might be found by a systematic evaluation of input and output data and an index of performance. Probabilistic methods are suggested for organizing the memory for optimulll search procedure. The system as outlined is very general and a human operator is left to make the final decision as to whether the control found by the so-called automaton is satisfactory. Only cursory attention is given to the possibility of replacing the human operator. A considerable effort has been directed toward the investigation of learning in automatic control systems at the Control and Information Systems Laboratory, School of Electrical Engineering, Purdue University. An informal introduction to learning control systems has been given by Fu. 10 One of the specific systems investigated at Purdue Universityll is discussed in this paper. Philosophy of Learning Cont-rol Systems The structure of a learning control system, as visualized at present, is distinguished from an adaptive system mainly in that in addi,tion to a conventional adaptive system the learning system has memory and logic. Data obtained through adaptation is stored and later utilized .to improve the system performance. Thus, whereas an adaptive system may take advantage only of its immediate past experience, a learning system is able to recall and use plant adjustment data obtained through adaptation while operating under similar environmental conditions in past time. An adaptive system will optimize a slowly time varying plant to a given index of performance (IP), often through a hill-climbing technique, by modifying the controller or plant parameters. Thus, if a sufficient range of parameters is available, the adaptive system will become optimum for a given index of performance. A basic constraint is that the plant should vary slowly enough that the adaptive loop is able to track the minimum of the index of performance and maintain relatively constant performance. In some instances, the plant parameters vary so quickly that the adaptive loop cannot maintain optimum performance although some adaptive action does take place. This is one type of situation in which the learning system proposed by the authors may be applicable. Learning systems of the type discussed also yield superior performance in systems with slowly varying parameters which change suddenly but are relatively constant between changes. The learning system would be designed so that it would store in memory a quantized measure of the plant parameters which are varying or of the environment causing the variation, the best index of performance obtained by the adaptive system, and the corresponding corrections necessary to obtain this performance. It is important to realize that the hill-climbing technique' used in the adaptive portion of the system may not converge to the minimum of the IP surface if the measurements of the IP are noisy. Thus all environmental parameters affecting the IP must be measured. Only if a separate hill-climb is performed for each combination of environmental parameters and all environmental parameters are considered will the IP surface be free of noise and the adaptive portion of the system operate properly. Thus when a previously occurring set of plant parameters is again encountered, the best learned corrections would be immediately set from memory and adaptation carried out from that point. The system would, of course, be designed so that the contents of the memory are continuously updated by storing the most recent best settings of the corrections. Assuming then, that the varying parameters may be quickly identified, and that the memory interrogation is faster than the adaptive action, the system performance, after sufficient operat- 317 A COMPCTER-SIMULATED ON-LINE EXPERIMENT IN LEARNING CONTROL SYSTEMS :: Figure 1. General Plant. ing time, would approach the optimum performance. In the general learning sHuation the plant and plant parameters are shown in Fig. 1. In Fig. 1, e == (eh e2, ..., en) is the environment vector which is considered to be measurable but uncontrolled. It includes all plant inputs and environmental factors which cannot be changed in order to improve the IP. d == (d h d2 , ••• ,d p ) is the plant parameter adjustment vector and includes plant inputs which may be changed in order to improve the IP. c == plant output vector, and I is the plant index of performance. Now each component ei of the plant environment vector e, corresponds to a separate environmental or input variable such as temperature, pressure, etc. There are several methods of hill-climbing the d vector to the minimum of the IP surface, but the one chosen here is to hill-climb each component of d separately and sequentially, the best values of each of the p parameters being stored in memory corresponding to each er, along with the corresponding direction of increase or decrease in each parameter of d and the lowest value of I. The stored adjustment vector corresponding to er is labeled drM. (Superscript M indicates a stored or memorized parameter. ) The operation of the system is then as follows. Upon recognition of the occurrence of er d is set to drM from a search of memory, and hill-climbing proceeds. The computer program should be arranged to facilitate a fast search of memory for previously occurring er. This may be implemented by programming the computer to search the most frequently occurring er's first. The program must also be capable of setting up new storage locations for er's which have not previously occurred. Now let us consider that a particular er occurs, given that it has previously occurred k times. Immediately drM (k) is set from memorv and parameter dj is to be adjusted, where: j == k - a p .. These conditions are app::::::"j>O plied to indicate sequential nature of adj ustment of d/s as mentioned pre) viously. W For measurement purposes, each of these variables, ei, is assumed ,to be quantized into qi levels where the qi'S for different values of i (i == 1, .. ~, n) are not necessarily equal. Then the number N of possible different environment vectors e is n N == IT qi i==1 e will thus be relabeled e == er ; r == 1, 2, • . . , N when referring to the rth possible environment. The index of performance, I, is considered to be a scalar function of er , d and c, i.e., I == f(e r , d, c) where c == F (e r , d). Thus, for a given er , I has at most a single minimum as a function of d. I is then a p-dimensional hyper-surface in the adjustment parameters dj (j == 1, 2, . . ., p). There results, then, a p-dimensional hill-climbing problem corresponding to each different er • Then d{(k + 1) is adjusted to d{(k + 1) == d{M(k) + s{M(k) 6dj where 6 dj S{M (k) == increment of d == ± 1 (stored direction j in which to increment d{M (k» Let us now consider the logic required to program the hill-climbing operation and the modification of computer memory. Define m r(k j + 1) _ - {I0 for (Ir(k + 1)-IrM(k» for (Ir(k + 1)-IrM(k» where Ir(k + 1) == I (e r, dr(k + 1) IrM (k) == I (er, drY (k) ) L O} >0 318 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 parameters affecting the system are assumed to be measurable and measurement noise is neglected. All environmental parameters er are assumed to be constant during one hillclimbing step. In the experimental example the same assumptions were made except that er was assumed constant for four hill-climbing steps. The memorized compensation parameter then becomes d{M(k + 1) == d{ll(k) + m{(k + 1) s{M(k) 6dj The memorized compensation vector is given by d rM (k + 1) == drY (k) - dj rll (k) + d{"AI(k + 1) where d{M (k) == (Oh O2 , ••• OJ-h d{M (k), Experimental System OJ+h ••• Op) d{M (k + 1) == (Oh O2 , ••• OJ-h d{M (k OJ+ h ••. 01.) A learning system of the type discussed above with second order plant has been simulated on the IBM 1710-GEDA hybrid computer installation. The system was assumed to be subjected to a pseudo-random sequence of environmental conditions which change the damping coefficient and undamped natural frequency of the system at discrete intervals. The environment was changed sufficiently often that purely adaptive action (i.e, hill-climbing on the IP surface) could not optimize the system during the period of constant environment. + 1), The memorized value of the index of performapce becomes IrM(k + 1) == IrM(k) + m{(k + 1) (Ir(k + 1) - IrM(k» and the stored direction in which to increment d{lI (k + 1) is given by s{M(k + 1) == - sgn [(Ir(k + 1) IrM(k» (d{(k + 1) - d{ll(k»] In the above development, all environmental u I T c t I I ----J r .--------.1 ~-- AnaIoc) to I I I Divital di9tol I Compute, conve,.. 1 1 0 -_ _...._ _ _ ~--.---------L---------=--=--=-:-====--=--=--=--=Figure 2. Experimental Learning System. I I I 319 A COMPCTER-SIMULATED ON-LINE EXPERIMENT IN LEARNING CONTROL. SYSTEMS The system was subjected to a fixed amplitude square wave input in order to facilitate the computation of an index of performance of the quadratic type. In the plant, as shown in Fig. 2, el and e2 are environmental parameters which are constrained to remain constant over two periods of the square wave input. Thus, at each occurrence of an (eh e2) pair four computations of the IP are carried out, one at each transition of the square wave.. After each computation of the IP, the compensation parameters d l and d2 are adjusted by a two-dimensional hill-climbing technique described above in order to minimize the performance index. Memory is used to store the current best values of d l and d2 for a given set of (eh e2) values, as well as the directions in which dl and d2 were being adjusted, and the best value of the IP found previously. Thus when a previously occurring (et, e2) pair reoccurs, the best values of d l and d2 are set from memory and the best direction to increment d l and d2 is known. Adaptation then proceeds and the latest values of the best d 1 and d2, direction and IP replace the old values in memory. For the particular system under investigation, el and e2 were each allowed to take on five different values, Le .• ql - (12 == 5. Thus twenty~ five different (et, e2) combinations were possible. In order to simulate the limitations on computer memory capacity, only the sixteen most probable (eh e2) combinations were allowed to have corresponding d h d2, etc. data stored with them. In this system no compensation or hill-climbing took place on the nine least probable (eh e2) combinations. Probabilities were computed by counting occurrences of (et, e2) pairs and a continuous check was made for the 16 most probable. In the hill-climbing technique, the increments used for changing dl and d2 were of fixed size and were not reduced even after learning was essentially complete. The changes in el and e2 were generated by storing in memory a pseudo-random sequence of (eh e2) pairs two hundred long. The (eh e2) pairs were set from memory on the digital pots just prior to the square wave transition and the sequence of (eh e2) pairs repeated after each two hundred values (fifty different (eh e2) pairs since each pair was repeated at least four times in a row) . The performance index (IP) used was of the form T IP == I-=-(Yl £2 + Y2 X2 2 + Ya u 2) dt f o where £, X2, U are as defined in Fig. 2, and Yh Y2, Y:kmay be changed as desired. In this example, the parameter el was allowed to take on the values 4, 5, 6, 7, or 8 and e2 to have the values 20, 30,40, 50 or 60. An outline of the digital computer program and flow diagram for the system under consideration are given in Appendix A. DiscuSsion of Results An investigation of the IP surface, for the particular system and performance indices under investigation, was carried out. The system may be redrawn as shown in Fig. 3 if we let a == el + d1 and b == e2 + d2. Let the input R (s) be a step input of amplitude A. Then A R(s) £(s) == s A(s I + a) ... "" I 'h 0 2 ~ ,Q"'TIJ X2 (s) == S2 + as + b u (s) == S2 + as + b A As For an index of performance given by I == f 00 (YI £2 + Y2 X2 2 + ya u2 ) dt o c(s) Figure 3. Simplified System Block Diagram. PROCEEDINGS-SPRING JOINT COMPUTER CONFERE~CE, 1964 320 For the performance index used in this example )'1 == 0, )'2 == 1, ),a == 1 Parseval's theorem yields jco 1==_1_ 21rj f (-'II [ds) d-s)] -jco )'2 [X2 Aft~r (s) X2 (-s)] + Thus + )'3 [u (s) U _ A2 [b + 1] _ k[b + 1] I 2ab ab (-s)]} ds some manipulation, the integrand be- The constant IP contours for this performance index are shown in Fig. 4 for k == 1.0. We note that for this case, the index of performance is very insensitive to e2 + d 1 == b but very sensitive to el + d l == a. This theoretical conclusion was verified experimentally as may be seen from the trajectories plotted in Fig. 4 for several (eh e2) values. The system trajectories required approximately two and onehalf hours operation to reach the points indicated. The hill-climbing was not completed at this stage but the time was sufficient to illustrate that the system was learning. In this comes A2()'1 + )'3) [ -),I] [_ s + + + -),-2-+-a-2 s )'1 (S2 )'2 + a 2)'1] )'3 + as + b) (S2 - as )'1 + b) + )'3 The IP may be evaluated from standard tables12 to yield _ A2[b()'1 + )'3) +)'2 + a 2)'1] I 2ab Then for given )'h ),2, and ),s, contours of IP may be plotted in the (a, b) plane. I I I I I \ 1"--- 60 l.P. -0;'01 2.!5 hr. corr.ponds to 0 level on Fi9. 3.!5 t =Z.5hr 40 "--- + 1.0 hr. corresponds to C level on Fi,. 3.!5 Z5 I:l • " ' , .'f\. 8 level corr,sponds to 8 lev,1 (.. adaptive steps) on FiV. 3. !5 t·"o:·~~ '-.~""O" o• ~" 20 r- ~ ~ o o /' j/ 0.03 .' ~ . \.t:lI.OIw. .' ~/- - - O.O~ Hillel. .", Trajectory // t:o?~~ 0.10 ..!..1.v.1 .i:Q '\. ·.t=0 40 80 120 Figure 4. Index of Performance for I = k (bat 1) LQ.. 0.2!5 0.5 " 160 200 321 A COMPCTER-SIMULATED ON-LINE EXPERIMENT IN LEARNING CONTROL SYSTEMS 60 3 IP. = 1 0 ~ A A [X~ 2 + U ] dt A A 40 A A B ILl 0 LEVEL A - LEVEL B - AFTER 4 ADAPTIVE STEPS LEVEL C - AFTER I HOUR OPERATION LEVEL D - AFTER 2.5 HOURS OPERATION A INITIAL A PERFORMANCE A Z A <{ A A B :IE a:: 0 B IL. a:: B ILl a.. IL. 0 20 x ILl 0 ~ ( ~ ) (:0) (:: ) Environment probability.06 .06 -06 -06 .04 ·06 I. P. = f 3 o [x~ + u2 ] .04 .04 dt ~z 40~ r <{ :IE a:: o IL. a:: ILl a.. IL. o 20 x o ilr - ILl Z - A CD ~ ~ (:~)(560) (6~) probability .02 .02 ~ INITIAL .04 .04 .06 LEVEL A - LEVEL B - AFTER 4 ADAPTIVE STEPS LEVEL C - AFTER I HOUR OPERATION LEIiEL 0 - AFTE" 2.5 HOURS OPERATlO~ probability PERFORMANCE ~ - ~D ~ ~ ABeD ~ n I-r-- o Environment ~ _04 -04 I I I 1 I 7 (2 0) (;0) (:0) (;0) (:0) 8 (2 0) .02 .02 .02 -06 .02 .02 I (~o) .02 I I 1.- la I I (:0) (5~) .04 .02 I (:0) .06 (:~) probability Figure 5. Learning System Performance. length of time the IP decreased by a factor of approximately six. The improvement in IP for each different (eh e2) pair may also be seen from Fig. 5, which was plotted from computer type-out data as discussed in Appendix A. Note the IP scale in Fig. 5 was not normalized as are the contours in Fig. 4. It is noted that this particular performance index becomes zero as a ~ 00, b ~ 00, and does not have a particular minimum. Thus after sufficient operating time, the d 1 and d2 adjustments would simply reach the maximum values allowed for in the system design (100 and 1000 respectively). It appears that the IP is zero for b == -1 but this value 322 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 of b invalidates the derivation for the IP since Parseval's theorem is valid only for asymptotically stable systems. Fig. 5 is a plot of IP versus (eh e:!) pairs for the index of performance investigated. Four points are plotted for each (eh e2) pair corresponding to four different times in the learning process. Level A represents the IP measured before compensation. The particular sequence of (eh e2) pairs given in this test was such that the first twenty-five pairs seen by the system were all different (from left to right as plotted in Fig. 5). After all twenty-five pairs occurred once, the order of occurrence was no longer the same. Level B represents the IP for each pair after the first four adaptive steps for each (eh e2)' The first sixteen (eh e2) pairs seen by the system were compensated initially and information was stored. The last nine, in order of first appearance, had no information initially stored, and thus showed no improvement. Level C represents the IP after approximately one hour of running. Here it is seen that the system had determined the sixteen (eh e2) pairs occurring most frequently (most probable) and had continuously learned on them. The nine least probable pairs had not improved from their initial values. Level D represents the IP at the conclusion of the test for each pair. Here again, of the sixteen most probable, the ones with highest probability learned faster. Of course adaptation would normally be carried out on all (eh e2) pairs and not just on the 16 most probable. The learning system then operates as an adaptive system when a given (eh e2) pair or environment first occurs, but has the capability of utilizing past information about the best compensation parameter setting when a previously occurring environment reoccurs. In this case, given sufficient operating time, the learning system is capable of reducing the index of performance to the minimum possible value for the 16 most probable (eh e2) pairs (environmental conditions) even though the environment is changing so rapidly that an ordinary adaptive system would fail to improve the index of performance significantly. Concluding Rema'rks Several particular problems currently under investigation are the application of stochastic approximation techniques 1:i,H,lii to hill-climbing when the IP surface is disturbed by noise due to unmeasurable environmental parameters and the application of pattern recognition techniques to measurements of the IP surface in order to increase the rate of learning. Present studies are also concerned with methods of quantizing the environment parameters to yield efficient learning operation subject to the constraint of finite computer memory capacity. The principle of learning outlined in this paper, that is, partial hill-climbing of the IP surface at each occurrence of a particular environment and memorization of the best compensation found for that environment, seems to be a relatively simple idea; and yet, as illustrated by the example, it yields great improvement in the system performance over that for a simple adaptive system. It is felt that this concept of learning control is particularly applicable to process control. Acknowledgement The authors wish to thank Dr. J. E. Gibson for his stimulating discussions. This work is in part supported by the Research Contract AF AFOSR 62-351 and National Science Foundation Grant G-14609. APPENDIX Description of the Computer Program The flow diagrams shown in Figs. 6, 7, and 8 illustrate the system logic and timing. A more detailed examination follows: 1. The (eh e2) values were punched on cards and stored in memory at the beginning of the program. The program did not take advantage of this fact in determining the probability of each (eh e2) pair. 2. The type-out of results was controlled manually by the computer operator by means of a Program Switch. If the switch was on and a cycle of 200 (eh e2) values had been completed, the type-out was performed, except at the beginning of the program where type-out was performed after the first 100 (eh e2) values. A COMPCTER-SIMULATED ON-LINE E:JPERIMENT IN LEARNING CONTROL SYSTEMS Figure 6. Flow Diagram (Part I). Figure 7. Flow Diagram (Part II). 323 324 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 5. If (eb e2)i =1= (eb e2)i-h then memory was searched to see if el had ever occurred previously. Next e2 was checked for previous ocurrence. Then (e h e2) i was assigned an address . where information was stored concerning its probability and location of index of performance and compensation information. The probability was incremented next (actually the number of recorded occurrences for (eh e2) i was increased by one) . 6. At this point memory was organized so that only the sixteen most frequent (eh e2) .pairs had compensation information stored. If (eh e2) i had no information stored, then d 1 and d 2 were set to zero. CHANGE DIRECTION OF dj INCREMENT Figure 8. Flow Diagram (Part III). 3. When 105 (eh e2) values had occurred, the total number of recorded occurrences of each (eh e2) pair was divided by ten and a counter was reset to count the next 10 5 values. Then each new occurrence of a particular (eh e2) pair caused the number of recorded occurrences for that (eh e2) to be increased by one. Thus the more recent occurrences were weighted more heavily. 4. When a new pair, (eh e2), was selected, it was compared to the previous pair, (eh e2)i-l' If they were the same, the search procedure for (eh e2) and its information was bypassed, since the location of information for (eh e2L was the same as for (e h e2)i-l' . As shown in 8 below, an indicator was placed in memory for each (eh e2) pair showing which d j was operated on 1ast. Then the program would operate on the other d j when that (eh e2) pair occurred again. In order for the program to operate on the same d j when an (eh e2) pair occurred twice or more in a row, the indicator was changed when (eh e2)i was determined to be the same as (eh e2)i-l. 7. The following information was stored for the sixteen most probable (eh e2) pairs: a) whether d] or d2 was operated on most recently b) the value of d j (j == 1,.2) that resulted in the best (lowest) IP value c) the slope (direction of change) for d j (j == 1, 2) d) the sign of d j (j == 1,2) e) the best previous value of IP. 8. If information was stored for (e], e2) h the program next determined from an indicator which d j was operated on last. Assume this was d 1 • The stored value of d 1 was then set, and operations commenced on d2. The indicator was then set to show that the program acted on d2 last (see 4 above). Next, the slope was determined. a) If the slopE! was positive, the program added the standard increment, fj, d, to d 2 and the sum was compared with 9999, since this was the maximum setting on the digital potentiometer. If the sum was greater than 9999, then d 2 was set to 9999. If not d2 was set equal to the sum in question. b) If the slope was negative, and d2 was greater than fj,d, then fj,d was subtracted from d 2. If d 2 was less than fj, d, the sign of d 2 was changed and the magnitude of d2 was not changed. The sign of d j was made negative or positive, respectively, by switching an amplifier in or out of the feedback loop (see Fig. 2). If the sign was negative the amplifier was in, thus yielding a subtractive correction. A COMPUTER-SIMULATED ON-LINE EXPERIMENT IN LEARNING CONTROL SYSTEMS c) Note that the previous values of d 1 and d2 were left in memory. The new d j were stored later only if the new IP to be measured was lower than the previously stored value of the IP (see 13 below). 9. The proper values of e}, e2, d 1 and d2 were then set on the analog computer and the IP integrator was set for integration. 10. The step input was generated next, using a comparator on the GEDA. A delay followed, during which the IP was computed. At the end of the delay, the IP was measured and the IP integrator was disconnected and reset to zero. 11. If no information was stored for (et, e2)h the system had completed the cycle and the next (eh e2) pair was obtained. 12. When information was stored for (et, e2) h and if the measured IP was greater than the stored best previous IP, then the slope indicator was changed on dj , so that dj would be incremented in the opposite direction the next time the same (eh e2) pair occurred. If the slope of dj was changed two consecutive times, then the present measured value of IP was stored and d] and d2 were not changed in memory. This procedure was implemented to allow for the possibility that an erroneously low value of IP was stored due to any random noise pulses introduced into the system at some previous time. If the slope of dj had not changed twice consecutively, only the slope indicator was changed in memory and the system had completed the cycle. 13. If the measured value of IP was lower than the stored value, then the measured value, along with the present d 1 and d2 values, were placed in memory and the cycle was complete. References 1. J. K. HAWKINS, "Self-Organizing Systems -A Review and Commentary," Proc. of the IRE, Vol. 49, No.1, January, 1961. 325 4. 0.· G. SELFRIDGE, "Pandemonium: A Paradigm for Learning," Proceedings of the Symposium on Mechanization of Thought Processes, National Physics Laboratory, Teddington, England, Her Majesty's Stationery Office, London, Vol. I, pp. 513-531, 1959. 5. F. ROSENBLATT, "The Perceptron, A Theory of Statistical Separability in Cognitive Systems," Cornell Aeronautical LaDoratory, Tr. No. VG-1196-6-1, January, 1958. 6. B. WIDROW, "Pattern Recognition and Adaptive Control Symposium and Panel Discussion on Discrete and Adaptive Processes," JACC, June, 1962. 7. J. G. TRUXAL, "Adaptive Control," Proceedings of the International Federation of Automatic Control, 1963. 8. E. MISHKIN and L. BRAUN, "Adaptive Control Systems," McGraw-Hill, 1961. 9. G. K. KRUG and E. K. LETSKII, "A Learning Automaton of the Tabular Type," Automation and Remote Control, Vol. 22, No. 10, March, 1962. 10. K. S. Fu, "Learning Control Systems," COINS Symposium, June 17-18, 1963, Evanston, Illinois. 11. J. E. GIBSON, K. S. Fu. et aI., "Philosophy and State of the Art of Learning Control Systems," Report TR-EE63-7, CISL, Purdue University, November, 1963, AFOSR5144. 12. H. M. JAMES, N. B. NICHOLS, and R. S. PHILLIPS, "Theory of Servomechanisms," pp. 369-370, McGraw-Hill, 1947. 13. H. ROBBINS and S. MUNRO, "A Stochastic Approximation Method," Annals of Mathematical Statistics, Vol. 22, 1951, pp. 400407. 2. M. MINSKY, "Steps Toward Artificial Intelligence," Proc. of the IRE, Vol. 49, No.1, January, 1961. 14. J. KIEFER and J. WOLFOWITZ, "Stochastic Estimation of the Minimum of a Regression Function," Annals of Mathematical Statistics, Vol. 23, 1952, pp. 462-466. 3. D. GABOR, W. P. L. WILBY, and R. WOODCOCK, "A Universal Nonlinear Filter, Predictor and Simulator which Optimizes Itself by a Learning Process," lEE Proc., Vol. 108, Part B, 1961, p. 422. 15. H. J. KUSHNER, "Hill-Glimbing Methods for the Optimization of Multi-Parameter Noise Disturbed Systems," Trans. of ASME, Series D, Jour. of Basic Engineering, Vol. 85, No.2, June, 1963. A HEURISTIC PROGRAM TO SOLVE GEOMETRIC-ANALOGY PROBLEMS Thomas G. Evans Air Force Cambridge Research,Laboratories (OAR) Bedford, Massachusetts for College Freshmen of the American Council on Education). Furthermore, if one were required to make explicit the reasoning by which he arrived at his answer, prospects are good that the results would correspond closely to the description of its 'reasoning' produced by INTRODUCTION The purpose of this paper is to describe a program now in existence which is capable of solving a wide class of the so-called 'geometricanalogy' problems frequently encountered on intelligence tests. Each member of this class of problems consists of a set of labeled line drawings. The task to be performed can be concisely described by the question: 'figure A is to figure B as figure C is to which of the given answer figures?' For example, given the problem illus- A I . I I~I At this point, a large number of questions might reasonably be asked by the reader. Four, in particular, are: (i) vVhy were problems of this type chosen as subject matter? (ii) How does ANALOGY go about solving these problems? (iii) How competent is ANALOGY at its subject matter, especially in comparison to human performance? c B I ANALOGY. I ~ Figure 1. (iv) \Vhat has been learned in the construction of ANALOGY and what implications might this study have for the further'development of problem-solving programs in general? trated as Fig. 1, the geometric-analogy program (which we shall subsequently call ANALOGY, for brevity) selected the problem figure labeled 4 as its answer. It seems safe to say that most people would agree with ANALOGY's answer to this problem (which, incidentally, is taken from the 1942 edition of the Psychological Test The remainder of this paper constitutes an attempt to answer these questions in some detail. \Ve first deal with a variety of motivations for this investigation and attempt to place it in the context of other work in related areas. Next we turn to detailed consideration of the problem type and of the mechanism of the ANALOGY program. Finally, we present some answers to 327 328 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 the remaining two questions raised above. (A more detailed discussion of all these issues can be found in Ref. 1). M otivaUons and Background In our opinion ample general justification for the development and study of large heuristic problem-solving programs has been provided (both through argument and through example) by previous workers in this area. We shall not attempt to add to it. Given that one is interested in the construction of such programs, a number of reasons can be advanced for the choice of geometric-analogy problems as a suitable subject matter. Some of these are: (i) Problems of this type require elaborate processing of complex line. drawings: in particular, they require an analysis of each picture into parts and the determination and use of various relationships among these parts. This is an interesting problem per se and one which can reasonably be expected to be of great practical importance in the near future. (ii) The form of the problems requires one to find a transformation that takes figure A into figure B and takes figure C into exactly one of the answer figures. This situation provides a natural opportunity for trying out certain ideas about the use of explicit internal 'descriptions' (here, of both figures and transformations) in a problem-solving program. Furthermore, more speculatively, it presents an interesting paradigm of 'reasoning by analogy,' a capacity which may playa large role in far more sophisticated problem-solving programs in the future. (In Section 5 we discuss the possible relevance of ANALOGY to the introduction into problemsolving programs of more powerful learning mechanisms than have yet been achieved.) (iii) Problems of this type are widely regarded as requiring a considerable degree of intelligence for their solution and in fact are used as a touchstone of intelligence in various general intelligence tests used for college admission and other purposes. This suggests a non-trivial aspect of any attempt to mechanize their solution. We shaD now attempt very briefly to place ANALOGY in the context of earlier work in re- lated areas. Two aspects of ANALOGY must be considered: (i) ANALOGY contains a substantial amount of machinery for the processing of representations of line drawings, including decomposition into subfigures, calculation of relations between figures, and 'pattern-matching' computations. Thus we must relate it to other work in picture processing and pattern recognition. (ii) ANALOGY is a complex heuristic problem-solving program, containing an elaborate mechanism for finding and 'generalizing' transformation rules. Thus we must relate it to other work on the development of problem-solving programs. We turn first to the picture-processing aspect. The essential feature of the treatment of line drawings by ANALOGY is the construction, from relatively primitive input descriptions, of more 'abstract' descriptions of the problem figures in a form suitable for input to the rulefinding program. The fundamental programming technique underlying this method is the use of a list-processing language, in this case LISP,2,3 to represent and process the figures in question. Work in picture processing, for pattern-recognition purposes, involving some elements of description, is found in Grimsdale et al.;t Marill et al.,ri and Sherman/i among others. Sutherland 7 and Roberts R have used, for quite different purposes, internal representations of line drawings similar in some respects to those used in ANALOGY. Kirsch 9 has worked with complex line drawings primarily as a vehicle for programs involving the analysis of Englishlanguage sentences pertaining to such pictures. Hodes 10 and Canaday 11 have used LISP expressions for figure description in much the same way that we have, though the development of machinery for manipulating such descriptions was, of necessity, carried much further in ANALOGY. Evidently the first advocacy of 'scene description' ideas (for use in pattern recognition) occurs in Minsky.l:! To place ANALOGY with respect to other work with problem-solving programs, we shall simply list a number of developments in the construction of problem-solving programs which have influenced, in ageneral way, our approach to the design of ANALOGY. These include LT A HEURISTIC PROGRAM TO SOLVE GEOMETRIC-ANALOGY PROBLEMS 329 (the Logic Theorist) Ii! and, more recently, GPS (the General Problem Solver) 14 of Newell, Simon, and Shaw, the plane-geometry theoremprover Jr; of Gelernter and Rochester, and SAINT, the formal integration program of Slagle. w Summary of the Solution Process, with Example To exhibit as clearly as possible the entire process carried out by ANALOGY, we now sketch this process, then examine its operation on an example. The sample problem we shall be considering is shown as Fig. 2 (where the Pi's are not part of the problem figures but labels keying the corresponding parts of the figures to cases, including ,the example to be discussed below, was quite simple. It merely separated a problem figure into its connected subfigures; e.g., figure A of the above example consists of the three objects labeled PI, P2, and P3. It later became desirable to have a more sophisti- B A I Figure 3a. 2 3 4 5 ~~~!3Jrl Figure 3b. Figure 2. expressions we shall give below). Before treating the example, we shall summarize the entire solution process. Given a problem such as that above, ANALOGY proceeds as follows: First, the input descriptions of the figures are read. Currently these descriptions, given as LISP expressions in a format to be illustrated below, are hand-made; however, they could well be mechanically generated from scanner or lightpen input by a relatively straightforward, quite 'unintelligent' program embodying line-tracing techniques already described in the literature. The descriptions represent the figures in terms of straight line segments and arcs of circles Xto any desired accuracy, at the cost of longer and longer expressions). Examples of the descriptions are given below. The first step taken by ANALOGY is to decompose each problem figure into 'objects' (subfigures). The decomposition program originally written, which was sufficient to handle many cated decomposition program with, in particular, the capability of separating overlapped objects on appropriate cues. For example, suppose problem figure A is as in Fig. 3a and figure B is as in Fig. 3b. The decomposition program should be able to separate the single object of figure A into the triangle and rectangle on the basis that they appear in figure B, from which point the remaining mechanism of parts I and II could proceed with the problem. While a decomposition program of the full generality desirable has not yet been constructed, the most recent version of the program is capable, in particular, of finding all occurrences of an arbitrary simple closed figure x in an arbitrary connected figure y; for each such occurrence the program can, if required, separate y into two objects: that occurrence of x and the rest of y (described in the standard figure format-note that this 'editing' can be rather complex: connected figures can be split into non-connected parts, etc.). 330 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 The type of decomposition illustrated above might be called 'environmental,' in that, e.g., figure A is separated into subfigures on the information that these subfigures are present, already separated, in figure B. An interesting extension to the present part I of ANALOGY might be to incorporate some form of 'intrinsic' Figure 4a. decomposition in which 'most plausible' decompositions are generated according to Gestaltlike criteria of 'good figure.' Such an extension could widen the problem-solving scope of AN ALOGY considerably to include many cases where the appropriate subfigures do not appear already 'decomposed' among the problem figures. For example, suppose problem figures A and B are as shown in Figs. 4a and 4b, respec- o Figure 4b. tively. A decomposition into the square, triangle, and circle seems necessary to state a reasonable transformation rule. This example, incidentally, illustrates one potentially useful 'intrinsic' decomposition heuristic: roughly, choose decompositions into subfigures which have as much internal symmetry (in some precise sense) as possible. Next, the 'objects' generated from the decomposition process are given to a routine which calculates a specified set of properties of these objects and relations among them. The program is designed so that this set can be changed easily. As a sample of a relation-calculating subroutine, we cite one that calculates, for figure A of our example, that the object labeled P2 lies inside that labeled P3 and generates a corresponding expression (INSIDE P2 P3) to be added to the part I output description of figure A. The method used involves calculating all intersections with P3 of a line segment drawn from a point on P2 to the edge of the field (all figures are considered as drawn on a unit square). In this case P2 lies inside P3 since the number of such intersections is odd, namely one (and P3 is known to be a simple closed curveif it were not, the calculation just described would be performed for each closed curve contained in P3). To do this, a substantial repertoire of 'analytic geometry' routines is required for part I, to determine, for example, intersections of straight line segments and arcs of circles in all cases and combinations. Other relation routines available in part I calculate, for example, that in figure A of our example PI is above P2 and P3 and in figure B that P4 is to the left of P5. The principal business of part I, aside from decomposition and the property and relation calculations, is a set of 'similarity' calculations. Here, part I determines, for each appropriate pair of objects, all members from a certain class T of transformations which carry one object of the pair into the other. The elements of Tare compositions of Euclidean similarity transformations (rotation and uniform scale change) with horizontal and vertical reflections. Given descriptions of virtually any pair of arbitrary line-drawings x and y, the routines of part I will calculate the parameters of all instances of transformations from T that 'map' x into y. A HEURISTIC PROGRAM TO SOLVE GEOMETRIC-ANALOGY PROBLEMS More precisely, an acceptable 'map' is a member of T for which T (x) is congruent to y up to certain metric tolerances which are parameters in the corresponding programs. This routine is, in effect, a pattern-recognition program with built-in invariance under scale changes, rotations, and certain types of reflections. It consists essentially of a topological matching process, with metric comparisons being made between pairs of lines selected by the topological process. In Ref. 6 Sherman introduced some topological classification into a sequential decision tree program for the recognition of hand-printed letters, but the notion of systematically usi~g the topological information to determine which metric comparisons ~re to be made seems to be new. This type of organization for pattern recognition has its own advantages (e.g., flexibility-the metric parts can be changed easily with no effect on the overall structure) and difficulties (e.g., sensitivity to metrically small changes in a figure which affect the connectivity-but this sensitivity can be largely removed by suitable pre-processing). Incidentally, it may be worth noting that if we suppress the metric comparisons entirely we get a general, and reasonably efficient, topological equivalence algorithm for graphs (networks) . The set of techniques we have just been describing, based on the use of a list-processing language to perform processing of line drawings by manipulating their list-structured descriptions, is by no means limited in applicability to the uses to which we have put it in part I of ANALOGY. To the contrary, it is our view that the representation of line drawings used here and the corresponding processing routines form a suitable basis for the development of a quite powerful 'line-drawing-manipulation language' with potential usefulness in a wide variety of applications. Regardless of whether the present investigation turns out to have a measurable influence on the art of designing problem-solving programs, it seems probable that the principal short-range contribution of ANALOGY is in the picture-processing by-products just described. (Incidentally, these techniques were discussed briefly from an ANALOGY-independent point of view in Ref. 17.) 331 After the similarity information is computed for every required pair of objects, both within a problem figure and between figures, this information, together with the decomposition and property and relation information, is punched out on cards in a standard format for input to part II. (For a typical set of figures, the total output of part I, punched at up to 72 columns/ card, might come to 15 to 20 cards.) Part II is given these cards as input. Its final output is either the number of the solution figure or a statement that it failed to find an answer. Its first step is to generate a rule (or, more frequently, a number of alternate rules) transforming figure A into figure B. Such a rule specifies how the objects of figure A are removed, added to, or altered in their properties and their relations to other objects to generate figure B. Once this set of rule possibilities has been generated, the next task is to 'generalize' each rule just enough so that the resulting rules still take figure A into figure B and now take figure C into exactly one of the answer figures. More precisely, for each 'figure A ~ figure B' rule and for each answer figure, part II attempts to construct a 'common generalization' rule which both takes figure A into figure Band figure C into the answer figure in question. This process may produce a number of rules, some very weak in that virtually all the distinguishing detail has been 'washed out' by 'generalization.' Hence it is necessary at this point to pick the 'strongest' rule by some means. This entire process requires a complex mechanism for manipulating and testing the rules and deciding which of the several rule candidates, the results of different initial rules or of different 'generalizations,' is to be chosen. The principal method embodied in part II at present is able to deal quite generally with problems in which the numbers of parts added, removed, and matched in taking figure A into figure B are the same as the numbers of parts added, removed, and matched, respectively, in taking figure C into the answer figure. A substantial majority of the questions on the tests we have used are of this type, as is our present example; virtually all would be under a sufficiently elaborate decomposition process in part I; this restriction still permits a wide variety o-f transformation rules. It should be mentioned 332 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 that the methods of part II have been kept subject.;.matter free; no use is made of any geometric meaning of the properties and relations appearing in the input to part II. The more detailed workings of both parts I and II are best introduced through examining the process sketched above at work on our example. To convey some further feeling for the nature of the input to part I, we exhibit part of it, namely, the input description of figure A. The LISP expressions look like: (DOT (0.4 . O.S» (See «0.3 . 0.2) 0.0 (0.7 . 0.2) 0.0 (0.5 . 0.7) 0.0 (0.3.0.2») (see «0.4 . 0.3) 0.0 (0.6 . 0.3) 0.0 (0.6 . 0.4) 0.0 (0.4 . 0.4) 0.0 (0.4 . 0.3») ) The first line above corresponds to the dot (at coordinates x == 0.4 and y == O.S on the unit square; the coordinate pairs in the other expressions are interpreted analogously). The next two lines correspond to the triangle (See stands for simple closed curve. All connected figures are divided into three classes-dots (DOT), simple closed curves (SeC), and all the rest (REG). This is solely for reasons of programming convenience; no other use is made of this three-way classification). Each nonconnected figure is represented simply by a list of descriptions of its connected parts. A curve (which may consist of an arbitrary sequence of elements chosen from straight line segments and arcs of circles) is represented by a list in which coordinate pairs alternate with the curvatures of the line elements between (all curvatures are zero here since the lines in question are all straight). Similarly, the next two lines above correspond to the rectangle; the entire description of figure A is then a list of the descriptions of these three parts. The format corresponding to the non-SeC figures like the Z-shaped subfigure of figure e is similar though somewhat more complex; it looks like: (REG «VI V2 (0.0 (0.55 . 0.5) 0.0 (0~45 . 0.3) 0.0» (V2 VI (0.0 (0.45 . 0.3) 0.0 (0.55 . 0.5) 0.0) ) ) ) where VI and V2 are the two vertices (here, endpoints) of the figure. The coordinates of VI and V2 are given to part I in a separate list. They are VI == (0.45 . 0.5), V2 == (0.55 . 0.3). Here, the top-level list describes the connectivity by stating which vertices are connected to which and how often-sublists describe in detail the curves making these connections. (By vertex we mean either an endpoint of a curve or a point at which three or more curves come together.) The complete details of the input format are given in Ref. 1, along with many examples. When the input shown above corresponding to problem figure A and the corresponding inputs for the other seven figures are processed, the output from part I is, in its entirety, the ten LISP expressions shown below. For brevity, all similarity information concerning non-null reflections has been deleted. Also, we have replaced the actual arbitrary symbols generated internally by ANALOGY as names for the parts found by the decomposition program by the names PI, P2, etc., which appear as labels on our example figures above. The ten output expressions are: (1) «PI P2 P3) . «INSIDE P2 P3) (ABOVE PI P3) (ABOVE PI P2) ) ) (2) «P4 P5) . «LEFT P4 P5») (3) «P6 P7 PS) . «(INSIDE P7 P6) (ABOVE P8 P6) (ABOVE PS P7») (4) «P2 P4 «(1.0 . 0.0) . (N.N» «1.0. 3.14) . (N.N»» (P3 P5 « (1.0 . 0.0). (N.N»») (5) «PI PS «(1.0 .0.0) . (N.N»») (6) NIL (7) ( (P9 PIO PII) (PI2 PI3) (PI4 PI5) (PI6 PI7) (PIS) ) (S) ( «INSIDE PIO P9) ABOVE PII P9) (ABOVE PII PIO» «LEFT PI2 PI3» «(INSIDE PI5 PI4» «ABOVE PI7 PI6» NIL) (9) ( «P6 P9 « (1.0 . 0.0) . (N.N»» (P7 PIO « (1.0 . 0.0) . (N.N» «1.0 . -3.14) . (N.N»» (PS PII « (1.0 . 0.0) . (N.N»») A HEURISTIC PROGRAM TO SOLVE GEOMETRIC-ANALOGY PROBLEMS «P6 PI3 « (1.0 . 0.0) . (N.N»») (P7 PI2 « (1.0 . 0.0) . (N.N» «1.0 .-3.14) . (N.N»») «P6 PI4 « (1.0 . 0.0) . (N.N»» (P7 PI5 « (1.0 . 0.0) . (N.N» «1.0 . -3.14) . (N.N»») «P6 PI6 « (1.0. 0.0) . (N.N»» (P8 PI7 « (1.0 . 0.0) . (N.N»») «P7 PI8 « (1.0.0.0) . (N.N» -3.14) . (N.N.»») ) «1.0. (10) ( ( «PI P1I «(1.0.0.0) . (N.N»») NIL NIL «PI PI7 «(1.0 . 0.0) . (N.N»») NIL) . (NIL NIL NIL NIL NIL) ) To explain some of this: The first expression corresponds to figure A. It says figure A has been decomposed into three parts, which have been given the names PI, P2, and P3. Then we have a list of properties and relations and similarity information internal to figure A, namely, here, that P2 is inside P3, PI is above P2, and PI is above P3. The next two expressions give the corresponding information for figures B about Euclidean similarities between figure A and figure B. For example, P3 goes into P5 under a 'scale factor = 1, rotation angle = 0, and both reflections null' transformation. The next two expressions contain the corresponding information between figure A and figure C and between figure B and figure C, respectively. The seventh list is a five-element list of lists of the parts of the five answer figures; the eighth a five-element list of lists, one for each answer figure, giving their property, relation, and similarity information. The ninth is again a fiveelement list, each a 'similarity' list from figure C to one of the answer figures. The tenth, and last, expression is a dotted pair of expressions, the first again a five-element list, a 'similarity' list from figure A to each of the answer figures, the second the same from figure B to each of the answer figures. This brief description leaves certain loose ends, but it should provide a reasonably adeq,uate notion of what is done by part I in processing our sample problem. The ten expressions above are given as arguments to the top-level function of part II 333 (optimistically called solve). The basic method employed by solve, which suffices to do this problem, begins by matching the parts of figure A and those of figure B in all possible ways compatible with the similarity information. From this process, it concludes, in the case in question, that P2 ~ P4, P3 ~ P5, and PI is removed in going from A to B. (The machinery provided can also handle far more complicated cases, in which alternate matchings are possible and parts are both added and removed.) On the basis of this matching, a statement of a rule taking figure A into figure B is generated. It looks like: ( (REMOVE Al «ABOVE Al A3) (ABOVE Al A2) (SIM OB3 Al « (1.0 . 0.0) . (N.N») ») (MATCH A2 « (INSIDE A2 A3) (ABOVE Al A2) (SIM OB2 A2 « (1.0 . 0.0) . (N.N»») . «LEFT A2 A3) (SIM OB2 A2 « (1.0 . 0.0) . (N.N» «1.0 . 3.14) . (N.N»» (SIMTRAN « (1.0 . 0.0) . (N.N» «1.0 . 3.14) . (N.N) I»~»~) (MATCH A3 « (INSIDE A2 A3) (ABOVE Al A3) (SIM: OBI A3 « (1.0 . 0.0) . (N.N»») . «LEFT A2 A3) (SIM OBI A3 « (1.0 . 0.0) . (N.N»» (SIMTRAN « (1.0 . 0.0) . (N.N) I»~»~) ) The A's are used as 'variables' representing objects. The format is rather simple. For each object added, removed, or matched, there is a list of the properties, relations and similarity information pertaining to it. (In the case of a matched object, there are two such lists, one pertaining to its occurrence in figure A and the other to its occurrence in figure B.) There are two special devices; the (SIM OBI ... ) - form expressions give a means of comparing types of objects between, say, figure A and figure C; the other device is the lIse of the SIMTRAN expressions in the figure-B list for each matched object. This enables us to handle conveniently some additional situations which we shall omit from consideration, for brevity. They are treated in detail in Ref. 1. 334 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 The above rule expresses everything about figures A and B and their relationship that is used in the rest of the process. (The reader may verify that the rule does, in some sense, describe the transformation of figure A into figure B of our example.) Next, a similarity matching is carried out between figure C and each of the five answer figures. Matchings which do not correspond to the ones between figure A and figure B in numbers of parts added, removed, and matched, are discarded. If all are rej ected this method has failed and solve goes on to try a further method. In the present case, figures I and 5 are rejected on this basis. However, figures 2, 3, and 4 pass this test and are examined further, as follows. Choose an answer figure. For a given matching of figure C to the answer figure in question (and solve goes through all possible matchings compatible with similarity) we take each 'figure A -7 figure B' rule and attempt to fit it to the new case, making all matchings between the A's of the rule statement and the objects of figure C and the answer figures which are compatible with preserving add, remove, and match categories, then testing to see which information is preserved, thus getting a new, 'generalized' rule which fits both 'figure A -7 figure B' and 'figure C -7 the answer figure in question.' In our case, for each of the three possible answer figures we get two reduced rules in this way (since there are two possible pairings between A and C, namely, PI ~ P8, P2~P6, and P3 ~ P7, or PI ~ P8, P2 ~ P7, and P3 ~ P6). In some sense, each of these rules provides an answer. However, as pointed out earlier, we want a 'best' or 'strongest' rule, that is, the one that says the most or is the least alteration in the original 'figure A -7 figure B' rule and that still maps C onto exactly one answer figure. A simple device seems to approximate human opinion on this question rather well; we define a rather crude 'strength' function on the rules and sort them by this. If a rule is a clear winner in this test, the corresponding answer figure is chosen; if the test results in a tie, the entire method has failed and solve goes on to try something else. In our case, when the values for the six rules are computed, the winner is one of the rules corresponding to figure 2, so the program, like all humans so far consulted, chooses it as the answer. The rule chosen looks like this: ( (REMOVE Al «ABOVE Al A3) (ABOVE Al A2) (SIM OB3 Al « (1.0 . 0.0) . (N.N) ) ) ) » (MATCH A2 « (INSIDE A2 A3) (ABOVE Al A2» . «LEFT A2 A3) (SIMTRAN ( «1.0 . 0.0) . (N.N» «1.0. 3.14) . (N.N) ) ) ) ) ) ) (MATCH A3 « (INSIDE A2 A3) (ABOVE Al A3» . «LEFT A2 A3) (SIMTRAN «(1.0.0.0) . (N.N»»») Again, it is easy to check that this rule both takes figure A into figure B and figure C into figure 2, but not ~nto any of the other answer figures. Further Examples and Comments (a) Examples We first exhibit several additional examples of problems given to ANALOGY: (i) (See Fig. 5) A 00 Figure 5. Here the rule involves changes in the relations of the three parts. ANALOGY chose answer figure 3. (ii) (See Fig. 6) This case involves both addition and removal of objects. ANALOGY chose answer figure 2. (iii) (See Fig. 7) 335 A HEURISTIC PROGRAM TO SOLVE GEOMETRIC-ANALOGY PROBLEMS c B A ~ ~ I 2 3 4 c B OJ U [±J 5 ~~§]~~ I 2 3 4 []0[±JG~ Figure 6. Figure 8. c A A c B [§] I 2 :3 5 4 B 2 5 3 5 4 [QJ@]~GE9 Figure 7. Figure 9. Note that this case required the more powerful decomposition program. Here ANALOGY chose figure 3. (iv) (See Fig. 8) The rule here simply involved a rotation. AN ALOG Y chose figure 2. fore, the outer object is removed and the inner one enlarged. This illustrates some of the flexibility of the procedure and the dependence of the answer choice on the range of allowed answers as well as on A, B, and C. (v) Here is an example of a failure by ANAL0Gy to agree with the human consensus which favors figure 5. ANALOGY chose figure 3. (See Fig. 9) Here ANALOGY chose figure 3, using an extension of the part II techniques discussed above. This extension, employed after failure of the basic process, involves systematic substitution of certain specified relations (e.g., LEFT for ABOVE) for others in the part II input descriptions, thus making it possible for ANALOGY to relate the 'vertical' transformation taking A into B to the 'horizontal' transformation of C into 3. (vi) In the problem of Fig. 1, the large circle of answer figure 4 was replaced by a large square and the problem rerun. Again figure 4 was chosen but by a different rule. Now, instead of the inner object being removed, as be- (vii) (See Fig. 10) A B Figure 10. c 336 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 C01nparison with Human Performance We can only roughly compare the performance of ANALOGY with that of humans on geometric-analogy problems, since ANALOGY has not yet been given the complete set of such problems from any test for which scores are available. However, as some indication, we cite scores on the ACE tests based on a period of years including those editions of the test from which most of the problems on which AN ALOG Y was tested were selected. These scores are for a large population of college-preparatory students; the median score, on a test consisting of- 30 such questions, ranged from 17 for 9th grade to 20 for 12th grade. pur estimate is that, on the same tests, ANALOGY, as it currently exists, could solve between 15 and 20 problems. Given, in addition, certain changes (mostly in part I, e.g., a more powerful decomposition program and additional properties and relations) for which we have reasonably wellworked-out implementations in mind, AN ALOGY should be capable of perhaps 25 successful solutions. (b) (c) The Use of LISP The use of a list-processing language to construct the ANALOGY program appears to have been a suitable choice; most notably, its capability at handling intermediate expressions of unpredictable size and 'shape' (such as our figure descriptions and transformation rules) is of great value. We especially wish to praise LISP as a convenient and elegant language in which to write and debug complex programs. The ease of composition of routines, the highly mnemonic nature of the language, and the good tracing facilities all contribute greatly to effective program construction. In return for the use of such a language one pays a certain price in speed and storage space, which, in the case of ANALOGY, at least, was a very acceptable bargain, since the necessity of machine-language coding would have made the entire project unfeasible. Incidentally, the ANALOGY program (apparently the largest program written in LISP to date) is so large that parts I and II must occupy core separately. The consequent limited (and one-way) communication between the parts was a serious design constraint but proved to have some compensating advantages in simplicity. ANALOG Y and Pattern-Recognition in Problem-Solving Programs In this section we shall consider certain aspects of the design of problem-solving machines. To aid this discussion we shall specify (rather loosely) a subclass of problem-solving machines and carry out our discussion in terms of these though the ideas involved are by no means limited in applicability to this class. The machines we have in mind are typified by GPSl-l in that the problem to be solved by the machine is to transform one specified 'object' or 'situation' (whatever this may mean in a particular subject-matter context) into another by applying an appropriate sequence of transformations chosen from a class available to the machine. A wide variety of problems may be cast in this form (again see Ref. 14 or other discussions of GPS by the same authors). As in GPS, subgoals may be generated and attacked by such a machine and elaborate schemes of resource allocation may be required. However, these aspects do not concern us here. Our interest lies in the basic task of the machine; given a pair of 'objects,' it must choose an 'appropriate' transformation, i.e., one contributing to the goal of transforming one of the given 'objects' into the other. It is a widely-held view, with which we agree completely, that for a machine to be capable of highly intelligent behavior on a task of this kind, in a rich environment of objects and transformations (and, in particular, to be capable of learning at a level more advanced than that of present machines), the critical factor is that it have a good internal representation of both its subject matter ('objects') and its methods ('transformations'), as well as an elaborate set of 'pattern-recognition' techniques for matching transformations to object pairs. Probably this means a quite 'verbal' representation of both objects and transformations as expressions in suitable 'description languages.' Furthermore, these matching techniques must be represented in a form in which they themselves are capable of being improved as the machine gains experience. The central role which 'pattern-recognition' techniques must play in sophisticated problem-solving programs and the corresponding importance for effective learning of autonomous improvement in the perform- A HEURISTIC PROGRAM TO SOLVE GEOMETRIC-ANALOGY PROBLEMS ance of these techniques are well expressed in Minsky.I 2 There we find: In order not to try all possibilities a resourceful program must classify problem situations into categories associated with the domains of effectiveness of the machine's different methods. T~ese pattern-recognition methods must extract the heuristically significant features of the objects in question. Again from Ref. 12 we have: Again from 12 we have: In order to solve a new problem one uses what might be called the basic learning heu,ristic-first try using methods similar to those which have worked, in the past, on similar problems. Here, the problem is, of course, to have pattern-recognition techniques possessing, or able themselves to learn, criteria of 'similarity' appropriate to the subject matter in question. The 'fixed-length property-list' schemes (see Ref. 12) which characteristically have been used to perform this pattern-recognition task in current problem-solving programs have two principal defects which limit their extension to harder problems: (i) ""hile, in principie, given enough sufficiently elaborate properties, one can make arbitrarily fine discriminations, in practice a given set of properties will begin to fail rapidly as situations become more complex. In particular, for 'situations' which must be treated as consisting of interrelated parts, the 'global' nature of the scheme in question leaves it helpless. (ii) Such a scheme is very limited in its learning capabilities, since it has access to very little information about its component properties; in particular, it is incapable of "knowledgeably' modifying its tests or adding new ones-it can only modify the weightings given to the results of these tests in its 'decisions.' In view of the limitations of the 'propertylist' pattern-recognition scheme just mentioned, we can formulate some requirements for a pattern-recognition scheme suitable tc replace it as a 'transformation-selecting' mechanism. First, the scheme must have access to a representation of each 'object' in terms of a 'descriptive framework' for the subject matter in 337 question which is suitable in that useful relationships between 'objects' can be extracted relatively simply-from the corresponding representations. Furthermore, the transformationselecting rules of the pattern-recognition apparatus should themselves be expressed in a representation suitable for a 'learning mechanism' to revise the set of rules (i) by adding new rules and deleting those old ones which prove least useful as experience associates certain object pairs with certain transformations and (ii) by replacing a set of particular rules by a 'common generalization' rule again represented in the same language. Such facilities could go far toward removing the limitations of which we have spoken and providing both a powerful rule language (the rules can be stated in terms of the 'descriptive framework' we have postulated for the 'objects') and a learning mode more sophisticated than any yet incorporated in such a general problem-solving program. So far we have been enumerating desirable features in a 'pattern-recognition' mechanism to be used as a transformation-selection device within a large problem-solver. What has all this to do with ANALOGY, which is not even a problem-solving program of the class we. have been considering? We suggest that ANALOGY can, under a suitable (rather drastic) reinterpretation, be to some extent viewed as a patternrecognition program having, to the limited degree appropriate for its particular environment, all the features we have listed. First, the 'objects' are the problem figures of ANALOGY and the suitable 'descriptive framework' appropriate to these objects is the 'subfigure and relation' representation used as the input part I generates for part II of ANALOGY. (Thus part I of ANALOGY corresponds to the apparatus that generates this representation for each object; that is, it goes from a representation of the 'problem objects' which is convenient for input to the problem-solver to one which is in a form suitable for internal use.) The generation·jn ANALOGY of a transformation rule taking one answer figure into another can be thought of as corresponding to the first kind of learning we listed above, namely, the adding of rules as, with experience, the machine associates certain object pairs with certain simple or com- 338 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 posite transformations. Finally, the common generalization of two rules in ANALOGY corresponds to the second kind of learning we mentioned, namely, the generation of a common generalization of several rules associating 'objects' and 'transformations.' Furthermore, ANALOGY's process of choosing between 'common generalizations' of different rule pairs mirrors a process of selectively incorporating only those generalizations with the greatest discriminatory power. Under this interpretation, ANALOGY appears as a model for a patternrecognition process with all the characteristics mentioned. The potential value of ANALOGY, viewed in this way, as a suggestive model for the construction of such pattern-recognition mechanisms for use within problem-solving programs may prove to be the chief product of our work with ANALOGY and the best justification for having carried it out. References 1. T. G. EVANS, PH.D. Thesis, Department of Mathematics, MIT, June, 1963 (soon to be available as an AFCRL Technical Report). 2. J. MCCARTHY, "Recursive functions of symbolic expressions," Comm. ACM, Vol. 3, April, 1960. 3. J. MCCARTHY et al., LISP 1.5 Programmer's Manual, MIT, revised edition, August, 1962. 4. R. L. GRIMSDALE, F. H. SUMNER, C. J. TUNIS, and T. KILBURN et al., "A system for the automatic recognition of patterns," Proc. lEE, March, 1959, Vol. 106, pt. B, pp. 210-22l. 5. T. MARILL, A. K. HARTLEY, T. G. EVANS, B. H. BLOOM, D. M. R. PARK, T. P. HART, and D. L. DARLEY, "CYCLOPS-I: a secondgeneration recognition system," F JCC, Las Vegas, Nevada, November, 1963. 8. L. ROBERTS, PH.D. Thesis, Department of Electrical Engineering, MIT, June, 1963. 9. R. KIRSCH, personal communication. 10. L. HODES, "Machine processing of line drawings," Lincoln Laboratory Technical Memorandum, March, 1961. 11. R. CANADAY, M.S. Thesis, Department of Electrical Engineering, MIT, February, 1962. 12. M. L. MINSKY, "Steps toward artificial intelligence," Proc. IRE, January, 1961, pp. 8-30. 13. A. NEWELL and H. A. SIMON, "The logic theory machine," IRE Trans. on Information Theory, Vol. IT-2, #3, September, 1956, pp. 61-79. 14. A. NEWELL, J. C. SHAW, and H. A. SIMON, "Report on a general problem-solving program," Proc. ICIP, Paris, France, June, 1959, pp. 256-264. 15. H. GELERNTER and N. RuCHESTER~ "Intelligent behavior in problem-solving machines," IBM J. Res. and Dev., Vol. 2, #4, October, 1958, pp. 336-345. 16. J. SLAGLE, PH.D. Thesis, Department of Mathematics, MIT, June, 1961. 17. T. G. EVANS, "The use of list-structured descriptions for programming manipulations on line drawings," ACM National Conference, Denver, Colorado, August, 1963. Acknowledgements The assistance of the Cooperative Test Division of the Educational Testing Service, Princeton, New Jersey, in providing a large set of geometric-analogy questions from its files is gratefully acknowledged. 6. H. SHERMAN, "A quasi-topological method for the recognition of line patterns," Proc. ICIP, Paris, France, June, 1959, pp. 232238. Thanks are also due to the Educational Records Bureau, New York, N.Y., for the statistics on human performance on geometrical-analogy questions cited in Sec. 4b. 7. 1. SUTHERLAND, "Sketchpad: a man-machine graphical communication system," SJCC, Detroit, Michigan, May, 1963. Most of the computation associated with the development and testing of ANALOGY was performed at the MIT Computation Center. EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM Larry E. Travis System De'velopment Corporation Santa Monica, California 1. SIGNIFICANCE OF THE THEOREMUTILIZING PROBLEM (1.2) Synthesizing Sequences of Sy'mbol Transf or'mations (1.1) Computers as Expert Problem Solvers \Ve must be a little more specific about the kind of task we are interested in. In logic, constructing a derivation is constructing a sequenc~ of permissible symbol transformations by which given premises can be transformed into desired conclusions. The permissible symbol transformations are the basic rules of inference. Thus in a system where the permissible transformations include detachment (Le., transforming P and P ~ Q into Q), interchange of conditionals and disjunctions (Le., transforming P ~ Q into '"""' P v Q and P v Q into '"""' P ~ Q), and commutation of disjunctions (Le., transforming P v Q into Q v P), an example of a derivation is the following: There are a large number of difficult intellectual tasks which consist of synthesizing a partialordering. To program a computer we must synthesize a sequence of computer instructions; one way of constructing a logical derivation is by synthesizing a sequence of inference-rule applications which transform given premises into desired conclusion; one way of choosing among alternative moves in a game is by synthesizing and evaluating trees of possibly ensuing moves. Can we program a computer to perform such tasks? Can we program a computer to perform such tasks expertly, Le., well enough that it can by itself surpass a skilled human being in the performance of such tasks? A decade of work on the subject by Gelernter, Minsky, Newell, Samuel, Shaw, Simon, Slagle, Tonge, and others has given us a clear yes in answer to the first question. But the answer to the second is still very much in doubt. The comparisons so far have been with Navy boots, high school sophomores, and college freshmen. The question at issue concerns the difference between performing a sequence-synthesis task well and performing it poorly if at all. We would ask: What is skill in tasks like theorem proving and program writing?' Suppose we have two machines each of which is able to prove at least some theorems but one rather like an unskilled student and other rather like a skilled mathematician. How do they differ? GIVEN PREMISES: (1) P ~ (Q ~ R) (2) P (3) ,-,R DESIRED CONCLUSION: '"""'Q DERIVATION STEPS: (4) Q ~ R Detachment transformation of Premises (1) and (2) (5) ,-,Q v R (6) R v ,-,Q 339 Interchange transformation of Step (4) Commutation transformation of Step (5) 340 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 DERIVATION STEPS-Continued (7) --R ~ --Q Interchange transformation of Step (6) (8) --Q Detachment transformation of Premise (3) and Step (7) Similarly in computing, constructing a program is constructing a sequence of available symbol transformations by which given input can be transformed into desired output. The available symbol transformations are the primitive computer instructions. Similar characterizations can be offered for game playing, puzzle solving, manipulating algebraic expressions, and other tasks. There are, of course, important differences among these tasks, especially in the way the tasks are defined for the problem solver, but for the present our concern is with what they have in common. (1.3) Skill at Sequence S"nthesis It would appear that one thing most sequence-synthesis tasks have in common, at least insofar as human problem solvers are concerned, is a way in which skill at their performance is acquired. Becoming a skilled logician or a skilled programmer mainly consists in learning useful combinations of the available primitive transformations and in developing a sensitivity to the occasions on which it is appropriate to apply these combinations. As we shall see, there is more to it than this, but this is a good place to start. Some examples will clarify our conception of skill. The easiest examples to come by are those where the skilled human being takes explicit note of a useful combination of primitives. Thus the logician adds defined rules of inference to his system. Or he just simply adds a theorem to his list of established statements. For instance, very early in the development of a system for the propositional calculus the logician will notice the usefulness of the transposition theorem (Q ~ R) ~ (--R ~ ,..."Q) and, after proving it with a sequence something like Steps (4) through (7) of our example above, will add it to his list of axioms and other proved theorems. Henceforth in developing a derivation, whenever he needs to transform an expression whose structure is like the antecedent of the theorem into an expression whose structure is like the consequent of the theorem, he merely cites the theorem. He does not indicate, as is done in the example above, all the primitive transformations, the primitive rules of infer ence, which are required actually to effect the transformation. Theorems are usually thought of as additions to the stock of assertions rather than as additions to the stock of transformation rules of a logical system. But we shall be primarily interested in them in their role as explicit representations of defined, complex transformations, i.e., transformations which can be effected by combinations of primitives. In programming, the defined, complex transformations are the subroutines and macroes, some explicitly formulated, labelled, and stored on tape but many more, accumulated in the experience of writing many programs of many different kinds, "stored" more or less completely only in the programmer's head. (1.4) Construction Problems and Existence Problems An important distinction is to be noticed between the kinds of sequence-synthesis problems represented by programming problems on the one hand and by theorem-proving problems on the other. The programmer has to produce the actual sequence of primitive transformations while the logician does not. This arises from a difference in the reasons why the two are interested in synthesizing sequences of primitive transformations. In logic the question of interest is whether such a sequence connecting premises and conclusion exists at al~; if it can be shown to exist the logical problem is settled, and though the logician may be worried about problems of elegance and efficiency, these are extralogical worries. The programmer, on the other hand, must synthesize an actual sequence of primitives. There is usually no question of whether such a sequence exists; the programming requirement is production of one to do some work. The programmer needs the actual sequence, not just the knowledge that one exists. \Ve shall refer to the programmer's problems as construction problems, opposing them to the logician's existence problems. The distinction is EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM 341 not a neat one because the logician usually uses methods such that from a demonstration of existence of a sequence of primitive inference rules he could automatically (and tediously) produce an actual sequence if called upon to do so. But the distlnction is worth noting because it is construction problems with which we shall be primarily concerned in the present paper. ally to write and run what can reasonably be called a theorem-utilizing program. We shall first describe the program. Then we shall report some results obtained with it and some conclusions to be drawn from these results. Finally, we shall indicate what appears to be a reasonable direction for future research on the problem. (1.5) Related Research 2. DESCRIPTION OF A THEOREMUTILIZING PROGRAM There has been a large amount of work in the recent past concerned with programming computers to prove theorems _(e.g., Wang,42 Gilmore,7 Davis and Putnam,3 and Robinson l9 ) and with programming computers to program themselves (e.g., Kilburn et al. 9 and Amarel).1 Though several of these authors, e.g., Wang and Amarel, respectively, have commented on the apparent importance of a mechanical theorem prover's being able to save and utilize previously-proved theorems and of a self-programming machine's being able to save and utilize previously-developed subroutines, there has been little work directly on the problems raised. Programs with at least some ability to utilize previously-proved theorems (or their counterparts) are Newell, Shaw, and Simon's Logic Theorist, 15,16 Gelernter's Geometry Theorem Prover,5,6 Slagle's Automatic Integrator,22 and Simon's Heuristic Compiler.21 We would suggest that it is because it attempts to prove theorems by using previously-proved theorems that the Logic Theorist is of special importance even though it turns out to be a rather weak theorem prover in comparison with other systems which use alternative formulations and methods. Wang avers that Newell, Shaw, and Simon's approach to proving theorems by machine is rather like trying to kill a chicken with a butcher knife. It is true that there are more efficient ways to dispatch a chicken, but there may be no better way to learn how to use a butcher knife. We turn, then, to reporting an investigation concerned not with theorem-proving or selfprogramming machines per se but with the more specific problem of how such sequencesynthesizing machines might be made efficiently to exploit previously-developed sequences when later given a problem of constructing a more complex sequence. Our approach has been actu- (2.1) Criteria of Program Evaluation Our goal has been to construct an automatic theorem-proving system which proves theorems by efficiently utilizing previously-proved theorems. The criterion of success is realization of a theorem prover which becomes progressively better as it accumulates and otherwise modifies its store of previously-proved theorems. It should be remarked that improvement of the store is not merely a matter of accumulation and, as a matter of fact (as clearly demonstrated by Newell, Shaw, and Simon'sl8 work), under certain conditions simple increase in size of the store Gan hinder rather than help problem-solving ability. This ability is more a matter of what particular previously-proved theorems are avaiiable and what information is stored about them and how this information is used, than it is a matter of how many are available. It is specifically to these points of how to select useful theorems for remembering, how to abstract them and their proofs, and how to use the abstracted information that the research here reported has been directed. (2.2) The Program's Problem Domain For our experiments with theorem proving we devised a special problem domain rather than selecting some standard domain such as the propositional calculus, group theory, or plane geometry. We shall presently give our reasons for doing this, but first let us describe the domain. It will probably be simpler if we first present problems in the domain as if they were problems of programming a very simple computer best characterized as a vector adder. The computer has no data memory. It is given as input an ordered n-tuple of numbers and operates on this n-tuple according to a given 342 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 program. The program is a simple sequence with each of its constituent basic instructions operating directly on the output of the immediately preceding instruction in the sequence. Each of the basic instructions adds a quantity (possibly zero or negative) to each term of the n-tuple on which it operates. Thus each of these instructions can also be represented as an ordered n-tuple of numbers. Consider an example of a 4-tuple adder which has as its basic instructions (S) <0,1,2,3>, (T) <3,2,1,0>, and (W) <0,-1,1,0>. A typical programming problem might be that of synthesizing a sequence of these instructions which will transform <4,9,2,7> into <10,10,10,10>. As can easily be verified, one program which will do this is (S), (T), (T), (W), (W), (W), (W). The first instruction (S) transforms <4,9,2,7> into <4,10,4,10>, the second instruction (T) transforms this into <7,12,5,10>, etc., until operation of the final instruction (W) gives an output of <10,10,10,10>. Problems of the kind we are considering can just as well be interpreted as theorem-proving problems in what might be called a tally calculus. Thus, rather than the n-tuple <4,9,2,7> we might have the expression AAAA/BBBBBBBBB/CC/DDDDDDD and we might ask whether AAAAAAAAAA/BBBBBBBBBB/ CCCCCCCCCCIDDDDDDDDDD can be derived from it given basic rules of inference corresponding to (S), (T), and (W). These rules are to be interpreted in such a way that (W), for instance, means: make no change in the A's, delete one B, add one C, and make no change in the D's. In such a tally calculus there might well be existence problems as well as construction problems. Most of the calculi with which we have actually worked, however, have been supplied with basic rules of inference sufficient to make possible the derivation of any expression in the calculus from any other expression. Our reason for choosing' such calculi has been that we are interested in problem-solving mechanisms and not in whether some particular derivations are possible, and we learn little from applying these mechanisms to problems which have no solution. It will be a very long time before we can hope to have non-algorithmic problem-solving mechanisms powerful enough that their failure to discover a solution is presumptive evidence that no solution exists. We have imposed on our problem solver requirements additional to those of simply constructing a sequence of transformations which will transform a given input n-tuple into a desired output n-tuple: (1) There are lower and upper bounds on the numbers intervening between input and output; thus we might require that no intervening number be less than or greater than 19. (2) Even within these bounds, there can be certain proscriptions; thus we might proscribe any n-tuple containing the number 8 from intervening between start and goal. ° A comment is in order concerning our reasons for choosing such an apparently artificial problem domain. Our intention was to design a vehicle with which we could conveniently investigate utilization of theorems or subroutines, one simple enough that most of the program design effort could be concentrated directly on such utilization rather than on matters of other kinds, e.g., pattern recognition. As we shall see, in this we were only partially successful. Further, we wanted problems possessing some of the features of programming or theorem-proving problems but not so many of these features that an extremely complex program would be required to solve any problems at all. Finally, we wanted a problem domain which did not present difficult problems for us. This allows us to take an Olympian view of what our mechanical problem solver is doing. Let us see how our problems of programming a vector adder compare with programming or theorem-proving problems which people actually have to solve. In our case, there are no difficulties associated with merging two lines of processing into one. Formally this is because the constituent transformations (the basic instructions or rules of inference) are all singleargument functions, a situation which holds neither in real-life programming nor in reallife theorem proving. Further, there are no difficulties in our case associated with branch- 343 EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM ing or iteration or recursion. Thus our problems are far sinlpler than actual programming or theorem-proving problems, but they have enough of the features of these actual problems to be interesting. In particular, our problems involve discovering a combination of basic transformations which will effect some particular complex transformation and, further, of discovering an acceptable ordering of these basic transformations. The requirement that they be ordered in a particular way derives from our imposing bounds and proscriptions on an acceptable solution. Thus, in the example given above, if we were to stipulate that no 4-tuple containing a number larger than 10 might intervene between start and goal, the suggested sequence (S), (T), (T), (W), (W), (W), (W) would no longer be a solution but the identically constituted though differently ordered sequence (W), (W). (W), (W), (S), (T), (T) would be. These various aspects of our problem domain should become clearer with the examples to follow. (2.3) Spatial Interpretation of the Problem Domain An Olympian overview is facilitated by giving the n-tuples a spatial interpretation. Thus in the case of pairs (2-tuples), we can correlate the start and goal pairs, the .intervening pairs, and the proscribed pairs with cells in a grid. In Figure 1, for instance, we correlate a start pair <4,10> with the cell marked "S," a goal pair <4,14> with the cell marked "G," and the proscribed pairs with the blacked-out cells. Given the pairs <2,1>, <2,-1>, <-1,2>, and <-1,-2> as the basic transformations available, the problem can be interpreted as that of working out a path (like the one in the figure) with these basic transformations as constituent steps, with no step ending in a blacked-out cell, and which begins at S and ends atG. By supplementing the set of basic transformations of the example so that we have one transformation for each of the eight legal knight's moves in chess, we have the problem of moving a knight around on a chessboard. We have christened the system defined by the four listed transformations the half-knight calculus (HKC) and that defined by the full set 19 16 15 14 13 12 II 10 o I 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 Figure 1. Spatial Interpretation of a 2-Tuple Transformation Problem. The problem consists in constructing a path, like the one indicated, from S to G. The path must be constituted only of allowed steps and none of these steps can end in a blacked-out cell or out of bounds. of eight the full-knight calculus (FKC). In order to avail'ourselves of the two-dimensional spatial interpretation, nlost of our work has been with 2-tul'le systems like these. (Both the HKC and FKC are, by the way, complete in the sense that their basic transformations are sufficient to transform any pair into any other pair.) The powers of human visual perception and spatial intuition are very useful in analyzing our mechanical problem solver's performance, in comparing and evaluating its solutions and attempted solutions, and in deciding on procedures which might be useful to it. There is some danger in using this spatial interpretation, however, for one is likely to be misled concerning the nature and difficulty of the problems insofar as the computer is concerned. We would specifically warn the reader against a tendency to look upon the problems as problems of running a maze with the connectivity of the nodes corresponding to the directly perceived connectivity of cells in the grid. And we would suggest that he can grasp the difficulty of solving the problems involved without any aid 344 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 from visual perception and spatial intuition, as the computer must solve them, if he thinks of what his own situation would be in the fiveor six-dimensional case. (2.4) A Succession of Problems from Easy to Hard Stipulation of lower and upper bounds and of proscribed n-tuples defines a problem field. For our experiments with 2-tuples we defined five fields, each with a lower bound of 0 and an upper bound of 19 but with the differing sets of proscribed pairs represented by the blackedout cells in Figures 2 through 6. It should be understood, again with respect to possibly misleading aspects of the spatial interpretation, that the research has not been concerned with designing a system which would learn local properties of the five fields. Our reasons for restricting the number of fields rather than using a different one for each different problem was solely one of convenience. Since only a part of each field is relevant to a particular problem, the five fields serve to determine a variety of ordering constraints sufficient for our present purposes. 19~~~~-+-+-+~~~4-+-~~~-+-t-r~ 18~~~-+-+-+~~1-~+-+-~~ri-t-t-r~ 17~~~-4-+-+~~4-4-+-+-~~~-+~-r~ 16~~~-4-+-+~~4-4-+-+-~~~-+~-r~ 15~~~-4-+-+~~4-4-+-+-~~~-+~-r~ 13~~~-4-+-+-+~~4-~+-~~~-+~-r~ 12 II 10 8·1-~~f---+-+-+~~4-4-+-+-~HH~-+-+-I o I 2 3 4 5 6 7 8 9 10 II 12 13 1<1 15 16 17 18 19 Figure 3. Second Problem Field. The problem illustrated is Problem 24: within the given field of proscribed pairs, transform <18, 6> into <11,12>. Given the five fields, a succession of 60 progressively more difficult problems was deter13 12 18 1I~~~-+-+~~4-4-+-+-~~~-+~~4-4 10 17 16 15 14 t---11-+-+..........1-+-+- ~~~-+-+~~~4-+-+-~~-+-+~~4-4 1--1---4-+1--1---4-+- o I 2 3 into <18,19>. o I 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 Figure 2. First Problem Field. The problem illustrated is Problem 6: within the given field of pros~ribed pairs, transform <17, 10> into <17,8>. mined by using each of the fields 12 times and for each use defining the numbers for the start pair and the goal pair with a semi-random process controlled so that the difference be- EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM 345 (2.5) Storage of Theorems 19 17 16 15 14 12 II o I 2 4 3 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 Figure 5. Fourth Problem Field. The problem illustrated is Problem 48: within the given field of proscribed pairs, transform <3, 8> into <11,18>. 17 ~~-+~~4-+-~~-+- 13 12 II 10 o I 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 Figure 6. Fifth Problem Field. The problem illustrated is Problem 55: within the given field of proscribed pairs, transform <9, 15> into <18,1>. tween the numbers was likely to be greater for the later problems in the succession. Figures 2 through 6 represent Problems 6, 24, 37, 48, and 55, respectively. We can now briefly describe the program with which we are investigating theorem utilization and which so far has been used to test some ideas concerning how utilization of solutions of the easy problems early in the succession of 60 problems might facilitate solution of the more difficult problems later in the succession. The most important idea was that once a problem was solved the solution should be generalized and precisely categorized in a certain way, with the solution's possible relevance to the solution of later, more difficult problems being determined by its categorization. We wished to devise something more efficient than mere addition of new theorems to an unordered theorem list which then had to be scanned in its entirety each time a search was made for possibly useful theorems, as was the case with Newell, Shaw, and Simon's original Logic Theorist (with the exception of one experiment where the theorems were categorized and weighted on the basis of their experienced usefulness in connection with particular methods) . One can use various techniques to make the selection of theorems possibly useful for a particular problem more efficient than a bruteforce search of an unordered list. For instance, Newell, Shaw, and Simon used certain similarity tests in the search of the Logic Theorist's theorem list; they experimented with enhancing search efficiency by pre-computing the in:formation for each theorem needed by these tests and storing it along with the theorem, thus obviating its computation each time the list was searched. Although the indexing was inverted, this represents a simple case of the kind of theorem categorization which we have wished to implement and test. Stefferud:!:1 has modified the Logic Theorist so that previouslyproved theorems are stored not on a simple list but as part of a complex list structure (called a theorem map) which compactly represents the syntactical structures of all the theorems available. Theorems are selected for possible usefulness in connection with a given problematic expression by a single operation matching the syntactical structure of the expression against this theorem map. That part of the map with which an appropriate match is obtained determines the whole set of previously- 346 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 proved theorems which have the appropriate syntactical structure for use with the given expression. Our approach is not unrelated to Stefferud's but we would like to work out a theorem-selection system which uses clues to the usefulness of a previously-proved theorem more powerful and more selective than whether or not there is a precise match between the syntactical structure of part of the theorem and the syntactical structure of part of the ex-pression to be proved. What such clues might be in the particular case of a propositional calculus like that with which Stefferud is working we leave for consideration in another place; here let us see what we mean for systems like the half-knight and full-knight calculi. It will be recalled that the problem of constructing a derivation in one of these calculi consists in discovering what combination of basic transformations will transform a start pair into a goal pair and in ordering these transformations so that none of the intervening pairs are out of bounds or on the list of proscriptions. For the experiments here reported what was saved of any particular successful derivation was information about how many applications of each basic transformation were required plus the defined transformation which this combination was able to effect. This was abstracted from the order in which the basic transformations were needed and from the particular start pair and goal pair which had defined the problem. Thus, as illustrated in Figure 7, the sequence <-1,-2>, <2,1>, <-1,2> might be used to solve the problem of getting from <1,2> to <1,3>. What will be saved about this solution is that it uses one each of the indicated basic transformations and that this combination will effect the defined transformation <0,1>, i.e., it will transform any pair into . Now if the system is later confronted with the problem, perhaps as a subproblem of some larger problem, of transforming <11,8> into <11,9> in the problem field also as illustrated in Figure 7, it might retrieve and try to use the information it has saved from the earlier solutions; but after it has retrieved the information it has the additional task of ordering the basic transformations in a manner which satisfies the constraints of the new problem. Be~ause of the 19 18 ~~-+~~4-+-~-+-+~4-+-~-+-+-r4 17 16 15 t--f--i-+- 1" ~I--+-+13 12 ~~-+~~+-+-~~~-+~~+-+-~- 11~~-4-+~~4-+-+-~-4-+-r-r;-+-~~ 10 o 1 2 3 " 5 6 7 8 9 10 11 12 13 1" 15 16 17 18 19 Figure 7. Use of a Previously Developed Solution to Solve a New Problem. A combination of allowed steps which has been discovered to solve the problem <1, 2> to <1, 3> can later be used to solve <11, 8> to <11, 9>. But a different ordering of the steps will be required for an acceptable solution. role they serve as defined transformations, let us call the abstracted solutions "theorems" even though they may not look much like what the logician usually refers to with this term. The categories on the basis of which theorems are filed and retrieved must of course be relevant to the information which is saved about the theorems. Four easily computable binary properties have been used to determine these categories. For each theorem let us call the change it makes in the first term of a pair on which it operates its X-value and the change it makes on the second term its Y -value. The four binary properties used to categorize a theorem are whether its X-value is positive, whether its Y-value is positive, whether its absolute X-value is greater than or equal to its absolute Y-value, and whether its absolute Y -value is greater than or equal to its absolute X-value. Tests to determine the values of these properties for a particular theorem are arranged on a simple discrimination tree (which can easily be expanded or otherwise modified for experimentation with alternative proper- EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM ties). When filed, a theorem is sorted down this three into one of eight categories corresponding to octants in a Cartesian coordinate system. The theorems in one category are put on an ordered list with the position of a theorem on the list being determined by the sum of its absolute X-value and its absolute Y-value. The basic transformations of a calculus are placed on these lists right along with the defined transformations. It will be noticed that the FKC has one basic transformation for each of the eight lists. (2.6) Retrieval of Theorerns For our problem domain, problems can be characterized with exactly the same properties used to characterize theorems. Thus the primary technique used to select a theorem possibly relevant for solution of a problem is to sort the problem down the same discrimination tree used to separate and file theorems. Given the relevant list of theorems, the first selection is that one which leaves the smallest remaining problem. (More precisely: given a problem of transforming 8 into G, first selected is that theorem T for which the sum of the absolute X-value and absolute Y-value of the problem T (8) -to-G is smallest.) Lower priority selections are made by alternately taking theorems from above and below the first choice on the theorem list being used or, if the list is too short to provide enough selections as determined by an externally specified parameter, by making selections from the lists for neighboring categories. We should make two clarifying remarks before proceeding with program description. There is a difficult question when specifying a system such as the one being described concerning the criteria which should be used in selecting a theorem to be tried. About the only thing which recommends the size of T (8) -to-G as a primary criterion in our case is that it is easily and quickly computable. A more sophisticated system would attempt an evaluation of the difficulty of actually being able to realize T (8) with T given the ordering constraints; -and an evaluation, with measures better than size alone, of the problem T (8) -to-G. Gelernter's oft-cited use of a semantic model was directed 347 toward evaluating the counterpart of T (8)to-G for his Geometry Theorem Prover. (Working backwards, what his system actually had to evaluate was 8-to-T-l(G). It did this by testing whether T-l (G) was true in a model determined by 8, the given premises.) A second clarifying remark concerns sorting problems down the same discrimination tree used to file theorems. In general, there might be properties which would be useful for categorizing theorems but which would be inapplicable to problems or vice versa. In our present case, for instance, the total number of basic transformations involved in their definition might be a property useful for categorizing· theorems. Problems have no such property. There would be, however, a corresponding problem property whose value could be tested at the same point in the sorting tree. There would have to be such a problem property else the theorem property would not be a useful basis of categorization. It should not be forgotten that the categorization is for purposes of determining relevancy of a theorem to a problem. To continue with program description: The first thing the program does when attempting to solve problems is to construct a tree of possibly useful theorems .. The number of branches from each node is determined by an externally specified parameter N. The program selects N theorems possibly useful for the problem 8-to-G; then it recursively selects N theorems for the problem T (8) -to-G determined by the best T of its first N selections; then it recursively selects N theorems for the problem U (T (8) ) to-G determined by the best U of its second N selections; etc. Recursion is terminated either when a closure is effected, i.e., when a sequence of selected theorems together transform 8 into G (ignoring the ordering constraints) ; or when a maximum allowable depth of recursion, as determined by an externally specified parameter, is reached. At the point when recursion is first terminated, the tree will have the structure indicated in Figure 8. The non-maxim urn-depth terminal nodes will be extended to full depth only if necessary, Le., if the theorem choices from the completed part of the tree do not result in a solution. 348 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Third N selections (for problem U(T(S»-to-G) N =3 Second N selections (for problem T(S)-to-G) First N selections (for problem S-to-G) \ \ \ mechanisms uses a recursive technique of breaking problems into subproblems, and we shall say something about this particular, very important mechanism in the next section. (2.8) Breaking Problems into Subproblems :' Figure 8. Tree of Theorem Selections. The tree is extended in. the manner indicated until closure is effected or until a recursion limit is reached. If the top line of theorems (T, U. V, etc.) does not result in a solution, other lines (T, U, V', etc.) will be tried. The tree will be extended from the lower nodes, e.g., that for T', wh~n but only when necessary. (2.7) Application of Retrieved Theorems Given the tree of selected theorems, the program now picks up from the top down (as the tree is represented in Figure 8) sequences of theorems which effect a closure, trying one after another until it attains a solution, runs out of allowed time, or exhausts the tree. Given a closure-effecting sequence of theorems, the program still has a difficult task to accomplish for it must order the constituent basic transformations so that the path of pairs intervening between start and goal is in bounds and does not include any proscribed pairs. The ordering mechanisms constitute the longest and most complex part of the program. (The entire program is approximately 4500 IPL instructions in length.) However, we shall not here describe these mechanisms because their operation is not directly relevant to our present theme of theorem utilization (though, as we shall see, their complexity and the fact that they don't work very well are directly relevant to conclusions to be drawn concerning effective use of theorems by machine). One of these A number of authors, especially Minsky,1l·12 have remarked on the importance of enabling problem-solving programs to transform a large problem into a constellation of smaller problems, "to plan" as Minsky calls it. This has turned out, however, to be a very difficult capability to implement, and there are not many (if any) operating programs which have it, though Newell, Shaw, and Simon 17. 18 have done some work toward incorporating a planning capability in their General Problem Solver. We suggest that theorems (or, more generally, what we have called abstracts of previously developed sequences) might well be primary instruments of planning. When planning, a human sequence synthesizer usually is especially interested in discovering a special kind of subproblem, viz., one very like problems he has already solved. A good deal of human sequence-synthesizing activity is an attempt to transform a problem into a constellation of subproblems each of which belongs to a familiar kind of problem, familiar in the specific sense of the problem solver's having available abstracts of sequences previously developed as solutions for problems of just this kind. The kinds of familiar problems will be more or less general and the abstracts of previous solutions more or less complete, and that's why, even after retrieval of the abstracts, the familiar problems are still problems. So in our case, the retrieved theorems often won't work. We shall presently want to examine why, but here we simply want to point out the connection between planning and theorem utilization. There are besides theorem utilization other kinds of planning, many of which appear to be special to particular problem domains. One such special kind is used by our present program when faced with the task of ordering a set of basic steps so that none of them result in a proscribed pair of numbers. When starting on any problem, the program has no way of determining which if any of the given proscriptions in the problem field are relevant to 349 EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM the problem. The reader can grasp the nature of this difficulty by imagining himself given the proscriptions simply as a list of pairs of numbers rather than in their spatially interpreted form. He then has no easy way of telling which of these pairs might get in the way as he attempts to work out a path between one pair of numbers S and another pair G. The program proceeds, to begin with, as if none of the proscriptions were relevant. As it tries one path after another it compiles a list of those proscribed pairs which have prevented success. This list's reaching a certain size, as determined by an externally specified parameter, is taken to indicate that success is unlikely without taking into consideration the specific configuration of proscriptions which are relevant. I Taking its occurrence on the list as evidence that a particular proscription is relevant, the program determines rows and columns (speaking in terms of the spatial interpretation) of relevant proscriptions and then looks for gaps in these rows and columns through which a successful path is apparently going to have to proceed. Cells within these gaps are then selected as midpoints defining possible subproblems, each such cell M determining the subproblems S-to-M and M-to-G. Several such 1\1's might be selected at one time, and they are put on an ordered list with the position of an M on the list being determined by criteria like the number of proscriptions in its immediate vicinity. The order of this list determin~s the order in which attempts are made to solve the subproblems determined by the M's. Given a pair of subproblems S-to-M and M-to-G on which to work, the system attacks these recursively, approaching each with the same mechanisms it originally applied to the problem S-to-G. Thus S-to-M might be broken into S-to-M' and M'-to-M, and so on. The only difference between operations of the program at one subproblem level and at another is that the lower the level the less the effort allowed per subproblem before reporting failure. The amount of effort expendable is controlled by reducing the potential size of the tree of theorem selections at each recursion onto a subproblem, as illustrated in Figure 9. This reduction also provides an effective means of ensuring that recursion will terminate. l)At SUbprobl.... Lev.1 0 2)At Subproblem Level 1 3)At Subproblem Lev.1 2 Figure 9. Theorem Selection Trees at Different Subproblem Levels. The amount of effort spent on subproblems is controlled by reducing the potential number of theorem combinations which can be tried each time descent is made to a lower level. o I 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 Figure 10. Recursive Solution of Subproblems of Problem 37. The original problem S-to-G was solved by breaking it into the subproblems S-to-M and M-to-G and recursively attacking these. The problem M-to-G was further broken into M-to-M' and M'-to-G before success was achieved. 350 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 A solution obtained by the method of determining subproblems and recursively attacking them is illustrated in Figure 10. The original problem <8,9> to <18,19> was broken into <8,9> to <1,12> and <1,12> to <18,19>, and the latter of these in turn into <1,12> to <1,14> and <1,14> to <18,19> before success was attained. This simple example does not illustrate the frequent occurrence of a number of alternative midpoints being tried and discarded before usable ones are hit upon. 17 16 15 14 I--~-+- (2.9) Deciding Which Theorems to Save We earlier raised the problem of improving a theorem file in ways other than simply adding to it. There is built into the program a mechanism for periodically culling from the file theorems which have been of little use. We shall not report more fully on the mechanism in this paper, however, because our experiments have not yet provided us with the opportunity to test the mechanism and to compare it with alternatives. 3. RE$ULTS OF EXPERIMENTS (3.1) Vindication of Theorem Utilization Sometimes the designer's product doesn't work at all; the airplane crashes or the ship sinks. So the first questions to ask about a complex problem-solving program are "Does the program work when all the pieces are put together? Does it solve problems ?" We are happy to report in the present case that the answer is yes. The program does solve problems, in some cases q.uite elegantly (Figure 11 shows one machine solution of Problem 56 in the FKC), in other cases very inelegantly (Figure 12 shows another machine solution of the same problem), but of course in some cases not all (Figure 13 shows a good but unsuccessful try for Problem 38 in FKC). More to the specific point of the experiments, the problem-solving power of the system was substantially increased by its ability to utilize theorems. We tested this by comparing its performance on Problems 51 through 60 without any transformations in its file except the basic ones of the FKC and its performance on these same ten problems with its file containing the o I 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Figure 11. A Good Machine Solution to Problem 56. The program is able to solve problems-in some few caS2S, elegantly. (The illustrated solution is in the FKC: allowed steps corr€sponding to the eight legal knight's moves in chess.) basic transformations of the FKC plus all the defined transformations (about 60) which it had accumulated from solving those which it could of Problems 1 through 50 and associated subproblems. Almost all of the time of the theo- 17 16 15 14 13 12 t--t--1---t-I--~-+- ~~~~~+-~r;~~~~~~~r;- 1I~~~~~~~+-~~-4~~~~+-~~ 10 o I 2 3 .. 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Figure 12. A Poor Machine Solution to Problem 56. The program is able to solve problems--but, in many cases, very inelegantly. (The illustrated solution is in the FKC.) 351 EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM 19 19 18 18 17 17 16 16 15 15 104 104 13 13 12 12 II II 10 10 o I 2 3 4 5 6 7 8 9 10 II 12 13 104 15 16 17 18 19 Figure 13. A Good But Unsuccessful Machine Try. The program in any of its versions was unable to solve Problem 38. To get an idea of the kind of ordering constraints the program is up against, the reader might try moving a knight from S to G without letting it touch down on any of the blacked-out cells. remless machine was spent in searching for combinations of the basic transformations which together would transform start pair into goal pair. Given its difficulty in discovering any combination at all which would effect this closure, it was unable to solve any but the simplest of the ten problems. Successful solution of the more difficult problems required having a variety of these combinations easily available from which to discover one which could be ordered to satisfy the ordering constraints. The machine with theorems was quickly able to discover combinations of basic transformations which effected a closure and could spend its time attempting to arrange these combinations in an acceptable order, quickly discovering an alternative combination in case further work on some given combination appeared futile. Examination of the relative performance of the two machines (the theoremless machine and the machine with theorems) on Problem 60 is instructive. Figure 14 illustrates the solution which the machine with theorems was able to achieve in 48.5 minutes (and in 1,276,809 IPL cycles). This solution involved a total of o I 2 3 04 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 Figure 14. A Hard-Won Machine Solution of Problem 60. Even with the use of previously-provided theorems, the achievement of this solution required a great deal of effort (over a million IPL cycles). Without theorem utilization the program was unable to sol'9'e the problem at all. (The illustrated solution is in the FKC.) 13 different closures. In contrast, the thooremless machine in 60 minutes effected only two closures, with combinations which it was un .. able to order satisfactorily, and fruitlessly spent the rest of the hour allowed it trying to effect a third. In at least one important sense the HKC and FKC derivations required in the present experi= ments are not trivial; this is with respect to the very large number of basic transformations required to effect them. For the problem illustrated in Figure 15, for instance, the solution achieved by our program (with theorems) is 44 steps long; an apparently optimum solution (achieved by a human being) is 28 steps long. A question raised by these performanees is whether there are any general strategies (heuristics) , other than utilization of previously developed subsequences, powerful enough to enable synthesis of very long sequences in non-trivial cases like theorem proving and programming. It might be suggested that all the machine needs is something like the spaceperception ability which enabled the human 352 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 17~~~~~~~~~~~4-+-~~-+~~ 16~~~~~~~~~~-b~~~~~~H 15 14 13 12 i--~".q..J- 1--~:!IIo.,A-i--1---4~- i--........~""" "~~~-N~~~~~~~~+-~~~~~ 10 I--~~~~~+-~~~-r~~r-~-+~, o I 2 3 4 5 6 7 8 9 10 " 12 13 14 15 16 17 18 .19 Figure 15. Machine and Human Syntheses of Long Sequences. The machine-produced sequence (heavy lines) is 44 steps long, the human-produced sequence 28 steps long. The human's visual abilities enabled him to get along without much prior experience on this kind of task. But without such abilities the machine needed prior experience to synthesize a sequence so long. (The illustrated solutions are in the FKC.) being to synthesize his 28-step sequence without much prior experience on this kind of task. But this is just to shift the problem to another domain. Given the spatial interpretation of the problem, the human being makes extensive use of experience, if not ontogenetic then phylogenetic. So the question still stands. Certainly no mechanical game player which looks ahead move by move is able to come close to looking 28 (or 44) moves ahead. Samuel's checker player 20 might achieve such farsightedness but it is of interest to note that this is with a technique which is a kind of utilization of previously-developed (trees of) sequences. Whenever the checker player, using what Samuel calls its rote memory, applies to positions X moves ahead a previously-computed evaluation which involved looking "X moves ahead, this provides it with an evaluation identical with what it would achieve by directly synthesizing and evaluating sequences X+ Y moves in length. The effect is cumulative so that even- tually the checker player is able to look ahead a very great distance. Samuel's work, by the way, leads to an important conclusion relative to what we have called the theorem-utilization problem: however powerful theorem utilization might be, attainable degree of theorem generalization and attainable efficiency of theorem retrieval impose limits. Samuel was able to use retrieval techniques as efficient as binary search but was unable to achieve any significant degree of generalization; as a consequence his file of experience would reach an unmanageable size before it enabled his machine to play good checkers at any stage of the game. He had to turn to techniques other than "theorem utilization" in order to achieve good mid-game playing ability. It should be noted, however, that these other techniques were not alternative ways of synthesizing longer sequences but were improved ways of evaluating given sequences of limited length. Hence, our question still stands: are there any strategies, other than utilization of previously-developed sequences, powerful enough to enable synthesizing of very long sequences in difficult problem domains if this is what we have to do, say to program a computer? We might apply the question directly to Newell, Shaw, and Simon's General Problem Solver (GPS).17 The primitive operators used by GPS correspond to what we have called basic transformations of a sequence-synthesizing problem domain. Given this correspondence, the many similarities between the operation of our program in its theoremless mode and the operation of GPS are obvious. Of course, adding to GPS an ability to accumulate and efficiently to use defined operators may not prove especially difficult, though it raises questions of exactly the kind with which we are concerned in this paper. To complete our case for the indispensability of theorem utilization for difficult sequence synthesis, we might temporarily look up from our experiments and appeal to authority. After making the point that non-trivial logical deductions usually require many steps of inference, Carnap2 (pp. 36-37) remarks, "In practice a deduction in science is usually made by a few jumps instead of many steps. It would, of 353 EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM course, be practically impossible to give each deduction which occurs the form of a complete derivation in the logical calculus, i.e., to dissolve it into single steps of such a kind that each step is the application of one of the rules of transformation of the calculus .... An ordinary reasoning of a few seconds would then take days." We suggest that, at least in this case, as it goes for men, so it goes for machines. (3.2) Failure to Satisfy Goal Criterion We must now turn to reporting and analyzing less positive results of our experiments. Over-all, our program was not a very good problem solver. For the HKC, in two runs (differing in respects inessential to our present interest) through the entire succession of 60 problems, the program was unable to solve 33 in one case and 30 in the other. For the FKC, in two runs the numbers were 21 and 17. Of course, increasing the time allowed per problem would have improved this performance somewhat, but we experimented enough with this variable safely to conclude that the time limit would have had to be extended unreasonably to assure success with most of the unsolved problems and, in any case, time taken to achieve solution is an important measure of the power of a heuristic problem solver. Even more disappointing than these tallies of unsolved problems was their distribution. It was mainly problems in the later part of the succession of 60 which were not solved, the difficult problems requiring for their solution use of solutions obtained for problems occurring earlier in the succession. The obvious conclusion is that "theorem utilization" wasn't working very well. At first we thought that perhaps the failure was a matter of the succession of problems not being long enough, i.e., that the program just needed more simple problems from which to build up a store of experience before being able successfully to attack the difficult ones. We attempted a minor "empirical test of this possibility by giving the program a second chance on the unsolved problems from the first run through the succession of 60, a second chance during which it could use for any given problem theorems acquired after its first attempt on the problem in addition to the theorems which had been available at the time of the first attempt. No more of the difficult problems were solved with the additional theorems than had been solved in the first place. (3.3) Implications of the Failure for Theorem Utilization In the difficult cases the program was failing because it did not possess techniques powerful enough to deal with severe ordering constraints. For problems like Problem 48 illustrated in Figure 5, for instance, even when the program had discovered a usable combination of basic steps, ordering them in the precise way necessary to avoid the proscribed cells required something the program does not possess, viz., a sharp sensitivity to patterns formed by these cells. The best the program could do on Problem 48 is illustrated in Figure 16. Of course, the sharp sensitivity alone would be of little value. There would be no point to the program's being able to recognize a pattern of proscriptions if this didn't indicate 17 16~~-+~~+-~~-+~~~~~-+~~ 15 1.~~-+~~+-~~-+~~+-~~~~~ 13 12 II 10 o I 2 3 • 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 Figure 16. Machine Inability to Achieve the Precise Ordering Required to Solve Problem 48. Appearances notwithstanding, the program is a long way from attaining an ordering of steps which will move it through the tight part of the illustrated problem field. Most of the program's failures are attributable to weaknesses of its ordering procedures-despite the fact that these constitute the bulk of the program. (The illustrated attempt is in the FKC.) 354 PROCEE:DINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 something for it to do. Recognition of a narrow channel through which the successful path must proceed would have to suggest procedures for selecting the particular basic steps and an ordering of them which would effect a path of acceptable dimensions; similarly for recognition of a required sharp right bend, or of a required detour to the left, or of a circumscribing rectangle within a certain size range and with entrance and exit required at certain places, etc. So the system would need" a much more powerful pattern-recognition capability than it presently has; but it would also need a system connecting this capability with methods of behaving appropriately given certain patterns. trivial, long sequences. One can, of course, write a more and more ~omplex program, but at some point he runs out of programming time or out of program storage space in his computer. The limits we are talking about are practical. Also, a point of special importance for artificial intelligence research (see Kelly and Selfridge8 ), there is a serious question whether the more and more complex program would not necessarily have to be more and more ad hoc with respect to a particular problem domain. These possibly rash statements are at least partly based on our experience with the present program. It will be recalled that information about the order of basic transformations in a successful sequence is discarded by the present program when that sequence is abstracted and stored as a previously-proved "theorem." There is no point to saving information about order when the pattern-recognition system used for retrieval of theorems relevant to the solution of particular problems is able to recognize only direction and distance aspects of the problems and nothing about ordering constraints. However, given the appropriate pattern-recognition capability there would be reason for saving information about order. For our particular problem domain (as well as for many others), one of the things which makes a problem difficult is its ordering constraints. Because our system is unable to recognize clues involving these constraints, and with these clues to retrieve relevant information from its file of past experience, it falls down badly on problems involving severe ordering constraints. We need, then, two complementary things: improved pattern-recognition capability and different kind's of information saved about our previously-proved theorems. tend~d It might be contended that we are begging the question. Surely there are ways to satisfy severe ordering contraints other than the retrieval and use of orderings previously worked out. To this charge we reply that there might well be such alternative ways but there appear to be sharp limits on how far one can go with them. This is related to the point we made earlier about the apparent necessity of theorem utilization for the mechanical synthesis of non- We have tried a long list of techniques into increase the ordering ability of our program. Our hope has been to achieve a program of sufficient problem-solving capability that we could use it as a vehicle for experimenting with alternate schemes for accumulating, culling, abstracting, filing, retrieving, and applying theorems; and we did not want the ordering aspects of the problems in our problem domain to enter into these inyestigations concerning theorems because of the very great difficulty of programming the pattern-recognition mechanisms which would be required. Hence, our valiant try to get the program to handle the ordering aspects of its problems by means other than theorem utilization. The try has failed, however, and so we have not so far been able to experiment with alternate schemes of theorem utilization; the reason is that changes in performance due to changes in these schemes are masked by effects due to weaknesses in parts of the program which were supposed to perform their functions without theorem utilization. We salvage something from the failure if we learn a lesson concerning the importance of theorem utilization even for those parts of our problems where we thought we could get along without it. The reader will be helped in deciding whether he agrees with us that we can't get along without it, if we indicate some of the things we did in trying to do so. We have already described probably the most important thing, viz., having our program look for gaps through which a successful. path apparently will have to proceed and use these gaps as bases for breaking a given problem into subproblems. EXPERIMENTS WITH A THEOREM-UTILIZING PROG.RAM Several other things were tried, many of them rather difficult to implement. Actually the reason we conducted as many as four separate runs through the succession of 60 problems was that with each successive run we hoped (to no avail) that we had devised significantly improved ordering techniques. We tried a technique for shortening paths, and thereby reducing the likelihood of violating proscriptions, by replacing parts of unacceptable paths with theorems spanning the same gaps but involving fewer basic steps. This caused over-all performance to deteriorate; the program did better by considering many alternative paths, rather than considering only a few and spending a lot of time on each of these trying to edit it into acceptable form. We tried techniques so easy as blindly reversing the order of steps in a path and so difficult as determining and effecting permutations which would usually move a bothersome part of a path from an area containing proscribed cells into an area containing none or at least fewer. We programmed and tried a complex process intended to relax the ordering constraints on a problem by discovering pairs, alternative to the given start and goal pairs, into which the given pairs could easily be transformed but lying in an area allowing greater freedom of choice. We programmed and tried a complex process which packed the steps of a path into an area of minimum dimensions. None of these things worked very well. The simple fact of the matter is we have been unable to devise and realize effective ordering techniques which make no use of previous experience on simpler problems. It might be suggested that we have boxed ourselves in with the kind of theorem utilization that we do have, that rather than selecting theorems which determine the constituent steps of a path and then being faced with ordering and re-ordering these steps, a problem solver in our domain should select basic steps one by one, selecting a step only if this can be combined with previously selected steps in a manner that satisfies ordering constraints. It seems to us that the experiment reported above with our program in theoremless mode is relevant to an answer. In this mode the program has no alternative but to select basic steps ~ne by one, and it will be recalled what a difficult time it has 355 merely selecting a long sequence of steps which will effect a closure let alone a sequence which satisfies severe ordering constraints as well. 4. GENERAL CONCLUSIONS (4.1) The Significance of Theorem Utilization (Again) We want now to indicate the significance we find in our experiments for matters beyond better ways of programming a computer to move a knight around on a chessboard. First, let us repeat-for emphasis-our main point concerning the apparent importance of using previously-developed, defined transformations for tasks involving synthesis of long, orderconstrained sequences of certain given basic transformations. That the ability to save and use previous experience in this way is important for a man is apparent enough to anyone who will carefully observe himself proving a theorem or writing a computer program. Per-. haps we have gone some way toward determining just how important such an ability might be for a problem-solving automaton as well. But we have said enough about this. (4.2) Importance of Pattern Recognition for' Problem Solving Second, we would remark on the importance of pattern-recognition capabilities to effective theorem utilization and hence to effective problem solving. The conclusion to which we are led is that a bottleneck presently preventing more powerful problem-solving machines is perceptual impotence. Our machines won't be good problem solvers until they are good theorem utilizers, and they won't be good theorem utilizers until they are good pattern recognizers. Hard as this bottleneck might be to break, unfortunately it isn't the only thing inhibiting progress. ( 4.3) Program Organization Difficulties Third, we would say something about some thorny problems of program organization that arise in connection with a complex heuristic program. We have discovered as have other experimental programmers (e.g., cf. Newell 13 ) that a problem-solving program which breaks 356 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 problems into subproblems and recursively attacks these, breaks the subproblems into subsubproblems and recursively attacks these, etc., will likely be a rather poor problem solver unless there are some rather strict controls imposed on its recursion. Such a program is likely to dig itself into a hole and never get up and out of the hole to a position from which it can recognize that it should have been digging in a different location in the first place. Perhaps even worse it is not able to relate the different subproblems which it defines at different levels of recursion to each other to discover overlap, duplication, and the possible significance of joint solution of subproblems quite separated from each other on the subproblem tree. And, finally, when it fails at some point and has to decide what subproblem to turn to next, obviously it would be a better problem solver if it were able to base its decision on some kind of overview of the subproblem tree rather than turning automatically to the next subproblem one level up from the point in the tree at which it failed, but this is precluded in the case of uncontrolled recursion. We can report that all these difficulties (and more) arose in our case, and we have no general resolution of them to offer. However, it is perhaps worth reporting that we found very useful, indeed even indispensable, the technique of building up a tree of subproblem records (isomorphic with the tree of subproblems solved and unsolved, pending and disposed of) which could be consulted by the program at any stage or level of recursion when deciding whether to create a new subproblem or whether to go to work on some previously created subproblem. Even though the techniques used for examining and comparing the records on this tree were quick and dirty, controlling recursion with the tree served to prevent much duplication of effort and misinterpretation of the relative importance of particular subproblems. (4.4) Harder Problems (for Programs and for Programmers) Finally, something should be said about where we see the main difficulties if we are to implement an experience-utilizing system of the kind here discussed for proving theorems in a calculus of greater complexity and sophis- tication than the HKC or FKC, say for the propositional calculus or the predicate calculus, or for programming computers more powerful and interesting than vector adders. The first problem will be in determining a useful set of properties with which to categorize theorems or subroutines and a corresponding set with which to categorize problems and then, of course, developing pattern-recognition procedures which enable detection of these properties. The second, and it will be recalled from our discussion above that we consider this a problem complementary to the categorization problem, will be determining how successful solutions should be abstracted, i.e., what kinds of information about them ought to be filed, and then, of course, developing procedures which can perform the desired abstraction. But these two problems are overshadowed by one of a kind which has not yet been solved, by us or by anybody else, even for systems so simple as the HKC and the FKC. This is the problem of generalizing a number of particular, previouslydeveloped solutions into a single solution schema. We haven't said anything about this generalization so far because our present program doesn't do it, instead saving and using only particular solutions. But skilled human problem solvers obviously aren't so limited in their use of previous experience and much of their power derives from their ability to use general schemata which have been built from many separate, particular solutions. We can easily illustrate the kind of general schemata we are talking about by looking at the HKC. Figure 17 shows a human solution of Problem 58 in the HKC. (Problem 58 was much too difficult for our program, in any of its several different versions, to solve.) Several parts of the successful path might have been constructed by using general schemata (and were so constructed, if we are to believe the introspection of the human who solved the problem.) Thus in synthesizing the lower part of the path manifesting a regular alternation of the basic steps <-1,-2> and <-1,2>, the problem solver could have put to good use a general schema enabling left movement through any required distance within minimum vertical dimensions simply by alternately applying EXPERIMENTS WITH A THEOREM-UTILIZING PROGRAM 19 18 17 16 15 (Q ~ -.,P), etc., but with representing and using the general transposition schema under which are subsumed all four theorems with which one can transpose a conditional. The problems of machine representation and use are hard enough. Maybe their solution will suggest some ideas about how to solve the more interesting problem of getting a machine to generalize and establish such schemata by itself, or at least with no more help than a human student has. But these problems of machine learning of general concepts are so hard as to suggest that maybe we ought to relinquish this faith that artifacts can be made to think. Maybe we ought to become Platonists all. ~~-+~~~~~-+~~~+-~-+=T~~ ~~-+~~4-+-~-+~~~~~~~~~ ~~-+~~4r~~-+~~4,~~-+~~~ ~~-+~~~~~-+~~~~~-+~~~ ~~-+~~+-~~-+~~~~~-+~~ 1-4 13 12 11 10 ~~-+-6~~~~~~~4-~~-+~~ o 1 2 3 -4 5 6 7 8 9 10 II 12 13 1-4 15 16 17 18 19 Figure 17. Human Solution (of Problem 58) Requiring Extremely Complex Ordering. The human problem solver used general schemata. Thus rather than moving from <12, 2> to <4, 2> by remembering and applying some combination of steps which effects a specific <-8, 0> transformation, he used a general schema for moving left any distance required. (The illustrated solution is in the HKC; notice the allowed steps.) these two basic steps. As it now stands, the program saves from its experience much more specific information, viz., information about how to effect the left-moving transformations <-2,0>, <-4,0>, <-3,2>, <-5,2>, <-3,-2>, <-5,-2>, etc. But it can not generalize this specific information into the schema enabling left movement through any desired distance. A human being apparently has no difficulty making such a generalization. In fact, even rats don't have any difficulty in making a generalization as simple as the one illustrated (cf. Krechevsky).lO But we are surely up against one of the hard problems of computer learning. For the present we are concentrating on methods of representing general schemata within a computer and methods of using them to solve problems. To take an elementary case, in the propositional calculus we are faced not with representing and using as inaependent, isolated items the particular theorems (P ~ Q) ~ (-.,Q ~ ~P), (P ~ -.,Q) 357 ~ ACKNOWLEDGEMENTS The research described is part of the independent research program of System Development Corporation. The particular computer program described was conceived while I was participating in the Research Institute in Heuristic Programming held at RAND during the summer of 1962. I wish to acknowledge my indebtedness to Institute staff members for their guidance during the initial stages of writing and debugging the program. Also I want to thank J. Feldman, T. Ruggles, and S. Shaffer for helping me construct the list-processing system required for - efficient running of the program on a Philco 2000. Special credit is due Dr. Feldman as designer of the initial version of the system (see Feldman 4 ) . The system in its present form is a one-address variant of IPL-V ( see Newell et al, 14) , REFERENCES 1. AMAREL, S. On the automatic formation of a computer program which represents a theory. In Self-organizing system, 1962, edited by M. Yovits et ale Washington: Spartan, 1962. Pp. 107-175. 2. CARNAP, R. Foundations of logic and mathematics. Chicago: University of Chicago Press, 1939. 3. DAVIS, M., and PUTNAM, H. A computing procedure for quantification theory. Journal of the ACM. 7 (1960) :201-215. 4. FELDMAN, J. TALL-A list processor for the Philco 2000 computer. Communications of the ACM. 5 (1962) :484-485. 358 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 5. GELERNTER, H. Realization of a geometry theorem-proving machine. In Information processing. Paris: UNESCO, 1959. Pp. 273-282. 6. GELERNTER, H., et al. Empirical explorations of the geometry theorem machine. Proceedings of the 1960 Western Joint Computer Conference. Pp. 143-147. 7. GILMORE, P. A proof method for quantification theory. IBM Journal of Research and Development. 4 (1960) :28-35. 8. KELLY, J., and SELFRIDGE, 0. Sophistication in computers: a disagreement. IRE Transactions on Information Theory. IT-8 (1962) :78-80. 9. KILBURN, T., GRIMSDALE, R., 'and SUMMER, F. Experiments in machine learning and thinking. In Information processing. Paris: UNESCO, 1959. Pp. 303-309. 10. KRECHEVSKY, 1. "Hypotheses" in rats. Psychological Review. 39 (1932) :516-532. 11. MINSKY, M. Heuristic aspects of the artificial intelligence problem. Lincoln Laboratory Group Report 34-55, 1956. 12. MINSKY, M. Steps toward artificial intelligence. Proceedings of the IRE. 49 (1961) :8-30. 13. NEWELL, A. Some problems of basic organization in problem-solving programs. In Self-organizing systems, 1962, edited by M. Yovits et al. Washington: Spartan, 1962. Pp. 393-423. 14. NEWELL, A., et al. Information Processing Language-V Manual. Englewood Cliffs: Prentice:-Hall, 1961. 1964 15. NEWELL, A., and SIMON, H. The logic theory machine. IRE Transactions on Inform.ation Theory. IT-2(1956) :61-79. 16. NEWELL, A., SHAW, J., and SIMON, H. Empirical explorations of the logic theory machine. Proceedings of the 1957 Western Joint Computer Conference. Pp. 218-239. 17. NEWELL, A., SHAW, J., and SIMON, H. Report on a general problem-solving program. In Information processing. Paris: UNESCO, 1959. Pp.256-264. 18. NEWELL, A., SHAW, J., and SIMON, H. The processes of creative thinking. In Contemporary approaches to/ creative thinking, edited by H. Gruber et al. New York: Atherton, 1962. Pp.63-119. 19. ROBINSON, J. Theorem-proving on the computer. Journal of the ACM. 10(1963): 163-174. 20. SAMUEL, A. Some studies in machine learning using the game of checkers. IBM Journal of Research and Development. 3 (1959) :210-229. 21. SIMON, H. Experiments with the heuristic compiler. Journal of the ACM. 10 (1963) : 493-506. 22. SLAGLE, J. A heuristic program that solves symbolic integration problems in freshman calculus. Journal of the ACM. 10 (1963) :507-520. 23. STEFFERUD, E. The logic theory machine: a model heuristic program. RAND Memorandum RM-3731-CC, 1963. 24. WANG, H. Toward mechanical mathematics. In A survey of mathematical logic, by H. Wang. Amsterdam: North-Holland, 1963. Pp. 224-268. ANALYTICAL TECHNIQUE FOR AUTOMATIC DATA PROCESSING EQUIPMENT ACQUISITION Solomon Rosenthal Directorate of Data A utoma.tion Headquarters, U.S. Air Force Washington 25, D. c. In this paper we describe inconsiderable detail the "Analytical Technique for Automatic Data Processing Equipment Acquisition" developed and documented with Mr. Ernest L. Holt and Mr. Joseph T. Averitt for the Directorate of Data Automation, Headquarters, U.S. Air Force. This is the first public description of this Technique and how to use it. This system, including a weighted factor selection method, was designed to be flexible, impartial, efficient and effective. Flexibility is achieved by permitting factor and weight modifications which best suit each individual selection. To maintain impartiality the factors considered and the weights assigned are varied prior to solicitation of proposals. The development of the technique was not influenced by applications to be processed on selected equipment,_ nor by specific equipment characteristics or considerations for any vendor. Equipment selection time is shortened, the resulting selection is valid, and the documentation produced fully supports the selection. Due to the built-in flexibility, impartiality and efficiency of the technique, it is acceptable to the ultimate user of the selected equipment, to approval authorities, and to vendors-in short, it is effective! The Technique is designed to facilitate competitive selection of Electronic Data Processing Equipment regardless of the purpose for which the equipment is to be acquired. It is equally applicable for centralized or pick-your-own concepts of selection. There are four parts, each designed as part of an integrated whole. The parts are: I Instructions for preparation of specifications for submission to vendors. II Instructions for submission of EDPS proposals. III Instructions for validation of EDPS proposals. IV Instructions for scoring of EDPS proposals. PART I The "Specifications Module," or "Specs for Specs" as some like to refer to it, was developed to provide a standard arrangement, organization and approach to the problem of preparation of the EDPS specifications for submission to vendors. A more complete description would provide a table of contents for specifications, and detailed instructions for the contents of each chapter and paragraph. Here" rather than present the prescribed format, we will discuss the material to be included in the specifications. Detailed knowledge of the other three parts is not required to implement anyone part, but to insure effective use of the Technique, those charged with the responsibility for its overall administration must be thoroughly familiar with all four parts. 359 360 ~ROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Specifications prepared in accordance with the standard format provide sufficient information about the job to be done to permit interested vendors to submit proposals that are truly responsive, and also to establish for the scorer a firm basis for application of the scoring module to be described later. Prospective bidders must be fully informed about intended locations and the user's organizational mission and objectives. The user's plans for conversion from any existing system; for centralized or decentralized programming in the case of multiple installations; his equipment delivery and implementation schedule; and his intention to pilot test before ordering more than one system, are all bits of information a vendor needs to know before investing in a proposal. Any conditions or minimum standards, with which the vendors must comply, must be documented. These specific requirements, or restrictive criteria, should be held to a minimum, and must not be designed to eliminate any vendors from consideration. If these rules are followed, it seems reasonable to state that non-compliance with any of these specifics could result in a proposal being eliminated from competition. Restrictions might justifiably apply in such areas as: Delivery (Dates and Places) Cost (Maximum Funds Available) Timeliness (Allowable delay between remote station inquiry and response, etc.) Support (Programming, maintenance, systems analysis, etc.) Physical Environment (Room size, controls, floor) Equipment Characteristics (Print positions, character sets, random access storage, etc.) Compatibility (With data banks, programs, other interface requirements) Expansion Capability (Anticipated workloads) Software (Monitors, executives, scientific subroutines, common languages, etc.) Others We must remember that to qualify for inclusion in the above list an item must be a firm, unalterable requirement. The heart of a specifications package is that part which documents the job to be done. For each data processing task, a statement is required which fully describes a representative sample of the total workload of that type. As large a percentage of the total as is practicable must be stated for each task. A sufficient number of task statements must be made to indicate collectively the requirement for all computer characteristics involved, i.e., an adequate number to enable determination of the configuration and capacities needed to accomplish the tasks. A complete statement includes a general description of the job and the function~l areas involved (personnel, materiel, accounting, scientific research, etc.). All processing requirements must be described in narrative form. All items, documents, files, master files, tables, etc., which are to. be used during---or produced by each processing step, should be discussed. Sort runs, edits, and so forth should be explained. Flow charts, consecutively numbered, describing the data system must be prepared. Each processing step must be identified and numbered to correspond to numbered paragraphs in the narrative. All inputs to, and outputs from, every chart must be clearly labeled to show their source or destination and to tie all the charts together. All inputs and outputs must be described in terms of the number of Alpha and Numeric characters contained. If sample document forms are available they should be included. A key item in our system is the "Workload Statement Chart" which must be supplied with the specifications; it provides a tabular summary of the information contained in the job statement just covered. A separate page or chart should be produced for each application or task. A more complete description would leave nothing to the imagination in the instructions for completing this chart. Here, we merely present a recommended "Workload Statement Chart" (Figure 1). Just one word of caution; the entries must serve to explain what came before, not to confuse the reader. In other words it should be simple for anyone to relate each entry to a corresponding part of the narrative and applicable flow chart. ANALYTICAL TECHNIQUE FOR AUTOMATIC DATA PROCESSING EQUIPME~T ACQUISITION 361 These prefaces should include, In the First: This part is used in preparation of instructions to vendo1"s. These instructions must be tailored to the requirements of the particular selection by those forwarding specifications and requests for proposals to vendors. In the Second: We request that your proposal be prepared as outlined in the attached "instructions for proposalorganization." • FigUre 1. Workload Statement Chart. The final entry on the "Workload Statement Chart" for each task should be "The above statement represents 'X'% of the total workload of this type." The final entry on the last chart should be "The above statement (s) represent(s) 'X'% of the known total workload." The percentages indicated will be used later in proposal preparation to project an estimate of the total time each proposed equipment will require to accomplish the total requirements. In most cases it will be advantageous to give prospective bidders a des'cription of the intended site (s). Include such information as available floor space, loading factors, maximum distance allowable beteen main computer and remotes, etc. To enable the vendors to propose a complete system, including power converters, regulators and other items needed, tell them what kind of facilities exist. You might also ,vish to tell them your current inventory of such things as magnetic tapes and control panels to give them a basis for quoting some trade-in values. PART II The "Proposal Module" serves a dual purpose. First, it provides those requesting a proposal with a complete guide to designing that request. Second, it provides the vendors with a "recommended" standard arrangement and indexing of the many subjects to be covered in a proposal. Therefore this module should have two prefaces -one to be read by those preparing the request, the other to accompany the request, to be read by those responding with proposals. * * * * * * * * * * * This format is not intended to restrict the contents of your proposal. The entire contents of Chapters I thru IX must relate to the specific configuration of equipment on which your proposal is based. Your proposal should include only that one equipment configuration adjudged by you to be the best for the purpose of this selection. If you consider it necessary to propose more than one configuration, each must be submitted as a separate and complete proposal ( Chapters I thru IX). Following Chapter IX, you may furnish any additional chapters which you consider pertinent to your proposal. When proposals are prepared in a standard format a giant step has already been taken towards simplifying the comparison of proposals. I t will be possible to pull from each proposal a chapter or paragraph, a prescribed chart or table, and compare the proverbial apples with apples, not with oranges or mangoes. The ,standard format in which proposals should be submitted, and the kind of information to be included, is: Chapter I General A letter of transmittal and a brief summary, for executive review, of the highlights of the proposal. Chapter II Responsiveness A statement of the ability to satisfy the restrictive conditions or minimum standards given in the specifications (see Part I-The "Specifications Module"). Chapter III Equipment configuration and costs A table (Figure 2) showing all components of 362 ~ROCEEDINGS-SPRING IWl: III. ~ 1II1IS2IPt'I(--"" ....> ..... ... - . . u.. (1.>_ ..... (_ !!.I!r.!l~ ..... I-u.. >-- ~to~::.'!...._ .... Figure 2. Equipment Configuration and Costs. Figure 4. Timing Estimate. the proposed system with complete pricing information. Chapter IV Systems and Timing 1. An explanation, with flow charts, of the vendor's approach to the solution of the problem, when the approach constitutes a departure from that given in the specifications. (In some cases, vendors are asked to design or redesign a data system.) Chapter V Questionnaire A questionnaire tailored by the requestor to the requirements ,of the submitted specifications and characteristics of each component structions make it clear that every question must be answered, and that each question will be answered with a number, a percentage, a price or other such specifics. Chapter VI Equipment Factors Technical literature giving detailed descriptions and characteristics of each component of the proposed configuration. Requirements for site preparation should be made known here. 2. Tables (Figures 3 and 4), called "Timing Estimate-Detail" and "Timing Estimate-Summary." These tabulate all time required by the main processor and associated peripheral devices. When these tables are completed properly, it is easy to compare them with the "Workload Statement Chart" (Figure 1) and to determine whether or not each step has been timed. There must be line entries on the "Timing Estimate Charts" for corresponding entries on the "Workload Statement Charts." ....- Quod o. I~ '~ - - .. -. IIopot1c t_ ::::. ~ I Proooaar I,....". ~r._ 1=> Totaln-.d Figure 3. Timing Estimate Stated Workload. eha pter VII Programming and Software A list and explanation of the software packages available for use on the proposed equipment, ·with emphasis on those particularly useful in the work to be performed. Chapter VIII Support 1. A description of the training available which meets the requirements of the prospective user. 2. A general discussion of the proposed maintenance plan, including a specific quotation of required preventive maintenance. 3. A statement about back-up equipment available for use in emergencies. Chapter IX Contract A copy of the contract (GSA schedule in the case of Federal Government), and any contractual deviations proposed for negotiation which would apply to the proposal submitted. When both specifications and proposals are prepared in a standard way, vendors know what ANALYTICAL TECHNIQUE FOR AUTOMATIC DATA PROCESSING EQUIPMENT ACQUISITION their equipment, if selected, would be expected to do, they can prepare their proposals most efficiently and economically, and a sound basis for objective comparison and selection is established. (p) Coatrol (J) PART III The "Validation Module" also serves two purposes. It provides the technicians with standard guidelines for determining the proposal's compliance with the specifications and its accuracy and adequacy. The validators are not charged with determining which of the proposed equipments is best but with determining whether or not the proposals being validated are acceptable. They do not compare one proposal with another. First the specifications package and the instructions (tailored for the particular acquisition) to the vendors are read. N ext the proposals are examined. After the validator is familiar with the specs and the proposal which he is to validate, he is ready to check all statements, answers to the questionnaire, tables and charts in a vendor's proposal. Claims about storage capacities, simultaneity, time estimates, and special purpose devices are some areas requiring detailed consideration. The validator must have at his disposal a comprehensive library of other publications from the vendor whose proposal is being checked, recognized technical publications from other sources and a vast storehouse of personal knowledge and experience in both the equipment and subject matter fields. During this process the validator may determine that a proposal does not comply with one or more of the requirements, is not adequate to accomplish the tasks described in the specifications, or that some inaccuracies exist in the proposal. When any such determination is made, the validator is required to prepare a statement containing the specific area or portion of the proposal considered unresponsive, inadequate or otherwise in question. This statement, with recommendations for correction or elimination from further consideration, goes to a reviewing authority for ,decision. If the reviewing authority concurs with the validator's statements concerning non-compliance or inaccuracies, the or Cod. ~ (q) Operator loco iJlclmiDC' 3,63 jor _ Lobel_ ~ prT CHARAC rEIUSTICS ~\1\ S;Jl.'c.:ci Instructions Peripheral Equlpmt"nt iO ,0 CapCit.:ity ';0 Main Memory ~o Magnetic Tape Peripheral Equipment Compatibility Program Tapes I:: I I I !O '; Cards Switchability Magnetic Tap~ Printers Other Peripherals 10 Reliability Minor differences would be understandable, but the differences are not likely to be minor. The magnitude of the differences can be glimpsed in the startling contrasts of two distinct groups which are presently working their w;ay to management positions. One group has a background in Finance and the other in Engineering. If these two groups were asked, independently, to use the Weighted Scoring Technique to evaluate a computer selection, I suspect the results would be very interesting. Joined as they may be in a common cause, identifiable principally, perhaps, with corporate well-being, they still would regard the method in much different ways and in all likelihood the major weights assigned by these two groups would differ considerably. Shown in Figure 2 is an example of the way these two groups might assign weights to the four major categories listed in Appendix I. How do you reconcile such differences of opinion? Moreover, does any reconciliation or compromise improve or enhance the acceptability of result= ant 'weighted scores' ? The difficulty-weakness--if you prefer, in the Weighted Factors Selection Method is that points assigned to the cost section cannot be 00 10 13 Specia.l Features Problem Timing5 Central Proce5sor Limited Input/Output Limited Balanced best from the evaluator's point of view, and each different. 's Other Cr..aracteristics EXPANSION POTENTIAL 200 Slack Time Central ProcesHor Peripberal Equipment MaxilTlllm Expanaiora Memory Peripheral Equipment 150 50 100 50 l5 25 SAMPLE WEIGlIT!NGS OF MAJOR CATE=RIES I SYSTEM SUPPORT Group With Financial Background 100 Program A •• i.stance Development Writing Converting Training Maintenance Offered Progra.m Testing Exi .ting Software Sort/Merge COBOL FORTRAN Report Generator Other Group With Eng. Background 10 Cost 850/0 Equipment Characteristics 10 10 20 50 250/0 Expansion Potential 50/0 25'fo System Support 5% 25% 10 10 10 10 10 }4'igure 1. 25% Figure 2. 370 PlROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 reconciled to the points assigned to the equipment characteristics section or to any other section, or even from item to item within a section. In other words, as the four categories presently exist in the Weighted Factors Selection Method, there is no common denominator between categories, thus there is no meaningful way of relating points from one category to another, or for that matter, between the elements within a category. COST-VALUE TECHNIQUE-GENERAL The Cost-Value Technique, like the Weighted Factors Selection Method, also recognizes the necessity of evaluating the 'extras' offered in various computer systems. The Cost-Value Technique, however, is unique in its handling of these 'extras.' The Cost-Value Technique studies any extras offered in the proposals to determine whether the claimed 'extras' are truly important extras or mere incidental elements that appear to be extras. For instance, a sixty-nanosecond memory and a ten-thousandcards-a-minute reader in themselves are not important extras, if extras at all; the important extra is how much slack time exists in the proposed system because of these high-speed units. For every 'extra' considered important, a study must be initiated to determine the value for the extra. The really distinguishing feature of the Cost-Value Technique is in the assignment of the value associated with these important extras. The value assigned is in terms of costs (dollars). A brief explanation may be helpful at this point because the significance of the value assignment in cost may not be readily apparent. By 'simply' assigning a cost-value to the various important 'extras' offered by the various vendors, a technique has been found to overcome the seemingly insurmountable problem (no objective method of relating points assigned the various items evaluated) of the Weighted Factors Selection Method. A common denominator has been established with which all. offered extras may now be related. Although the costvalues assigned to the various extras will still be a matter of each individual selection and will continue to reflect the opinions of the cost-value assigners, a value when assigned can be understood, examined, discussed, and changed in- dependently of all other individually assigned values. Another very important benefit derived by the use of cost assignment of value in the Cost-Value Technique is that management, for perhaps the first time in the twenty-year history of computers, can understand what is going on in an evaluation and is able to make informed decisions on the value of any disputed extras. The cost-values established for the various extras found within a proposal are assigned as 'credits' to that proposal (weight added to the value side). This allows each proposal submitted to be evaluated by taking its total 'outof-pocket' cost and subtracting the cost-value 'credits' awarded that proposal to find the difference. The proposal having the smallest difference (since the 'value' of having the application completed remains constant for all vendors) automatically becomes the proposal selected by the evaluation technique as being most advantageous in terms of the amount of systems equipment and support that is being obtained for the money spent. An example of how the Cost-Value Technique might be applied is shown in Figure 3. ~ Sih' Prqx~n.ti,)n Electrical E:-..pansitlu POlo 11\1 •• 1 Flrt.t Tim .. 1'.1", ~ Sl,,'ond Tim., 1110'" Third Tlml' lHud, ,).000 10,000 0 Conlitructlon E'lui.pment Installation Equ.ipment T unaportation V('ndor Support Penonnel Training S, 000 !)YiIltcm SUIJport.' Pf:'rtllonnt!'l 140,000 0 Existing Program. Backup Facili.ties Doc: umentiltion Program L Da.u. Conver. Program TeatillK ContlnuiDg Coats Computer System Eq\lipment Central Proce •• or Peripheral Equipmem Magnetic Tape Unite Card Reader Printer Card Punch Optical SC&nDer Controller. Auxillary Equipment Operation and Maintenance Per.onnel Ma..... ger Analyst. Progra.nuners Operator Other. Program DevelopmeDt Supplies Magnetic Tape Printer Paper Card. TralnlnK Exi6tlng Pro~ranll. Backup Fa.cilitieli 0 0 0 3.000 0 10. ill:( !;"'O,, ,,":',(1('1(' Other Extra .. Timeline .. Inqwry ~n5. 000 :;0.0('11 Reports eeairOlble Compatibility Purcha.e Option 2.Z0.OOO 50.000 1.2.0.000 43.000 150.000 332..000 100.000 130.000 10,OO( l~. 00(1 il1. {lOP 515.000 100,000 400.000 35.000 ZO.OOO 0 12..000 10,000 7,000 Total COtit ,,~, Documll·ntabon Progra.m Testi.ng - 2. 862. 00 DIFFERENCE Total Cost. $2.. I6Z. 000 Total Credit. 400.000 $ Z, 46Z, 000 Figure 3. Total Credit. $400,000 COST-VALUE TECHNIQUE FOR EVALUATION OF COMPUTER SYSTEM PROPOSALS Having now considered the three major premises, the purpose of an objective evaluation technique, and the distinguishing feature (value assigned in cost terms as credits and fewer factor categories used) of the Cost-Value Technique, let us proceed to a detailed examination of the previously mentioned cost-value study of all extras offered. COST-VALUE STUDY-WITH EXAMPLES The Cost-Value Technique's approach to the 'extras' offered by the vendors is to appraise the offered extras to determine whether they are worthy of inclusion in the. evaluation, and if so, to determine the cost-value of these extras. To avoid any bias, or appearance of bias, on the part of the evaluators, this study should preferably be initiated before the proposals are received. It thus becomes necessary to deal. with hypothetical or realistically anticipated extras. A sample listing of items sometimes considered as 'extras' will be helpful to our study and for that purpose a sample listing, similar to one that might be found in the Weighted Factors Selection Method of evaluation as shown in Appendix I will be used. The sample will be used as a beginning for our study of the essentials and extras offered in computer system proposals. The list shown in Appendix I is arranged into four categories: Costs, Equipment Characteristics, Expandability Potential, and System Support. Other groupings of items could also be used. Some of the items, like those under costs, are not extras, but are included because they must be evaluated in selecting a computer system. In the following sections, each of the listed categories will be studied; one of the results of our study will be the identification of these items and categories that are considered important in the Cost-Value Technique of computer selection. Before examining each of the four categories, shown in Appendix I, these words of caution and care are furnished. 1. Methods described herein for cost-value determinations are by no means the only methods that could be used. 2. There is nothing sacred in any of the costvalues established in this report since the 371 value of any item depends upon the likelihood of the user's need for that item. For example: a. If the described system is to be used only for one or two applications, and the size and volume of these applications are fixed, then the cost-value of Expansion Potential is likely to be nil. _ On the other hand, if the described system is the first system to be installed in a growing company, the cost-value of Expansion Potential will be very high because every hour of available expansion might be regarded just as valuable as each hour in actual use. b. If the described system is to replace an existing, compatible system, the cost-value of some of the System Support items like 'personnel loaned' or 'program assistance' may have no cost-value. If, however, the computer is for a relatively inexperienced group, 'personnel loaned' or 'program assistance' might each have a cost-value as high as, or higher than $20,000 a :man year. Another general word of explanation before getting into the details of cost-value assignment. It will be of help to discuss the idea of cost-valuing the extra. Thus, if four vendors, A, B, C and D, respectively, offer 100 hours, 125 hours, 50 hours and 70 hours of program checkout time, then the amount of 'extra' to which a cost-value should be affixed is 50 hours for A, 75 hours for B, and 20 hours for D. The extra for each item considered for each vendor is the amount offered by that vendor for that item, minus the minimum amount of that item offered by any vendor. If the extras offered are in different terms : Vendor A offers 100 hours of prime shift time, prior to delivery for checkout ; Vendor B offers 125 hours, any shift, after delivery, for checkout; Vendor Coffers 50 hours of third shift time, prior to or after delivery, for checkout-then all the items have to be converted to their cost-value first, and then their difference taken, to determine their 'extra' cost-value. The four categories found in Appendix I and to be ·considered now are Costs, Equipment 372 PlROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Characteristics, Expandability Potential, and System Support. tiating costs that exist between systems, should be recorded for evaluation." Cost Items All cost items should be considered in the evaluation. Items such as the cost of supplies or personnel costs may prove to be nondifferentiating (no significant difference found between vendors) in a given selection, but they should not be deleted from the evaluation list until their costs have been found to be truly non-differentiating. The second major category of items given points in the Weighted Factors Selection Method was Equipment Characteristi~s. Treating cost items as one-time costs or continuing costs is a matter of cataloging. Two rules must govern any proper treatment of cost items: The costs must be spread proportionately over the expected life of the system, and the system costs must change to reflect the costs of any planned system expansion. If, for example, the life of a system is set at six years and if a uniform expansion rate of 10% a year is expected over the life of the system, then each of the cost items on the list should be charged (if applicable) to the yearly system cost for six years; and it would be expected that the equipment cost for the sixth year would be larger than the equipment cost of the second year. Another important consideration relating to cost items is that they should show the cost for individual pieces of equipment to be used. This should be done in all cases except when the system is to be used for less than one shift or when the entire system is to be purchased. The break-out shows which equipments are actually to be used for more than one shift (higher rental). The breakout also indicates which items might be more favorably leased than purchased under a split acquisition. Equipment Characteristics The Cost-Value Technique does not consider any equipment characteristics, in themselves, to be important extras. Instead, their significance is measured in terms of the running time of the system which in turn determines the system's cost and expansion potential. Typical of the kind of equipment characteristics now being discussed are: the relative speeds and capacities of the systems; hardware compatibility; switchability; reliability; and some other special features. These obviously are features (extras) that determine the time required to complete the applications specified and in turn determine how much slack time is available for expansion, and as such, will be evaluated in the next section on Expansion Potential. Sample problem timing items (time required to perform some specific set of sample problems) should not be evaluated. They should be used exclusively for the validation of the application timings quoted in the proposals. Other characteristics (size, weight, etc.) either" are included in the space requirements, which are costed; or will appear as special items under a new category, 'Other Extras.' This new category is necessary when using the Cost-Value Technique because many extras to be evaluated do not fit under any of the existing categories. The third major category given points in the Weighted Factors Selection Method was Expansion Potential. No cost items should be duplicative"; that is, the system should not be charged twice for the same equipment or service. For example, if a card reader is used both on-line and off-line, the full cost of the card reader should not be shown twice. Similarly, program development, if performed by users-not contractor personnel-and personnel cost should not both be costed. Expansion Potential The Cost-Value Technique looks at Expansion Potential in a new way, which is thought to be more meaningful, and evaluates the category by considering the value of the extra offered. Basically, the thought behind the cost items can be summed up by saying, "Any differen- To estimate the Expansion Potential of a system, it is necessary first to calculate the run- COST-VALUE TECHNIQUE FOR EVALUATION OF COMPUTER SYSTEM PROPOSALS ning time required by the system to complete all required applications. And, at this point, we are compelled to list the elements and aspects of a computer and its use that constitutes its running time. The items that must .be considered in any calculation of the running time of a system are listed below: 1. Speed a. Central Processor b. Peripheral Equipment c. Auxiliary Equipment 2. Capacity a. Central Processor b. Peripheral Equipment 3. Special Features a. Parallel Processing b. Simultaneous Operations c. Other 4. Reliability a. Switchability b. Error Detection and Correction Features 5. Preparation Time a. Set-up/Take-down b. Program Insertion c. Media Handling I! u. Non-productive Time a. Reruns b. Program Checkout 7. Software Efficiency a. Compiled Languages b. Assembled Languages Centra! processor or peripheral equipment speeds are not new approaches to establishing the comparative merits of competitive equipment systems. What mayor may not be new, but which nevertheless must be considered, is the interaction between the central processor and the peripheral equipment. If the time required to do the processing called for in the run is great enough, it might cause delay in the input and output of the data. Thus, what must be determined is the effective speed of the peripheral equipment when operated in conjunction with the central processor. Also, the effect of auxiliary equipment on the system must be ascertained to assure its adequacy to meet the requirements of the input device. 373 The capacity of the central processor is important because it determines the number of program steps, plus data that can be accommodated at one time. Normally, the larger the capacity the faster the system because fewer load steps are necessary and because sorting and merging functions can process larger volumes of data per pass. The capacity of the peripheral equipment is also important. The printer is a good example of what is meant here. A printer having 160-print positions may permit 'two-up' printing whereas a 120print position printer may permit only 'one-up' printing, thus doubling its printing time. Again, a printer capable of printing eight copies may be able to do in one printing run what it would take some 'six copy' printers two runs to do. In both of these cases, the capacitY of the specific peripheral equipment would materially affect the total running time of the applications. The special features offered will usually affect the total running time of the system. Obvious examples of this are parallel processing, buffering, and simultaneous operations. The use of these features usually requires some investigation, but their applicability to calculating run time is immediately apparent. Other special features shouid be examined to determine if their value is to decrease total rup. time. When such is th~ case, their influence on run time should be calculated and used. The use of the above three items, speed, capacity, and special features, are all relatively straightforward in determining total run time. Now, however, we must determine the influence that the systems' reliability exerts on its total run time. How is this influence to be determined? This area of determination is admittedly an area of 'guestimation' rather than definable fact. However, it is felt that a better 'guestimation' can be made here about its influence on total running time than can be made in the absolute on its proper weighted scoring influence on the total selection. The area of concern here is system reliability. This is made up of a number of sub-it€ms: reliability of individual units, number of units available, switchability of these individual units, the availability of error detection and correction 374 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 techniques, etc. The intent of considering the above items is to determine the number of hours of downtime per month that could be expected for each of the systems proposed and the frequency of the errors resulting from the systems unreliability. Historical information, if available, should be used for 'ball park' figures. From these figures, educated guesses can be made of the time required to correct the processing effect of these errors and the number of hours of system downtime during which the system is not available. The withdrawal of these two figures from the total time available of the system gives a new systems time available figure. The preparation time factors must be determined and added to the processing times of each run for each of the systems proposed. The set-up and take-down times are concerned with the time required to prepare the peripheral device for a run and to take-down completed work after a run. Program insertion time is the estimated time required to insert the program into the system's memory. Media handling time is the time required to change any media (magnetic tape reels, new printer paper, etc.) that must be changed while the run is in process. The non-productive time for an application must be estimated and its time added to the total system time. Non-productive time includes rerun time due to operator or programmer error-.. and program checkout time. The magnitude of this factor (usually assigned as a percentage of the productive time) is dependent upon the experience of users! personnel and an estimation of the :number of new programs to be checked out each month. This factor could change (should decrease as the system gets older) from year to year. Another big timing factor, upon which little time has been spent to date, is software efficiency. The efficiency of the software is a very important factor since low efficiency may increase the running time by 25 % or more. This factor is still in the 'questimation' stage, but with work reliable efficiency factors could probably be determined for all the systems proposed. Now that all the items necessary for the determination of total running time, except the application, have been discussed, we will see how this running time is used in determination of the expansion potential. The determination of expansion potential cost-value is made by determining the value of an extra 'time-block' (increment of time) and deducting from this value the cost of obtaining the extra 'time-block.' The cost-value of succeeding time-blocks should be determined until a zero value is obtained. The use of time-blocks should probably be explained. Time-blocks are used because additional time on a computer system is recognized to have a decreasing value as more of it becomes available. Thus, the first time-block is of more value than the second or any succeeding time-block. This is because the first timeblock is obviously much more likely to be used than any succeeding time-block; indeed, this probability of use is one of the ways of assigning a cost-value to a specific time-block. But, why use time-blocks rather than hourly increments or percentage expansion figures? Any of the three could be used. Time-blocks were used because of their convenience. With hourly increments, a great deal of computation is involved. Percentage expansion figures, while likely to be more accurate, involve considerably more work. The extra work required by hourly or percentage approach is not warranted, since the total result cannot be more accurate than value assignments given and the value assignments regardless of the time method used, are still just estimates. Whichever method is used, the important thing is the decreasing value assigned to later time units. The determination of this cost-value of expansion potential should only be done once. The expansion considered is the expansion potential in the system after· its last expansion phase (last year of stated expansion) has been met. At this point, the future expansion potential of the systems become the 'extra' being measured between systems. Let us take an example with two variations to illustrate the concept explained above. Assume that two systems in the $20,000 a month class are proposed to handle the applications specified. The running times for the two systems, when extended to include all the items COST-VALUE TECHNIQUE FOR EVALUATION OF COMPUTER SYSTEM PROPOSALS necessary, are found to be 150 hours a month for system A, and 200 hours for system B. Further, system A requires 60 hours a month of maintenance and system B requires only 40 hours a month. 1. If the application is one in which there is little likelihood of having additional expansion, then time-blocks may be made rather long and their per hour value rather small-say 100 hours per timeblock and $50 per hour for the first block, $25 per hour for the next time-block, and $10 per hour for each time-block thereafter. Thus, system A would have a value of (assuming 530 hours a month maximum for both systems) $8700 and system B would have a value of $8200. From these value figures would have to be subtracted the cost of the extra shift machine rental, the cost of the personnel required to operate the system, the cost of any extra equipment necessary, and the costs of the extra system maintenance required. If there was any value remaining, it would be called the cost-value and credited once to the system. 2. Ii the appiication is iikeiy to grow (or if there is the possibility of sub-leasing extra time to a second party), then timeblocks may be made relatively small and their per hour value rather high-say 20 hour blocks and $150 or more per hour for the first time-block, with each succeeding block decreased by $5.00 per hour. Again, the cost-value for the two systems would be computed by adding up the value and subtracting the cost to obtain these hours. With a large valuation placed on a time-block, the costs of going to higher speed or to larger numbers of peripheral equipment might be justified, if so, the cost of the higher speed units or extra units would also have to be subtracted. The cost-value remaining would be applied one time as a credit. The last major category given points in the Weighted Factors Selection Method was System Support. System Support The Cost-Value Technique considers the 375 value of the extra offered. There are several methods of assigning cost-values to the System Support items listed in Appendix I. The simplest and perhaps the best method of cost-value assignment is to simply request the other vendors to quote costs to supply a service equal to what is con3idered 'best' in each case. Thus, if one vendor offers 24 hours on-site maintenance, and the other vendors don't, it might prove very meaningful to ask the other vendors what the extra charge would be (and then credit the largest cost to the vendor already supplying the item, and the largest cost minus each specific vendor's cost for the item would be credited to each of the remaining vendors). The same type thing could be done for program assistance, personnel loaned, training, program testing, documentation, and special software. However, sometimes the costs quoted would be so excessive that it would not make a fair base against which to award value. For instance,· if a user was impressed by some special programming routine, he might well ask the various vendors for the cost of supplying such a routine. But, he might receive answers of tens and hundreds of thousands of dollars, where if he himself were to go out and procure such a routine he would not be willing to pay over five thousand dollars. In such a case, the five thousand dollars should become the base. Thus, in cases where the user places a value on a service, lower than a vendor's cost, this value figure becomes the base for determining the item's value. In some cases, however, Le,> maintenance offered, or documentation, the vendor may not be able to give cost figures for supplying service equal to vendors because he just doesn't have the facilities necessary to give equal service. In such a case, the cost-value of such a service must be individually determined and might be considerably higher than the costs charged by any other vendor. In such a case, this higher cost-value figure should become the base. These system support items might also be handled by making it a mandatory requirement that specific amounts of these items be supplied. (This method is not recommended since a small vendor may not be in a position to meet some of these requirements, and thus he would be 376 PlROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 eliminated for items that should properly be treated as 'extras' or at most specific cost items.) The cost-value of these items might also be ascertained by the user, by taking each item in turn and truly determining its value to him. If the vendor agrees to loan three programmers for three months, what is this really worth? He will get nine man months of work by an experienced programmer which might be the equivalent of fifteen months of programming by his own people (or a new person added), but it may also represent a very important time saving (ready to receive the system sooner). Here are a couple of cost-value items, the cost-value most closely representing the users needs should be chosen. However, the cost-value should never exceed the cost of having the service contracted by someone else, thus, just because a helping hand (personnel loaned) by the vendor will result in the user being ready for the system installation three months earlier and thus result in a $100,000 a month savings for those three months, this doesn't make the value of the personnel loaned $300,000, because, people of the same caliber might have been hired from a software consulting service for $40,000. This $40,000 would then be the c~st-value assigned. Some items like back-up available and debugging facilities are support items on which the vendors cannot be asked to change or improve, therefore, their cost-value has to be evaluated as the items are proposed. An approach to determining the cost-value of back-up would be to determine the probability of experiencing a catastrophic failure, and then determining the cost associated with carrying on the computer activities on the back-up facilities available. The costs times the probability of its happening should give the probable cost for the various systems. Again, cost-value credits can be made out of these costs figures, by simply taking the highest cost minus a specific vendor's costs to determine h~s cr€dits. Cost-value determination for debugging facilities could be handled in much the same way. Some items previously covered in other categories in the Weighted Factors Selection Method have not yet been discussed in the Cost- Value Technique. These items are covered under the next category. Other Extras There are many other extras that might be offered by a vendor. Items like desirable compatibility or memory lockout can be handled by determining the costs that will be eliminated by the inclusion of such abilities. Thus, the costs that would have to be paid to convert tapes of one sort to another would be saved if the two systems were compatible; this cost therefore becomes the cost-value of such compatibility, or the costs of the time and trouble that could be saved by the inclusion of a memory lockout device becomes its cost-value. An imp8.29 66.:l.e to12 ;;: ~:,~:----~:':n!-------!:!..lt~~;- - 9:: ~~: ]'~: 2~'-: _ _ _.,..__'---_ _ ::: ------'J::~.:!L: ::'D~ __ ;:z ::!: 5;: 21:: :t::---..·i'"--! . ----'!~:.:!"-:-- ~;i8 1;: U _ _--",,"'--,_ _ _-"-po"---_ _ _--"'LD-.D.1IL'_ _ =:g----:~:---~:~.~~;-- 30.311 ('0.00 5519 ::: ::.:: Figure 8. Central Processor Utilization Report. Figure 10. Programming Requirements Report. take-down time, and 'memory utilization. It further provides information for those applicable computer systems on the parallel data conversion that could be scheduled during each program run. After listing these forecasts, SCERT then summarizes the projections for daily, weekly and monthly average utilization time, and finally, prints the projections for peak period utilization. Programming Requirements Report (Figure 10) This report, which is also a recap by individual program run, projects the number of program steps required to put the program "on the air," and then makes an estimate of the user's programming effort in man months. It further reflects the number of program steps saved by using applicable sub-routines and utility programs furnished by the manufacturers. The programming effort in man months projection is a function of the number of steps in the program related to the programming language specified by the user in the Environment Definitions. This computation is made in relation to the experience of the user's programming staff correlated to the programming difficulty of the computer involved, and the nature of the individual program run. Satellite Computer and Auxiliary Equipment Utilization Report (Figure 9) Based on the pre and post run history developed during the SCERT- simulation, this,report forecasts for each computer run in the system, the satellite computer and auxiliary equipment utilization. Auxiliary equipment utilization is subdivided by function such as card to tape, tape to print, and other off-line data conversion operations. Also provided is the number of hours of data preparation work, such as key punch time, required to support each run. ----60.?2-"2."-17.3S - - - - - - - - - Figure 9. Satellite Computer and Auxiliary Equipment Utilization Report. Application Summary Report (Figure 11) This report is a summation of all program runs within each application area and is provided to assist the user in determining the priority to be assigned to the various applications for implementation purposes. It shows the monthly system utilization by application together with a projection of programming effort and associated cost. Cost Summary Report (Figure 12) This report is provided for each computer configuration evaluated. It is a management digest of the costs associated with installing a 391 THE USE OF A COMPUTER TO EVALUATE COMPUTERS F'ng WI,.o'" "'J"IIC:~ 1'l1l:4 _APPLII:.lrl0!il_!::El!IT~'L PIlOCESSQ~ COOI= "OUA'9 UTILlZATlO"---SATELLITE CQ"PUTU--PROUI....1fIIl EF..!'QIU PAtbIiClllILMOJIITtol uTILIZATION I_ •• ~rR ""IE~"'E • srsfEI'I BIIiO • C01'lOuTeR CnMPLI=MENT NUMliliR 1414. FIltEOUENCY JII 01. -~P"T 1'."T'.":,;~~~~;~!SOF~=CO.D __!!ECo.n'_'U1'I.E'--1/n---":~F:~;=--0-.".....~" '.'lLOC. OF~••eLS ~n" T1~ 1 ---r:.t=~::l~::. OS' !!l'S:' ~uf!:.~S "e;' ___q .~ 32 cif..4~r().~~--- 2 _"'''~TEw HIl' MSf:o~1'1Ic!_.8_3J _ _ U _ " T _ _ D.l..~ ... ~ _ _ S _ _ _ U.9 .. _ _ 0J.9J _ _--"?Q.ao ___ J ... 1C1 _ _ 1~ i§ii.27 Figure 11. Application Summary Report. SCI;RllNAl,.'I'SIS PlRTt! COST SU!"I .... R'f A£il'oRT. FORSYSTEMor"o. CO"'lPUTeACOMPLEIlfENT NUI'IBERH14 1 2 Figure 13. Detailed Systems Analysis. _____________ _____ tnttl ~42~7 __ ll~~Q~0~0~0'-~~~=-~~ ~~- • ......0-0_ _ _ _ _ _ _ _ ~"'__ COST DFp aYfR.i..&6;.tUllIIH_ _ _ _- - ' > Figure 12. Cost Summary Report. computer and operating it once all applications are programmed. Recurring costs show the monthly rental computed by SCERT, based on the projections of utilization. If manufacturers have several rental options, SCERT will have selected the optimum. Shown for comparison purposes are the purchase' costs which include the cost of maintenance. In addition to the equipment costs, the recurrent cost analysis also reflects personnel salaries for equipment operation and data preparation and program maintenance. The One-Time Cost Analysis shows the costs of acquiring data media such as magnetic tapes. Detailed Systems Analysis (Figure 13) This is an optional three-part report which provides a comprehensive analysis of each computer run as simulated by SCERT. Part I reflects for each input and output file the total buffered and unbuffered time, the input/output device and channel assignment selected by SCERT, the number of reels, if magnetic tape file, and the optimum records per block developed by SCERT. Part II of the report is an analysis of internal computation, including time, memory, and program steps by function; i.e., arithmetic computations, decision and control, data handling and input/output control. Part III, a Special Time Analysis, forecasts the time required for program insertion time, program delay time due to errors, end of job rewind time, and other special functions. FinaHy, all timing forecasts are totaled to a net program running time. This report, in addition to providing detailed documentation and back-up for all other SCERT reports, can be invaluable to an analyst or programmer when designing the running program. It will solve such programming problems as input/output channel assignments, and the determination of optimum record size, blocking factor and program segment size. COMPUTER COMPLEMENT REPORT R eport Explanation This report reflects descriptive information for each computer configuration to be evaluated. Column Explanation: 1 MODEL NUMBER-Identifies each component in the configuration. MANUFACTURER-Identifies man2 ufacturer of each component. QUANTITY IN SYSTEM-The com3 ponent quantity in each configuration. PURCHASE PRICE-For each com4 ponent and associated special features. 392 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE. 1964 5 6 7 8 9 10 11 12 13 MINIMUM MONTHLY RENTALThe minimum rental option for each component. FLOORING SQ. FT.-Represents the minimum number of square feet of physical and work areas requiring false flooring for each component. COOLING BTU/HR.-Shows BTU's required for cooling each component. POWER SUPPL Y-Reflects power requirements for each component. CENTRAL PROCESSING UNIT-A subdivision of this report which shows all equipment connected on-line to the main frame. PERIPHERAL DEVICES-The naille of each component shown for the configuration. SPECIAL FEATURES-Reflects all special features and control devices. All costs and other factors are shown with the appropriate device. SATELLITE COMPUTER-Reflects the satellite computer (if required) to support the central processor. TOTAL EDP SYSTEl\1:-Reflects totals for all components for purchase price, monthly rental, floor requirements, BTU's and power in KW and KVA. 5 6 7 8 9 10 CENTRAL PROCESSOR UTILIZATION REPORT Report Explanat'ion This report reflects analytical information computed by SCERT on each program run in the entire system for each computer configuration evaluated. Column Explanation: 1 PERIOD-Identifies the periodic occurrence of the computer run. 2 FREQUENCY-Identifies the frequency of occurrence within the period. 3 RUN NUMBER-An identification number assigned by the analyst. RUNNING TIME-This is the pri4 mary projection developed by SCERT 11 12 simulation and represents elapsed running tIme of the program. It is based on optimum utilization of memory, instruction power and simultaneity. SET-UP-Reflects net amount of setup and take-down time required. TOTAL TIME-A summation of running time and set-up time. MEMORY CHARACTERS UTILIZED-Represents total requirements for each program, including instruction, work areas, I/O data, and software routines. An asterisk indicates memory has been exceeded, and SCERT has segmented. PARALLEL PROCESSING-When equipment has the capability of performing parallel processing; SCER T calculates the effective parallel conversion time available. This figure is computed within the parameters of memory, operational time and I/O channels available during each run. RUNS TOTAL-A summation of the SCERT projections of running time for a.II runs occurring in the period indicated. AVERAGE TIME-The time required for all runs plus a proration of the running time for runs occurring more or less frequently than the period indicated. NORMAL MONTH TOTAL-A summation of the total time required for all computer runs occurring monthly or more frequently. PEAK MONTH TOTAL-The normal month time plus the time required if all quarterly, semiannual and annual runs were to Occur during one month. SATELLITE COMPUTER AND AUXILIARY EQUIPl\1:ENT UTILIZATION REPORT Report Explanation This report reflects analytical information computed by SCERT on each program run in the entire system for each computer configuration evaluated. THE USE OF A COMPUTER TO EVALUATE COMPUTERS Column Explanation: 1 PROGRAM RUN NUMBER-An identification number assigned by the analyst. 2 SATELL1TE COMPUTER-A projection of the hours of utilization of the satellite computer to support each program run. In this example, off-line equipment was used for normal data conversion operations, consequently, no utilization is reflected for a satellite computer. 3 OFF-LINE EQUIPMENT UTILIZATION-During the SCERT simulation, a pre and post history is developed for each computer run. SCERT anaiyzes this history to determine the off-line requirements for converting source document data to a media acceptable by the configuration being evaluated. It also determines the post run requirements for such off-line functions as printing, punching, etc. This off-line utilization is reflected in terms of hours per average month by type of function for each run, such as tape to print and card to tape. 4 DATA PREPARATION-Machine time requirements for all necessary data preparation such as key punching and paper tape punching for each program run are computed and reflected in terms' of the hourly requirements per average month. In this exa'mple, source documents were not defined, therefore, data preparation time was not computed. PROGRAMMING REQUIREMENTS REPORT Report ExplarULtion This report reflects analytical information computed by SCERT on each program run in the entire system for each computer configuration evaluated. Column Explanation: 1 RUN NUMBER-An identification number assigned by the analyst. 2 3 4 5 393 PROGRAMMED BY USER-A projection of the number of program steps which must be programmed by the user's staff to prepare each program. SUB-ROUTINES-A quantitative reflection of the number of program steps which will be incorporated into each program run as a result of the utilization of manufacturer furnished su b-routines. UTILITY ROUTINES-A quantitative reflection of the number of program steps in the manufacturer furnished utility routines used in each run. PROGRAMMING EFFORT IN MAN MONTHS-A quantitative projection of the numb~r of man months required to write each program run. This projection, although it may be no closer than plus or minus 15 % of actual program effort required, because of the technique and the factors used in arriving at it, represents the best possible projection that can be achieved. The factors used for arriving at this projection are the number of programmers and their experience levels, the complexity of the computer run, and the language to be used. APPLICATION SUMMARY REPORT Report ExplarULtion This report reft.ects the summarization of information computed by SCERT at the program run level for each computer configuration evaluated. Column Explanation: 1 APPLICATION CODE-The first character of program run number will be assigned by the analyst to identify each application in the system. PROGRAM TIME~A projection of 2 the CPU utilization hours for each application for an average month. SET-UP TIME-Reflects net set-up 3 and take-down time required. 4 SATELLITE COMPUTER UTILIZATION-A projection of the utilization 394 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 5 6 hours in support of the central processor for each application for an average month. PROGRAMMING EFFORT IN MAN MONTHS-A summarization of programming manpower effort required for implementing all of the computer runs in each application. PROGRAMMING EFFORT DOLLARS COST-An estimation of programming costs, based on an extension of the number of man months of effort multiplied by the average monthly salary of a programmer, as furnished by the user through the Environment Definitions. 6 7 COST SUMMARY REPORT Repo'rt Explanation This report reflects the SCERT projections for all costs associated with the operation of the entire system for each computer configuration evaluated for an average month. Column Explanation,' 1 BASIC HOURS-Represents the number of hours in the rental option selected by SCERT. SCERT analyzes, in terms of the number of hours of component utilization, all options offered and then selects the optimum one. 2 EXTRA USE-The number of hours of extra use required beyond the basic hours provided in the option selected by SCERT. 3 MONTHLY RENTAL-Represents the total basic hours pI us extra use hours monthly rental for each component during an average month. 4 PURCHASE PRICE/XX-The purchase price of each component amortized over an XX month period. The period of amortization is selected by the user and defined to SCER T through the Environment Definitions. lV[ONTHLY MAINTENANCE-Represents the maintenance charges for each purchased component during an average month. 8 9 10 MONTHLY COST-A total of' the amortized purchase price plus the monthly maintenance cost. This is a figure which can be compared to the monthly rental to assist in determining purchase versus rental. DATA PREPARATION-A dollar figure representing the personnel costs required to prepare input data on a recurring basis to achieve the operation of the total system. This ·projection is based on factors furnished through the Environment Definitions, such as number of data preparation personnel and their mean salary, the user's estimate of their data preparation production, i.e., key strokes per hour; and the computations made by SCERT which reflect the total data preparation requirement for an average month. EQUIPMENT OPERATION-A dollar figure representing the projection of equipment operation costs for an average month. This estimate is computed on data defined to SCERT, such as numbers and mean salaries of operation personnel; and the computations made by SCERT to determine the hours required for processing the entire system on the simulated configuration. PROSRAM MAINTENANCE - An estimate of the user's personnel costs to maintain all computer runs in this system. DATA MEDIA COSTS-Represents the costs computed by SCERT for unexpendable data media necessary for the implementation of this system. Examples are magnetic tapes or any other type of interchangeable data storage media. DETAILED SYSTEMS ANALYSIS REPORT Report Explanation This report reflects detailed analytical information computed by SCERT on each program run in the entire system for each computer configuration evaluated. THE USE OF A COMPUTER TO EVALUATE COMPUTERS Column Explanation: 1 2 3 4 5 6 7 8 9 10 11 12 PART I, INPUT/OUTPUT ANALYSIS-This part of the report is a complete analysis of all I/O functions in the system. INPUT/OUTPUT FILES-A column showing file name. FILE NUMBER - An identification number assigned by the analyst. NUMBER OF RECORDS-The number of records for each of the files. RECORD SIZE-The optimized record size computed by SCERT for each file. RECORDS PER BLOCK-The optimum file block size computed by SCERT. NUMBER OF REELS-The number of reels required for each tape file. INPUT/OUTPUT MEDIA-A code specifying the type of media assigned to each I/O file. If the computer is a multi-channel machine, SCER T determines the channel assignment for each device and indicates it. BUFFERED TIME-An estimate of that I/O time which can be buffered (or overlapped) by other operations during each program run. UNBUFFERED TIME-A projection of that I/O time which cannot be overlapped and is therefore net time during each program run. PART II, INTERNAL COIVIPUTATION ANALYSIS-During the simulation of each program run, SCERT compiles by functional category the number of program steps, associated memory utilized by work .areas, and required time for all internal \.:omputation. FUNCTION-Based on the internal computer activity codes specified by the analyst, SCERT determines a distributional breakout for the four functions of Arithmetic Computation, I/O Control, Data Handling, and Decision and Control. 395 13 PROGRAM STEPS-The number of instructions to be programmed by the user and to be furnished by manufacturer's software. 14 MEMORY USED-A total of the memory utilized for a function, including requirements for program steps, work areas, tables and I/O areas. 15 INTERNAL TIME-Time required for the computer to perform a function. 16 PART III, SPECIAL TIME ANALYSIS-An open ended analytical technique incorporated into SCERT to highlight for the program analyst the time consumed by those functions which are meaningful to the design of the Isystem. 17 PROGRAM INSERTION TIME-A projection of the optimum time required to insert the program into memory based on available peripheral hardware. If SCERT determines that segmentation is required, the added time for inserting the various program segments is also computed and shown in this column. 18 PROGRAM DELAY TIME, MULTIMEDIA CHANGES-SCERT determines the number of reels required for tape files, and if during a program run an insufficient number of tape stations are available for each multi-reel file, it computes the time delay for rewind, take-down and set-up time for each reel. 19 END OF JOB REWIND TIME-The time required to rewind the longest reel. 20 SET-UP AND TAKE-DOWN TIME -The optimized time required to set up peripheral devices before each run and to take down completud work after the run. It is based on the number of operators and an analysis of the runs previous to and subsequent to each run. A GENERAL-PURPOSE TIME-SHARING SYSTEM Jules I. Schwartz, Edward G. Coffman, and Clark Weissman System Development Corporation Santa Monica, California INTRODUCTION by the system is the PDP-l (manufactured by Digital Equipment Corp.), which is the major input/output vehicle for the various remote devices. Since June 1963, a Time-Sharing System has been operational at the System Development Corporation in Santa Monica. This system was produced under the sponsorship of ARPA and has utilized ideas developed at both Massachusetts Institute of Technologya·4 and Bolt, Beranek, and Newman,l.l1 as well as some original techniques. Time-sharing, in this case, means the simultaneous access to a computer by a large number of independent (and/or related) users and programs. The system is also "general purpose," since there is essentially no restriction on the kind of program that it can accommodate. The system has been used for compiling and debugging programs, conducting research, performing calculations, conducting games, and executing on-line programs using both algebraic and list-processing languages. The remote input/output devices available to users include Teletypes, displays, and other computers. These devices can be run from within SDC, and from the outside, with the exception of displays, which can be operated only a short distance from the computer. It is expected that computers to be used at remote stations win eventuany inciude the CDC 160A, the DEC PDP-I, and the IBM 1410. (Currently only the 160A is being used, from an installation 400 miles distant from the Q-32.) Figure 1 is a description of the system's remote equipment configuration. This paper is divided into four major discussions. These are: (1) an outline of the capabilities provided for the user by the equipment and program system; (2) a description of the system's operation, with an analysis of the system scheduling techniques and properties; (3) a somewhat detailed description of two of the currently operating system service programs; and (4) a conclusion and summary. CAP ABILITIES FOR THE USER Equipment Configuration The major computer used by the TimeSharing System (TSS) Executive is the AN/ FSQ-32 (manufactured by IBM). Also used Figure 1. Remote Equipment Configuration. 397 398 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 The AN/FSQ-32 computer is a 1's-complement, 48-bit-word computer, with 65,536 words of high-speed (2.5 usec. cycle time minus overlap) memory available for programs, and an additional 16,384 words of high-speed memory available for data and input/output buffering; the latter memory is called input memory. The PDP-1 also has access to the input memory; thus, this memory serves as the interface between the two computers. In addition, the Q-32 has an extremely powerful instruction repertoire, including access to parts of words for loading, storing, and arithmetic; it also has an extensive interrupt system. Figure 2 shows the principal components of the system and the important information-flow paths throughout the system. As implied in the figure, each main memory bank (16K words) is individually and independently accessible by three control units: the central processor unit, the high-speed control unit, and the low-speed control unit. High-speed I/O, lowspeed I/O, and central processing can take place simultaneously out of different memory banks, or, with certain restrictions, out of the same memory bank. The high-speed and lowspeed I/O operations originate, of course, from Ilo Highspeed AUXiliary memory control unit ! Central process::.ng unit Figure 2. TSS Hardware System. PDP-l requests by the central processor unit. The low-speed control unit can service two or more· low-speed I/O devices simultaneously, while the high-speed devices can only be operated individually, mainly because their cycle time ap-· proaches that of core memory. A memory-protection mechanism and an interrupt, interval (quantum) clock (not shown in the figure) are also integral parts of the TSS computer system. On a bank-by-bank basis only, the memory protection mechanism provides the capability for inhibiting, under program control, the writing of information into one or more memory banks. The quantum clock has the following characteristics: 1. It can be set under program control to a time interval (quantum) anywhere in the range from a few msec. to 400 msec. 2. It can be made to interrupt computer operations after the set interval has elapsed, or after any power-of-two multiple of the set interval (up to a multiple of eight) has elapsed. 3. Under program control, it can be activated and reset. A summary of the pertinent device characteristics is given in Table I below. The disk file shown in Figure 2 is currently being incorporated into the system. The Time-Sharing System as it Looks to the User The time-sharing user today communicates with the Time-Sharing System primarily by means of Teletype. He has at his disposal six basic commands to the system. Briefly, these commands are: • LOGIN: The user is beginning a run. With this command he gives his identification and a "job number." • LOAD: The user requests a program to be loaded (currently from tape, eventually, from disk). Once this command is executed, the program is an "object program" in the system. • GO: The user starts the operation of an object program or restarts the operation of an object program that has been stopped. Once the user gives this com- A GENERAL-PURPOSE TIME-SHARING SYSTEM 399 Table I. Characteristics of the AN /FSQ-32 Storage Devices AVERAGE ACCESS TIME DEVICE SIZE WORD RATE Core Memory 65K 2.5,.,.sec./wd. Inut/Output Core Memory 16K 2.5 p.sec./wd. Magnetic Drums 400K 2.75 p.sec. fwd. 10 msec. Disk File 4000K 11.75 p.sec./wd. 225 m·sec. Magnetic Tapes 16 Drives 128 ,.,.sec./wd. (High density) 5 to 30 msec. (no positioning), depending on whether the tape is at load point, and whether it is being read or written. mand, he can send Teletype messages to either his object program or the TimeSharing System. • STOP: The user stops the operation of an object program. • QUIT: The user has finished a particular job. Upon receipt of the QUIT, the TimeSharing System punches a card with certain accounting information on it and removes the object program from the system. • DIAL: The user may communicate with other users or the computer operators with this command. • Break point: When a specified point in the program is reached, notifies the user, and (on options) displays registers, and stops or continues the program. As many as five break points are allowed simultaneously. In addition to these basic commands, the user has available to him a variety of on-line program debugging, or checkout, functions which give immediate access to any .part(s) of an object program. Briefly, these debugging functions include: • Open: Displays the contents of the given memory or machine register and uses this as a base address for other debugging commands. • Modify open register address: Changes the address of the opened register by the given increment or decrement. • Insert: Inserts the given value into the opened register. • Mask: Inserts values by the given mask. • Mode: Displays values according to specified mode (floating, decimal, octal, Hollerith) . • Dump: Dumps a given set of registers, either on Teletype or tape. The actual commands to perform these functions usually include a symbol or address with one or two unique Teletype characters. Additional Facilities Available to System Users The commands and devices mentioned so far are facilities available to users· or users' object programs directly through the Time-Sharing System's Executive. With these facilities one could run and debug programs that exist in a binary form. To make the system more useful, however, a number of additional devices (called service routines) are available to users. These are themselves run as object programs, so it is clear that there is no limit to the number of service routines that can eventually be made available. These service routines include programs to file and update symbolic information; compilers; a fancy desk calculator; tape-handling routines; and a number of others including some advanced routines utilizing interpretive techniques. A more detailed description of these interpretive routines appears in the time-sharing applications section, below. 400 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 SYSTEM OPERATION AND TIME AND SPACE ALLOCATION System Operation The discussion so far has been primarily on the operation of the system from the user's point of view. The following is an over-all description of the system and how it operates. Basically, the system operates as follows: All object programs are stored on drum, put there as a result of the LOAD command. When a program's time to operate arrives, or, preferably, ahead of this time, it is brought into high-speed memory. If bringing a program into its area in memory causes a stolage conflict with another program, the latter must be restored to its place on drums (a process called s.wapping) . A prograni"s turn will end when it initiates an input or output request, when a machine or program error is detected, or when its time is up, the time allotted being determined prior to its turn. At the completion of its turn, its machine environment (e.g., accu. mulator, index-registers, etc.) is saved, and it either resides in memory until its next turn or is written on drums. This mechanism is controlled by the time-sharing Executive. As stated before, there is no restriction on the type of object program that can run in the system. Therefore, as much input/output equipment as possible is made available to object programs; thus, object programs may use tapes, displays, and Teletypes for input and output. Other computers can also be treated as input/output devices; further, disk storave is available to object programs. Since it is impractical, in such a system, to have specific Teletypes or tapes referred to by object programs, input/output is done in a general fashion, with all input/output devices given arbitrary names by the object programs and declared to be files used by the object program during its run. Thus, only the Time-Sharing System knows what physical tape drivers, Teletypes, or areas of drums are being used. The Time-Sharing System's Q-32 Executive occupies 16,384 words of memory, leaving the remainder of memory for object programs. The Executive that exists in the PDP-1 is primarily concerned with maintaining the flow of infor- mati on to and from the remote devices. It does relatively little decision-making. However, it does determine the kind of input/output device concerned, the type of conversion necessary (if any), and the particular channel of the device with which it is communicating. The time-sharing Executive in the Q-32 has eight major components. These include routines that perform input/ output, perform on-line debugging, interpret commands, assign storage, and schedule object programs. By far the most distinctive feature of the time-sharing Executive, compared to other monitors or executive systems, is the scheduler. Accordingly, a more detailed description of time and space scheduling follows. Time Allocation and User Capacity The first problem considered in the TimeSharing System (TSS) scheduling design was the determination of the minimum amount of time to be given each program during a response cycle of the system. A response cycle is that period of time during which all active programs (i.e., programs requiring central processing time) are serviced. Clearly, to satisfy TSS objectives, this quantum of time (q) must be at least as great as the average amount of time required by an object program to produce a response. Here, of course, we refer only to those programs designed to communicate with a user station (display or keyboard device), and to those programs for which a fast response is desired and can reasonably be expected. In other words, a user requesting a matrix inversion will (and must) expect to wait considerably longer than a user wishing only to see the contents of some register in his program. Initially, it was obviously not possible to determine a priori the distribution of objectprogram operating times, nor was it even possible to define or classify the group of users requiring these data. The currently available information regarding user programs, and, to some extent, the experience of others, indicated that a q of 50 msec. was sufficient. The extensive recording now being performed during TSS operation is accumulating data that will much more accurately indicate the necessary q size. A GENERAL-PURPOSE TIME-SHARING SYSTEM In the following section, "worst-case" situations are being treated. "Worst-case" situations are being treated because they, by definition, give the overload threshold or capacity of the system; because they simplify the problem of having to cope with the distributions of object-program sizes and operating times; and because TSS will be operating at, or near, capacity for a high percentage of· the time, if the present rate of usage continues. In some cases the "worst-case" values that are used had to be estimated. There is considerable evidence, however, to support the estimates given in the following approximation of the maximum number (nma:J of active users that can be serviced in one response cycle, when given the size of the response interval (tr), the quantum size, and the hardware constraints. In the current version of TSS the "worstcase" response cycle consists of the following recurrent, non-overlapping sequence of operations: dumping of the last program operated; loading of the next program to operate; allocation of the time interval for operation. For the values of q and tr that are of interest, the number of active programs in the system can be much larger than the number of menlory-protected programs that can be held in core memory at one time; therefore, the above sequence will virtually always be necessary for the operation of each program. Assuming (as is presently the case) that object programs are not relocatable, we have (in view of the regular, cyclic operation of TSS) the following simple relation, tr ( 1 - 7]) 2t,. + q (1) where til represents an average value for the time it takes to transfer a program from drum storage to core memory or vice versa, and 1] is the fraction of time ( overhead) used by the Executive during each response cycle. The fraction of overhead (1]) is a difficult quantity to evaluate, and it depends to some extent on n lll :IX ' Because of the complexity of TSS operation, it is also diffi"cult to estimate 1] through recording during TSS operation. From experience to date with the system, it is estimated that 7r ranges from two per cent to 401 fifteen or twenty per cent depending on existing circumstances. Equation (1) shows that, without major revisions in hardware, a significant inprovement in nmn can be achieved only through a decrease in the quantity (2ts + q). In particular, if object programs can be made dynamically relocatable, this quantity can be reduced to the value of 2ts alone. Clearly, this is the best one can do, simply because the speed of the highspeed I/O sectio~ in swapping programs in an uninterrupted sequence represents a fundamental iIpper bound on TSS capacity. Further improvement necessitates an extensive increase in core-memory size, so that at least some active programs can remain in memory during consecutive response cycles. An increase in nmn brought about by an increase in the speed of the high-speed I/O section is not economically feasible as can be seen from the equipment description given earlier. Assuming dynamic relocatability of programs, equation (1) changes to : n llUlX L (1 - 7]) tr/2t.. (2) In practice, the extent to which the optimum is attained depends on the distributions of object-program sizes and operating times. If 2t,. is substantially larger than q, Equation (2) can, for all practical purposes, be considered an equality. Relocatability at load time would, of course, also significantly increase n max , but the improvement that could be expected would be substantially less than that given in Equation (2). For a more specific evaluation of the improvement, a knowledge of the distributions just mentioned is necessary. The linear relationships between nmn and tr given by Equations (1) and (2) are shown graphically in Figure 3 with the following parameter values: q = 50 msec., 1] = 0.20, and 20 15 10 Figure 3. Response Time vs. Number of Users. 402 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 2ts = 100 msec. (corresponding to a program size of 16K words). At present, a value of 2.0 sec. is being used for tr, with a resulting n max of about 11. Up to now, the efficiency of central processor utilization (relative to un overlapped I/O time) has been considered of secondary importance, providing that user requirements have been met. Admittedly, this computational efficiency3 is rather low in the "worst-case" situations. As will be seen in the next section, however, the way in which object programs are sequenced tends to maximize this efficiency for any given load situation. Clearly, the installation of dynamic relocatability in the system would allow an efficiency up to 50 % since q can be made equal to 2t, without affecting Equation (2). It should be emphasized that nmn does not represent the maximum number (Nmax ) of user stations that can be active at one time; it represents only the maximum number of user programs that can be serviced in a fixed response interval under the assumptions given earlier. It has been conservatively estimated that the associated object program is in need of central processor time only ten to twenty per cent of the time during which a user station is in use. Accordingly, it may be possible to make N max considerably larger than n max without significantly jeopardizing user-response requirements. Three important factors figure in the estimate of p = nmax/Nmax: 1. Relative to computer processing speeds, many applications (e.g., debugging, gaming) consume considerable user time in thinking and output analysis. 2. The average user is less than professional in his use of input devices. A slow manual-input rate, coupled with occasional typing or format errors, will certainly tend to make p small. 3. Generally, computer output to user stations takes as much as one to ten seconds. The estimate of p given above was based on the observation of these three factors during system operation and has been justified by the results of the limited amount of recording currently available. In obtaining the precise distribution of the quantity p it will be possible to determine the probability of overload for a given N max, or to determine the N max necessary for a given probability of overload. It should also be pointed out that, ultimately in TSS, as in a telephone exchange, several more user stations may be allowed than can actually be in use at one time. The extent to which Nmax can be exceeded must again be determined by a distribution obtained in the same manner as for p. Sequencing and Priorities The sequence in which object programs are allocated time is determined by a priority scheme that favors the smaller programs that do not use low-speed I/O time. The amount of time allocated is given by the total time available (tr ), divided by the current number (n) of object programs requesting central processing time. When n= n max , the time allocated is given by the minimum quantum discussed in the previous paragraphs. The priority scheme was adopted to prevent low-speed I/O that was initiated by object programs from degrading the response of those users not using low-speed I/O. Users whose programs require low-speed I/O must expect poorer response, not only because of the lowspeed operations, but also because of possible conflicts in object-program I/O requests. Each object program in the system receives a priority according to the criteria in Table II. Table II. Priority Criteria PRIORITY 1 2 PROGRAM CHARACTERISTICS i I Program is less than 16K and does not use low-speed I/O. Program is less than 32K and uses ~ow-speed I/O or, program is be ~ween 16K and 32K and no low- speed I/O. S ;Program in excess of 32K. During any given interval of time, Priority 1 programs will receive service first; Priority 2, second; and Priority 3, last. To prevent degradation of response by low-speed I/O, main A GENERAL-PURPOSE TIME-SHARING SYSTEM 403 Memory Bank: 1 4 2 Lr~1 o 50 "....---161< progr8III8 45 B Priority 2 5. 32K Priori ty 3 }!!'iori ty 1 ~ 32K . _ - - 3 2 K programs 46 f 16K (N III&X ) 35 . 30 Figure 4. Main Memory Allocation. 15 memory is partitioned and allocated as shown in Figure 4. Because of the relatively small number of current TSS users, this storage allocation procedure has not yet been imposed on object programs. In the future when the number of Priority 2 and 3 users begins to cause a significant degradation in Priority 1 response, this scheme will be fully implemented as described. Figure 4 shows that Priority 1 and 2 programs can be multiplexed, but Priority 3 programs preempt practically the entire machine. The priority scheme cannot solve the problem arising when a Priority 3 program undertakes a lengthy, low-speed I/O transfer. The majority of programs using low-speed I/O, however, concern tape transfers, which involve no searching, that take from 50 to 75 msec. When a program completes operation prior to the expiration of its time allocation for any of the reasons given in the second paragraph of this section, the remaining time will be redistributed among the remaining users requesting service. As a result, the large Priority 2 and 3 users will generally receive more time than the Priority 1 users, thus increasing the potential utilization of central-processor time. Space AUocation Although the timing and speed limitations on TSS capability have been of concern, storage limitations are presently far more severe. Storage limitations can be largely removed, however, by acquisition of additional drum space up to the maximum of about 600K. Figure 5 gives a rough idea of how much drum storage must be provided for object programs, to achieve a balance between the speed and capacity of the system. The curves are obtained by letting n llWX == pNIlI:lx in Eq. (1), tr == 2.0 sec., 'YJ == 0.2, and p == 0.2. -=::::===:..-__ 10 _~Curre~n~t~drum~c~apac~ity~_ _ for 32K programs 51~~_'--r~_r-'-~~__r-~______~. 5'0 1bo 1~0 ~ ~o 3~0 Jx, 4~0 3bo 5bo (q msec) 5 N max · F 19ure. = .1::!L . p 2t tr • +q In the initial TSS model (with only eight TTY users), auxiliary memory drum storage was partitioned and allocated in a fixed manner to provide an early working model of the system. This technique proved quite satisfactory at the time, but the number of input stations has now increased to about 48. To accommodate the additional users, a more efficient use of drum storage was necessary. The present method meets this requirement by allocating storage in a contiguous, "head-to-tail" fashion. The adapatability of this storage-allocation method requires searching an inventory of available drum space each time a new program enters the system, and periodically redistributing drum space to maximize the available amount of contiguous drum space. A possible disadvantage of this method is the additional overhead produced, especially when programs must be reshuffled to allocate a sufficient amount of contiguous drum space for a new program. Here again, the performance of this storage allocation technique must be evaluated by statistical recording, since the performance depends strongly on the distribution of program sizes, and on the rates at which programs enter and leave the system. However, at present (and in the foreseeable future) the above rates are so low that the additional overhead produced is negligible. Future Improvements in TSS Scheduling There are many ways, including both hardware and software additions to the system, in which the capacity and scheduling efficiency of 404 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 TSS can be enhanced. The more or less obvious hardware improvements include: • • • • Additional core memory Additional drum memory Relocation mechanism Disk storage The effects of a relocation mechanism and additional drum storage have been described in the previous sections. Additional main memory can be expected to allow for a larger Executive system, larger object programs, and greater scheduling efficiency. However, a substantial improvement in scheduling efficiency must be predicated on the existence of a relocation mechanism, when one makes the obvious assumption that the memory size is small compared to the total size of all active object programs. The disk file, which is just now being installed and checked out, will supplant tapes for all those applications in which disks are faster and use less machine time. It is expected that disks will be used to store a program library for TSS usage and to store large data bases for object program usage. It is conceivable that disks will also be used for program swapping. The first use of the disk file promises to eliminate a high percentage of manual operations associated with program loading from tapes, and thus to reduce greatly the corresponding delay experienced by users. The second use should save considerable time for the user whose application involves searching through large data bases. Although the estimates given in this paper are based realistically on current experience, it is not unlikely that user characteristics will evolve quite differently than predicted. Program sizes and/or operating times may grow to a point that invalidates the "worst-case" figures given in this section. It is possible, howeve'r, to counteract a certain amount of this degradation by certain improvements in scheduling logic. One improvement would be obtained by taking advantage of the fact that a fairly large class of users exists for whom responses substantially greater than one or hvo seconds are quite acceptable. In short, it is possihle to assign response levels to each user and to service each user j~st frequently enough to ensure his level of response. Furthermore, the disk file can be used for swapping those pro':' grams for which short responses are not necessary. Provided that disk access is in parallel with other high-speed I/O activities, the effective swapping speed can retain the same order of magnitude as for drums. There are many programs that do not alter themselves during their execution. Thus, as another software improvement, these programs could be treated by the system in two sections: an instruction section and an environment (data) section. During a program's execution it -would never be necessary to write the instruction section back on drums; only the environment se~tion and the machine conditions at interrupt would be written back on drums. These and other improvements to TSS are under present investigation. Of principal concern in the investigation of these system changes is the amount of overhead they produce. In some cases the increase in overhead exceeds the expected "improvement" in operating speed and efficiency. TIME-SHARING APPLICATIONS To illustrate the "general purpose" nature of the Time-Sharing System, we focus on two interesting programming systems currently operating on TSS as service systems for the user. The first, IPL-TS, is a complete listprocessing system for the Information Processing Language V developed by Newell, Simon, and Shaw.I2 The second, TINT, is an on-line Teletype INTerpreter for theJOVIALalgebraic language developed by SDC.I6.17 When the Time-Sharing System is equipped with these two programming-language systems, the user is immediately provided with a familiar programming system to ease his transition to programming for time-sharing, and allowed to use, with little or no modification, any code he may have previously written in IPL-V or JOVIAL for other machine systems. IPL-TS Description IPL-TS executes interpretively IPL-V code written in accordance with the latest published IPL-V conventions. I3 Some exceptions are noted, particularly in the IPL-TS I/O conven- A GENERAL-PURPOSE TIME-SHARING SYSTEM tions dictated by machine limitations and timesharing procedures. More significant, however, are the extensions provided by IPL-TS in the areas of mode of code execution, and improved on-line communication. IPL-TS can operate in one of two modes at the programmer's option: the "production" mode or the "debugging" mode. The production mode is designed for maximum code execution, and is used essentially for checked out code. Though code is still executed interpretively, J1 suppression of all debugging functions in the production mode has produced a four-fold increase in execution rate over debugging-mode operation. Execution rates of over 400,000 cycles per minute, which compare favorably with other non-time-sharing IPL systems, are common. To the IPL-TS user, production-mode operation is analogous, as we shall see later, to the TINT user compiling his checked-out code with the Time-Sharing JOVIAL Compiler (JTS). Debugging-mode operation, on the other hand, is designed for maximum user efficiency and greater on-line programmer control over the execution of his program. The debugging mode a!!o\vs all the standard IPL options; it also permits a number of on-line functions not common to IPL systems. These include: 1. Optional breakpoint action at any moni- tor point, whereby the currently executing program is suspended until completion of the execution of anyon-line, programmer-specified routine; 2. On-line, symbolic program composition and/or debugging; 3. Optional automatic or on-line programmer-controlled execution of a full "back trace" routine that prints up to the last 100 interpretation cycles. This routine is executed by IPL-TS automatically at each system-detected error occurrence as a debugging diagnostic; and 4. A flexible, "thin skinned" system er~or trap mechanism permitting programmer specification of trapping actions for all system-detected errors. TINT Description TINT is a two-pass interpretive program 405 system for time-sharing use, and operates upon a subset of the JOVIAL problem-oriented language. TINT includes a generator, a set of operator subroutines, and the interpreter. The generator was acquired from a current IBM 7090 JOVIAL compiler and was modified to handle the particular JOVIAL dialect used by TINT. The operator subroutines and the interpreter are original code developed specifically for TINT. The generator (first pass) scans the input JOVIAL statements and translates them into an intermediate Polish prefix language. Grammer checking is performed during the translation. The language subset allowed may include the arithmetic, relational, and Boolean operators; procedure calls; data table, array, and item (integer, floating point, and Hollerith) declarations; and the GOTO, IF, STOP, READ, and PRINT statements. The READ and PRINT statements were added to the language specifically for time-sharing operation. Operator subroutines comprise the primitive functions used by the interpreter to perform the actions specified In the intermediate language. The interpreter (second pass) scans the intermediate language for the current operator prefix and its arguments, and executes the corresponding operator subroutine that computes on these arguments. .. The TINT user is permitted a number of options in composing and executing his code: He may reference code stored in a binary library tape of his own composition; he may file away any current code on tape for subsequent use, Or for compilation with JTS after the code has been exercised and debugged; and, he may optionally execute code from a prestored tape or from the Teletype. On-Line Program Composition Both IPL-TS and TINT allow the user to write symbolic programs on-line and to execute them immediately, by themselves or in conjunction with previously coded routines. With IPLTS, the programmer uses the special system routine, Linear IPL (LIPL),* which accepts * LIPL was designed and coded by R. Dupchak while consJlltant to the RAND Corporation, Santa Monica. 406 PROCE.E'DING8-SPRING JOINT COMPUTER -CONFERENCE, 1964 L1PL tlF-AOY! ~T ACKTICST=(A"! AI A2 A3.ACKTf.ST) A"!=(JIS4 I"!$CYC.O$ JI~7 10$H3z$ JIS7 HI+I"! JI61 I"$SICC.O,," JIS7 1"$lIM~::$ JIS1 I"!+IM JI61 " ... ACK($ JIS7 1111411 JIST III~." JIS7 I!ONlo! JIS7 III"')=~ J157) LOGIN 0173 JDX.25 $OK LOG ON I_ LOAD TINT 1196 SWAIT "LOAD OK GO SMSG IN • A"! sns UP PHINT LINE' • AI .,,1) AI=(\IIH3 JI~" JSM SAVES H3 COpy IN W0. STA~T AND CLOr.K IN $311" "BEGINS NEW PROGIMM" N f' $ "NUMBER Of' CASF.S" SUHX f' $ "SUH Of' VALUES" XRA~ j; $ "AK(TH. MEAN" SDEV f' S "STD. DEVIATION" ~EAD N. SUMX S XBAR " SUMX/N $ PRINT 6H(MEAN a). J(BAR $ SDEV • «SUHX •• 2.0-N.XBAR •• 2.1lI)/CN-I » •• 0.5 S PRINT 8H(S. D. ,. ). SDEV $ TERM $ "CAUSES EXECUTION Of' P~OGKAM" ITEM ITEM ITEM ITEM • A2=( 1"14" JilT IIINI! 7M9-1 .JJ~5.JH) 9-1:(JII i 7119-2 I"!NII JI2S.9-3) IIIN" J i l l A2.9-3 5I!IKI 1111411 11'11411 J i l l 142 JI25.J'j) 9-~=(\"+1 • A2 COMPUTES AC(MW.N") • A3=(III+9 JI6W IWH3 IIWOI l\WII J i l l JI57 10+29 J1611 .$3 H!+49 JI6M I"NII JI51 JI5S IIWII J9.J3IIJ • A3 COMPUTF:S DF:LTA H3 AND oaTA Tim:. ALSO SETS VAtllABLES IN Pf 112 +READY Program Reference Statements 115 +READY DUMP (Figure 1.6b, line 143) produces an alphabetically ordered listing of program identifiers with their current values. Array subscripts are stepped automatically, contiguous zero-valued array elements are omitted, and empty elements (Le., those never receiving a wl!ue) are flagged. 117 ::: 110 112 114 120 115 ZPUIT PRINT 102 116 +READY Program reference statements (see Figure 1.4) allow the user to display various vital conditions of his program. These statements are not retained. They are acted upon immediately, then discarded. Through their use, a complete, dynamic record of both control flow and data usage may be obtained. gi~iN~I~: ZPU!T (51), TABLE(500) X : 0 Y = I. I STATEJIIENT NI!JT IN LANGUAGE. TYPaGRAPHICAL ERRI!JRS IIAY BE CI!JRRECTED IIIIIEDIATELY ~Y: S~STITUTING A C8RRECT STATEIIENT VIA KEYBIJARD. 103 +READY CV !:~:g~ cv I PRINT 103, X • Y F8RIIAT(2XF5.2.F8.5) ANY STATEIIENT 8R SEQUENCE .F STATEIIENTS IIAY BE ri:iF5ED BY IIIIIEDIATE EXECUTIIIIN AFTER ENTRY. =ItOI • :8102 X Y :ERIHIR TRANSFER paINT N DBES N.T EXIST +READY START I =8103 O. 1.00000 :~~;~~ END I!J~ ;"f~!~ ENCI!JUNTERED DURING EXECUTII!JN m 122 123 124 125 126 127 128 129 +READY +READY +READY +READY +READY +READY +READY +READY X : X + DELX DEL Y = X. Y*DELX Y : Y + DEll 2 TABLE( 1> : X TABLE( 1+1 > : Y IFex - 1.>1,1,3 3 DB " J : I, I, 2 X : TABLE (J) :~ !~~~~ ARIT~ri+«TABLE(J+I>-TA8U;(2)}/( TABLE( 1+1>- rABLE(2».50 ) IIIXED III!J~~ DEC8MPl!JSITII!JN ERRI!JReS) • !:~:g~ g~ STATEMENTS IN ERRI!JR AT TIllE I!JF ENTRY ARE NeT ACCEPTED. 130 +ERRIlR m m!:~:g~ m!:~:g~ 135 +READY 136 +READY 139 +READY SUBSTITUTII!JN PlAY BE IIADE VITHI!JUT RE-ENTERING PRBGRAII ~~t0~~gA~L~~~;1 >-TABLE(2) )/( TABLE( 1+1 )-T~8LE(2) ).50.; PRINT 104, X , ZPLST FilRIIATCF5.2,51 AI> 104 " ~~~~T~~) = ZPLI!JT( K+I ) END Figure 1.6a. Sample Program: Creation and TEsting. REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM 140 110 112 115 115 115 115 115 115 135 155 135 135 155 135 135 137 141 START 0 +R£ADY =IlOI • X Y =111102 O. 1.00000 =0103 =111103 0.20 1.04000 =111103 0.40 1.12520 =111103 0.60 1.25798 0.80 1.45926 =111103 =0103 1.00 1.75111 O• • =1104 0.20 • =1104 =1104 0.40 0.60 ·=111104 0.80 =11104 =11104 1.00 =1,.04 1.20 • =ERR8R VALUE 8; SUBSCRIPT IS Z£R0, NEGATIV£, 0R EXCEEDS DIII£NSI0N DUIIP +READY CHAR=-O. 14191581 £-08 Dnx= 0.20000000£-00 DELY= 0.42026726£-00 142 +READY 145 +READY ERR8R £DIT(rs.5) DUIIP ILLEGAL CHARACTER IN TEXT DUIIP CHAR=-O.OOOOO DELX= 0.20000 on Y= 0.42027 1= 15 J= 15 K= 51 x= 1.20000 Y= 2.17158 TABLECI)= O. TABL£(2)= 1.00000 TABLE(5)= 0.20000 TABLE(4)= 1.04000 TABLE(5)= 0.40000 TABLE(6)= 1.12520 TABLE(?)= 0.60000 TABLE(8)= 1.25798 TABLE(9)= 0.80000 TABLEClO)= 1.45926 TABL£C1 1>= 1.00000 TABLECl2)= 1.75111 TABLE(15)= 1.20000 TABLE(14)= 2.17158 TABLE(500)= O. ZPL0TC I )=-6.09524 ZPL0T(2)- 6.69524 ZPL0T(5)=-6.09524 i~:i~;;~:~:g;;~! DUIIP ALWAYS IIAY BE INTERRUPTED. 144 145 146 110 112 114 127 115 127 115 127 115 127 115 127 115 127 115 127 133 135 133 135 133 135 133 135 133 135 133 135 133 135 137 419 +READT TRAC£ K +READT TRAP 101./158. +READY START 0 = IlOI • X Y =1102 =TRAP TRANSFER till 2 (125 ) =TRAP TRANSFER TI I (115 ) =1103 O. 1.00000 =TRAP TRANSFER Til 1 (115 ) =0103 0.20 1.04000 =TRAP TRANSFER T8 1 (115 ) =1103 0.40 1.12320 =TRAP TRANSFER TI I (115 ) =1103 0.60 1.25798 =TRAP TRANSFER Til I (115 ) =1103 0.80 1.45926 =TRAP TRANSFER till 1 (115 ) =1iII03 1.00 1.75111 =TRAP TRANSFER Till 3 (128 ) =TRACE K= 1 =0104 O. • =TRACE K= =0104 0.20. =TRACE K= =11104 0.40 =TRACE K= 12 =0104 0.60 =TRAC£ K= 20 =0104 0.80 =TRACE K= 53 =0104 1.00 =TRAC£ . K= 51 =0104 1.20 • =ERRflR VALUE 0; SUBSCRIPT IS ZER0, NEGATIVE, IR EXCEEDS DIIIENSIIilN • Figure 1.6c. Sample Program: Creation and Testing (continued) . 147 +READY ALTER 157./137. 157 +ALTER ZPUT( K) = BLANK 1371+ALT£R ALTER 110. IIOI+ALTER BLANK = ZPL0T( I ) 1I02+ALTER ALTER. 1102+ERR0R D0 128.0 REFERENCES UNDEFINED LABEL 148 +READY ALTER 137./157. 157 +ALTER 4 ZPL"TC K) = BLANK 1571+ALTER ALTER. 149 +READY TRACE* K 150 +READY TRAP* 101.1158. 151 +R£ADY START 0 110 =I101 • 112 =0102 x Y 115 =0103 O. , 1.00000 115 =0105 0.20 1.04000 115 =0103 0.401.12320 Q.60 i. 25'198 0.80 1.45926 115 =0105 1.001.75111 155 =0104 O. 135 =0104 0.20 135 =0104 0.40 155 =0104 0.60 155 =0104 "b.80 135 =0104 1.00 1.20 155 ="104 158 =577 152 +READY i~; ~:i~ * * Figure 1.6d. Sample Program: Creation and Testing (continued). Figure 1.6b. Sample Program: Creation and Te.sting (continued) . CHECK (Figure 1.6f, line 232) is an abbreviated INDEX in that only erroneous and suspicious items are displayed (i.e., only those INDEX lines marked with an asterisk). A UDIT generates cross-reference information based on the execution of the program, showing which sections were never executed and which variables were never set, or set but never used. This concise, post-mortem summary of incomplete control flow and data usage is a powerful aid in ensuring the thoroughness of program debugging. 201 = 20~ = 203 : 204 : 205 : 20~ : 207 = 208 = 209 : 210 : 211 = 212 : 215 = 214 = 215 : 216 : 217 = 218 = 219 = 220 : 221 = 222 = 223 = 224 : 225 : 226 = 227 : 228 = 229 = 250 = NUIIBER 201. PR2IGRAfil SAfilPLE DIMENSI"N ZPUT(51), TABLE(500) DELX:.2 X:O Y:I. 1=1 READ 101, CHAR, ZPL0T BLANK: ZPL"Te I ) 101 F0RIIATC52AJ> PRINT 102 102 F0RfIIATC5XI HX5XI HY> G0 T" 2 I PRINT 103,X, Y 103 F"RfIIATe2XF5.2,F8.5) 1= 1+2 X:X+DELX DELY:X*Y*DELX Y:Y+DELY 2 TABLEe 1>=X TABLECI+I>:Y IFeX-I. )1, 1,3 5 00 4 J:I,I,2 X:TABLECJ) CF ~~0;~~f~~~~~+1 )-TABL£<2»/( TABlE( 1+1 )-TABLEe2)>*50. PRINT 104,X,ZPL0T 104 F0RfIIATc F5. 2, 51 At> 4 ZPL"TC K>=BLANK SnlP 17 END Figure 1.6e. Sample Program: Creation and Testing (continued) . 420 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 231 +READY INDEX 1 2 3 4 5 101 102 103 104 BLANK CHAR DEll DELY I J K >!oSAI'IPLE TABLE X ZPl0T 232 +READY CHECK 5 ~33 = **SAI'IPLE +213. +219. +222. +228. +207. +209. +211. +214. +227. +208. +207. +203. +217. +206. -222. +222. +224. 201. 202. +204. -219. +205. -220. 202. +228. 3. As in most compilers, the sequence of translated code for arithmetic expressions may differ from that produced by other compilers and slight discrepancies due to variations in truncations JIlay occur. -221. -212. -221. -222. -207. -210. -213. -226. -228. -225. -216. -218. +215. -224. -223. -225. -217. -215. -219. -220. -224. -228. +219. -213. -221. -213. +220. +216. +223. -217. -223. -216. -226. +218. -224. -217. +207. -208. +225. -226. -218. +207. 201. +READY Figure 1.6f. Sample Program: Creation and Testing (continued) . Built-in Subroutines Since only program statements are retained, many excellent testing and debugging commands would not be available under program control, but would require the presence of the user at the moment of execution. To overcome this limitation, most of these statements have been designated as "built-in subroutines," a concept completely analogous to FORTRAN built-in functions. These statements, without change in their form, may be made the operand of a subroutine CALL statement. In this way, all the console testing and debugging features which may be of value are also available under program control. COMP ATIBILITY Studious regard has been paid to maintaining consistency with other FORTRAN compilers. Programs written in the system language are acceptable without change to conventional FORTRAN IV processors. FORTRAN IV programs are acceptable to the experimental system with the following limitations: 1. The program must be written with statements from the system subset. 2. A restriction of all one-pass translators is that the source-deck ordering must have the declarative statements precede the imperative statements. Of course COMMENT and FORMAT statements may appear anywhere. 4. Some minor differences in the internal representation of program constants, caused by different conversion routines, may also create slight differences in numerical results. 5. Individual source programs are limited to approximately 400 statements. This limit may often be circumvented by segmenting oversized programs into smaller subprograms. 6. Other Factors: a. No arithmetic function statements; b. No logical, complex, or double-precision variables; c. Number of elements (i.e., constants, variables, arrays, and functions) must be less than 190; d. Only one continuation card; e. No magnetic tape I/O; f. Some minor restrictions on equated variables; g. ConstantsReals: 8 digits, with magnitude within range 10-32 to 1032 or with zero magnitude, Integers: 10 digits; h. Array names must appear in a DIMENSION statement prior to any other appearance; i. Maximum I/O record size is 133 characters; j. Array names used as arguments must be declared in COMMON. EXAMPLES A program exhibiting many of the features available in this system is depicted in Figures 1.5 and 1.6. Figure 1.5a shows the final, correct version of the program. Figure 1.5b shows the correct output produced as a result of execution (see START statement, Figure 1.5a, line 128). REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM 101 101 101 102 103 104 105 106 1"7 108 109 110 111 112 113 114 II~ 116 117 118 119 120 121 122 123 124 I~ 126 127 128 CIlIIIIAND THIS IS A SAIIPLE PRIlGRAII. -READY C -READY C . PReGRAII SAIIPLE -READY DIIIENSIIlN ZPLIIT (~2), TABLE (~OO) -IREADY -!READY X =0 Y : I. -IREADY I : I -IREADY READ 101, DELX ,cHAR ,lPLilT -IREADY FIlRIIAT (f'7.4,~3A I ) -IREADY 101 PRINT 102 -IREADY 102 FIlRIIATC5I(IHX7XIHY) -IREADY 2 TABlE CJ) = X -IREADY TABLE (J +1) : Y -IREADY I PRINT 103, X, Y -tRUDY FIlRIIAT<2KF 7 .4". 8.~) -tRUDY 103 IFCX·I.)~,3,3 -tREADY 51 =1 +2 -tREADY X=X+DELX -tRUDY DELY =X.Y .DELX -IRUDY Y = Y + DELY -tRUDY GeTe 2 -tRUDY 3DI!4J=!.!.2 -IREADY X = TABLE(J) -IREADY K =I.+(CTABLE (J+1)-TABLE (2»)/CTABLE -1.0+10./50. 101 -ERRIIR 04117. PARENTHESES NeT IN BALANCE. 101 -READY HENRY =2.E -~50 •• a..1IG' (2 •• 50.110. )-1.+10./50. > 101 = HENRY= 0.1502585{£-06 101 -READY ReeTI=(-25.~ORTr(2' ••• 2-4 •• I •• 2.))/(2 •• t> 101 -ERReR ARITIIiIETIC DECIJIIPeSITleN ERRIIR (s) 101 -ERRIR IIIXED IlIlDE 101 -READY ReeTI=(-2'.~ORTr(25 ••• 2-4." •• 2.»/(2 •• I.) 101 = Rlln 1=-0.8025765.1£-01 101 -READY HENRY=2.E-~50 •• a..IG' (2 •• 50./10. )-1.0+10.150.) 101 = HENRY= 0.15025851E-06 101 -READY VAL= I.ICesr (50. HI.IIGr CABSr CSINf (50./2. )ICf'Sr (50./2.») 101 = VAI.=-0.9711499EE 00 101 -READY AREA=2 •• 10 •• 5.*5INr(J.1416/10.) 101 = AREA= 0.30901768£ 02 101 -READY ARC=2.*5QRTI" (4 ••• 2+1.3333*2 ••• 2) 101 = ARC= 0.92375753E 01 101 -READY ARC =2 •• (4 •• 4.+4 •• 2 •• 2./3. ) •• 0.5 101 = ARC= 0.92376041£ 01 101 -READY S=11..fGr«I.~INr(45.»/(I.-SINr(45.»> 101 -READY 101 = B= 0.12594177£ 01 101 -READY S =SIN' (45. > 101 = S= 0.8509035:2E 00 101 -READY G=0.5>11..I'IGr«I.+.7071>/0.-.7071» 101 = G= 0.88135995E 00 101 -READY E =20.*IITANr (20./4. )-4./2.*lIG' (4 ••• 2+20."2) 101 = E = 0.1540664.1£ 02 101 -READY 0=(2./(3.1416*10. » •• 0.5*5IN' (10.) 101 = 0=-0.13726357£-00 101 -READY 0=0.7978ISQRTrOO.)*5INr(10.) 101 = 0=-0.137249111[-00 101 -READY C 101 -READY the 7090 before you obtained correct resuIts?" (Number of runs were summed.) 4. "How many problems did you debug on the 7090?" 5a. "Approximately how many hours of training did you have on the terminal console ?" 5b. "How many debugging hours ?" 6. "How many problems did you debug on the terminal console?" Because this experiment was of limited scope, the experience reported must be taken cautiously. There are many variables which affect the usefulness and economy of this approach, and continuing field trials will yield more precise information. Figure 1.7. Examples of Command Mode Operation. SUMMARY USER A B C D E F G H I TOTALS SRI EXPERIMENT BACKGROUND SKILLS 7090 FORTRAN FORTRAN TYPING NO. RUIIS NO. PROGRAMS DEBUGGED HIGH HIGH HIGH LOW LOW NONE HIGH HIGH MEDIUM NONE NONE NONE NONE 5 NONE 12 6 NONE J NOllIE 10 NONE 4 3 40 MEDIUM MEDIUM LOW LOW LOW LOW NOllIE 5 '2 REMOTE COMPUTING NO. HOURS NO. PROGRAMS DEBUGGED TRNG D8UG 2 21'3 2 I 3 2 3 illS 2 3 3 6 4 3 6 4 2 I 2 2 2 11/2 fill! 2 5 2 2 3 16 16 28 26 3 I Figure 1.8. Results of SRI Tryout. The students were divided into two groups, I and II, and given the same set of problems to be solved in FORTRAN. Group I was told to do the odd-numbered problems on the IBM 7090 and the even-numbered ones on the remote-computing terminals. Group II reversed this polarity. The chart in Figure 1.8 shows the answers given by nine of the participants (the tenth failed to return his questionnaire) to the following questions: 1. "How much FORTRAN experience have you had?" (Answer was evaluated HI, LO, MED, NONE.) 2. "Have you had any typing experience?" (Answer was evaluated HI, LO, MED, NONE.) 3. "How many times did each problem go to The time-shared use of a computer provides a convenient, economical service to numerous remote users. This access is enhanced by use of conversational, source-language debugging techniques. Although the experimental system is oriented to the IBM 1050 terminal, the FORTRAN language, and scientific applications, the techniques described are useful with other terminal devices, programming languages, and application areas. Preliminary opt:rating experience indicates that systems such as the one described have considerable potential in enabling personnel less skilled in the programming art to rapidly obtain solutions to their problems. ACKNOWLEDGEMENTS Among the several people making significant contributions to the system, the authors wish to specifically acknowledge the work of Miss Geneva Butts, who implemented a major part of the source-language debugging features, and Mr. Murray Kizner, who implemented the translator routines for many Command features. REFERENCES 1. E. G. ANDREWS, "Telephone Switching and the Early Bell Lab. Computers," Bell System Technical Journal, March 1963. REMOTE COMPUTING-AN EXPERIM ENTAL SYSTEM 2. A RAND Symposium, "Economics of Remote Computing," Datamation, September 1961, October 1961, November 1961. 3. R. L. PATRICK, "So You Want To Go On Line," Datamation, October 1963. 4. D. B. BREEDON and P. A. ZAPHYR, "Pros and Cons of Remote Computing," Control Engineering, January 1963. 5. G. L. BALDWIN and N. E. SNOW, "Remote Operation of a Computer by a High Speed Data Link," Proc. F JGG, December 1962. 6. C. STRACHEY, "Time Sharing in Large, Fast Computers," Proceedings of the Intel'national Conference on Information Processing-UNESCO, June 1959. 7. J. MCCARTHY, "Time Sharing Computer Systems," Management and the Computer of the Future, Chapter 6, John Wiley & Sons, Inc., 1962. 8. F. J. CORBATO, "An Experimental Time Sharing System," Pl'oc. SJCC, May 1962. 9. F. J. CORBATO, et a!., "The Compatible Time Sharing System-A Programmer's Guide," The M.LT. Press, May 1963. 10. W. V. CROWLEY, "Why Stretch?" Proc. ACM National Conjel'ence, September 1962. 11. S. BOlLEN, E. FREDKIN, J. C. R. LICKLIDER, and J. MCCARTHY, "A Time Sharing Debugging System for a Small Computer," Pl'oc. SJCC, May 1963. 12. W. CLARK and J. C. R. LICKLIDER, "OnLine Man-Computer Communication," Proc. SJCC, l\1:ay 1962. 13. J. C. R. LICKLIDER, "Man-Computer Symbiosis," IRE Transactions on Human FactOl'S in Electl'onics, March 1960. 14. L. C. CLAPP and R. Y. KAIN, "A Com·puter Aid for Symbolic Mathematics," Proc. F JCC, November 1963. 15. H. TEAGER and J. MCCARTHY, "TimeShared Program Testing," Proc. ACM National Meeting, September 1959. 16. C. N. MOOERS, "The Reactive Typewriter," ACM Communications, January 1963. 17. G. J. CULLEN and R. W. HUFF, "S~lution of Non-Linear Integral Equations Using OnLine Computer Control," Proc. WJCC, April 1962. 423 18. T. MARILL, D. EDWARDS, and W. FEURZEIG, "DATA-DIAL: Two-Way Communication with Computers from Ordinary Dial Telephones," ACM Communications, October 1963. 19. R. HEAD, "The Programming Gap in Real Time Systems," Datamation, February 1963. 20. W. A. HOSIER, "Pitfalls and Safeguards in Real Time Systems," Datamation, April 1962 and May 1962. 21. T. A. HALDIMAN, "Management Techniques for Real Time Computer Programming," ACM Journal, July 1962. 22. W. FRANK, W. GARDNER, and G. STOCK, Programming On-Line Systems," Datamation, May 1963 and June 1963. 23. D. ISRAEL, "Simulation Techniques for the Test and Evalution of Real Time Compiler Programs," ACM Journal, 1963. 24. R. HEAD, "Real Time Programming Specifications," ACM Communications, July 1963. 25. FORTRAN General Information Manual, TRM .... ....., ........... 1?£\1'"n'"O ..&. l\Tll1'Y1ha ... JJ"""'.1. V".&. .L ......... , \,;I..I. ...... ~,)Q Q()'7A ~v-uv.-x. .L" 26. IBM 7040/7044 General Information Manual, IBM Form Number D22-6645. 27. IBM 1301 Disk Storage, IBM Form Number D22-6576. 28. IBM 7320 Drum Storage, IBM Form N umber G22-6717. 29. IBM 7740 Communications Control System, IBM Form Number A22-6753. 30. IBM 1050 Data Communications System, IBM Form Number A24-3020. 31. H. FERGUSON and E. BERNER, "Debugging Systems at the Source Language Level," ACM Communications, August 1963. 32. M. WILKERSON, "The JOVIAL Checker," Proc. W JCC, April 1961. 33. H. SCHWARZ, "An Introduction to ALGOL," ACM Communications, February 1963. 34. G. M. WEINBERG and G. L. GRESSETT, "An Experiment in Automatic Verification of Programs," AC,M' Communications, October 1963. REMOTE COMPUTING -AN EXPERIMENTAL SYSTEM Part 2: Internal Design J. M. Keller, E. C. Strum, and G. H. Yang Development Laboratory, Data System.s Division IBM Corporation New York, N. Y. INTRODUCTION 1. Immediate error diagnostics; This is the second of two papers dealing with the experimental remote-computing system. Part 1 described the system as viewed by a user who is unaware that he is jointly sharing the central computer with numerous other users. This paper (Part 2) describes the internal design of the system, with attention focused on those features which are of general interest and applicable to the design of other programming systems. 2. Program alteration without recompiling; 3. Extensive symbolic debugging aids; 4. Ready availability of the source version of the user program; 5. A user program that is : a. dynamically relocatable, h. easiiy interruptibie, and c. storage protected. This paper is introduced by a description of the over-all control structure and data organization .. Each of the principal sUbsystems is then described. The paper concludes with some remarks regarding possible extended applications. An appendix describes in some detail the algorithms used in the decomposition/recomposition of arithmetic expressions. SYSTEM ORGANIZATION Program.s The experimental remote-computing program is divided into three major system areas (Figure 2.1) : 1. The Scheduler, which is responsible for maintaining awareness of the total system status and for ordering and assigning tasks to the other system parts; 2. The Process Control system, consisting of the Translator, which reduces the user's input statements to an equivalent internal form (see below); the Interpreter, which executes this internal form; and the Process Control program, which regulates these two subsystems on the local level; 3. The I/O Control system, which is responsible for monitoring and operating all OBJECTIVES An operating system servicing numerous online users must meet certain design objectives that might be regarded as secondary or even unnecessary in conventional operating systems or compilers. But these objectives become paramount when the psychological and practical effects of sustained, immediate access to a computer are considered. Thus primary attention must be given to attaining: 425 426 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 T he Terminal Header .- -- - I SUPERVISOR ---- - - -- - - - - - - -- --, I I I I I I I I I L -----c=J --0 -.0 r----I I'"'""T""...JL..JI_-, A PROGRAM AREA I I I SCAN I I I +---EXCHANGE --~FLOW ~ @ PROGRAM AREA I I I I IL _ _ _ _ ~ - - - DATA FLOW Figure 2.1. General Block Diagram of System. I/O attachments, including the communications exchange. Data Organization At the system level, there are three principal data constructs: For each terminal in the system, there is a Terminal Header record (Figure 2.2) containing the following information: 1. Current status: a. operating mode, i.e., Command or Program (Rf. Part 1), b. terminal status, i.e., I/O wait, busy, or dormant, c. control information, i.e., should the system interrupt automatic status (execution) and return to manual status (statement entry) or continue automatic status, d. type of terminal component active, e. terminal ID, f. storage allocation block; 2. Header information for Command mode execution: a. formats for data output, b. system use records; 3. Temporary locations for random storage access: a. current user identification, b. name of current active program, c. location of active program for this terminal, d. name or location of subprogram called by current program. The M aste'I" Block CURRENT STATUS INFORMATION DA TA OUTPUT FORMATS ~--------------------- RECORD OF SYSTEM USE CURRENT USER I.D. r--------------------CURRENT ACTIVE PROGRAM r--------------------ACTIVE PROGRAM ADDRESS r--------------------CALLED SUBPROGRAM Figure 2.2. Terminal Header. For each statement in the language (Rf. Part 1), there is a Master Block record (Figure 2.3) containing the following information: 1. A statement type identifier; 2. A statement class identifier; 3. The symbolic, external statement identifier with associated control characters for recognizing the statement name on input and for recreating it on output; 4. Various indicators which denote intrinsic statement characteristics for checking purposes; 5. Addresses for transfers of control to the various major system routines, e.g., Translator, Interpreter; etc. REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM INDICATORS IC~S PROCESS CONTROL INFORMATION STATEMENT CODE CODE ELEMENT CONTROL TABLE CONTROL ----------- 427 CHARACTERS AND STA TEMENT CONTROL TABLE ----------- R-INDEX TABLE SYMBOLIC r - - - - - - - - - - STATEMENT - - - - - - - - - - N-INDEX TABLE IDENTIFIER I SCAN ADDRESS LIST ADDRESS I LINK ADDRESS INTERPRETER ADDRESS PARAMETER STACK TEMP STACK I PROGRAM LIST (STA TEMENT AND ELEMENT LISTS) I PROCESS CONTROL ADDRESS I PROCESS CONTROL ADDRESS - PROCESS CONTROL ADDRESS Figure 2.3. Master Block Record. Figure 2.4. User-Program Layout. The Master Block is used either as a dictionary, when information concerning the statement is needed, or as a switching center, when control flow within the system is dependent upon the statement type. A single record for both of these activities provides considerable flexibility in adding new statements, in modifying control conditions, and in making basic system modifications. The User Program Layout For the entire system there are two large, fixed areas (Figure 2.4) reserved for occupation of the various active user programs, Programs brought into these areas are relocated under program control; all I/O to and from these areas is overlapped. The duration of occupancy is determined either by overstepping a time limit or by the occurrence of one of several specific conditions (see following section on user-program organization). The layout of the user program is divided into two parts: 1. The statement and element records (see following section) which comprise the user program; and 2. The header. The header is further subdivided into two parts: 1. List control and other controls for the program records (see following section) ; and 2. Control words used by the Process Control program to keep track of program status. USER-PROGRAM ORGANIZATION The User Program The user's source-pTogram statements are mapped into equivalent internal records, which are classified and controlled by list structures. These records and their controlling elements constitute the user's program (see Figure 2.4). Every statement of the user's program is reduced to an individual statement record; and every element (name or label) is reduced to an individual element record. These records are inserted and chained on lists in the program area in the order of their appearance and creation. Control is maintained through tables of list-control words in the header portion of the user program. All addresses in the user program are relative to its base in order to facilitate relocation. 428 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Records Element Records An element in the source program is defined as a label, constant, variable, array, or function name. Every source element maps into a fixed-length, internal element record (see Figure 2.5) containing the following information: REFERENCE NUMBER mE I I MODE SIZE AL TER NUMBER STATEMENT CODE R(C) * SIZE NEXT ADDRESS ARRAY/COMMON/EQUIVALENCE ADDRESS INDICATORS .. NAME VALUE Figure 2.5. Element Record. 1. Reference number-a unique internal numeric identifier, assigned by the system. It is used for all internal referencing by the system. 2. Type-denotes the type of element, i.e., label, constant, variable, array, function. 3. Mode-denotes the mode, real or integer, of elements referring to numeric quantities. 4. Indicators-contains properties attributed to the element by declarative statements and/ or execution. These include storageallocation, and indications of element usage at object time. 5. Name-the external alphanumeric identifier. 6. Value-either the numeric value of the element or supplemental information for an array or function. 7. Next address-address of the next element record. 8. Array COMMON EQUIV ALENCEaddress of the value, if the element is in COMMON or is an array; or an offset address, if the element is equated to an array. Statement Records Every source statement maps into a variablelength, internal statement record (see Figure 2.6), which contains in coded form all informa- INDICATORS LABEL NEXT ADDRESS NEXT CLASS ADDRESS I R(A) R(B) PARAMETER OPERATOR R(D) FUNCTION OPERATOR R(SQRD I R(C) temp + temp temp - R(C) temp C = A*B + C/SQRT(D) Figure 2.6. Statement Record. tion present in the source statement. Each record begins with two standard words containing the foHowing information: 1. Alter number-a unique internal numeric 2. 3. 4. 5. 6. identifier assigned by the system. I t denotes the position of the statement relative to all others in the program; it is referenced by the user when modifying the program, manually requesting information, or starting execution. Statement code-identifier of the particular statement type. Indicators-reflects usage of the statement during execution. Label-refers to the associated external statement number, if any. Next address-address of the next statement record. N ext class address-address of the next statement record of the same type. The remainder of each statement record contains one or more words. Their number and content depend on the particular statement type. For example, an arithmetic-statement record contains the macro representation of REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM the translated expression, while a DO statement contains references to the indexing parameters. Lists The objective of providing for alteration of individual statements was the deciding factor in determining the internal record organization. The conventional table-oriented approach appeared much less attractive than the classification of records by lists. I, 2, 3, 4, 5, 6 Most compilers use tables to record information necessary for referencing and validating data usage and control flow. In this system, the same information is kept in the statement and element records. However, organizing these records on lists allows for increased flexibility in the compiling system. 7 • 8, 9, 10 For example, deletion and insertion of statements for program modification is easily provided. Time-consuming recompilations become completely unnecessary as a result of this altering provision. In addition, errors resulting from improper control flow and from invalid variable references can be diagnosed earlier in the compilation pl"OCeSS than is common with conventional compilers. Element Lists 429 records in source sequence (i.e., ordered by "alter number") ; 2. One of the class lists, consisting of all statements of a particular class (e.g., arithmetic, control, DO, I/O, allocation declarations, etc.). The entry list is used both to control execution sequence and to provide the proper ordering when the source program is reconstructed from the internal form. The class lists are extremely useful in performing checking operations (e.g., checking DO loops for proper nesting and control transfers). List Control Every list is controlled by a single control word pointing to the first and last records. For the statement lists there is a small table of control words for statement control (see Figure 2.4). Another similar table controls the element lists. In addition there is also a master table controlling the symbolic names of reserved system symbols: library functions (e.g., SIN, SORT, etc.) ; built-in functions (ABS, FLOAT, etc.) ; and system subroutines (DUMP, EXIT, etc.). Addressing Each element record is chained onto one of 26 element lists, each list consisting of all those element records whose symbolic names have the same initial letter. Element records within each list are ordered alphabetically by symbolic name. There are two additional list3 which link numeric elements as either integer or real constants. This set of element lists provides two significant features: Two tables exist for control of the element and label identifiers (see Figure 2.4). These are the R-index, or internal-identifier reference table, and the N -index, or numeric-label table. For every element that appears in a program, an entry for its internal identifier, R, is made in the R-index table; similarly, for every statement label, N, an entry is made in the N -index table. 1. The symbol look-up is more efficient since only the set of symbols with the same initial letter are considered; 2. Fully alphabetized symbolic cross-reference listings and memory dumps are easily provided. Every element or statement in the program can be accessed in an "asociative" manner by sequentially searching the lists until a match is found for the requested symbolic name or alter number. Each element or labeled statement can also be located in a "direct-look-at" manner l l by using the internal identifier for the element or label as an entry to the R- or Nindex table. Thus the flexibility of associative list searching and the efficiency of direct element fetching are both incorporated in the system. Statement Lists Each statement record is chained onto two lists: 1. The entry list consisting of all statement 430 PROOEE'DINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 FUNCTIONAL DESCRIPTION The Scheduler The purpose of any real-time, multiprogramming supervisory program is to synchronize, control, and monitor system operation. 12 , 13, 14 The program is responsible for determining what things are to be done, and by whom, to what, where, and when each is to be done. It has the duty of maximizing system throughput 'and ensuring reliable operation, and, in this case, of maintaining rapid and level response times at the terminal consoles. At the nucleus of this supervisory structure (see Fig.ure 2.1) is the Scheduler,15,16 which controls the: 1. Process Control program, which in turn directs the processor routines that translate and execute user programs; and 2. I/O Control program, which coordinates the communications exchange, random storage devices, tape units, reader, and online printer. The Scheduler performs continual sequential sampling of the subsidiary subsystems and maintains pertinent status data in the terminal headers. When data has been received from the terminal, the Scheduler examines the terminal header and decides whether to transmit a request to the random-storage I/O queue to fetch the user program (Program mode), or, if no program is required, save the data in a to-be-processed queue (Command mode). There are two kinds of termination: temporary, and return-to-manual. Temporary interruption frees the system for use by another terminal and may occur for the following reasons: 1. The allotted time interval has expired; 2. Input data is requested: 3. Output buffer is filled; 4. An external subprogram is invoked. The return to manual status occurs when: 1. .An error condition occurs; 2. A STOP or PAUSE statement is executed; 3. The end of the program is encountered; 4. The user requests an interrupt from the terminal. The Process Control Program The Process Control program (see Figure 2.7a) accepts information from the Scheduler and coordinates the activities of the processor programs. All of the appropriate Process Control routines and service routines must be initialized (1) to process the terminal header if the termnial is in the Command mode, or (2) to process various parts of the program header and list if in the Program mode. The Process Control program maintains an action code in In either event, the Scheduler passes to the Process Control program all information necessary for processing the input message--such as locations of the terminal and program headers, the location of the input message, and the operating mode of the terminal. Even if no message has been received for a given terminal, its active program will be fetched from random storage and the Process Control program entered, if the terminal header shows that the program is in the automatic state (i.e., in the process of execution). After each return from the Process Control program under this condition, the Scheduler must determine whether the automatic state should be terminated. Figure 2.7a. Genera! Diagram of Process Control Flow REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM each header to determine the next task to be performed upon the user program. The Process Control program may require the input statement just received to be scanned by the translator; it may require continuation of execution, of a DUMP, or of a LIST. The user program may be in the ALTER mode; it may be in the process of being tested for certain conditions which may prohibit further execution. Data for an input statement may be awaited or output of multiple-error messages may be in effect. When the user program is in the manual mode, the Process Control program has the responsibility of examining the process codes returned from the translator and of taking the necessary action. When the user program is in the automatic mode, execution may be temporarily halted and, in some cases, the program may be returned to manual-mode status (see above). When a subprogram "call" is made, execution halts until the next cycle for this terminal. At that time, the called subprogram becomes the user's active program and is brought into memory in place of the calling program. When a RETURN is effected or if an error occurs, the calling program is reactivated. 431 In order that the user may always be aware of the status of his program, condition codes are printed at the terminal whenever a change of status occurs. He is notified when input (a statement or data) is requested; when an error occurs; when and why execution was terminated; when a system statement (see Figure 2.7d) occurs (e.g., DUMP, INDEX, TRACE); and, optionally, when a subprogram call is made. When execution runs off the end of a program, and when a STOP or PAUSE is encountered, he is informed that his request for interruption of execution has been recognized. In short, the Process Control system always knows what is happening in the user program, and continually keeps the user informed of the status of his job. The objective is to provide the remote user with a more complete awareness of his program's status than is obtainable at a conventional computer console. The Translator Scan Routines The Translator (see Figure 2.7b) is responsible for transforming the source-language program to the internal form. 17 • 18, 19 (See Figures 2.5 and 2.6.) A preliminary scan is first used to identify arithmetic statements. For all other statements, the statement operator is collected and used to reference (via a Master Block routine) the corresponding master record. Control then passes to the translation routine for the particular statement type, e.g., GOTO, RETURN, PRINT, DIMENSION, etc., Every statement's decomposition goes through the same basic phases to form element and statement records. These involve the use of several service routines to collect the element name, find its record in a list or create a new record, and validate the statement and element usage. Figure 2.7b. General Diagram of Process Control Flow ~Translator. As each element in a statement is collected, a search is made to determine if it has previously appeared in the program. If the element has been previously used in the same statement, the record appears on a current element working list; otherwise, it may be found on the element list in the user's program or, alternately, on the master list of reserved and 432 PROCE,EDINGS----SPRING JOINT COMPUTER CONFERENCE, 1964 system names. If no record is found, then the element is new to the program, and a record is created for it and placed on the current working list. The information put into the record depends on the variable itself and the kind of statement it appears in. The mode indicators for an element depend either on its initial letter or on its appearance in an INTEGER or REAL declarative statement. The type and variable indicators depend on the statement type. A variable appearing in a storage-allocation statement is nagged according to its declaration as an array, common, or equated variable. If all source elements have previously appeared, no new element records result from the translation of a statement. However, a statement record must always be created. The Master Block record for the statement type provides the statement code. Statements that involve a list of variable, such as DIMENSION, EQUIV ALENCE, or COMMON, contain a count of the variables used followed by the Rnumber of the variables. Statements which involve a list of labels, such as: GO TO (5,6,7,8), I or IF (J-5) 12,3,12 contain an item count followed by the numeric labels. If the execution of a statement will change the value of a variable, the identifier of that variable is placed in a special field in the statement (see Figure 2.6). Statements containing arithmetic expressions or input/output lists involve specialized decomposition routines. The master Translator passes to both these routines essentially the same input: a string of words, each word containing either an operator or an R-number. The decomposition routine transforms these elements into an ordered set of arithmetic macros consisting of an operator and two operands in every word. These macros are then returned to the master Translator and added to the statement record (see Figure 2.6). The input/output list-decomposition routine also returns a macro set of executable operations, (A detailed description of the arithmetic decomposition is contained in Appendix I.) Scan Diagnostics Throughout the translation-scan phase, checking occurs for syntax and compositiontype errors. Illegal statement operators and invalid statement forms are detected early in the translation. Lack of a label on a FORMAT statement or the presence of a label on a declarative (where control may not flow) violate the definition of the statement type. Illegal uses of variables, such, as a simple variable name followed by a parenthesis, are detected by testing indicator bits in the element records. The same checking of element records is used to detect mixed-mode errors in the arithmetic expressions. The number of subscripts following an array name is checked for agreement with the number declared in the DIMENSION statement for that variable. In general, the Translator detects all syntactic errors which are within the context of a single statement and those semantic errors which occur in the use of the elements in the statement. Link Routines If the statement has no errors, the Process Control program decides whether to save the statement record and its related element records as part of the user'g program. A statement record is either added to the end of the entry and class lists or, if the .ALTER mode is active, is inserted somewhere into these lists. To accomplish this linking, space for the new record is found, and the address of this area, relative to the program area base, is inserted as the "next" address in the preceding record on the list. If the statement record is successfully put into the program area, the element records are linked to their respective lists. Every new element record also causes its address (relative to the user program base) to be entered into the R-index table. Link Diagnostics Before a new record is actually chained to a list, certain checks for consistency of referencing are made. These are partially accomplished through use of the N -index table, in which all references to labels are recorded. For every label in the program there is a corresponding REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM entry in this table. In each entry there are two fields: the first specifies the relative address of the labeled statement record; the second specifies how the label is referenced, Le., from an I/O statement, a DO statement, or a branch type statement. These entries are set up and checked before the statement is linked to the lists. Examples of, the errors detected at this phase are: 1. Duplication of statement numbers; 2. Referring to a FORMAT statement from a branch statement; 3. Referring to an executable statement from an I/O statement; 4. Using an illegal statement as the end of a DO (e.g., a branch type) ; 5. Referring, as the end of a DO, to a statement which precedes the DO. Another type of consistency error detected at this time is based on ordering of statements. To link a statement into a list, the preceding statement must be available. In the case of an ALTER insertion, the succeeding statement is also available. It is possible, then, to check for violation of such precedence rules as: 1. Declarative statements must precede executable ones; 2. The first executable statement following a branch-type statement must be numbered (i.e.. every section of the program should be potentially executable). I t is important to note that all these consistency and precedence errors are reported to the user immediately after the statement is accepted by the system. Most conventional compilers report all composition-type errors throughout the entire program before going on to check for consistency errors. In this system, diagnostics are provided as early as possible. When the END statement is first linked to the program list, or thereafter at the end of an AL TER sequence, several specialized routines check completeness of control flow and data referencing. 433 Storage Assignment The value of a simple variable or a constant is stored in the element record. However, storage for all arrays and any variable appearing in common must be specially assigned. Because of their interaction, all allocation declarations must be entered before storage can be assigned; on the other hand, storage must be assigned as soon as possible since partial execution of the program may be requested at any time. The Link routine, on recognizing the first executable statement, assigns storage on the basis of all declarative statements, which are linked on the same class list. After an ALTER sequence involving a storage allocation the same operation is .again performed. Storage Control When a statement or element record is to be linked to a list, it is moved from a temporary working area to the program area. Space for successive records or data storage is at first assigned sequentially throughout the program area. When the user deletes ( via ALTER) any statement or variable from the program, the associated records are unlinked from the program and chained to a "null" list ordered by size of record. When space is needed for a new record, the null record that best fits (i.e., large enough but with minimal "trim") is selected; this technique prevents wasteful fragmentation of the null-storage areas. If no record on the null list satisfies the space requirement, but the total size of the scattered null records would provide enough space, then a "squeeze" is performed by moving every record in the program to a contiguous storage area. All references to relative addresses in the program area are then changed to reflect this relocation. The Interpreter Execution of the user's program is done in an interpretive fashion 20 ,21,22,23,24 on a statement-by-statement basis under control of the Process Control program. This control program sends to the Interpreter the address of the statement to be executed. Upon successful 434 PROCEE'DINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 INTERPRETER SYSTEM STATEMENTS GET: GET: ADDRESS ADDRESS VALUE VALUE RECORD RECORD R NO, STATEM. INDEX CHECK AUDIT R NO, DUMP STA CHECK: N NUMBER LIST RESET CHECK: N DO DO Figure 2.7c. General Diagram of Process Control Flow -Interpreter. execution of this statement, the relative address of the next statement is saved in the program header. At this time, an indicator is turned on in the statement record, showing that the statement has been executed. The Interpreter can be broken into several parts (see Figure 2.7 c) : 1. The master interpreter, which decodes the statement type; 2. The service subroutines used by all statement routines; 3. The macro interpreter used for arithmetic expressions; 4. The various statement routines. Decoder To interpret a statement, a code is fetched from the statement record and, using the Master Block, control is transferred to the appropriate Interpreter routine. Service Routines These subroutines are used to fetch element and statement records and to address value words for variables and constants. In the In- Figure 2.7d. General Diagram of Process Control Flow -System Statements. terpreter, all fetching is done by a direct "lookat" of a table entry for an address. This is in contrast to the associative referencing used in the Translator. Execution speed is considerably increased by this elimination of list searching. All references to a variable in a statement record are by its internal identifier. This number is used as a key to the R-index table to access the relative address of the element record. Whenever a value word is fetched for use, an indicator for "variable used" is turned on in the element record. Similarly, if a value is stored into a value word, an indicator for "variable set" is turned on. Macro Interpreter The evaluation of an arithmetic expression can be expressed in hardware terms; Le., the system has an instruction repertoire of six two-address instructions, and is equipped with a group of pushdown registers. 25 , 26, 27, 28, 2<1 Execution of the statement in the interpretive REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM mode is analogous to machine execution and involves several steps: 1. Fetch the next macro (Le., instruction) to 2. 3. 4. 5. be executed; Fetch operand values; Decode the instruction operation; Perform the specified operation; Store the result in a push-down stack. Subscripts of arrays and arguments of functions are indicated by a special operator. When this operation is encountered, an entry is made in a push-down parameter stack. Values are fetched from this stack for computing an array address or for passing arguments to a function. Functions are also indicated by a special operator. When this operator is encountered, the function-element record is fetched. All records for librar'y routines point to their actual machine coding within the system. All other functions are called from random storage. Statement Routines Every statement has a particular Interpreter routine associated with it. There are several categories which should be discussed: 1. Arithmetic---the macro interpreter is used to evaluate the expression to the right of the ":=" and store its value in the left. . hand variable. 2. Branch-the macro interpreter is used to evaluate the arithmetic expression for an IF; a service routine is used to fetch the value for I in GO TO (. •.. ), 1. The proper transfer point is chosen from the list of numeric labels in the statement. This numeric label is used to access the N-index table for the relative address of the statement to which control should flow. 3. DO loops-the initial execution of a DO statement creates an entry in a push-down stack controlling DO nesting; it also initial. izes the value of the DO index variable and flags its element record as an active DO index. The execution of the last state.. ment in the range of a DO is detected through checking of an indicator turned on in the translation process. After execution of this last statement, the DO state- 435 ment is fetched again and its index is tested and incremented. Execution continues with the statement following the DO until the indexing condition is satisfied. 4. Input/Output-an I/O macro interpreter is used to compute addresses of values to be passed to the appropriate input/output service routine. A table is generated in the execution process to handle variables in the list controlled by implied DO's. Execution Diagnostics Choosing the interpretive approach to execution necessarily means sacrificing speed. For debugging purposes, this is not often a serious impediment-especially since diagnostics are possible for many errors never detected in conventional execution. These include detecting: 1. A value word not being set before used: 2. A subscript value not being valid; 3. A DO index being reset in the range of the DO; 4. A computed GO TO parameter not being in range; 5. The size of an integer exceeding its limits; 6. The existence of an illegal value in an I/O list with implied DO's. Input-Output Control System (IOCS) The prime responsibility of the Input-Output Control system is to select, from the respective queues built up by the Scheduler, the next task or combination of tasks to be performed by the individual I/O units. Upon completion of a given task, the Scheduler is notified either directly through program switch indications or indirectly through the terminal header. Before relinquishing control to the Scheduler, the IOCS initiates the next task for that channel device based on the queue information. It also maintains control surveillance over all I/O buffer areas to prevent overflow. The I/O attachments consist of disk, drum, magnetic tape, card reader, on.line printer and communications exchange. The disk is used as a permanent storage for user programs. The drum serves as a rapid 436 PROCEEDINGS-..;SPRING JOINT COMPUTER CONFERENCE, 1964 access storage device for the repeated shuffling of user programs in and out of memory. The magnetic tapes, card reader, and printer are used in a conventional manner. The communications exchange has some interesting capabilities not found in more conventional I/O equipment. The Exchange The IBM 7740 communications control system 30 • 31 is used to buffer and control the traffic flow between the communications network and the IBM 7040 computer. It is a stored-program computer with a rather specialized instruction repertoire designed for real-time applications. The instructions possess powerful logic and data manipulating facilities, through somewhat limited arithmetic capability. Instructions are fixed in size, one instruction per 32-bit word, while data is composed of strings of 8-bit characters. Addressing is at the character level, up to a maximum of 64K characters (i.e., 16K words). The 7740 program performs several communications-oriented functions. First, it accomplishes line and terminal control by generation, recognition, and manipulation of control characters, in order to establish a connection to the remote terminals, and to determine the operation to be performed. Second, it provides message control, so that'the messages may reach their intended destinations: they are logged in, monitored for correctness, and converted from the various transmission codes to the codes acceptable to the other devices in use. Third, it provides protection to ensure the proper disposition of messages, and to ensure the correction of transmission errors wherever possible. To simplify these functions, the 7740 has several hardware and programming capabilities not often found in conventional computers. 1. The most striking of these is the ability to operate in an independently controlled hierarchy of modes. In increasing order of priority (that is, decreasing order of interruptability), these are: a. The normal mode. The normal activities involved in polling, addressing, and monitoring of all communications devices are conducted in this mode. Because of the large number of lines, processing is on a continuous service basis, whereas a conventional computer attains I/O overlap by yielding independent control to the devices and servicing them on an interrupt basis. b. The I/O mode. This mode is used to control input/output between the 7740 and the 7040. A special uninterruptable state, called copy mode, is used for the actual transmission of information. c. The attention mode. This mode is entered when service (not connected with any hardware malfunction) is needed (e.g., servicing the interval timer). d. The service mode. This mode is entered if malfunctions are detected. Entry to the service, attention, or I/O copy modes may be initiated by the machine; entry to any mode may also be initiated by the program. In addition, it is possible to inhibit mode change so that tables or programs used in several modes may be protected (this is analogous to disabling a channel on a conventional computer). Associated with each mode is a pair of machine registers which contain the complete status information. Mode change is automatically accomplished by storing this information into the cells associated with the old mode and picking up the corresponding information from the cells associated with the new mode. 2. Time-stamping, essential to control in any communications or real-time environment, is provided for by the interval timer, which is automatically updated by the machine every few milliseconds. This timer is used in conjunction with attention-mode programs to provide a programmed real-time clock, and a programmer-accessible interval timer. 3. Information about each of the communication channels (or lines) is maintained in fixed positions of core storage using two channel-control words, one pair for each line involved. The current status informa- REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM tion of these words is manipulated by both the hardware and the programs in order to control the flow of information within the system. 4. In order to facilitate the acquisition and transmission of data, the memory of the 7740 is considered by the hardware to be divided into blocks of 32 characters, each of which begins on an 8-word boundary. The first 30 characters of each block are used to store data, while the last two provide a I6-bit chaining address used to indicate where the next block of information is located. These chain addresses, supplied by programming, are uS€d by the hardwa-re to advance automatically to the next character location. Because storage is not infinite, it is possible to place a special indicator in the chain-address location of any block. When this buffer-block signal is detected by the machine in the process of acquiring a new block, automatic entry into the attention mode occurs, thus enabling the program to accurately control the available storage pool. CONCLUDING REMARKS The Translator described performs a mapping of a source program to an equivalent, list- structured, internal form. This method may be called "selective" or "differential" compiling, because statements may be inserted, replaced, and deleted without retranslating the entire source program. In addition, this approach provides rapid, comprehensive reference and diagnostic data. And finally, the process is reversible; the source program may be regenerated in its original form, or in a related form. 32 Interpretive execution provides the means for complete source-language debugging. Information on the dynamic behavior of data use and control flow can be applied to improve optimization of the generated object code.The implementation and description of the rem9te-computing system has naturally been done in a time-sharing context. Nevertheless, the techniques used are equally applicable to a 437 conventional compiler operating under a monitor system. Standard hardware devices in a conventional configuration were adapted to this purpose through programming. However, system performance could be substantially improved by use of a special machine organization designed to perform the same functions. ACKNOWLEDGEMENTS The authors wish to acknowledge the contribution of two associates: Miss Harriett Cohen for the stQrage allocation routines, and Mr. Dan Davis for the I/O list decomposition and interpreter routines. APPENDIX I-EXPRESSION DECOMPOSITION /RECOMPOSITION Introduction The primary purpose of any formula translator is to reduce expressions to a form that provides the fundamental order in which operations should be performed to produce correct results. Implicit in this form should be a record of the order in which partial results are developed, accumulated, and reused. The techniques and traditional programming tools generally applied to accomplish this are :33, 34, 35, 36 1. Forward scan; 2. 3. 4. 5. Push-down list; Forcing tables; Ordered macro list; Implied push-down temporary indications; 6. Chaining and string concatenation. Forcing tables are used to produce an order of operation based on the real or assumed hierarchy of arithmetical or mathematical operators. Push-down lists in this respect often work on a LIFO (last in-first out) principle. Macros are used as a form which approaches as nearly to a machine-executable form as can be used while retaining its machine-independent structure. In addition to the operator and the operand elements, the macro form often 438 PROCEEDINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 contains a reference to the temporary result that the operation will produce, such as Tl or T2, implying a push-down order to partial results. String-manipulation techniques of chaining and concatenation are often employed to facilitate translating operations. Requirements The Arithmetic Translator includes not only the traditional decomposition but also a recomposition 37 phase to restore the statement or expression to its original form from the compressed macro string generated during decomposition. The macro string generated, therefore, must satisfy several requirements: 1. It must be easily interpreted, saving time; 2. It must be compact, saving space; 3. It must be recomposable. The decomposition translator must detect all errors in logic and syntax. It must supply the number, order, and mode of all operations to be performed by the Interpreter. The recomposition translator should develop a string in the original sequence; all necessary punctuation must be restored. In short, it must produce a string identical in all respects to the original, except for the removal of redundant parentheses. The resulting string, when decomposed again, should produce a macro string identical to that originally decomposed. OPERATOR Symbol + Nome DECOMPOSITION Left Op Right Op RECOMPOS ITION Old Op New Op plus 1 Techniques Forcing Tables The forcing tables in Figure 2.8 are used as follows. The decomposition table is used to cause the- generation of macros based on the relative hierarchy of related or successive operators. If the value for the right operator is equal to or greater than the value for the left operator, then a macro based on the left operator is generated. The recomposition table is used to decide when parentheses are necessary to maintain the hierarchy implicit in the order of macros previously generated. If the value for the new operRtor is greater than or equal to the value for the old operator, then the string developed around the old operator during a previous concatenation must be enclosed within parentheses before further concatenation can take place. Push-Down Lists The lists used in the decomposition translator are the operator and variable lists which hold those elements awaiting further action from a forcing situation. The recomposition translator has an operator list used essentially for the same purpose. In addition, it uses two lists which contain control words of partial strings awaiting further action. The "work list" contains the control words of strings which are to be concatenated into a single string with a single control word. This control word is then placed on the "string list" until a later call for further concatenation is encountered. minus Macro Strings multiply / 4 divide exponentiation so subscript fo function replacement 0 7 7 0 7 •.mary minus 0 comma left poranthesis right poranthesis 0 0 0 0 end of messoge (EOM) Note: The zero code signifies that the operator is illegal when appearing in the specified role. Figure 2.8. Forcing Tables for Translator. Each macro contains an operator byte and one or two variable bytes. The operator is a basic operation plus an indication of the modes of the variable bytes. Either or both variable bytes may contain a temporary indication. These do not have to be specific temporary indications, since owing to the ordered structure of the macros, both the Interpreter and the recomposition translator use a push-downaccumulator for storing and fetching partial re": suIts of execution and partial strings developed through concatenation. For the same reason, no indication need be kept in the macro of the REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM temporary to be generated by the operator (such as Tl or T2). To save space, macros with temporary indications may be compressed during packing procedures by indicating left and/ or right temporaries iiI the operator byte, thus eliminating all bytes for temporary indications. For example, when the macro string generated in the example is compressed in this manner, it re3ults in 22 bytes, or, at four eight-bit bytes per word, less than six words of storage. The interpreter accesses macros-in order-for exeCution of the statement, building up temporary re3ults and using them in turn when later lnacros call for them. An EOM (End of Messa.ge) operator signals the end of the macro list. The recomposition translator accesses the macros and builds up temporary strings in much the same manner as the Interpreter. Also from this simplified, compact macro form it is but a simple step to generate machine-language code; either temporary locations can be implicitly addressed by the machine itself or else explicit storage addresses can be used. Chaining and Concatenation In the recomposition translator, when an operand is not an intermediate temporary, it is developed as an element in the output string and placed in an empty word in a pool. I t is treated as a one-element string and assigned a 439 ~ x = «-B) + SQRTF(B**2-4. *A*C))/(2. "A) 0 MACRO OUTPUT STRING .ill. .Yl PUSH DOWN LISTS QE. V2 Var x % --.f:' yrr( temp SQRTF + fo Figure 2.10a. Example of Decomposition-Part I. control word. When strings are to be joined together, the last word of the first string refers to the linking operator (which is developed in the empty pool), and in turn, the operator refers to the first word of the next or preceding string. The two or more control words are combined into one which references the first and last words of the concatenated chain or string of elements. When the recomposition translator eventually encounters the EOM operator, there is only one chain represented by a control word on the string list. This chain or scrambled string is then unraveled into a sequential list of all the elements in the recomposed statement or expression. Diagnostics i. Mixed I'kde ii. Mixed fvIode in a Function Argument iii. Illegal Use of Function or Array Name Without Arguments iv. Simple Variable, Constant, or Expression follawed by left Parentheses vi. Illegal Number of Arguments in Function Illegal Mode of Function Argument The decomposition performs complete diagnostic checking. Wherever possible, error checking continues even though some errors 1 vii. Fixed ta Float Exponent viii. Level of Nesting of Functions Exceeds Maximum Number of Eight (8) ix. Illegal Successive Operators xi. Uneven Number of Parentheses MACRO OUTPUT STRING xii. General Syntax Error OP xiii. Expression Begins with Illegal Operatar xiv. Mode of Variable Not Set x = «-B) + SQRTF(B·*2-4. *A*C))/(2. 'A) 0 Illegal Parenthetical Order Vl V2 PUSH DOWN LISTS .2!. temp I'kde Not Set For Arry Arguments in Function 4. xvi. Number of Parameters in Function Exceeds Declared Maximum xvii. Number of Arguments in (Defined) Function Specified to be Zera xviii. I'kde of Actual Argument is Not Set xix. Illegal Operator in Parameter or Illegal Position for Comma + A / /' 7 temp /- / temp C Only these errors cause an immediate error return. All others return for further error checki':!S.. SQRTF ...¥ fo AuS ....C condition - (illegal in FORTRAN) .. Var • X •• Figure 2.9. Arithmetic Translator DIagnostIcs. Figure 2.10b. Example of Decomposition-Part II. 440 PROCEEDINGS-SPRING JOINT- COMPUTER CONFERENCE, 1964 1 x = ({-B) + SQRTF{B**2-4. *A*C))/{2.*A) 0 MACRO OUTPUT STRING ~ PUSH DOWN LISTS Yl:... Vl ~ ~ um X / A temp C y temp temp ~ temp fo SQRTF + temp ymP' ~ /" 4. ~ rr / ~ ~ Ym( /' temp last operator on the operator list, action is taken to output a two- or three-byte macro: a. The last operator on the operator list and the last one or two variables on the variable list, depending on the operator, are removed and incorporated into an output macro. b. For each macro generated for the output string, except for comma-operator macros, a temporary indication is generated on the variable list. ~ X = «-8) + SQRTF(8.... 2-... ·A·C»/(2. ·A) 0 temp Figure 2.10c. Example of Decomposition-Part III. have already been encountered. (See Figure 2.9 for a list of decomposition diagnostics.) Decomposition Rules (Figure 2.10) 1. When all action has been taken with a new operator or variable encountered in the forward scan~ it is placed on the appropriate push-down list. 2. An array name or function name followed by a left parenthesis generates two additional operators for the operator list: a subscript operator or function operator, and a comma operator for the· initial parameter. 3. When the forcing value of a new operator equals or exceeds the forcing value of the 1 MACRO INPUT STRING PUSH DOWN LISTS OP OP Wori< String um..,¥ -8 -" fo I 0 .Y!.. V2 ... A temp C temp temp ·"7 8··2 " / yA' / /- temp C SQRTF ... ·A temp temp 2. A temp temp X temp temp Current N_Op Figure 2.11a. Example of Recomposition-Part I. Recomposition Rules (Figure 2.11) x = «-8) + SQRTF(8·"2-... "A"C»!2. ·A) 0 MACRO OUTPUT STRING OP Vl ::!2. um ... A temp C temp temp + I 0 Yare ./ /' / X / ~ ~ / ~ ~ )4 temp fo PUSH DOWN LISTS OP SQRTF temp temp 2. A temp temp X tem,. temp Figure 2.10d. Example of Decomposition-Part IV. 1. On each new macro encountered in the forward scan, the right operand (V2), if it exists, is always considered for action before the left operand (VI). a. If Vi of a macro is not a temporary indication, it is developed in a word from the empty pool, and assigned to a control word which is placed on the intermediate work list. b. If Vi of a macro is a temporary indication, the last control word on the string list is removed and placed on the work list. 2. For any operator in a macro except the comma operator the last action taken is to 1 441 REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM X «-8) + SQRTF(B**2-4. *A*C))/(2. *A) <:> 2 x = «-8) + SQRTF(**2-4. "A"C»/(2. "A) 0 MACRO INPUT STRING VI OP ~ B um um 4. A temp C temp temp ------------ temp .--e-- + / -8 ~ ~ ~ ~ ~ ~ 8*"2-4. *A*C temp temp 2. A temp temp X temp ~- 4. A temp C temp temp String (-B~*A) --~ ~ (-~.*A) ~ temp fa SQRTF temp / SQRTF SQRTF -£0 ~ Work OP V2 Vl OP Work :;:.. PUSH DOWN LISTS MACRO INPUT STRING PUSH DOWN LISTS OP temp 2. A temp temp X temp temp· -0 X =(-B + SQRTF(B**2-4. *A*C))/(2. *A) Current NewOp temp <:> Figure 2.11d. Example of Recomposition-Part IV. Figure 2.11h. Example of Recomposition-Part II. place the operator temporarily on the operator push-down list, and to combine the control words on the work list, linking t}"'~;-r "" ...... ""....... C!t-r;nrrC! 6a.:J tr.JI"".L ......... tnrrat'ha-r '"'vo ...... "'.I..I.¥..L ;n+n ... .I..I.\lV Ana V.I..&.'\:;: ron,..,+ ... Al '-'V.l..!. .... .L V~ word, which is placed on the string list. a. For subscript and function operators, the name and parameter strings referenced on the push-down work list are linked in order, separated by appropriate parentheses and commas. X =«-B) + SQRTF(B**2-4. *A*C)/(2. *A) 0 MACRO INPUT STRING OP Y!.... ~ 2 4. A temp C temp temp ~ .....-s-- ~'C) ~~ ~*A*C) ----- -a---- -B+~*A*C) ~ ~ (2.*A) SQRTF -/ ----- Work ---r.- temp fa PUSH DOWN LISTS OP temp temp 2. A temp temp X temp (-8 + SQRTF(B**2-4*A*C)) temp 0 b. For arithmetic unary or binary operators, the one or two strings refe-renced on the work list are linked with the operator. 3. Whenever a control word is removed from the string list, an operator is removed from the operator list and tested against the new operator from the current macro. a. When the right forcing-value of the new operator equals or exceeds the left forcing-:value of this last operator from the operator list, parentheses are placed at the ends of the string referenced by the control word just placed on the work list. b. If the string represents V2, and the left forcing-values of the new and last operators are equivalent, parentheses are placed at the ends of the string. Current New OP / Figure 2.11c. Example of Recomposition-Part III. Extensions There is no limit to the length of the statement string that can be used as input to this type of decomposition translator. Any mathematical language based on hierarchical rules of operation-for purposes of computation similar to that in arithmetic formulas-can be decomposed and recomposed just as easily using forcing tables and the other traditional techniques. The macro form pro- 442 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, duced could be of quite a different form depending upon the nature of the interpretive scan. It would, of course, have the same implied order of operation. The operators involved need not be only unary or binary operators; they need not be only arithemtical or functional. Boolean operators, logical operators, ternary operators, or any others could be easily handled in this manner. REFERENCES 1. A. NEWELL (Ed.), "Information Processing Language-V Manual," Prentice-Hall, 1961. 2. H. GERLERNTER, J. HANSEN, and C. GERBERICH, "A FORTRAN-Compiled ListProcessing Language," ACM Journal, April 1960. 3. A. J. PERLIS and C. THORNTON, "Symbol Manipulation by Threaded Lists," ACM Communications, April 1960. 4. A. EVANS, A. PERLIS, and H. VAN ZOEREN, "The Use of Threaded Lists in Constructing a Combined ALGOL and Machine-Like Assembly Processor," ACM Communications, January 1961. 5. J. WEIZENBAUM, "Knotted List Structures," ACM Communications, March 1962. 6. J. WEIZENBAUM, "Symmetric List Processor," ACM Communicat~ons, September 1963. 7. R. BROOKER and D. MORRIS, "A General Translation Program for Phrase Structure Languages (Lists) ," ACM Journal, January 1962. 8. H. W. LAWSON, "The Use of Chain List Matrices for the Analysis of COBOL Data Structures," Pl'oc. ACM National Conference, September 1962. 9. H. D. BAECKER, "Mapped List Structures," ACM Communications, August 1963. 10. P. R. KOSINSKI, H. KANNER, and C. L. ROBINSON, "A Tree-Structured Symbol Table for an ALGOL Compiler," Pl'OC. of ACM National Confef'ence, August 1963. 11. P. M. SHERMAN, "Table Look-at Techniques," ACM Communications, April 1961. 12. L. R. TURNER, A. MANOS, and N. LANDIS, Initial Experience on Multiprogramming 1964 on the Lewis Research Center 1103 Computer," Proc. of ACM National Conference, August 1960. 13. N. LANDIS, A. MANOS, and L. R. TURNER, "Initial Experience with an Operating Multiprogramming System," ACM C;ommunications, May 1962. 14. A. B. SHAFRITZ, A. E. MILLER, and K. ROSE, "Multi-level Programming for a Real-Time System," Proc. EJCC, December 1961. 15. E. F. CODD, "Multiprogram Scheduling," ACM Communications, June 1960, July 1960. 16. E. F. CODD, "Experience with a Multiprogram Scheduling Algorithm," Proc. of ACM National Conference, August 1960. 17. J. WEGSTEIN, "From Formulas to Computer Oriented Language," ACM Communications, March 1959. 18. B. ARDEN and R. GRAHAM, "On GAT and the Construction of Translators," ACM Communications, July 1959. 19. M. E. CONWAY, "Design of a Separable Transition Diagram Compiler," ACM Communications, July 1963. 20. The 701 Sp,.eedcoding System, IBM Form Number 24-6059. 21. The 705 Print. System, IBM Form Number 32-7855, 1957. 22. Bendix Intercom 1000 Programming System, Bendix Computer Division, 1958. 23. W. R. BRITTENHAM, et aI., "SALE (Simple Algebraic Language for Engineers) ," ACM Cornmunications, October 1959. 24. R. E. MACHOL, W. J. ECCLES, and J. C. BAYS, "There's Still a Place for Interpreters," Pl'oc. ACM National Conference, September 1962. 25. W. LONGERAN and P. KING, "Design of the Burroughs B5000 System," Datamation, April 1961. 26. R. W. BARTON, "A New Approach to the Function Design of a Digital Computer," Proc. W JeC, April 1961. 27. J. ANDERSON, "A Computer for Direct Execution of Algorithmic Languages," Proc. EJCC, December 1961. REMOTE COMPUTING-AN EXPERIMENTAL SYSTEM 28. A. C. D. HALEY, "The KDF. 9 Computer System," Proc. F JCC, December 1962. 29. C. B. CARLSON, "The Mechanization of a Push-Down Stack," Proc. FJCC, November 1963. 30. 7740 Communications Control System Principles of Operation, IBM Form Number A22-6753. 31. 7740 Communications Control System Communications Control Package, IBM Form Number C28-8160. 32. J. J. ALLEN, D. P. MOORE, and H. P. ROGOWAY, "SHARE Internal FORTRAN Translator (SIFT) ," Datamation, March 1963. 443 33. K. SAMUELSON and F. L. BAUER, "Sequential Formula Translation," ACM Communications, February 1960. 34. H. D. BA~CKER, "Implementing a Stack," ACM Communications, October 1962. 35. C. L. HAMBLIN, "Translation to and from Polish Notation," The Computer Journal, October 1962. 36. R. J. EVEY, "Application of PushdownStore Machines," Proc. F JCC, November 1963. 37. K. IVERSON, "A Programming Language" (Chapter 5), John Wiley & Sons, Inc., 1962. MULTICOMPUTER PROGRAMMING FOR A· LARGE SCALE REAL-TIME DATA PROCESSING SYSTEM G. E. Pickering, E. G. Mutschler, and G. A. Erickson UNIVAC Division of Sperry Rand Corporation San Diego, California INTRODUCTION Fundamental in the design philosophy of this system is the Unit Computer concept. In essence, the concept is a recognition of the need for computers of varying capability among ships of various types and operating modes. It solves this requirement by the use of standard computers operating in multiples to obtain increased ca:pacity and functional capability, rather than use of several different computers each possibly of a different shape and size. However, in solving this varying computer capacity requirement, the Unit Computer concept established the need for multicomputer programs. The multicomputer programming techniques discussed in this paper were conceived and implemented in a large scale tactical data system developed for the U.S. Navy. Many of the details of this system are classified and cannot be discussed here. * However, it can be stated that the subject system is a man/machine complex, primarily intended for fleet air defense and surface operations and maneuvering. The objectives of the system are: to provide commanders of forces afloat with a broad picture of the current tactical situation; to assist in directing operations in time to intercept and destroy potential threats; and to present the means to coordinate various weapon systems in a combat environment. These are achieved by automating, to a high degree, the collecting, processing, exchanging, and evaluating of large quantities of data through use of computers and digital data processing techniques. This paper concerns the programming problems encountered in designing and implementing operational programs requiring more than one Unit Computer and describes the techniques developed to solve two intrinsic problem areas: • EXECUTIVE CONTROL IN A MULTICOMPUTER COMPLEX The system objectives are accomplished in real-time. Data are received by the system from various sensing devices such as shipboard and AEW radars, sonar, IFF equipment and ECM equipment which are in continuous contact with the outside environment. Data entering the system are processed, analyzed and used by the system to influence 01' alte·,. an event during the pl'og'ress of that event, i.e., real-time. • DATA TRANSFER BETWEEN COMPUTERS A third problem is assignment and distribution of tasks between computers. To deal with this subject in the detail required would necessitate discussing classified material. That this is a problem of complexity should be obvious. However, it is evident that distribution of tasks between computers is a system design problem which must be uniquely solved for each multicomputer system. ':'This paper was originally prepared for presentation at the 19()1 Eastern Joint Computer Conference. Security regulations prevented its release until now. 445 446 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 It should be emphasized that the multicomputer programming techniques discussed in this paper have been tried and tested for the last five years. Several multicomputer programs, employing two and three computers, have been delivered to the Navy. Thus, multicomputer programming, as discussed in this paper, is not theory, but actuality. mately 7 days, the real~time modulus of the system may be somewhat less. The real-time modulus of the system is so defined that in scheduling tasks for execution (see discussion under Section II-Program Control in a Multicomputer Complex), the modulus of the clock will not be exceeded. In preparation for the subject of multicomputer programming, it is first necessary to discuss pertinent characteristics of the Unit Computer developed by UNIVAC to satisfy the Unit Computer design philosophy. In a multicomputer program, the real-time clocks within all computers are synchronized. Upon initiation one computer transmits the content of its real-time clock to the other computers, which then set their clocks accordingly. DATA TRANSFER LOGIC I. THE UNIT COMPUTER The Unit Computer is a general-purpose, stored program, solid-state machine. * Only those characteristics most pertinent to the subject of multicomputer programming are discussed in this paper. Real-time Clock Among the features that make possible the use of the computer in real-time applications is the real-time clock. The clock is contained in a special register within Magnetic Core Memory and can be referenced or set by the computer program. Time is maintained accurate to the nearest 2- 10 second. The clock is incremented automatically by the computer upon signal of a 1/1024 cps crystal controlled oscillator. When the system is started, the clock is set to zero and time is automatically maintained relative to the start of the system problem. Although the modulus of the clock is approxi- A total of 14 input and 14 output channels is provided in the Unit Computer. Of these, two input and two output channels are especially designed for intercomputer communications. Each input channel and output channel consists of 30 data lines and 3 control lines. The special input/output channels differ from a normal input/output channel primarily in use of the control lines and in the associated channel logic. The input/output logic of the Unit Computer requires that peripheral equipment request each input or output data word transfer. The computer then acknowledges receipt of the input or availability of an output. All input channels, including the special input channels for intercomputer transfer, and all normal output channels use Request and Acknowledge logic. The control lines associated with request and acknowledge logic are shown in Figure 1. These are Input Data Request Line and Input Acknowledge Line for normal input channels and EXTERNAL FUNCTION I I ~ PERIPHERAL \ EQUIPMENT II lid 1&1 0 30 DATA UNES I --'t------J-- --~---J--- Z Z UNIT u COMPUTER ~ S ~ ;:) 0 EXTERNAL INTERRUPT IZZ I I~ IS I~ INPUT ACKNOWLEDGE I \ \ 30 DATA UNES I -\-----/---- I I ----""""'----1-I Figure 1. Normal Channel Configuration. *A more thorough description can be found in "Datamation," December 1961, p. 45. PERIPHERAL EQUIPMENT MULTICOMPUTER PROGRAMMING FOR A LARGE SCALE PROCESSING SYSTEM Output Data Request Line and Output Acknowledge Line for normal output channels. In addition to these two lines normal input channels have an External Interrupt control line and normal output channels have an External Function control line. .The External Interrupt control line allows the external equipment connected to that channel to signal the computer of its immediate data transfer requirement which is honored by an interruption of the computer program. The External Function control line is used to specify a function desired of the external equipment connected to that output channel. An External" Function Word to a tape control unit, for example, may. specify Rewind Tape Unit 2. The two special output channels use Ready and Resume logic for intercomputer data transfer control. The transmitting computer signals that it is Ready with an output data word which is interpreted by the receiving computer as an Input Data Request signal. The receiving computer upon accepting the transfer will send back an Input Acknowledge signal which is interpreted by the transmitting computer as a ThiS reverse logic is necessary so that th~"'t;~"~s~i~tting computer will exhibit to the receiving computer the same control and tin1ing characteristics as do the peripheral equipment. ROQllTYlO The control lines associated with Ready and Resume logic of the intercomputer channels are illustrated in Figure 2.' These are Input Data Request (or Ready) and Input Acknowledge (or Resume). In addition, intercomputer channels utilize an Input Buffer Status signal. The Input Buffer Status signal originates in the receiving computer and is defined by the state of the Input Buffer Active/Inactive Designator. The transmitting computer can sense the Input Buffer Status signal, thus determining status of the receiving computer's input buffer on that intercomputer channel. A special output register (Cl), incorporated in the computer, alternately services the two output channels reserved for intercomputer communication. All output intercomputer communications are time-shared through this special ,output register which holds data until a. Resume sign~l is received from the receiving 447 computer. However, if a ~esume signal is not received within 32 to 64 seconds, the register is automatically cleared and the computer is notified of this condition by an Internal Interrupt called the Intercomputer Failure Monitor. This 32- to 64-second time interval is necessary to insure that intercomputer output with other computers is not suspended indefinitely. The Interrupt utilizes special entrance registers in Magnetic Core Memory for notification of faulty intercomputer output channels and to distinguish which of the channels is faulty. Data transfer in or out of the Unit Computer, whether between computers or between computer and external equipment, is handled by buffered transmission of data with timing under control of the receiving computer or the external equipment respectively. The buffering process transfers consecutive words, starting at a given initial address through a given terminal address, on a specified input or output channel. A single computer instruction will initiate a buffer mode of data transfer. Once established, buffer transmission employes independent access to memory; the entire buffering operation proceeds to completion with no additional program references. Thus, the buffer mode of data exchange provides for input/ output operating asynchronously with the main computer program; the computer continues execution of program instructions in the normal sequence. The sequence of data and control signals for a normal transfer of data from one computer to another would proceed as follows: a) Receiving computer sets Input Buffer Acti ve Signal; b) Transmitting computer 'detects Input Buffer Active Signal; c) Transmitting computer places data on 30 data lines; d) Transmitting computer sets Ready which becomes Input Data Request in Receiving computer; e) Receiving computer detects Input Data Request; f) Receiving computer samples 30 data lines; g) Receiving computer sets Input Acknowledge line (returned to Transmitting computer as Resume); 448 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 h) Transmitting computer senses Resume line; i) Transmitting computer drops data lines and Ready line. Steps c) through i) of this sequence are repeated for every data word. Input Buffer Active remains energized during the entire transfer of a block of words. In most cases it is desirable for the program to be informed of the completion of a particular buffer transmission. This is made possible by internal interrupts generated within the computer. If an input buffer mode via a specified channel was established with a buffer monitor, an Input Buffer Monitor InJ;errupt is generated wheh the buffer mode terminates (Le., transmission has been completed ~s indicated by the current buffer address equal to the terminal address) . Likewise, an output buffer established with monitor would generate an Output Buffer Monitor Interrupt when the buffer mode terminates. A Buffer Monitor Interrupt is associated with each input channel and each output channel; a separate entrance register in computer memory is reserved for each Interrupt. The unique entrance address thus defines the source of the Internal Interrupt request for action by the appropriate Interrupt routine. COMPUTER INSTRUCTIONS In addition to instructions normal to arithmetic, logic, and index operations there are several instructions which add tremendously to the power of the computer in a real-time application. Those instructions which are mentioned or inferred later in the paper are now described. Repeat-The Repeat instruction sets up a repeat mode whereby the instruction immediately following the Repeat instruction is executed n times, as specified in the Repeat instruction. In initiating the repeat mode, n is transferred to a special register which retains the number of executions remaining throughout the repeat mode. If n is zero, the instruction to be repeated is skipped. This instruction also enables automatic address modification (increment, decrement, or .increased by a pre-set constant) of the repeated instruction after each individual execution. Ordinarily the repeated instruction searches a table for coincidence. If coincidence does not occur, the repeat mode terminates and the instruction following the repeated instruction is executed. If coincidence occurs, the repeat mode terminates and the instruction following the repeated instruction is skipped. This method of searching a table for coincidence is used by the Executive Routine in controlling the execution of various tasks of the main program, as discussed later. A principal advantage of a repeated search is the reduction in instruction execution time since a memory reference is not required to read in a next instruction. As used in the Executive Routine, a repeat search takes 8 + 11.2 (N) microseconds where N is the table item number on which coincidence occurs. A non-repeated search to accomplish the same end would require 8 + 40 (N) microseconds. Thus, the time required for program control is reduced by an approximate factor of 3 through use of the Repeat instruction. Compare-This instruction compares the signed value of the operand with the signed value contained in either or both the arithmetic registers, A and Q. Skipping the program's next sequential instruction is also allowed by this instruction if the Skip condition is met. The Compare instruction is used with the Repeat instruction to perform the search described previously. The execution time given in that description is for a repeated Compare. As will be described later in Section II of this paper, the time at which a task is scheduled to be executed is compared with the contents of the real-time clock. Return Jump-The Return Jump instruction provides for transferring control to a specific set of instructions (subroutine), executing that subroutine, and returning control to the next instruction following the Return Jump. The Return Jump makes possible the modular program construction used whereby each separately defined task is implemented by a subprogram, which is referenced by the ExecutIve Routine, and the identical subprogram is used in each program performing that task. A subprogram js a high-level subroutine; a subprogram usually references lower-level subroutines. MULTICOMPUTER PROGRAMMING FOR A LARGE SCALE PROCESSING SYSTEM External Function-The External Function instruction has two meanings, one for intercomputer channels and another for normal input/ output channels. If the instruction specifies an intercomputer channel, the connected computer is interrogated as to the status of its input buffer on the connected channel. If the interconnected computer's input buffer is active (Le., an input buffer has been initiated and the connected computer is ready to receive data), the next instruction following the External Function is skipped. If the connected computer's input buffer is not active, the next instruction (which is normally a Jump) is executed. For normal input/output channels, the operand of the External Function instruction (normally a control code) is transmitted to the external equipment connected to that channel. Store Input Channel-This instruction stores the contents of the specified input channel at the address specified by the operand. An Input Acknowledge signal is then sent over the specified channel thereby informing the external equipment of the computer's availability to receive additional data and to indicate that the previous input was received. Since buffer mode of data transfer is normally used, the Store Input Channel instruction is used primarily to generate Input Acknowledge signals to External Interrupts. Initiate Input Buffer (With Monitor) and Initiate Output Buffer (With Monitor)-These instructions establish an input or output buffer, respectively, via the specified input (or output) channel. Subsequent transfers of data, executed at a rate determined by the external device, are made directly into (or occur directly from) Magnetic Core Storage starting at the address specified by the operand. The storage address initially established is advanced by one for each individual transfer. The next current address is maintained throughout the buffer process in the lower half word of the Buffer Control Register for the specified channel; the last address of the transfer is maintained in the upper half word of the same Buffer Control Register. Each input channel and each output channel has a unique Buffer Control Register associated with it. The buffer mode will continue until it is superseded by subsequent initiation of a new 449 buffer via the same channel or until the upper and lower half words of the Buffer Control Register are equal. Should the latter occur, a Buffer Monitor Interrupt is generated, causing the computer program to be interrupted and the Buffer Monitor Interrupt routine for that channel to be executed. Multicomputer programs use the Initiate Input Buffer (With Monitor) and Initiate Output Buffer (With Monitor) instructions exclusively in transmitting data between computers. II. EXECUTIVE CONTROL PHILOSOPHY The system which made necessary multicomputer programming techniques has a number of functions that must be performed. In accomplishing these functions, the design objective is to use both men and machines to best advantage. Repetitive and routine operations are performed automatically by machines, whereas decisions of tactical importance are made by men. Although the focal point of the equipment complex is one or more computers, the system also employs high-speed digital data communication facilities, radar video 'processors, analog-to-digital data converters, digitalto-analog converters, and typical computer peripheral equipment such as magnetic tape and Teletype, * all of which provide for automatic inputs to the computer and/or automatic outputs from the computer. A complete display of complex and data entry devices allow men to monitor an ever-changing, yet accurate picture of the current tactical situation and to enter their intelligence and tactical decisions into the computer. All equipment is connected directly or indirectly to the computers. For each function the computers must: • receive data from the appropriate system equipment, consistent with data transfer rates of each equipment; • correlate, process and (as necessary) evaluate data in performing the specified system function; • present the results of processing and evaluation to men for their interpretation and decisions; *Teletype isa trade mark of the Teletype Corporation. 450 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 transmit pertinent data and human decisions, as appropriate, regarding that data to the ultimate users, again consistent with data transfer rates of the equipment involved. The system operates in real-time, therefore, operational requirements dictate the response times for individual system functions. To relate a real-time system to real-time data processing then, one might say that real-time data processing is processing done immediately as a result of an input for the purpose of providing an essentially instantaneous response or output. >I: Processing must, as a result, be completed at a rate greater than the input data rate or else the processing will fall behind so that eventually the system will be operating out of real-time. The system saturation point is reached when processing rate equals the input rate. From the foregoing discussion, it is clear that any real-time system must have, at least, a data processing capability sufficient to handle the average input rate from all input functions. Moreover, since system inputs occur randomly, it must be assumed that all inputs could occur simultaneously. This would represent the peak load. The problem then, is how to schedule, or queue, the functions to be performed in such a way that under peak load conditions all functions would be performed within their real-time requirements. An Executive Control Philosophy has been developed and implemented to solve the scheduling (or multiprogramming) problem. Executive Control Philosophy is a general term-explained specifically it means a recognition of the system's tasks in order of system priority; distributing them among the system's computers; and then controlling them in the individual computers of the system by an Executive Routine within each computer. The Executive Control Philosophy is based on the fact that each function and concomitant handling of peripheral equipment defines a :: 1. Element ej is assumed to be in position /117 having x, y coordina tes lux and lup I I Fn the Phase II pseudo wire length number for element ej when in position 1m is given by: == llll::i j -----}---.-~ (Iyx (ek) - /11" I a (T j ) - 1 nil k ' Y.r (ek) - tn)' : ) where ej, t: T j , ek t: T j k =F i, Fl.ln + and a (T) > 1. To get an expression for FllI the phase I pseudo wire length number, it is necessary first to define an auxiliary function F j,lI*j for three separate cases. Case 1: a (T) = 2 Fi.n*j = I yx (ed where T j = {ek, ed. tux + 1 I yy (ek) - IllY I Case2: a (Tj) = 3 F i,ll*j = max { ( ! Yx (ek) - tnx I + : yY (ek) !II~' : ), ( , yx (enJ - IlIx I + I yY (euJ - Iny I ) } where T j = {elll, ek, ej }. Case 3: a (T j ) >3 For this case a pseudo location t'l~ is first defined, its x and y components given by: 1 tj/ j == a (T ) ~ Yx (ek) where ek t: T j and k =F i. ) ~ Y.'· (ek) where ek j 1 lj/j == a (T t: T j and k =F i. j Furthermore, let a function Gt j be given by: G.*j = I r Yx (e k) - - 1 a (T j ) !ix*j : + where ek t: T j and k =1= i. - i y~. 1 ~ all k (ek) - A/j I) AN ALGORITHM FOR PLACEMENT OF INTERCONNECTED ELEMENTS Here again Fi.n == 7 since F 1.11. 3 == F i •n·2 == 4 and Fi.n*l == 7. FPII for the case in Figure 11 would be equal to 11, the length of both wires. Then Fi.n*j can be expressed as: F i,n *'J = [ a (T 1) ~k 1 ;111 _ j (ek) - lnx I + I yy (ek) where ek £ T j and k =1= i. ( I Yx lny I )] - Gt j l1 > 1. One of the advantages of choosing the functions in this manner is the ease of computation. Another is that they tend to closely approximate the real situation. As an example, consider the element ei which has only one transductor T j • Suppose ex (T j ) == 2. Then F:.~ is merely the dog leg distance between the two elements contained in T j • Suppose ei is contained in two transductors Tl and T2 such that a (T 1 ) == ex (T2) == 2, Tl == { ek, ei }, and T2 == { e nu ei }. That is, ei is connected to two different elements as shown in Figure 11. Then Ff,In would be equal to seven which is the length of the longest wire. Suppose we also had Ta == { em, ei } with ex (T3) == 2 as shown in Figure 12. @ C C 2 Length 6) =4 For another example, suppose el belongs only to transductor T j where a (TJ == 5 and T j == { eh e2, e3, e4, e5 }. Then Fi~~ can be interpreted as the average distance from element el to the other elements in Tjo Suppose M == { th 12 } where 11 and t2 are one unit apart, and both positions are at some distance from the other elements in T j as shown in Figure 13. Then F l\ - F l.I2 == 1 which is desirable because the wiring would be done in a manner similar to Figure 14 below, and the difference in actual wire length between el in /1 and el in 12 is only 1 unit. Suppose positions 11 and 12 and elements e2, e3, e4, and e5 are located as shown in Figure 15 below, that is, interior to the set. Then Ff\ - FP2 == O. That is, the two positions are equivalent and el ll1:ay ,be placed in either position with no difference in wire length. Once again the function approaches the real situation. For the case shown in Figure 5, Ff.l == Fi.2 == O. Fi.} would be greater than zero only if f 1 were a position located outside the rectangle formed by e2, ea, e4, and e!) as in Figure 11. For Figure 11, Fi.l - Ff,2 == 1 as would be expected. cy 1 Length F[~ for the case in Figure 12 would be equal to 15. Now the phase I measure Fi~n for wire length is defined as : Ffn == max {F t . ·j} where ei £ T j and ex (T j ) =7 Figure 11. c em) C 3 Length C ~ei) 2 =4 J 1 Length =7 Figure 14. 6) Figure 12. 6) 6) Figure 13~ 6) g]g] 6) ~ g] 485 0) C0 6) Figure 15. 486 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 which may .be contained in any unconnected set. This number is a function of the number of empty positions on the board and the amount of avialable space in the computer. In short, a given placement program is restricted as to the size of the solution matrix which it can solve. By restricting the number of elements in an unconnected set, the size of the matrix is restricted. This means that the sets may not be maximal in some cases. In many problems, the majority of the transductors are such that a (Tj ) 2. But the case where a (Tj ) > 2 must be handled consistently. Furthermore, it is observed that given any element ei there is a large probability that ei t: T j such that a (Tj ) > 2. Table 1 shows the result of a survey made on a large computer printed circuit board. The board had 729 transductors with the distribution as shown. N is the number of transductors with the given a (T j ) . In other words, the sets T j were sorted into groups according to their cardinal number. = 5. The Procedure for Forming Maximal Unconnected Sets Given a set S of elements ei and a set of transductors (a typical transductor be1ng T j ) an unconnected set U is defined as follows: j-l z U U m (the union of sets U l. U 2 , ={ U eh ek t: S I there is no T j such that ei £ T j and ek £ T j }. There are many ways in which the set U can be formed. It is desirable to form a family U' { U h U 2 , Us, ... , U r , • • • U q }. Each member of S is contained in at least one of the sets in U/. Also, it appears desirable that the unconnected sets should overlap one another. I In dealing with a large number of elements it is not feasible to form all unconnected sets. Instead a technique is used which causes some overlap. About 1).. of the elements in each set will be contained in other sets. Figure 16 is a flow chart which shows the technique for obtaining the unconnected set U j given the sets Uo, U h U 2 , ••• , U j - I • The following nomenclature is used in Figure 16. = Eo E W = The original set of elements which are eligible for inclusion in an unconnected set. A working set containing elements eligible for inclusion in the current unconnected set. A working set containing some of the elements not eligible for inclusion in the current unconnected set. Elements in Ware connected to elements which are in the current unconnected set U j • max (X:!h X:!:!) (X:!h x:!:!) > max (XI h XI:!) if min (Xl h XI:!) or min -I 10 r---------------4--------------/-~ otherwise > max (Xl:!, X:!:!) (Xl:!, X:!:!) > max (X11' X:! if min (X11, Xl:!) or min I ) otherwise -I 10 r------------/---~--------------~ This scheme has the intuitive appeal, if one interprets the range of possible values of X as a grey scale, of detecting a noisy pattern on a noisy background. The general rule for the recognition of a figure is that min (F) Figure 10. Convergence of a 250 x 1000 Linear System. problem in pattern recognition, which while simple, is certainly not trivial. The experiment will be described in two stages: first the procedure by which the patterns are generated will be described and analyzed, and then the associative memory which was used and its functioning will be discussed. It is desirable in order to simplify the analysis of the results that the pattern generation scheme assign approximately half of the possible inputs to the class of patterns and half to the class of non patterns. It is also desirable that the procedure have an intuitive appeal as a pattern generator, that is that the patterns should be recognizable by some relatively simple logical test. The following satisfies both of these criteria. Consider a four-element mosaic X11, Xl:!, X21, X:!2, where each element can assume anyone of a discrete range of val ues, x. The pattern which is tested for in this experiment consists of a bar, either vertical or hori- > max (B) where F designates the figure elements and B designates the background elements, i.e., the lightest element in the figure is to be darker than the darkest element in the background. Thus a contrast enhancing procedure which converted all elements of a field to black or white depending on whether they were greater than or equal to the minimum value in a potential figure or less respectively would reconstruct a black figure on a white background with no degradation, had the input mosaic pattern actually been in the class of figures. The probability that an arbitrary four-element pattern, with a grey scale of 48 steps, is recognizable as a bar is 0.3197, which provides an adequate density of patterns for the purposes of the present experiment. The associative addressing matrix used in this experiment had an effective size of 9600 columns, each of which was 192 bits in length. The actual matrix was 1/48-th this size, 192 X 200, but a technique devised by Dr. H. Everett of WSEG makes it possible to use this smaller matrix in such a way as to be equivalent to the larger memory. The matrix is filled with ones and zeroes using a random number generator, with the probability of a one being entered in any bit position being a prechosen value p: in this example p == 0.2. PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1~64 504 The threshold was selected to be ~ == 3. Using these parameters it is possible to define the statistical behavior of the memory in generating the associative addresses. The probability that any particular a ~lement is a one, P ,is given by 4 P == L (~ ) (0.2)i(0.8)4-i = 0.0256 i == 3 Consequently the mean Hamming weight of the a vectors is He' == 245.76 with a standard deviation of (Ta. = 15.48. This says that the number of a elements which Cycle 1 2 3 4 5 6 7 8 Totals will be operated on in any single relaxation cycle is between 215 and 277 with a 95 % likelihood, which is a satisfactory description of the extent of the alteration in Q introduced by relaxation. The associative memory was shown a sequence of 400 patterns constructed. by using a random number generator to assign values to the Xij' Whether a particular mosaic input was a pattern or not was determined by applying the logical test. I t is worth noting that these 400 points were drawn from an environment of 5, 308, 416, (48 4 ) , possible mosaic configurations. The scores of the device are given. in Table I, as accumulated over cycles of fifty exposures. TABLE I Number of "patterns" in the cycle 30 34 25 31 33 29 26 26 234 The results tabulated in the preceding paragraph summarize quite well the operation of the associative memory as a pattern recognizer. The fact that roughly the same number of patterns and nonpatterns have been misclassified, out of a population of 234 patterns and 166 non patterns, is an indication of unbiased performance in the classification of mosaic figures. These results are shown graphically in Figure 11 where the raw scores, accumulated over cycles of fifty exposures, and a smoothed curve are plotted versus cycle number. Pattern Errors Nonpattern Errors 27 10 4 8 2 3 4 7 65 2 12 16 6 11 10 8 4 69 originally used by Smith 21 in his studies of a perceptron-like computer model, ADAP II. This particular function was chosen because it embodied several special features whbh allow a testing of the associative memory functions and because it permits a qualitative comparison of Smith's results and those achieved using the associative memory. The "Smith function" is given by F = Xl X2 2xa 0 . x .. + II which has a mean of F - k1k:! 4 K- Experiment No.5 This experiment is devoted to an investigation of the behavior of the associative memory when used to store a simple algebraic function + + ka where k i is the number of nonzero steps in the discrete range of the i-th variable. For test purposes this function has several advantages. It involves a cross product term which has proven 505 APPLICATION OF AN ASSOCIATIVELY ADDRESSED, DISTRIBUTED MEMORY . .. . , ~r -I '. 10 .. .. "" ~ l- -I 10 Figure 11. Scores of the Pattern Recognition Program on Cycles of 50 Points Each. to be a very difficult function for perceptronlike (single layer perceptrons) devices to learn. It has a term which has a zero coefficient and hence; although this variable enters into the associative addressing, cannot affect the functional value. Finally the basic form is a iinear summation of terms, a functional form most easily stored in the quasi-linear associative memory. In all of the tests which make up this experiment n == 9, i.e., Xi == (0, 1, ... , 9). The matrix is 192 x 9,600 with the Everett scheme for permuting a base matrix being applied to a basic 192 x 200 array. The essential parameters of the experiment are p == 0.2 8==3 which gives a mean Hamming weight of Ha == 261.12 A random number generator was used to assign random values (0 - 9) to Xt, X2, Xa, and X.t and the corresponding F s was calculated using these random variables. The convergence plot for this test is shown in Figure 12. E is defined in a slightly different manner here than before. In the case of the contrived compatible linear systems where E was' first introduced, the functional values were tightly clustered about the mean and bounded away from zero so that no I I I I I ~ 10 Figure 12. Convergence for F!< with p = 0.2; (j = 3. difficulty could be encountered in defining E to be the average absolute percentage error. Here the function is very broadly spread, and furthermore not bounded away from zero; therefore the following related definition for E is used in discussing this experiment. D -r;, 1 ~ ~==D L IR til F\ ! i == 1 where D == 60 in this experiment. The following test is actually the crux of this whole series of experiments since it demonstrates the actual operation of the associative memory. This test is concerned with the memory's performance on the F s function using the parameters given in the preceding paragraph. The convergence plot for this particular test has already been given in Figure 12; however, the detailed behavior of the memory is lost in a smoothed measure of convergence such as E. Since the primary function of the associative memory is the recall (or estimation) of functional values it is highly desirable to present the memory output in a form where its performance can be conveniently judged. Two questions which occur naturally when one considers an associative memory are: 1. How will it perform in a new field of data 506 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 ~ 100 ~ Ii /'\ /1 1\ 0 I I 0 b 1 I!~ I I Fs i 1 c!l l 50_ ,\ ' I~· j ( V 0 o V\ o o \ \ 0 \ r \ \ I \ \ ~ ~, ! I I- I 11 :~ I 0 I I :i \I I ~ 0- 0- Figure 13. Test of a Filled Memory on Random Points in X Space (0 predicted, • actual values). = = points, if forced to estimate without new data being entered? 2. What is the span of its memory; that is how well will it do if shown the same set of data points with which the memory was originally filled, compared to a new field of data points? Both of these questions have been investigated. The memory was "filled" by drawing 3000 data points at random as described before, and then by setting the relaxation coefficient to zero, k = 0, further changes in the Q vector were prevented. First a sequence of random points, presumably new ones, were used to query the memory. The results of this test are shown in Figure 13. E = 0.10444 which indicates that the terminal performance of the associative memory while in the learn cycle, E = 0.10248, is probably a measure of its storage accuracy over the entire functional space. The second question actually answers this by showing that the detailed memory of the actual data points entered in the memory has been sacrificed for a type of distributed functional memory for the entire functional space. Figure 14 shows this phenomenon in a very striking fashion. The points Figure 14. Test of the Survival of Detailed Memory Entries (0 = predicted, • actual values). = immediately preceding the end of the storage period are all queried, with the unexpected result that the memory has a detailed retentivity period of perhaps 500 points, or half a cycle, and that beyond this the individual data points have been lost individually and have been assimilated instead in a collective form as a "smoothed" function. In conclusion then, the series of tests which made up Experiment No.5 has demonstrated the behavior of the associative memory on an actual notrivial problem. The data could just as well have been drawn from any number of real-life problem situations; however, it seemed appropriate to discuss a well-behaved analytic function in this paper to permit the reader to readily judge the performance of the memory. CONCLUSION In view of the length of some of the arguments which have been presented, it seems especially desirable to give a succinct description of the mathematical model for an associative memory developed in this paper. A function (commonly a decision function) f (X) is defined over an n-dimensional input APPLICATION OF AN ASSOCIATIVELY ADDRESSED, DISTRIBUTED MEMORY space, in which each of the variables is restricted to assume one of a discrete set, k i in number, of values. Each of these input points is mapped linearly onto a vertex of an Ndimensional unit hypercube, n N == L kL, i == 1 all of whose vertices make up the so-called Sspace. The function f (X) is the desired memory fill. A metric Si is defined by S==X·A where A is an (N x R) matrix whose columns are points in S-space. These points, Ah constitute a set of R fixed reference points in S-space. The metric Si defines the distance of X from Ab and was shown to be complete. A much weaker metric, ab is defined on S by the logical operation I if Si. :::::". 0 ai == { 0 if St :::::::,. 0 where 0 is a threshold value; 0 L. 0 L. n. It was shown that for some A (i.e., for a sufficiently large number of fixed reference points) there exists at least one hyperplane, Q, in the M space in which a is defined, such that the distance at which the point a, whose pre-image as defined above is X, lies from Q is the function f (X) multiplied by a suitable normalizing constant. Such a hyperplane may be found by the iterative procedure : ~I~;: Q" == Q' - a . Q' + 6. 'I the distance d between words is computed and the entries are organized accordingly. d 2 was defined as a function of the N-tuples representing the points in S-space and one must show that the same distance can be calculated for any of the proposed image spaces. In this appendix we shall prove that d 2 is the same for X and X', Sand S' and a and a'. Following our earlier usage the vectors in S-space will be assumed to have N elements grouped in n' partitions or variable ranges in each of which a single one occurs. Now let A be the complete N x 2:\" binary matrix in which every possible N bit binary word appears as a column of the matrix. The S matrix is then a (1 x 2:\") row matrix formed by multiplying A by X. If we consider two vectors, say X and X', in S-space the distance between them is found from (3) to be d 2 == n - X'XT (3) The first step in demonstrating that the weak metric on S is a complete one is to show that d 2 as given by (3) may be computed from S and S' and non-specific knowledge about S-space: XA == Sand X'A == S' form X' A (XA) T == S'S'1' or X' AATX'l' == S'S'1' (9) The unusual properties of the complete binary matrix make it possible to greatly simplify the left hand side of (9) through a manipulation of AAT ' f 'I =1= J. ( • . T ~ ( •, au -- 2:\"-2 , 1 AA (8) -'- ai.j) ~ l a'I' Thus we may finally conclude that the output of this particular mo('el of an associative memory (at a time where <}' is the estimate of Q) upon being interrogated with the address (associative criterion) X is == _ 2:\"-1 ajj - 'f 1 ._ . J 1 - irrespective of any ordering of the At in A. Thus AA'r can: be decomposed to the sum of two par- ticularly simple matrices, i.e., which was proven in Appendix 2 to converge. F (X) 507 a (X) .- Q' 1 + 2:\"-2 I 1 1 This formula may now be re-introduced into (9) to give the following very simple form 1 APPENDIX 1. COMPLET.ENESS' OF THE. d 2 METRIC In the,' parallel' associative addressing of data, 1 XT 1 + 2:\"-2 x,x'r == S'ST (10) 508 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 If one notes that the effect of the matrix operation in the left hand member is merely to tally twice the nonzero elements of X' and XT and that this number is n, then (10) can be reduced and the resulting equation solved for X'X'l' X'XT == 2:-2 8'8 T { - n22x-2 } (11) Finally the distance between X and X' is found to be d 2 == n - 2:- 2 8'8'1' - { n22x-2 } (12) where the right hand term is expressed only in terms of 8, 8' and parameters descriptive of the general characteristics (n and N) of 8-space as was desired. This proves that for some R, in this case R == 2 x , the metric defined on 8 is a complete one, that is to say, that the location of a point in 8-space is unambiguously specified by the 8 vector. First consider the form of the 8 matrix, 8 == X . A. I~ will have integer elements with all integer values 0 thru n being represented. This can be decomposed into the following useful form n 8 L == i == 8(i) (13) 1 where each of the 8(i) is the matrix whose elements are either i or O. This representation of 8 allows one to represent a analytically in the following form. n ~ 8(i) 1 i == (14) 8 where 8 is the threshold value chosen for the logical operation in (6). The basic problem is: given a and a', is it possible to compute the distance between X and X', or in other words is the weak metric a complete in the extended space? Note that the value of the threshold 8 against which a is computed is left unspecified, and hence can be considered as one of the parameters of the problem. Assume 8 to be n; in other words the association vector must have a 1 in precisely every code position in which the X vector has a 1 if the corresponding a element is to be nonzero. As we have noted before, this result is not affected by the presence of extra I's in the association vector. a is now a single term and may be written as a == ~ 8 (n) n (I4-a) It is now possible to compute the number of nonzero elements in a rather simply. If there are n partitions with k j, discrete values possible in the i-th partition, and if the association matrix is the same complete binary matrix discussed before, the number of elements whose value is n in 8 is n cpo (n) == i rr 2 == kI (15) 1 - 1 If we consider another vector with (n - 1) of the partitions identical, with a single element different in a partition possessing k j elements, then the number of ai == n in the same relative positions of the two a vectors is n CP1 (n) == IT i == 2 ~I - 1 X 2 kJ - 2 (16) 1 i # j This argument is simply extended to the case where the two X vectors differ in an arbitrary number, p L. n, of partitions j(p) n CPI' (n) == IT 2 k, i == 1 i#j IT 1 j == 2 kJ-2 j (1) (17) It is possible to express X as a function of a as defined by (I4-a), cpo(n) as given by (15) and A, or more precisely AT. We shall now prove the following theorem. X =[.~n) ]-1 {aAT _ .~n) (1 ... 1) } (18) First consider the term aAT. aL is a one, if and only if, Sl is equal to n, which is to say that the binary cycles of the rows of. A, assumed to be ordered as shown in the previous example for this argument (which correspond to a 1 in the elements of the X matrix) must all be positive, or equal to one in this case. Thus, the operation of matrix multiplication followed by the logical decision (6) is exactly analogous to a double logical extract operation. If we now reverse 509 APPLICATION OF AN ASSOCIATIVEI:.Y ADDRESSED, DISTRIBUTED MEMORY this operation and extra a against AT, i.e., the logical operation n L ajAij == f3j i == (19) 1 The rows of A which contributed to making Sj, pass the threshold test will now be "in phase" with the variations of at; that is they will be positive in all cases when aio is positive. All other rows will assume all possible binary arrangements while these are fixed and consequently will be positive and zero equally often. Thus the "in phase" rows will contribute a 1 for each 1 in a, or what is the same thing, if R is understood to represent a general "in phase" row of A and RT is the corresponding column of A'£ n a . RT L == i == aj, == t 511 Now choose an arbitrary Qo E17"}, and let Ql be the image of Q() under P PQo == Ql Case 1. If Qo E S then == llad IllaH111 lli!ill is the unit vector along the normal from ai PQo == Qu the origin to the hyperplane '7fi. and Q() is a fixed point of the transformation and the theorem is trivially true. k t cannot be specified more tightly without knowing the orthogonal projection of the line segment [Qi, Q~] on the intersection of '7fi and '71"1+1. Case 2. Let Qo E17"1 ; Qu E S then Qo =F Ql Proof: Since T i , i +1 is a linear transformation we may write IITJ• J+lQi -.- T j, j+lQJIIIITj , j+1 T j-l, jQ~-1 - T j , j+lTj - 1 , jQt- 1·11 (26) etc. for j steps in all ; where Qt. Q~ E'7rj etc. However; stopping with the step shown in (26) and using (25) we may write IITQi -TQ~II == kjllQI - Q~II =::: kj i ITQ~-l - T~-lll j = "IT i == kiilQl 1 QolI (27) where The R hyperplanes are considered in a cyclical order; therefore on the R + I-st projection the images will be in '7fh and from (26) and (27) we have: IIPQl - PQolI == KIIQl - QolI (28) where R K == IT kl i == 1 P is a projectivity of "'1 onto itself, and as such (as noted in Remark 1) has at least one fixed point, or in general some fixed subspace S such that for every uES Pu==u R IT kl < 1 i =·1 Equation (28) may be expressed as K where the T in the left hand norm is T j , j + 1 and in the right hand norm is Tj-l. j. Now by repeatedly applying this reduction, j times in all, we obtain IIT j , j+l Qi - T j , j+l QAII and at least one kj, =F 1. k j == 1 is equivalent to saying that the norm of the segment [Qi, Ql)] is the same as the norm of the orthogonal projection of [Ql, Q~] on the intersection of 17"j and '7fj +h say Pj' But then for all pj to be 1 would require that the pj be pairwise parallel, i.e., pj parallel to pj +h etc., and Qo == Ql necessarily, in contradiction .to the assumption that Qo =I=- Ql. Therefore at least one kl =F 1 and jjPQj - = PQj-lll== KllQj - Qj-lll (29) where we need not show superscripts on the Qj since all points and their images under Pare in 17"1 alike. Equation (29) may be recursively solved to yield 'IPQj - PQj-lll == IIQj+l-QjllL. Kjll Ql - QolI where (30) Now we wish to determine the Cauchy difference for the sequence {Qj}; IQj+m - Qjll. Using (30) repeatedly we obtain IjQj+m - QjllL (Kj+m-l + ... + kjllQl - + Kj+m-2 QolI after the usual introduction of the missing terms and appropriate regrouping. llQj+m - QjllL. Kj (II-=- ~m) IIQl - Qoll (31) but for a suitable choice of j the right hand side of (31) can be made less than any E > 0; therefore the sequence {Qd converges to a point Q; QE17"l' 512 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, We must still prove that Q is indeed a fixed point of P, i.e., Q E S. IIPQ-QII L IIPQ-PQjll + IIPQj-QII L KIIQ - Qjll + IIQj+l- QII but both of the right hand terms have been shown to go to zero as j increases; therefore II PQ - Q II < E for every E > 0 and Q is shown to be a fixed point of P, or PQ == Q => Q E S Corollary: If the system of hyperplanes defines a determinate system of linear equations with a unique solution point, then this point is the only fixed point under any of the possible projective cycles P, i.e., any reordering of the indices, and the above proof shows that Lim pn Q' == Q n ~ 00 where Q' is any point in any plane and Q is the unique solution point of the system. The foregoing argument completes the convergence proof for the algorithm proposed for filling the associative memory. REFERENCES 1. W. L. McDERMID and H. E. PETERSON, "A Magnetic Associative Memory System," IBjlf Jou1'flal 4, pp. 59-62, 1959. 2. J. R. KISEDA, H. E. PETERSON, W. C. SEELBACH, and M. TEIG, "A Magnetic Associative Memory," IBM Journal 5, pp. 106-121, 1961. 3. R. R.: SEEBER and A. B. LINDQUIST, "Associative Memory with Ordered Retrieval," IB"f,l Journal 6, pp. 126-136, 1962. 4. R. R. SEEBER, "Cryogenic Associative Memory," National Conference of the Association for Computing Machinery, August 23, 1960. 5. R. L. BOYEL, "A Semantically Associated Memory," pp. 161-169 in Biological Prototypes and Synthetic Systems 1, edited by E. E. BERNARD and M. R. KARE, Plenum Press, New York, 1962. 6. B. WIDROW, "Generalization and Information Storage in Networks of Adaline 'Neu- 1964 rons'," pp. 435-461, in Self-Organizing Systems, 1962, edited by M. C. YOVlTS, et al., Spartan Books, Washington, D. C., 1962. 7. McDERMID and PETERSON, OPe cit. 8. KISEDA, et al., OPe cit. 9. SEEBER and LINDQUIST, OPe cit. 10. KISEDA, et al., OPe cit. 11. K. S. LASHLEY, Brain Mechanisms and Intelligence: a quantitative study of injuries to the brain, University of Chicago, Chicago, 1929. 12. F. ROSENBLA TT, "The Perception-a theory of statistical separability in cognitive systems," Cornell Aeror.autical Laboratory Report No. VG-1196-G1, January 1958. 13. F. ROSENBLATT, Principles of Neurodynamics - perceptrons and the theory of brain mechanisms, Spartan Books, Washington, D. C., 1962. 14. F. ROSENBLATT, "Perceptual Generalization over Transformation Groups," pp. 63-100 in Self-Organizing Systems, 1960, edited by M. C. YOVITS and S. CAMERON, Pergamon Press, New York, 1960. 15. B. WIDROW, "Adaptive Sampled-Data Systems-A Statistical Theory of Adaption," 1959 WESCON Convention Record, Part 4. 16. B. WIDROW and M. E. HOFF, "Adaptive Switching Circuits," Tech. Report No. 1553-1, Stanford Electronics Lab., Stanford, California, June 30, 1960. 17. B. WIDROW, "Generalization and Information Storage in Networks of Adaline 'Neurons'," Self-Organizing Systems, pp. 435461, Spartan Books, Washington, D. C., 1962. 18. D. M. Y. SOMMERVILLE, An Introduction to the Geometry of N Dimensions, Dover Publications, Inc., New York, N.Y., 1958. 19. R. V. SOUTHWELL, Relaxation Methods in Engineering Science-a treatise on app't'oximate computation, Oxford University Press, Oxford, 1940. APPLICATION OF AN ASSOCIATIVELY ADDRESSED, DISTRIBUTED MEMORY 22. HERBERT BUSEMANN and PAUL J. KELLY, Projective Geometry and Projective Metrics, Academic Press, Inc., New York, New York, 1953. 23. W. V. D. HODGE and D. PEooE, Methods of Algebraic Geometry, Cambridge University Press, Cambridge, 1947. 513 20. F. S. SHAW, An Introduction to Relaxation Methods, Dover Publications, Inc., New York, 1953. 21. J. W. SMITH, "ADAP II-an adaptive routine for the LARC computer," unpublished report (Navy Management Office) communicated by the author, September 1962. DESIGN OF AN EXPERIMENTAL MULTIPLE INSTANTANEOUS RESPONSE FILE* E. L. Younker, C. H. Heckler, Jr., D. P. Masher, and J. M. Yarborough Stanford Research Institute Menlo Park, California alphabetic code of ~ach search word. If a word is not contained in the dictionary, it is automatically rejected. After all words of the search question have been entered, the document magnetic store is interrogated by the search question in superimposed code form. The comparison between the search word and the document indexes is made for all documents simultaneously and the machine instantaneously determines if any documents in the file include the search question. If there are none, the machine indicates visually that there is no response. If there is at least one, the machine counts the number of responding documents and displays this number. Then it types out the indexes of all responding documents on the same typewriter that was used to ask the question. SUMMARY An experimental model of an electronic reference retrieval file in which all file entries are interrogated simultaneously has been designed and constructed. The experimental model is designed to store and search on a file of indexes to 5,000 documents. A document index consists of a decimal accession number and up to eight English word descriptors that are closely related to the contents of the document. The vocabulary required to describe the documents is held in a machine dictionary that has a design capacity of 3,000 words. In the model delivered to the sponsor, Rome Air Development Center, the storage capacity is only partially used. The specification for the delivered model calls for the storage of approximately 1,100 documents that were selected from the ASTIA (now DDC) Technical Abstract Bulletin and of the vocabulary needed to describe them (about 1,000 words). The document indexes and the dictionary words are stored in wiring patterns associated with arrays of linear ferrite magnetic cores. INTRODUCTION Memories that can be searched in parallel and from which stored information is retrieved on the basis of content have received considerable attention for application to retrieval file problems. 1, 2, 3,4 This paper describes the design of an experimental retrieval file based on the work reported by Goldberg and Green. 3 Since the contents of the semipermanent magnetic memory used in the experimental file can be searched in parallel and multiple responses A search question, consisting of one to eight descriptors in their natural English form, is entered by means of an electric typewriter. During entry of the search question, the dictionary magnetic store is interrogated by the * The work described in this paper was supported by Rome Air Development Center under Contract AF 30(602)2772. 515 516 PRCCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 to the search question are permitted, the system is called MIRF-M ultiple Instantaneous Response File. 5 LOGICAL ORGANIZATION OF THE MIRF SYSTEM The logical organization of the experimental MIRF system is illustrated by Fig. 1. Information pertaining to the document indexes and to the descriptors used in the document indexes is contained in two major units called MIRF units. A MIRF unit is basically a magnetic memory in which information is permanently stored in the wiring associated with the magnetic cores. The Document MIRF is the principal element of the system. It contains fdr each stored document index the document accession number and the descriptors (in coded form) that describe that document, as well as a superimposed search code that is used in the searching process. The Dictionary MIRF has two functions. During the input phase of operation it translates the alphabetic code of the English word descriptor that is entered from the typewriter into the binary serial number assigned to that English word for use inside .the machine. During the output phase, the Dictionary MIRF translates the binary serial number of a word that is obtained during a search into the alphabetically coded form of that word. After the binary serial number of an input English word has been generated, this binary VISUAL DISPLAY YES/NO YES RESPONSE COUNT number is translated by a logical process in the Search Code Generator i~to a search code that is assigned to the particular English word. The search codes of successive words of a search question are superimposed by adding them together, bit by hit by an inclusive-OR operation. When the search question is complete, the superimposed search code of the question is compared with the superimposed code section of the Document MIRF. Each document index whose search field includes the superimposed code of the search question is said to respond to the question. Frequently more than one document will respond. By a logical process for resolving multiple responses,6 the accession number of a particular responding document is generated. Then the binary serial numbers of the English words contained in this document index are generated one at a time. By means of the Descriptor Selector, each serial number is transmitted to the Dictionary MIRF, whe're it is translated to the alphabetic code of the English word. This process is repeated for each responding document. SYSTEM DESIGN Jltlagnetic Implementation of the MIRF Unit The MIRF units of the experimental model use an interesting modification of the Dimond Ring7 translator in which the drive and sensing functions are interchanged. Information is stored in unique wiring patterns associated with an array of linear ferrite cores as il1. ONE COUPLING LOOP AND ONE DIOOE PER FILE ITEM ONE MAGNETIC ELEMENT PER TEST BIT ~ ~ -HrR4-++--_~T_E_M_n_t>I---'1 ___ DICTIONARY MIRF DESCRIPTOR ALPHABETIC CODE SECTION DESCRIPTOR SERIAL NUMBER SECTION DOCUMENT MIRF ACCESSION NUMBER SECTION , SUPERIMPOSED DESCRIPTOR CODE SERIAL NUMBER SECTION SECTION i SEARCH CODE GENERATOR I INPUT-OUTPUT TYPEWRITER ~ " " " " " : :: :: :: :: :: :: "" :::: ITEM 3 -v -~-\--- I + I Figure 1. Simplified Block Diagram of MIRF Experimental Model. TO DETECTOR ITEM I + CONTROL : ITEM 2 DESCRIPTOR SELECTOR ~ ~ " + DRIVE ----4~'--- - PULSE INPUT TEST PATTERN SELECTOR SWITCHES Figure 2. Core-Wiring Arrangement for MIRF Memory. DESIGN OF AN EXPERIMENTAL MULTIPLE INSTANTANEOUS RESPONSE FILE lustrated by Fig. 2. Each item of stored information (a document index in the Document MIRF or a descriptor in the Dictionary MIRF) is represented by a conductor that passes through or around each associated core in a unique pattern determined by the information it contains. In series with each conductor is a diode. The cathodes of many diodes are connected together to form the input to a detector amplifier. Notice that one core is required for each bit of information, but that each core can be associated with a particular bit of many item conductors. Each core has an input winding that can be selected by means of a switch. All cores whose selector switch is closed will be energized when a drive pulse is applied. A voltage will be induced in each item conductor that threads an energized core, but no voltage will be induced in conductors that do not thread the core. A test can be made on the information stored in many cores by selecting a particular set of cares and energizing them. In order for an item to match the test information, its conductor must pass outside of every energized core. Then no voltage will be generated in the item wire and the input to the detector amplifier will be held near ground through the item diode. Voltages will be induced in the conductors of items that do not match the test; the polarity of these voltages is chosen to back-bias the associated diodes. If no item matches the test information, a voltage will be induced in every item conductor and every diode will be back-biased. The input to the detector will then assume a significantly negative voltage. Thus, the presence or absence of desired stored information can be determined by applying the drive currents to a particular set of cores. This is a function of an associative or contentaddressed memory: to indicate the presence or absence of certain information based on the detailed contents of a search question without regard to the actual location (or address) of that information. Now consider in more detail how a bit of information of a search question is compared with information in a MIRF unit. Figure 3 illustrates how a test is made to determine whether or not the test bit is logically "included" in the stored information. This cir- CONDUCTOR # I (ITEM WHOSE K th BIT 517 = ONE) CONDUCTOR #2 ( ITEM WHOSE K th BIT = ZERO) TIMING PULSE FLlP- FLOP '----'-----' HOLDING K th BIT Figure 3. Circuit for Testing Inclusion. cuit is typical of those used in the superimposed section of the Document MIRF. One core is used to store the kth bit of many items. The kth bit of the search question is stored in a flip-flop whose one side is connected by way of an AND gate to a drive amplifier, which in turn is connected to the primary winding of the kth core. The conductor of an item whose kth bit is equal to one (Conductor 1) passes outside the kth core. On the other hand, the conductor of an item whose kth bit is equal to zero (Conductor 2) threads the core. If the flip-flop stores a one, the primary winding of the core will be energized when the timing pulse is applied to the AND gate. A voltage will be induced in Conductor 2 (indicating a mismatch) but none will be induced in Conductor 1 (indicating a match). If the flip-flop stores a zero, the primary winding will not be energized because the timing pulse will be blocked at the AND gate. No voltage will be induced in either conductor, and a match will be indicated on both lines. Therefore, it can be seen that a stored one bit includes both a test one and a test zero, while a stored zero bit includes only a test zero. The circuit for testing for identity between the test bit and the information stored in the MIRF is shown in Fig. 4. This circuit is typical of those used in the alphabetic descriptor portion of the Dictionary MIRF. The jth bit of many items is stored in a pair of cores jA and j B. The jth bit of the test question is stored in a flip-flop. In this case, both the one and zero sides of the flip-flop are connected to AND gates 518 PRCCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 CONDUCTOR #1 (ITEM WHOSE j th BIT = ONE I CONDUCTOR #2 ( ITEM WHOSE j th BIT = ZERO) FLIP-FLOP L--~-' HOLDING j th BIT Figure 4. Circuit for Testing ldentity. whose outputs control drive ampliffers that are connected to the primary windings of the cores jA and ju. The conductor of an item whose jth bit is one (Conductor 1) bypasses core jA while the conductor of an item whose jth bit is a zero threads core jA' The threading of core ju by the two conductors is the reverse of the wiring of core jA' If the flip-flop stores a one, the primary winding of core jA will be energized when the timing pulse occurs. No voltage will be induced in Conductor 1 (a match indication) but a voltage will be induced in Conductor 2 (a mismatch indication). If the flip-flop stores a zero, the primary winding of core J B will be energized. In this case, a voltage will be induced in Conductor 1 but not in Conductor 2. Thus it can be seen that the bit stored in the MIRF must match the test bit identically for a match indication to be obtained. 2. Basic Operations Using the MIRF Units Two types of operations involving the MIRF units are basic to the operation of this experimental model. One operation tests to see if certain information is contained in the MIRF. The other uses information that is contained in the MIRF to generate a number in a flipflop register external to the MIRF unit. Examples of these basic operations are given in the following paragraphs. a. Testing of Information Contained in the MIRF· Unit Dictionary MIRF-During the input of the English words to form a search question, the Dictionary MIRF is tested to see if the input word is contained in the vocabulary (that is, if it is a valid descriptor). This is done by gating the alphabetic descriptor register to the drive amplifiers associated with the alphabetic portion of the MIRF (50 bits long, two cores per bit). As a result, 50 drive amplifiers are energized and 50 primary windings in the MIRF carry current. If one of the stored words has a bit pattern in the alphabetic portion that matches identically the energized set of primaries, the match detector will indicate a match condition. If not, the match detector will indicate a mismatch condition. The output of the match detector is used to determine the next step in the logical sequence. It is important to note that the test is applied to the entire Dictionary MIRF simultaneously and that a match or mismatch signal for the entire MIRF is obtained in about 5 microseconds. Document M I RF-After all words of the search question have been typed, the superposition of their search codes is held in the search code accumulator. At the beginning of the actual search operation, the flip-flops of the search code accumulator are gated to their associated drive amplifiers. A particular set of drive amplifiers is energized and current flows in a corresponding set of primary windings in the 80 bit superimposed code field of the Document MIRF. If the detailed bit pattern represented by the energized primaries is included in any of the superimposed fields of the stored document indexes, a match condition is indicated by the match detector. If not, a mismatch indication is given. The test is made on the entire contents of the document MIRF simultaneously and a YES/NO response is obtained in about 5 microseconds. It should be pointed out that the criterion for a match is inclusion, not identity. A document index includes the search question if the following conditions of the superimposed' search code portion of the index are satisfied. First, for every bit of the index search field that is a one, the corresponding bit of the search question is either a zero or one. Second, for every bit of the index search field that is a zero the corresponding bit of the search question is a zero (in other words a binary one includes both DESIGN OF AN EXPERIMENTAL MULTIPLE INSTANTANEOUS RESPONSE FILE a one and a zero, but a binary zero does not include a binary one). b. Generating Nurabers by the MIRF Process-The generation of the serial number of an input descriptor illustrates this operation. Assume that an English word has been typed in and that the test for valid descriptor is true. Because a match is obtained when the alphabetic descriptor register is gated to the Dictionary MIRF, one item wire in the MIRF is effectively isolated: namely, the wire that is uniquely related to the input descriptor. The detailed wiring pattern of this wire in a group of cores outside the alphabetic code field contains the binary serial number of the input descriptor. By gating the alphabetic descriptor register to the MIRF and at the same time causing cuttent to flow in the primary winding of a core that is in the serial number portion of the MIRF, the binary value associated with that core for the selected line can be determined. The presence of current in the additional winding tests for a binary one in that position. If the match detector indicates a match, the value is indeed one. However, if a mismatch is obtained, the value must be zero. The sequence for generating the serial number is as ioHows: First the flip-flop register that will eventually hold the serial number is cleared to all ones. Then the alphabetic descriptor register is gated to its drive amplifiers and a drive amplifier associated with the parity bit of the serial number is energized. The output of the match deteator is observed. If a match condition is observed, it is known that the parity bit is actually a one and the parity bit flip-flop in the serial number register is not changed. If a mismatch is observed, it is known that the parity bit is zero and the parity bit flip-flop in the serial number register is not to zero. The next step is to energize the drivers associated with the alphabetic descript<1r register and a driver associated with the least significant bit of the serial number. Again the output of the match detector is observed and the flip-flop assigned to the least significant bit is either allowed to stay at one or is changed to a zero. This procedure continues for thirteen steps. At the end of this time, the 12-bit serial number and its parity bit will have been generated and stored in the serial number register. 519 CIRCUIT DESIGN Three principal types of transistor circuits are used in the experimental model: transistors are used as switches to drive the primary windings of the MIRF cores; discriminator-amplifier circuits are used to accept the voltage generated on the secondary windings of the MIRF cores (this is the match detector circuit) ; and transistor logic circuits are used for the over-all control of the MIRF operations. All three types were designed at SRI. 1. MIRF Driver The drive currents that are required by the ferrite cores iIi the Document and Dictionary MIRFs are furnished by circuits such as the one shown schematically in Fig. 5. Four MIRF driver circuits are mounted on one printed circuit plug-in board, as shown in Fig. 6. Each circuit is capable of supplying the required 2 amperes at low impedance. The power transistor that delivers the drive current (Type 2N1905) is driven by a push-pull emitter follower that provides 60 milliamperes of base drive current into 2N1905. The output power transistor has rise-and-fal! time capabilities of less than 0.3 microsecond. The actual current in the load is nearly linear because of the inductive nature of the load and builds up to the 2 ampere amplitude at the end of approximately 10 microseconds. The overshoot voltage induced when the transistor is turned off is clamped by a silicon diode to -36 volts. The clamp prevents excessive voltage spikes from appearing across the output transistor while still allowing the load inductance to recover within 10 microseconds. Two protective features of the MIRF driver circuit should be noted. One is a fuse, which is inserted in series with the load to protect against excessive load currents. Before the winding of the magnetic circuits internal to the MIRF assembly can be damaged by too much current from, say, an accidental short circuit, the fuse wire will open up. The second protective circuit includes a square-loop memory core that is threaded by the lead going to the transistor load. This core is normally biased off, but if the drive current exceeds a safe value the square-loop core will switch and induce a voltage in a sense lead. The voltage in 520 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 necessary for the MIRF discriminating amplifier to differentiate between these two signals and generate a standard logic level output of -6 volts for a mismatch and 0 volts for a match. The circuit for the amplifier is shown in Fig. 7. In order to distinguish between very closely spaced match and mismatch signals, two thresholds are employed in the amplifier. The first threshold is provided by a ,lN3605 silicon diode at the input to the amplifier. This diode does not pass signals unless they exceed approximately 0.5 volt. After passing the first threshold, the signal is amplified in a feedback amplifier with a gain of about 50. If the amplified signal then exceeds the second threshold of 3 volts, a mismatch signal is delivered at the output of the amplifier. INPUT +12. Il'jgure 5. Schematic of MIRF Driver. 3. Logic Circuits "7,1'1,. "5 ... __&ii.-.Q.,Q7. ~ QI,. QS _ _ _ _ _ _ _ _ _ _-Q.Z, QII,GIO,. g' 1It15,1It1" • • 1II13 kl. !;lOll . . . . . , ItU • • I ' " fIl' ........ l..1"TQJI'.MT Figure 6. Component Assembly of MIRF Driver Board. the sense lead is amplified and used to turn off the system clock. The purpose of this circuit is to protect the 2N1905 transistor against excessive heat dissipation from currents that are excessive but not large enough to burn out the fuse wire. In the flip-flop register and over-all control circuits, resistor-transistor logic is used. Highly reliable circuits that operate in the 100-kc frequ'ency range have been developed. The basic gate circuit is shown in Fig. 8. This circuit in typical use performs a simple majority operation. If one or more of its three inputs are at a negative potential, the output is held at ground potential. Since ground is defined as the one state in this system, and a -6 volt potential is defined as a zero state, the basic gate performs the "not and" or NAND operation. All the passive components shown in Fig. 8, plus one resistor and two capacitors, are contained in one physical element supplied by Centralab, Inc. These components are screened on a passive substrate to a tolerance of 3 % for the resistors (5 % design tolerance) and 2. MIRF Discriminating Amplifier The electrical output of the MIRF magnetic modules is generated by a very large diode gate including almost 300 diodes. Under the worst conditions a match signal from this array can reach a level as high as 0.4 volt. On the other hand, a mismatch signal from the same array may only generate a potential of 0.6 volt. It is ..... Figure 7. Schematic of Discriminating Amplifier. DESIGN OF AN EXPERIMENTAL MULTIPLE INSTANTANEOUS RESPONSE FILE -12v 521 circuit. The gate circuit is also used as a preamplifier for an emitter-follower circuit. The basic logic circuits, e.g., gates, one shots, flipflops, etc., are mounted on plug-in logic boards. A typical logic board, with seven gate circuits mounted on a printed circuit board, is shown in Fig. 9. -6.4v 6.aK MAGNETIC DESIGN 6.aK General Ccmsiderations The magnetic design oi a MIRF unit is centered in the individual magnetic core, which acts as a transformer with a multiturn primary winding and many single-turn secondary windings. When current flows in the primary winding, the magnetic core must be capable of producing a flux change of sufficient time duration and amplitude to generate the desired signal in secondary windings. The amplitude of the induced voltage is determined primarily by the characteristics of the diode associated with the secondary winding. The duration of the induced voltage is determined primarily by noise on the secondary winding and the consequent delay required before sampling of the output can be accomplished. 1. 6.aK 51 K +12v - Figure 8. RTL Circuit Designed for the MIRF System. TEST PTS.7,'.5,4,',2,8'" CR'S 7, I. !Io, 4, 3, 2,.1 0'7, Q6. 05, 04, Q3 , Q2 •• aI CENTlU.LA. UNITS It•• '" ALL COIIJI'OII[.T "U • • [.S _aD LI!.TTO.!4IMT ,ItO_ Figure 9. Component Assembly of Gate-Logic Board. 10 % for the capacitors. The substrates are encapsulated with a Durez coating, and are ready for mounting to a printed circuit card via their projecting leads. The gate circuit is a basic part of every logic circuit employed in the machine. By itself it performs the combinatorial function of logical conditions. Two gate circuits properly interconnected form a bistable, or flip-flop, circuit. Two gate circuits interconnected in a slightly different way form a monostable, or one-shot, The cross-sectional area of the magnetic core is proportional.. to the product of the amplitude and duration of the voltage induced in the secondary windiqgs (this is usually referred to as the volt-second area of the induced voltage pulse). This was kept reasonably small by using a high-quality germanium diode (the 1N500) which requires a back-biasing voltage of only 0.6 volt in order to perform. properly in the diode circuit associated with the input to the discriminating amplifier. The circumferential length of the magnetic core is determined primarily by the number of secondary windings associated with the core and the mechanical design of the supports for these windings. In the MIRF units of the experimental equipment, the core has the capacity for 2,000 secondary windings. The core's mean circumferential length is 7 inches; its cross section is a square, 1,4 inch on a side. Two other considerations influenced the selection of the magnetic cores used in the MIRF units. One is the requirement that the core be made in two pieces so that the array of cores 522 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 can be separated into two portions to facilitate initial wiring and changes in wiring. The other is the necessity of using commercially available parts. The number of cores needed in this experimental equipment is too small to justify the design and production of a core . of special size or shape. 2. Details of the DictionaTY and Document MIRF Units The individual cores used are the same for both the Dictionary and Document MIRF. Each core is composed of two V-shaped ferrite structures (Allen Bradley part no. VC 892-141C), which have been specially modified at the factory to permit a maximum of 0.0005 inch air gap in each leg when two such structures are joined together to produce a MIRF core. To drive each core, a twenty-turn primary winding is provided. This consists of two ten-turn windings distributed in such a manner as to minimize the leakage flux and the resulting noise signal (see Fig. 10). The primary winding drives the core from an 18-volt voltage source through a transistor switch driver. The output voltage induced upon each secondary winding is an essentially rectangular voltage pulse having a droop of 0.1 volt in 10 microseconds, from 0.8 volt at the leading edge to 0.7 volt just prior to the trailing edge. The maximum primary current, 0.7 ampere, occurs at 10 microseconds after the beginning of the pulse. To accommodate the expanded capacity of the MIRF document file (5,000 documents) three primary windings will be driven in Figure 10. Details of Primary Windings. parallel, so that a maximum driver current of 2.1 amperes is required. The performance requirement of the magnetic circuits is that consistent and easily separable match and mismatch signals be generated at the diode end of the item wires (see Fig. 2) when a set of primary windings is driven. The design objective was that a maximum match signal of 0.1 volt and a minimum mismatch signal of 0.6 volt should be realized within 1.5 microseconds after the application of the primary drive pulses, and that pulsing of the MIRF cores be repeated for many cycleR at a 50-kc clock rate. To achieve these goah, noise due to ringing and leakage flux had to be minimized. A MIRF unit contains many cores (the Document MIRF has 234 and the Dictionary MIRF has 140), each with a separate primary winding; further, each core is associated with more than a thousand single-turn secondary windings. The secondary windings pass through or around all cores in the unit and so form a long rope. The capacitance between wires in the rope, the inductance of these wires, and the inductance of the primary windings are intercoupled in a very complex manner. In the development of the MIRF units, substantial noise on the secondary (item) windings was experienced due to ringing currents in the primary windings. This noise was reduced to a negligible level by inserting a Type DI52 diode in series with each primary winding and shunting each primary by a 1000 ohm resistor. A lowamplitude noise signal of about 5 Mc, due to inductance and inter-item capacitance of the secondary windings, was also observed. Such noise could be reduced to a very low level by filtering at the input to the discriminating amplifier, but in the experimental system this was not necessary. Noise due to leakage flux must be kept small in order to hold the maximum match signal at 0.1 volt. A secondary wire that represents a match item must pass outside all energized cores. Since in the worst case, 57 cores may be energized, the maximum permitted noise due to leakage flux at each core is less than 2 millivolts (this corresponds to a leakage fl ux of 14of one per cent at each core). In the experi- DESIGN OF AN EXPERIMENTAL MULTIPLE INSTANTANEOUS RESPONSE FILE mental model two methods are used to. reduce leakage flux. One is distributing the primary winding on the cores to compensate for the magnetic potential drop by a corresponding rise in magnetic potential at the points where the drop occurs. As Fig. 10 shows, the winding has a linear spacing except at the points where the air gaps occur; there two turns are closely spaced. The second method uses cancellation of induced voltages to reduce the effect of leakage flux. The common end of many item wires, instead o.f being connected to ground, as shown in the simplified diagram of Fig. 2, is actually connected to a wire that lies in the item wire ro.pe and passes outside of all cores. The voltage induced in the "cancellation lead" at any core by leakage flux is approximately equal to that induced in item wires and is opposite in polarity (relative to the input terminals of the discriminating amplifier). MECHANICAL DESIGN 1. The MIRF Module Implementing the wiring-patterns-on-cores method of storage illustrated by Fig. 2 presented a challenging mechanical design problem. It was necessary that the physical structure containing the magnetic cores and the associated wiring be made in two parts that could be easily separated. It was desirable to. fabricate submodules of wiring patterns, so that the permanently stored information could be changed mechanically in relatively small blocks. 523 are cemented to a l/s-inch-thick phenolic board. Each bobbin carries a ten-turn winding. The windings on pairs of bobbins are connected in series to form the primary winding for one of the magnetic cores. An item tray is a l!J.6inch thick phenolic bo.ard with a field of shallow bobbins that matches the field of coil bobbins. The bo.bbins on the item tray are slightly larger than the coil bobbins, permitting item trays to be stacked up on the coil bobbin assembly. One item tray can accommodate 286 item wires. The diodes that are connected in series with the secondary windings and form the input circuit to the discriminating amplifier are mounted on the edge of the item tray. A MIRF module is assembled by sliding up to seven item trays into position on the coil bobbin assembly. One set of U cores is then inserted into the set of coil bobbins and held in place by a plate with a silicone-rubber pad. The other set of U cores is then dropped into position on the opposite side of the bo.bbin coils. Finally, the top plate (also with a spongy pad) is dropped into position to hold the entire assembly intact. The two sets of U cores are held together under slight pressure from the silicone pads. I I Separate MIRF modules are used to store the information concerning document indexes and dictionary words. In each, the co.res are arranged in a rectangular pattern and are supported by long bobbins. These bobbins are firmly attached to a base structure and carry the primary windings for the cores. A MIRF module is a complete assembly of magnetic cores, primary windings for the cores, and submodules of secondary windings with their associated diodes. The construction of a module is illustrated by the exploded view of Fig. 11. The principal parts of the assembly are the base, or coil bobbin, assembly and the item wiring trays. The. coil bobbin assembly consists of. a field o.f paper bobbins (two per magnetic core) that Figure 11. Exploded View of MIRF Module. 524 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 A complete item tray is shown in Fig. 12. The item wires' start in the upper left corner of the trays, where they are connected to a common bus bar. They pass from left to right in the first row of cores, then back and forth until they emerge in the lower left center part of the tray. The wires then run to assemblies of diodes, where each wire is connected to its own individual diode. The output side of the diodes (the cathodes) are connected together and wired to a small connector, which is seen in the lower left hand portion of the tray. Even though each tray contains detailed wiring for 286 items, only two wires run from the tray to the external discriminating amplifier. Figure 12 also shows a pair of primary coil bobbins with the two U cores inserted. A closeup of a MIRF module with the top plate removed is shown in Fig. 13. The tops of one set of U cores can be seen as well as four item trays. The connectors for the output of the item tray3 can be seen in the lower center part of the photograph. The discriminating amplifier circuits (one for each of the seven item trays that can be included in a module) are located on the circuit board that is mounted in front of t!le magnetic module. 2. Wiring of the Item Trays The item trays in the Document and Dictionary MIRF units store more than one-third of a million bits of information. To ensure the greatest possible accuracy of the wired-in information, two steps were taken. First, the raw data for the documents were computerprocessed to give a set of punched cards that contain the detailed wiring information. Second, a wiring scheme was devised, which presented the detailed wiring information to a wireman in a very simple form, and which included a means of checking the accuracy of the wiring as the wiring was actually done. In this scheme, the path that a wire was to take was delineated by a set of lights in an array of incandescent lamps. Figure 12. MIRF Item Tray. An over-all view of the item-tray wiring equipment (wiring aid) is shown in Fig. 14. The empty wiring tray is placed on the wiring jig in front of the operator. A card is then Figure 13. Close-up of Document MIRF Module (Top Plate Removed). Figure 14. Over-all View of Item Tray Wiring Equipment. DESIGN OF AN EXPERIMENTAL MULTIPLE INSTANTANEOUS RESPONSE FILE placed in the punched-card reader and a pattern of lights is set up in the wiring jig. Number 36 Nyleze wire is taken from a spool through a tensioning device to the top of a special wiring tool (shown in the hand of the operator). The wire from the bottom of the wiring tool is first soldered to the common bus shown in the upper left part of the wiring tray. The tool is then moved along the path specified by the pattern of lights, leaving the wire wound in the desired pattern around the item tray bobbins. Correct wiring at a bobbin is indicated by a light turned on to yellow brilliance. If a light is off, or is on at white brilliance after the wiring tool passes a bobbin position, a wiring error is indicated. 3. Alternative Method of Fabricating Item Trays Alternative methods of preparing wired-in information that may be more easily automated than stringing of small wire have been investigated. One alternative is illustrated by Fig. 15, which shows an item conductor in the form of a metallic path etched on a thin, coppercoated Mylar sheet (half-ounce copper on 2-mil Mylar). It will be noted that the item conductor is connected to a bus at the top of the sheet and to another bus at the botton1. These copper areas are used for connecting the item conductor to the common bus at one and to a diode at the other. This sheet contains one item, but two item conductors could easily be placed on Figure 15. MIRF Item Conductor Formed by Metallic Path on Mylar Sheet. 525 one sheet, one being associated with one leg of the magnetic core and the other with the other leg. The experimental model contains a submodule of 75 items on Mylar sheets. DELIVERED EXPERIMENTAL EQUIPMENT The experimental Multiple Instantaneous Response File System is an all-soljd state equipment. Transistor drive circuits capable of supplying two amperes of current to magnetic circuits, special discriminating amplifiers capable of operating reliably with a poor signal-tonoise ratio input signal, and transistor logic circuits were designed for high reliability, low cost, and moderate speed. About 300 current drive transistors, 2500 logic transistors, 2500 printed gate circuits (a group of 6 resistors, 2 capacitors and their interconnecting wiring on a passive substrate) and 5,000 diodes are used in the system. Except for sequences involving the input-output typewriter, the system oper- ' ates synchronously under the control of clock pulses derived from a 50-kc transistor multivibrator. Figure 16. Front View of Experimental MIRF Equipment. 526 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE , 1964 , Figure 18. Front View of Equipment with Document MIRF Module in Extended Position. Figure 17. Rear View of Experimental MIRF Equipment (Doors Removed). The experimental equipment shown in Figs. 16 through 18 was delivered to Rome Air Development Center in July, 1963. A front view of the equipment is shown in Fig. 16. The main equipment cabinet, the input-output typewriter, and the display and control unit can be seen. Figure 17 shows a rear view of the equipment cabinet with the doors removed. The right hand portion of the cabinet contains logic circuits for control of the system, arranged in modules of plug-in transistor logic boards. The Dictionary MIRF unit is contained in the center portion of the cabinet. Directly beneath the MIRF unit are two modules of drive circuits which provide current to the MIRF. In the left hand portion of the cabinet are the Document ]\IIRF and the transistor circuits for providing drive currents to it. It will be observed that space has been allowed for one additional MIRF unit in the center section and for two additional MIRF units in the left hand section. This is to provide for the expansion of the Dictionary MIRF to 3,000 words and expansion of the Document MIRF to 5,000 document indexes. A front view of the cabinets that house the MIRF units and their drivers is shown in Fig. 18. Here the Document MIRF unit has been pulled out to show it in its extended position. Below the MIRF units the wiring side of the transistor drive modules can be seen. The format of the typewritten record of a search in the experimental model is shown in Fig. 19. The first two lines, "Stanford Research Institute Project 4110," etc., are a manually typed heading for the subsequent search. The . heading was typed while the typewriter was effectively disconnected from the rest of the equipment. The search question consists of three words: "coding," "computers," "digital." This line was also typed manually. The rest of the printout is the machine's response to the search question. Seven documents responded. For each one, a four-digit accession number and the English words that describe the document are printed on a single line. The asterisk prefix on some words have been copied from the ASTIA abstract. It will be observed that the three search words appear in every respondSTANFORD RESEARCH INSTITUTE PROJECT 41Hl' MUL TIPLE INSTANTANEOUS RESPONSE FILE CODING, COMPUTERS, DIGITAL. 0'156 *CODING, DIGITAL COMPUTERS, DATA PROCESSING SYSTEMS, LANGUAGE, 0'2.0'1 RADAR PULSES, RADAR SIGNALS, 'CODING, DIGITAL COMPUTERS, 0'420' DESIGN, DIGITAL COMPUTERS, *LANGUAGE, CODING, ANAL YSIS, 0'540' ~IGITAL COMPUTERS, ERRORS, LANGUAGE, CODING, MATRIX ALGEBRA, fJ727 LANGUAGE, *CODING, *HANDBOOKS, DATA PROCESSING SYSTEMS, DIGITAL COMPUTERS, 0732 DIGITAL COMPUTERS, CODING, TELETYPE SYSTEMS, DISPLAY SYSTEMS, MAPS, 0'824 DATA PROCESSING SYSTEMS, DIGITAL COMPUTERS, OPERATIONS RESEARCH, CODING, Figure 19. Format of Typewritten Record of a Search. DESIGN OF AN EXPERIMENTAL MULTIPLE INSTANTANEOUS RESPONSE FILE ing set of indexes. It should be especially noted that the search words appear in different positions and different order in the different responding documents. This independence of order of the search words and the position of the corresponding descriptors in the document indexes is an important result of the superimposed coding of the search field. CONCLUSIONS From experience with the Experimental MIRF it is concluded that interrogation of the magnetic storage units and the over-all control of the system can be accomplished with reliable circuits of modest complexity. Storage of the document index information in wiring associated with arrays of cores that are physically separable appears feasible; arrays of cores can be separated, submodules of wired information can be changed, and the core arrays reassembled in a reasonably short time. More work on the mechanical design of the magnetic modules is needed, however, to permit easier and faster changing of the stored information. Based on the performance of the experimental model, which contained a file of more than 1,000 document indexes, it is concluded that with the present design a system building block should contain about 5,000 document indexes. It appears that as many as ten such building blocks could be combined in a system whose over-all control is little more complex than that for a single building block. Therefore it is concluded that files of the order of 50,000 indexes could be built with no major changes in the basic concepts or circuits used in the experimental model. Easy communication between a human operator and the Experimental MIRF System has been demonstrated. The machine's response to a search question is essentially instantaneous in terms of human reaction time and the information content of the response is sufficient to allow the operator to start the document search with a general question and to use the information received to define a more specific question. In this way it is possible to home-in quickly on the documents of special interest. Several automatic features of the equipment 527 have proved to be useful. One of these is the capability of accepting a synonym in the search question and automatically translating it into the synonymous descriptor contained in the machine's vocabulary. Another feature is the capability of automatically modifying the search question inserted by the human operator and initiating a new search. For example, if any of the input words have attached to them a "see-also" reference, that see-also reference will be substituted for the original word to form a new search question. ACKNOWLEDGEMENTS The development of the Experimental Multiple Instantaneous Response File was s:ponsored by the U. S. Air Force, Rome Air Development Center, under Contract No. AF 30 (602)2772. The authors wish to acknowledge the contributions of their colleagues at Stanford Research Institute, especially C. B. Clark, D. C. Condon ,and V. Sanford. REFERENCES 1. A. E. SLADE and C. R. SMALLMAN, "Thin Film Cryotron Catalog l\lemory," p.roc. of the Symposium on Superconductive Techniques for Computing Systems, Washington, D. C., May 1960, published in Solid State Electronics, vol. 1, pp. 357-362 (September 1960). 2. J. R. KISEDA, H. E. PETEP..8EN, W. C. SEELBACH, and M. TEIG, "A Magnetic Associative Memory," IBM J. of Res. and Dev., vol. 5, pp. 106-121 (April 1961). 3. J. GOLDBERG and M. W. GREEN, "Large Files for Information Retrieval Based on Simultaneous Interrogation of All Items," Large Capacity M emory Techniques for Computing Systems, M. C. Yovits, Ed., pp. 63-77, MacMillan Co., New York, 1962. 4. M. H. LEWIN, H. R. BEELITZ, and J. A. RAJCHMAN, "Fixed Associative Memory Using Evaporated Organic Diode Arrays," AFIPS Conference Proceedings, vol. 24, pp. 101-106 (November 1963). 528 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 5. E. L. YOUNKER, D. C. CONDON, C. H. HECKLER, JR., D. P. MASHER, and J. M. YARBOROUGH, "Development of a Multiple Instantaneous Response File-the AN / GSQ-81 Document Data Indexing Set," to be published as a Rome Air Development Center Technical Documentary Report. 1964 6. E. H. FREI and J. GOLDBERG, "A Method of Resolving Multiple Respo!lses in a Parallel Search File," IRE Trans., EC-10, pp. 718722 (December 1961). 7. T. L. DIMOND, "No. 5 Crossbar AMA Translator," Bell Labs Record, vol. 29, pp. 62-68 (February 1951). RESEARCH IN AUTOMATIC GENERATION OF CLASSIFICATION SYSTEMS Harold Borko, Ph.D. System Development Corpo'ration Santa Monica, Califm'nia INTRODUCTION time reading the titles and authors of several thousand books before we could find the one for which we were looking. If we knew the author, and the books were arranged alphabetically by author, we could locate the book quickly. On the other hand, if we didn't know the author, but knew the subject content of the book, we would want the books arranged by subject category in order to search the file efficiently. Finally, if all we knew was that the book we sought was a big black one which we could recognize, we This paper is concerned with the organization of information, in the form of documents, for efficient storage and retrieval. By documents we mean books, technical reports, articles, memoranda, letters, photographs, data facts, etc.-all forms of memory file organization ranging from documents in a library to data in a real-time command-and-control system. Therefore, the implications of this work are applicable to a field broader than the concerns of the ordinary library. "'(1<71"'\111,.1 VVVII.A..I.\A. l;trr. t"ha +;lao "' ...... "' .... o"arl hu \""'-J.I.'\JL /">A11"'\'" • .1..1..1.'-'0 U..L..I..'-4t.l..I.,f)'-'\..I. V J .L.1.~~C "'.L~"'" rr"ha nAi .... t ...I.. .1.1.,", .t'V.L.L.L'-I being made is that there are various ways of organizing a file, and whether or not a particular method of file organization is efficient depends upon the search strategy. Furthermore, no one method of file organization would be equally efficient for all search questions. This is an important, if obvious, point and one which is often overlooked. In actual practice, most of the information retrieval research has been concerned with document files because the most highly organized collection of documents in existence today is the library, and in doing research on methods of organizing information, one must compare the adequacy of proposed new techniques with existing library methods. Procedures which will improve information storage and retrieval in a library will probably be sufficiently powerful to help improve other methods of file organization. The central theoretical problem of classification as a method of organizing documents is that only one principle at a time can be utilized for gathering items together. ~his principle can be alphabetic arrangement by author, color coding based on the binding of the book, subject classification, or and other scheme-as long as only one principle is used at a time. PURPOSES OF DOCUMENT I CLASSIFICATION The reason for maintaining a collection of documents is to have an available store of information and to be able to retrieve desired information rapidly and with confidence. The value of classification is that it il}creases efficiency in locating this desired information. If we tried to locate a book on a particular subject in a library that did not use any system of classification, we would have to spend a long Since a document collection is a store of information, it is usually desirable to organize this store according to subject matter. By establishing clearly demarcated groups, or classes, of documents on related topics, the number of documents to be scanned can be reduced to reasonable proportions. This, in essence, is the purpose of classification. A classification system 529 530 PROCEEDINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 is a scheme for organizing a mass of material into groups so that related objects are brought together in a systematic fashion. Objects in one group are selected so as to be more like each other than objects in any other group. However, before this aim can be realized, two questions must be answered: 1) How many classes shall be established? 2) What shall be the measure of similarity and, hence, what is the principle to be used in determining class membership? If one is interested in automatic procedures, one has to answer a third question, namely: 3) What principles of classification are most amenable for use in an automated document classification system? All three of these problem areas are being studied, and some results are already available. DEVISING A CLASSIFICATION SCHEDULE The classification of knowledge is not a new problem. Even in ancient times, man sought to organize information of the world around him into categories for efficient retrieval. What is new, perhaps, is the application of mathematical techniques to the classification problem. The older forms of classification, from ancient times through the Dewey Decimal System, were attempts to impose logical subdivisions on the whole field of knowledge. The surprising thing is not that these systems were imperfect, but rather that they succeeded as well as they did. Melvil Dewey first proposed his Dewey Decimal System in 1876, and it is still in extensive use. Now, as a result of new inventions and accelerated research, the traditional boundaries between the sciences are breaking down. It is time to reexamine the concept of classification, to go back to basic principles and to study the various methods of deriving a classification system. Factor An.alysis-Borko In 1958, Tanimoto12 published a theoretical paper on the applications of mathematics to the problems of classification and prediction. Specifically, he pointed out how the problems of classification can be formulated in terms of sets of attributes and manipulated as matrix func- tions. An actual application of matrix mathematics to the analysis of a collection of documents was made by Bork02 in 1961. The aim of this study was to determine whether it was possible to derive a reasonable classification schedule for a collection of documents by factor analysis,6 a mathematical technique which enables one to isolate the underlying variables in a domain of events. This method has been used by psychologists to determine the underlying variables of intelligence, personality, creativity, ability, etc. In Borko's classification study, factor analysis was used to discover the relationship of key content words as they are used in psychological literatUre. Approximately 600 psychological abstracts were selected for study. These were key punched in their entirety, and by means of a computer program called FEA T9 (Frequency of Every Allowable Term), a frequency count was made of all words, and 90 tag terms were selected for further analysis. These data were arranged in the form of a matrix consisting of 90 terms and 618 documents. A portion of this matrix is reproduced in Table 1. The number in each cell represents the number of times a given word occurred in a particular document. Table 1 shows that the term "child (children)" did not occur in document number 74, occurred twice in document number 307, and three times in document number 374. The term "level (s)" occurred once in documentnumbers 74, 626, and 674 and did not occur in the other documents in the example. --¢a --- --- ---.s ---s::: I;l) r:fJ r:fJ ~ r... Q) r:fJ ~ 0 ~ U Abstract # 74 307 0 1 0 1 0 1 4 1 32~ 575 626 647 653 674 r... ;:g :E til r... ~ c$ .~ ~ btl 0 Q3 '0 ~ I;l) r:fJ r:fJ > ---til '0 0 ~ ~ ~ U ~ ~ p... r:n. 0 1 0 1 0 0 0 0 0 1 0 0 0 1 0 0 1 1 0 1 1 0 1 'I 0 0 1 0 2 2 2 1 2 1 3 0 2 0 0 0 I Table 1. A PortiQn of the Document Term Matrix RESEARCH IN AUTOMATIC GENERATION OF CLASSIFICATION SYSTEMS Based upon the data in the document-term matrix, one can compute the degree of association among the terms as a function of their occurrence in the same set of documents. A measure of this association is the correlation coefficient. This is a decimal number which varies from +1.000 to -1.000. A +1.000 would mean a perfect correlation, namely, that every time word X occurred, word Y appeared in the same document; a zero correlation would indicate no relationship; and a negative correlation would mean that if the' word X occurs in a document, then word Y is not likely to occur. correlation between each of the 90 words with every other word (a total of approximately 4000 correlations), one ~reates the term-term correlation matrix (Table 2). This matrix expresses the actual associations which occurred among selected words in a sample of documents. These statistical procedures, preparatory to the factor analysis, are important in demonstrating a method for translating a conglomeration of words and documents into a set of vectors which can be processed mathematically. Factor analysis, when applied to. the correlation matrix, enables one to determine the basic underlying variables which account for the relationsamong the words as expressed in the vectors. It enables us to mathematically determine which words are related and form a set; these sets, in turn, are interpreted as classification categories for grouping the original sample of documents. The formula for computing' the correlation coefficient is as follows: r x)" == v' NlXY - (lX) (lY) [NlX:! - (lX):!] [N~Y:! - (lY):!] By applying this formula and computing the Ability Achievement Activity Ability ~chievement .272 ~272 -.028 026 1-. [Activity -.028 -.026 IAnalysis .048 .080· -.041 -.002 .119 -.025 IAnxiety &31 Analysis Anxiety .048 .080 -.041 -.002 .119 -.025 ..030 .030 Table 2. A Portion of the Correlation Matrix In the experiment just described, the original 90-column matrix was reduced to 10 vectors which accounted for 62·% of the total, and it is assumed most all of the common, variance. These vectors were then rotated mathematically to achieve a simpler and more meaningful structure of the hyperspace. They were then interpreted by the investigator as ten classification categories into which the original· sample of 618 psychological reports could be groupedand by implication, all psychological literature. To illustrate how the factors were interpreted, let us examine the words which had significant loadings on the first factor. Term # 33 10 70 2 63 Word girls boys school achievement reading Factor Loading .74 .73 .30 .20 .18 There were only five words with significant loading. It is fairly obvious that the concept underlying these terms deals with the achievement of boys and girls in school; consequently, this factor was interpreted as academic achievement of boys and girls in school; consequently, in a like manner. These included factors named 532 PlROCEEDINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 experimental psychology, social psychology and community organization, school guidance and'" counseling, clinical psychology and psychotherapy, etc. Thus, we have arrived at answers to the two questions posed ~arlier in this paper: How many classes should be established, and what shall be a measure of similarity? The application of factor analysis enables one to determine the number of categories which should be established in order to adequately describe a given sample of documents. Furthermore, it provides a statistical technique, or principle, for measuring the similarity of content based upon the co-occurrence of key content terms. There are many questions still to be answered before one can decide on the usefulness of this technique for classification. These questions include: 1) Are the categories stable; do they hold from one sample of psychological literature to another? 2) Are the categories valid; can all documents be reasonably classified into these categories? 3) Are the categories useful; do they lend themselves to automated document classification? 4) Is the technique a general one; can it be applied to documents other than psychological reports? Before reviewing the studies designed to answer these questions, it would be well to first examine some other mathematical techniques for deriving classification schedules. Clump T heory-Parker-Rhodes and Needham At the Cambridge Language Research Unit in England, Parker-Rhodes was also inter~sted in classification theory ,and a mathematical basis for forming classes of documents. Interestingly, he considered the use of factor analysis but rejected it on two grounds, one theoretical and the other practical. From a theoretical point of view, Parker-Rhodes claimed that "the statistical type of technique has its place only after we have discovered whatever classification there may be. For then it is up to the statis- tician to say how nearly the properties of particular elements of the universe are inferable from a statement of the classes to which eacp. belongs .... This is quite a different enterprise from that of finding the classes themselves"lo (page 4). On the practical side, factor analysis is rejected as being incapable of handling "really large universes." Having decided to avoid the statistical concept of determining the probability of class membership, Parker-Rhodes restructured the problem in terms of locating clumps "in a Boolean lattice representing all possible subsets of the universe." Within a Boolean lattice there are many ways of defining clumps, and in fact, many different clumps are defined. Without getting involved in details, it can be broadly stated that "members of a clump must be more like each other, and less like non-members, than elements of the universe picked at random" (page 9). Thus we see the relationship between the theory of clumps and the theory of classification. The method used for locating clumps within the lattice remains to be worked out. Initial procedures for clumping are described by Needham. s Research aimed at improving and testing these procedures is still going on. However, even now these techniques have been applied to a 346 x 346 matrix which is beyond the capabilities of presently available factor analysis programs. Latent Cla,88 Analysis-Baker The similarity between document classification and the problems inherent in the analysis of sociological questionnaire data was recognized by Baker. He then proposed an information retrieval system based upon Lazarsfeld's latent class analysis.! As Baker points out, "The raw data of documents, the presence or absence of key words, is amenable to latent class analysis without modification of either the analysis or the data. The latent classes and the ordering ratios yielded by the analysis provide the basis for a straightforward means of classification and retrieval of documents" (page 520). The latent class model assumes that the population-that is, the number of documents in the sample--can be divided into a number of mutually exclusive classes. Usually the number of RESEARCH IN AUTOMATIC GENERATION OF CLASSIFICATION SYSTEMS classes is determined by the investigator, although it is conceivable that this parameter can be determined mathematically. One starts by selecting the key words which characterize each class of documents. Then latent class analysis is used to compute the probability that a document having a certain pattern of key words belongs to a given class. Each of the 1000 documents are then analyzed to determine whether they contain one or more of the four terIl)s. Sixteen (24) response patterns are possible, ranging from ++++ to 0000. A x2 test enables one to estimate the latent structure from the observed data. Having obtained a latent structure which fits, one can compute an ordering ratio, which is the probability that a document having a given word pattern belongs to a particular latent class. For example, a document with all four key words present has a probability of .998 of belonging to class 1, i.e., it is concerned with computer automated instruction. Baker gives the following example: Let us assume that we have 1000 documents in our file. We are interested in classifying these documents into two classes-those dealing with computer automated instruction and those not directly related to this topic. We select as the key words in our search request the following: 1. computer. 2. automated. 3. teaching. 4. devices. Table 3 shows the relationships between the response pattern, expected frequencies, and ordering ratios of the 1000 documents analyzed, in terms of their latent class structure. Expected Frequency Response Pattern Class 1 I Total Fitted Ordering Ratios Class 1 Class 2 158.76 .24 159.00 __ 1___Of) RA. 216 -=-- lOR 00 ---=-- 68.04 68.04 17.64 45.36 45.36 11.76 29.16 7.56 7.56 19.44 5.04 5.04 3.32 2.16 .96 2.16 .56 8.78 19.44 5.04 8.60 2.39 5.04 77.76 20.16 45.36 20.16 181.20 69.00 70.20 18.20 54.14 64.80 16.80 37.76 9.95 12.60 97.20 25.20 50.40 23.48 183.36 t-~ r Class 2 ! I ++++ +++0 ... -++0+ +0++ 0+++ ++00 +0+0 0++0 +00+ 0+0+ 00++ +000 0+00 00+0 000+ 0000 533 .998 9RO - - =~ .986 .969 .969 .838 .700 -.700 .772 .768 .600 .200 .200 .100 .142 .012 .002 020 e- _ _ _ .014 .031 .031 .162 .300 .300 .218 .232 .400 .800 .800 .900 .858 .988 Table 3. Expected Frequency of Response and the Ordering Ratios Based Upon the Estimated Latent Structure l The table readily reveals the applicability of latent class analysis for information retrieval. This application is still in the theoretical and experimental stages. It has yet to be tested with empirical data from actual files. AUTOMATED DOCUMENT CLASSIFICATION The preceding discussions of factor analysi~ clump theory, and latent class analysis all dealt with methods for devising empirically based 534 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 classification categories. These and other researchers have been investigating mathematical methods for deriving classification categories because of their belief that empirical classification systems will provide a more efficient means for the classification and the retrieval of information than the traditional methods of document classification. This belief has been subjected to scientific tests and evaluations. In a study by Borko and Bernick,3 an attempt was made to test the hypothesis that a classification system derived by factor analysis provides the best possible basis for automatic document classification and would result in more accurate automatic classification of documents than would be possible using more traditional classification categories. -Maron,7 in pursuing his interests in automatic indexing and classification, worked with 405 abstracts of computer literature which had been published in the IRE Tramactions on Electronic Computers, Volume EC-8. In essence, Maron proposed a set of 32 subject categories which he felt were logically descriptive of the computer abstracts. Then he selected 90 clue words in such a manner that they would be good predictors of his 32 categories. The 405 documents were divided into two groups-260 abstracts made up the experimental group and the remaining 145 comprised the validation group. Maron classified all 405 documents into the 32 categories. Working with the documents of the experimental group only, he computed the value of the terms in the Bayesian prediction equations. He then used this formula to automatically classify the documents into their categories. Automatic document classification was correct in 84.5 % of the cases in the experimental group and in 51.8 % of the cases in the validation group. Borko and Bernick decided to test the hypothesis that a higher percentage of correct classifications could be made using the same set of documents if a factor-analytically derived classification system were used instead of Maron's logically derived categories. However, their results were approximately the same as those obtained by Maron. Because of the nature of the experimental design used, it was impossible to determine whether the difficulty lay in the mathematically derived classification sys- tem or whether the factor score method used to predict correct document classification was not as effective as the Bayesian prediction equation. Another series of experiments were designed and executed. 4 It was concluded from this series that, while there was no significant difference between the predictive efficiency of Bayesian and factor score methods, automatic document classification is enhanced by the use of a factoranalytically derived classification schedule. Approximately 55 % of the documents were automatically and correctly classified. While this 551'0 current automatic classification of the documents is statistically very significant, it will have little practical significance until greater accuracy can be demonstrated. Up to this point the criterion for correct classification has been the human classifier, but this is not necessarily the best criterion. We know that humans are not perfectly reliable, and therefore, it is not possible to predict human classification with perfect accuracy. The ultimate criterion of the usefulness of any indexing and classification system is whether it retrieves relevant information in response to a search request. Automatic document classification procedures should be evaluated on how efficiently they retrieve information and not on how well they can match the imperfect human classifier. This is a much more difficult problem, but research is already under way to evaluate the retrieval effectiveness of automatic document classification. As work progresses on the evaluation and improvement of techniques for automatic document indexing and classification, it can be anticipated that the bottleneck which now exists between the collection and the processing of documents will be eliminated and automated storage and retrieval systems will become possible. BIBLIOGRAPHY 1. BAKER, F. B. Information Retrieval Based Upon Latent Class Analysis. Journal of the Association of Computing Machinery, Vol. 9, No.4, Oct. 1962,512-521. 2. BoRKO, H. The Construction of an Empirically Based Mathematically Derived Classification System. Proceedings of the RESEARCH IN AUTOMATIC GENERATION OF CLASSIFICATION SYSTEMS Spring Joint Computer Conference, San Francisco, May 1-3,1962, Vol. 21, 279-289. (Also available as SDC document SP-585.) 3. BORKO, H., and BERNICK, M. Automatic Document Classification. Journal of the Association of Computing Machinery, Vol. 10, No.2, April 1963, 151-162. (Also available as SDC document TM-771.) 4. BORKO, H., and BERNICK, M. Automatic Document Classification: Part II-Additional Experiments. Journal of the Association of Computing Machinery, Vol II, No.2, April 1964. (Also available as TM771/001/00.) 5. BRITISH STANDARDS INSTITUTE. Guide to the Universal Decimal Classification (UDC). British Standards House, London, 1963. 6. HARMAN, H. H. Modern Factor Analysis. University of Chicago Press, Chicago, 1960. 535 7. MARON, M. E. Automatic Indexing: An Experimental Inquiry. Journal of the Association of Computing Machinery, Vol. 8, No.3, July 1961, 407-417. 8. NEEDHAM, R. M. The Theory of Clumps, II. M. L. 139, Cambridge Language Research Unit, Cambridge, England, March 1961. 9. OLNEY, J. C. FEAT, An Inventory Program for Information Retrieval. SDC document FN-4018, July 1960. 10: PARKER-RHODES, A. F. Contributions to the Theory of Clumps, M. L. 138, Cambridge Language Research Unit, Cambridge, England, March 1961. 11. SHERA, J. H. and ·EGAN, M. E. The Classified Catalog: Basic Princivles and Practices, American Library Association, Chicago, 1960. 12. TANIMOTO, T. T. An Elementary Mathematical Theory of Classification and Prediction, IBM, New York, 1958. INFORMATION STORAGE AND RETRIEVALANALYSIS OF THE STATE OF THE ART G. N. Arnovick, J. A. Liles, and J. S. Wood Information Storage and Retrieval Systems Systems Engineering Space and Information Systems Division North American Aviation, Inc. INTRODUCTION In anticipation of the rather broad category of readers to whom this paper is directed, an attempt is made to reasonably satisfy those with considerable background in IS&R as well as those with less familiarity. Consequently, an attempt is made to avoid laborious and detailed descriptions of coordinate indexing, storage media, etc. At the same time, it is recognized that some readers may, for example, be technically expert in hardwar~ but require some familiarization with the specific software notions of IS&R. It is emphasized, therefore, that the central aim is to present a critique rather than to educate or merely inform. Information retrieval, like the weather, stimulates a good deal of verbal clamor and speculation, yet remains vexingly elusive and unmanageable. Hopefully, both the weather and recorded information will eventually prove amenable to some form of human control. In the mea:ntime, there must be a continuing effort to achieve balance in the evolution of the concept and equipment aspects of information storage and retrieval (IS&R). It is reasonable to expect overemphasis on equipment capabilities. The emergence of IS&R as a distinct discipline is largely attributable to the significant advances of modern computer technology. These considerations necessitate inclusion of the somewhat fundamental (though cursory) earlier sections on hierarchic and coordinate indexing. They should be ignored or quickly scanned by the informed reader. Those who have been moderately exposed to the subject should be interested in the sections on Problems of Coordinate Indexing and, to a lesser extent, on Relationships in Coordinate Indexing. The descriptions and evaluations of probabilistic and automatic indexing should be useful even to those with extensive documentation background. Nevertheless, steps must be taken to correct the deficit in systems and concepts, or the situation will be analogous to a surveyor pacing off chains with a precision micrometer. The purpose of this review is to assess present capabilities in the field and the extent to which these capabilities are effectively utilized. However, in view of the clamor that has already been made about IS&R, the inclination here is to avoid a massive item-by-item listing of available systems and techniques. Instead, general classes of the operations and equipment involved in IS&R are summarized and evaluated. Specific examples are cited for illustration. The greater portion of the review is devoted to a discussion of indexing for this is considered to be the most important single factor in IS&R. 537 538 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 For example, it is impossible to embark upon an exposition of coordinate indexing without touching on the coordination problems that develop at the retrieval end ("false drops" as one instance). Considerable space in the section on indexing is in fact dedicated to the problem of coordination failures. INDEXING The schemes for indexing documents are multifarious and growing. There are any number of ways in which all the existing and presently conceivable schemes may be organized, but the plan in Figure 1 seems convenient for obviating some of the ambiguities and redundancies usually encountered. Kent 19 gives the following definitions of document and aspect systems: The last part of the section on indexing touches lightly on the subject of storage media to convey a feeling for the ways in which indexes are made readily available to the browsing inquirer. Document systems may be defined as those information systems that involve the recording of all characteristics concerning a single document on one record or, less commonly, on a single discrete set of records. The remaining aspects of information handling-storage and retrieval, abstracting, dissemination, reproduction and display, and communication links-are treated more summarily. Conclusions are presented at the close of the review. I HUMAN DECISION BASED ON INSPECTlON OF DOCUMENT OR ABSTRACT MANUAL 1 INDEXING I r I Aspect systems may be defined as those information systems that involve the recording, 1 1 [ MECHANIZED --.. I J I I 1 BIBLIOGRAPHIC, INDICATIVE (DOCUMENT RETRIEVAL) CONCEPT, INFORMA TlVE (DATA RETRIEVAL) I I BROAD TOPIC INDEX I AUTHOR INDEX SPONSORI NG ORGANI ZA TI ON INDEX REPORT NUMBER INDEX CONTRACT NUMBER INDEX CHRONOLOGICAL INDEX ------"- I 1 NONHIERARCHIC (COORDINATION) I 1 I SUBJECT (A PRIORI SUBJECT HEADS) USES PREASSIGNED STANDARDIZED SET OF KEY TERMS 1 I etc 1 HIERARCHIC (CLASSIFICATION) etc . • . • LIBRARY OF CONGRESS DEWEY DECIMAL UNIVERSAL DECIMAL BLISS CUrrER FACETED CLASSIFICATION STATISTICAL PROBABILISTIC AUTD-ENCODING, etc SEMANTIC, SYNTACTIC ANALYSIS THESAURUS, SEMANTIC CODE, AUTHORITY LIST DESCRIPTORS ASTIA PAL etc Figure 1. Breakdown of Indexing Schemes. KEYWORD (A POSTERIORI SUBJECT HEADS) USES WORDS THAT OCCUR IN TEXT; MAY RELY ON SYNONYMS, "SEE" REFERENCES, RELATIONSHIPS (LINKS, ROLES, etc) I UNITERM etc (IN GENERAL THE SUBJECTS ARE CREATED BY WORDS IN TEXT; ALMOST EVERY SYSTEM HAS SOME AUTHORITY LIST) INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART on one record or a discrete set of records, of the numbers (or other unique identification) of all documents that have a characteristic ( or aspect) in common. In general, indexing systems that locate specific information are characterized by greater depth of indexing than systems that merely locate documents. But increasing the depth amplifies the concomitant problems by orders of magnitude. Indicative indexing is a working reality; information indexing, at least where a large general store of information is concerned, is a monster whose powerful potenti~n needs to be harnessed. Almost all present working systems still rely on human indexers, even if all other aspects are automated. CONVENTIONAL SUBJECT CLASSIFICATION In conventional subject classification, all documents and books are filed in a given slot according to an established, predetermined hierarchy of subjects. This system is poorly suited to a modern technical library for the following reasons: Lookup is a two-step operation involving a search for documents under a subject head followed by a search of subjects under each document to choose those which are pertinent. Overlapping of disciplines is increasing. The process of interpolating new terms (updating) in a rapidly developing subject is cumbersome. A conventional universal classification scheme may include the entire store of an aerospace-oriented library in one small corner of its structure, whereas aerospace may span a colossal and varied coHection of highly overlapping topics. Most hierarchies are ai'tificial. The natural structure of biology and chemistry does not extend to all knowledge. Extensive cross referencing alleviates the confinement of one heading to a reference, but it burdens the user and, carried to extremes, renders chaotic the already synthetic hierarchies. 539 FACETED SUBJECT CLASSIFICATION Faceted classifications attempt to introduce greater flexibility by permitting the free combination of various categories or facets. Subj ectsare not arranged in a fixed descending order but are combined in an order prescribed according to classes. The chief advantage in addition to flexibility is that the classification may be adapted to the information that is being classified. Nevertheless, a document is permanently pigeonholed, and most of the inherent disadvantages of subject classification are still present. In fact, faceted classification fails in general because it does not solve the problems of the searcher. If the hierarchy is not a natural one (and most are not), it represents a point of view. This system is suitable for well-established subjects and, hence, for the filing of certain (e.g., reference) books. Enough time has elapsed for" the user to have become familiar with the indexing point of view. However, the outlook of the searcher and the order of the index may be completely disparate in considering a current, complex subject. HIERARCHIES AND RETRIEVAL EFFICIENCY To better understand the problem of classifying knowledge in rapidly evolving fields, the following example is cited. Even if retrieval is lOO-percent effective, problems still remain. Herner and Herner 16 found in a recent study involving atomic energy materials that the coincidence between the content of research reports and reference questions in atomic energy is very small. Further investigation revealed that once reports have been used as sources of current information, their significance falls away rapidly. These results powerfully emphasize the fact that the time available for matching viewpoints is essentially zero. COORDINATE INDEXING A natural consequence of the development of high-speed computers was the attempt to find new ways of processing data including documented information in machine-tractable form. One reasonable approach was to question the 540 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 continuing necessity for creating artificial relationships between subjects as in hierarchic classifications. Every document defined its own subject. How then could the relationships between concepts and data be determined by the actual environment in which they were embedded? The discussion that follows treats the concept of coordination as originally conceived. It presupposes few or no a priori associations between concepts. Certain significant words or data are taken from the text of a document, and the searcher is then free to combine any number of such key terms,as he chooses in the attempt to match, not viewpoints, but what he actually has in mind with what the author is actually talking about. This approach is the ideal. The ways in which it can be implemented and the extent to which the ideal is approached in practice need to be examined. The methods of implementation and storage techniques and will be discussed under that subject. The ideal is compromised when associations are made between index terms. In a recent issue of American Documentation, it is argued that the introduction of 'relationships is tantamount to denial of Uniterm indexing and return to subject heading classification (attributed to Cutter in 1876).2 Certainly lexical and grammatical ties play an important part in information content. But considering the expense and state of the art of linguistic analysis,4 the attitude is that coordinate indexing (no relationships) and subject heading classification (inflexible with respect to association) are at opposite poles; something flexible and capable of expressing relationship lies between. The general concept of coordinate indexing is well understood and will be sketched only briefly; it involves three basic steps: 1. Accession numbers are assigned to the documents in the store. Consider 10 documents lettered A to J comprising a closed store. 2. From each document, the indexer picks out a representative number of wordtokens (single occurrences of a word in a document) and assigns a code number to each word (a given word distinct from any other; it may occur in several docu- ments). Assuming emergence of a total vocabulary of 30 numbered words, the result may be depicted as shown in Tables 1 and 2. Table 1. Documents Ordered by Term Number (Serial List) Document Number Words IA BCD E F G H I J CD 3 5 3 16 8 21 7 11 28 2 6 7 12 17 11 22 CD 8 4 3 4 CD 13' 2 19 23 8 2 29 8 14 18 20 13 24 CD 30 4 25 CD 15 9 9 5 26 18 10 15 19 11 27 In this listing, the 30 words are represented by 52 word tokens. 3. The list is inverted by grouping all document accession numbers under a given word. For example, word number 1 (circled) occurs under document letters A, C, H, I, and J. The inverted list looks like this (for the first 10 words) : Table 2. Documents Ordered by Term Number (Inverted List) Word Number Document Number 1 2 3 4 5 6 A A A A A B @E B B C ® I D J I J 7 8 9 10 ©© ®® C D c F I A search defined in terms of words 1, 7, and would be found in documents C and H, which have these numbers in common. Problems of Coordinate Indexing Indexer Variability The practical difficulties incurred by straight coordinate indexing arise primarily because the effect of viewpoint can never be eliminated. It will occur wherever decisions are made, and INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART indexing at the present stage is largely a decision-making process. In a superficial experiment conducted at Space and Information Systems Division of North American A vaition, the IS&R group selected a series of documents and compared the indexes produced by different people (none of whom, however, was a trained indexer). The variations were enormous. Without attempting to generalize the few results of this test, the following trends were noted: Thos'e who indexed documents within their own specialization tended to read (rather than scan) the document and pick many more than the average number of terms. Those with technical background who had not specialized in the field tended to be more selective and pick terms that seemed to be emphasized either by frequency of occurrence, placement in quotes, or other attention-drawing devices of the author. Nontechnical people tended to pick a fair number of terms, chiefly selecting those which looked "technical" and often ignoring "common" terms with a meaning specialized in the particular documents. Certain individuals consistently pick few terms, others pick many. Citing the most extreme example, a document on mathematics was indexed by one person (a mathematician) with 52 terms and by another (an aeronautical engineer) with 5 terms. There was no effort to evaluate document relevancy as a function of the resultant indexes. Extensive formal studies of this kind are being made at Documentation Incorporated (indexing reliability tests and study of teaching/learning aids) 6 and IBM (study of effect of educational background on encoding performance) .31 Savage reports no significant effect of educational level on ability to index. 1s Clearly, the human factors in manual indexing need to be explored more fully before effective automation of the process can be achieved. Specifically, indexing techniques and term selection need to be assessed in terms of retrieval effectiveness. 541 Search Variability-Coordination Failures A second problem is the variability with which the searcher (or requester) can express his inquiry. Often it happens that he is unable to express it. A document on germanium rectifiers might be overlooked in a search for germanium diodes (near-synonym) or for germanium transistors (inclusive class). Conceivably, a requester might even verbalize his inquiry as "a germanium thing that permits current to flow in one direction but not in the other." This problem is one of a group that is categorized as coordination failures. These are best portrayed by example. Returning to the set of documents, A to J, and their filial terms, 1 to 30 (Table 1), the following specific topics and words are assigned: DocumentE on: sodium chloride, potassium chloride, and potassium iodide Document G on: air-to-ground missiles Word no.: 2-potassium I6-sodium 17-chloride IS-iodide 2 I-air 22-ground 23-missiles Table 3 lists the types of coordination failure with examples of searches in which they might occur. The examples are purposely trivial for the sake of illustration. Relationships in Coordinate Indexing There are some purists who feel that any presumption of relationship between index terms is flatly a reg-ression to the construction of hierarchies with all of their concomitant 542 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 vexations, subjectivity, and artificiality. Others insist that there must be an ideal marriage of coordination and classification to produce optimum communication between the information store and its user. Sensibility lies with the middle group that comprises not only the greatest number of documentation practitioners (those actually engaged in meeting the information needs of scientists and engineers) but also some of the most advanced thinkers in the field. Table 3. Examples of Coordination Failures Using Pure Coordinate Indexing (Selection and Juxtaposition Only of Terms Occurring in Text) Type of Coordination Failure Example Request Search Terms 1. False coordi- a) Ground handling Ground (22) nation of missiles Missiles (23) b) Sodium iodide 2. Incomplete coordination Air-to-air missiles Sodium (16) Iodide (18) Documents Retrieved Reason for Failure G (air-toground missiles) Insufficient depth of requesteasily rectifiable by narrowing specificity of request. E (sodium and potassium -chloride, potassium iodide, but NOT sodium chloride) Need to show stronger relationship than mere co-occurrence or juxtaposition. Possible solutions (links, roles, etc.) are discussed under relationships. Air (21) G (air-toMissiles (23) ground missiles) The search terms are sufficient to describe the request but insufficient to describe document G, which is nevertheless retrieved. A meager solution is the use of logical negation (air AND NOT ground), but the requester or searcher must be aware of possible exclusions. 3. Synonym failure Table salt Table salt NONE, but document E is relevant Index does not provide for synonym "table salt-sodium chloride." A thesaurus or subject authority list is needed. 4. Generic search failure All documents on alkali halides Alkali halides NONE, but document E is relevant Index does not provide for generic name of chemical compounds, i.e., "alkali halides NaCI, KCI, KI." A thesaurus or authority list is required. 5. False relationship Ground-toair missiles Ground (22) Air (21) G (air-toground missiles) Exact matching of terms (i.e., no absent terms, no superfluous terms), but a stronger relationship than mere cooccurrence or juxtaposition must be shown. INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART 543 Table 3. Examples of Coordination Failures Using Pure Coordinate Indexing (Selection and Juxtaposition Only of Terms Occurring in Text) (Cont.) Type of Coordination Failure 6. Failure to retrieve "next best" documents Search Terms Documents Retrieved a) (Refer to Table 1) 3,6,4,31,32 NONE. The requester, however, might find it advantageous to examine document B, which has three of the search terms (3,4, 6) b) (Refer to Table 1) 3,31,32 still NONE Example Request , , , Reason for Failure No facility for pulling documents with at least some of the search terms (optimally, the most terms) ; lack of browsing capability. Same problem as above; merely shrinking the request does not necessarily improve the retrieval. If a searcher began a process of successive elimination of terms, in the ,given example he could make, a~ _... m~ny -- --- ~~ -- (\ ~ 1 ) ~I (\ ~ 2) - 1h --LU attempts before getting a "hit." This. would only be practical on computers and even then is not the most efficient approach. P hysical Proximity A primitive kind of association between words in a text is the distance in terms of words that separates them. This association is thought by some to be useful in machine translation as a word and its dependents tend to be grouped together, but it seems to have little application to indexing. It could serve a useful analytical function if index terms were tagged according to the part of a document in which they occurred, such as title, abstract, and main body. The relative importance of these parts could then be analyzed. A related study has been made by workers at IBM. Resnick30 and Savage report that the results of indexing from an abstract and from the title of a document are not significantly different. It should be realized, however, that the IBM indexing studies are based on a comparison of indexing terms with keywords supplied by users of a Selective Dissemination of Information (SDI) System. A certain bias is thus already inherent in the tests. Synonyms Words with similar meanings can be handled in the input by using "see" references. In a descriptor system, all words considered near enough in meaning are mapped into a single term (descriptor) via a subject authority list or they are referred to a thesaurus. The process can be done by the human searchers or, prefer- 544 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 ably, can be handled entirely within the system dictionary. If the system is to serve its users with a minimum of inconvenience, the constraints imposed on the uS'ers and indexers must be reduced to a minimum. Humans should be free to make decisions (formulate requests, select keywords). The system should unburden them of the processing and matching operations. Partial Implication or Near-Synonyms This problem is much like the preceding one except that now provision is made for generic groupings of specific terms and specific filials of generic terms. These are usually handled by "see also" references. In other words, some documents are filed under the given term but other potentially relevant documents may be found under related terzns. This technique inGorporates into the system the main advantages of hierarchic arrangements. Semantic Relationships As a rule, the association of index terms should not be stressed too heavily. It is sometimes helpful to show directional relationships in answer to the question "Does A affect B, or does B affect A?" For example, the terms "diffraction" and "ultrasonic" could pertain either to ultrasound used to diffract light or to the diffraction of ultrasound. Even the expression "utrasonic diffraction grating" can be ambiguous in this respect. The occurrence now and then of such false or ambiguous correlations led to the creation of associative links that connect various terms from a document in one group, other terms from the document in another group, and so on. To further clarify the meanings involved, semantic roles were devised to delineate the function of a given term within a given document. 12 The foregoing ultrasonic diffraction grating problem could be handled nicely by assigning "ultrasonic" a role indicator telling whether it is "acted upon" (by an acoustic grating) or "acting upon something" (light). A present working system that uses roles and links extensively is the one serving the American Institute of Chemical Engineers.26 However, such devices are best suited to such highly structured disciplines as chemistry since it is important for the chemist if he is interested in a given chemical as a catalyst not to be bothered with documents on this chemical as a reagent, dye indicator, or insecticide. Another device is to join words inseparably as bounds terms (precoordination). A noun and its descriptive adjective might be treated as a single concept. This has both good and bad consequences. It requires a well-tutored indexer and means that a search for one of the terms will not, in general, pull a document indexed with the bound term. In particular, however, it is useful for concepts such as chemical compounds (especially in a chemical library) or the occasional ambiguous term like air-toground missiles. It is beneficial for pairs of terms that always coordinate. Relationships between words are invoked in the hope of decreasing false coordinations. But an exaggeration of their importance is not desirable, for they can simultaneously exclude relevant documents. A good system will combine the various types of coordinate indexing, avoiding extremes but using relationships sparingly. ProbabiZ'istic Indexing Indexing, like searching, may be thought of as a binary operation. Every word in the text of a document is either chosen as an index term or it is not. In a search, a document is either picked (assumed relevant) or it is not. This means that when a request is made, the user has reason to be wary of the output. He may get a handful of documents, most of which will hopefully be relevant to his request. Or he may be handed a list of several hundred accession numbers. He may then undertake the project of scanning the documents or narrow his request. Then he may exclude documents of interest. All this because the system called every document either good or no good. Probabilistic indexing, proposed by Maron and Kuhns,24 provides for the assignment of weights to index terms and the ranking of retrieved documents according to some criterion of relevance. Instead of being chosen on a zeroone, go or no-go, basis, the words of a document are weighted by the indexer according to their significance to the documents. Maron and Kuhns, in their own experimental work, use an INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART eight-level rating, i.e., 0, :Is, %, .... %, 1. Relationships are also introduced, but on a purely statistical basis relying on the frequency of occurrence of word pairs in all the documents of the collection. The most promising one of several mathematical devices used to relate the word pairs is a coefficient of association. The coefficient of association is, as described by the authors, essentially a measure of the excess of joint occurrences of a given pair over expecta tion based on random co-occurrence. An alternative to Maron's "excess" was presented by Stiles at the recent symposium on materials information retrieval attended by one of the authors.35 The chi-square test used to correlate physical experimental data is applied to word co-occurrences. Probabilistic Indexing Experiment A controlled experiment was conducted with 110 articles, selecting and categorizing keyword~ then working backward to coordinate documents and categories. In this way, eventually, for every request there was an answer document which, when retrieved, satisfied that request. Conventional (binary) and probabilistic techniques were compared, and the results were expressed in terms of the number of retrieved documents that had to be read before hitting the answer documents. It turned out that the conventional system would require the user to read approximately thirty percent more retrieved documents to obtain the same number of answer documents as oppOSed to the basic Selection process of probabilistic indexing. Considerable improvement was achieved with elaboration of the basic selection process. Proba bilistic Indexing-Critique Indexer subjecti~ity is greatly increased. The decision-making process would seem more difficult, but it may subsequently prove to be worth it. According to the authors, the work of the indexer is facilitated; as it now stands, he is too constrained by having to say yes or no to a possible index term. The ability to make too fine a breakdown of the weighting scale from 0 to 1 is questioned. An analogous problem arises in education when 545 a teacher has to grade pupils in a highly supjective field. In probabilistic indexing a scale of 0 to 3 (or 0, Va, 2h, 1) would seem to be the maximum. However, the psychometric problem of scaling is not an easy one to resolve. The premise that one can go from index terms that are highly significant of a document to documents that are highly relevant to a request is accepted with much reservation. It is probably a correct premise in most instances, but it overlooks the situation in which a request will b.e quite well and perhaps best satisfied by a document in whose over-all text the search terms playa relatively minor role. The old subject-classification problem-to describe the subject of a document-does not wholly describe its information content. There are many formulations of the information retrieval problem in terms of statistical and information theory, but most of them skirt the problem. In fact, the circumstances are often tantamount to a search for the exact solution to a problem when most people are not sure just what the problem is. The probabilistic indexing approach, on the other hand, has inherent potentialities for the eventual over-all mechanization of IS&R. This, JYlaron's later work on automatic indexing and Edmundson and Wyllys' insight into the true implication of information theory for IS&R, seem to be paths to the library of the future. Probabilistic indexing is still a manual indexing' procedure. It appears to be the natural transition from manual to automatic indexing because as mechanized statistical methods of selecting terms from a document come more and more into use, probabilistic indexing will provide a richer foundation on which to build. MECHANICAL INDEXING Any discussion of mechanized indexing should be prefaced by citing the single major limitation that prevents any of the elegantly devised theoretical schemes from being put into practice. This is the requirement for total text input. The machine processing of total text is not efficient for anything but experimental analysis as long as printed text must be translated into machine-readable text (via keypunching, for example). Working automatic 546 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 indexing systems, like machine translation systems, must await the perfection of print-reading and character-recognition devices. refine the statistical behavior laws narrowing their specification and reducing the undefinable to a minimum. Mechanized indexing is attacked from two different angles, statistically and linguistically. The fundamental concept of statistical key word selection is illustrated in Figure 2, which is designed to convey a graphic impression of the concept and is not necessarily the only procedure for implementing the concept. The diagram represents a smoothed frequency count of the word occurrences in a document as a function of the rank order of the words. This is statistical selection in its crudest, most unrefined form. It flatly presupposes that common words will occur most frequently, the esoteric but un pertinent words most infrequently. The middle-frequency words-those remaining after the extremes of the curve have been removedare considered to be the most significant. The same process can then be applied to pairs of the significant words. Statistical The common basic assumption of statistical selection processes is that the linguistic properties and laws of literature are describable in terms of populations and frequencies of word occurrences and their associations. The difference between statistical description of language and statistical thermodynamics is basically quantitative, but this quantitative difference is sufficient to manifest itself in a qualitative form. In general, the numbers treated by thermodynamics are larger and the variables fewer than in linguistics. The result is that given a norm of behavior in linguistics, the deviations from the norm are more pronounced than in thermodynamics. There is little evidence in the physical world of less than maximally probable events. In language, on the other hand, exceptional occurrences are witnessed regularly. The problem confronting mathematical linguistics, then, is to define the variables and to INT£IVAL I'~~~' cJ rItE IN OIf~Qtl I~ . .# tll.1R.o\so,.",C ACCEIER.o\TION lO SUCH ,..~ri' "'' '0 'i>"~ '>J.c.,w.c. ,,(!.d' GaAPH lANK Figure 2. Idealized Frequency Count of Words Occurring in a Document Entitled "The Ultrasonic Acceleration of Diffusion." In general, this first-approximation procedure screens out the many clearly irrelevant words leaving a working body that can be treated by more precise techniques. Examples of Statistical Selection Three proposals for automatic, statistically based indexing merit attention. One has many ramifications in use today; the other two are still in the ivory tower having been subjected only to carefully controlled experiments. Auto-encoding The pioneer study of mechanized indexing,22.23 which led to the key word in context (KWIC) index, relies on the computer recognition of individual words and counts their frequency in a text. Eliminating the common words, the most commonly occurring topical words are used (without further discrimination) as index terms. The product is a mechanically prepared concordance. Recognizing the limitations of word indexing, Luhn has standardized the vocabulary by combining words with the same root and then combining the counts of synonymous words. A normalized form is selected after being looked up in a thesaurus. Relationships are handled by computer analysis of word pairs in cases where significant words occur together. INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART The search terms are develo.Ped analo.go.usly fro.m an essay-fo.rm request by matching search terms and index entries. Automatic Document Classification Academic-level arguments to. the co.ntrary no.twithstanding, there is much to. be gained by a scheme fo.r rapidly and auto.matically fitting do.cuments into. sensible catego.ries. The pro.blem facing library practitio.ners is particularly acute in this respect, because while tho.se slightly remo.ved fro.m the bo.o.kshelves debate the co.mparative merits o.f classificatio.n versus do.cument-independent o.rganizatio.n o.f info.rmatio.n, these peo.ple are pressed to. shelve their documents in a ratio.nal fashio.n. Besides, as -Bo.rko. argued so. co.nvincingly at a recent linguistic seminar,6 peo.ple (scientists included) tend to. think naturally and with facility in such an o.rdered way, i.e., catego.rically. The co.nflicts arise when the flexible adaptive human psycho.Io.gical system co.nfro.nts rigid, Io.ng-standing systems so.lidified o.n paper. A happy so.lutio.n appears fo.rthco.ming in auto.matic classificatio.n. Wo.rk has been do.ne in this area by Maro.n 25 and Bo.rko. et a1. 5 •6 In bo.th instances the researchers rely o.n statistical co.rrelatio.n between wo.rds and do.cuments, and between wo.rds. The two. experiments are similar in several respects. Fo.r example, do.cuments are searched by co.mputer fo.r the o.ccurrence o.f specific clue wo.rds relating to. specific a prio.ri subject heads, o.r catego.ries. The essential difference lies in the derivatio.n o.f the catego.ries and assignatio.n o.f the do.cuments. Maro.n devises his o.wn catego.ries and assigns the do.cuments by means o.f a Bayesian predictio.n fo.rmula. Borko., o.n the o.ther hand, derives his catego.ries by matrix co.rrelatio.n o.f the clue wo.rds, extractio.n o.f eigenvecto.rs by facto.r analysis, and ro.tatio.n o.f these eigenvecto.rs to. find the best catego.ries. Do.cuments are then assigned by facto.r sco.re predictio.n techniques. Bo.rko has do.ne co.nsiderable wo.rk in the last co.uple o.f years in co.mparing his o.wn and Maro.n's wo.rk. A related effo.rt in do.cument asso.ciatio.n by linguistic analysis has been undertaken by Salto.n at Harvard. 33 Auto-matic Indexing (Mat'on) Maro.n's scheme25 fo.r auto.matic indexing o.perates by generating a prio.ri -subject heads 547 and searching do.cuments by co.mputer fo.r the o.ccurrence o.f these subject heads (called clue words). The intentio.n is a go.o.d o.ne, to. ultimately merge pro.babilistic and auto.matic indexing. Automatic Indexing (Edrnundson and Wyllys) -Rela,tive Frequency Approach This technique 10 represents a p.ew co.ncept in the statistical analysis o.f text and co.mes elo.sest to. bridging the gap between IS&R and info.rmatio.n theo.ry. Instead o.f treating a do.cument as the universe o.f words as in the primitive diagram in Figure 2, the frequency o.f a wo.rd in ~ do.cument is co.mpared with the frequency o.f the same wo.rd in general use. Vario.us statistical criteria are suggested and evaluated. The upsho.t is that a wo.rd that is rare -in general use but frequent in a do.cument is a significant measure o.f that do.cument's co.ntent. This co.rresPo.nds to. the surprise element o.f info.rmatio.n theory. Citation Indexing Several studies have been made in recent years using the biblio.graphic materials and ".,,;+.n. .... .: __ .... ___ ........... ! __ ...l ~l"a.UVUo:s ! _ ....l-. ........... _ CVULa.H1t:U HI ....... _L_ UVCU1l1~IH~ -.!J..L. .... ___"'1! __ .... _Ll __ ~lLH~r ulr~cLly fo.r cro.ss-referencing purpo.ses o.r as a statistical vehicle.l2.13.20.33 These techniques explo.it the dynamical histo.rical evo.lutio.n o.f science and the language used to. express it. One o.f the autho.rs (Wo.od) is wo.rking o.n interlingual citatio.n statistics in Russian and English aco.ustics jo.urnals in an attempt to. derive systematically fro.m current termino.Io.gy an accurate technical termino.Io.gy fo.r machine translatio.n and info.rma tio.n retrieval fro.m fo.reign language references. Linguistic The creation o.f indexes by auto.matic linguistic analysis penetrates deeply into. the realm o.f machine translatio.n (MT). In this case, it calls fo.r the auto.matic translatio.n o.f the so.urce language into. a target IS&R language fro.m which indexes can be created. Present MT effo.rts are aimed at syntactic analysis, o.f which there are two. types: 1. Immediate-co.nstituent o.r phrase-structure analysis-the segmentatio.n o.f sentences into. successively smaller machinetractable parts. 548 }lIROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 2. Dependency analysis-the assignment of a governing word to every word in a sentence (or other lexical unit), i.e., the creation of a complete dependency structure. Further elaboration of linguistic analysis is beyond the scope of the present survey. A clear description and reconciliation of the two approaches are given by Hays.15 Klein and Simmons actually implement both procedures simultaneously.21 Finally, a thorough state-of-theart treatment of syntactical analysis was presented by Bobrow at the last Joint Computer Conference. 4 Extension of Concepts A desirable future system would unify automatic statistical indexing applying the relativefrequency approach of Edmundson and Wyllys as a criterion for assigning weights to index terms as required in probabilistic indexing. A certain amount of skepticism is evoked by the assumption that statistical selection will create the best indexes because some of the problems are semantic. However, it will probably create the best automatic indexes for some time to come. Linguistic analysis has a serious drawback if used to find a common language for IS&R and MT in that there is a major difference of goals between IS&R and MT. In MT, the user supposedly has a document, wants to know what it says, and is therefore interested in fidelity of translation. In IS&R, the user is interested in finding a peculiar (sic) type of information and wants the documents that contain it regardless of how it is presented. On the other hand, linguistic analysis will serve IS&R advantageously if used to learn how people think and communicate ideas~ An intermediate step to probabilis~ic indexing would be indexing on two levels, viz., terms which are indicative of what a document is about and other terms pertaining to items of information not immediately contained in the topic. In other words, the following assignation of values would be made to every word in the document: o- Nonsignificant word 1 - Significant nonindicative word 2 - Significant indicative word Indicative is interpreted to mean that the word is directly related to the theme of the document. Such distinctions are psychologically facile for human indexers. STORAGE AND RETRIEVAL PRESENTATION OF INDEXES If an automatic literature search is conducted, the user will, in general, never see the index. He will make a request, and the system (via librarian or computer) will do the search for him. There still remains a strong need for the user, on occasion, to see the index. The forms of index presentation, all of which can be readily produced by machine, are card form, book form, and pictorial display. Card Form Every library user is familiar with the standard card catalog. The cards may represent units of a document file, in which case one card contains all the printed information pertinent to a document. They may be units of an aspect system, each card representing an index term and containing a list of document numbers. They can also be punched or otherwise marked for manual or machine searching, and they may have a window with a microfilm of the document or abstract (aperture card). Book FO'rm Conventional and Rotated Book-form indexes stem from the familiar list of topics at the back of almost every nonfiction book. The frustration often engendered by most book indexes led to improvements, such as the extremely detailed and cross-referenced index of Chemical Abstracts. An advanced development is the rotated, or permuted, index. Strings of key words, such as the significant words of titles or auto-abstracted sentences (KWIC), are listed several times, alphabetically according to each word in every such string. Many experimental machine-generated index printouts of this type are now in use and are proving to be very effective. For example, the RotaForm Index, now a regular part of Index Chemicus, is a computer-generated molecular formula index that repeats and rotates INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART 549 a chemical formula for every chemical element that it contains. projected on a viewing screen. The number of frames and superimposable strips is expandable. Columnar Hypothetical SystemsThe Semantic Road Map An attempt to present coordinate indexes of the uniterm type in book form is the columnar index, such as the Scan-Column Index and Tabledex. These indexes print adjacent columns of document numbers under word numbers and are rapidly scanned to locate documents with a large number of desired key words. The limitations for an untrained user are pretty obvious. Clearly, the fatigue factor would be high. The book-form index is, of course, a closed index; once it is printed, terms cannot be entered or deleted. Pictorial Display The ability to visualize a coordinate index in Gestalt form is much to be desired. It has the added advantage that it can, in principle, be viewed on the same screen as the document or abstract itself. Termatrex and Minimatrex Conceivably, one might project any. bookform index on a screen. But the displayed index used to great advantage by a number of firms today is the Termatrex system. Document accession numbers are located on the nodes of a coordinate grid that is set against an illuminated background. When a number of Termatrex cards, each one representing a key word, are superimposed, the light will show through the nodes that are common to all of the key words. In this way, the user can g~t a rapid over-all glimpse of how many documents he would have to inspect as well as their accession numbers. Also under development by Jonker Business Machines, the creators of Termatrex, is Minimatrex, an improved, miniaturized, mechanizable version of Termatrex. The coordinate grids are photographed on film strips by grouping broad categories of terms onto corresponding frames of the strip. In its present form, the Minimatrex strips have 8 fra'mes, each one a 100-by-100 matrix representing 10,000 documents. As many as 12 strips can be superimposed, and the resultant retrieval pattern is There is one hypothesized system which, although far too advanced conceptually to be operational even in the near future, deserves special attention because it foresees the possibilities of a truly automated era. Loren Doyle of SDC 9 gives an intriguing picture of a space-age library system. Giving credit to the relative-frequency notion of Edmundson and Wyllys,lO Doyle seeks to "increase the mental contact between the reader and the information stored so the reader can proceed unerringly and swiftly to identify and receive the message for which he is looking." The semantic road map idea recognizes the human proclivity to organize concepts in spatial relationship to each other to allow the free evolution of concepts, one as the outgrowth of another, linking the retrieval process more intimately with the process of natural creative thought. Doyle provides a scheme analogous to the one by which we would find an apartment number, given an address number, street, zone number, town, county, state, and country, none of which were familiar to us. vVe would begin with a reasonably compact map of the world and proceed until we had a houseplan of the apartment building. The projected library user sits before a console with a panel of control buttons and a viewing screen. He begins with a map of the contents of the library whereon he views concepts whose mutual associations are indicated by proximity,' heavy lines, thin lines, dotted lines, large letters, small letters, and arrows that point toward distant cousins. Through a rapid succession of such maps, he homes in on the message for which he seeks. The documents themselves are in the form of "document proxies," or abstracts in map representation similar to the search maps. This permits rapid scanning of a large number of document abstracts to find those worth reading. The maps are, of course, automatically generated. One set of techniques for realizing such 550 PROCEEDING~PRING JOINT COMPUTER CONFERENCE, 1964 a system is given along with the description of a preliminary experiment. Don Swanson,38 Dean of the Graduate Library School, University of Chicago, gave a somewhat similar man-machine concept for a library request console tied into a storage and control system. However, his discussion, unlike Doyle's, was centered on console-library request techniques as a feedback link which is part of an on-line search process system for future automated library systems. Savage emphasizes that to describe the effectiveness of a system in terms of magnitudes and system parameters (number of books in a library or number of descriptors in a computer memory) does not tell how well the system works. What is needed is a standard criterion of relevancy, one that is as useful as miles 'per gallon for measuring automobile economy. Savage characterizes human evaluations in a somewhat more, precise way than is usual, points to some of the difficulties of ever generating a metric mea,sure of system performance, and suggests the direction of study to obtain more accurate measures. It is important to realize that any metric criterion of adequacy will have to be tested by way of obtaining feedback from the ~users of a real system. . .. ' MECHANICAL SEARCHES The original mechanized searches were made by serially operating machines, which operate most efficiently by reading the whole store to find what is needed. Serial searching is still advantageous when the store 'can be broken up and grouped (as in multiple-pun~hed card decks) ; input, updating, and purging are also simpler then. B'y its very nature, however, serial searching is less efficient than applying the lookup principle, which is feasible with random access machines having large memories. This principle also allows for indefinite expansion of a large file. SERIAL AND RANDOM ACCESS The factors which must be considered in a comparative evaluation of serial and random access techniques (search and lookup) are file size, length of search, economy of money and time, effort expended, and the inherent efficiencies of -retrieving a given type of information. In the face of the growing number of available computer systems, the thorough-going and detailed evaluation of any systems or combinations of equipment presents an enormous task. An interesting and eriginal approach to the problem by automation of the selection of systems suitable to a particular need is given by Arnovick. 1 Essentially, a complete list of system component characteristics and ancillary information is generated and stored in a computer. A complete set of application requirements is compiled and matched against the system information stored in the computer; the computer analyzes the application information and synthesizes tin optimum general system. LOGICAL OPERATIONS It is convenient to think of searches as comprising two kinds of logical operation, conjunctive and disjunctive. Searches are narrowed by the former because taking the logical products and differences of terms (AND, AND NOT) makes the request more specific. Disjunctive operations are useful for broadening the search; the logical sum (OR) makes for greater variability. Disjunctive operations and combinations of disjunction with conjunction (logical products of logical sums, etc.) are especially helpful for processing request terms in a machinestored thesaurus or otherwise pinpointing a request. It has been pointed out that although Boolean functions do, in fact, accomplish the stated effects, from the systems optimization standpoint, they do not do so efficiently. It is demonstrated that Boolean combinations will yield either too much irrelevant material or too little relevant materia1. 38 In the attempt to find a more flexible and optional treatment of the retrieval problem using keywords, Del Ballard (formerly of RCA) has developed a new approach to search logic. The technique is one that is easily implemented on a general-purpose computer and provides the following: INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART 1. Retrieval of "best available" documents 2. Ranking of documents by the degree to which they match a request in terms of number of keywords 3. Upper and lower bounds on either the number of keywords or number of documents 4. Sensitivity to various attributes of keywords, e.g., rank, order, :numerical magnitude, ranges of values 5. Arrangements of search terms into groups that fit the requester's thought process, including nesting of groups It is stressed that the grammatical problem is circumvented by grouping concepts without specification of the relationships between them. It has proven to be a pliable, rough, and ready tool suitable for the general practitioner in information retrieval. MANUAL-DEDICATION OF SPACE A systematically arranged index can be searched by proceeding according to the rule of systematization as in the standard card or book catalog. The rule may be simple, as in alphabetic filing, or it may rely on a certain amount of prior knowledge on the part of the searcher as in chemical and biological hierarchies. In more recent developments, such as the permuted index and manual coordinate indexing cards, the searcher is guided more rapidly to a specific location of information. A technique that is both efficient and adaptable for eventual machine searching makes use of the Dedication of Space principle. In essence, a particular geometric point or region is used to represent an accession number. The matching of like points for several terms yields documents containing those terms. This is a form of the coordination principle. The best known example is the peek-a-boo card, one version of which is a standard punched card of the IBM type coded so that each space represents a document number. Matching is achieved by aligning several term cards and noting the document numbers. (holes) through which light can be seen. If no documents are found in a search, terms can be eliminated by trial and error until a "see-through" occurs. In computer searches, 551 it is: possible to generate a "minimum search criterion" which, for ,a given set of terms, will yield those documents containing the largest number of terms in the set. STORAGE MEDIA Paper Media Paper holds a certain record for longevity as the prime medium for recording information. That it can be fairly permanent has been demonstrated by the Dead Sea scrolls. It will continue to be prominent for some time, although in recent decades it has taken unprecedented forms, such as punched cards and paper tape. An interesting speculation presents itself. Although digital information can be stored on paper, the technique of sensitization does not make use of the intrinsic properties of the medium, i.e., holes are punched or magnetic materials are deposited. Researchers in the Soviet Union have uncovered a slight piezoelectric sensitivity in wood which may indicate possibilities for the uS.e of paper. In general, cards and paper tape are used either as 'storage media for small manual or semiautomated systems or as the input medium for large automated systems. Punched cards and punched paper tape executed by automatic writing equipment are used for the latter. Small systems use peek-a-boo cards, keysort cards, edge-notched cards, uniterm cards, or modifications of these. Magnetic Media Tapes Magnetic tape is extremely well suited to the long-time storage of large quantities of data in machine-language form. I t is also one of the most rapid search media. For this reason, it is used most extensively as the peripheral memory of computers and, therefore, for the storage of older, less frequently needed information. Its nature is generally more permanent and less flexible than other magnetic media. Because it is most adaptable to digital data, magnetic tape usually works best for the storage of index terms and document numbers. There are times when it is advantageous, ac- 552 :PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 cording to some people, to record entire text in machine language on tape; but this is not conceivably practical except for data. However, there is a system called VIDEOFILE (an RCA development) which stores documentary and pertinent bibliographic information using video techniques. The document image is converted to a video signal and is stored with an index number. It has the advantage of furnishing rapid and direct hard-copy printout and is amenable to communication media. Drums Magnetic drums are better suited than tapes for the internal memory of a computer because of their more rapid (many orders of magnitude) random access. Being a lookup device, the drum is well adapted to storage of the internal functions of a computer, such as dictionaries, thesauri, and program instructions. Moreover, it is the cheapest type of internal, or main, memory. Discs Another dimension is added by the use of disc memory units, which have characteristics similar to drums but with a considerable gain in access time and storage space. Given an address, the information is located on a certain disc in the stack, a certain sector of the disc, and a certain band wi thin the sector. Magnetic Cores Magnetic core arrays provide still faster retrieval, but the expense per unit storage space is the highest of all internal memories. Cores work well in conjunction with magnetic tape as the peripheral storage media. Cards Magnetic cards need to be searched serially, but they have the advantage of flexibility over tape with respect to additions or deletions. However, the search rates are slower. The cards are usually stored in cartridges and may be used in conjunction with film chips, in which case the identifying codes are in magnetic digital form whereas the information is in the form of a photographic image. Magnetic coding is sometimes used in c ...... l"T ;.l..1J.l,.V ..... t-", t-hn ., H-; ~""''''''''•.H,''''VV\.A. IIr.••n ..... .Lv.J.""" \...I.'-'~V.1..I..I.5 .... v v U'C'C.I:-'J..y lIJ.l'C Ull,l- mate aspect of abstracting mechanization. FEEDBACK Document profiles remain fairly stable, while recipient profiles are subject to continual modification through feedback. In the SDI system, response cards are provided along with abstracts of documents whose profiles suitably overlap the recipient profile. The recipient returns the response card with an indication that he (1) had no interest in the document, (2) found the abstract useful and sufficient, (3) found the abstract interesting and requests the total document. His profile is then updated on the basis of his responses. The depth of matching, i.e., the extent to which the recipient profile and document profile overlap, is a controllable parameter. EXPANSION OF SDI The skill profile now employed by Space and Information Systems Division is one example of the profile matching concept. The skill file can be searched when management wants to know who is knowledgeable in a given area. PURPOSE OF ABSTRACTING ABSTRACTING VERSUS INDEXING For the purpose of definition, it is important to differentiate functionally between an abstract and an index, a detailed descriptor index in particular. One can almost visualize a document being screened through several gradations of processing. At one end would be the single subject heading or "catchword" (Schlagwort). At the other end of the range is the total document itself. In order of increasing depth of representation, the intermediate steps would be a bibliographic index, a complete document profile as used in selective di~,semination, a more detailed informative index using descriptors or Uniterms, scope notes, a detailed index with roles and links, a so-called supplied title (a title written by the document processor), a one- or two-sentence synopsis, a telegraphic abstract, and a detailed summary or abstract. However, the user will not want to be bothered with more than three of these includr 556 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 ing the total document. Under the proper circumstances, the continuum seems to fall spontaneously into the parts necessary for that set of circumstances. For example, a trade journal, well attuned to the needs of its readers, may give a list of one-sentence descriptions of worthwhile reading matter (supplied titles) providing the reader with sufficient stimulus to procure the document. However, if a library is to get information to a large and diverse clientele, it cannot hope to cater to the individual user as the 'editors of a trade journal can to their readers. It cannot expect to handle an influx of up to a thousand documents per day, evaluate their content, and dramatize their highlights in the way that a trade journal can. Not knowing the specific needs of the user, the alternative extreme would be to give him every potentially pertinent document. Such a system would entail a disproportionate waste of time. The index and search techniques will help the user to find things which might be what he thinks he wants. The dissemination operation will help him receive things which might be what he had not thought of or did not know about. To conserve the user's time, abstracts of those documents with possible or latent interest are supplied to him. Therefore, an abstract is not an index, for an index indicates what a document is about, or it tells where the information is located. An abstract is not a document in the strict sense of the word for it does not tabulate data nor does it give detailed hookup and assembly instructions for an instrumentation complex. An abstract is not identical with either an index or a document, but it incorporates features of both. STANDARDIZATION OF ABSTRACTING Several studies are being conducted to find operational definitions of abstracting. The two mainstreams of effort are directed toward examining the effect of various abstracting philosophies on requesters and their satisfaction and toward evaluation of different abstracts of the same or similar documents. Several parameters are involved, and in most studies one or more of these are controlled. In one investigation,S the size of the natural language abstracts is held to a fixed percentage (10 per cent) and the work of different abstracters is compared. A unique part of the same investigation is devoted to the study of "term diagrams," which incorporate the "road map" idea of Loren Doyle (see Indexing) to show quantitative and conceptual associations between key words. These studies are aimed at eventual automation of document condensation. The American Institute for Research is conducting an extensive inquiry into the operational definition of abstracting, the development of rules guided by task performance on the part of both abstracter and user, and the evaluation of efficiency. 8 A survey is being made among abstracting services, publishers, and users to gather ideas from those experienced in the field. Simulated job conditions are set up to measure user comprehension and proficiency under these conditions, given both full and abstracted texts. AUTOMATIC ABSTRACTING The principal efforts in automatic abstracting are being carried out at Thompson-Ramo W001dridge, Inc. System Development Corp., Planning Research Corp., and at International Business Machines. Without going into the detaIled procedures for abstracting, the primitive approach is to first find statistically significant keywords (content words with a high frequency of occurrence) and then find those sentences which contain the greatest number of such keywords. "The results so far are conspicuous by their lack of coherence; partly because of habit patterns in read~ng, -they seem to present disconnected ideas with a los8 of flow between the ideas. Of course, the abstracts suggest rather than condense or represent the document contents. Nevertheless, they are surprisingly good on occasion. The problems just mentioned have been instrumental in the inception of more probing studies into the nature of human abstracting. CRITICAL COMMENT ON AUTOMATIC ABSTRACTING As an incidental preface, the importance of the abstract in general should be considered. It would be difficult to overemphasize the place of the abstract or document summary in the mechanized system of the future. Apart from its utility to the user in some form, it will be the INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART prime medium of the system, even the main unit of communication. An abstract is a useful and logical unit because it can be printed on a frame of microfilm or on a coded (punched or magnetic) card or on a single sheet of paper. Even advanced mechanized systems will probably use manually produced abstracts. Automation of the abstracting process will be one of the last stages in the evolution of fully automated systems. The reason is evident after a moment's reflection. Difficult as the problem of automatic language translation is, it is one order of complexity lower than the job of abstracting. In many ways the difference between the two is analogous to the disparity between a dictionary translation and a translation turned out by a truly bilingual specialist in the field. The latter, if a good translator-, will not simply convert words or even phrases. He will restate the author's thoughts in the new medium. The abstract must do this same thing. Automatic abstracting comprises the ultimate refinement of automatic translation and automatic indexing. It must translate ideas from expanded to compressed language, and it must be selective, It must also be endowed with a product all its own, the attribute of paraphrase. REPRODUCTION AND DISPLAY There are a number of hard copy reproduction methods available, including: Copy Camera Ozalid Ditto Process Photoengraving Electrography Photogravure Letter Press Photolithography Li thogra phy Rotogra vure Mimeograph Thermofax Offset Printing There is a method to suit almost every conceivable need. In general, the obtaining of higher-quality, cleaner copy calls for a greater investment. The situation is not so well resolved in the case of display systems. The technology of display has generally lagged behind the advancement of data processing and transmission. However, government and industry have begun to exhibit a resurging interest in computer-generated large-scale displays. The development of 557 high-capacity comman9 and control systems within the Air Force has emphasized the need for sophisthicated data-presentation subsystems. 14 There are three main areas of technological effort: Projection, using a stable light modulator, such as film or selenium plate. Light valve, also a projection device, with direct electronic control of the image. This has advantal!es of speed and elimination of expensive film and the disadvantages of resolution and brightness (two urgently requisite factors in IS&R display). Electroluminescence, no projection, experimental prototypes only. Display surface acts both as light source and modulator. . In general, for the specific applications of IS&R, there are no suitably inexpensive systems for reproduction and display. Essentially what is needed is a low-cost, low-volume device to meet the demands of a circulating library replaced by a noncirculating, automatic transmission-displaY-reproduction system. Admittedly, for many applications, microfilm display can be made inexpensive, particularly where it is sufficient to exhibit an entire sequence of frames or film. But for IS&R, which is a selective application, research and production have not come up with adequate equipment. COMMUNICATION LINKS Hardware development in IS&R will ultimately make it unnecessary that a requester be physically present at the information center. Links for communication between remote request stations and the center, which are now available or in advanced stages of development, range from conventional telephone lines to systems employing microwave devices. A closedcircuit television system (CCTV) will lend itself very readily to application in IS&R. Therefore, the communication system might be as simple as one permitting a requester to initiate a search by telephone and whose results, in the form of abstracts, are returned to him by telephone or mail. With abstracts and documents stored on microfilm and a CCTV /telephone system, the requester could, in a more advanced system, view abstracts or the document itself. 558 P,ROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 The ultimate in a communication link between the user and the information center would permit the user to deal directly with an automatic IS&R unit (computer) to initiate a search, call up on a viewing screen the documents resulting from the search, and reproduce these in the form of hard copy. While all the functions involved are conceptually simple, considerable development and system integration effort must be accomplished before the ultimate system becomes a reality, especially in the development of CRT line scanning (800 to 1200 lines) . In the area of volume hard-copy reproduction, to take one example, the state-of-the-art survey for this report revealed no systems that could be called ideally suited to IS&R if one considered high-cost trade-offs for systems such as electrostatic-video printing. Among the lesser problems, considerable work in human factors for such a system might be done. Detailed answers to questions of viewing-screen brightness and contrast and ambient illumina- tion should be made available. While all the individual functions involved are conceptually simple, considerable development and systemintegration effort must be accomplished before this becomes a reality. Much groundwork has been done,32 but the extent to which experience in viewing conventional TV visual presentations can be applied to the prolonged reading of text for an ideal IS&R visual· link is questionable. SUMMARY The status of information storage and retrieval is summarized in Figures 4 (Equipment) and 5 (Techniques). The sizes of the boxes are drawn to indicate their relative "strength" of development of the given area. Note that the "Equipment" circle is drawn larger than the "Document Processing" circle to show the greater strength of development in this area. A notable exception is the equipment for graphics and display, microfilm reaoing, and reproduction in particular. SPECIAL AND GENERAL PURPOSE COMI'UTERS Figure 4. Equipment State of the Art. Figure 5. Technique State of the Art. 559 INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART In two respects, IS&R is not yet an integrated science in the sense of chemistry or even such an interdiscipline as bionics. First of all, the contributing disciplines are diverse and overlapping, but hardly unified. Secondly, although many nonconventional systems are in operation throughout the world, they have little in common by way of a unifying description. The most glaring deficits are the software lag and lack of a truly empirical basis. Today is part of an era of machines that can analyze data, read and recognize printed symbols, recognize audible and verbal signals, be conditioned and learn, teach humans or other machines, and even make decisions. Thin film memories are reducing access times to the nanosecond realm. A billion items of distinct information can be stored in a space the size of a paperback novel. But trailing far behind these amazing creations in the equipment field are the concepts of how people communicate with each other, particularly when the scientist or engineer presenting his results or theory on paper does not know who in what remote corner will want to read it, or when the person seeking information has some vague unfulfilled desire and-does not knov: -who in what remote COl~ner has the happy cure for his troubles. Each one hopefully relies on those who are entrusted with getting the answer to the need and finding the market for the goods. In the meantime, advanced thinkers are passing over the concepts-equipment gap and are developing theories for conceptual machines not yet in existence. And-emphatically-they are needed. But concurrent with these efforts is a greater need for less speculation and conjecture, less projection of pet ideas and systems, and more experimental investigation and scientific method. The ultimate criterion of how well a retrieval system works is the extent to which retrieved information meets the need of the user. From the outset one is confronted with problems because user need is already an idealization. The corresponding measure of how well this need is met might be called user satisfaction, which is ab;o an idealization. Because user need and user satisfaction are highly unquantitative and highly indeterminate, they must be approximated. The expression of user need is a request and the vague notion of user satisfaction is simulated by the notion of relevancy_ The situation might be visualized as in Figure 6. ¢ SATISFACTION 0 I I I I I .. REQl£ST DOCUKNT IGOALS AND OBJECTIVES BY PROXY) OR ABSTRACT IAN ANSWER BY APPROXIMATION) ¢ RELEVANCY 0 Figure 6. Contrast Between Ideal and Real Information Retrieval. lUlU l{){"! P A PUV .LJ..&..L-I.L..I..I.. '-''-'' .L"~..&. .l..L .L 1. ARNOVICK, G. N., "Automatic Analysis and Synthesis of Specifications for Machine Systems in. Information Storage and Retrieval," presented at the Annual Meeting of the American Documentation Institute, Miami, Florida, December 10-15, 1962. 2. ARTANDI, S., ·and T. C. HINES, "Roles and Links-Or For~ard to Cutter," American Documentation 14 (1): 74-77, January 1963. 3. BAGG, T. C., and MARY E. STEVENS, Information Selection Systems Retrieving Replica Copies: A State-of-the-Art Report, National Bureau of Standards Technical Note 157, December 31, 1961. 4. BOBROW, D. G., "Syntactic Analysis of English by Computer-A Survey," Proceedings of the Fall Joint Computer Conference, November, 1963, pp. 365-388. 5. BORKO, H., and MYRNA BERNICK, "Automatic Document Classification," Journal of the Association for Computing Machinery 10 (2): 151-162, April 1963. 560 ~ROCEEDINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 6. BORKO, H., "Some Recent Experiments in Automatic Document Classification," Linguistic Research Seminar, The RAND Corporation, Santa Monica, California, January 9, 1964; "Research in Automatic .Generation of Classification Systems" (1964 SJCC). 7. CORNERETTO, A., "Associative Memories; A Many-Pronged Design Effort," Electronic Design ( ): 40-55, February 1963. (An excellent introduction and survey.) 8. Current Research and Development in Scientific Documentation, No. 11, National Science Foundation, Office of Science Information Service, Washington, D. C., November 1962. 9. DOYLE, L. B., "Semantic Road Maps for Literature Searchers," Journal of the Association for Computing Machinery 8 (4) : 553-578, October 1961. 10. EDMUNDSON, H. P., and R. E. WYLLYS, "Automatic Abstracting and IndexingSurvey and Recommendations," Communications of the Association for Computing Machinery 4 (5) : 226-234, May 1961. 11. FORD, J. D., JR., and E. H. HOLMES, A Comparison of Human Performance Under Natural Language and Term Diagram Procedures for the Production of Report Summaries, TM-622, System Development Corporation, February 6,1962. 12. GARFIELD, E., "Citation Indexes for Science," Science 122: 108, 1955. 13. GARFIELD, E., "Citation Indexes in Sociological and Historical Research," American Documentation 14 (4): 289-291, October 1963. 14. Guide for Group Display Chains for the 1962-1965 Time Period (advance copy), Directorate of Engineering, Rome Air Development Center, New York, March 27, 1962. 15. HAYS, D. G., Grouping and Dependency Theories, RM-2646, The RAND Corporation, Santa Monica, California, September 8, 1960. 16. HERNER, S., ·and M. HERNER, "An Experiment in the Use of Reference Questions in the Design of a Classification System," report to the National Science Foundation, 17. 18. 19. 20. 21. 22.' 23. 24. 25. 26. 27. Washington, D. C., Herner & Co., January 1962. IBM General Information Manual: Selective Dissemination of Information, International Business Machines Corporation, White Plains, New York, 1962. IBM Workshop on Information Storage and Retrieval, San Jose, California, June 26-29, 1962. KENT, A., "Machine Literature Searching and Translation. An Analytical Review," Advances in Documentation and Library Science, Vol. III: Information Retrieval and Machine Translation, Inerscience, New York, 1960, pp. 13-236. KESSLER, M. M., "Bibliographic Coupling Between Scientific Papers," American Documentation 14 (1): 1~25, January 1963. KLEIN, S., and R. F. SIMMONS, "Syntactic Dependence and the Computer Generation of Coherent Discourse," Mechanical Translation 7 (2): 50-61, August 1963. LUHN, H. P., Auto-Encoding of Documents for Information Retrieval Systems, International Business Machines Corporation, Yorktown Heights, New York, 1958. LUHN, H. P., The Automatic Derivation of Information Retrieval Encodements from Machine-Readable Texts, International Business Machines Corporation, Yorktown Heights, New York, 1959. MARON, M. E., and J. L. KUHNS, "On Relevance, Probabilistic Indexing and Information Retrieval," Journal of the Association for Computing Machinery 7 (3): 216244, March 1960. MARON, M. E., "Automatic IndexingAn Experimental Inquiry," Journal of the Association for Computing Machinery 8(7): 404-417, July 1961. MORSE, R. D., "Information Retrieval," Chemical Engineering Progress 57 (5) : 55-58, 1961. (This is the best account of links and role indicators.) Pacific Southwest Navy Research and Development Clinic, Panel on Information Systems and Computer Technology, Berkeley, California, October 16-18, 1963. INFORMATION STORAGE AND RETRIEVAL-ANALYSIS OF THE STATE OF THE ART 561 28. RCA-NBC, Before the Federal Communications Commission, Petition of Radio Corporation of America and National Broadcasting Company, Inc., for Approval of Color Standards for the RCS Color Television System, June 25, 1963. 29. RENOLD, W., Journal of the Society of Photographic Instrumentation Engineers 1 (2): 44-48, December 1962/January 1963. 30. RESNICK, A., "Relative Effectiveness of Document Titles and Abstracts for Determining Relevance of Documents," Science 134 (3484): 1004-1005, October 6, 1961. 31. RESNICK, A., "Comparative Effect of Different Educational Levels on Indexing in a Selective Dissemination System," International Business Machines Corporation (in press) . 32. RICHMOND, PHYLLIS A., "What Are We Looking For?" Science 139: 737-739, February 22, 1963. 33. SALTON, G., "Some Experiments in the Generation of Word and Document Associations," Proceedings of the Fall Joint Computer Conference, December 1962. 34. STERN, J., "An Application of the Peek-ABoo Principle to Information Retrieval:; Proceedings of the Symposium on Mate- 35. 36. 37. 38. 39. rials Information Retrieval, Dayton, Ohio, November 28-29, 1962. STILES, H. E., "Progress in the Use of the Association Factor in Information Retrieval," Proceedings of the Symposium on Materials Information Retrieval, Dayton, Ohio, November 28-29, 1962. SWANSON, D. R., "Information Retrieval: State-of-the-Art," Proceedings of the Western Joint Computer Conference, Los Angeles, California, September 9-11, 1961. SWANSON, D. R., "State-of-the-Art Paper," Conference on Libraries and Automation, sponsored by the National Science Foundation, The Library of Congress, The Council on Library Resources, Inc., Airlie Foundation, Warren, Virginia, May 26-29, 1963. VERHOEFF, J., W. GOFFMAN, and J. BELZER, "Inefficiency of the Use of Boolean Functions for Information Retrieval Systems," Communications of the Association for Computing Machinery 4 (12): 557-558 ; 594, December 1961. WILLIAMS, J. H., JR., "A Discriminant :Method for Automatically Classifying Documents," Proceedings of the Spring Joint Computer Conference, November 1963, pp. 161-166. TRAINING A COMPUTER TO ASSIGN DESCRIPTORS TO DOCUMENTS: EXPERIMENTS IN AUTOMATIC INDEXING M. E. Stevens and G. H. Urban National Bureau of Standards Washington, D. C. INTRODUCTION National Bureau of Standards in recent months. The SADSACT (Self-Assigned Descriptors from Self And Cited Titles) method is based on the two working hypotheses: 1. That, given only a "teaching sample" of previous human indexing of representative items, the machine can derive statistics of association between substantive words used in the titles or abstracts and the descriptors assigned to these items such that patterns of substantive words in the titles and cited titles of new items can be used for automatic assignment of descriptors to these new items, and 2. That, as in the case of citation indexes, the bibliographic references cited by an author can provide useful clues to the subject of his own paper. During the past five to ten years, increasing interest has developed in the use of machines as a substitution for human intellectual effort in the indexing or classification of the subject content of documents. In terms of practical applications, the greatest interest to date has been in keyword indexing from the significant words actually occurring in titles, abstracts, or full texts. Pioneering use of computers for this purpose, by Luhn s and Baxendale,2 has been followed by the development of a number of KWIC (keyword-in-context) and similar programs. This is "derived" or "derivative" indexing as opposed to "assignment" indexing. 6• s In derivative indexing, the indexing terms used are identical with words chosen by the author himself. In assignment indexing, the terms used are typically drawn from a classification schedule, a subject heading list, or a thesaurus of descriptors, and they are assigned on the basis of decision-judg!llents about the appropriateness of these terms as indicative of the subject matter of the document in question. Approaches to indexing by computer based upon assignment of indexing terms or upon automatic classification techniques have been reported by Baker,! Borko,3.4 Maron,9.10 Needham,ll and Swanson,16 among others. In addition, we hope to demonstrate an automatic indexing technique that will minimize time and cost of preparation by limiting the amount of input material required and by capitalizing on byproduct data generation from one-time keystroking of descriptive cataloging information plus the text of the titles of biblio'graphic references cited in new items. THE TRAINING SEQUENCE AND INDEXING PROCEDURE In training the computer to assign descriptors to our documents, we first select a sample, presumably representative, of items already Small-scale experiments in automatic assignment indexing have also been conducted at the 563 564 PROCEEDINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 in our collection. These have all previously been indexed to some pre-established indexing-term vocabulary, as the result of human subject content analysis. In our experiments to date, the "teaching sample" has consisted of approximately 100 items drawn from a collection of about 10,000 technical reports in the fields of information retrieval and potentially related research including computer design, programming languages, mathematical logic, linguistics, pattern recognition, psychology, operations research, and the like. These teaching sample items are ones that had all previously been indexed by analysts at ASTIA (now the Defense Documentation Center) in the year 1960. Descriptive cataloging information, abstracts, and lists of descriptors assigned were available. This information was keystroked on Flexowriters and the texts of the titles, descriptors, and abstracts were converted to punched cards for input to the computer. In programming the SADSACT method for trial on the IBM 7090-7094 computers, the cards for each of the documents in the teaching sample are read into the computer, one item at a time. A list of descriptors assigned to these items is formed, and the frequency of use of each descriptor for the sample as a whole is tallied. The text of the title and abstract of each document is then processed against a "stop" list to eliminate the common and nonsignificant words. Each remaining word from document title and abstract is associated with each of the descriptors previously assigned -to that document. Sorted lists of English words and their descriptor co-occurrence relations are formed and read out, using three output tapes. When all the documents in a teaching sample have been processed, the next step is to identify the "validated descriptors"-that is, those that have occurred three or more times in a 100-item sample. * (The "names" of other descriptors assigned to not more than one or two items are * In our first teaching sample, 219 different descriptors had been used by the DDC indexers, and 72 occurred for three or more items in the sample. Our second teaching sample, formed by substituting different items for 30 items withdrawn from the first to be used as test material, gave 221 descriptors, of which 70 were validated. retained as "candidate descriptors" but word co-occurrence data is not retained for these.t) For these validated descriptors, the word-descriptor association lists are then merged into a master vocabulary list which gives for each word the identity of the descriptors with which it co-occurred and the relative frequency of its co-occurrence with each descriptor. The SADSACT automatic indexing method r therefore, uses an ad hoc statistical association technique in which each word may be associated either appropriately or inappropriately with a number of different descriptors. In the subsequent indexing procedure, reliance is placed on patterns of word co-occurrences and on redundancy in new items to depress the effects of non-significant or inappropriate· word-todescriptor associations and to enhance the significant ones. The SADSACT indexing procedure is carried out as follows. The text of the title of a new item and of titles cited as bibliographic references by the author is keystroked, and the byproduct punched paper tape is converted to cards for input to the computer. This input material is p·rocessed against the master vocabulary list to yield, for each word matching a word in the vocabulary, a "descriptor-selectionscore" value for each of the descriptors previously associated with that word. When all words from titles and cited titles have been processed, the descriptor scores are summed and for some appropriate cutting level,t those descriptors having the highest scores are assigned to the new item. The actual score value derived reflects both a normalizing factor (based, for example, on the ratio of the number of previous co-occurrences of this word with a particular descriptor to the tOne of the distinctions to be emphasized in assignment as opposed to derivative indexing is that alth::mgh a word or phrase occurring in a title or in text may coincide orthographically with the "name" of a descriptor or subject heading, its occurrence mayor may not result in the assignment of that descriptor to the item in which it occurred. tIn machine tests to date, the cutting level has been arbitrarily set so that exactly twelve descriptors are assigned to each item. However, since these are print~d out in decreasing order of selection-score values, results of other cutting levels can readily be computed. TRAINING A COMPUTER TO ASSIGN DESCRIPTORS number of different words co-occurring with that descriptor) and a weighting formula that . gives greater emphasis to words occurring in "self" -title (the author's own choice of terminology) than to those occurring in cited titles. Similarly, greater emphasis is given to words that coincide with the names of descriptors. The latter weighting enables the assignment of "candidate" as well as "validated" descriptors, if words so coinciding occUI several times in the input material. An extension of this feature would allow similar treatment to be given to n-tuples of substantive words" (such as "automatic character recognition") that had occurred with some frequency in the teaching sample material, so that "potential descriptors" could be used to provide change and growth in the indexing vocabulary. RESULTS TO DATE In a previous informal report, we have noted the results of applying the SADSACT method to DOCUMENTS: 565 to eight new (1963) items to our first 100-item teaching sample, consisting of selected documents indexed by DDC in the spring of 1960. These results were disappointing in terms of comparison against discriptors assigned by DDC indexers, largely because of changes in indexing philosophy and vocabulary such that many descriptors used by human indexers in 1963 were not available to the machine. However, 49% of the descriptors judged by a subject matter specialist to be relevant to these items were correctly assigned by the computer program. Our first report also covered tests on 30 "new" items drawn from the teaching sample itself in order to provide, quickly, a basis for evaluation in which the indexing vocabulary could be held constant. Where titles and abstracts had been used for the teaching sample procedure, for this test the titles and cited titles only were used, so that a reasonable proportion of the input was new rather than TABLE 1. Machine "hits" against DDC assignments, tests run against T .S. # 1 only Short Title of Item Perceptrons and the theory of brain mechanism • (Pattern recognition with an adaptive network Pattern recognition with an adaptive network Invariant input fora pattern recognition machine An adaptive character reader Theory of files Learning in pattern recognition Reorganization by adaptive automation Finite state languages formal representations Some computer experiments with predictive coding On categorical and phrase structure grammars Error correcting codes from linear sequential Method of improving organization in large files Machine program for theorem 'proving Pattern target analysis Tables of Q functiQns for two Perceptron models Lineal inclination in encoding information TOTAL AVERAGE % Hits AllDDC Descriptors % Hits Descriptors Available to Machine 37.5 33.3 50.0 25.0 37.5 25.0 33.3 50.0 20.0 45.5 42.9 33.3 50.0 33.3 60.0 50.0 50.0 66.7 20.0 55.6 50.0 50.0 66.7 20.0 71.4 50.0 33.3 57.1 60.0 0.0 25.0 22.2 66.7 42.9 66.7 75.0 0.0 33.3 25.0 66.7 75.0 100.0 100.0 0.0 40.0 40.0 549.2 34.3 713.1 44.6 858.1 53.6 % Hits Validated Descriptors 60.0 50.0) 50.0 33.3 ,.,,... £\ IO.V 566 PROGEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 TABLE 2. Machine "hits" against DDC assignments, tests run against both T.S. #1 and T.S. #2 T.S. #2 T.S. #1 % All Flowgraphs for nonlinear systems Activity in nets of neuron-like elements Survey of cueing methods in education Facet design and analysis of data Automatic recognition of speech Problems in nonlinear control systems Guiding principleR for time and cost A'pproach to pattern recognition using linear threshold devices Simultaneous multiprogramming of electronic computers Design and study of correlation instrumentation The learning model approach Soviet computer technology Computer applications of Boolean functions Sequential encoding and decoding Design of a language for Simplex system users Automated teaching a review TOTAL AVERAGE % % A vailable Validated % All % % Available Validated 14.3 20.0 33.3 14.3 20.0 33.3 9.1 16.7 33.3 18.2 33.3 66.7 42.9 50.0 60.0 42.9 50.0 60.0 37.5 42.9 50.0 12.5 14.3 16.7 25.0 33.3 28.6 50.0 66.7 50.0 25.0 16.7 28.6 25.0 66.7 25.0 40.0 100.0 100.0 40.0 100.0 100.0 44.4 50.0 57.1 33.3 37.5 42.9 28.6 40.0 66.7 21.4 30.0 50.0 22.2 44.4 80.0 22.2 44.4 80.0 27.3 50.0 37.5 30.0 50.0 75.0 33.3 60.0 75.0 27.3 50.0 25.0 30.0 50.0 50.0 33.3 60.0 50.0 36.4 36.4 50.0 27.4 27.4 37.5 75.0 75.0 75.0 75.0 75.0 75.0 55.6 55.6 71.4 55.6 55.6 71.4 579.1 36.2 764.6 47.8 981.8 61.4 506.8 31.7 671.1 41.9 888.5 55.6 "source" material. For these 30 items, the average "hit" accuracy was 64.8 %. This included the occasional assignment of candidate descriptors in cases where words in title or cited titles coincided with the names of candidate descriptors frequently enough to build up a score above the cutting level. (For example, the candidate descriptor "Sleep" was assigned to an item entitled "Human performance as a function of the work-rest cycle.") Considering validated descriptors alone, the machine assignment accuracy was 77.4 %. These results were suspect, however, because of the probable source material bias. Accordingly, substitutes were prepared for each of these first "test" items to form a second teaching sample consisting of 70 items from teaching sample # 1 and 33 additional items. Items from teaching sample #1, now genuinely "new," and some additional items were then run against teaching Eample #2. Items new to both samples were also run against sample # 1. Results for 101 machine tests on 59 different items are shown in Tables 1, 2, and 3 in terms TRAINING A COMPUTER TO ASSIGN DESCRIPTORS TO DOCUMENTS: 567 TABLE 3. Machine "hits" against DDC assignments, tests run against T.S. #2, both with cited titles and with abstracts Cited Titles % All Construction of convolution 66.7 codes by sub-optimization 40.0 Iterative circuit computers 20.0 Use of context cues in teaching machines 42.9 Model for communication with learning Word length and intelligibility 28.6 Machine correction of garbled 71.4 English text Multiple task study on automatic 50.0 data processing Independence of attributes in memory 50.0 Use of models in experimental 42.9 psychology Linear recurrent binary error 42.9 correcting codes Adaptive sample data systems 42.9 Common programming language task 80.0 A '1 Some requirements for production control data Remembering the present state 25.0 of a number of variables Papers on the APT language 57.1 Topics in the theory of discrete 100.0 information channels Some aspects of problem solving 28.6 Predictive model for self-organ25.0 izing systems Adaptive switching circuits 29.9 Techniques of pictorial data reduction 71.4 Combining of classes condition 37.5 in learning On some relations between 57.1 human engineering Human performance as a func9.1 tion of the work-rest cycle The simulation of human thought 85.7 Studies of communication channels 14.3 Concept of the ideal observer in 44.4 psychophysics Learning in random nets "1 ~":I:.':> TOTAL AVERAGE 1177.7 45.3 % Abstracts % Available Validated % All % % Available Validated 66.7 66.7 66.7 66.7 66.7 40.0 25.0 40.0 28.6 40.0 20.0 40.0 25.0 40.0 28.6 42.9 50.0 28.6 28.6 33.3 33.3 71.4 40.0 71.4 14.3 85.7 16.7 8"5.7 25.0 85.7 50.0 50.0 75.0 75.0 75.0 66.7 50.0 66.7 60.0 25.0 28.6 33.3 33.3 33.3 40.0 42.9 50.0 57.1 57.1 66.7 50.0 80.0 t>v.V 75.0 80.0 100.0 28.6 80.0 14.3 33.3 80.0 50.0 50.0 80.0 100.0 25.0 33.3 25.0 25.0 33.3 80.0 100.0 80.0 100.0 57.1 100.0 80.0 100.0 80.0 100.0 28.6 25.0 33.3 33.3 28.6 50.0 28.6 50.0 33.3 66.7 38.5 83.3 50.0 55.6 83.3 60.0 29.9 57.1 50.0 38.5 66.7 66.7 55.6 66.7 80.0 57.1 66.7 42.9 42.9 50.0 14.3 25.0 18.2 28.6 50.0 85.7 16.7 44.4 85.7 50.0 50.0 71.4 14.3 33.3 71.4 16.7 33.3 71.4 50.0 37.5 100.0 100.0 100.0 1241.6 45.9 1368.1 50.7 1609.2 59.6 .............. 1312.5 50.5 1544.6 59.4 568 PROGEEDINGS----SPRING JOINT COMPUTER CONFERENCE, 1!164 of the percentages of descriptors originally assigned by nnc indexers which were also assigned by machine. These results show, even more than had been expected, a significant drop below the "hit" accuracy obtained when the same items were run under partially "source material" conditions.§ The overall average "hit" accuracy for these tests is only 40.1 Cfr, considering all descriptors assigned to these items by nne. However, in spite of the use of test items drawn from the same time period as the teaching samples in order to maximize consistency of indexing vocabulary, 19.17c of the descriptors assigned 'by nDC were not available to the machine. When corrected for this not-available factor, the average "hit" accuracy was 48.2 %-. Other descriptors used by nnc for these items were available to the machine only as candidate descriptors and could only be assigned if input words for the test items coincided with the names of these descriptors. Considering only the descriptors that were freely available to machine, the validated descriptors, the computer assignment accuracy was 58.1 Cfr. In only one test ca s.e , for these 59 items, were no hits achieved by the machine method ("Pattern target analysis") . Figures 1 and 2 represent, respectively, machine assignments for title-and-cited-titles and for title-and-abstract of an item subsequently evaluated by several members of our staff, as shown by initials. Checks to the left of the descriptors indicate agreements with nnc assignments. The machine missed three nnc assigned descriptors for this item: "Electronic Circuits," "Machine Tools," and "Servomechanisms," the latter two not being available to the machine system. Figures 1-2 illustrate the present format for computer printout, with assignments in order of decreasing selection scores (numeric data to the right of descriptor names) for three different weighting formulas. These figures also illustrate differences in order of machine selection and of actual descriptors chosen by the § This is apparently due to the fact that although only each item's own title was source material, the weighting formulas to emphasize significance of words in the self-titles changed the effective proportion of the new material in the cited titles. 000191294550 PAPERS O~ THE APT LANGUAGE pEseRI PTOR NAME liN rUTliEMATieAL COMPUTER PATA AUTOMA TI C 5')6 510 ~~~GE 48~1 SCIENrlFJC RESEARCH IIUMA:. E;>JGI:-IEEP.ING COMMUNICATION SYSTEMS ClJMMU"IICATlONS THl:ORY MACHINE TRANSLAT ION PATA rRMISMISSION SYSTEMS (fIFORMATlON THEORY MATHEMATICAL LOGIC SCIENI JFlC REPORTS _______________ 330 326 310 300L-_____________ 251 241 246 244 240 1/T~.O DESCP.IPTOR NAME LANGUAGE AIJTOMATIC KACHlI-lE TRANSLATJON PKOGRAMMING IIATHEHATICAL COMPUTER DATA DIGII.I\L COMPUTERS M;lfHt:MAT I CAL lOGIC COMPUTERS UATA STORhGE SYSTEMS DISPLAY SYSTEMS 2356 1')89 1460 1212 1111 1001 9),8 918 915 765 ~DI~ M3 COMMUNICATIONS THEORY. 610 DESCRIPTOR NAME I/N.O PATA STORAGE SYSTEMS 1952 I~OHPUTERS !1Y (". w,""y.erefjl'C6 19_Q_6~_ _ _ _ _ _ __ ,-IATHlMATlCAL LOGIC . i 1769 ." DI\U PROCESSING SYS TENS· WI '+FP.ft61_5~4~2_________- - - "--M'4:fHll'lATiCAL COMPUTER DAJA' 1529 . CODING O'f C. ,tr?P- . _ 1239 OiSPlAY SYSTEMS I 1139 Y', Figure 1. ~0-"~t29~~0.-fAPERS()N--IHE_ APT LAN~~~G!; _ _ DfSCR(PTC_R_.~_~_E_________~l~_ AUTOl'ATIC MATHEMATICAL .l'lAI'\GUAGE ,/ PRCGRAf'lMING CC~PUTER 3365 DATA-' ····-·-····-3050-·------2473 ----------------2375"----------- g~~~~~ETICS ----.-.--------~~~~.---------HUI'AN ENG If\EER I NG 1519 CO~~UNICATI(j~S THECRY·--- --. - - - - - ' - - 1 5 0 0 - - - · - - - TRACKI"G 1265 CO~PUTER -LOGIC .--- ------- -----.-- 1 2 1 2 - - - - - - - - - SCIENTIFIC R(PCRTS 1205_ __ --J--foIACHINE TRANSL..-TIOi---·------1020 DESCRIPTCR ·NAME - . -"/LANGUAGE ---_ ... -_. ---- --l/TN-.O -" ... ----------9819-------- --7::~~~~~"'~~~NSnYlciN------~!!~'-------AUTOMATIC 6372 . ,..ATHEHATICAl CC"PUlER"'OAlA'" 6262 - - - - - - DESIGN 5472 DIGITAL COlo:PUTERS . --·-------3366 ~~;~~~~~~-T~qt\S }HECRY ig~~'-------- DATA STORAGE SYSTEI'S 27<;0 MATHEMATICAL LCGfC---------2730-SCIENrIFIC__ ~E?_O~~ _____ . 245J _ _ _ _ _ __ DESCRIPJ'JR NAIIE 1/N·JL _ _ vLANGUAGE I""'PROGRAf'MING - - - - - - - - - 21251 --14n9- _ _ ·¥"·=~~~!~~I~.RA.N~_L~_U~" _ _ _ _ _ ~~~!!. _ _PESIGN __________________ MATHEMATICAL C(MPUTER DATA DICITAl COI/PUTERS . COMPUTERS ...... - -._. --vg:~ :~~g~~~h~-~-S.~~~~-E-MS --2~~~·~={I}~~-~-ti;~~A~c-RY Figure 2. ________ 11Jl28_ _ _ _ _ _ _ _ 8618 7524 6018-------- ~:~~~---!-~-~.~"------ TRAINING A COMPUTER TO ASSIGN DESCRIPTORS TO DOCUMENTS: SADSACT technique using cited titles as against abstracts as input. In our earlier tests, the same items had been run both with cited titles and with abstracts against the teaching sample in which they had been included (that is, as both partially and entirely source material items). For these items, the 64.87r hit accuracy for the cited title case improved to 75.07c average accuracy in the abstracts case. However, when the new test items were run both ways against teaching sample #2, there was no significant difference in the average accuracies obtained as between using titles-and-cited-titles only and using titles-and-abstracts from the same items. Problem of Evaluation The problems of evaluation of any indexing method whatsoever, whether carried out by man or by machine (or by men aided by machines) are crucial. O'Connor 12 has aptly pointed out that: " ... the question, 'Is mechanized indexing possible?' by itself is not answerable, because it is not intelligible. More positively expressed, if you want to ask clearly about the possibility of mechanized indexing; you should have both some notion of 'good retrieval,' and some notion of hO\:tl good YOLI \vant the retrieval to be which is permitted by mechanized indexing." 569 In advance of evaluations based on retrieval tests, we have relied, first, on comparisons with the prior human indexing. We have been faced with the mechanics of assuring consistency-ofindexing criteria as necessary for any realistic assessment of machine as compared to manual indexing of the same items. As we have noted, this is not possible even for "training" and "test" items which are all documents of interest in our collection and which were all indexed in approximately the same time period by the same DDC (ASTIA) indexer-staff. As between "teaching sample # 1," "teaching sample #2" (of 100 and 103 items, respectively, with 70 % overlap), and of test items "new" to both these samples there are obvious differences as to the descriptor-vocabulary and indexing philosophy. This type of discrepancy was even more marked in the first run of eight (1963) items against the (1960) basis of teaching sample # 1. More serious, however, are the questions of the O'ConnQr remarks previously cited: those of evaluation of any indexing method whatsoever. We have, in our test data, a provocative case in point. In Table 4 we show the machineassigned descriptors for the paper by L. G. Roberts of l\1.I.T. entitled "Pattern recognition with an adaptive network." Although exactly 12 descriptors were required, by our present TABLE 4. Descriptors assigned by computer to "Pattern Recognition with an Adaptive Network" Data Sampling Learning Simulation Computers Data Processing Systems Design Digital Computers Mathematical Prediction Mathematical Logic Digital Systems Mathematical Computer Data 570 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 program, to be assigned, none of the 12 so assigned by machine is entirely irrelevant or inappropriate. As it happens, this same document was indexed twice by nnc personnel, first as an unpublished report, later as a reprint of the version appearing in the open literature. Indexer A assigned to this item the four descriptors: "Coding," "Digital Computers," "Mathematical Computer Data," and "Programming." Indexer B assigned: "Digital Computers," "Electrical Networks," "Identification Systems," "Programming," "Simulation," and "Test Methods." It is of interest, first, that although our machine method "hit" score was 50 % with respect to Indexer A, and only 33.37c with respect to Indexer B, at the same time A's agreement with B was exactly 33.3 % and B's with A was 507c. More significantly, however, the machine method was the only one of the three which assigned the crucial descriptor "Learning" (reflecting the self-organizing and adaptive features of the mechanism described) to the item. Indexer B assigned two inappropriate descriptors-"Test Methods" and "Electrical N etworks," but the SADSACT technique did not make either mistake, even though the word "network" did appear in the item's own title. A second problem of evaluation we are faced with involves the consequences of an assumption that the previously-given hurpan indexing decisions, for a sampling of our collection, are in fact "good enough." This is a misleading assumption. As a case in point, of the approximately 200 items handled either in the teaching samples or as "new" items, only one could be appropriately indexed under the descriptor "Computers" that also involved analog com- these items, if "Computers" was a relevant puters or systems. Thus, for all but one of descriptor, so also was the descriptor "Digital Computers." There is considerable debate about claims for "consistency" of machine, as opposed to manual, methods of indexing. In some of our tests, for example, we assigned both the descriptor "Computer" and the descriptor "Digital Computers" in 41.2 % of the cases in which either was assigned, but the human indexers assigned both if either was used only 33.3 % of the time. In general, very little data are available on interindexer consistency or on the likelihood that the same indexer would index the same item the same way on different occasions. Painter 13 reports figures for interindexer consistency as ranging from 5470 similarity of indexing to 82 %. Her data for the case of DDC show that: "One hundred and eleven starred descriptors were used the first time which did not appear the second (41 %) and 98 starred descriptors were used the second time and not the first (36% )." Jacoby and Slamecka7 report even less favorable figures, with interindexer consistency as low as 18 % in some cases. In the light of such indications, the automatic assignment indexing techniques appear promising. In addition, however, we have initiated the investigation of relevance jUdgments on the part of subject matter specialists reviewing the full texts of our test items. Evaluation by Relevance Judgments The data shown in Tables 1-3 compare machine assignments with those previously made by human indexers. An alternative method of evaluating the test results is to determine the TABLE 5. Average per-cent agreement, one or more people with machine assignments No. of Indexers 2 3 4 5 No. of Items No. of Descriptors Assigned 12 9 6 3 1 1 4 10 10 50.0 47.9 40.0 54.2 66.7 52.7 45.7 58.9 66.7 5.8.3 55.6 68.3 100.0 66.7 53.3 80.0 100.0 75.0 60.0 90.0 OVERALL 47.4 52.9 61.6 67.9 76.0 TRAINING A COMPUTER TO ASSIGN DESCRIPTORS TO DOCUMENTS: probable relevancy of those descriptors that are assigned by the machine. Twenty-five of the items in the test runs have therefore been submitted to one or more members of our staff, all of whom are users of the collection. They were asked to choose 12 descriptors for each item exclusively from the list of descriptors actually available to the machine. The results of these evaluations are shown in Table 5, which gives the summary data for judgments of relevance by one or more persons (including the initial DDC indexer) of descriptors assigned by the SADSACT technique. It should be noted that these judgments are based upon the independent assignment by the human analysts of these same descriptors to the same item, rather than by review of the machine assignments. Thus the reslJ-lts are probably conservative. The results in Table 5 also show the percentages of machine-assigned descriptors judged by human analysts to be relevant for cases where the machine assigns fewer than 12 descriptors. It is evident that the fewer descriptors assigned by machine, the better the chance that human evaluators will judge those assi-gned to be relevant. It is also evident that the more the people who index, the better is the chance that one or more will assign a descriptor that the SADSACT model also assigns. Tables 6 and 7 provide more detailed data for the cases where independent evaluations were made by several people. Table 6 shows machine-ind'exer and interindexer agreement for the 10 cases where four of our own "customers~~ had analyzed the items. It provides, for each of th( different descriptors assigned by machine at th~ 6-descriptor cutting level, indication of the iem for which these assignments occurred, and the agreement with assignments of that descriptor. Table 7 gives, for 20 cases in which four or more people made independent evaluations, the chances that the machine will choose descriptors also chosen by people for the first descriptor assigned, first two, ... up to the total of 12. Taking both the "hit" accuracy an.d the human agreement data into account, the' SADSACT results compare favorably with those reported by Maron!l (51.870) and by Bork03 in an experiment using the same computer literature abstracts as had been used by Maron 571 (46.5 %). Our results to date also appear to fall within the range of agreement-data for human interindexer consistency. Future Considerations Our present plans call, first, for continued testing and evaluation of a number of new items, varying such parameters as the size of the teaching sample, the number of descriptors to be assigned, and weights for both priorassociation occurrences and descriptor selection. This additional experimentation can be carried out using the present preliminary program. Depending on the results of continued testing, we except to explore applicability of the method to other subject matter areas (both for more homogeneous and more heterogeneous collections) if they are favorable, and to study means for improvement by various means of reintroducing human judgment into the process, if the results are discouraging. If the promise of the early results is substantiated on further runs, we shall also wish to re-investigate and refine the programming techniques. For our preliminary evaluation, many ad hoc choices and arbitrary assumptions were made. These will require modification. Additional programming refinements should provide for consolidated vocabulary-descriptor associations for larger teaching samples, incorporation of occurrences of n-tuples of significant words as "potential descriptors," experimentation with various weighting formulas, and the like. The SADSAGT indexing procedure differs from most other efforts at realization of assignment-type indexing by machine in two principal respects. First is the emphasis on a posteriori, completely mechanizable, techniques that nevertheless are based, in a sampling sense, upon prior judgments of people with respect to specific requirements of real jobsituations. The second differentiating aspect is the insistence not only upon obtaining the consensus of the collection as the basis of vocabulary compilation and word-descriptor associations, but also of going beyond the individualistic usage of language and terminology by a particular author. This latter emphasis tends to reduce the effect of linguistic idosyncracies and consequent scatter of like items under a number of different indexing terms. TABLE 6. Agreements, by descriptor and item, of one or more persons with first six descriptors assigned by machine to each of 10 test items I~ "'C % Number of persons agreeing with machine assignment Descriptor Coding 5 CCCS*, LRBE, CPLT Theory SHT Errors 4 WLI, SCC LRBE 66.7 100,0 SCC tz:j t;:j i t:I Z GJ rn. "'C ~ zGJ c..... 0 z 1-3 (") CCCS, LRBE 0.0 0 a:: "'C c:::: 1-3 TTDI CCCS 100.0 tz:j ~ 100.0 0 (") WLI, CPLT, PAL CPLT WLI PAL z"'%j tz:j ~ tz:j 66.7 Z 66.7 J:r:j ..... (") ASDS MTSA ~ 0) H:>o Intelligibility Bibliography 100.0 WLI WLI, TTDI Design Classification TTDI 100.0 cecs Data Processing Systems - 0 100.0 CCCS,CPLT Machine Translation Automatic Synthesis 1 CCCS, LRBE Language Human Engineering 2 TTDI Data Transmission Systems Electronic Circuits Information Theory 3 Agreement Assignments of this Descriptor ~ 0 0 MTSA ASDS SHT SHT 1\1TSA PAL 0.0 66.7 100.0 100.0 MTSA LRBE 0.0 LRBE 0.0 Data Storage Systems Data ASDS, PAL MTSA 100.0 ASDS Mathematical Logic Communications Theory Programming Digital Computers Probability Reasoning Computers Circuits Tracking Documentation 0.0 0.0 50.0 SCC 1-:3 100.0 CPLT, PAL I~ t-I Z 100.0 PAL, SHT CPLT TTDI Iz Cl 100.0 SCC TTDI SHT SHT SCC *N ote: Items are identified by initials of first 4 significant words in titles, see Tables 1-3. SCC MTSA 50.0 :> (i 0 ~ ~ 100.0 c::: 100.0 0.0 t,%j ~ 0.0 1-:3 1-:3 0 :> rn. rn. t-I Cl Z t:I t,%j rn. (i ~ t-I ~ 1-:3 0 ~ llJ 1-:3 0 t:I 0 (i c::: ~ t,%j Z 1-:3 rn. at -:t ~ 574 PROCEEDINGS---:SPRING JOINT COMPUTER CONFERENCE, 1964 TABLE 7. Number of descriptors assigned by machine, also chosen by one or more human indexers Item 1 2 3 4 5 6 7 8 9 10 11 12 CCCS 1 2 3 4 4 5 6 6 7 7 7 7 ICC 1 2 2 3 4 5 6 7 8 8 8 8 UCCT 1 2 2 3 4 5 5 5 5 5 5 5 MCL 1 2 3 4 5 5 5 5 5 6 6 6 WLI 1 1 1 1 2 2 3 3 4 4 5 5 MCGE o 1 1 3 4 5 5 6 7 7 7 MTSA 1 2 3 4 4 4 5 5 5 5 5 5 lAM 1 2 2 2 3 3 3 3 3 3 4 5 LRBE o o 1 2 3 3 4 4 4 5 5 6 ASDS 1 1 2 2 2 3 4 5 5 5 5 6 CPLT 1 2 3 4 5 6 6 6 6 6 6 6 SRPC o 1 2 2 2 3 3 3 3 3 4 4 RPSN o o o 1 3 3 4 4 4 1 2 3 4 2 5 2 PAL 1 5 6 6 7 7 8 8 TTDI 1 2 3 3 3 4 4 4 4 5 6 7 PM SO 1 1 1 1 1 1 1 1 1 1 1 1 HPFW 1 1 1 1 1 1 2 2 2 2 2 2 SHT 1 2 3 4 5 5 6 7 7 1 2 3 4 4 5 5 6 7 7 CIO,}> o 1 2 3 4 4 5 4 SCC 4 4 4 6 6 6 6 6 6 70.0 66.7 66.3 64.0 62.5 60.7 56.2 52.8 51.0 49.1 47.1 Average Per-cent Chance that people 75.0 will agree It also tends to produce a normalizing effect so that the various items in the collection are indexed more consistently than is possible if each item is processed as an entirely independent and self-sufficient entity. The word "training" as applied to our proposed method for automatic indexing is somewhat of a misnomer in terms of our actual experiments to date. It is justified in the sense that a training sequence, using a "teaching sample," gives the machine a representative set of "acceptable" input-to-desired-output examples. It is not justified, as yet) in the sense that, the system as presently programmed could significantly modify or adapt its procedures, without human intervention, to new vocabulary or subject emphases in the input items. There is reason to suppose, however, that the method can be adapted to new and changing vocabularies (both of "descriptors" and of author terminology) by throwing out for human inspection: occurrences of words not on the general stop-list, yet not in the vocabulary; and occurrences of non-purged words in title-andcitations coinciding with the name of a descriptor, when that descriptor has not been assigned (e.g., an early occurrence of the word' "plasma" in a physics paper, but not involving enough other words previously associated with blood chemistry, hematology, and the like, to assign physiological or medical field descriptors) . If this proves feasible, potentialities of TRAINING A COMPUTER TO ASSIGN DESCRIPTORS TO DOCUMENTS: 575 training the machine system to recognize new descriptor-assignment possibilities, to sharpen its own recognition-selection-assignment processes in terms of current real-life input, and to recognize and display anomalies, changes, and trends, may supply significantly improved problem-solving capabilities in the changing world of literature search as aided by machine. Although the SADSACT results to date are quite limited, the procedure indicates some promise of being quick, relatively inexpensive and consistent, and it capitalizes on one-time key stroking of descriptive cataloging information available for multiple purposes. 8. 9. 10. REFERENCES 1. BAKER, F. B. "Information Retrieval Based upon Latent Class Analysis." Journal of the Association for Computing Machinery, 9:4 (October 1962) 512-521. 2. BAXENDALE, P. B. "Machine-Made Index for Technical Literature - An Experiment." IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 2:4 (October 1958) 354-361. 3. BORKO, H. "The Construction of an Empirically Based Mathematically Derived Classification System." In PROCEEDINGS 1962 SPRING JOINT COMPUTER CONFERENCE, American Federation of Information Processing Societies, 1962, pp. 279-289. 4. BORKO, H., and M. D. BERNICK. "Automatic Document Classification." JOURNAL OF THE ASSOCIATION FOR COMPUTING MACHINERY, 10:2 (April 1963) 151-162. 5. EDMUNDSON, H. P., and WYLLYS, R. E. "Automatic Abstracting and IndexingSurvey and Recommendations." COMMUNICATIONS OF THE ACM, 4:5 (May 1961) 226-234. 6. HERNER, S. "Methods of Organizing Information for Storage and Searching." AMERICAN DOCUMENTATION, 13:1 (January 1962) 3-14. 7. JACOBY! J., and V. SLAMECKA. "Indexer Consistency under Minimal Conditions." 11. 12. 13. 14. 15. Bethesda, Md., Documentation Inc., Nov. 1962 Iv. Contract AF 30 (602) 2616, Project 4594: RADC TR 62-426: AD-288 087. LUHN, H. P. "A Statistical Approach to Mechanized Encoding and Searching of Literary Information." IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 1:4 (October 1957) 309-317. MARON, M. E. "Automatic Indexing: An Experimental Inquiry." In MACHINE INDEXING, PROGRESS AND PROBLEMS, Washington, D. C., American U., 1961, pp. 236-265. MARON, M. E., and J. L. KUHNS. "On Relevance, Probabilistic Indexing and Information Retrieval." JOURNAL OF THE ASSOCIATION FOR COMPUTING MACHINERY, 7:3 (July 1960) 216-244. NEEDHAM, R. M. "A Method for Using Computers in Information Classification." In INFORMATION PROCESSING 1962 (Proceedings of IFIP Congress, Munich), Cicely M. Popplewell, Editor (Amsterdam, North-Holland Publishing Co.), pp. 284287. O'CONNOR, J. "SmIle Remarks on Iviechanized Indexing and Some Small-Scale Empirical Results." In MACHINE INDEXING, PROGRESS AND PROBLEMS, Washington, D. C., American U., 1961, pp. 266-.:279. PAINTER, A. F. "An Analysis of Duplication and Consistency of Subject Indexing Involved in Report Handling at the Office of Technical Services, U.S. Department of Commerce." Rutgers State U., Ph.D. Dissertation, 1963, p. 135. Washington, D. C., Office of Technical Services, PB 181501. STEVENS, IVr. E. "Preliminary Results of a Small-Scale Experiment in Automatic Indexing." To be published in PROCEEDINGS, NATO Advanced Study Institute on Automatic Document Analysis, Venice, 1963. SWANSON, D. R. "Automatic Indexing and Classification." Paper presented at the NATO Advanced Study Institute on Automatic Document Analysis, Venice, 1963. EXPERIMENTS IN INFORMATION CORRELATION* J. L. Kuhns and C. A. Montgomery The Bunker-Ramo Corporation Canoga Park, California 1. INTRODUCTORY REMARKS ability to assimilate and correlate, thus extending the critical limit? It is reasonable to suppose that information systems of the futp.re will have control over portions of a document as well as over full text. In addition, we suppose that many users will want output in order to verify hypotheses or to form new hypotheses. This paper is concerned with the facilitation of the second procedure. Thus the emphasis is not on the two classical problems of information systems, namely, not retrieving relevant material, or retrieving irrelevant material; but rather on the problenl of assimilability of the output by the user. Experiments performed in connection with an earlier project on information retrieval and correlation provided some interesting preliminary answers to these questions. The library for this series of experiments consisted of a simulated intelligence file of sentence fragments developed from a book'~Kogun-The History of the Japanese Army in the Pacific." A machine-produced concordance of all words contained in the fragments served as an index to the file. The particular aspect of the assimilability problem which will concern us is the effect of information loads of varying sizes upon the user's ability to assimilate and correlate data. It seems plausible to assume that the user's ability to correlate information is a function of the volume of information to be assimilated; and moreover, that this ability will improve with increasing volume until a certain critical volume is reached, beyond which the load of information causes the user's correlation ability to deteriorate rapidly. This assumption gives rise to two fundamental questions: The experiment 'of particular significance here.is one which dealt specifically with the time dimen,sion. All fragments containing any reference to time of day (for example, "noon," "0645," "dawn," "8 :00 a.m.") were retrieved. About 25 fragment cards were retrieved in all. The fragments of this set had at least two attributes in common: they concerned the Pacific War in some way, and contained some reference to time of day. The randomly ordered cards were presented to several persons, who noticed only that all cards referred to battles in the Pacific War. When the cards were reordered into two groups of night versus day, however, a striking correlation emerged-namely, that all daylight attacks had been initiated by U.S. forces and all 1. What is this critical volume in a given situation? 2. Is it possible to structure information in some way so as to improve the user's * The work reported in this article was supported by the National Aeronautics and Space Administration under Contract NASw-538. 577 578 PIROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 night attacks by the Japanese. (This correct inference, incidentally, was nowhere explicitly stated in the book.) The results of this experiment suggest that the critical volume of information may in fact be extremely small. Only 25 cards were involved, and each card contained a single fragment, which in this case was a minimal piece of information. Moreover, these fragments had certain attributes in common-"Pacific War" and "time of day." The correlation described above, however, was not discovered until the set of fragments was structured in terms of conceptual subgroups-"night," "day"-included in the semantic range represented by "time of day." These findings suggest that structuring of data can extend the critical limit on volume, and further, that the elements of this structure should be subsets consisting of closely related members of the main set. The results of the earlier experiment thus established a point of departure for the present investigation. 2. EXPERIMENTAL DESIGN A considerably more complex series of experiments was designed to test whether the "Kogun" discoveries provided essentially correct answers to the two questions raised above. We decided to create a set of data structured along the lines described above-that is, a set whose structure is based on subsets consisting of closely related members of the main set. A second set of data more loosely structured than the first would be provided by the main set itself. A third set would consist of unstructured data compiled randomly. The ability to assimilate and correlate the information presented in these variously structured sets of data would then be tested by requiring experimental participants to list inferences suggested by each set. Based on the "Kogun" experiment, it was assumed that the first set of data, being the most highly structured, and therefore, most easily assimilated, would be most productive of inferences; and the random data, least productive. The scope of the initial experiments was limited to generating and testing the first set of data. The type of structure represented by the first set of data is most conveniently generated by "clustering" techniques. The object is to cluster the data in natural groupings whose members are associated through co-occurrence of concepts. The procedure is as follows: if two items of the data set have a concept in common they are said to be associated. A cluster is then defined as a collection of items which are mutually associated as well as being maximal with respect to this property. This last condition means that an item cannot be added to the collection without violating the property of mutual association. The definition of association can be strengthened by stipulating that several concepts must occur in common. The technique used is a mathematical procedure for exhaustively listing these clusters. 3. EXPERIMENTAL PROCEDURE Materials to be used for the creation of the three data sets were a fragment file of sentences generated from a collection of documents on advanced propellants and a machine-produced concordance which served as an index to the file. The first step in the experimental procedure consisted of selecting a single fragment called the F 1 fragment, which would serve as the Figure 1. Illustration of F 1 star. EXPERIMENTS IN INFORMATION CORRELATION nucleus for a set consisting of all fragments related to the F 1 fragment through co-occurrence of concepts. This is diagrammed in Figure 1. The lines between fragments indicate the co-occurrence of concepts; for example, if F 1 contains the words "small motors" and F:! contains the word "micro-burner," then the same concept can be said to occur. The set so formed is called the Fl "star." Clusters of fragments associated by co-occurrence of concepts in addition to the F 1 defining concepts can then be derived from the Fl star. As an illustration, in Figure 1 the set {F 1, F:!, F;{} is a cluster, so are {FJ, F .. }, {F), F:;}, {F), F H }. These clusters can serve as the maximally structured set of data for the experiments, while the members of the F 1 star would constitute the loosely structured set. Since the magnitude of the task of developing these structures manually is directly proportional to the number of fragments involved, it is essential that the membership of the star should not be too large. On the other hand, if its membership is too restricted, the star will not contain enough information for valid generalizfttions to be made. As the membership of the Fl star depends on the number of concepts contained in the F 1 fragment and their distribution throughout the fragment file, it can be seen that selection of the F 1 fragment is the crucial step in the experimental procedure. Attempts to limit the size of such stars by arbitrarily restricting the number of fragments taken from each document proved unsuccessful, as did several other artificial means of coping with an oversupply of related fragments. Many potential F 1 fragments were selected, found inappropriate, and rejected before the following F 1 fragment was chosen: "Space mission studies have indicated the need for small rocket motors of low thrust (of the order of 100 lb.) for attitude control, midcourse trajectory corrections, rendezvous, and so forth." Concepts used to define the F 1 star were as follows: small and motors or micro and motors burners burners engines engines 579 space trajectory control and attitude rendezvous All cards containing these concepts were then retrieved from the file. There were 24 in all, yielding an F 1 star of 24 members, which proved to be a workable amount. A list of concepts was then developed for each of these fragments and the list for each fragment was compared with that of all other fragments in the Fl star to determine which fragmen~s had the greatest number of concepts in common. A correlation matrix was created in which fragments having two or more concepts in common were labeled as strongly correlated, those having only one common concept as weakly correlated. "Strong" and "weak" clusters were then derived from the matrix by the mathematical technique mentioned above. Since a greater degree of relatedness or structure is required by the defining criteria for strong versus weak clusters, these clusters represent legitimate materials for testing the user's ability to assimilate and correlate information having various degrees of structure. It was expected that inferences derived from strong clusters would exhibit some distinctive property as compared to weak clusters. A further test of assimilation and correlation ability could be achieved by imbedding the clusters in successively larger segments of text, on the theory that correlations obvious in the clusters as represented by the fragment cards would be obscured if the clusters were instead represented by the documents from which they were culled. We therefore selected 12 clusters -6 strong and 6 weak-which were represented by four levels of material: Level D Description Cluster is represented by the fragment cards. C Cluster is represented by the sentence on the fragment card plus the six sentences (3 preceding, 3 following) which form its immediate context within the document. 580 B A PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Cluster is represented by a machine-produced extract of the document. Cluster is represented by the document itself. The clusters were then divided into four groups of three clusters each. The material for level A of the first group of clusters was placed in a folder marked for the first participant along with the level B material for the second group of clusters, the level C material for the third, and the level D material for the fourth, so that none of the four participants (specialists in the field of advanced propellants) would have the opportunity to draw inferences from the same cluster on two levels. These elaborate precautions were to prevent the participants from becoming familiar with the same cluster through its representations on various levels, causing a tendency to make the same inference for the different levels rather than looking for new correlations which might be exposed in proceeding to a level where the cluster is less deeply imbedded in peripheral material. The participants were instructed to begin with level A and proceed to level D, observing the time limits (given below) imposed- for reading the material and making inferences: Table of Maximum Times Level Reading Time A B C D One-half hour 20 minutes 15 minutes 5 minutes Time for Deriving Inferences 20 20 20 20 minutes minutes minutes minutes 4. RESULTS A total of 152 inferences were listed by the participants. Of these, exactly half were derived from strong clusters and half from weak. The level from which the largest number of inferences was derived was the A level, which also contained the most material. 51 inferences were listed for the A level, 42 for the B level, 34 for the C level and 25 for the D level, indicating a strong correlation between quantity of material and number of inferences produced, which we had intended to offset to some extent by imposing time limits for reading and infer- cel1ce listing. The S-16 cluster (representing level D) and the inferences derived from it are listed below. S-16 Cluster (Level D) This motor uses small quantities of propellants which can be made on a laboratory scale yet is large "enough that the heat transfer losses are small. The combustion gases are cooled to a much greater extent relative to that in a larger motor, thus lowering c (characteristic velocity) . Space mission studies have indicated the need for small rocket motors of low thrust (of the order of 100 lb.) for attitude control, midcourse trajectory corrections, rendezvous, and so forth. Inferences for Cluster S-16 1. Inferences- at .A-level 1. Propellant evaluations do not apear to be standardized. 2. Inferences at B-level 1. Hypergolic rocket propellants permit optimum utility of small thrust engines. 2. Injector configuration is a critical parameter in achieving maximum performance of small, low thrust rocket engines. 3. High energy, toxic rocket propellants can be handled without hazard. 3. Inferences'at C-level 1. Propellants can deteriorate upon standing over long periods of time. 2. Impurities may be the cause of uncontrolled burning and/or detonations and decrease performance of propellants. 3. In small motors heat transfer through components lowers performance significantly. 4. Better mixing of propellants increases performance. 5. N 2 0 2 and N2H4 are the best state-of-theart combinations which satisfy requirements. 6. Testing on small sub-scale is sufficient for accepting or rejecting a propellant combination for use on larger motor tests. EXPERIMENTS IN INFORMATION CORRELATION 581 4. Inferences at D-level 1. Low thrust, efficient, simple rocket motors have a variety of potential research and development applications. 2. In a small motor, combustion gases are cooled to a greater extent than in a large motor probably because of the greater ratio of heat transfer surface to combustion chamber volume in the small motor. The inferences were then uniquely classified into the following categories: Propulsion Systems Propellant Combinations Propellant Evaluations Reliability Performance Design Chambers Storage and Handling Combustion Efficiency Combustion Stability Ignition Rea t Transfer Problem Areas Extrapolation of Test Results Applications for Small l\10tors The inferences derived on the various levels were compared to determine whether the classes of inferences had members from all levels; or whether, as we had postulated, the relations between members of a cluster. would be obscured when the cluster was represented by large segments of data, so that the same type of inference could not be derived from all representations of the same cluster. This assumption proved false in the present investigation, which might be attributed to the fact that the critical volume was not reached due to the overgenerous time limits. The same types of inference were frequently listed for levels A through D. The original fragment cards making up the clusters were then assigned to as many of the above categories as were applicable in order to compare the extension of the clusters with that of the inferences derived from them. Intersections of categories were plotted for the clusters and for inferences made at the D level-i.e., those derived directly from the clusters themselves. A correlation coefficient was used to determine the degree of "overlap" between clusters and inferences, thus measuring their conceptual closeness. Although the correlation coefficients for both strong and weak clusters were good,. inferences derived from strong clusters proved significantly more highly correlated with their parent clusters than did inferences derived from weak clusters. The strong clusters scored .61, the weak ones .43, where the coefficient is on a scale from -1 to +1 with 0 being the point of random correlation. Although these data support to some extent the trends indicated by the "Kogun" experiments, they also raise some interesting questions which can only be answered by comparative data of the type outlined above in describing our design for these experiments. One of . these questions arises from the fact that the participants were able to recognize relations between members of a cluster and make inferences based on these relations even when: the cluster was represented by large segments of text. Would the same apply for more loosely structured sets of data-if the 24 members of the F 1 star, for example, were imbedded in successively larger segments of text, would the basic relation still be discernible, or would inferences derived be diffuse and general rather than highly correlated with the star from which they were derived? vVhat sort of inferences would be derived by applying the layering technique to a random selection of fragments related only by coming from literature in the same field? A second type of question is. more thoughtprovoking, and less readily answerable. It departs from the notion of an inference as a measure of the ability to assimilate and correlate information and considers instead the nature of an inference. We have discussed the correlation betwen basic data and inferences derived from these data, but is it reasonable to expect such correlation to exist? Perhaps the mark of a good inference is that it exhibits little conceptual relatedness to the data from which it was derived. Again, what is a "good" inference? Should "good" be interpreted as "non-trivial"? How much information should be carried in an inference? A systematic approach to these problems is explored in the following section. 582 PlROCEEDINGlS-SPRING JOINT COMPUTER CONFERENCE, 1964 5. QUANTITATIVE EVALUATION OF INFERENCES In developing a systematic procedure for evaluating inferences, it is convenient to think of the presented information set S as having three components: information content, volume, and structure. Let "s" denote the proposition expressed by the content of the set, and suppose that the analyst, after examining the set, produces the inferences (hypotheses, conjectures) h], ... , hu. By varying the structure, or volume, or content of S, different collections of inferences will be generated. The problem is then to develop a comparative evaluation of the "goodness" of the two collections. In order to do this we consider a quantitative evaluation based on the notion of a subjective probability function or credibility function or betting function. Such functions are discussed in References 1, 2, 3. We denote this function by "B" (for "betting function" or "belief"). Thus B (h, s) is the analyst's evaluation of the degree of belief in the hypothesis h after considering the evidence s. B (h) is his degree of belief in the hypothesis h prior to considerings. We suppose that B obeys the laws of probability so that B gives the value of the betting quotient (ratio of amount bet to total stake) that should be offered for a bet on the hypothesis in question. * In addition, the quotient should be psychologically fair in the sense that the bettor must alternately be willing to bet on the negation of the hypothesis with the complementary quotient. Let us suppose for the present that we actually can obtain three fundamental values B (h), B (h, s), and B (s). Since the "goodness" of a collection of inferences should in some way depend on their validity, novelty, dependency on s, and non-triviality, let us see how the'se things can be defined in terms of the B-function. It seems appropriate to define the novelty of h to be B (11), where Ii is the negation of h. Thus the novelty of h is taken to be B(h) == 1 - B(h) i.e., the complement of the degree of belief in the inference before examining the supporting ::'Thus betting schemes based on B will not lead to a net loss in every possible case. (See Reference 1, 3.) data. We define the validity of the inference to be B (h, s) ; i.e., the degree of belief after examining the supporting data. It appears that the "score" of an inference should increase with increasing validity and novelty and that the product of validity and novelty would therefore be a good choice for the score. On the other hand, an inference quite independent of s should score zero. Independence is characterized by the condition B(h, s) =- B(h) so that the product of validity and novelty for an independent inference is B (h). B (h). Thus, we propose to score inferences as the product of validity and novelty less the independence value of this product. That is to say, we take the final score to be G(h, s) == [B(h, s) -B(h)] . B(h) The goodness of the inference is therefore measured as the product of the change in degree of belief and the novelty. Let us now test this scoring schema for various extreme conditions: (1) h is tautologous (e.g., h is "A is A"); (2) h is contradictory (e.g., h is "A is not A") ; (3) h is trivial (e.g., h == s); (4) s logically implies h (deductive inference) ; (5) h is independent of s (the supporting data does not relate to h). These five cases are shown in the table below. As well as showing the notions of validity and novelty, we also include the notions of "strength of supporting data" and "relative validity." We now examine how the procedure would apply to some actual data. We use the "Kogun" experiment discussed above. The hypotheses to be considered are: (hd Japanese attack at night in a particular case; (h:!) Americans attack in day in a particular case; (h a ) Japanese almost always attack at night; (h.d Americans almost always attack in day: (hz;) h3 and h4 (i.e., the assertion of the conjunction of ha and h 4). The supporting data s is that given by the 25 fragment cards. Since the source of the data is from the book "Kogun" and consists of supposedly factual reports, let us assign a rather high value to B (s), say 0.9. Disregarding s, we consider each of the hypotheses hI and h2 as quite plausible TABLE OF EXTHEME CONDITIONS Notion Definition his tautologous his contradictory h is trivial (h == s) s logically implies h h is independent of s t.%j >< '"d Strength of Supporting Data B (s) B (s) B (s) B (s) B (s) B (s) t.%j ~ Relative Validity B (s and h) B (s) 0 B(s) B (s) B(s) B(h) t.%j Novelty B (h) 0 1 B(s) B (11) B(h) Z ~ rn Validity B(h, s 1 0 1 1 B (h) Z (Novelty) x (Validity) B(h,s)B(li) 0 0 B(s) B(li) B(h) B(1i) z ~ ~ ~ ~ Score [B (h, s) - B (h)] B (h) 0 0 B2 (s) B2(h) 0 o ~ ~ > ~ ~ o Z (") o ~ ~ t.%j toot >-~ ~ o Z <:it 00 ~ 584 PlROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 and, therefore, assign a high a priori degree of belief to both, say 0.9. Thus, the novelty of each is 0.1. The analysis of the hypotheses ha and h4 is more subtle. Presuming we know nothing about the Pacific War, we imagine that perhaps the specific military situation would mean that sometimes night is better fo~ attack, sometimes day. On the other hand, perhaps the general properties of the military situations of the kind discussed are such that night is always better or day is always better for an attack. This second consideration makes us think that B (h3) is near 0.5 (because we cannot choose between night and day). The first consideration makes us think that B (h a) is less than 0.5. Therefore, we take B (h a) == 0.4. Similarly B (h .. ) == 0.4. The hypothesis hr;, however, is completely surprising, so we take B (hr;) == 0.1. To estimate the validity, we note that h t and h2 are contained in s. Thus, B (hI, s) == B (h2' s) == 1. For h3 and h .. , we note that s expresses frequency of occurrence of certain situations and that the relative frequency of occurrence of these situations as described in ha and h4 is unity. Thus, we take B (h a, s) == B (h 4 , s) == 0.9 (slightly less than unity). Similarly, we take B (hl), s) close to 0.9 but somewhat less, say 9.8. * The relative validity and scores can then be computed. The results are shown in the table below and are seen to be intuitively satisfactory. TABLE OF KOGAN INFERENCES Notion Definition hI h2 h3 h4 h5 Strength of supporting data B (s) .9 .9 B (s and h) .9 .9 .9 Relative Validity .9 .9 .81 .81 .72 Novelty Validity B(h) .1 .1 1.0 1.0 .6 .9 .9 B (h, s) .6 .9 (Novelty) x (Validity) B (h, s) . B (h) .1 .1 .54 .54 .72 Score [B(h, s) -B(h)] B(h) .01 .01 .30 .30 .63 The problem of obtaining the B values is still open and it should be approached keeping the following considerations in mind. 1. The inferences obtained often contain words indicating qualitatively the degree of belief (e.g., "it appears that," "possibly," "probably," etc.) ; there is evidence that there is a comparative notion of belief susceptible of quantization. 2. There may be difficulty in obtaining an honest appraisal of B (h) after the analyst has examined the data s. Possibly this can be avoided by using a referee to make the appraisals. *The normative condition, B(h a & h 4, s) = B(h 3 , h4 & s). B(h 4, s), is considered in this appraisal. .8 3. Comparative evaluations are desired and the quantitative evaluations are for this purpose. Thus consistency in the evaluations is required only to the extent of the comparative results. 6. CONCLUDING REMARKS In this study, it was assumed: a. that a user requirement in certain information systems is the obtaining of data to form hypotheses b. that this ability to form hypotheses is dependent upon the assimilation and correlation of the data presented c. that the ability to assimilate and correlate information is in turn dependent upon EXPEnIMENTS IN INFORMATION CORRELATION the volume and structure of the information presented. A measure of assimilability called the critical volume was postulated based on the results of earlier information correlation experiments. An experimental design was developed to determine the relation between this measure and the degree of structure in the data presented. A mathematical technique was used to produce two sets of structured data. Assimilation of information represented by these data was then tested by requiring experimental participants to derive inferences from the data presented. Inferences derived from more highly structured materials proved more highly correlated with the basic data set. Thus in the course of these experiments, methods for systematically structuring data and measuring assimilability were designed and tested on a small scale. In addition, a procedure 585 for quantitatively evaluating hypotheses (I.e., the procedure for measuring assimilability) is proposed as a tool for a feedback improvement to the structuring procedure. Results of these experiments, although preliminary, are generally positive, and suggest realistic approaches to further studies of information correlation. REFERENCES 1. CARNAP, R., "The Aim of Inductive Logic," in Logic, Methodology and Philosophy of Science: Proceedings of the 1960 International Congress, Stanford University Press, 1962. 2. POLYA, G., Patterns of Plausible Inference, Vol. II, Princeton University Press, 1954. 3. SAVAGE, L. J., "Bayesian Statistics," in Re- cent Developments in Information and Decision Processes (ed., Machol, R. E., and Gray, P.), The Macmillan Company, New York, 1962. SOME FLEXIBLE INFORMATION RETRIEVAL SYSTEMS USING STRUCTURE MATCHING PROCEDURES* Gerald Salton and Edward H. Sussenguth, Jr. Computation Laboratory, Harvard University Cambridge, Massachusetts it is possible to represent the syntactic structure of the text excerpts by abstract trees, and a tree matching procedure becomes necessary to compare the extracted information with the requests. 4,5 As before, an exact matching procedure would not be very helpful, since many different ways can be found to express the same ideas or requests. What is needed instead , is a procedure which permits inclusion of partly unspecified information, and w4ich provides for the possible relaxation of the various conditions that render a complete match impossible at any given time. Graph matching techniques are, of course, also applicable to the comparison of items of information which exhibit inherently a multi-dimensional structure, such as electrical or pipeline networks, street or geographical maps, chemical molecular structures, and so on. INTRODUCTION The comparison between stored information identifications and requests for information is one of the principal tasks to be performed in automatic information retrieval. In so-called descriptor systems, where information is represented by sets of independent key words, this operation is relatively simple, sInce it consists of a comparison between the respective "vertors" of key words. In many retrieval SyStems it has been found necessary or expedi~nt to use more complicated constructs for the identification of information. Notably "role" indicators are often added to identify various types of key words, and "links" specify a variety of relations between key words. A complete identification for a document or an item of information is often represented by a graph, consisting of nodes and branches between nodes, to identify respectively the key words and relations between key words.1,2,3 The matching of such information graphs with graphs ,representing requests for information is a relatively complicated and time consuming operation, partlcularly since the request structure can be made to match the information structure only partially and incompletely. Extensive experience has been gained in the past with structure matching programs which operate on a "node-by-node" or a "piece-bypiece" basis. 6,7,8 In the node-by-node approach, the nodes of the two structures are compared one at a time, until either the complete structures match, or else an incompatibility arises; in the latter case it becomes necessary to backtrack to a point where there is agreement and try again with different elements. In the pieceby-piece approach, a dictionary of basic substructures is used to break a given structure into pieces which are then matched as a whole. The graph matching problem arises also in document retrieval systems where certain significant portions of text are extracted and compared with the search requests. I~ such cases, * This study was supported in part by the National Science Foundation under Grant GN-82. 587 588 PROCE'EDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Neither of the two techniques works well for any but the simplest structures. The node-bynode method usually requires extensive backtracking, involving the comparison of many hundreds of nodes for even very simple structures. The piece-by-piece approach, on the other hand, suffers from the fact that no standard, well-defined method exists for breaking a given structure into substructures. A topological structure-matching procedure has been programmed for the 7090 computer which does not depend on a specific ordering of the nodes, nor on the presence or absence of certain specified substructures. Little or no backtracking is required, and the method can be used to detect complete as well as partial matches. The basic idea is to determine certain simple properties of the nodes of the two structures to be matched, and to equate. those subsets of the nodes in the two structures which exhibit equivalent properties. A standard procedure is then used to form new matching subsets, and to break down already existing subsets into sets with fewer members. The procedure is completely determinate except in cases where it is necessary to resolve certain symmetries in the connection pattern of the nodes; in that case a guess. (assignment) is made as to the correct solution; such a guess may later prove to have been right or wrong, and if wrong, may require some backtracking. In most practical problems, however, little backtracking seems to be needed. t Computer experiments indicate that the topological procedure is much more efficient than either the "node-by-node" or the "piece-by-piece" approach. Operating automatic retrieval systems based on the use of relatively complex structures (as oppo~ed to sets of unconnected key words) seem therefore to become a practical possibility instead of merely a theoretically desirable goal. An example is given first to illustrate the partial matching procedure, as well as the methods which may be used to alter one or both of the structures to be compared in order to make a match between them more likely. The procedure is then applied to the matching of t A related strategy has been used by Unger to detect complete, rather than partial, isomorphisms between directed graphs.9 document graphs with request graphs, and to a retrieval system based on the comparison of syntactically analyzed document excerpts with a stored phrase dictionary. THE STRUCTURE MATCHING PROCEDUREt Consider first the problem of determining whether the graph of Fig. 1 (b) is contained in the graph of Fig. 1 (a), that is whether Fig. 1 (b) is a subgraph § of Fig. 1 (a). The nodes in the two structures are labelled arbitrarily from CD to ® and from ® to @ respectively, and the connection pattern of the nodes is represented by the binary connection matrices § shown in Figs. 2 (a) and 2 (b). Since no additional information is furnished about either the nodes or the branches of the two graphs under consideration, all relevant properties of these graphs are in fact derivable from the con- 0) DICT!OOARY STRUCTURE b) QUERY STRUCTURE Figure 1. Directed Graphs. t Theoretical foundations and proofs of convergence of the method are given in detail in Reference 10. The theory as well as applications to chemistry are also more fully treated in References 11 and 12. * A graph G consists of a set X (the nodes) and a set of relations between certain pairs of nodes (the branches). A matrix C such that C Ji = 1 whenever there is a branch from node Xi to node xj' and is 0 otherwise, is called the connection matrix of graph G. A 8ubgraph H of G is obtained by removing from G certain nodes as well as all branches adjacent to the removed nodes. A partial g'raph J of G is obtained by removing from G some of its branches. A partial 8ubg1'aph K of G is a subgraph of a partial graph of G. A completed pa.rtial 8ubgraph L of G is a partial subgraph to which branches are added so as to preserve all original paths between the nodes included in L i specifically, if node z is removed from G in forming L and if there exist paths from x to z and from z to y in G, then a path exists from x to y in L for all x, y included in L. SOME FLEXIBLE INFORMATION RETRIEVAL SYSTEMS I 2 I 2 3 4 5 6 789 I I I I I I 3 4 Set Numbe a b c I a I 5 6 b XX' XXI' XXII' XXIII' Criterion for Set Formation Assignment Corresponding Sets {a)={3) Outward connections, sets XIX' Partition (b,c)={2,5) (a)={3) (b)={2) (c)= - I I t t DICTIONARY STRUCTURE a) 1 1 1 c I 7 8 9 XIX' 589 Figure 3h. Set Correspondences Resulting from Improper Assignment. b) QUERY STRUCTlRE Figure 2. Connection Matrices for Graphs of Figure 1. reachable from a given node by outgoing (or incoming) paths of length k; nection matrices. The computer program is therefore based on the manipulation of binary matrices of the type shown in Fig. 2. The heart of the algorithm consists in using various properties of the nodes and/or branches of the graphs in order to generate pairs of sets which must match if the two graphs are eventually to match. The following properties are particularly useful for this purpose: a. the kth order outward (or inward) degree of the nodes, that is the number of nodes b. labels or identifiers which may be associated with nodes or branches; c. the connectivity patterns of sets of nodes, that is, the nodes reachable from a given set of nodes by paths of length k. The procedure for the graphs of Fig. 1 is outlined in Fig. 3. Only connections of length 1 have been used to simplify the exposition. The initial set correspondences are shown in Fig. 3 (a), lines I to IV. Set II, for example, is constructed by noting that the set of all nodes I"\f " Set Number I II III IV V VI VII Criterion for Set Formation Corresponding Sets Outward degree 1 Outward degree 2 Inward degree 1 Inward degree 2 (b) s: (l,2,3,4,6,8,9) (a) S:{2,3,4,9) (b) S:{2,4,5,6,7,8) (c) s: (2,5,7,8) Partition (a) S (2,3,4,9) {b)S (2,4,6,8) (c) S (2,5,7,8) VIII IX X XI Outward connection, sets sets Inward connection, sets sets XII XIII XIV Partition V VI VI VII (b,c) S (2,4,5,6,7,8) (c)S(2,6,7,8) {a)s (1,2,3,4,9) {a,b}s (l,2,3,4,6,8,9) --------------------------------------------------------------------------------- r I {a}S (2,3,4,9) (b}S {2,4,6,8} (c}S (2,7,8) .-------- -.--------._-----------.------------ ------------------.-----.------XV XVI XVII XVIII Inward connection, sets XIV Partition (a,b}s (1,2,3,4,6,8,9) (a) S {2,3,4,9} {b)s: (2,4,6,8) {c} S (2,7,8) ----------------------------------------------.---------------------_.-----------XIX XX XXI XXII XXIII Assignment Outward connection, sets XIX Partition (a)=(2) {b,c)s: (6,7,8) (a)={2) {b)S (6,8) {c)S (7,8) Figure 3a. Correspondences Formed for the Graphs of Figure 1. .... 1"\11t·txT~ .... A .Aano................ C) " " ..... " L U ..... "'5L'I::;'I::; " ;..... ~.l.l 4-h~ "H'C ....... ~_n \.{U'CJ..y ~+_ •• ~+.,_~ i:jl..l U~"U.l'C, must correspond to the set of all nodes having at least outward degree 2 in the dictionary structure. The only node of outward degree 2 in Fig. 1 (b) is ® ; there are four nodes in Fig. 1 (a) that have outward degree 2 or greater; ® must therefore correspond to either nodes ®, 0, @) or @. At this point it is necessary to generate smaller sets from the ones shown on lines I to IV of Fig. 3 (a). This is done by noting, for example, that the set of query nodes contained in both sets I and III of Fig. 3 (a) can correspond only to dictionary nodes which are also contained in sets I and III (plus possibly in other sets). The on:Iy query node contained in both sets I and III is @ ; in the dictionary structure, nodes ®, @), ®, and ® are both in sets I and III, so that set {b} must be contained in sets {2, 4, 6, 8}. The set of possible correspondents of node @ has then been reduced from the seven nodes of set I to the four nodes of set VI. This "set partitioning" procedure is performed by the computer by comparing the columns of the binary matrices exhibited in 590 PROCEEDlNGS-SPRING JOINT COMPUTER CONFERENCE, 1964 --------.------------------------------------------ NODES SETS I a b c 1 2 3 4 5 6 7 8 9 I 0 1 1 i i 1 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 i 0 0 0 1 1 1 1 1 1 I III III IV 1 1 0 0 0 1 0 0 0 0 0 0 0 J 1 0 0 Figure 4. Matrix Representation of Sets I-IV of Figure 3a. Fig. 4. Node@, for example, has column vector 1010; the only nodes of the dictionary structure including the pattern 1010 are nodes ®, 0, ® and ® with vectors 1111, 1110, 1010, 1011 res pecti vely . The partitioning process yields three new sets labelled V, VI and VII. New sets (VIn to XI) are added using outward and inward connections of length 1 from the set;; V, VI, and VII. (The outward connection of the set {c} is empty, and therefore it is not included in the table.) The ,artitioning process is repeated, yielding sets XII, XIII, and XIV. Since the possible - correspondents of node ® have changed, it is not redundant to test the connectivity again. When this is done and another partition performed, sets XVI, XVII, and XVIII result. These pairs of sets are identical to those produced by the previous partitioning, and no simple properties of the nodes can be used at this point to generate new sets which would in turn result in a reduced partition. II An "assignment" is therefore made by postulating the correspondent for node ® (set XIX). This assignment represents a guess which must later be verified for correctness. Partitioning of the sets XVI to XX yields the sets XXI to XXIII. New assignments (not shown in Fig. 3 (a» of @ first to ® and then to ® then produce two one-to-one correspondences between the graphs of Fig. 1: ®~®I @ ~ 0~ ® ~ and G)J {®~® (~~ ® ~ ~G) If it is desired to obtain other possible corII The problem of choosing a "good" property set to be used for the generation of set correspondences is, in general, unsolved. For the example at hand, additional properties mi)?,'ht, however, have been used. For instance, second order inward degrees yield the correspondence c 2, 6, 7 after the first partition. respondences, it is now necessary to go back to the sets XVI to XVIII and attempt other assignments for node @. Sets XIX' to XXIII' of Fig. 3 (b) illustrate the assignment ® ~ 0. This assignment yields a partition which is seen to be improper since node © cannot be included if! the empty set. The assignment a .~ ® is therefore not useful since it leads to an incompatibility. The other two possible assignments for node@ , do, however, furnish acceptable one-to-one correspondences as follows: @~0} f®~® ®~ ® andl@~0 0~® 0~®. The four mappings obtained are easily verified by comparing Figs. 1 (a) and 1 (b). A flowchart of the complete procedure is shown in Fig. 5. The procedure is seen to be --------------, I I I I I ' - - -_ _--.._ _ _ _....J - - - - - - - - - - - - - : I I I I I I I I I I I I I ..J Yes NO MATCH EXISTS BETWEEN THE STRUCTURES - - - - CARDINALITY VIOLATION Figure 5. Simplified Diagram of Graph Matching Procedure. iterative since the generation of corresponding sets is followed by a partitioning process, followed again by the generation of new sets, and so on. Alternate applications of partitioning followed by formation of new sets will result in one of three situations: 1. the membership of each set is reduced to one, thus exhibiting the complete match between the given structures; 2. an incompatibility arises between pairs of corresponding sets, that is, a cardinality SOME FLEXIBLE INFORMATION RETRIEVAL SYSTEMS 591 violation is found to exist between pairs of corresponding sets; in that case no match exists in general; 3. no incompatibility arises but repeated application of the partitioning procedure will not result in the formation of new sets; in that case, more than one match is generally possible, and it is necessary to perform an arbitrary assignment of correspondents for one of the nodes. If a cardinality violation is detected, that is, if, for example, a set A is found to be included in a set B which has fewer members, as happened in the example for sets XXIII', then the two structures being compared obviously cannot match. The comparison process can therefore be stopped immediately, unless the incompatibility resulted from a previous assignment; in the latter case, only that particular assignment can be discarded, and other possible assignments must be tried before deciding that the two structures do or do not match. The procedure to be followed in case of cardinality violation is shown by broken lines in Fig. 5. In practice, cardinality violations normally arise early for graphs which do not match, so that the procedure is very rapid in such cases.# THE ADAPTIVE MATCHING PROCESS In the example described in the preceding section, four isomorphisms were detected between the graph of Fig. 1 (b) and that of Fig. 1 (a). Clearly, it is possible to increase or decrease the number of matches (or, alternatively, to increase or decrease the probability of a match between any two given structures) by suitably relaxing or tightening the conditions which affect the matching process. If, for example, the unilateral connections (directed branches) in Fig. 1 are replaced by bilateral connections, and therefore the non-sym,metric connection matrices of Fig. 2 are changed into the symmetric ones of Fig. 6, then eight additional isomorphisms will be found between the two graphs. In fact, three different isomorphisms will then exist between the "triangle" {a, b, c} and each of the triangles {2, 6, 7}, :# Non-matching graphs of fifty nodes required an average of less than one-half millisecond on the 7090 computer during a test run. t 2 3 4 5 6 789 t I I 1 I 1 2 t I 3 4 5 6 7 8 9 0) a b t t t I t t 1 1 1 a t b I 1 1 c t 1 t t t cJ I I! lJ J 1 t 1 t DieT IONARY STRUCTURE b) QUERY STRUCTURE Figure 6. Symmetric Connection Matrices Derived from Graphs of Figure 1. {2, 8, 7}, {4, 2, 8} and {9, 4, 8} included in the dictionary structure. Another possible way of relaxing the conditions which are operative during the matching process is to permit the introduction between any two nodes in the query structure of a variable number of intermediate nodes. This process replaces the query structure of Fig. 1 ( b) by the new structures of Figs. 7 (b) and ? ( d). (The broken lines indicate indirect eonnections.) Since each of the intermediate nodes mayor may not match a given node in the dictionary structure, it is now necessary to test whether the query structure is a compieted partial subgraph (rather than a subgraph) of the dictionary structure. A comparison of Fig. 7 (b) with the partial subgraph of Fig. 1 (a) represented as Fig. 7 (a), and a comparison of Fig. 7 (d) with Fig. 7 (c) reveals at least two additional completed partial subgraphmatches that could be obtained in addition to the four subgraph matches already exhibited in the preceding section. Further completed partial subgraph matches not shown in Fig. 7 are also possible. 0) PARTIAL SUBGRAPH OF DICTIONARY STRUCTURE OF FIGURE 1 b) MODIFIED QUERY STRUCTURE ~ ~7 PARTIAL SUBGRAPH ~ DICTIONARY STRUCTURE OF FIGURE 1 c) d) MODIFIED QUERY STRUCTURE Figure 7. Matching Partial Subgraphs. 592 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 I I 2 :3 4 5 6 7 8 9 a) 2 :3 4 5 6 7 8 9 I 1 1 1 1 I I I I t 1 1 1 1 I 1 1 I abc a b c I I 1 1 I 1 1 t I DICTIONARY STRUCTURE b) QUERY STRUCTURE Figure 8. Path Matrices (Including Indirect Connections) for Graphs of Figure 1. To determine whether a gral'h is a completed partial" subgraph of another graph, it is no longer sufficient to know whetRer two nodes are directly connected or not, but it is also necessary to know whether a (possibly indirect) path exists between any pair of nodes. Thus the connection matrices of Fig. 1 must be replaced by the "path matrices" shown in Fig. 8 in which the i_jth element is 1 whenever a path exists from node i and to node j. * * Each 1 in the matrices of Fig. 8 thus indicates either a direct or an indirect connection between the corresponding nodes, and use of the algorithm of Section 2 with the path matrices of Fig. 8 (instead of the connection matrices of Fig. 2) will generate the isomorphisms derived in Figs. 3 as well as a number of additional completed partial subgraph matches including those exhibited in Fig. 7. Consider now, on the other hand, methods which will tighten the requirements to be met for a satisfactory match. Instead of specifying less information than for the directed subgraph comparison, it is now necessary to add restrictions to the graph of Fig. 1. A possible method consists in adding labels to the unlabelled branches of the graph to simulate, for example, various types of relations between the nodes. Another possibility is the addition of labels to the nodes of the graph so as to restrict the correspondents of a given labelled query node to only those nodes in the dictionary structure which carry the same label. Consider first the two graphs shown in Fig. ** The path matrix may be generated automatically as a sum of powers of the corresponding connection matrix.l 3 a ) DICTIONARY STRUCTURE b) QUERY STRUCTURE Figure 9. Abstract Graphs Including Branch Labels. 9. Clearly these graphs are identical with those shown in Fig. 1, except for the added branch labels which distinguish three types of relations denoted respectively by the digits 1, 2, and 3. The binary connection matrix of Fig. 2 may now be replaced by a numeric branch label matrix as shown in Fig. 10, whose i_jtll element I 2 :3 4 5 6 7 8 9 2 2 :3 :3 2 I :3 :3 4 2 :3 5 6 2 t 7 8 9 abc a b c :3 :3 2 2 :3 1 :3 Figure 10. Branch Label Matrix for Graphs of Figure 9. is n if there exists a branch of type n from node i to node j, and is 0 otherwise. In the set generation and partitioning procedure, it is then possible to keep with each node a list of branch labels of all outgoing (or incoming) branches, Set Number Criterion for Set Generation Corresponding Sets I Outward degree 2 (or greater) and outward branch label (3,3) (a} S(2,9} II Outward degree 1 (or greater) and branch label (2) (b) S (l,2,4,6,8) III Inward degree 2 and branch label (2,3) (c) S(2,7,8] IV Connections from set I includ- (b,c] S(4,7,8) ing branch label (3) V Connections from set II and branch label (2) (c) S (2,6,8,7} VI Connections into set III and branch label (2) (b) S (1,6,8,4} VII Connections into set III and branch label (3) fa} S{3,4,2,9} Figure lla. Set Correspondences for Graphs of Figure 9. SOME FLEXIBLE INFORMATION RETRIEVAL SYSTEMS I II II V V VI 'II 593 9 a b e 1 2 3 4 5 6 7 8 1 0 0 0 0 0 1 0 1 0 1 0 1 0 0 0 1 1 1 0 0 0 1 0 0 0 1 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 1 0 1 0 1 1 0 0 0 1 0 0 0 0 0 1 0 1 0 0 0 0 1 1 1 0 0 1 0 0 0 0 1 0 0 1 (a}£{2,9) (b}£{4,8) (e) £{7,8} ~o ®IERF OPTiOUE ®~1 CORTEX 2 ~@ CHRONAXIE Figure llb. Set Inclusion Matrix for Sets of Figure lla. and, obviously, given a pair of corresponding sets not only must the nodes match as before, but the branch labels must match as well. The matching procedure is illustrated in Fig. 11. The branch labels make it possible to generate a large number of sets at the outset. The set partitioning procedure illustrated by the set inclusion matrix of Fig. 11 (b) then results in the formation of the three small sets reproduced in the figure. Assignment of @ to either node ® or node ® finally produces two one-to-one mappings as follows: ®~®} {@~® ®~@ and @~® ©~® @~(f) A comparison of the graphs of Fig. 9 can be used to verify that these two mappings are the only ones which obey the branch labelling restrictions. As a last extension, consider now the two Syntol graphs2 of Fig. 12. These graphs correspond to an actual document abstract and to a search request, respectively, as encoded under the Syntol system, and may be seen to be identical with the structures of Fig. 9 except for the added node labels. In order fully to represent the system, it is no;v necessary to add node label matrices to the connection matrices and Figure 12b. Typical Syntol Query Graph. to the branch label matrices. The node label matrices may be represented either as a table including all the node names together with the (possibly vacuous) corresponding labels, or alternatively as a full matrix whose i_jtlt element is 1, whenever label j is attached to node i. The node labels serve the same purpose as the branch labels, since they restrict the number of possible correspondents of a given node to only those nodes which either carry the same label, or else carry no label, thus indicating that they can match any node whatsoever that satis;.. fies the remaining restrictions .. The procedure used to determine whether the query graph of Fig. 12 is a subgraph of the document graph is outlined in Fig. 13. Since all nodes are labelled, an immediate correspondence is established between the nodes of the two graphs under consideration (sets I, II and III of Fig. 13). It remains to determine whether the connections and branch labels are preserved. Sets IV of Fig. 13 reveal an incompatibility, since node ® has two outgoing branches with a branch lab~l (3), whereas the corresponding node ® has only one such outgoing branch. Since a set containing two elements cannot be contained in a set containing only one element, Set Number I II III IV Figure 12a. Typical Syntol Document Graph. Criterion for Set Generation Corresponding Sets (a) S{3) Node label "cortex" {b}S{8) Node label ''nerf optique" {c)S(7) Node label "chronaxie" Direct connections from set I {b,c)S{2} with branch label (3) Figure 13. Set Correspondences for Graphs of Figure 11 Using Direct Connections. 594 PROCEEDINGS--SPRING JOINT COMPUTER CONFERENCE, 1964 Set I !INumber I Criterion for Set Generation I II III IV I I I Corresponding Sets I Node label "cortex" {a} ~ (3) Node label "nerf optique" (b) ~ {8} Node label "chronaxie" (c) ~ {7} 'Paths from I with branch label {b,c} ~ {2,7,8} (3) V Paths from II with branch label VI Paths into III with branch label VII Paths into III with branch label VIII Paths into II with branch label (c) ~ {7} (2) {b} ~ (l,2,4,6,8) (2) (a) ~ [2,3,4,9} (a) ~ {2,3,4,9} (3) (3) a variety of possible relations between terms to a single one (represented by an unlabelled branch). Replacement of directed by non-directed branches further reduces the ability to discriminate between a variety of relations, since a relation from A to B is now equivalent to one from B to A. Finally, removal of node labels simplifies both the search requests and the document identifications, since it eliminates from consideration some of the terms used as identifiers. A possible strategy for the gradual broadening of matching criteria is as follows: Figure 14. Set Correspondences for Graphs of Figure 11 Using Indirect Connections. the subgraph test fails, and there is no need to proceed further. It is therefore necessary to relax the matching conditions by taking into account indirect connections and intermediate nodes. The path matrix of Fig 8 is now used to verify that complete paths (rather than direct connections) and path labels are preserved by the correspondence in the node labels. The set correspondences, shown in Fig. 14, demonstrate that to each outgoing and incoming labelled path in the query structure there corresponds a path with similar properties in the document structure. The query graph therefore matches the document graph when indirect connections are taken into account. An adjustable procedure for the comparison of query and dictionary structures can now be outlined. The process uses the same matching algorithm throughout, and is modified only by altering the matrices which represent the connection patterns and the branch or node labels. The exact strategy used in the progressive alteration of the matrices may be made to depend on the type of document collection being processed, and on preliminary retrieval tests. Clearly, the weaker the restrictions which affect the matching process, the more matches are likely to be obtained, and the larger therefore the collection of answers to a given search request. In general, elimination of the branch labels from the query and document graphs reduces a. Use unmodified query structure Q and dictionary structure D and test whether Q is a subgraph of D; b. If the preceding test is negative use path matrix including indirect connections to determine whether Q is a completed partial graph of D; c. If the preceding test is again negative, selectively remove branch labels by altering branch label matrix and test again using first only direct connections (subgraph test), then indirect connections; d. If matching conditions must be further relaxed, replace unilateral by bilateral connections and use symmetric connection matrices first with direct and then with indirect connections; e. Finally, selectively remove node labels and test again for subgraph and then for incomplete partial graph. A retrieval system using graph matching procedures in conjunction with natural language data is outlined in the next section. A SENTENCE MATCHING PROCEDURE FOR DOCUMENT RETRIEVAL A simplified automatic document retrieval system is shown in Fig. 15. 5•14 This system makes use of the standard statistical procedures, including the computation of word frequency counts, word associations based on cooccurrence in the same sentences or texts, document associations based on .co-occurrence of words, and document relevance coefficients.l 5 ,16 In addition, a dictionary or thesaurus may be used if available to normalize the vocabulary. SOME FLEXIBLE INFORMATION RETRIEVAL SYSTEMS INCOMI NG TEXT OR SEARCH REQUEST word can of course function as a semantic label by itself. A syntactic analysis is then performed which determines a dependency structure for the words of a sentence, and also generates a syntactic label for each word. A typical dependency tree, resulting from an automatic syntactic analysis, is shown in Fig. 16. A syntactically analyzed sentence can of course be represented as before by direct and indirect connection matrices, as well as syntactic and semantic label matrices; moreover, these matrices can be generated automatically from the output furnished by the syntactic analysis program. 17 DICTIONARY LOOK-UP TO OBTAIN SYNTACTIC AND SEMANTIC LABELS r----------- ------------1 EXPANSION OF SEMANTIC LABELS I I L__TH~~~3~~R~~ ~~~E!~~~R!3~H2:J r------------ I ------------, L__'=-NE !~~~':.!~C_SEN~E~':~ ~X2~~2:.I.::~J r----------- - I ________ ---, IN I COMPUTATION OF SENTENCE SIGNIFICANCE I STRUCTURAL 1 r.WCHING --I I I I SYNTACTIC ANALYSIS OF SIGNIFICANT SENTENCES AND STRUCTURAL MATCHING WITH CRITER ION PHRASES I EXPANSION OF SEMANTIC LABELS I I I ~=~~~=~~=~=~J[~~~~--~==~~~ LT~~O~~~ 5!~12S!~C~L TE~M_~~R~~A~I5:~J COMPARISON OF SEARCH REQUEST WITH DOCUMENT IDENTIFICATIONS AND POSSIBLE DOCUMENT CORRELATIONS It is now possible to compare the set of analyzed sentences or search requests with a set of "criterion phrases" included in a phrase dictionary. Each criterion phrase is representative of one or more subject categories, and if a match is obtained between a criterion phrase and an analyzed sentence or search request, the corresponding subject categories can be attached to the matching sentences or requests. To retrieve a set of documents in answer to a given search request, it is then sufficient to compare the subject categories attached to the requests, with the subject identifiers attached to the documents as outlined in Fig. 15. / / OPTIONAL STEPS / COMPULSORY STEPS SIMPLIFIED SYSTEM USING STRUCTURAL MATCHING FIGURE 15 Figure 15. Simplified System Using Structural Matching. The quantitative procedures may be supplemented by choosing a set of significant sentences, as determined by the statistical process, and using them to perform a structural analysis. Specifically, each word is furnished with one or more thesaurus category numbers (the semantic labels) as a result of the dictionary look-up procedure. If no dictionary is available, each 0 ®/ I'I I I 'I '(IDa I I I I I R I• /~@) I J I SYNTACTIC} LABELS ~ I i iI I 8 A typical criterion phrase is shown in Fig. 17. Each criterion phrase is represented, as before, by an identification number and control _ 0 ~/J' \!:Io 595 I I I I I :@ I I I I I ...... 0 ...... 0 @l @l @: S V C /~0 @o, I I I I I I I I I I I I A S V A HE TEXT I I iI I I I I I 0 I SEMANTIC} BECAUSE LABELS CONTAINS SECRET INFORMATION RETRIEVAL IS Figure 16. Typical Syntactic Dependency Tree. VITAL 596 PROCEEDINGS-SPRING JOINT· COMPUTER CONFERENCE, 1964 ® ®o ®/l o ; SYNTACTIC} (A:O) LABELS i SEMANTIC} (0;4, LABELS 023) I : (S~C) ' :' (0;7) (i)/~® THESAURUS CATEGORY 014: (I.N.FORMATION, \ DOCUMENT (5),\ FACT (5), ) DATUM, DATA ETC. I ' II I SYNTACTIC} LABELS I The matching process between a given criterion phrase and a sample sentence or search request is identical with that used in the preceding section for document graphs. That is, two principal criteria must be satisfied: 1. Given a specified node of the criterion phrase, all those sentence nodes are selected which have matching syntactic and semantic labels; 2. From among those sentence nodes which obey the restriction of part 1, some subset must be chosen whose direct (or indirect) connection pattern is identical with the connection pattern of the corresponding nodes in the criterion phrase. Consider, as an example, the criterion phrase of Fig. 17 and the sentence of Fig. 16. Clearly, both syntactic and semantic labels of nodes CD and @, and .of nodes ® and ® will match properly. However, there exists a path from node a to node ® in the criterion p·hrase, while no such path exists from node ® to node 0. Therefore, the structure matching procedure will not be successful for the given example. On the other hand, it can be easily verified that the trees of Fig. 18 will, in fact, prope-rly match the criterion phrase of Fig. 17. Several methods are provided in the system for adjusting the matching process. First, the matching algorithm itself is adaptable, since node and branch labels can or cannot be taken into account, and direct as well as indirect con- , I I , I I I I I A S PR 0 , I I I I I I I I I I I I SEMANTIC} THE RETRIEVAL OF INFORMATION LABELS (a) Figure 17. Typical Criterion Phrase. information, the di]'ect and indirect connection matrices, the syntactic and semantic node label matrices, and the category indicators which identify the subject classes for the given phrase. The semantic node labels attached to the sample criterion phrase are decoded in Fig. 17. I ",@ I 0 I I THESAURUS CATEGORY 017 (RETRIEVAL, ) PROCESSING, ORGANIZATION, SEARCH, ETC. ,"'-.,. :, SYNTACTIC} S LABELS V SEMANTIC} HE LABELS + A DISCUSSES INFORMATION A 0 AND DOCUMENT RETRIEVAL (b) TREE STRUCTURES WHICH MATCH THE CRITERION PHRASE OF FIGURE 17 FIGURE 18 ® (0 I I , I I I I I I I I I I I I I I I I SYNTACTIC} ~ LABELS , I SEMANTIC } LABELS T~E I S PR I I I I I PROCESSING OF , A I I I DOCUMENT CONTENT (e) ® ®/i! /Y 0)/! I 9I SYNTACTI C} U~~ SEMANTIC } LABELS I : I I I I I I A S ~ DOCU~ENT I I FILE ORGANI:iATION (d) TREE STRUCTURES WHICH MATCH THE CRITERION PHRASE OF FIGURE 17 FIGURE 18 Figure 18. Tree Structures Which Match the Criterion Phrase of Figure 17. nections can be used. Second, it is possible to provide the criterion phrases with a smaller or larger number of syntactic and semantic labels, SOME FLEXIBLE INFORMATION RETRIEVAL SYSTEMS thus restricting or enlarging the possible sentence nodes which are compatible. Finally, the thesaurus which can be used to replace text words by thesaurus categories, as well as the criterion phase dictionary can be enriched to ensure inclusion of a larger variety of possible sentence structures. The system is presently being tested in order to determine the practical effectiveness of these various measures. REFERENCES 1. M. DETANT, Y. LECERF, and A. LEROY, "Travaux Pratiques sur l'Etablissement des Diagrammes," Enseignemept Preparatoire aux- Techniques de la Documentation Automatique, Euratom Report, Ispra (February 1960). • 2. J. C. Gardin and F. LEVY, "Le SyntolSyntagmatic Organization Language," Proceedings of the IFIP Congress-62, Munich (August 1962). 3. R. BARNES, "Language Problems Posed by Heavily Structured Data," Communications of the ACM, Vol. 5, No.1 (January 1962) . 4. G. SALTON; "The Manipulation of Trees in Information Retrieval," Communications of the ACM, Vol. 5, No.2 (February 1962). 5. G. SALTON, "Some Hierarchical Models for Automatic Document Retrieval," American Documentation, Vol. 14, No.3 (July 1963). 6. E. MARDEN and H. R. KOLLER, "A Survey of Computer Programs for Chemical Information Searching," National Bureau of Standa'rds, Report No. 6865 (May 1960). 7. H. SHERMAN,."A Quasi-topological Method for'the Recognition of Line Patterns," Information Processing, Proceedings of the International Conference on Information Processing, Paris (June 1959), pp. 232238. 597 8. R. L. GRIMSDALE, F. H. SUMMER, C. J. TUNIS, and T. KILBURN, "A System for the Automatic Recognition of Patterns," Proc. lEE, Vol. 106 (1959), pp. 211-221. 9. S. H. UNGER, "GIT-A Heuristic Program for Testing Pairs of Directed Line Graphs for Isomorphism," ACJl.-I National Confe1'ence, Denver (August 1963). 10. E. H. SUSSENGTH, JR., Doctoral Thesis, Harvard University (in preparation). 11. E. H. SUSSENGUTH, JR., "A Graph Theoretic Algorithm for Matching Chemical Structures," Journal of Chemical Documentation (to ~ppear). 12. G. SALTON and E. H. SUSSENGUTH, JR., "Automatic Structure - Matching Procedures and Some Typical Retrieval Applications," Report No. ISR-4 from the Harvard Computation Laboratory to the Air Force Cambridge Research Laboratories (August 1963) . 13. F. E. HOHN, S. SESHU, and D. D. AUFENKAMP, "The Theory of Nets," IRE Transactions on Electronic Computers, Vol. EC6, No.3 (September 1957). n ".4. ___ !1....1_ A. ..... "- _ _ .... +.:". Q .... 14. U. i::)ALTUN, i\..[' It:XIUlt: .l'1.U\'Ul11aL.l~ ..::J~ ...,"'CUI for the Organization, Storage and Retrieval of Language Data (SMART)," Report ISR-5 to the National Science Foundation, Harvard Computation Laboratory (January 1964). 15. v. E. GIULIANO, "Automatic Message Retrieval by Associative Techniques," Fi1'st Congress on the Information System Sciences, Hot Springs (November 1962). 16. H. E. STILES, "The Association Factor in Information Retrieval," Journal of the Association for Computing Machinery, Vol. 8, No.2 (April 1961) . 17. A. LEMMON, "The Criterion Routine," Report No. ISR-5 to the National Science Foundation, Harvard Computation Laboratory (January 1964). .r""'t & ........ _...... ~1 .,.n+n.~ SOME IMPROVEMENTS IN THE TECHNOLOGY OF STRING MERGING AND INTERNAL SORTING Martin A. Goetz Applied Data Research, Inc. Princeton, N. J. GENERAL strings the same size thus resulting in an "effective" power of the merge equal to the way of the merge. * The read-forward Oscillating Technique described in this paper is directed toward this goal. Sort/merge programs for magnetic tape computer systems are of two basic classes: 1 Digital (or Radix) and Collation The Von-Neuman (T/2),1 Polyphase,2,3 and Cascade 4 merge techniques begin after all the data is processed by the internal sort subprogram and the initial strings are placed on tape (Figure 1). The Von-Neuman Technique places the initial strings on only half the available tapes; the Polyphase and Cascade place data on all the unused tapes. Depending on the number of tape units, one of the merging techniques will prove superior over the others. The read-backward Oscillating Technique5 writes several initial strings, transfers to the merge sub-program, merges the strings and returns to the internal sort-hence the name Oscillating (Figure 2). As the number of tape units used for sorting increase, the effectiveness of the Oscillating Sort increases. Given T input tapes, the effective power of this technique is T-2. This technique, previously considered only for r8ad-backward tape systems, is developed for read-forward only tape systems as described in the following pages. The digital sort is useful number of cases and is not examined in this paper. The collation sort is the more general type of sort and is composed of two basic sub-programs: 1. The first sub-program internally sorts a group of data. The group of data after being sorted is referred to as a "string" or "initial string." Such a sub-program is referred to as an internal sort. 2. The second sub-program merges two or more strings. It will produce as output longer strings and will eventually produce one string which contains all the data. Such a sub-program is referred to as a "merge," or "string merge." The input data enters the internal sort subprogram only once and the merge sub-program one or more times. The various sortingsystems in use today all try to minimize the'execution time of the merge sub-program. This is accomplished by maximizing the' way of the merge while at the same time keeping input ~: For a d~st~n~tion b:tween "way of the m:rg"," and "effective power of the merge" please see glossary of sorti merge terms. 599 600 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 ExUIP1e - 6 tapee 16 etr1nqa '1'2 '1'1 VOH-1IBtJMUI - ..--to......... - I"e.d baoJarard. '1' '1'4 '1'3 '1'1 ~~ ~~ ~I .u. l' I~ l~ .u. ' .IJ. 7 -U,u.. 8 ,i). .IJ.. 11 10 .ij, ,u.. '1'2 ,jJ. 5 n 2 6 .u.. ,u. 12 ,u. 1 9 16 0 '1'. 5 '1' 4 '1'3 '1'6 5 ~!~It::lil: I -]I ~[ 4 lIS '1'6 • :Ut~· to, 0 ," t~,. JJ..t O DIlI!IIUft1111 10 CASCAD\! - r.4-:to:rwar4 -..--- IL_ IL_ i58 ·EB .u. .u. ~E§ ~m .u. .u. 16 6 .u. ..u. 8 11 9 .ij. 12 U 10 ,jj. 13 110 10 ..u. 1_ 15 11 11 15 16 .. .u. ---- ..--tor00z4 12 l' 110 _,111 POLYPHASE " ,u. 5 15 16 -11-b8 .,u. .u. ~ffi ..u. 10 6 11 .u. ~~ ~m .u. .u. .JJ. 7 8 n 12 l' -II- ,IJ.. 1~ !~ _-_1 LlIGBIIID _ . .u..et1«l ....._ _ _ .u..et1«l. (ooutb-torazod; ... IIIM'IIIIIImIll _-_I 1I1D&l................t_ t_.-....,.. (~) DoUl>l.............._ _ _ o LlIGEIIID ........ d:1NO't1GD. 1:aIH.. . . t~ d1ncrt1oa~ (_~.to_ .. 1I1D&l' ....- 1DU_ _ _ - . . . (~) Do@lo""",,1DU"'t _ _ writ..,.. C.u.) Figure 1. Distribution of Strings During Internal sort for Von Neuman (N / 2), Cascade and Polyphase Merge. READ-FORWARD OSCILLATING MERGE The read-forward Oscillating Merge allows the use of a preceding internal sort technique which produces variable size string lengths (e.g., Replacement-Selection/i). It also allows, of course, techniques which produce fixed-size strings (e.g., successive merging 1 ) • This paper presents an example in which the internal sort is a Replacement-Selection. The Replacement-Selection Technique used as an internal sort produces an initial string length almost twice the size of the memory available for sorting. Since the Oscillating Merge Technique must merge initial strings formed by this technique, it is important that we review the nature of the output produced by the Replacement-Selection Technique. Given a memory for building strings that can hold R records, the expected string length for random data for the first string7 is l.73R; for successive strings, 2.0R; without replace- Vl'1tu... (~) 1DU _ _ oeq_ or - . . _ writ..,. ... t_. Figure 2. Tape Motion of Read-Backward Oscillating Merge. mente l.OOR. Given T tape units, the string on the first tape is estimated to be l.73R; for the last tape unit. l.OOR. for all other tape units. 2.00R. This variability in string length reduces the advantages of the read-backward Oscillating Technique. 8 As will be demonstrated, this variability also reduces the effectiveness of the read-forward Oscillating Technique. The read-forward Oscillating Technique can also be used with an internal sort that produces initial strings of a fixed size. In this case, the initial string length is l.OR for all tape units. THE TECHNIQUE It is more convenient to describe the technique when the initial string lengths produced by the internal sort are fixed in length (l.OR), It will then be shown how the read-forward Oscillating Merge operates when used with the Replacement-Selection Technique in which variable length strings are produced. Given N tape units (T 1 to TN) available as work tapes. (In the example· shown in Figure 3, N is equal to 5.) SOME IMPROVEMENTS IN THE TECHNOLOGY OF STRING MERGING AND INTERNAL SORTING IlIPtl'1' S'l'Alt'l' 01" 'rAPZ 110 REWIND TIll! LEGEND Y Arrow point _bon poa1t1on ot read/wr1te head. OIl. tape. S1r>gl. arrow 1Dd1cat.. tap. 1'Qd1ng. l~) arrow 1Dd1oat_ tape wr1t1Dg. (,u) Double Dotted. arrow 1nd1cat.. tape re'W1Ddlngo(1) Figure 3D JIO RlllfIIIlD TIME 3 4 lig]K! !., 1Ir--l Jl. II ~ 21 ~L-....J; ~I 11r---l ~ 17 18 13 18 19 20 9 10 11 12 14 15 16 11r---J 19 ~ L-.J n 5 6 7 8 Figure 31" 1:: t ~~:. 17 I I !!1 210 I I 9 16: 12 I ; j" : I I I ; 1 ··! ! '" 1 ~~~ ~::~ ~~~ ~: ~~: ~: ~: • 13: • I +II 5 I 8 I I I : : , 4I I I 1 I "I I : i r---t 20 1. Initial strings are written on all tape units (T 1 to Tx). After each string is written on successive tape units, the tape is rewound. The last tape unit is not rewound (Figure 3A). 2. Tape units Tl to T X- 1 are merged onto tape units T x, at which time all tape units are rewound (Figure 3B). 3. As soon as tape unit Tl is rewound (this tape unit contains only one string of data), a string is written on tape unit T 1 and then it is rewound. Strings are consecutively written on tape units T2 to T x - t • All tape units except TX-l are immediately rewound (Figure 3C). TX-l is not rewound. 4. A merge of Tl to T X - 2 and Tx is performed onto T X - 1 ' Then all tapes are rewound (Figure 3D). 5. The cyclings shown above continue for N cycles at which time all tape units contain a string length of size (N -1) times the initial string length (Figure 3E). 6. The (N -1) size strings are merged, the power being (N -1) as shown in Figure 3F. 7. The pattern shown continues until the data is exhausted, at which time one string will be formed, similar to the readbackward Oscillating Technique. Note that after 16 strings were merged, the rewind interlock time was equal to rewind time for N -1 strings. This time is a relatively small portion of the rewind time. Note that rewind time is minimized by starting the development of the next string in a group, although it will not be merged until the next cycle (String 21 and Strings 17-20). • : : I 7 U .. 12 13 lS Figure 3. Tape Motion of Read-Forward II 601 Os~i1lating Merge. When using the Replacement-Selection Technique in conjunction with the read-forward Oscillating Technique, initial strings are constrained not to exceed twice the number of records (R) in memory. After 2.0R records are written on tape, the string is completed. Note that with other merge techniques the minimum string length might be LOR as a lower limit· and "all" records to be sorted (an entire file) as the upper limit. When using the read:'forward Oscillating Technique, the upper limit is set at 2.0R. When 2.0R records can not be formed, dummy records (and blocks) 602 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 are substituted to form 2.0R records. Since 2.0R is the expected length, excessive dummys will not be formed. 2.0R may not be an optimum figure. It will, however, ciosely approximate the optimum as the number of tape units increase. To the degree that the read-forward selection technique must generate dummy records to produce string lengths of 2.0R, it is less efficient than the read-backward technique. As stated above, however, this reduction in efficiency should be slight under most circumstances. THE PROBLEM OF WRITING ON TAPE USING THE READ-FORWARD OSCILLATINGMERGE The read-forward Oscillating Merge herein proposed requires that data be written on the front of a tape without destroying information further down the tape which will be subsequently read. Depending on the computer systenl and the tape units, this may cause problems. For some computer systems, the read head may not be positioned properly due to start-stop time variations, automatic bypassing of unwritable tape (bad spots), effects of the erase head or differences in writing density. For systems with a tape rewrite feature, there would be no problem. For systems which allow tape erase, gaps on the tape can be program generated which will solve the problem. If neither of these features are available "hash" blocks can be inserted to protect information which must be subsequently read. The technique may not be applicable to older systems where there is no programmed error control. Because of the wide variety of tape systems, this problem is not covered in more detail. It has been investigated, and it can be shown that the additional programming to cope with this problem is trivial. Polyphase or Cascade Merge Technique. The Read-Forward Oscillating Technique might be considered even in systems which allow backward reading. This is particularly the case in systems where tape reverse interlock is high. The foregoing presentation of the read-forward technique should not be construed as a recommendation of this technique to the exclusion of other methods. The selection of proper sorting techniques is a complex problem dealt with in detail in other papers ri, H, 9, 10 and which is not completely formalized at this time. VARIABLE LENGTH RECORD SORTING USING THE REPLACEMENT-SELECTION TECHNIQUE INTRODUCTION Another area where efficiencies' in sorting can be attained, is in the internal sort. Since the number of merging passes is based on the number of internal strings, it is desirable to minimize the number of strings formed by the internal sort. The number of strings are minimized when the amount of data sorted at one time (the length of the string) is maximized. The sorting techniques in use today limit the string length for variable size records to the number of records that can be stored in memory. This section describes a sorting technique that permits an initial string to be formed that contains approximately twice as much data as can fit into the working storage a vailable in memory. 7 GENERAL CONCLUSION The variable-record length internal sort segment uses a modified version of the Replacement-Selection Technique. 6 This technique has previously been applied to the sorting of fixedsize records or to a variable-size record converted into a fixed format. The technque described offers the same potential as the Tead':bac:kwArd Oscillating Technique, namely: as the number 'of itapes increase, this technique will perform the sorting task almost twice as efficiently as the N /2 (Von Neuman) and more efficiently than the The proposed technique temporarily "disjoins" a \7ariable-size record into one or more fixed-size "pieces" (re£erned 1il0> :M 'segments) and at selection time ,combines the separate segments of the record. No expansion of the record occurs. SOME IMPROVEMENTS IN THE TECHNOLOGY OF STRING MERGING AND INTERNAL SORTING The selection of an "optimum" fixed-size segment storage area is either determined by the user (based on his knowledge of the data) or assigned by the computer program. The optimum size is one which will produce the longest string on tape without causing average size records to be broken up into a large number of segments. Long strings are produced by selecting a fixed storage segment size into which the records will fit without the need for excessive "fill" when the last "segment" of the record is moved into the fixed segment area. The selected segment size should be such that all keys for each record appear in the first disjointed segment when subdivided. A brief review of the selection logic for con·· ventional fixed size Replacement-Selection internal sorting follows: 603 against the 4th, etc. In this manner a set of first round winners is selected. The addresses of the winners are stored. 3. In a similar way succeeding rounds of winners are selected until one final winner is selected. Addresses of the winners of each round are recorded. This process is called initialization of the tree and requires R-1 comparisons. (See Figure 5.) RBCORD 1 RBCORD 2 winner RBCORD 3 RBCORD 4 winner RECORD 5 winner RBCORD 6 winner RBCORD 7 Core memory is divided into 4 parts .. (1) Instructions (2) Input Areas (3) Output Areas (4) A string building area (work storage area) The string bUllQlng .area is further subdivided into pockets (or sl9tS) equal to a fixed size plus about 12 characters to hold information tlsed during the selection process. The selection process is similar to a tournament match in which the selected record at each level is referred to as a "winner." Assume as an example, the string building area can contain 12 records. The logic for selection is as follows: 1. A string building area is filled with records (R) from the input area. (See Figure 4.) 2. The first record in the string building area is compared against the second, the third RECORD 1 RECORD 2 I~ORD ! 4 5 7 8 9 10 11 12 6 w1nner RBCORD 8 RBCORD 9 w1nner RBCORD 10 winner RBCORD 11 winner RBCORD 12 -Figure 5. Logical Relationship of Records in Tournament. 4. The final winner is moved to the output and its position in the string building area is replaced with a new record from the input area. 5. If the new input record could be put out in sequence with respect to the last record that was put out, it can participate in the current tournament. Otherwise, it may not participate in the current tournament. 6. Each subsequent record selected requires Log~R comparisons. This selection process is referred to as a "scan." 7. When no records can participate, the current tournament is over and a new string must be formed. U~ING Figure 4. Example of Records in String Building Area. THE REPLACEMENT-SELECTION TECHNIQUE FOR VARIABLE LENGTH RECORDS A record in the input area is divided so that one or more segments of it may be moved into 604 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 slots in the string building area. The first such segment, containing the key, is called the header; all other segments are trailers. Each segment of a record, except the last, contains as part of its indicative information, the address of the segment of the record which is its immediate successor. When the string building area is completely filled it is "initialized." Only headers participate in this initialization and only headers are winners in each "round" of the tournament. Initialization is concluded when a final winner has been determined. At this point, the winner is moved to the output area. Immediately, its place in the string building area is filled from the input area. Whenever a "segment" is moved from the string-building area, that slot is immediately filled. This is true irrespective of whether a header or trailer segment was moved out (Figure 6). As segments are moved out they are rejoined in the output area into a variable length record. Any "fill" in the last segment is deleted at this time. After a string building area slot has been refilled, a "scan" takes place if one is necessary. It is necessary to "scan" if either a RECORD SEGMENTS IN STRIN3 BUILDING AREA INPUT 58 RECORD RECORD RECORD header was moved out of the string building area or if a header was moved into the string building area. In other words, the only time a scan is not performed is when a trailer is replaced by a trailer. After the scan, the next, "segment" is moved into the output area according to the following rules: 1. If the last segment moved to the output contained a reference to a successor (which must be a trailer), the successor is moved. This accomplishes the result of assembling the records which were segmented initially. 2. If the last segment moved was without a successor, the current final winner of the tournament (which must be a header), is moved to the output. CONCLUSION The logic for processing variable-length records using the Replacement-Selection Technique requires an additional address in the tournament tree which is used to chain between segments of a variable-length record. Additional logic is required to p~ocess variable length records. Records are segmented into header and trailers. After the selection process, they are recombined into a variable length record. The logic of the Replacement-Selection Technique requires modifications as described. As in the Replacement-Selection Technique for fixed size records, data is moved only twice. The overall logic of the Replacement-Selection Technique is retained and all its advantages are exploited. 6 6A ACKNOWLEDGEMENT The author wishes to express his appreciation to Mr. Warren F. Spalding of Applied Data Research, Inc. for his assistance in the organization and editing of this paper. GLOSSARY OF SORTING AND MERGING TERMS USED WITHIN THIS PAPER * 3 R3~or 1 Figure 6. Segments in String Eui.U:ng Ar:::a. Backward read A feature available on some magnetic tape system whereby the magnetic tape units can transfer data to computer storage while mov- SOME IMPROVEMENTS IN THE TECHNOLOGY OF STRING MERGING AND INTERNAL SORTING ing in a reverse direction. Normally used, if available, during the external sort phases to reduce rewind time. Balanced sorting See: (T/2)-way merging. Cascade merging A technique used in a sort program to merge strings of sequenced data. Given T work tapes, merging is performed at T-1 on part of the data, T-2 on parts of the data, and so on. Strings of sequenced data are distributed in a Fibonacci Series on the work tapes preceding each merge. The effective power of the merge varies between T -1 and '1'_2 but in all cases is less than the power of the Polyphase Merge. CF: effective power of the merge. Collating Sequencing a group of records by comparing the key of one record with another record until equality, greater than, or less than is determined. Collating sequence The sorting sequence; a description of the sort key for a file of records. Collating sorting A sort which uses a technique of continuous merging of data until one sequence is developed. Computer limited See: process limited. Digital sorting A sort which uses a technique similar to sortIng on tabulation machines (e.g., IBM Sorter). The elapsed time is directly proportional to the number of characters in the sequencing key and the volume of data. Also "radix sort." Effective power of the merge Equal to S~, where S is the number of input strings and N is the average number of times each element of data is read. Fibonacci series . A series where the current number is equal to the sum of the two preceding numbers: i.e., * Source: A Glossary of Sorting and Merging Terms -Communications of ACM, May 1963. 605 1, 2, 3, 5, 8, and so on. Some sort programs distribute strings of data onto work tapes so that the number of strings on successive tapes form a Fibonacci series. Fixed size records Denumerable file elements each of which has the same number of words, characters, bits, fields, etc. Cf: variable-length records. Generalized sort A sort program which will accept the introduction of parameters at run time and which does not generate a program. Generated sort A production program which was produced by a sort generator. Input tapers) Tape(s) containing a file in arbitrary sesequence to be introduced into a sort/merge program. Insertion method See: Sifting. Item See: Record. Key Also, sequencing key; criteria; sequencing criteria. The fields in a record which determine, or are used as a basis for determining, the sequence of records in a file. Magnetic tape sorting A sort program that utilizes magnetic tapes for auxiliary storage during a sort. Major key The most significant key in a record. Merge A program that performs merging. Merging The forming of a single file of sequenced records from two or more files of sequenced records. Multifile sorting The automatic sequencing of more than one file, based upon separate parameters for each file, without operator intervention. 606 PROCEEDINGS-SPRING JOINT COMPUTER CONFERENCE, 1964 Multipass sort A sort program which is designed to sort more data than can be contained within the internal memory of a central computer; Intermediate storage, such as disc, tape, drum, etc. is required. Optimum merging patterns The determination of the sequence in which specific sorted tapes in a file should be processed so as to minimize the total number of merge passes required to create a single file of sequenced records. Order of the merge The number of input files to a merge program. Also: power of the merge. Oscillating merge A technique used in a sort program to merge strings of sequenced data. For tape systems that permit backward reading, the effective power of the merge is equal to T -2. Output tape(s) Tapes containing a file in specified sequence as a result of a specific sort/merge process. Pass The processing of each file record once for the purpose of reducing the number of strings of sequenced records and increasing the number of sequenced records per string. Phase An arbitrary segmentation of a,sort program. Many sorts axe segmented into,=

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.3
Linearized                      : No
XMP Toolkit                     : Adobe XMP Core 4.2.1-c041 52.342996, 2008/05/07-21:37:19
Producer                        : Adobe Acrobat 9.0 Paper Capture Plug-in
Modify Date                     : 2008:11:17 01:52:49-08:00
Create Date                     : 2008:11:17 01:52:49-08:00
Metadata Date                   : 2008:11:17 01:52:49-08:00
Format                          : application/pdf
Document ID                     : uuid:6a7cbebe-aae0-48f2-b0a9-3da05d7fe8c0
Instance ID                     : uuid:0080f2a4-97ab-4f7f-bbac-1ad5cf312cab
Page Layout                     : SinglePage
Page Mode                       : UseOutlines
Page Count                      : 636