5P41_RR00785 13_SUMEX_Annual_Report Year_11_Jun84 5P41 RR00785 13 SUMEX Annual Report Year 11 Jun84

5P41_RR00785-13_SUMEX_Annual_Report-Year_11_Jun84 5P41_RR00785-13_SUMEX_Annual_Report-Year_11_Jun84

User Manual: 5P41_RR00785-13_SUMEX_Annual_Report-Year_11_Jun84

Open the PDF directly: View PDF .
Page Count: 233

Download
Open PDF In Browser	View PDF

SUMEX
SUMEX
STANFORD UNIVERSITY
UNIVERSITY
STANFORD
MEDICAL EXPERIMENTAL
EXPERIMENTAL COMPUTER
COMPUTER RESOURCE
RESOURCE
MEDICAL
RR - 00785
00785

ANNUAL REPORT
ANNUAL
REPORT - YEAR 11
Submitted
Submitted to
BIOTECHNOLOGY
BIOTECHNOLOGY RESOURCES
RESOURCES PROGRAM
PROGRAM
NATIONAL
NATIONAL INSTITUTES
INSTITUTES OF
OF HEALTH
HEALTH

June
June 1,1984
1,1984

STANFORD
STANFORD UNIVERSITY
UNIVERSITY SCHOOL
SCHOOL OF MEDICINE
MEDICINE
Edward
Edward A. Feigenbaum,
Feigenbaum, Principal Investigator
Investigator
Edward
Edward H. Shortliffe, Co-Principal
Co-Principal Investigator
Investigator

DEPARTMENT
OF
DEPARTMENT
OF HEALTH
HEALTH AND
AND HUMAN
HUMAN SERVICES
SERVICES
PUBLIC
HEALTH
SERVICE
PUBLIC HEALTH SERVICE
NATIONAL
INSTITUTES
OF
NATIONAL
INSTITUTES
OF HEALTH
HEALTH
OF
DIVISION
DMSION
OF RESEARCH
RESEARCH RESOURCES
RESOURCES
BIOTECHNOLOGY
RESOURCES
BIOTECHNOLOGY
RESOURCES PROGRAM
PROGRAM
ANNUAL
ANNUAL PROGRESS
PROGRESS REPORT
REPORT
PART
PART I.,
I., TITLE
TITLE PAGE
PAGE

1.
1. PHS GRANT
GRANT NUMBER:
NUMBER:

6P41RR00786-11
SP41RR00785-11

2. TITLE
TITLE OF GRANT:
GRANT:

SUMEX
SUMEX
Stanford
University
Medical
Stanford
University
Medical
Experimental
Computer
Resource
Experimental
Computer
Resource

3.
3.

NAME
NAME OF
OF RECIPIENT
RECIPIENT INSTITUTION:
INSTITUTION:

4.
4. HEALTH
HEALTH PROFESSIONAL
PROFESSIONAL SCHOOL:
SCHOOL:

Stanford
University
Stanford
University

School
of Medicine
Medicine
School of

S.
5. REPORTING
REPORTING PERIOD:
PERIOD:
Sa.
5a.
Sb.
5b.

FROM:
FROM:
TO:
TO:

08-01-83
08-01-83
07-31-84
07-31-84

6. PRINCIPAL
PRINCIPAL INVESTIGATOR:
INVESTIGATOR:

6a.
6b.

NAME:
NAME:
TITLE:
TITLE:

6c.

SIGNATURE:
SIG NATURE:

7. DATE
DATE SIGNED:
SIGNED:

8. TELEPHONE:
TELEPHONE:

Edward A.
A. Feigenbaum,
Feigenbaum, Ph.D.
Ph.D.
Edward
Professor of
of Computer
Computer Science
Science
Professor

?%.1
AIM Projects
Projects
II.A.4.1.
Project
11.,4.4.1. PATHFINDER
PATHFINDER
Project
II.A.4.2.
II.A.4.2. RXDX
RXDX Project
Project
II.B.
1I.B. Books, Papers, and Abstracts
Abstracts
H.C.
J1.C. Resollrce
Resource Summary
Summary Table
Table

Appendix
Committee
Membership
Appendix A.
A. AIM
AIM Management
Management
Committee
Membership
Appendix
Growth
Appendix B.
B. Community
Community
Growth and
and Project
Project Abstracts
Abstracts

E. A. Feigenbaum

106
106
121
129
130
130
136
136
142
149
149
159
159
166
166
167
175
179
179
183
183
185
190
191
191
197
202
202
202
202

203
203
209
209

Appendix
Appendix

5P·n
SF’41 RR0078S-11
RR0078S-11

List
List of
of Figures
Figures
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure

1:
2:
3:
3:
4:
4:
5:
5:
6:
6:
7:
8:
9:
10:
10:
11:
11:
12:
12:
13:
13:
14:
14:
15:
15:

Current
Current SUMEX-AIM
SUMEX-AIM Decsystem
Decsystem 2060 Computer
Computer Configuration
Configuration
Current
Current SUMEX-AIM
SUMEX-AIM 2020 Computer
Computer Configuration
Configuration
Current
AX Computer
Current Shared V
VAX
Computer Configuration
Configuration
SUMEX-AIM
SUMEX-AIM Ethernet
Ethernet Configuration
Configuration
SUIV1EX-AIM
SUMEX-AIM File
File Server
Server {SAFE}
{SAFE}
SUt\.tEX-AIM
Development
A....X}
SUMEX-AIM Development Vax
Vax {ARDV
{ARDVAX}
Total
by Month
Total CPU
CPU Time
Time Consumed by
Month
Peak Number
Number of Jobs by
by Month
Month
Peak Load Average
by Month
Average by
Month
Monthly
by Community
Monthly CPU
CPU Usage
Usage by
Community
Monthly
Terminal
Connect
by Community
Monthly Terminal Connect Time
Time by
Community
Cumulative
by Project
Cumulative CPU
CPU Usage
Usage Histogram
Histogram by
Project and Community
Community
TYMNET
TYMNET Terminal
Terminal Connect
Connect Time
Time
ARP
ANET Terminal
ARPANET
Terminal Connect
Connect Time
Time
SUMEX-AIM
SUMEX-AIM Growth
Growth by Community
Community

...
in
Ill

10
10
11
12
13
13
14
15
31
32
32
34
35
37
45
45
209

E. A. Feigenbaum

5P41 RR00785-U
RR00785-11

Annual
Annual Report
Report

I.
Description
I. Narrative
Narrative
Description
This
This third
third year
year of
of the 5-year
5-year renewal
renewal of
of the
the SUMEX
SUMEX resource
resource grant
grant has been an
active year
year not
not only
only for
for the
the SUMEX
SUMEX staff,
staff, but
but for
for the SUMEX-AIM
SUMEX-AIM community
community involved
involved
active
in developing
Successes
developing expert
expert systems.
systems.
Successes in developing
developing such systems,
systems, many
many of
of them
them
stemming
stemming from
from projects
projects in the
the SUMEX-AIM
SUMEX-AIM community,
community, continue
continue to
to stimulate
stimulate strong
strong and
growing
growing interest
interest in AI
AI research
research on many
many educational,
educational, governmental,
governmental, and industrial
industrial fronts.
fronts.

This is an annual
annual report
report for
for the Stanford
Stanford University
University Medical
Medical EXperimental
Experimental
This
computer
computer resource
resource for
for applications
applications of
of Artificial
Artificial Intelligence
Intelligence in Medicine
Medicine (SU1vfEX-AIM)
(SUMEX-AIM).
It
It covers the period
period between
between May
May 1, 1983 and April
April 30, 1984.
This
This third
third year
year of
of the 5-year
5-year renewal
renewal of
of the
the SUMEX
SUMEX resource grant
grant has been an
active
active year
year not
not only
only for
for the
the SUMEX
SUMEX start,
staff, but
but for
for the SUMEX-AIM
SUMEX-AIM community
community involved
involved
Successes
in developing
developing expert
expert systems.
systems.
Successes in developing
developing such systems, many
many of
of them
them
stemming
stemming from
from projects
projects in the
the SUMEX-AIM
SUMEX-AIM community,
community, continue
continue to
to stimulate
stimulate strong
strong and
growing
growing interest
interest in AI
AI research
research on many
many educational,
educational, governmental,
governmental, and industrial
industrial fronts.
fronts.
In addition,
addition, this
this past
past year
year has seen concurrent
concurrent development
development of
of SUMEX-AIM
SUMEX-AIM as a
distributed
distributed scientific
scientific resource.
resource. Our
Our approved
approved project
project goals focus principally
principally on the
merging of
of state-of-the-art
state-of-the-art community
community research
research in biomedical
biomedical AI
AI applications
applications with
with new
merging
computing
computing tools
tools and on the challenges they
they will
will bring
bring to
to the
the SUMEX-AIM
SUMEX-AIM community
community and
resource. The
The SUMEX
SUMEX start
staff continues
continues to
to exploit
exploit these advances in professional
professional
resource.
workstations and communication
communication technology,
technology, while
while at
at the
the same time
time maintaining
maintaining our
our high
high
worksta.tions
standards for
for a computing
computing resource.
resource.
sta.ndards
This
This third
third year
year also saw the initiation
initiation of
of a number
number of
of SUMEX-AIM
SUMEX-AIM pilot
pilot projects.
projects.
pilot projects
projects provide
provide new activities
activities and research
research directions
directions for
for the community
community to
to
These pilot
replace
replace existing
existing projects
projects which
which have matured
matured and moved
moved off
off the
the SUMEX-AIM
SUMEX-AIM resource.
resource.
The
The earlier
earlier phases of
of the SUMEX-AIM
SUMEX-AIM resource were characterized
characterized by
by the building
building
of a3 national
national community
community of
of biomedical
biomedical AI
AI collaborators
collaborators around
around a central
central resource located
located
of
St,nnford University.
University.
Beginning with
with 5 projects
projects in 1973, the AIM
AIM community
community grew
grew to
at Stanford
Beginning
11
11 major
major projects
projects at our
our renewal
renewal in 1978. This
This past
past year
year saw the
the completion
completion of
of two
t.wo long
term
DENDRAL
and PUF'F
jVM. There
term and successful projects
projects on SUMEX-AIM;
SUMEX-AIM;
DENDRAL
PUFF/VM.
There
currently
currently are 13 fully-authorized
fully-authorized projects
projects plus seven pilot
pilot efforts.
efforts.
t-.lany
Many of
of the computer
computer programs
programs under
under development
development by
by these groups
groups are maturing
maturing
into
useful
intro tools
tools increasingly
increaingiy
useful to the respective
respective research or
or clinical
clinical communities.
communities. \Ve
We
continue to
to seek out
out new AI
AI applications
applications in our
our community
community of
of biomedical
biomedical and computer
comput.er
continue
scientists
The
community
scientists who
who interact
interact through
through electronic
electronic media.
media.
The SUMEX-AIM
SUMEX-AIM
community is
beginning to evolve as a highly
highly distributed
distributed resource,
resource, with
with the
the SUMEX
SUMEX staff
staff and computer
computer
beginning
facilit.y
facility serving
serving a.<;
as the
t,he backbone
backbone to electronic
electronic communication
communication and systems
systems support.
support. The
The
community is becoming
becoming more and more
more involved
involved in personal
personal computers
computers and professional
professional
commnnity
workststiors,
more heavily
heavily dependent
dependent on network
network communication
communication facilities
facilities for
for
workstations,
and more
in
teractions, collaborations,
interactions,
collaborations, and sharing.
sharing.

The following
following sections
sections cover
cover the activities
activities of
of the SUMEX-AIM
SUMEX-AIM resource
resource this
this past
The
year, including
including brief
brief summaries
summaries of
of the
the our
our objectives,
objectives, a characterization
characterization of
of biomedical
biomedical AI
AI
year,
rese:1fch,
resesrch, resource organization
organization and operating
operating procedures,
procedures, recent
recent core progress
progress in system
system
development
development and basic AI
AI research, and progress
progress in the collaborative
collaborative projects.
projects.

Feigenbaum
E. A. Feigenbaum

5P41
5P41 RR00785-11
RR00785-11

Summary
Summary of
of Research
Research Progress
Progress

I.A.
of
Progress
I.A. Summary
Summary
of Research
Research
Progress
LA.1.
of
and
I.A.1. Overview
Overview
of Objectives
Objectives
and Rationale
Rationale
SUMEX-AIM
SUMEX-AIM ("SUMEX")
(“SUMEX”) isis aa national
national computer
computer resource
resource with
with aa dual mission:
mission: 1)
1)
promoting
promoting applications
applications of
of computer
computer science
science research
research in
in artificial
artificial intelligence
intelligence (AI)
(AI) to
biological
biological and
and medical
medical problems,
problems, and
and 2)
2) demonstrating
demonstrating computer
computer resource sharing
sharing within
within aa
national
community
of
health
research
projects.
The
central
SUMEX-AIM
facility
The central SUMEX-AIM facility is
is
nationa. community
health
projects.
located
physically
in
the
Stanford
University
Medical
located physically
Stanford University Medical School and serves as
as aa nucleus for aa
community
community of medical
medical AI
AI projects
projects at universities
universities around
around the country.
country. SUMEX
SUMEX provides
provides
computing
computing facilities
facilities tuned
tuned to the needs
needs of AI
AI research and communication
communication tools to
facilitate
facilitate remote
remote access,
access, interinter- and intra-group
intra-group contacts,
contacts, and the demonstration
demonstration of
developing
developing computer
computer programs
programs to biomedical
biomedical research collaborators.
collaborators.

I.A.1.1.
is
Intelligence
LA. 1.1. What
What
is Artificial
Artificial
Intelligence
The
The subfield
subfield of computer
computer science known
known as
as Artificial
Artificial Intelligence,
Intelligence, or
or AI,
AI, deals with
with
symbolic
reasoning
using
large
amounts
of
heuristic
knowledge.
Many
of
world’s
symbolic re-aoning using
amounts
heuristic knowledge. Many of the world's
difficult
problems
electronic
electronic or
or mechanical
mechanical
difficult
problems are symbolic,
symbolic, such as troubleshooting
troubleshooting
equipment,
medical
diagnosis
and
therapy
planning,
and
configuring
elemental
equipment, medical diagnosis
therapy planning,
configuring elemental parts
parts into
into a
whole system.
system. For
For these kinds
kinds of
of problems,
problems, AI
AI offers
offers new opportunities
opportunities for
for developing
developing
computerbased solutions.
computer-based
solutions.
AI also uses heuristic
heuristic
In addition
addition to
t.o using
using symbolic
symbolic representations
representations of
of knowledge,
knowledge, AI
methods
thumb, judgmental
judgmental rules
mles that
that
met,hods for
for processing
processing information.
information. Heuristics
Heuristics are rules
rules of
of thumb,
aid in finding
plausible
other areas of
of computing
computing in its
its
bk solutions.
solutions. AI
AI is distinguished
distinguished from
from other
finding plauso’
att,ention
heuristic (non-algorithmic)
(non-algorithmic)
att.ent,ion to
to both
both symbolic
symbolic (non-numeric)
(non-numeric) information
information and heuristic
methods
methods for
for solving
solving problems.
problems.

Placing AI
AI in
in Computer
Science
Placing
Computer
Science
The major
major focus of
of AI
AI is understanding
understanding intelligence
intelligence through
through construction
construction (or
(or
The
programming) of
of machines
machines that
that behave intelligently.
intelligently. That
That is a grand
grand goal. In
In the
the shortshortprogramming)
term, AI
AI research
research focuses on non-numerical
non-numerical problem
problem solving
solving in
in order
order to
to build
build experience
experience
term,
with problem
problem solving
solving methods,
methods, techniques
techniques for
for representing
representing various
various kinds
kinds of
of knowledge,
knowledge,
with
interfaces with
with users.
users, and numerous
numerous other
other issues.
issues.
interfaces
of t,he
the distinguishing
distinguishing features
features of
of problems
problems for
for which
which AI
AI methods
methods have
have been
One of
t,hat the problems
problems are not
not well-structured.
well-structured. That
That is,
is. one does not
not already
already
developed is t*hat
developed
know in advance
advance (from
(from the
the problem
problem description
description alone)
alone) what
what the
the best
best method
method is for
for solving
solving
know
the problem.
problem, In
In short,
short, there
there are no algorithms.
algorithms. Broadly
Broadly speaking,
speaking, AI
AI substitutes
substitutes
the
exploratory search
search for
for precise,
precise, algorithmic
algorithmic solution
solution methods.
methods.
explorat.ory

Expert Systems
Systems and
and Applications
Applications
Expert
The national
national SUMEX-AIM
SUMEX-AIM resource
resource is an outgrowth
outgrowth of
of a long,
long, interdisciplinary
interdisciplinary line
line
The
of artificial
artificial intelligence
int.elligence research
research at
at Stanford
Stanford and
and elsewhere
elsewhere concerned
concerned with
with the
the
of
development
of
concepts
and
techniques
for
building
"expert
systems"
[1].
An
"expert
development of concepts and techniques for building “expert systems” [l]. An “expert
system"” is an inteliigent
intelligent computer
computer program
program that
that uses knowledge
knowledge and
and inference
inference procedures
procedures
sy,item
to solve
solve problems
problems that
that, are difficult
difficult enough
enough to
to require
require significant
significant human
human expertise
expertise for
for their
their
to
solution. For
For some fields
fields of
of work,
work, the
the knowledge
knowledge necessary
necessary to
to perform
perform at
at such
such aa level,
level,
solution.

E. A.
A. Feigenbaum
Feigenbaum
E.

Overview
Overview of Objectives
Objectives and Rationale
Rationale

5P41
SP41 RR00785-11
RR00785-11

plus the inference
inference procedures
procedures used,
used, can be thought
thought of as
as aa model of the expertise
expertise of the
expert
expert practitioners
practitioners of that
that field.
Two
features
Two important
important
features that
that distinguish
distinguish expert
expert systems from
from conventional
conventional
programs
Expert
flexible in the sense
Expert systems are flexible
sense
programs are flexibility
flexibility and understandability.
understandability.
that
in the sense
that they
they can be
be changed and extended
extended easily, and they
they are understandable
understandable
sense
that
that they
they can explain
explain the contents
contents of
of their
their own
own knowledge
knowledge bases
bases and their
their own
own lines of
reasoning
features are especially
especially important
important in medicine,
medicine, where
where knowledge
knowledge is
is
reasoning [10].
[lo]. These features
changing
changing rapidly
rapidly and where practitioners
practitioners have to understand
understand the reasons for
for a program's
program’s
decisions because
because they
they have to accept responsibility
responsibility for following
following (or
(or not
not. following)
following) those
decisions.
The
The application
application areas range from
from medicine
medicine to electronics,
electronics, from
from machinery
machinery to
software.
software. The problems
problems ra.nge
range from
from diagnosis and troubleshooting
troubleshooting (analysis)
(analysis) problems
problems to
planning
pla,nning and configuration
configura.tion (synthesis)
(synthesis) problems.
problems. Knowledge
Knowledge bases
bases for
for expert
expert systems are
built
built iteratively
iteratively -- usually
usually through
through long interactions
interactions over
over many
many months
months between
between a human
human
specialist.
specialist who
who underst.ands
understands the details
details of
of the domain
domain and a knowledge
knowledge engineer
engineer who
who
understands
understands the programming
programming details
details of
of the system.
system.
The
The knowl-edge
knowledge of
of an expert
expert system
system consists of
of facts and heuristics.
heuristics. The
The "facts"
“facts”
constitute
constit,ute a body
body of
of irrformation
&formation that
that is widely
widely shared, publicly
publicly available,
available, and generally
generally
agreed upon by experts
experts in a field.
field. The
The "heuristics"
“heuristics” are the mostly-private,
mostly-private, little-discussed
little-discussed
judgment (rules
rult's
rules of good judgment
(rules of
of plausible
plausible reasoning,
reasoning, rules
rules of
of good guessing) that
that
charact.erize
charact,erize expert-level
expert,-level decision making
making in the field.
field. The
The performance
performance level of
of an expert
expert
system
system is primarily
primarily a function
funct.ion of
of the size and quality
quality of
of the knowledge
knowledge base that
that it
it
possesses.
is the
possesses. One of
of the key
key ideas in maintaining
maintaining flexibility
flexibility and understandability
understandability
clean separation
program that
separation of
of elements
elements of
of the knowledge
knowledge base from
from elements
elements of
of the program
that
interpret
base.
interpret the knowledge
knowledge base.
The
building expert
The major
major issues
issues in building
expert systems,
systems, at the moment,
moment, are:
•l selecting
problem (in
importance,
selecting an appropriate
appropriate problem
(in terms
terms of
of size, difficulty,
difficulty,
importance,
decomposability,
decomposability, risk)
risk)
•l selecting
selecting a representation
representation and control
control structure
structure (or
(or framework
framework system
system that
that
sllpplies
both),
supplies both),
•l set.t.ling
problem,
set,t.ling on an
a.11appropriate
appropriate vocabulary
vocabulary and conceptualization
conceptualization for
for the
the problem,
•l finding
finding an available
available expert,
expert,
transferring the expert’
expert's
knowledge into
into the
the program
program (knowledge
(knowledge engineering),
engineering),
l• transferring
s knowledge
refining t,he
the knowledge
knowledge base with
with feedback
feedback from
from test
test cases,
cases,
l• refining
packaging t,he
the system
system in a form
form that
that is acceptable
acceptable to
to end-users,
l• packaging
l• validating
validating the quality
quality of
of the
the program’
program's
s advice.
advice.
of the best
best known
known expert
expert systems
systems is MYCIN
MYCIN [3], a program
program in which
which the
the
One of
separation of
of knowledge
knowledge (of
(of medicine)
medicine) from
from the rest
rest of
of the
the program
program was carefully
carefully
sepa.rstion
engineered. (The
(The abstracted
abstracted ca.se
case of
of an arbitrary
arbitrary knowledge
knowledge base and a framework
framework
engineered.
interpreter, plus
plus auxiliary
a.uxiliary programs,
programs, was achieved
achieved in the EMYCIN
EMYCIN system
system [16],
to which
which
interpreter,
[lS], to
other domains
domains can be added to
to build
build a diagnostic
diagnostic system
system in those domains.)
domains.)
knowledge of
of other
knowledge
Cllrrent.ly authorized
authorized projects
projects in
in the
the SUMEX
SUMEX community
community are concerned
concerned in
in some
Currently
way with
with the
the application
application of
of AI
AI to
to biomedical
biomedical research*.
research·. The
The tangible
tangible objective
objective of
of this
this
way
approach is the
t.he development
development of
of computer
computer programs
programs that
that will
will be more
more general
general and effective
effective
approach

•

Bri€'f abstracts
abstracts of
of the
the various
various projects
projects can
can be
be found
found in
in Appendix
Appendix BB on
on page
page 209
200 and
and more
more detailed
detailed progress
progress
Brief
summaries in
in Section
Section IIII on
on page
page IHI.
60.
summaries

A. Feigenbaum
Feigenbaum
E. A.

RR00785-111
5P41 RR007851

Overview of
of Objectives
Objectives and
and Rationale
Rationale
Overview

consultative tools
tools for
for the
the clinician
clinician and
and medical
medical scientist.
scientist. There
There already
already have
have been
consultative
promising result*s
results in
in areas such as chemical
chemical structure
structure elucidation
elucidation and
and synthesis,
synthesis, diagnostic
diagnostic
promising
consultation, molecular
molecular biology,
biology, and
and modeling
modeling of
of psychological
psychological processes.
consult,ation,
much is yet
yet to
to be learned
learned in the
t.he process of
of fashioning
fashioning a coherent
coherent
Needless t.o say, much
scientific discipline
discipline out
out of
of the
the assemblage of
of personal
personal intuitions,
intuitions, mathematical
mathematical procedures,
procedures,
scientific
emerging theoretical
theoretical structure
st.ructure comprising
comprising artificial
artificial intelligence
intelligence research.
research. State-of-theState-of-theand emerging
art programs
programs are far
far more
more narrowly-specialized
narrowly-specialized and
and inflexible
inflexible than
than the
the corresponding
corresponding
art
of human
human intelligence
intelligence they
they emulate;
emulate; however,
however, in
in special
special domains
domains they
they may
may be of
of
aspects of
comparable or
or greater
greater power,
power, e.g., in
in the
the solution
solution of
of formal
formal problems
problems in
in organic
organic chemistry.
chemistry.
comparable

I.A.1.2. Impact
Impact of
of AI
AI in
in Biomedicine
Biomedicine
I.A.1.2.
There is a certain
certain inevitability
inevitability to
to the
the field
field of
of Artificial
Artificial Intelligence
Intelligence and
and its
its
There
applications, in particular,
particular, to
to medicine
medicine and biosciences. The
The cost
cost of
of computers
computers will
will
applications,
continue to
to fall
fall drastically
drastically during
during the
the coming
coming two
two decades. As itit does, many
many more
more of
of t,he
the
continue
practitioners of
ot the
t.he world’
world's
professions will
will be persuaded
persuaded to
to turn
turn to
to economical
economical automatic
automatic
practit,ioners
s professions
information processing
processing for
for assistance in managing
managing the
the increasing
increasing complexity
complexity of
of their
their daily
daily
information
They will
will find,
find, from
from most
most of
of computer
computer science, help only
only for
for those problems
problems that
that
tasks. They
have a mathematical
mat.hematical or
or statistical
statistical core, or
or are of
of a routine
routine data-processing
data-processing nature.
nature. But
But
ha.ve
such problems
problems will
will be relatively
relatively rare,
rare, except
except in
in engineering
engineering and
and physical
physical science. In
In
management, indeed
indeed in most
most of
of the
the world’
world's
work, the
the daily
daily tasks
tasks are
medicine, biology,
biology, management,
medicine,
s work,
requiring symbolic
symbolic reasoning
reasoning with
with detailed
detailed professional
professional knowledge.
knowledge. The
The computers
computers
those requiring
that will
wiII act
act as intelligent
intelligent alrsivtants
assistants for
for these professionals
profeSSionals must
must be endowed
endowed with
with
that
symbolic
symbolic reasoning
reasoning capabilities
capabilities and knowledge.
knowledge.
The
The growth
growth in medical
medical knowledge
knowledge has far
far surpassed the
the ability
ability of
of a single
practitioner to master
practitioner
master it
it all,
all, and the computer's
computer’s superior
superior information
information processing
processing capacity
capacity
there
by offers
thereby
offers a natural
natural appeal. Furthermore,
Furthermore, the reasoning
reasoning processes
processes of
of medical
medical experts
experts
are poorly
poorly understood;
understood; attempts
attempts to
to model
model expert
expert decision-making
decision-making necessarily
necessarily require
require a
degree of
of introspection
introspection and a structured
structured experimentation
experimentation that
that may,
may, in turn,
turn, improve
improve the
the
quality
quaiity of
of the
the physician's
physician’s own
own clinical
clinical decisions, making
making them
them more
more reproducible
reproducible and
defensible.
New insights
defensible. New
insights that
that result
result may
may also allow
allow us more
more adequately
adequately to
to teach m~dical
medical
students
students and house staff
staff the techniques
techniques for
for reaching
reaching good decisions, rather
rst,her than
than merely
merely to
to
offpr
offer ;).n. collection
collect ion of
of facts
facts which
which they
they must
must independently
independently learn
learn to
to utilize
utilize coherently.
coherently.
The
The kno'wledge
knowledge that
that must
must be used is a combination
combination of
of factual
factual knowledge
knowledge and
heuristic
The latter
1atLer is especially
especially hard
hard to
to obtain
obtain and represent
represent since the
heuristic knowledge.
knowledge. The
experts
expertIs providing
providing it
it are mostly
mostly unaware
unaware of
of the heuristic
heuristic knowledge
knowledge they
they are using.
~ledical
hledical and scientific
scientific communities
communities currently
currently face many
many widely-recognized
widely-recognized problems
problems
relating
relating to the rapid
rapid accumulation
accumulation of
of knowledge,
knowledge, for example:
example:
•l codifying
codifying theoretical
theoretical and heuristic
heuristic knowledge
knowledge
•l effectively
effectively using the wealth
wealth of
of information
information implicitly
implicitly available
available from
from textbooks,
textbooks,
journal
articles
and
other
practitioners
journal articles
other practitioners
•l disseminating
disseminating that
that knowledge
knowledge beyond
beyond the intellectual
intellectual centers where
where it
it is
is
collected
collected
•l customizing
customizing the presentation
presentation of that
that knowledge
knowledge to individual
individual practitioners
practitioners as
as
well
well as
‘3s customizing
customizing the
t,he application
application of the information
information to individual
individual cases
cases
\Ve
We believe that
t.hat computers
computers are an inevitable
inevitable technology
technology for
for helping
helping to overcome
overcome

E. A. Feigenbaum
Feigenbaum

Overview of
of Objectives
Objectives and Rationale
Rationale
Overview

5P41 RR00785-11
RR00785-11

While recogOlzmg
recognizing the
the value
value of
01 mathematical
mathematical modeling,
modeling, statistical
statistical
these problems.
While
problems.
classification, decision
decision theory
theory and other
other techniques,
techniques, we believe
believe that
that effective
effective use
use of
of such
classification,
methods depends on using
using them
them in conjunction
conjunction with
with less
less formal
formal knowledge,
knowledge, including
including
methods
contextual
contextual and strategic
strategic knowledge.
knowledge.
Artificial inklligence
intelligence offers
offers advantages
advantages for
for representing
representing and using
using information
information that
that
Artificial
will allow
allow physicians
physicians and scientists
scientists to
to use
use computers
computers as
as intelligent
intelligent assistants.
assistants. In
In this
this way
way
will
envision a significant
significant extension
extension to
to the
the decision-making
decision-making powers
powers of
of specific
specific practitioners
practit.ioners
we envision
without
without reducing
reducing the importance
importance of
of those individuals
individuals in that
that process.
Knowledge is power,
power, in the profession
profession and in the intelligent
intelligent agent.
agent. As we proceed
Knowledge
to model
model expertise
expertise in medicine
medicine and its
its related
related sciences,
sciences, we find
find that
that the power
power of
of our
our
to
programs derives
derives mainly
mainly from
from the
the knowledge
knowledge that
that we are able to
to obtain
obtain from
from our
our
programs
collaborating practitioners,
practitioners, not
not from
from the sophistication
sophist,ication of
of the inference
inference processes
processes we
collaborating
observe them
them using. Crucially,
Crucially, the knowledge
knowledge that
that gives power
power is not
not merely
merely the
the
knowledge of
of the textbook,
textbook, the lecture
lecture and the
the journal,
but the knowledge
knowledge of
of good
knowledge
journal, but
good
practice--the
experiential knowledge
knowledge of
of good
knowledge
practice--tlle experiential
good judgment
judgment and good
good guessing,
gue88ing, the knowledge
of the practitioner's
practitioner’s art
art that
that is orten
often used in lieu
lieu of
of facts
facts and rigor.
rigor.
of
This
This heuristic
heuristic
knowledge
knowledge is mostly
mostly private,
private, even in the
the very
very public
public practice
practice of
of science. It
It is almost
almost never
never
taught
taught explicitly,
explicitly, is almost
almost never
never discussed and critiqued
critiqued among
among peers, and most
most orten
often is
not even in the
t.he moment-by-moment
moment-by-moment awareness of
01 the
the practitioner.
practitioner.
not
Perhaps the the most
most expansive
expansive view
view of
of the significance
significance of
of the work
work of
of the
the SUMEXSUMEXPerhaps
AIhl community
communit.y is that
that a methodology
methodology is emerging
emerging for
for the systematic
systematic explication,
explication, testing,
testing,
AIM
dissemination, and teaching
teaching of
of the heuristic
heuristic knowledge
knowledge of
of medical
medical practice
practice and scientific
scientific
dissemination,
performance.
performa,nce. It
It may
may be less
less important
important that
that computer
computer programs
programs can be organized
organized to
to use
use
this lcnowledge than
Lhan that
that the knowledge
knowledge itself
itself can be organized
organized for
for the use of
of the
the human
human
this
practitioners
practitioners of
of today
today and tomorrow.
tomorrow.
Evidence of
of the impact
impact of
of SUMEX-AIM
SUMEX-AIM in promoting
promoting ideas such as
as these, and
Evidence
developing the pertinent
pertinent specific
specific techniques,
techniques, has been the explosion
explosion of
of interest
interest in medical
medical
developing
artificial intelligence
intelligence and the specific
specific research efforts
efforts of
of the SUMEX
SUMEX community.
community.
artificial
As
SU?rlEX has
ha.s entered
entered its second decade, we have found
found that
that the small
small community
community of
of
SU!\fEX
researchers
970'ss has now
researchers that
t.hat characterized
characterized the AIM
AIM field
field in the
the early
early 11970’
now grown
grown to a large,
accomplished, and respected
respected research community.
community. The
The American
American Association
Association for
for Artificial
Artificia.1
accomplished,
Intelligence
Intelligence (AAAI),
(A,4.41), the principal
principal scientific
scientific membership
membership organization
organization for
for the AI
AI field,
field, has
members, over
over 1000 of
of whom
whom are members
members of
of the medical
medical special interest
interest group
group
4000 members,
known as
as t.he
the AAAI-M.
AAAI-M.
This subgroup
subgroup was founded
founded by
by members
members of
of the SUMEX-AIM
SUMEX-AIM
known
This
community who
who were active
active in AAAI
AAAI and is the only
only active
active subgroup
subgroup in the Association.
Association.
community
The organization
organization distributes
distributes semiannual
semiannual newsletters
newsletters on medical
medical AI
AI and provides
provides a focus
The
for co-sponsoring
levan t medical
co-sponsoring re
relevant
medical computing
computing meetings
meetings with
with other
other societies
societies (such as
as the
American Association
Association for
for Medical
Medical Systems
Systems and Informatics
Informatics -- J\.-\MSI).
k4MSI).
Medical AI
AI papers
American
Medical
prominently featured
featured at
at’ both
both medical
medical computing
computing and artificial
artificial intelligence
intelligence meetings,
meetings,
are prominently
artificial int.elligence
intelligence is now
now routinely
routinely featured
featured as
as a specific
specific subtopic
subtopic for
for specialized
specialized
and artificial
sessions at
a,t medical
mcdicsl computing
computing and other
other medical
medical professional
professional meetings.
meetings. For
sessions
For example,
example,
members of
of the AIM
.41bf community
communit,y have represented
represented the field
field to
to physicians
physicians at the American
American
members
of Pathology
Pathology and American
American College
College of
of Physicians
Physicians meetings
meetings for
for the last
last several
several
College of
A mere decade ago, the words
words "artificial
“artificial intelligence"
intei!igence” were never
never uttered
uttered at such
years. A
The growing
growing interest
interest and recognition
recognition are largely
largely due to
to the activities
activities of
of the
the
conferences. The
SU?.lES-.4IM community.
community.
SU!'.fEX-AIM
Another indication
indication of
of the growing
growing impact
impact of
of the SUMEX-AIM
SUMEX-AIM community
community is its
its
Another
effect, on medical
medical education.
education.
effect
For
For reasons such as
as those outlined
outlined above, there
there is an
increasing recognition
recognition of
of the
t,he need for
for a revolution
revolution in the way
way medicine
medicine is taught
taught and
increasing

E. A.
A. Feigenbaum
Feigenbaum

Overview
Overview of Objectives
Objectives and Rationale
Rationale

5P41 RR00785-11
RR00785- I I

medical
Computing
medical students
students organize
organize and access
access information.
information.
Computing technology
technology isis routinely
routinely
cited as
as part
part of this
this revolution,
revolution, and artificial
artificial intelligence
intelligence (and SUMEX-AIM
SUMEX-AIM research)
research)
generally
generally figures
figures prominently
prominently in such discussions. Such diverse organizations
organizations as
as the
Na.tional
Na.tional Library
Library of Medicine,
Medicine, the American
American College of Physicians,
Physicians, the Association
Association of
American
American Medical
Medical Colleges, and the Medical
Medical Library
Library Association
Association have all called for
for
sweeping changes in medical
medical education,
education, increased educational
educational use
use of computing
computing
technology,
technology, enhanced research in medical
medical computer
computer science, and career development
development for
people working
working at the interface
interface between
between medicine
medicine and computing;
computing; reports
reports of all four
four
organizations
orga,nizations have
ha.ve specifically
specifically cited
cited the role of
of artificial
artificial intelligence
intelligence techniques
techniques in future
future
medical
programs
medical practice
practice and have used SUMEX-AIM
SUMEX-AIM
programs as
as examples of where
where the
technology
technology is
is gradually
gradually heading.
heading.
In summary,
summary, the logic which
which mandates
mandates that
that artificial
artificial intelligence
intelligence playa
play a key
key role in
enhancing
enhancing knowledge
knowledge management
management and access
access for biomedicine
biomedicine -- a logic in which
which we have
long believed
gradually become evident
evident to much
much of the biomedical
biomedical community.
community. We
believed -- has gradually
are encouraged
encouraged by
by this
this increased recognition,
recognition, but
but realistic
realistic about
about the significant
significant research
cha.llenges
cha.llenges that
that remain.
remain. Our
Our goals are accordingly
accordingly both
both scientific
scientific and educational.
educational. We
We
continue
continue t.o
to pursue
pursue the research objectives
objectives that
that have always
always guided
guided SUMEX-AIM,
SUMEX-AIM, but
but
must
must also undertake
undertake educational
educational efforts
efforts designed to inform
inform the biomedical
biomedical community
community of
of
our
our results
results while
while cautioning
cautioning it
it about
about the challenges remaining.
remaining.

A. Feigenbaum
Feigenbaum
E. A.

Details of
of Technical
Technical Progress
Progress
Details

5p·H
5P4 1 HH00785-11
R.R00785- 11

I.A.2.
Details
of
Progress
I.A.2.
Details
of Technical
Technical
Progress
I.A.2.1.
Facility
Management
and
I.A.2.1.
Facility
Management
and Operation
Operation
The
The following
following materia.l
materia.1 covers the SUMEX-AIM
SUMEX-AIM resource activities
activities over
over the past
past
year
year in greater
greater deta.il.
detail. Individual
Individual sections
sections cover
cover progress
progress in ;
•l Facility
Facility Management
Management and Operation
Operation

Timesharing Systems
Systems
•l Timesharing
•l Professional
Professional \Vorkstations
Workstations
•l Networking
Networking and Communications
Communications
These sections
sect.ions outline
outline accomplishments
accomplishments in the context
context of
of the resource staff
staff and resource
management.
management. Details
Details of
of the progress and plans
plans for
for our
our external
external collaborative
collaborative projects
projects are
presented
presented in Section
Section II
II beginning
beginning on page 69.

I.A.2.2.
Facility
Management
and
I.A.2.2.
Facility
Management
and Operation
Operation
SUMEX-AIM
continues
SUMEX-AIM
continues to
to manage and
and operate
operate it's
it’s computing
computing resources in a
effect.ive
effect,ive and efficient
efficient manner
manner conducive
conducive to
to providing
providing a reliable
reliable and robust
robust computing
computing
environment.
environment.
While
While the
the previous
previous year
year (Year
(Year 10) involved
involved a major
major move from
from the KIlO
KIlO Tenex
Tenex
syst.em
system to a new DECsystem
DECsystem 2060, this
this year
year saw more emphasis
emphasis to
to our
our gradual
gradual move to
to
distributed
distributed processing,
processing, while
while continuing
continuing to
to improve
improve our
our excellent
excellent timesharing
timesharing environment
environment
on the 2060. This
This development
development is covered
covered in full
full in section
section I.A.2.2
I.A.2.2 starting
starting on page 15.
Ollr
Our continued
continued movement
movement to
to professional
professional workstations
workstations has taken
taken on several
several forms.
forms.
We
We have continued
continued to
to acquire
acquire Lisp
Lisp machines
machines for
for use
use by
by the SUMEX
SUMEX community
community while
while at
the same time
time investigating
investigating t.he
the use of
01 remote
remote virtual
virtual graphics
graphics and new lower
lower cost
workstat,ions
workstations such as
as the Apple
Apple Macintosh,
Macintosh, Sun workstations,
workstations, and others
others that
that are
appearing
The
The development
development of
of professional
professional workstations
workstations is covered
covered in
apprnring on the market.
market.
more
more detail
detail in section
section I.A.2,3
I.A.2.3 starting
starting on page 21.
SUMEX
SUMEX cont,inues
continues to
to expend
expend a great
great deal of
of effort
effort in the
the support
support and development
development
of
Key
communications facilities.
facilities.
Key to
to our
our ability
ability to provide
provide the
the
of our
our networking
networking and communications
maximum
maximum computing
comput,ing power
power available
available to
to the
t,he greatest
greatest number
number of
of users is a mechanism
mechanism for
for
making
making it
it, irrelevant
irrelevant where
where that
that user is physically
physically located.
located. By
By having
having a robust
robust networking
networking
and communications
communications environment,
environment, we are able to extend
extend our
our facility
facility to
to any
any user or
or group
group
of
of users, thereby
thereby making
making available
available to
to them
them the power
power and convenience
convenience of
of SUMEX.
SUMEX.
Further
Further information
information on the
the progress made in networking
networking and communications
communications can be
found
found in section
section I.A.2.4
I.A.2.4 starting
starting on page 23.
In
In the area of
of fa.cility
fa.ci!ity management
management and operation,
operation, several
severa.! noteworthy
noteworthy events
events
occurred
occurred over
over the past
past year
year which
which will
will be explained
explained in more detail
detail here.

E. A. Feigenbaum
Feigenbaum

Details
Details of Technical
Technical Progress

5P41 RR00785-11
RR00785-1 I

SUA/EX/IIPP
Facility
SUhdEX/HPP Welch
Welch Road
Road Computing
Computing
Facility
A major
major development
development this
this past year
year at SUMEX
SUMEX was the move of the Heuristic
Heuristic
Programming
Project
to
their
new
location
at
701
Programming Project
their
location
701 Welch
Welch Road, adjacent
adjacent to the Stanford
Stanford
Medical
Medical Center.
Center. Since this
this group
group is a major
major user of the SUMEX-AIM
SUMEX-AIM resource and the
focus for most
most of the core AI
‘41 research, a good deal of effort
effort was expended to provide
provide a
robust
robust computing
computing environment
environment at their
their new location.
location. This
This development
development involved
involved several
stages and levels of technical
technical deveL:;pment,
devei;;pment, ranging
ranging from
from construction
construction of the machine
machine
room,
room, new cable and wiring
wiring installation,
installation, procurement
procurement and setup of networking
networking hardware,
hardware,
to major
major new developments
developments in the networking
networking software
software and a twisted
twisted pair
pair Ethernet
Ethernet
communication
communicat.ion link
link between
between this
this site and the main
main SUMEX
SUMEX Computer
Computer room.
room. All
A!! of the
hardware
hardware and facilities
facilities purchases were funded
funded from
from sources other
other than
than SUMEX.
SUMEX.
\Ve
We setup the general
genera! communications
communications capabilities
capabilities for
for the two
two buildings
buildings occupied
occupied by
by
the HPP.
HPP. This
This involved
involved wiring
wiring up local terminals,
terminals, installing
installing local Ethernets
Ethernets (both
(both 3 and 10
10
mt'gabit
megabit capability),
capability), and acquiring
acquiring and installing
installing networking
networking hardware
hardware such as
as terminal
terminal
interface
interface processors (TIP)
(TIP) and gateways,
gateways, as
as well
we!! as
as extending
extending the current
current SUMEX
SUMEX TIP
TIP
and GATE\\,
A
Y
software
to
handle
both
3
and
10
megabit
network
traffic.
GATEWAY software
handle both
megabit network traffic.
But
But the most
most important
important and most
most interesting
interesting development
development in this
this process was the
twisted pair'
“twist,ed
pair’ ethernet
ethernet developed
developed by
by the SUMEX
SUMEX engineering
engineering staff
stall to allow
allow high
high speed
reliable
reliable communications
communications between
between this
this Welch
Welch Road facility
facility and the SUMEX
SUMEX machine
machine room.
room.
Further
Further information
information on this
this new ethernet
ethernet can be found
found in Section
Section I.A.2.4
I.A.2.4 on page 23.
II

HPP
HPP researchers are routinely
routinely using this
this link
link to
to communicate
communicate with
with SUMEX
SUMEX and the
the
central
central university
university network.
network. In addition,
addition, various
various Lisp
Lisp machines
machines and printers
printers located
located in the
HPP
HPP facility
facility and connected
connected to
to a local network
network are able to communicate
communicate with
with the
the
university
university network.
network.
The
The end result
result is that
that we have successfully
successfully been able to
to extend
extend the
the SUMEX
SUMEX
compnting
providing a high
computing environment
environment to a remote
remote site,
site, providing
high speed link
link to
to the facilities
facilities of
of
SUMEX
processing. We
SUMEX While
while also allowing
allowing for
for local
local distributed
distributed processing.
We see
see this
this experience
experience has
being most
most valuable
valuable in the future
future as
as we move further
further into
into a distributed
distributed environment,
environment,
while
provided by
by large
while still
still needing
needing the sharing
sharing of
of resources and communication
communication links
links provided
t.imesharing
timesharing systems
systems and local area networks.
networks.

Dig£tal Equipment
Equipment Corporation
stops development
0/86-6it
product line
Digitnl
Corporation
stops
development
of S&bit product
line
Digital Equipment
Equipment Corporation,
Corporation, a long
long time
time supplier
supplier of
of high
high speed 36-bit
36-bit
Digital
timesharing computers
computers to
to the
the Artificial
Artificial Intelligence
Intelligence community,
community, announced
announced that
that itit was
timesharing
stopping all
all development
development of
of future
future 36bit
36-bit products,
products, and instead
instead sta.rting
starting a program
program to
to
stopping
provide a migration
migration path
path to
to its
its line
line of
of VAX
V A..,{ minicomputers.
minicomputers.
provide
Many DEC
DEC 20 customers
customers had
had been anticipating
anticipating a new yet
yet unannounced
unannounced machine
machine
Many
from DEC
DEC code named
named the
the'‘Jupiter’
Jupiter'.
which had been reported
reported to
to be a order
order of
of magnitude
magnitude
from
, which
faster than
than the
t.he current
current KLlO
KLlO processor
processor used in DEC20’
DEC20's
DEClO's.
However, DEC’
DEC's
fa.ster
s and DEClO’
s. However,
s
announcement means this
this effort
effort has stopped,
stopped, and we can expect
expect no more
more 36bit
36-bit products
products
announcement
from Digits!
Digital Equipment
Equipment Corporation.
Corporation.
from
The effect
effect of
of t’
this
announcement to
to the
the AI
AI community
community is disappoint,ing,
disappointing, alt,hough
although
The
his announcement
not totally
totally unexpected.
unexpected. The
The DECsystem20
DECsystem20 has been the
the predominant
predominant timesharing
timesharing machine
machine
not
to support
support Artificial
Artificial Intelligence
Intelligence based research,
research, but
but yet
yet researchers
researchers have been in
in
used to
of more
more processing
processing power
power and
and larger
larger address spaces for
for quite
quite a few years.
years. DEC
DEC has
need of
clearly decided
decided to
to devot,e
devote their
their resources
resources to
to VAX
VAX development.
development. For
For those
tho~e in
in need of
of
clearly
greater 36bit
3u-bit processing
processing power
power or
or address space, you
you must
must now
now look
look to
to newer
newer less
less
greater

E. A.
A. Feigenbaum
Feigenbaum

RR00785-11
5P41 RR0078S-11

Details of
of Technical
Technical Progress
Progress
Details

experienced
experienced companies
companies such as
as Foonley
Foonley and Systems
Systems Concepts
Concepts for
for follow
follow on 36-bit
36bit
products
products which
which those firms
firms are preparing.
preparing.
The
The impact
impact of
of this
this decision
decision on the
the SUMEX-AIM
SUMEX-AIM community
community must
must be examined
examined in
conjunclion
conjunclion with
with our
our development
development of
of AI
AI systems
systems on personal
personal Lisp
Lisp machines.
machines. We
We have
outlined
outlined very
very clearly
clearly our
our plans
plans to
to move
move to
to distributed
distributed Lisp-based
Lisp-based workstations
workstations for
for AI
AI
t,his is clearly
clearly where
where we see
see the
the AI
AI computing
computing market
market heading.
heading. These
Research, and this
ma.chines
machines offer
offer much
much better
better cost./performance
cost/performance ratios
ratios than
than timesharing
timesharing machines,
machines, high
high
resolution bit-mapped
bit-mapped screens, and powerful
powerful Lisp
Lisp programming
programming environments
environments for
for the
resolution
development
development, and eventual
eventual dissemination
dissemination of
of AI
AI based systems. However,
However, this
this is not
not too
too say
we still
still do not
not see
see a role for
for the large timesharing
timesharing machine
machine in our
our environment.
environment. We
We still
still
of a large central
central mainframe
mainframe computer
computer as the
the anchor
anchor for
for a large
believe in the use of
con1munit.y of
of users. The
The mainframe
mainframe also functions
functions as
as a central
central facility
facility for
for communication
communication
community
collaboration, and provides
provides fast
fast Lisp
Lisp cycles for
for program
program development
development where
where the
and collaboration,
application
application is not
not in need of
of a specialized
specialized workstation.
workstation.
Other SUA/EX
SUMEX
C’omputing
Facilities
Other
Computing
Facilities

SUMEX continues
continues to
to support
support other
other mainframe
mainframe computers,
computers, file servers, professional
professional
SUMEX
workstations,
workstations, and assorted
assorted printers
printers and terminals
terminals for
for use
use by
by the
the SUMEX-AIM
SUMEX-AIM community.
community.
1. The
The SUMEX-AIM
SUMEX-AIM File
File server,
server, based on a VAX
VAX 11/750
11/750 computer,
computer, continues
continues to
to

of the workstation
workstation users within
within SUMEX-AIM.
SUMEX-AIM. The
The use
use of
of
serve the needs of
SAFE by
by users of
of our
our 2060 is minimal.
minimal. We
We plan
plan to extend
extend the
the use
use of
of SAFE
SAFE in
SAFE
future by
by providing
pro\-iding more
more convenient
convenient access
access by
by 2060 users than
than is currently
current,ly
the future
available.
available.
The VAX
VAX 11
11,/780
computer system,
system, originally
originally purchased
purchased with
with DARPA
DARPA funds
funds
2. The
/780 computer
and previously
previous!y located
located in Margaret
Margaret Jacks Hall
Hall on campus,
campus, has become a
SUMEX-AIM resource this
this past
past year.
year. The
The system
system was moved
moved to a new location
location
SUMEX-AIM
Stanford campus
campus which
which provides
provides a better
better environment
environment for
for a computer
computer
on the Stanford
of this
this size. This
This VAX
VAX is now
now shared
shared between
between the
the Computer
Computer Science
of
Department
Department and the SUMEX-AIM
SUMEX-AIM community.
community.
3. SUMEX
SUMEX continues
continues to support
support a wide
wide range of
of professional
professional workstations
workstations from
from
vendors as
as Xerox,
Xerox, Symbolics,
Symbolics, and Hewlett
Hewlett Packard
Packard for
for the development
development
such vendors
testing of
of AI
AI applications.
applications. Additional
and testing
Additional work
work has been started
started to explore
explore
use of
of the Apple
Apple Macintosh
Macintosh and Apple
Apple Lisa
Lisa within
within SUMEX.
SUMEX. More
More
the use
information on these developments
developments can be found
found in section
section I.A.2.3.
I.A.2.3.
information

E. A. Feigenbaum
Feigenbaum

5P41 RR00785-11
RIl00785-11

Details of
of Technical
Technical Progress
Progress
Details

7
G3nmil
Central Proceaor
Processor
DECKLlo-E
DEC
KLlO-E
2M words
warns of
of memoly,
memory, c¶cho
Cache
2M

RH20
RH2o
L

RH20
RtKo

MassBus

" . .sus

DisK colltr0ller
Controller
D&k
and Dnvc
Onve
and
DECRP07
DEC
UP07

Disk Controlkr
ContrOller
Disk
and Dnve
Drive
MCI
DECRP07
DEC
R PO7

RH20 I
I
RH20

IJNIEUS
UNIBUS

MusSus
M&s&m

4
OIB20
DIE20

DEC AN20
ARPAnet
Interlace

and OnY8

DECRPOe

MEIS
MassBus
Ettlemet
Interlace

K1.INIKUne

~glnQTTY

DEC LP·26
LP-26
DEC
urm Pnnta
UnePnntar
t-

6 Une Scannens

DEC OH·',
96 Unes total

III

6L.ines

TYMNET
Interface

Figure
1:
Figure
1:

f>. .A. Feigenballm

Current SUMEX-AIM
SUME?(-AIM Decsystem
Decsystem 2050
2060 Computer
Computer Configuration
Configuration
Current

DEC
DEC LPlO
LP10
Line
Une Printer
Printer

Tape Drives
and Controller

DEC BAlD

Console TTY

20ECTU·78
2 DEC TU-78

l/O
bus
1I0bua

Disk ContrOller
~

fo-'

11/40FROUTEND
111.-0 FRONT END

RR00785-11
5P41 RR00785-11

Details
Details of
of Technical
Technical Progress
Progress

Central
Central Processor
Processor
DEC
KS·10
DECKS-10
512K
512K words
words Memory
Memory
LA-36
LA-36 Console
Console

Unibus
Unibus Adapter
Adapter

UNIBUS
UNIBUS

Massbus
Massbus Adapter
Adapter

Massbus Adapter

Ethemet
UNIBUS
Interface

I
Line Scanner
DEC OZ·"
f-16 Lines

MassBus

I
Disk
Disk Controller
Controller
and
and Drive
Drive
DEC
DEC RP·06
RP-06

Figure
2:
Figure
2:

UNli3US
UNIBUS

MassBus

r-l

Tape
Tape Controller
Controller
and
and Drive
Drive
DECTU....s
DEC TlJ-45

Current
Current SUMEX-AIM
SUMEX-AIM 2020 Computer
Computer Configuration
Configuration

11
11

E. A. Feigenbaum
Feigenbaum

RR00785- 11
5P41 RR00785-11

Details
Details of
of Technical
Technical Progress
Progress

Central
Central Processor
Processor
DEC
DEC VAX
VAX 11nOO
11 f70O
4 Mbytes
Mbytes Memory
Memory
Floating
Floating Pt
Pt Unit
Unit
LA·36
LA-36 Console
Console

MassBus
MSSBUS

Disk Controller
and Drive
DEC RP·06

Line Scanner
DEC DZ·"
16 Lines

Disk
Disk Controller
Controller
and
and Drive
Drive
DEC
DEC RP·07
RP-07

Ethernet
UNIBUS
Interface

Tape Controller
and Drive
DEC TU·n

Figure
3:
Figure
3:

E. .r\.
A. P('ig('nbaum
Feigenbsum

UNIBUS
UNIBUS

I-

Chaosnet
Chaosnet
UNIBUS
UNIBUS
Interface
Interface

Current
AX Comput,er
Current Shared
Shared V
VAX
Comput,er Configuration
Configuration

DeWIs
Details of
or Technical
Technical Progress
Progress

SP41
RR0078S-11
SF’
41 RR0078.5-11

Margaret Jacks
Jacks
Margaret
Heuristic Prog
Prog Prof
PrOI
Heunstic

Electrical
Electrical
Engineering
Engmeenng

Pine Hall
Hall
Pine
CSLI

10 Mb.t ethernet
I

SCORE 2060
2060
SCORE
Ether TIP
TIP
Ether
Dover Pnnter
Printer
Dover
Other CSD
CSD Eoulp
Equip
Other

.
I

Sumex VAX
CSLl2060
Oandeltons
Dandelions
NS
NS file
file server
server

Oncocin
OnCOCin Offices
Offices

piiiJq

SUMEX Offices
Offices
SUMEX

Medical
Center
Medical Center
SUMEX Mach
SUMEX
Mach Rm
Rm

Xerox Alto
Alto
Xerox
Dolph.n
Dolphm
Dandelions
Dandelions
SUN.
Devel
SUN· Devel
Ether TIP
Ether
TIP
Imprmt-10
Printer
Imprint·10
Printer

2060
2060
2020
2020
VAX·
VAX. File
File Server
Server
VAX·
VAX. Devel
Devel
Ether
Ether TIP
TIP
PDP·11
POP-1 1 . Devel
Oevel

Repeater
Repeater

~
M Gateway
Gateway

Ether TIP
Dandelions
HP 9836's

10
ethernet
10 Mbit
Mbit ethernet

Whelan Building
(Welch Road)
Whelan
Buflding (Welch
Road)
Heuristic
Heuristic Prog
Prog Proj
Proi
1.5
1.5 Mbit
Mbit
phone
phone line
line

10
10 Mbit
Mbit ethernet
ethernet
/I

'-====l~~ I

11
I

Dandelions
Dandelfons
LM·3600's
LM-3600’S
Ether
Ether TIP
TIP
Dolphins
Dorado
Dorado
!=laven Printer

3 Mb.t
Mbtt ethernet
ethernet
3

Figure
4:
Figure
4:

SUMEX-AIM
SUhlES-AIhl Ethernet
Ethernet Configuration
Configuration

13
13

E. .\. Feigcnhnrim

Det3
Dehilsils of
of Technical
Technical Progress
Progress

5P41
5P41 RR00785-11
RR00785-11

Central
Central Processor
Procesor
DEC
DEC VAX
VAX llnSO
I t n!x
22 Mbytes
Mbytes Memory
Memory

MassBus
MassBus 1

TUSS

UNIBUS
UNIBUS

Kenne<1y
600/1600 bpI

CDC 256 Mbyte
Removable·
MedIa
Disk Drive

Fujitsu Eagle
414 Mbyte
Disk Drive

It
System
System Industries
Industries
9900
8900 Disk
Disk Controller
Controller

Fujitsu Eagle
Eagle
Fuji&u
414 Mbyte
Mbyte
414
Disk Drove
Drive
Disk

Fujitsu Eagle
Eagle
Fujitsu
414 Mbyre
Mbyte
414
Disk Drive
Drive
Disk

TIlDe Drove
& Emulex
Controller

Disk Controller
and Drive
DEC RK07

Ethernet
UNIBUS
Interface

OZ·11
I--

Figure 5:5:
Figure

E. ;\.

Ft'i~l'nbd'lrn

Line Scanner
BLines

SUMEX-AIM File
File Server
SerVt'r {SAFE}
{SAFE}
suhim-AI?.1

14
14

Console TTY

Details
Details of Technical
Technical Progress

5P41
5P41 RROOi85-11
RR00785-11

Central Processor
DEC VAX 111750
2 Mbytes Memory

Console TTY

TU 58

UNIBUS
UNIBUS

Disk Controller
and Drive
DEC RK07

Ethernet
UNIBUS
Interface

Disk Controller
and Drive
DEC RK07

DEC OZ-11
Line Scanner
8 Lines

Figure 6:
6:
Figure

SUMEX-AIM Development
Development Vax
Vax {ARDVAX}
SUMEX-AIM
{ARDVAX}

A. Feigenbsum
Feigenbaum
E. A.

Progress
Progress -- Timesharing
Timesharing Systems
Systems

5P41
5P41 RR00785-11
RR00785- 11

I.A.2.3.
Timesharing
Systems
I.A.2.3.
Timesharing
Systems
Continued
Continued support
support and
and development
development ot
of our
our timesharing
timesharing systems
systems this
this past
past year
year has
has
concentrated
concentrated on
on several
several areas,
areas, including
including improvement
improvement of
of user
user services
services such
such as
as printing
printing
spoolers
spoolers and
and archiving
archiving support,
support, implementation
implementation of
of features
features from
from our
our KilO
KIlO Tenex
Tenex system,
system,
enhancing
interface
network
interface service,
service, correcting
correcting encountered
encountered system
system bugs, and
and
enhancing network
implementing
In
implementing new features
features for
for better
better user community
community support.
support.
In addition,
addition, we have
invested
system
invested further
further effort
effort in
in supporting
supporting the VAX/UNIX
VAX/UNIX
system in
in conjunction
conjunction with
with the
SUMEX-AIM
SUMEX-AIM file
file server
server installation.
installation.

DECsystem
2060/I'OPS-20
System
DECsystem
206O/TOPS-20
System
Support
Support of our
our main
main timesharing
timesharing machine,
machine, the DECsystem
DECsystem 2060,
2060, has continued
continued
during
during grant
grant year
year II.
11.

•l Hardware
development
Hardware
development
I.
1. The
The DECsystem
DECsystem 2060 system
system now in operation
operation at SUMEX
SUMEX differs
differs greatly
greatly
from
from our
our previous
previous KilO
KIlO Tenex
Tenex system.
system. Whereas
Whereas before, the KilO
KIlO system
system
and TENEX
TENEX software
software required
required much
much in-house development
development and support,
support,
life is easier with
with the 2060. Being
Being at Stanford
Stanford University,
University, where
where there
there
are at least 7 other
DECsystem
20's
with
similar
hardware
and
software
other DECsystem 20’s with similar hardware
software
is a great
great advantage.
advantage. We
We are able to
to share our
our experiences with
with other
other
sites, and have become an integral
integral part
part of
of the Stanford
Stanford DEC
DEC community.
community.
In addition,
addition, the DEC2060
DEC2060 hardware
hardware has been more
more reliable
reliable and easier to
maintain
maintain than
than the
the KilO
KIlO system.
system.
2. Additional
provide support
Additional modems
modems were added to
to the
the 2060 to
to provide
support for
for BELL
BELL
212A
baud protocols.
protocols. This
baud
212A 1200 baud
This adds an alternative
alternative to Vadic
Vadic 1200 baud
service.
use the Bell
service. Modems
Modems which
which use
Bell 212A
212A standard
standard are more
more widely
widely
available
available for
for much
much less
less cost than
than Vadic
Vadic modems
modems..

•a TOPS-20
TOPS-20 Monitor
AIonitor Software
Software Enhancements
Enhancements
1. A
A significant
significant enhancement
enhancement to
to our
our TOPS-20
TOPS-20 monitor
monitor occurred
occurred this
this year
year

when we implemented
implemented the
the software
software from
from our
our Tenex
Tenex system
system which
which allowed
allowed
when
extended support
support for
for the
the ‘'?'
feature of
of TOPS-20
TOPS-20 when
when parsing
parsing filenames.
filenames.
extended
?’ feature
This feature
feature allows
allows a user at
at any
any time
time to
to get
get a list
list of
of possible choices
This
when needing
needing to
to input
input a file
file name by
by just
just typing
typing ‘'?'.
This returns
returns an
?‘. This
when
actual list
list of
of file
file names, whereas
whereas in the
the standard
standard TOPS-20
TOPS-20 monitor,
monitor, just
just
actual
the st.ring
string ‘'input
filespec' was returned.
returned. This
This is a very
very significant
significant and
and
the
input filespec’
useful change to
to our
our TOPS-20
TOPS-20 system.
system.
useful
We continued
continued to
to keep up-to-date
up-to-date on
on the
the various
various bug
bug fixes
fixes and
and monitor
monitor
2. We
improvements that
that we received
received from
from DEC
DEC and
and other
other TOPS-20
TOPS-20 sites.
improvements
These included
included several
several fixes
fixes and
and rewrites
rewrites to
to the
the Internet
Internet IP/TCP
IP /TCP code
These
which went
went under
under a major
major revision
revision this
this past
past year.
year.
which
\Ve installed
installed the
the capability
capability for
for users to
to access
access their
their subdirectories
subdirectories as ifif
3. \Ye
they were
were the
the owners
owners of
of such.
such. While
While this
this may
may seem to
to be the
the logical
logical way
way
they
to implement
implement subdirectories
subdirectories to
to begin
begin with,
with, DEC’
DEC's
models of
of
to
s models
subdirectories was
was a bit
bit different.
different. Our
Our changes have
have since been installed
installed
subdirectories
on other
other DEC20’
DEC20's
at Stanford
Stanford and
and elsewhere.
elsewhere.
on
s at
4. We
'We inst,slled
installed the
the capability
capability to
to vary
vary the
the allocation
allocation of
of windfall
windfall cycles
cycles in
in
-1.

E. ,4.
A. Feigenbaum
Feigenbaum
E.

Progress - Timesharing
Timesharing Systems
Systems
Progress

5P41 RR00785-11
RR00785-11

accordance
accordance with
with user classes.
classes. This
This will
will allow
allow us more
more flexibility
flexibility in
assigning
jobs enough cycles to
assigning jobs
to run
run comfortably
comfortably while
while limiting
limiting their
their
usage
usage to a strict
strict percentage
percentage of
of the
the machine.
machine.
5. vVe
We installed
installed several
several new features
features in our
our TOPS-20
TOPS-20 EXEC
EXEC to
to facilitate
facilitate the
use
use of
of the system
system by
by the users. Among
Among these features
features was the ability
ability to
to
edit
edit any
any previous
previous command
commai.nd you
you had entered,
entered, and then
then re-execute
re-execute that
that
command.
command. This
This saves extra
extra keystrokes
keystrokes and has proven
proven to
to be very
very useful.
useful.
The
The code to
to do this
this came from
from the University
University of
of Texas
Texas at Austin.
Austin.
6. We
We switched
switched our
our system
system this
this year
year to
to using
using encrypted
encrypted passwords.
passwords. This
This
means that
that passwords
passwords are not
not stored
stored in any
any readable
readable form
form on t.he
the
computer
computer system,
system, and if
if an illegal
illegal user should
should gain access
access to
to the system,
system,
he/she
he/she would
would not
not be able to
to find
find out
out the passwords
passwords of
of any
any other
other users.
We
We feel this
this feature
feature is quite
quite important
important as the frequency
frequency of
of computer
computer
break-ins/attempts
break-ins/attempts increases.
7. Software
Software was added to
to our
our monitor
monitor in order
order to
to record
record the last
last reader
reader of
of a
file. Previously,
date of
of the last
last read was recorded,
recorded, while
while both
both
Previously, only
only the date
the writer
writer and date
date were recorded
recorded for
for creating
creating and writing
writing a file. This
This
gives users the ability
ability to
to determine
determine which
which other
other persons may
may have been
read
ing files
reading
files..

•a Printer
Printer Support
Support
The
The support
support on the 2060 for
for various
various printers
printers in the SUMEX
SUMEX community
community has
been greatly
greatly enhanced this
this past
past year.
year.
1. Support
Support was added for
for the Xerox
Xerox Raven
Raven printer
printer at the
the Welch
Welch Road
Road

facility
facility to allow
allow spooling
spooling and direct
direct output
output to
to the printer.
printer. In
In addition,
addition,
code was added to
to the spooler
spooler to print
print out
out a header
header page identifying
identifying the
user, filename,
job.
filename, and date
date for
for each job.
2. Similar
Similar spooler
spooler support
support was added for
for the
the Xerox
Xerox Dover
Dover printer
printer in
Margaret
hlargsret Jacks Hall.
Hall.
3. SUMEX
SUMEX installed
installed a Printronix
Printronix line printer
printer at
at Welch
Welch Road to allow
allow users
print out
out files remotely
remotely from
from the 2060. The
The Printronix
Printronix is connect.ed
connected to
to print
SUMEX
SUh?EX via
via a twisted
twisted pair
pair serial
serial line.

We transformed
transformed TENEX
TENEX software
software to the normal
normal TOPS-20
TOPS-20 line printer
printer
4. \Ve
spooler
spooler program
program to look
look out
out for
for users who
who had accidentally
accidentally printed
printed an
'unprintable
‘unprintable file',
file’, meaning
meaning a binary
binary file
file of
of some sort
sort which
which does not
not
cont:1in
contain legible
legible characters.
characters. We
We do this
this both
both by
by counting
counting the number
number of
of
binary
binary characters
characters in the first
first page of
of the file, and by
by not
not printing
printsing the file
if t.he
the count
count exceeds
exceeds a certain
certain threshold.
threshold. A
A similiar
similiar scheme is also used
taking
taking int.o
into account
account the vertical
vertical motion
motion of
of the first
first page.
5. Additional
Additional modifications
modifications were made to the LPlO
LPlO line printer
printer driver
driver
software in the TOPS-20
TOPS-20 monitor
monitor to
to improve
improve the reliability
reliability of
of using
using this
this
software
line printer,
printer, which
which came from
from our
our KilO
KIlO system.
system.
6. \Ve
We greatly
greatly enhanced our
our support
support of
of the IMAGEN
IMAGEN Imprint-lO
Imprint-10 laser
printa
/TCP Ethernet
printer this
this past
past year.
year. A
A new IP
IP/TCP
Ethernet interface
interface was installed
installed on
the prin
ter (discussed further
printer
further in Section
Section LA.2.4
I.A.2.4 replacing
replacing the existing
existing
serial
serial interface.
int,erface. This
This new interface
interface allows
allows for
for more
more efficient
efficient printer
prinber

Feigenbaum
E. A. Feigenbaum

Timesharing Systems
Progress - Timesharing

5P41 RR00785-11

operation, and greater
greater flexibility
flexibility in choosing output
output modes, number
number of
operation,
pages, and other
other features.
features. We have also implemented
implemented aa
copies, header pages,
TOPS-20 spooler for the Imagen
Imagen as
as well.
well.
TOPS-20
•l User
User System
System Software
Software

1. We have continued
continued to assemble and maintain
maintain a broad
broad range of utilities
utilities
1.
support software
software on the 2060. These include
and user support
include operational
operational aids,
statistical packages, DEC-supplied
DEC-supplied programs,
programs, text
text editors,
editors, text
text search
statistical
programs, file space
space management
management programs,
programs, graphics
graphics support,
support, texttextprograms,
formatting and justification
justification assistance, magnetic
magnetic tape conversion
conversion aids,
formatting
many more. We also are importing
importing software
soft,ware tools and packages
and many
wherever necessary to avoid
avoid reinventing
reinventing the wheel and wasting
wasting our
our own
own
wherever
efforts. Packages have been imported
imported from
from Texas Instruments,
Instruments, Columbia
Columbia
efforts.
University, the University
University of Texas at Austin,
Austin, Yale University,
University, and other
other
University,
Stanford sites.
Stanford
SUMEX has continued
continued to provide
provide to
to its users the latest
latest releases
releases of
2. SUMEX
various Lisp
Lisp dialects
dialects that
that run
run under
under TOPS-20.
TOPS-20. This
This past
past year
year we agreed
various
provide disk
disk space to store
store the 'official'
‘official’ version
version of
of Interlisp-20
Interlispfrom
to provide
from
XEROX due to the fact
fact that
that the machine
machine at Xerox
Xerox used to support
support
XEROX
Interlisp before
before was being removed.
removed. Interlisp-lO
Interlisp
is now officially
Interlispofficially
maintained at SUMEX
SUMEX by our
our staff
staff and XEROX
XEROX personnel.
personnel. In addition,
addition,
maintained
continue to support
support the full
full variet.y
variety of
of LISPUSERS
LISPUSERS packages.
we continue
Portable Standard
Standard Lisp
Lisp (PSL)
(PSL) developed at the University
University of Utah
Utah has
Portable
a.lso been installed
installed on SUMEX.
SUMEX.
also
We continue
continue to use
use MM,
MM, a very
very powerful
flexible mail
mail system,
system, on the
3. We
powerful and flexible
2060.
Electronic bulletin
bulletin boards
boards are also extensively-supported
extensively-supported
at
2050.
Electronic
at
SUMEX. These provide
rather informal
informal mechanism
mechanism for
for community
community
SUMEX.
provide a rather
Other bulletin
bulletin boards,
boards, read and contributed
contributed to
to
discussions and debates. Other
throughout the INTERNET
INTERNET
community, are available
throughout
community,
available for
for perusal
perusal at
SUMEX. These bulletin
bulletin boards
boards cover
cover such topics
topics as AI
AI Discussions,
Discussions, Micro
Micro
SUMEX.
Computers, Terminals,
Terminals, and \Vorkstations.
Workstations.
Computers,
SIJhIEX participates
part,icipates with
with other
other Stanford
Stanford sites in a general
general license for
for
4. SUMEX
access to
to the SCRIBE
SCRIBE text-formatting
text-formatting system
system from
from UNILOGIC,
UNILOGIC, including
including
access
versions
too run
run under
under TENEX,
TENEX, TOPS-20,
TOPS-20, and UNIX.
UNIX. SCRIBE
SCRIBE is the
the
versions t’
preferred
preferred tool
tool for
for text
text preparation
preparation at
at SUMEX.
SUMEX.
5.
S. Versions
Versions of
of various
various user utilities
utilities and system
system utilities
utilities were updated
updated
throughout
throughout the
the year.
year. These programs
programs included
included network
network server
server processes,
processes,
statistical
stat istical packages,
packages. system
system daemon
daemon programs,
programs, and several
several programs
programs for
for
processing
processing electronic
electronic mail.
mail.
6.
5. Various
Various system
system network
network tables
tables and networking
networking softwltre
softwa.re were updated
updated to
to
sccommoda,te
accommodate the ARPANET
ARPANET split
split that
that occurred
occurred this
this year.
yea.r. The
The network
network
change effectively
effectively split
split the
the ARPANET
ARPANET into
into two
two networks,
networks, the
the MILNET,
MILNET,
which
which is a secure private
private part,
part, and the
the rest
rest of
of the
the ARPANET,
ARPANET, which
which
operates
operates as the
the original
original ARPANET
ARPANET did
did before.
before .
• Docermentc?tion
Documentation and
and Education
Education

\Ve
\Vt> have expended
expended considerable
considerable effort
effort to
to develop,
develop, maintain,
maintain, a.nd
and facilitate
facilitate
access
access to
to our
our documentation
documentation so to
to accurately
accurately reflect
reflect available
available software.
software. The
The

Feigenbaum
E. A. Feigenbaum

Progress
Progress - Timesharing
Timesharing Systems
Systems

5P41 RR00785-11
RR00785-11

HELP
IHELP and Bulletin
Bulletin Board
Board subsystems
subsystems have
hazTebeen important
important in this
this effort.
effort. As
As
subsystems
publish a bulletin
bulletin or
subsystems are updated,
updated, we generally
generally publish
or small
small document
document
describing
more and more
more changes occur,
occur, it
it becomes more
more
describing the changes. As more
difficult
pointers. Within
difficult, for
for users to
to track
track down
down all of
of the
the change pointers.
Within manpower
manpower
limits,
limits, we are in a continual
continual process of
of reviewing
reviewing the existing
existing documentation
documentation
system
programs now
system for
for compatibility
compabibility with
with the programs
now on line and to integrate
integrate
cha.nges
This also will
will be done with
with a view
view toward
toward
changes into
into the main
main documents.
documents. This
developing
better tools
developing better
tools for
for maintaining
maintaining up-ta-date
up-to-date documentation
documentation, .

•l Software
Sharing
Software
Sharing
stated
previously, we firmly
firmly believe in importing
importing rather
rather than
than
stated previously,
reinventing
reinventing software
software where
where possible. As
As noted
noted above, a number
number of
of the
the
brought up are from
packages we have brought
from outside
outside groups.
groups. Many
Many a.venues
avenues
between the system
projects, other
exist
exist for
for sharing
sharing between
system staff,
staff, various
various user projects,
other
facilities,
and vendors.
The
facilities,
vendors.
The advent
advent of
of fast and
a.nd convenient
convenient
communication
communica.tion facilities
facilities coupling
coupling communities
communities of
of computer
computer facilities
facilities ha.s
has
ma.de
cooperation
made possible effective
effective intergroup
intergroup
cooperation and decentralized
decent,ralized
maintenance
maintenance of
of software
software packages.

1. As

2. The
The TENEX,
TENEX, TOPS-20,
TOPS-20, and UNIX
UNIX sites on the ARPANET
ARPANET have been a
good model
model for
for this
this kind
kind of
of exchange based on a functional
functional division
division of
of
labor
by-product
labor and expertise.
expertise. The
The other
other major
major advantage
advantage is that
that as a by-product
of
particular software,
personal
of the constant
cons.tant communication
communicat.ion about
about particular
soft.ware, personal
relat.ionships
between staff
relationships between
staff members
members of
of the various
various sites develop.
develop. These
collegial
collegial interactions
interactions serve to
to pass general
general information
information about
about software
software
tools
tools and to encourage
encourage the
the exchange of
of ideas among
among the sites. Certain
Certain
common
problems are now
common problems
now regularly
regularly discussed on a mUlti-site
multi-sit,e level.
3. vVe
We continue
continue to
to draw
draw significant
significant amounts
amounts of
of system
system software
software from
from other
other
ARP
ANET sites, reciprocating
ARPANET
developments.
reciprocating with
with our
our own
own local developments.
Interactions
backup support,
Interactions have included
included mutual
mutual backup
support, experience
experience with
with
various
various hardware
hardware configurations,
configurations, experience
experience with
with new types
types of
of computers
computers
and operating
operating systems,
systems, designs for
for local networks,
networks, operating
operating system
system
enhancements,
utility
or
language
software,
and
user project
project
enhancements,
software,
utility
or
collaborations.
We
collaborations.
We have been able to
to import
import many
many new pieces of
of
software
soft.wsre and improvements
improvements to
to existing
existing ones in this
this way.
way. Examples
Examples of
of
imported
program MM,
imported software
software include
include the message
message manipulation
ma.nipulat,ion program
Mhf, SAIL,
SAIL,
PASCAL,
A.X Ethernet
PASCAL, SOS, INTERLISP,
INTERLISP, the C compiler,
compiler, V
VAX
Ethernet code, the
PHOTO
program, ARPANET
host
PHOTO program,
ARPANET
host tables,
tables, various
various user utilities,
utilities, and
many
many others.
others.
4. Fin:.dly,
Finally, we also have assisted groups
groups that
that have interacted
interacted with
with SUMEX
SUMEX
user projects
projects in acquiring
acquiring access
access to
to software
software available
available in our
our community.
community.
\Ve
providing tape
preparation and copy
\Ve are repeatedly
repeatedly providing
tape preparation
copy service to
to many
many
SU~[EX-AIM
projects to
SUME.X-AIM projects
to aid in sharing
sharing their
their software
software with
with outside
outside
requestors.
requestors.

E. A.
A. Feigenbaum
Feigenbaum

5P41
5P 41 RR00785RR00785- 11

Progress - Timesharing
Timesharing Systems
Systems
Progress

DECsystem ZOL?O/TOPS-20
2020/I'OPS-20 System
System
DECsystem
Monitor Upgrade
Upgrade --- Our
Our 2020 system
system has continued
continued to
to run
run very
very reliably
reliably this
this
1. Monit.or
past year.
year. We
We have
have updated
updated the
the 2020 monitor
monitor with
with bug
bug fixes
fixes and
and performance
performance
past
improvements regularly.
regularly. There
There will
will likely
likely be few further
further monitor
monitor releases for
for
improvements
the 2020 since itit does not
not support
support extended
extended addressing
addressing and
and there
there are no plans
plans
the
to add
add this
this feature.
feature.
to
Demo Cont,rols
Controls --- We
\Ve continue
continue to
to use the
the 2020 system
system for
for demos of
of AI
AI systems
systems
2. Demo
developed at
at SUMEX.
SUMEX. This
This demo
demo system
system takes
takes advantage
advantage of
of the
the “pie-slice”
"pie-slice"
developed
scheduler in the
the TOPS-20
TOPS-20 release 4 monitor.
monitor. We
vVe now
now guarantee
guarantee dedicated
dedicated
scheduler
large fraction
fraction of
of the
the machine
machine but
but also allow
allow others
others to
to do useful
useful work
work
users a large
when the
the demo
demo demand
demand is low.
low. This
This system
system has nicely
nicely met
met the
the needs of
of both
both
when
groups.
groups.

v:4-YjUNIX Systems
We
11/780 system
\Ve continued
continued to
to provide
provide systems
systems support
support for
for the
the VAX/UNIX
VAX/UNIX 11/780
system
(named ‘'AIMVAX')
shared by
by the
the SUMEX-AIM
SUMEX-AIM community
community and Stanford
Computer
(named
AIMVAX’)
shared
Stanford Computer
Department. Various
Various efforts
efforts included
included supporting
supporting the
the UNIX
UNIX monitor,
monitor, installing
installing
Science Department.
network software,
software, and in bringing
bringing up various
various user subsystems.
subsystems,
new network
Further
Further development
development has continued
continued in support
support of
of the
the SUMEX-AIM
SUMEX-AIM File
File Server
Server
(SAFE) based on a VAX
V A..'C 11/750.
vVe successfully
successfully converted
converted SAFE
SAFE to
to Berkeley
Berkeley Unix
Unix 4.2
(SAFE)
llj7.50. We
server.
server, and with
with the help of
of the
the Computer
Computer Science Department,
Department, converted
converted the
the Ethernet
Ethernet
Pup software
software to
to run
run under
under UNIX
UNIX 4.2
Pup

E. A. Feigenbaum
Feigenbaum

Progress
Progress - Professional
Professional Workstations
Workstations

SP41
FjP41 RR0078S-11
RR00785-1 I

I.A.2.4.
Professional
Workstations
I.A.2.4.
Professional
Workstations
Our
Our ongoing
ongoing movement
movement to professional
professional workstations
workstations is taking
taking on several forms.
forms. We
We
continue
continue to carry
carry out
out our
our acquisition
acquisition plans
plans for
for acquiring
acquiring Lisp
Lisp machines
machines for
for use
use by
by the
SUMEX
SUhlEX community,
community, as well
well as investigating
investigating the
the use
use of
of remote
remote virtual
virtual graphics
graphics and new
This
lower
This work
work is prototypical
prototypical of
of what
what other
other
lower cost workstations
workstations in our
our environment.
environment.
groups
groups will
will face and we hope will
will serve to
to find
find effective
effective solutions
solutions to
to common
common problems.
problems.

Lisp-based Scientific
Workstations
Lisp-based
Scientific
Workstations
SUMEX
SUMEX carefully
carefully developed
developed and implemented
implemented our
our equipment
equipment acquisition
acquisition plan
plan for
for
year
year eleven by
by buying
buying seven Xerox
Xerox 1108 Lisp
Lisp machines
machines for
for use by
by SUMEX-AIM
SUMEX-AIM projects.
projects.
Two
purchased with
provide floating
Two of
of these machines
ma.chines were purchased
with special upgrade
upgrade packages to provide
floating
point. capability,
capability, expanded
expanded microcode,
microcode, and expanded
expanded memory.
memory. Our
Our experiences with
with these
machims
machines will
will be reported
reported in next
next years
years report.
report.
The
The XEnOX
XEROX Dolphin
Dolphin on loan to
to Rutgers
Rut*gers University
University was returned
returned to
to SUMEX
SUMEX this
this
year.
year. This
This Dolphin
Dolphin had effectively
effectively served the Rutgers-AIM
Rutgers-AIM community
community in setting
setting up their
their
Ethernet
provided initial
Now
Ethernet network
network and provided
initial exposure to
to the Lisp
Lisp machine
machine technology.
technology. Now
that
that that
that experiment
experiment is successfully
successfully completed,
completed, the Dolphins
Dolphins will
will be used for
for AI
AI system
system
development
development at Stanford.
Stanford.
SU~fEX
SUMEX installed
installed two
two SYMBOLICS
SYMBOLICS 3600 Lisp
Lisp machines,
machines, purchased
purchased with
with DARPA
DARPA
funding,
funding, for
for use
use within
within the Heuristic
Heuristic Programming
Programming Project
Project (HPP)
(HPP) at
at their
their new location
location at
Welch
Welch Road.
Road. \Ve
We are currently
currently awaiting
awaiting a new release of
of the Symbolics
Symbolics Operating
Operating System
System
software
before
we
can
provide
Ethernet
access
to
our
file
server
from
software before
provide Ethernet access to our file server from these machines.
machines.
The
The 3600'5
3600’s are used regularly
regularly by
by members
members of
of the HPP.
HPP.

We
We st.ill
st,ill are using 4 preproduction
preproduction models of
of the Dolphin
Dolphin workstations.
workstations. One
preproduct.ion
preproductzion model
model has been exchanged for
for a production
production system,
system, and we are on schedule
with
with XEROX
XEROX to exchange the remaining
remaining 4 machines
machines for
for production
production models at no extra
extra
cost.
cost,. This
This process is hampered
hampered by
by the rate
rate at which
which XEROX
XEROX themselves
themselves can get
prod
llction machines.
product,ion
machines.
\Ve studied
studied the benefits
benefits of
of buying
buying the extended
extended memory
memory and microstore
microstore upgrades
upgrades
to the Xerox
as
Xerox 1108 Dandelion
Dandelion announced
announced at AAAI-83
MI-83
as being
being "under
“under development."
development.” \Ve
We
concluded
concluded that
that some users would
would benefit
benefit greatly
greatly from
from these enhancements
enhancements and others
others not
not
a.t
a.t all. The
The most
most marked
marked improvements
improvements came from
from system
system which
which were extremely
extremely memory
memory
limit.ed.
NEOM)"CIN. SUMEX
limited, such as
as NEOMYCIN.
SUMEX will
will be acquiring
acquiring two
two 1108's
1108’s with
with the upgrades
upgrades for
for
full
full t.ime
time use
use and testing.
testing.
A close relationship
relationship between
between SUMEX
SUMEX and the newly-formed
newly-formed Center
Center for
for the Study
Study of
of
Langnage
Ln.ngusge and Information
Information (CSLI)
(CSLI) at Stanford
Stanford was established.
established. This
This has already
already benefit.('d
benefitsed
SU~1EX
SLJhlEX (and
(and the ONCOCIN
ONCOCIN project
project in particular)
particular) in the loan, by
by CSLI
CSLI to
to SUMEX,
SUMEX, of
of
two
two Xerox
Xerox 1108's
1108’s which
which have been in constant
constant use
use by
by researchers
researchers since January
January 12th.
12th. The
The
SUI,1EX
SUhlEX staff
staff assisted CSLI
CSLI in bringing
bringing their
their DecSystem20
DecSystem20 and network
network environment
environment online.
line. CSLI
CSLI has informally
informally expressed an interest
interest in working
working on the problem
problem of
of distributed
distributed
AI
AI computation
computation with
with SUMEX
SUMEX researchers.
researchers. CSLI
CSLI will
will have 110 1108's
1108’s on the Ethernet
Ethernet
wit.hin
within the ye:1r.
year. This
This resource suggests some exciting
exciting solutions
solutions to former
former compute-bound
compute-bound
problems.
problems. The
The ONCOCIN
ONCOCIN group
group has already
already implemented
implemented a preliminary
preliminary network-based
network-based
Interactor
to run
Int,eractor which
which permits
permits elements
elements of
of ONCOCIN
ONCOCIN
run concurrently
concurrently on different
different
machines.
of this
this writing,
writing, the Reasoner and its
its Debugger
Debugger have been made to
to run
run
ma.chines. As of
transparently
transparently in this
this mode, and to make good use of
of both
both processors.

21
21

E. A.
A. Feigenbaum
Feigenbaum

5P41 RR00785-11
RR00785- 11

Progress - Professional
Professional Workstations
Workstations

Virtual
Virtual Graphics
Graphic3
SU!'vlEX
SUMEX continued
continued the development
development of aa Virtual
Virtual Graphics
Graphics system
system written
written in
Interlisp-IO
and running
Interlisprunning on our 2060.
2060. Any
Any user running
running the V system
system on aa workstation
workstation
can then
then use
use the
t,he package on the 2060 to drive
drive the graphics
graphics display
display on the workstation.
workstation. A
A
current
application
is
to
take
nuclear
magnetic
resonance
data
on
the
2060
and display
current a.pplication is
nuclear magnetic
data
display
t,he
the atoms
atoms and
a.nd their
their bonds on a SUN
SUN workstation
workstation by using splines. This
This development
development is
is in
its infancy,
infancy, but
but, is
is opened ended and has great
great potential
potential with
with the price
price of workstations
workstations
capable
capa.ble of decent graphics
graphics reaching
reaching the two
two to three
three thousand
thousand dollar
dollar range. It
It allows
allows those
users who
who cannot
cannot afford
afford expensive lisp machines
machines to have full
full graphics
graphics power
power available
available to
them
them by doing
doing the actual
actual graphics
graphics applications
applications on a large time
time shared system,
system, and then
then
doing
doing t.he
the graphics
graphics itself
itself remotely
remotely on a less
less expensive workstation.
workstation. This
This development
development can
help users take
ta.ke advantage
advantage of the computing
computing power
power of the DECsystem
DECsystem 20, while
while providing
providing
many
many of the
t,he high
high speed
speed graphics
graphics advantages
advantages of the Lisp Machines.
Machines.

Apl)/e
Development
.4pple Workstation
Workstation
Development
SUMEX-AIM
SUMEX-AIM has initiated
initiated a development
development project
project to pursue
pursue the effectiveness
effectiveness and
possible use
of
low
cost
personal
workstations
within
our
environment.
use
personal workstations within our environment. After
After examing
examing a
number
number of new personal
personal computers
computers and workstations
workstations on the market,
market, such as
as the Hewlett
Hewlett
Packard
Packard 150.
150, IBM
IBM PC, Sun workstations,
workstations, and others,
others, we chose
chose the Apple
Apple Macintosh
Macintosh and
Apple
Apple Lisa on which
which to begin our
our work
work in this
this area. These machines
machines were chosen
technically
t*echnically due to their
their built
built in graphics,
graphics, networking,
networking, mouse, windows,
windows, and menu support.
support.
vVe
We also considered
considered the very
very beneficial
beneficial relationship
relationship formed
formed between
between Apple
Apple and Stanford
Stanford
University
which provides
provides us direct
direct access
access to Macintosh
Macintosh hardware
hardware and software
software
University which
documentation
plan to
documentation which
which is a necessity
necessity for
for the type
type of
of work
work we plan
to do.
Our
Our Macintosh
Macintosh development
development encompasses several areas ;
1. INFO-MAC
INFO-AIAC Discussion
Di.'3cu8sion List
List

An
An electronic
electronic discussion
discussion list
list was originated
originated at SUMEX,
SUMEX, and is currently
currently
maintained
maintained here, to
to foster
foster sharing
sharing and communication
communication among
among research groups
groups
and
pursuing the serious use
a.nd universities
universities that
that are interested
interested is pursuing
use of
of the Macintosh
Macintosh
within
within their
their respective
respective environments.
environments. This
This list
list has been highly
highly successful in
collaborating
collaborating on Macintosh
Macintosh development
development and the sharing
sharing of
of ideas. The
The
discussion
over 50 sites, and well
well over
over 1000 participants.
participants.
discussion list
list currently
currently contains
contains over

C Development
Development Environment
Environment
2. C
A vital
vital link
link in our
our development
development of
of Macintosh
Macintosh software
software is creating
creating a C based
A
development system
system on our
our VAX
VAX computers
computers for
for the
the coding
coding and downloading
downloading of
of
development
software. Utilizing
Utilizing existing
existing MC68000
MC68000 C cross compilers
compilers on our
our VAX,
VA.,X, we are
software.
developing the
the necessary linkages
linkages in order
order to
to make
make the
the appropriate
appropriate system
system
developing
to rout,ines
rOll tines in the
the Macint.osh
Macintosh ROM’
ROM's
for sophisticated
sophisticated graphics
graphics and system
system
calls to
s for
rela,t,ed functions.
functions.
related
3. Macintosh
Afacintosh print
print servers
servers

In order
order to
to effectively
effectively use the
the Macintosh
Macintosh as a stand-alone
stand-alone workstation,
workstation, we will
will
In
provide
the
ability
to
print
out
Macintosh
developed
files
on
our
IMAGEN
provide the ability to print out Macintosh developed files on our IMAGEN
printers.
laser printers.

Applebu8 to
to Ethernet
Ethernet Interface
Inter face
4. Applebus
This development
development involves
involves the
the hardware
hardware and
and software
software necessary to
to be able to
to
This

A. Feigenbaum
Feigenbaum
E. A.

Progress - Professional
Professional Workstations
Workstations

5P41
5P41 RR00785-11
RROO785-11

access
access various
various file and print
print servers on our
our Ethernet
Ethernet from
from aa Macintosh.
Macintosh.
Macintosh
blscintosh will
will be
be connected to the Apple
Apple network
network called Applebus.
Applebus.
hardware
hardware provides
provides an interrace
interface between
between Applebus
Applebus and 10MB
1OMB Ethernet.
Ethernet.
software
software necessary for this
this project
project involves
involves formulating
formulating Macintosh
Macintosh file
and block
block level I/O
I/O requests into
into properly
properly formatted
formatted Internet
Internet packets.
packets.

The
Our
Our
The
level

5. Virtual
Virtual Graphics
Graphics on
on a Lisa
Lisa

The
The Virtual
Virtual Graphics
Graphics System, as
as previously
previously reported,
reported, is in great
great need
need of a low
cost workstation
workstation on which
which it
it can run.
run. We
We have started
started a project
project to port
port the
Virtual
Virtual Graphics
Graphics system
syst,em to a Apple
Apple Lisa in hopes of providing
providing to our
our users
high speed
will report
report further
further on this
this project
project
speed graphics
graphics at remote
remote locations.
locations. We will
in next
next year's
year’s report.
report.
Anticipating
Ant,icipating the popularity
popularity of our
our Macintosh
Macintosh developments,
developments, we are fully
fully prepared
prepared
to make our
our efforts
efforts available
available to other
other research sites,
sibes, Universities,
Universities, and non-profit
non-profit
institlltions
institutions on a.a royalty-free
royalty-free basis in hopes of fostering
fostering continued
continued development
development and
communal
communal sharing.
sharing.
In addit.ion
addition to Lisp-based
Lisp-based scientific
scientific workstations,
workstations, we believe the use
use of
of low
low cost
workst.at,ions,
workstnt,ions, which
which offer
offer suitable
suitable local processing
processing power,
power, high
high resolution
resolution screens with
with easy
to
t.o use
use user interfaces,
interfaces, and networking
networking and communications
communications abilities,
abilities, are vital
vital to the
future
future of
of our
our resource. Our
Our Macintosh
Macintosh and Lisa
Lisa development
development efforts
efforts will
will allow
allow us to use
use
and experiment
experiment with
w&h these workstations
workstations in our
our environment.
environment.

Hewlett
Hewlett Packard
Packard Development
Development
SUl\fEX
SUMEX assists the ONCOCIN
ONCOCIN Project
Project is developing
developing a computing
computing environment
environment for
for
developing
part
developing AI
AI applications
applications based on HP
HP 9836 workstations.
workstations. These workstations
workstations were part
of
peripherals for
of aa. gift
gift from
from Hewlett
Hewlett Packard
Packard to the Oncocin
Oncocin Project.
Project. Additional
Additional support
support peripherals
for
t.he
the 9835's
9836’s included
included large capacity
capacity disk
disk drives,
drives, color
color monitors,
monitors, graphic
graphic tablets,
tablets, and a laser
printer. \Vork
proceeding to
print.er.
Work is proceeding
to network
network these machines
machines onto
onto the SUMEX
SUMEX Ethernet
Ethernet as
as soon
as
as suitablke
suit.ablke networking
networking hardware
hardware is available
available from
from HP.
HP. These machines
machines will
will be used for
for
projects within
new and
a.nd existing
existing projects
within Oncocin.
Oncocin.

A. Feigenbaum
Feigenbaum
E. A.

RR00785-11
5P41 RR00785-11

Networking and Communications
Net,working
Communications

I.A.2.5.
Networking
and
I.A.2.5.
Networking
and Communications
Communications
A
aspect of
A highly-important
highly-important
of SUMEX-AIM
SUMEX-AIM is effective
effective communication
communication with
with remote
remote
users and between
between the growing
growing number
number of
of machines
machines available
available within
within the SUMEX
SUMEX resource.
In addition
addition t.o
t.o the
the economic
economic arguments
a.rguments for
for terminal
terminal access,
access, networking
networking offers
offers other
other
advantages
including improved
improved inter-user
inter-user communications
communications and
a.dvantages for
for shared
shared computing,
computing, including
more
more effective
effective software
software sharing.
sharing.
Users accessing a remote
remote computer
computer will
will use a hardline
hardline connection
connection to
to the computer
computer
as
as aa. standard
standard of
of comparison.
comparison. Local
Local networks
networks stand
stand up well
well in this
this comparison
comparison but
but remote
remote
network
network facilities
facilities do not.
not. Data
Data loss
loss is not
not a problem
problem in most
most network
network communications;
communications; in
fact,
fact, wit·h
with the
t,he more
more extensive
extensive error
error checking
checking schemes, data
data integrity
integrity is higher
higher than
than for
for a
long distance
distance phone link.
link. On
On the
the other
other hand,
hand, remote
remote networking
networking relies upon
upon shared
shared use
use of
of
communication
communication lines for
for widespread
widespread geographical
geographical coverage at
at substantially
substantially reduced
reduced cost.
However,
However, unless enough
enough total
total line capacity
capacity is provided
provided to meet
meet peak loads, substantial
substantial
queueing
jams result
queueing and traffic
traffic jams
result in the
the loss
loss of
of terminal
terminal responsiveness. We
We continually
continually
monitor
monit,or the load statistics
statistics for
for our
our direct,
direct, dialup,
dialup, and TYMNET
TYMNET lines to
to avert
avert logjam
logjam
situations.
situations.

n-AfNET
TYMNET
TYMNET provides
provides broad
broad geographic
geographic covera.ge
covera,ge for
for terminal
terminal access
access to
to SUMEX
SUMEX from
from
throughout
throughout the country
country and increasingly
increasingly from
from foreign
foreign countries.
countries. With
With the installation
installation of
of
our
our new DEC2050
DEC2060 computer
computer system
system in January
January of
of 1983, we installed
installed new TYMNET
TYMNET
equipment.
After
After the initia.l
initia,l debugging
debugging of
of the new equipment
equipment (called
(called TYMCOM)
TYMCOM) the
equipment.
equipment
However, some months
months after
after the installation
installation it
it was
equipment has been quite
quite reliable.
reliable. However,
discovered
that
the
XON/XOFF
protocol
between
the
Tymcom
and
the
2050
had
discovered t,hat
XON/XOFF
protocol between
Tymcom
2060 had not
not
been properly
The
The number
number of
of user
properly specified
specified in the Tymcom
Tymcom and was corrected.
correct.ed.
complaints
complaints about
about connection
connection problems
problems have been greatly
greatly reduced.
reduced. This
This is thought
thought to
to be
the result
result of
of improved
improved "backbone"
“backbone” lines within
within Tymnet
Tymnet and the installation
installation of
of triple-duty
triple-duty
modems
modems which
which simplify
simplify things
things for
for the users
users..

•-tRPA.NET
\Ve
of
We retain
retain our
our advantageous
advantageous connection
connection to
to the Department
Department
of Defense's
Defense’s
ARPANET.
now
Agency
This
ARPANET.
now ma.naged
ma.naged by
by the Defense Communications
Communications
Agency (DCA).
(DCA).
This
connection
connection has facilitated
facilitated close collaboration
collaboration with
with the Rutgers-AIM
Rutgers-AIM facility
facility and many
many
other
ot.her computer
computer science groups
groups that
that are also on the net. \Ve
We have maintained
maintained good
working
working relationships
relationships with
with other
other sites on the ARPANET
ARPANET for
for system
system backup
backup and software
software
interchange.
day-bo-day working
working interactions
interactions with
with remot.e
remote facilities
facilities would
would not
not be
interchange. Such day-t.o-day
possible without
wit,hout the integrated
integrated file transfer,
transfer, communication,
communication, and terminal-handling
terminal-handling
capabilities
The
capa.hilities nnique
lrnique to the ARPANET.
ARPANET.
The ARPANET
ARPANET is also key
key to
to maintaining
maintaining ongoing
tt' lIectual contact.s
going in
intellectual
c0ntact.s between
between SUMEX
SUMEX projects
projects such as
as the Stanford
Stanford Heuristic
Heuristic
Programming
Progra.m.ming Project
Project authorized
authorized to
to use
use the net
net and other
other active
active AI
AI research groups
groups in the
ARPANET
ARPANET community.
community.
This
This past
past year,
year, SUMEX-AIM
SUMEX-AIM participated
participated in the split
split of
of the ARPANET
ARPANET into
into two
two
networks;
networks: the MILNET,
MILNET, which
which is a highly
highly secure strictly
strictly DOD-related
DOD-related part
part of
of the network,
network,
and the ARPANET,
which
ARPANET,
which is the remainder
remainder of
of the ARPANET
ARPANET sites. This
This latter
latter net
net
functions
NET can only
functions as
as we knew
knew t,he
the ARPANET
ARPANET before.
before. The
The MIL
MILNET
only be accessed
accessed via
via mail
mail
gat.eways.
ga.t.eways. No TELNET
TELI”:ET or
or FTP
FTP to
to MILNET
MILNET sites is allowed.
allowed. In
In addition,
addition, access
access to
to the
ARPANET
AC's
ARP,4NET T
TAC’
s (Terminal
(Terminal Access Controllers)
Controllers) was restricted
restricted this
this past
past year
year to
to only
only
t.hose
t,hose users who
who were granted
granted TAC
TAC access
access cards, which
which meant
meant their
their username
username was
rt'gistered
Network Information
registered with
with the Network
Information Center,
Center, and they
they were given
given a password
password with
with

E. A. Feigenballm
Feigenbslim

5P4l
5P41 RR00785-l1
RRO0785-11

Networking
Networking and Communications
Communications

which
which they
they could dial
dial into
into the ARPANET.
ARPANET. SUMEX
SUMEX arranged
arranged for
lor guest cards for
for those
users
users who needed such access.
access.
\Ve
We continue
continue to be
be called upon to interact
interact with
with outside
outside organizations
organizations which
which are (or
(or
wish to be) connected
connected to our
our IMP.
IMP. The
The line to Advanced
Advanced Information
Information and Decision
Decision
Syst.ems
Systems occasionally
occasionally causes
causes trouble
trouble requiring
requiring diagnosis. The
The intended
intended connection
connection to
Perceptronics
Perceptronics Inc. has evidently
evidently been canceled.

EI'HERNEI'
ETHERNET
A sllbstantial
substantial portion
portion of our
our system
system effort
effort this
this past
past year
year went
went into
into continued
continued
development
development of local Ethernet
Ethernet facilities
facilities which
which link
link the SUMEX
SUMEX resource hardware
hardware with
with
other
other parts
parts of the campus,
campus, namely
namely to 701
701 Welch
Welch Road, which
which is
is the new location
location of the
Heuristic
Heuristic Programming
Progra.mming Project,
Project, and to the Computer
Computer Science
Science Department
Department building
building on
campus. \Ve
We have also invested
invested a great
great amount
amount of
of effort
effort this
this year
year to begin our
our transfer
transfer to
a 10 megabit
megabit Ethernet,
Ethernet, while
while continuing
continuing support
support of
of our
our current
current 3 MB
MB ethernet.
ethernet.
Specific
Specific areas of
of Ethernet
Ethernet development
development include:
include:

-- We
We continued
continued support
support of
of the Sequin reliable
reliable packet
packet protocol
protocol and
Leaf
Leaf byte-level
byte-level file transfer
transfer protocol
protocol to enable our
our Xerox
Xerox D machines
machines to access
access
files on our
our DEC20
DEC20 systems. The
The Leaf
Leaf server
server had to be modified
modified on the DEC20
DECPO
t,his
this past
past year
year when
when we switched
switched to using
using encrypted
encrypted passwords. The
The LEAF
LEAF
server
server implementation
implementation for
for the 4.2 BSD release of Unix
Unix was also debugged and
installed
installed at SUMEX.
SUMEX. This
This allows
allows us to
to access
access files stored
stored on our
our VAX
VAX file
servers
servers from
from either
either our
our 10MB
1OMB or
or 3MB
3MB networks.
networks.

1. Leaf
Leaf server
server

The
protocol is built
built into
The Leaf
Leaf protocol
into the lowest
lowest levels of
of the Dolphin
Dolphin I/O
I/O system,
system, and
allows
allows any
any file on a remote
remote file
file server
server to
to be accessed
accessed as
as easily
easily as
as a disk
disk file in
both paged or
both
or random
random access
access mode. The
The latest
latest updates
updates to
to the Sequin
transport
t,ransport level have made marked
marked improvements
improvements in efficiency.
efficiency. The
The 2020 now
now
performs Leaf
performs
Leaf file
file transfers
transfers with
with a speed approaching
approaching that
that of
of XEROX's
XEROX’s
dedicated
dedicated file server.
server.
2. TOPS-20
Ethernet Server
We continued
continued to
to maintain
maintain and improve
improve the
TOPS-20
Ethernet
Seroer -- We

Et.hernet
under the
Ethernet service under
to t.he
the TELNET
TELNET and FTP
FTP
mentioned Leaf
server, and
mentioned
Leaf server,

TOPS-20
updates
TOPS-20 operating
operating system.
system. This
This included
included updates
programs, as well
previously
programs,
well as
as mail
mail software,
software, the previously
network table
table maintanence
maintanence programs.
programs.
network

Ethernet Ciate.way
Gateway ---- Our
Our Ethernet
Ethernet gateway
gateway software
software has continued
continued to
to run
run
3. Eth,ernet
reliably and effectively.
effectively. The
The previous
previous problems
problems with
with lost
lost packets
packets and delayed
delayed
reliably
terminal response has been fixed,
fixed, the
the cause of
of which
which was a bad memory
memory board
board
t.erminsl
software bug
bug in the
the TOPS-20
TOPS-20 operating
operating system.
system. Serious
Serious problems
problems that
that
and a software
affected our
our net
net connectivity
connectivity to
to other
other parts
parts of
of campus
campus were also discovered
discovered
affected
repaired this
this past
past year,
year, thereby
thereby providing
providing us with
with over
over 99%
99% net
net
and repaired
connect.ivity to
to the rest
rest of
of campus.
campus. The
The changes involved
involved board
board repair
repair and
connect.ivit,y
modifications to
to the topology
topology of
of the
the campus
campus Ethernet.
Ethernet.
modifications
The gateway
gateway itself
itself was generalized
generalized
The
networks where
where previously
previously itit had
had
networks
currently have two
two gateways,
gateways, each
currently
networks. two
two of
of which
which are 3MB
3MB and
and
networks,

to handle
handle 3 or
or more
more directly
directly connected
connected
to
only dealt
dealt with
with 2 such networks.
networks. We
We
only
handling the
the traffic
traffic between
between three
three local
local
handling
10MB network.
network.
one a 1OMB

4. Ethernet
Ethernet TIP
TIP (EtherTIP)
(EtherTlp) --- The
The EtherTIP
EtherTIP provides
provides multiple
multiple terminal
terminal access
access to
to
the Ethernet.
Ethernet. A
A PUP
PUP ethernet
ethernet operating
operating syst,em
system was written
written for
for MC68000MC68000t.he
and a MC6800O-based
MC68000-based EtherTIP
EtherTIP was built
built base.d
based on
on this.
this.
based processors, and

E. A.
A. Feigenbaum
Feigenbaum
E.

Networking
Networking and Communications
Communications

5P41 RR00785-11
RR00785-11

The
The EtherTIP
EtherTIP software
software has undergone
undergone further
further enhancements
enhancements in the
Portions
of
this
work
was
done
in
conjunction
Portions
this work
conjunction with
with the Stanford
Stanford
Science Department.
Department. Among
Among those enhancements
enhancements are the following:
following:

past year.
year.
Computer
Computer

a. It
It now
now accepts
accepts incoming
incoming connections
connections to
‘to line printer
printer ports,
ports, and for
for remote
remote
system
system diagnosis.
diagnosis.
b. It
It can simulate
simulate the
the "old"
“old” Stanford
Stanford EtherTIP
EtherTIP for
for users who
who have not
not yet
yet
made the transition
transition to the new environment.
environment.
c. The
The user interface
interface is more
more flexible
flexible to suit
suit the needs of an increasingly
increasingly
diverse
diverse user community.
community.
The
The EtherTIP
EtherTIP software
software has developed
developed into
into a very
very stable
stable system,
system, and one
enjoying
enjoying good use
use within
within the SUMEX
SUMEX community.
community.
5. 10 AlB/SEC
- SUMEX
SUMEX made a major
major move this
this past
past
hfB/SEC Ethernet
Ethernet Development
Development --

year
year to begin our
our transfer
transfer to
to a 10 megabit/sec
megabit/set network.
network. While
While the current
current 3
megabit/sec
megabit/set network
network continues
continues to
to serve us well,
well, many
many new workstations
workstations and
printers
printers are coming
coming on the market
market with
with only
only 10 MB/SEC
MB/SEC interfaces,
interfaces, and in
addition,
a.ddition, since 3 MB/SEC
MB/SEC networks
networks were only
only used a very
very few selected
settings.
settings. it
it is becoming
becoming increasingly
increasingly difficult
difficult to find
find replacement
replacement parts
parts when
when
failures
do
occur.
failures
occur.
Therefore,
Therefore, this
this past
past year
year saw several efforts
efforts involved
involved in installing
installing and
supporting
supporting the SUMEX
SUMEX 10MB/SEC
lOMB/SEC Ethernet;
Ethernet ;
a. Reworking
Reworking the
the entire
entire Ethernet
Ethernet system
system software
software to
to handle
handle both
both 3 and
and 10
megabit
megabit link
link level
level standards,
standards, i.e., addressing
addressing and encapsulation
encapsulation are
transparent
transparent to
to the
the user
user levels. We
We similarly
similarly made the
the network
network link
link level
protocols transparent
protocols
transparent to
to the
the the
the user level software.
software. In
In this
this way
way one can
communicated
protocols on a 10MB/SEC
communicated using
using PUP
PUP protocols
lOMB/SEC ethernet
ethernet and the
the
user software
software does not
not have to
to change.
protocols for
b. Adding
Adding address resolution
resolution protocols
for PUP
PUP and
and IP
IP so that
that the
the
3~tB/SEC byte
byte addresses can be translated
3MB/SEC
translated to
to 10 MB/SEC
MB/SEC hardware
hardware
addresses for
for the
the link
link level. This
This enables one to
to communicate
communicate using
using PUP
PUP
or IP
IP between
between 3 and
and 10 megabit
megabit hosts.
or
Integrating XNS
XNS and
and IP
IP into
into the
the PUP
PUP routing
routing mechanism.
mechanism.
c. Integrating
Solving some rather
rather subtle
subtle software/hardware
software/hardware integration
integration problems
problems in
d. Solving
order to
to simulate
simulate “ethernet”
"ethernet" on
on the
the HPP/Welch
HPP/\Velch Road
Road “twisted
"twisted pair”
pair"
order
ethernet.
ethernet.
e. Bringing
Bringing up the
the 3 MB/SEC
MB/SEC EtherTIP
EtherTIP on the
the 10 MB/SEC
MB/SEC network
network was a
e.
proof tha,t
that the
the above worked.
worked. ItIt was done without
without any
any changes to
to the
the
proof
TIP software
software itself
itself by
by simply
simply relinking
relinking itit with
with the
the 10 MB/SEC
MB/SEC system
system
TIP
software. This
This required
required only
only one additional
additional piece of
of logic.
logic. When
When a 10
software.
MB/SEC host
host wants
wants to
to communicate
communicate using
using PUP
PUP which
which is a 3 MB/SEC
MB/SEC
MB/SEC
protocol. then
then itit must
must find
find its
its PUP
PUP address
address from
from some host
host on the
the 10
protocol,
MB/SEC network.
network. The
The gateway
gateway maintains
maintains a translation
translation table,
table, and
and listens
listens
MB/SEC
requests, thus
thus translating
translating the
the 10 MB/SEC
MB/SEC hardware
hardware address into
into
for such requests,
for
"soft PUP
PUP address,”
address," and
and replying
replying to
to the
the requesting
requesting host.
host.
a “soft

HPP-SUAIEX Communication
Communication link
link -- The
The Heuristic
Heuristic Programing
Programing Project
Project
6. HPP-SUrZ1E.Y

A. Feigenbaum
Feigenbaum
E. A.

Networking
Communications
Networking and Communications

5F41 RR00785-11
RR00785- 11
5P41

(HPP)
701 Welch
Welch Road, adjacent
adjacent to
to the
(HPP) relocated
relocated from
from its campus location
location to
to 701
Stanford
Stanford campus. Since this
this group
group is a primary
primary user of
of the SUMEX
SUMEX computer
computer
facility
facility and the principal
principal focus for
for core AI
AI research, a communication
communication link
link
location and SUMEX
SUMEX machine room
room was imperative.
imperative. Several
between the new location
communication
communication schemes
schemes for
for establishing
establishing a reliable
reliable and relatively
relatively fast link
link were
m icrowave, laser, infrared,
infrared, direct
direct ethernet
ethernet (by
(by trenching
trenching
considered, namely;
namely ; microwave,
and placing
placing a direct
direct ethernet
ethernet cable), ATT's
ATT’s Tl
Tl service and others.
others.
All
All of the above schemes
schemes would
would have necessitated
necessitat,ed large budgetary
budgetary outlays
outlays and
some
some would
would have imposed lengthy
lengthy time
time delays (getting
(getting permits
permits and the like)
like)
The idea of using bare copper telephone pair
pair
due to jurisdictional
jurisdictional boundaries. The
already in place looked very
very attractive
attractive especially if
if reasonable
reasonable speed
speed and
already
reliability
reliability could be achieved. The
The wire
wire distance between the above mentioned
mentioned
locations
locations is approximately
approximately 2000
2000 ft.
ft. A
A design goal was established to
to try
try to
to
communication link
link with
with Ethernet
Ethernet type
type speed
speed ( 3MB/SEC)
3MB/SEC ) between
develop a communication
these
these two
t.wo locations.
locations.
Utilizing
Ut.ilizing high
high driving
driving capacity
capacity drivers
drivers (differential)
(differential) and ultra
ultra high speed,
speed, high
high
sensitivity
sensitivity receivers a transceiver
transceiver was designed
designed and tested for
for maximum
maximum
transmission
transmission speed
speed with
with maximum
maximum reliability.
reliability. The
The final
final configuration
configuration resulted
resulted
in aa, half
half duplex
duplex transmission
transmission over
over a bare copper twisted
twisted pair
pair in each direction
direction
utilizing Manchester
Manchester coding at
at a reliable
reliable transmission
transmission speed
speed of
of 1.25MBs/sec.
1.25MBs/sec.
utilizing
each direction
direction for
for an aggregate
a.ggregate speed
speed of
of 2.5MBs/sec.
2.5MBs/sec. This
This communication
communication link
link
each
has been
been in operation
operation for
for about
about six months
months now without
without any appreciable
appreciable down
time
time or
or noticeable error
error rate
rate or
or data
data delays. Many
Many HPP
HPP researchers are
utilizing
utilizing t,his
t,his link
link to communicate
communicate with
with SUMEX
SUMEX and the University
University Ethernet
Ethernet
network.
and printers
network. In addition,
addition, various
various Lisp
Lisp machines
machines
printers located in the HPP
HPP
facility
facilit*y a.nd
and connected to
to a local network
network there are also able to communicate
communicate
with the University
University network.
network.
with

Ii'VTERNET SOF'1.1VARE
One major
major issue
issue we face
face at SUMEX-AIM
SUMEX-AIM in support
support of our
our network
network environment
environment. is
the lack of standardization
standardization in network
network protocols
protocols among various
various vendors. Currently,
Currently, many
many
vendors
/TCP) protocols.
vendors are adding
adding support
support to their
their products
products for the Internet
Internet (IP
(IP/TCP)
protocols.
SUMEX
/TCP protocols
SUMEX continues
cont.inues to support
support the
the IP
IP/TCP
protocols on the DEC2060, and we are
current,ly alpha-testing
alpha.-testing a release
release of
of Interlisp-D
Inter-lisp-D which
which also supports
supports IP
IP/TCP
protocols. In
currently
/TCP protocols.
addition,
/TCP software
nddit,ion, we sucessfully adapted the IP
IPjTCP
software to our
our VAX
VAX systems running
running UNIX
UNIX
4.2BSD.
L~BSD.
This
This Vax
Vax TCP
TCP adaptation
adaptation involved
involved provisions
provisions for subnet
subnet routing,
routing, 3 MB/SEC
MB/SEC
byte swap problems, encapsulation
encapsulation problems
problems and 10
10 MB/SEC
MB/SEC debugging with
with our
our
byte
gateways.
gntewa.y.5.

27
27

E. A. Feigenbaum

Progress - Progress in Core Research

5P41 RR00785-11

I.A.2.6.
Progress
in
I.A.2.6.
Progress
in Core
Core Research
Research
Over
Over the past year
year we have continued
continued to support
support several core research activities
activities
aimed at developing
developing information
information resources, basic AI
AI research, and tools
tools of general interest
interest
to the SUMEX-AIM
SUMEX
SUMEX is providing
providing only
only partial
partial support
support for
for these
SUMEX-AIM community.
community.
projects,
project,s, with
with complementary
complementary funding
funding coming
coming from
from ARPA,
ARPA, ONR,
ONR, NLM
NLM and NSF
contracts
contracts and grants
grants to the Stanford
Stanford Heuristic
Heuristic Programming
Programming Project.
Project.

Core Research
Research
Core Research at SUMEX-AIM
SUMEX-AIM focuses
focuses on understanding
understanding the roles of knowledge
knowledge in
symbolic
symbolic problem
problem solving
solving systems, its representation
represent.ation in software
software and hardware,
hardware, its use
use for
for
inference.
inference, and its acquisition.
acquisition. We are continuing
continuing to develop new tools
tools for system
system builders
builders
and to improve
improve old ones.
ones. The
The research crosses
crosses aa number
number of application
application domains,
domains, as
as
reflected
reflected in the subprojects
subprojects discussed earler,
earler, but
but the main
main issues
issues that
that we are addressing
addressing in
this
this rese3rch
research are
a.re those fundamental
funda.mental to all aspects of AI.
AI. We
We believe this
this core research is
broadening
broadening and deepening
deepening the groundwork
groundwork for
for the design and construction
construction of
of even more
more
capable and effective
effective biomedical
biomedical systems.
systems.
As mentioned
mentioned above, although
although our
our style
style of research is largely
largely empirical,
empirical, the
questions
questions we are addressing
addressing are fundamental.
fundamental. The
The three
three major
ma.jor research issues
issues in AI
AI have,
since its beginning,
beginning, been knowledge
knowledge representation,
representation, control
control of
of inference
inference (search), and
learning.
learning. \Vithin
Within these topics,
topics, we wiII
will be asking
asking the following
following kinds
kinds of
of questions. As our
our
work
work progresses, we hope to leave behind
behind several
several prototype
prototype systems
systems that
that can be developed
by others
others in the
t.he medical
medical community.
community.
1.
1. Knowledge
Knowledge

Representation
Representation -- How
How can we represent
represent causal models and
structural
What
benefits of
st.ructura,l information?
information?
What are the relative
relative benefits
of logic-based,
logic-based, rulerulehased,
based, and frame-hased
frame-based systems?
systems ? How
How can we represent
represent temporal
temporal relations
relations and
events
events so that
t,hat rea.'Soning
reasoning over
over time
time is efficient?
efficient?

2. Knowledge
Knowledge Acquisition
Acquisition -- How
How can an expert
expert system
system acquire
acquire new knowledge
knowledge
wit,hout consuming
without
consuming substantial
substantial time
time from
from experts? Can
Can we improve
improve the
knowledge
paradigm enough
knowledge engineering
engineering paradigm
enough to
to make
make a difference?
difference? Can
Can automatic
automatic
learning
programs
be
designed
that
will
work
across
many
disciplines?
lea,rning programs
that will work
many disciplines? Will
Will
cooperative
cooperat.ive man-machine
man-machine systems
systems be able to open the communication
communication channel
channel
het,ween
expert and expert
expert system?
bet,wern expert
Knowledge Utilization
Utilization -- By
By what
what inference
inference methods
methods can a variety
variety of
of sources of
of
3. Knowledge
knowledge of
of diverse
diverse types
types be made to
to contribute
contribute jointly
jointly and efficiently
efficiently
knowledge
towa.rd solut,ions?
sollltions? What
\Vhat is the
the nature
nature of
of strategy
strategy and control
control information?
information?
toward

Plans for
for the Coming
Coming Year
Plunv
Several syst,ems
systems have been developed
developed in recent
recent years
years to
to serve as vehicles
vehicles for
for
Severa,
knowledge engineering
engineering and research
research on knowledge
knowledge representation
representation and its
its use. Knowledge
Knowledge
knowledge
acquisition (including
(including machine
machine learning)
learning) and
and advanced
advanced architectures
architectures for
for AI
AI will
will be the
the two
two
acquisition
areas of
of most
most new activity
activity in the coming
coming year.
year. Research on these topics
topics obviously
obviously must
must
draw on on-going
on-going work
work in representation
representation and
and control.
control.
draw
In part,iculsr,
particular, we will
will focus on
In
• Induct,ive
Inductive learning
learning of
of MYCIN-like
MYCIN-Iike rules
rules from
from case
case data
data in the
the domain
domain of
of
diagnosing disorders
disorders where
where the
the chief
chief complaint
complaint is jaundice;
jaundice;
diagnosing

• Learning
Learning from
from experience
experience in domains
domains where
where the
the means for
for interpreting
interpreting new
new

A. Feigenbaum
Feigenbaum
E. A.

Progress
Progress - Progress
Progress in Core
Core Research

SP'l1
5P4 1 RR0078S-11
RROO785-11

data
data are largely
largely contained
contained in the
the emerging
emerging (and
(and thus
thus incomplete
incomplete and not
not
wholly
wholly correct,)
correct) theory;
theory;
•l Learning
by watching
Learning by
watching
NEOMYCIN;
NEOMYCIN;

medical
medical

expert
expert

diagnose

cases
presented
cases presented

by
by

•l Investigating
Invest,igating complex
complex signal
signal understanding
understanding systems
systems for
for ways
ways to exploit
exploit and
represent
represent concurrency
concurrency with
with a view
view toward
toward hardware
hardware and software
software architectures
architectures
that
that may
may be capable of
of several
several orders
orders of
of magnitude
magnitude improvement
improvement in
performance.
performance.
Further
and the Heuristic
Further information
information on the core research at
at SUMEX-AIM
SUMEX-AIM
Heurist,ic
Programming
Project
can
be
found
in
the
Projects
section
starting
on
page
89.
Programming Project
found
Projects section starting

E. A.
A. Feigenbaum
Feigenbaum

5Phl RR00785-11
RR00785- 11
5P~1

Progress
Progress - Resource Operations
Operations Statistics
Statistics

I.A.2.
Operations
Statistics
I.A.2.7. 7. Resource
Resource
Operations
Statistics
The
The following
following data
data give an overview
overview of
of various
various aspects of
of SUMEX-AIM
SUMEX-AIM resource
usage.
usage. There
There are 5 subsections
subsections containing
containing data
data respectively
respectively for:
for:
1.
1. Overall
Overall resource loading
loading data
data (page 31).

2. Relative
Rela,tive system
system loading
loading by
by community
community (page 33).
3. Individual
Individual project
project and community
community usage (page 36).
4. Network
Network usa.ge
usage data
data (page 44).
5. System
System reliability
reliability data
data (page 44).
For
For the most
most part,
part, the
the data
data used for
for these plots
plots covers the
the entire
entire span of
of the
SUMEX-AIM
SUMEX-AIM project.
project. This
This includes
includes data
data from
from both
both the TENEX
TENEX KIlO
KIlO system
system and the
curren
current t DECsystem
DECsystem 2060. At
At the point
point where
where the SUMEX-AIM
SUMEX-AIM community
community switched
switched over
over
to the 2060 (February,
(February, 1983), you
you will
will notice
notice severe changes in most
most of
of the graphs.
graphs. This
This is
due to many
rnany reasons
rea.sons which
which I will
will mentioned
mentioned briefly
briefly here;
here ;
1.
1. Even
Even t.ho1]gh
though the Tenex
Tenex operating
operating system
system used on the KilO
KIlO was a forerunner
forerunner of
of

t.he
the current
current Tops20
Tops20 operating
operating system,
system, the Tops20
Tops20 system
system is still
still different
different from
from
Tenex is many
many ways.
ways. Tops20
Tops20 uses
uses a radically
radically different
different job
scheduling
Tellex
job scheduling
mechanism,
mechanism, different
different methods
methods for
for computing
computing monitor
monitor statistics,
statistics, different
different I/O
I/O
routines,
routines, etc. In general.
general, it
it can not
not be assumed that
that statistics
statistics measured
measured on the
Tenex
Trnex syst·em
syst,em correlate
correlate one to one with
with similar
similar statistics
statistics under
under Tops20.
Tops20.
2.
3. The
The KLlO
KLlO processor
processor on the 2060 is a faster
faster processor
processor than
than the KilO
KIlO processor
processor
used previously.
job running
previously. Hence, a job
running on the KLlO
KLlO will
will use
use less
less CPU
CPU time
time than
than
the sa.me
job running
sa,me job
running on the KilO.
KIlO. This
This aspect is further
further complicated
complicated by
by the
fact
fact that
that the SUMEX
SUMEX KilO
KIlO system
system was a dual
dual processor
processor system.
system.
3. The
The SUMEX-AIM
SUMEX-AIM Community
Community was changing
changing during
during the time
time of
of the transfer
transfer to
the
just been
t,he 2060.
SOGO.The
The usage of
of the GENET
GENET community
community on SUMEX
SUMEX had just
pha:sed
phased out.
out. This
This part
part of
of the community
community accounted
accounted for
for much
much of
of the
the CPU
CPU time
time
used by
by the AIM
AIM community.
community. Since the
the purchase
purchase of
of the 2060 was partially
partially
funded
funded by
by the Heuristic
Heuristic Programming
Programming Project
Project (HPP),
(HPP), an additional
additional number
number of
of
HPP
HPP Core
Core Research Projects
Projects started
started using the 2060, increasing
increasing the Stanford
Stanford
communities usage
usage of
of the machine.
machine. And
And finally,
finally, the move to
to the 2050
2060 occurred
occurred
communities
during
during a pivotal
pivotal time
time in the community
community when
when more
more and more
more projects
projects were
either
eit,her moving
moving to their
their own
own local timesharing
timesharing machines,
machines, or
or onto
onto specialized
specialized Lisp
Lisp
workstations.
workstatJions. It
It also was the time
time for
for the closure of
of many
many long time
time SUMEXSUMEXAIM projects,
projects, like
like Dendral
Dendrsl and Puff/VM.
Puff/VM.
AIM
Any
,4ny conclusions
conclusions reached by comparing
comparing the
the data
data before
before and after
after February,
February, 1983
should be done with
with caution.
caution. The
The data
data is included
included in this
this years
years annual
annual report
report mostly
mostly for
for
should
casual comparison.
comparison. Starting
Starting next
next year,
year, only
only data
data from
from the 2060 will
will be recorded
recorded on the
annual
annual report.
report. Readers will
will be referred
referred to
to previous
previous annual
annual reports
reports (such as
as this
this one) for
for
data
da.ta from
from the KIlO
KIlO Tenex
Tenex system.
system.

Feigenbaum
E. A. Feigenba.um

Progress
Progress - Resource Operations
Operations Statistics
Statistics

5P41
5P4 1 RR00185-11
RR00785fI 1
,

Ot'erail
Overall Res~urce
Reshree Loading
Loading Data
Data
The
over the life
life
The following
following plots
plots display
display several different
different aspect.s
aspects of system
system loading
loading over
of t.he
the current
current
t,he project.
project. This
This data
data includes
includes usage of the
the Tenex
Tenex KilO
KIlO system
system and the
DECsystem
DECsystem ~050.
2060.
These plots
of jobs
jobs
plots include
include total
total CPU
CPU time
time delivered
delivered per
per month,
month, the peak number
number of
logged in, and the peak load average. The
variable is
The monthly
monthly "peak"
“peak” value
value of
of a given
given variable
the average of
Thus, these
of the daily
daily peak values
values for
for that
that variable
variable during
during the month.
month. Thus,
"peak"
not reflect
reflect
“peak” values are representative
representative of
of average monthly
monthly loading
loading maxima
maxima a.nd
a.nd do not
the
days, which
which are much
much higher.
higher.
t,he largest
largest excursions
excursions seen
seen on individual
individual days,

800
800r
*‘Or

Total
Total CPU
CPU Usage
Usage
Hours/Month
Hours/Month

600
600 -

400
400 -

200
200 -

1983
$74
1975
1976
1977
1978
1979
1980
1981
1982
1984
1985
4
o~--~----~----~----~--~~----~----~--~~--~----~----~
1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985

1974

Figure
7:
Figure
7:

Total
Total CPU
CPU Time
Time Consumed
Consumed by
by Month
Month

31
31

E. A. Feigenbaum
Feigenbaum

5P41
5P41 RR00785-11
RROOTSS-11

Progress -- Resource
Resource Operations
Operations Statistics
Statistics
Progress

60
50
”

Peak Daily
Daily Jobs
Jobs
Peak.

4040
3030

‘.

2020
1010
O~--~----~----~--~----~----~--~-----L----~--~~--~I

I%74
1974

1975
1975

1976
1976

1977
1977

1978
1978

Figure
8:
Figure
8:

*r
8

7
7

1979
1979

1980
1980

1981
1981

1982
1982

1983
1983

1984
1984

1985
1985

Peak Number
of Jobs by
by Month
Month
Peak
Number of

Peak
Peak. Daily
Daily
Load
Load Average
Average

66
55
44
33
22
11
tI
OL-____
L -_ _ _ _L -_ _ _ _L -_ _

1974
p974

1975
1975

1976
1976

1977
1977

Figure
Figure 9:9:

E.
E. A.
A. Fcigenb3.um
Feigenbaum

1978
1978

1979
1979

1980
1980

____

1981
1981

Peak
Peak Load
Load Average
Average by
by Month
Month

32
32

1982
1982

1983
1983

1984
1984

I
1985
1985

Progress - Resource Operations
Operations Statistics
Statistics
Progress

RR00785-11
5P41 RR00785-11
Relative System
Loading by Community
Relative
System Loading
Community

The SUMEX
SUMEX resource is divided,
divided, for
for administrative
administrative purposes, into
into three
three major
major
The
projects based at the Stanford
Stanford Medical
Medical School (Stanford
(Stanford Projects),
communities:
Projects),
communit#ies: user projects
project’s based outside
outside of
of Stanford
Stanford (National
(Nutiomzl Alhf
common system
system
user projects
AlA! Projects),
Projects), and common
development
As defined
development efforts
efforts (System
(System Staff).
stajfi.
defined in the resource management
management plan
plan
approved
approved by the BRP
BRP at
at the
the start
start of
of the project,
project, the available
available system
system CPU
CPU capacity
capacity and
divided between
between these communities
communities as follows:
follows:
file space resources are divided
Stanford
Stanford
AIM
AIhI
Staff
Staff

40S;
40%
40S;
40%
20S;
20%

The
The "available"
“available” resources to be divided
divided up in this
this way
way are those remaining
remaining after
after
various
various monitor
monitor and community-wide
community-wide functions
functions are accounted
accounted for.
for. These include
include such
things as
as job
scheduling, overhead,
overhead, network
network service, file space for
for subsystems,
subsystems,
things
job scheduling,
documentation,
documenlation. etc.

The monthly
monthly usage
usage of
of CPU
CPU resources and terminal
terminal connect
connect time
time for
for each of
of these
The
three communities
communities relative
relative to
to their
their respective
respective aliquots
aliquots is shown
shown in the plots
plots in Figure
Figure
three
10 and Figure
Figure 11. As mentioned
mentioned on page 30, these plots
plots include
include both
both KilO
KIlO and 2060 usage
data.
data.

E. A. Feigenbaum
Feigenbaum

5P41
5P41 RRO0785-11
RR00785-11

Progress -- Resource
Resource Operations
Operations Statistics
Statistics
Progress

40
40-

National Projects
Projects
National

%CPUTime
%
CPU Time

30
30.

20
2010
10I%74
I

.A,
1975

1976

1977

1978

1979

P974

1975

1976

1977

1978

1979

40
JOT

% CPU
CPU Used
%
Used

1980
1980

1981

1982

1983

1984

1985
4

1981

1982

1983

1984

1985

Stanford Projects
Projects
Stanford

30
30-

20
2010
IOI

I
o~----~----~----~----~----~----~----~----~----~----~----~

1974
I%74

40
40

1975
1975

1977
1977

1976
1976

1978
1978

1979
1979

%
36 CPU
CPU Used
Used

1980
1980

1981
1981

1982
1982

1983
1983

19S4
1984

1985
1985

System
System Staff
Staff

20
20
10·
70 -

_I
b--L+

o.~----~--~~

p974 1975
1975
1974

__ ____ ____ ____ ____ _____

1976
197G

1977
1977

Figure
10:
Figure
10:

EE.. .-\.
‘-1. Feigenbaum
Feigcnb:Lum

1978
1978

1979
1979

1980
19SO

1981
1981

____

1932
1932

tvIonthly
Monthly CPU
CPU Usage
Usage by Community
Community

34
34

____

1£83
1553

1984
19c&

1935

Progress
Progress -- Resource
Resource Operations
Operations Statistics
Statistics

5P41
5P4 1 RR00785-11
RR00785- 11

6000
5000

National
National Projects
Projects

Connect
Connect Time
Time
Hours/Month

4000
4000 3000
3000 2000
2000 1000
7000 -

---f
1975
,‘
974
I
1976
1977
1978
1979
1980
1981
1983
1985
4
1982
O~--~----~----~----~----~----~----~--~~--~----~----~
1975 1976 1977 1978 1979 1980 1981 1982 1983 .1984
1984 1985

1974

8000
Fizz
7000
6000
6000
5000
4000
3000
2000
1000

Connect
Connect Time
Time
Hours/Month
Hours/Month

Stanford
Stanford Projects
Projects

1975
lo974
I
1976
1977
1978
1979
1980
I
O~----~----~----~----~----~----~----~----~----~----~~--~
1975 1976 1977 1978 1979 1980 1981
1982
1983
1984
1985
1985
7983
1984
1991
7982

1974

6000 6000

Connect Time
Time
Connect
Hours/Month
5000
5000 - Hours/Month

System Staff
Staff
System

4000 4000
3000 3000
2000 2000
1000 1000
O~----~----~--~~--~~--~----~----~----~----~----~----~I

1974
IS74

1975
1975

1976
1976

1977
1977

Figure 11:
11:
Figure

1978
1978

1979
1979

1980
7980

1981
1981

1982
1982

1983
1983

1984
1984

Monthly Terminal
Terminal Connect
Connect Time
Time by
by Community
Community
Monthly

35
35

E. A.
A. Feigenbaum
Feigenbaum
E.

1985
1985

Progress
Progress -- Resource
Resource Operations
Operations Statistics
Statistics

5P41
5P41 RR00785-11
RR00785-11

Individual
Individual Project
Project and
and Community
Community U8age
Usage
The
The following
following histogram
histogram and
and table
table show
show cumulative
cumulative resource
resource usage
usage by
by collaborative
collaborative
project
and
community
during
the
past
grant
year.
The
histogram
displays
the project
project
project and community during the past grant year. The histogram displays the
distribution
of
the
total
CPU
time
consumed
between
May
1,
1983
and
April
3D,
distribution of the total CPU time consumed between May 1, 1983
April 30, 1984,
1984, on
the
t,he SUMEX-AIM
SUMEX-AIhl DECsystem2050
DECsyst.em2060 system.
system.
In
entries include
include aa text
text summary
summary of the funding
funding sources
In the
the table
table following,
following, entries
(outside
(outside of
of SUMEX-supplied
SUMEX-supplied computing
computing resources)
resources) for
for currently
currently active
active projects,
projects, total
total CPU
CPU
consumption
consumption by
by project
project (Hours),
(Hours), total
total terminal
terminal connect
connect time
time by project
project (Hours),
(Hours), and
average file space
space in use
use by project
project (Pages, 11 page
page = 512
512 computer
comput,er words).
words). These data
data
were accumulated
accumulated for each
each project
project for the months
months between May,
May, 1983 and May,
May, 1984.
1984.
Several of the projects
projects admitted
admitted to the National
National AIM
AIM community
community use
use the RutgersRutgersAIM
AIM resource as
as their
their home base.
base. We do not
not explicitly
explicitly list
list these projects
projects in this
this annual
annual
report
report covering
covering the St.anford
Stanford SUMEX-AIM
SUMEX-AIM resource. We do record
record information
information about
about the
Rutgers
Rutgers resource itself,
itself, however,
however, and note its separate resource status
status with
with the flag
"[Rutgers-AIM]"
.
” [Rut’gers-AIM] ‘I.

E. A.
A. Feigenbsurn
Feigenbaum
E.

35
36

Progress
Progress -- Resource
Resource Operations
OperatGons Statistics
Statistics

5P41
5P41 RR00785-11
RROOi85- 11

National
National AIM
AIM (24.4%
(24.4% Total)
Total)
AIM
AIM Administration
Administration
AIM
AIM Pilots
Pilots
AIM
AIM Users
Users
ACT
ACT

I
I

Caduceus
Caduceus
SEeS
SECS

1
I

Models
Models of
of Human
Human Cog
Cog
Solver
Solver
Puff-VM
Puff-VM
Rutgers
Rutgers

1
P

Stanford
Stanford (59.6%
(59.6% Total)
Total)
AGE
AGE
AI
Handbook
Al Handbook
DENDRAL
DENDRAL
EXPEX
EXPEX
Guidon
Guidon
Hpp
Research
Hpp Research
HPP
Assoc
HPP Assoc
Med
Med Info
Info Sci
Sci
MOLGEN
MOLGEN
Oncocin
Oncocin
Protein
Structure
Protein Structure
RX
Stanford Pilots
Pilots
Stanford
Stanford Assoc
Assoc
Stanford

.....

Staff (16.0%
(16.0% Total)
Total)
Staff

Staff
Staff
Staff Assoc
Assoc
Staff

Figure 12:
12:
Figure

P
I

10
10

15
20
25
25
15
20
Percent of
of Total
Total CPU
CPU Used
Used
Percent

Cumulative CPU
CPU Usage Histogram
Histogram by
by Project
Project and
and Community
Community
Cumulative

37
37

E. A.
A. Feigenbaum
Feigenbaum
E.

5P41
5P41 RR00785-11
RR00785-11

Progress -- Resource
Resource Operations
Operations Statistics
Statistics
Progress

Resource Use
Use by
by Individual
Individual Project
Project -- 5/83
5/83 through
through 4/84
4/84
Resource

National
A1A[ Community
Community
i\‘
ational .4IM

Connect
Connect
(Hours)
(Hours)

0.37
0.37

33.88
33.88

2866
2866

58.15
58.15

895.52

6852
6852

1.38
1.38

209.34

750
750

ACT Project
Project
1) ACT
"Acquisition of
of
“Acquisition
Cognitive Procedures”
Procedures"
Cognitive
John R. Anderson,
Anderson, Ph.D.
Ph.D.
John
Carnegie-Mellon Univ.
Univ.
Carnegie-Mellon
NSF IST-80IST-80-15357
NSF
15357
2/81-2/84
2/81-Z/84

File
File Space (**)
(**)
(Pages)
(Pages)

CPU
CPU
(Hours)
(Hours)

$186,000
$186,000

CADUCEUS
2) CADUCEUS
"Clinical Decision
Decision Systems
Systems
“Clinical
Resea.rch
Resource"
Resesrch Resource”
Jack D. Myers,
Myers, M.D.
M.D.
Jack
Harry E. Pople,
Pople, Jr.,
Jr., Ph.D.
Ph.D.
Harry
University of
of Pittsburgh
Pit,tsburgh
Universit,y
NIH RR-01
r·m-Ol101-07
NIH
101-07
$1,507,717
7/80-5/8S $1.607,717
7/80-6/&5
$369,484
7/83-6/84 $369,484
7,i83-f/84
NLM Lh1037
Lr-.W371O-04
NLM
IO-04
$817,884
7/80-5/8S
7/80-S/85
$817,884
$196,710
7/83-5/84
7/83-6/84
$196,710
NLt\l New
New Invest
NLhI
Invest L~103889-02
Lh¶03889-02
Gordon
Gordon E. Banks.
Banks. M.D.
M.D.
$107,675
4/82-3/8.5
4/‘82-3jS.S
$107,675
$35,975
4/83-3/84
4/83-3,/84
$35,975
$35,975
4/84-3/85
4/84-3185
$35.975
3) CLIPR
CLIPR Project
Project
"Hierarchical
“Hierarchical Models
Models
of Human
Hllman Cognition"
Cognition”
\Valter
\Valter Kint.sch.
Iiint.sch. Ph.D.
Ph.D.
Peter
Pet.er G. Polson,
Poison, Ph.D.
Ph.D.
University
Llniversity of Colorado
Colorado
NI!\'1H
NIXtH MH-15872-14-15
PIIH-1.5872-14-16 (Kintsch)
(Kintsch)
7/81-5/8'1
7/,81-6,/&l $281,085
77;83-6/8-1
/ 8~~-o/8·t $59,878
$60,878
NSF
NSF (Kintsch)
(E;int.sch)
$200,000
8/8a-7/85
8,‘83-7/‘86
$200,000
mr-.l
(Polson)
IBhi (Poison)
Da.vid
David Kieras
Kieras
lTniversity
Universit.y of
of Arizona
Arizona
$364,000
1/82-12/84
l/82-12/8.4
$364.000
$145,000
1/84-12/84
l/84-12/84
$145,000

E.
E. A.
A. Feigenbaum
Feigenbaum

38
38

Progress - Resource Operations
Operations Statistics
Stat.istics
Progress

SP41
5P41 RR0078S-11
RROO785-11

0.65

61.20

303

264.61

9877.34

10500

S.76
356.23
5.76
“Problem Solving
Solving
"Problem
Expertise”
Expertise"
Paul E. Johnson, Ph.D.
Ph.D.
P::J.1l1
v.,'illiam
William B. Thompson,
Thompson, Ph.D.
Ph.D.
Control
Control Data
Dat.a Corp.
Corp. (Johnson)
(Johnson)
$90,000
1983-85
$90,000
Microelect.
Microelect. and
a.nd Info.
Info. Ctr.
Ctr. Univ.
Univ. of
of rvlN
MN (Plus
(Plus Two
Two Colleagues)
Colleagues)
1984-1987 $800,000
$800,000
H18'i-1987

492

4) PUFF-VM
PUFF-VM Project
Project
“Biomedical Knowledge
Knowledge
"Biomedical
Engineering
Engineering in
Clinical
Clinica. Medicine"
Medicine”
John J. Osborn,
Osborn, M.D.
M.D.
John
Med. Research Inst.,
Inst.,
Med.
Francisco
San Francisco
Edward
Edward H. ShortIiffe,
Shortliffe, M.D.,Ph.D.
M.D.,Ph.D.
Stanford University
University
Stanford
Johnson &
k Johnson
Johnson
Johnson
1 year
$SO,OOO
$50,000 (*)
(*)
year
5) SECS Project
Project
"SiI1llllation
“Simulation &
Sr Evaluation
Evaluation
of Chemical
Chemical Synt.hesis"
Synthesis”
of
W.
W. Todd
Todd Wipke,
Wipke, Ph.D.
Ph.D.
U.
1J. California,
California., Santa
Santa Cruz
Cruz
NIHEHS
ES02845-02
NIHEHS ESO~845-02
4/82-3/85
4/82-3/8,5
$257.801
4j84-3i85
$89,140
4/84-3/85
$89,140
Evsns &
& SlItherland
Slltherland Corp.
Corp.
Evans
Equipment
Equipment gift
gift
Value
$9S,000
Value
$95,000
Stauffer
Sta,uffer Chemical
Chemical Co.
$5,000
$6,000
5)
6) SOLVER
SOLVER Project
Project

E. A.
A. Feigenbaum
Feigenbaum

Operations Statistics
Statistics
Progress - Resource Operations

5P41
5P41 RR00785-11
RR00785-11

*** [Rutgers-AIM]
[Rutgers-AIM] ***
***
7) ***
Rutgers Research Resource
Rutgers
“Computers in Biomedicine"
Biomedicine”
"Computers
Saul Amarel,
Amarel, D.Sc.
Ca.simir Kulikowski,
Kulikowski, Ph.D.
Ph.D.
Casimir
Shalom vVeiss,
Weiss, Ph.D
Ph.D
Shalom
Rutgers U
U.... New Brunswick
Brunswick
Rutgers
NIH RR-00543-12
RR-00643-12
(Amarel, Kulikowski)
Kulikowski)
NIH
(Amarel,
12/82-11/83
$405,304
12/82-11/83
$405,304
(Kulikowski, Weiss)
Weiss)
NIH RR-02230-01
RR-02230-01
NIH
(Kulikowski,

0.52
0.52

38.59
38.59

1117
1117

65.85
55.85

2227.48
2227.48

2461
2451

.93
.93

118.75
118.75

686
585

57.36
57.36

3836.19
3836.19

9649
9649

-----------------

-------------------

----m-w

455.56
455.56

17654.52
17554.52

35676
35576

12/83-11/87
$3,198,075
12/83-11/87
$3,198,075
12/83-l
l/84 $989,275
$989,276
12/83-11/84

AIM Pilot
Pilot Projects
Projects
8) AIM
9) AIM
AIM Administration
Administration
10) AIM
AIM Users
10)

Community Totals
Totals
Community

E. ‘A.
2. Feigenbaum
Feigenbaum

40
40

Progress - Resource Operations
Operations Statistics
Statistics

SP41
5P41 RR0078S-11
RR00785-11

CPU
CPU
(Hours)
(Hours)

Connect
Connect
(Hours)
(Hours)

File
File Space
Space
(Pages)
(Paws)

1) AGE
AGE Project.
Project (Core)
(Core)
"Attempt
“Attempt to
Generalize"
Generalize”
Edward
Ph.D.
Edward A. Feigenbaum,
Feigenbaum, Ph.D.
Dept.
Dept. Computer
Computer Science
Science
ARPA
(U*)
ARPA MDA903-8Q-C-0107
MDA903-80-C-0107
(***)
(partial
(partial support)
support)

11.80
11.80

845.30
845.30

4076
4076

2) AI
AI Handbook
Ha.ndbook Project
Project (Core)
(Core)
Edward
A.
Feigenbaum,
Ph.D.
Edward
Ph.D.
Dept.
Dept. Computer
Computer Science
Science
ARPA
(**)
hID,4903-80-C-0107
(**)
ARPA4 MDA903-8Q-C-0107
(partial
(partial support)
support)

1l.03
11.03

980.94
980.94

4425
4425

3.72
3.72

183.81
183.81

2980
2980

4) EXP
EX Proj
ect
EXPEX
Project
"Expert
“Expert Explanation"
Explanation”
Edward
Edward H. Shortliffe,
Shortliffe, M.D.,Ph.D.
hl.D.,Ph.D.
Dept.
Dept. Medicine
Medicine
ONR
NR 049-479
ONR NR
049-479
$-156,622
1/81-12/8:3
l/81-12,,‘8:3
$456,622
ONR
NR049-479
ONR NR049-479
Michael
hlichael Gt'nesereth
Genesereth
1/84-12/85
$312.070
l/84-12/‘86
NSF IST83-12H8
IST83-12148
Bruce G. Buchanan
Buchanan
$330,000
3/84-2/87
3/‘84-3,‘87
$330.000 (*)
(*)
$99,410
3/84-2/8.5
3 /8+2;‘8Fi
$99,410 (*)
(“)

53.75
53.75

239l.40
2391.40

4920
4920

5) GUIDON-NEOMYCIN
GUIDON-NEO~lYCIN Project
Project
"Exploration
“Exploration of Tutoring
Tutoring
S:
6 Problem-solving
Problem-solving
Strategies"
Strategies”
Bruce G. Buchanan,
Buchanan, Ph.D.
Ph.D.
William
William J. Clancey,
Clancey, Ph.D.
Ph.D.
Dept.
Dept. Computer
Computer Science
Science
ONR/
NOOOI4-79-C-0302
ONR/ARIARI N00014-79-C-0302
$583,892
3/79-3/85
3,‘79-3/85
$683,892

45.44
45.44

4418.68
4418.68

5967
5967

Stanford
Stan ford Community
Community

3) DENDRAL
DENDRAL Project
Project
"Resource
“Resource Related
Related Research:
Research:
Computers
Computers in Chemistry"
Chemistry”
Carl
Carl Djerassi,
Djerassi, Ph.D.
Ph.D.
Dennis H. Smith,
Smith, Ph.D.
Ph.D.
Dept.
Dept. Chemistry
Chemistry
NIH RR-00612-13
NIH
RR-00612-13
$170,710
5/82-4/83
5/82-d/83
$170,710

41
41

E. A. Feigenbaum

Progress
Progress - Resource Operations
Operations Statistics
Statistics

5P41
5P41 RR00785-11
RR00785- 11

6)
Project
6) fv[OLGEN
kIOLGEN
Project
"Applications
“Applications of
of Artificial
Artificial Intelligence
Intelligence

106.92
106.92

7734.34
7734.34

10448
10448

239.97

14404.62

14389

4.79

635.43
635.43

1296
1296

t.o
t,o Molecular
Molecular Biology"
Biology”
Edward
Edward A. Feigenbaum,
Feigenbaum, Ph.D.
Ph.D.
Peter
Peter Friedland,
Friedland, Ph.D.
Ph.D.
Charles
Charles Yanofsky,
Yanofsky, Ph.D.
Ph.D.
Depts.
Depts. Computer
Computer Science/
Science/
Biology
Biology
NSF MCS-8310236
NSF
MCS-8310236 (Feigenbaum,
(Feigenbaum, Yanofsky)
Ya.nofsky)
11/83-10/84
$139,215
(*)
11/83-lo/84
$139,315
(*)
7) ONCOCIN
ONCOCIN Project
Project
"Knowledge
“Knowledge Engineering
Engineering
for
for Med.
Med. Consult.ation
Consultation” II
EdWard
Edwa.rd H. Shortliffe,
Shortliffe, M.D.,Ph.D.
M.D.,Ph.D.
Dept.
~[edicine
Dept. hfedicine
NLM LM-03395
(Shortliffe/ONCOCIN)
NLhI
LM-03395
(Shortliffe/ONCOCIN)
Edward
A.
Feigenbaum,
Edward A. Feigenbaum, Ph.D.
Ph.D.
7/79-5/84
7/79-6184 $497,420
$497,420
7/83-5/84
$95,424
7,/83-6/84
$95,424
NLM LM-00048
NLhf
LbI-00048
$196,425
7/79-5/84
7/79-6/84
$196,425
7/83-5/84
$39,502
7,‘83-6184
$39,502
ONR
NR 049-479
ONR NR
$456,522
(*)
1/81-12/83
l/81-12/83
$456,622
(*)
NIH RR-01513
NIH
RR-01613
$524,455
7/83-6/85
7!83-6186
$624,455
$220,371
7/83-6/84
7/83-6/84
$220,371
NLM L~1-04136
NLM
Lhl-04136
$211,851
8/83-7/85
8/83-7/86
$211,851
$50.517
8/83-7/84
8/83-7/84
$60,517
H.J.
H.J. Kaiser
Kaiser Family
Family Fdn.
Fdn.
$150,000
7/83-6/86
7!‘83-6/86
$150,000
$50,000
7/83-6/84
7;83-6,/84
$50,000
ONR
NOOO14-81-K-0004
ONR N00014-81-K-0004
Michael
Michael R
R. Genesereth
Genesereth (Shortliffe)
(Shortiiffe)
1/84-12/86
$512.070
l/84-12/86
$512,070 (*)
(*)
NSF IST83-12148
NSF
IST83-12148
Bruce
Bruce G. Buchanan
Buchanan (Shortliffe)
(Shortliffe)
3/84-2/87
$330.000
(*)
3 /84-2/‘87
$330.000
(*)
$99,410
3/84-2/85
3/83-2/85
$99,410 (*)
(*)
8) PROTEIN
PROTEIN Project
Project
"Heuristic
“Heuristic Compo
Comp. Applied
Applied
to
to Prot.
Prot. Crystallog.
Crystallog.” II
Edward
Edward A.
A. Feigenbaum.
Feigenbaum, Ph.D.
Ph.D.
Dept.
Dept. Computer
Computer Science
NSF MCS-81-17330
NSF
MCS-81-17330
$28,976
1/82-1/83
l/82-1/83
$28,976

E. A. Feigenbaum
Feigenbaum

42
42

Progress - Resource Operations
Operations Statistics
Statistics

5P411 RR00785RR00785-11
5P4
11

79.44
RADIX Project
Project
9) RADIX
"Deriving Medical
Medical Knowledge
Knowledge from
from TimeTime“Deriving
Oriented Clinical
Clinical Databases”
Databases"
Oriented
Robert L. Blum.
Blum. M
M.D.
Robert
.D.
Gio C.hl.
C.I,I. VViederhold,
\Viederhold, Ph.D.
Ph.D.
Gio
Depts. Computer
Computer Science/
Science/
Depts.
Electrical Engrg.
Engrg.
Electrical
IST-8317858 (Blum)
(Blum)
NSF IST-8317858
(*)
3/84-3186
3/84-3/86 $89597 (*)
NLM (Wiederhold)
(Wiederhold)
NLM
$291,192
S/84-llj86
S/84-11/86 $291,192

3140.27
3140.27

8777

10) Stanford
Stanford Pilot
Pilot Projects
Projects

61.55
61.55

4115.02
4115.02

6097
6097

11)
HPP Core
Core AI
AI Research
Research
11) HPP

383.07
383.07

29073.96
29073.96

42202
42202

12)
HPP Associates
12) HPP

57.37
57.37

1600.31
1600.31

2997
2997

13) Stanford
Stanford Associates

27.01
27.01

1016.59
1016.59

1681
1681

5.62
5.62

1315.64
1315.64

587
587

---------

----------

_------

1091.46
1091.46

71856.29
71856.29

110842
110842

CPU
CPU
(Hours)
(Hours)

Connect
Connect
(Hours)
(Hours)

Space
File Space
(Pages)

288.21
288.21
16.65
16.65

17591.82
17591.82
1983.43
1983.43

23292
23292
7847
7847

---------

-------------------

_------

304.87
304.87

19575.25
19575.25

31139
31139

CPU
CPU
(Hours)
(Hours)

Connect
Connect
(Hours)
(Hours)

File Space
Space
File
(Pages)
(Pages)

530.54
530.54

67375.43
67375.43

167863
167863

----=====

2382.43
2382.43

176461.50
176461.50

345520
345520

14)
Medical Information
Information Sciences
Sciences
1-i) Medical

Community
Community Totals
Totals

SUMEY Stf'!!
St a f f
SUMEX
1) Staff
Staff
2)
2) System Associates
Associates

Community
Community Totals
Totals

System
System Operations
Operations
1)
1) Operations
Operations

Resource
Resource Totals
Totals
(*)
(*)

Award
Award includes indirect
indirect costs.
costs.

(**)
awarded
to
(**) Supported
Supported by aa larger
larger ARPA
ARPA contract
contract MDA-903-80-C-0107
MDA-903-80-C-0107
awarded
to
the Stanford
Stanford Computer
Computer Science
Science Department:
Department:

43
43

E. A. Feigenbaum

Progress
Progress - Resource Operations
Operations Statistics
Statistics

5P41
5P41 RR00785-11
RR00785- 11

System
Reliability
System Reliability
System
better than
System reliability
reliabilit,y for
for the DECsystem
DECsystem 2060 has been much
much better
than with
with our
our
previous KilO
periods of
particular hardware
previous
KIlO system.
system. "Ve
We have had very
very few periods
of particular
hardware or
or software
software
problems. The
below covers the
period in which
problems.
The data
data below
the entire
entire period
which the SUMEX-AIM
SUMEX-AIM community
community
has used the 2050.
2060. The
The actual
actual downtime
downtime was rounded
rounded to
to the nearest
nearest hour.
hour.

1
1

Feb

Mar

Apr

Table
per Month
Table 1 : System
System Downtime
Downtime Hours
Hours per
Month - February
February 83 through
through Apr
Apr 83

11
11

15
15

26
26

13
13

16
16

28
28

May
hlay

Jun

Jul
Jul

Aug
Au

Sep
Sep

Oct
Ott

Nov
Nov

Dec
Dee

Jan

Feb

Mar
Mar

Apr
-Or

Table
per Month
Ta.ble 2 : System
System Downtime
Downtime Hours
Hours per
Month - May
May 83 through
through Apr
Apr 84

Reporting
period
Reporting period
Total
UpTime
Total Up Time
Pl\I
Phi Downtime
Downtitne
Act.ual
Act,ual Downtime
Downtime
Tot,al
Tor.al Downtime
Downtime
Mtbf
Mtbf
Uptime
Upt,ime Percentage
Percentage

462 days, 23 hours,
hours, 41 minutes,
minutes, and 42 seconds
454 days, 5 hours,
hours, 16 minutes,
minutes, and 57 seconds
1 days, 14 hours,
hours, 2 minutes,
minutes, and 55 seconds
7 days, 4 hours,
hours, 21
21 minutes,
minutes, and 50 seconds

8 days, 18 hours,
hours, 24 minutes,
minutes, and 45 seconds
2 days, 16 hours,
hours, 30 minutes,
minutes, and 16 seconds
98.45

Network Usage Statistics
Network
Statistics
The
plots in Figure
The plots
Figure 13 and Figure
Figure 14 show the monthly
monthly network
network terminal
terminal connect
connect
tim('
broader term
time for
for the TYMNET
TYMNET and the INTERNET
INTERNET usage. The
The INTERNET
INTERNET is a broader
term for
for
what
previously referred
usage. Since many
what was previously
referred to as
as Arpanet
Arpanet usage.
many vendors
vendors now
now support
support the
INTERNET
protocols (IP
JTCP) in addition
JTCP
INTERNET protocols
(IP/TCP)
addition to
to the Arpanet,
Arpanet, which
which converted
converted to
to IP
IP/TCP
in Janllary
198~3, it
between Arpanet
Janua.ry of
of 1083,
it is no longer
longer possible to
to distinguish
distinguish between
Arpanet usage and
Internet
Internet usage
usage on our
our 2060 system.
system.

E. A.
A. Feigenbaum
Feigenbaum

44
44

Progress
Progress- -Resource
Resource Operations
OperationsStatistics
Statistics

SP'll
RR00785-11
5P4
1 RR0078511

1400

TYMNETConnect
ConnectTime
Time
TYMNET
Hours/Month
Hours/Month

‘400

1200
7200
10001000
800 800
600 600
400 400
200
zoo-

jr
4

OL---~----~----~----~----~----~----~--~~--~----~----~
I?374
I
1975
1974
1975 1976
1976 1977
1977 1978
1978 1979
1979 1980
1980 1981
1981 1982
1982 7983
1983 1984
1984 7985
1985

Figure 13:
13:
Figure

TYMNET
TYMNET Terminal
Terminal Connect
Connect Time
Time

1200

ARPAnet Connect
Connect Time
Time
r ARPAnet
Hours/Month
Hours/Month
1000
1000 ‘200

800
800 600
600 400
400
t

200
200
t

O~----~----~----~----~----~----~----~----~----~----~~--~
1975
1976
1977
7978
1979
1980
1981
! 982
1983
1984
1985

It374
I
1974

1975

1976

1977

Figure
Figure 14:
14:

1978

1979

1980

1981

1982

1983

1984

ARP
ANET Termina.l
ARPANET
Terminal Connect
Connect Time
Time

45
45

E. A.
A. Feigenbaum
Feigenbaum
E.

1985

Progress
Progress -- SUMEX
SUMEX Staff
Staff Publications
Publications

5P41
5P41 RR00785-11
RR00785- 11

I.A.2.8.
SUMEX
Staff
I.A.2.8.
SUMEX
Staff Publications
Publications
The
The following
following are
are publications
publications for
for the
the SUMEX
SUMEX staff
staff and
and include
include papers
papers describing
describing
t,he
the SUMEX-AIM
SURIEX-AIM resource and
and on-going
on-going research as
as well as
as documentation
documentation of
of system
system and
and
program
EX-AIM community
program developments.
developments. Many
Many of
of the publications
publications documenting
documenting SUM
SUMEX-AIM
community
research are
are from
from the individual
individual collaborative
collaborative projects
projects and
and are
are detailed
detailed in
in their
their respective
respective
reports
(see
Section
II
on
page
69).
Publications
for
the
AGE
and
AI
Handbook
reports (see Section II on page 69). Publications for
AGE and AI Handbook core
core
research
resesrch projects
projects are
are given
given there.
1.
1. Carhart,
Carhsrt, R.E.,
R.E., Johnson, S.M.,
S.M., Smith,
Smith, D.H.,
D.H., Buchanan,
Buchanan, B.G.,
B.G., Dromey,
Dromey, R.G.,
R.G.,

and Lederberg,
A
Lederberg, J., Networking
Networking and
and aa Collaborative
Collaborative Research
Research Community:
Community:
A
Programs.
IN
Cllse
Case Study
Study Using
Using the DENDRAL
DENDRAL
Programs.
IN P. Lykos
Lykos (Ed.),
(Ed.),
CO~1PUTER
AND
COMPUTER NETWORKING
NETWORKING
AND CHEMISTRY,
CHEMISTRY, ACS
ACS Symposium
Symposium Series,
Series,
No. 19,
1975.
19, 1975.
2.
2. Levinthal,
Levinthal, E.C.,
E.C., Carhart,
Carhart, R.E.,
R.E., Johnson, S.M.,
S.M., and Lederberg,
Lederberg, J.:
Computer,'!
Computers Talk
Talk to Computers.
Computers. Industrial
Industrial Research, November,
November, 1975.
1975.

When

3. Wilcox,
- A Machine-Independent
Programming
Wilcox, C.R.,
C.R., MAINSAIL
MAINSAIL
Machine-Independent
Programming System.
Syatem.

Proc.
Proc. DECUS
DECUS Symposium
Symposium 2(4),
Z(4), Spring,
Spring, 1976.
1976.
4. Wilcox,
Project:
for Software
C.R.: The AL4INSAIL
MAINSAIL
Project: Developing
Wilcox, C.R.:
Developing Tools
Tools for
Software
Portability.
Portability. Proc.
Proc. SCAMC,
SCAMC, October,
October, 1977, pp. 76-83.
5. Lederberg,
and
Lederberg, J.L.:
J.L.: Digital
Digital Communications
Communication8
and the Conduct
Conduct of
of Science:
Science: The
New
New Literacy.
Literacy. Proc.
Proc. IEEE
IEEE 66(11), November,
November, 1978.
1978.
6. Wilcox,
MAINSAIL - Language
Language
Wilcox, C.R.,
C.R., Jirak,
Jirak, G.A.,
G.A., and Dageforde,
Dageforde, M.L.:
M.L.: MAINSAIL
Afanual. Stanford
Manual.
Stanford University
University Computer
Computer Science Report
Report STAN-CS-80-791,
STAN-CS-80-791,
1980.
1380.
C.R.,
Jirak,
G.A.,
and
Dageforde,
M.L.:
AIAINSAIL
C.R.,
Dageforde,
M.L.:
Jirak,
G.A.,
MAINSAIL
- Implementation
Implementation Overview.
Overview. Stanford
Stanford University
University Computer
Computer Science Report
Report
STAN-CS-80-792, 1980.
STAN-CS-80-793,

7. \Vilcox,
\Vilcox,

addition, a substantial
substantial continuing
continuing effort
effort has gone into
into developing,
developing, upgrading,
upgrading,
In addition,
extending documentation
documentation about
about the
the SUMEX-AIM
SUMEX-AIM resource.
resource. These efforts
efforts include
include user
and extending
help files, and introductory
introductory notes, an ARPANET
ARPANET Resource Handbook
Handbook entry,
entry, and
and
guides, help
policy guidelines.
guidelines.
policy

E. A.
A. Feigenbaum
Feigenbaum
E.

46
46

Progress
Progress - Future
Future Plans
Plans

5P41 RR00785-11
RR00785-11

I.A.2.9.
Future
Plans
I.A.2.9.
Future
Plans
Our
plans for
plans for
Our plans
for the next
next grant
grant year
year are based on the Council-approved
Council-approved plans
for our
our
5-year
plans for
5-year renewal
renewal that
that began in August,
August, 1980. In
In addition
addition to the specific
specific plans
for the
the next
next
grant
period to serve
grant year,
year, we include
include a summary
summary of
of the overall
overall objectives
objectives for
for this
this 5-year
5-year period
as
background. NearNear- and long-term
plans for
as a background.
long-term objectives
objectives and plans
for individual
individual collaborative
collaborative
projects are discussed in Section
beginning on page 69.
projects
Section II
II beginning

Ot.erall
Overall Goals
Goals
The
The goals of
of the SUMEX-AIM
SUMEX-AIM resource are long-term
long-term in supporting
supporting basic research
in artificial
broad range of
biomedical
applying these techniques
techniques to
to a broad
of biomedical
artificial intelligence,
intelligence, applying
problems, experimenting
with
technologies
promote scientific
with communication
communication
technologies to
to promote
scientific
experimenting
problems,
interchange,
better tools
interchange, and developing
developing better
tools and facilities
facilities to
to carryon
carry on this
this research.
research. Just
Just as
as
the tone of
proposal derives
of our
our renewal
renewal proposal
derives from
from the
the continuing
continuing long-term
long-term research
research objectives
objectives
of
philosophy
of the SU~fEX-AIM
SUMEX-AIM community,
community, our
our approach
approach derives
derives from
from the methods
methods and philosophy
already
We will
will continue
continue to
to develop useful
useful knowledge-based
knowledge-based
already established
established for
for the resource. We
software
biomedical research based on innovative,
softwa,re t,ools
t,ools for
for biomedical
innovative, yet
yet accessible computing
computing
t.echnologies.
t,echnologies.
For
For us it
it is important
important to
to make systems
systems that
that work
work and are exportable.
exportable. Hence, our
our
approach
is
to
integrate
available
state-of-the-art
hardware
technology
approach
to integrate available state-of-the-art hardware technology as a basis for the
the
underlying
software research and development
development necessary to support
support the AI
AI work.
work.
underlying software
SUMEX-AIM
SUMEX-AIM will
will retain
retain its broad
broad community
community orientation
orientation in choosing
choosing and implementing
implementing
its
its resources. \Ve will
will draw
draw upon
upon the expertise
expertise of
of on-going
on-going research efforts
efforts where
where possible
and
This orientation
orientation has
a.nd build
build on these where
where extensions
extensions or
or innovations
innovations are necessary. This
proved to be an effective
proved
effective way
way to build
build the current
current facility
facility and community.
community.
We
We have built
built ties to a broad
broad computer
computer science community;
community; have brought
brought the
the
rE'sults
results of
of their
t.heir work
work to
to the AIM
AIM users;
users: and have exported
exported results
results of
of our
our own
own work.
work. This
This
broader
particularly active
broader community
community is particularly
active in developing
developing technological
technological tools
tools in the form
form of
of
new machine
machine architectures,
architectures, language support,
support, and interactive
interactive modalities.
modalit.ies.

Toward
Distributed Resource
Resource
Toward a Afore
More Distributed
The
The initial
initial model
model for
for SUMEX
SUhLIEX as
as a centralized
centralized resource was based on the high
high cost
of
powerful computing
of powerful
computing facilities,
fa.cilit,ies, which
which were not
not readily
readily duplicated.
duplicated. This
This role is evolving,
evolving,
though,
though, with
with the
t.he introduction
introduction of
of more
more compact
compact and inexpensive
inexpensive computing
computing technology.
technology.
Our
by community
Olur fnture
future goals are guided
guided by
community needs for
for more
more computing
computing capacity
capacity and
improved
build more
improved tools
tools to build
more effective
effective expert
expert systems, and to
to test
test operational
operational versions
versions of
of
AI
programs in real-world
In order
order to
to meet
meet these needs, we must
must take
take advantage
advantage
AI programs
real-world settings.
settings. In
of
of a ralll,;e
range of
of newly-developing
newly-developing machine
machine architectures
architectures and systems. As a result,
result, SUIvIEXSUhsIEXAIM
AIM will
will become a more
more distributed
distributed community
community resource with
with heterogeneous
heterogeneous computing
computing
facilit,ies
fncilit.ies tE'thE'rE'd
tethered to
t,o each other
other through
through communications
communications media.
media. Many
Many of
of these machines
machines
will
physically near the projects
projects or
will be located
located physically
or biomedical
biomedical scientists
scientists using
using them.
them.

The C{Jntinuing
Role of
Continuing Role
of SUAfEX-Central
SUMEX-Central
Even
Even with
with more
more distributed
distribut.ed computing
computing resources, the central
central resource will
will continue
continue
to play
play an important
important role as
as a communications
communications crossroad,
crossroad, as
as a research
research group
group devoted
devoted to
integrating
integrating the new software
software and hardware
hardware technologies
technologies to
to meet
meet the needs of
of medical
medical AI
,41
projects, and as
applicat.ions,
applications, as a spawning
spawning ground
ground for
for new application
application projects,
as a base
base for
for local AI
AI
projects. A
projectas.
A key
key challenge
challenge will
will be to
to maintain
maintain the scientific
scientific community
community ties that
that grew
grew
naturally
previous colocation
naturally out.
out of
of the
t,he previous
colocation within
within a central
central facility.
facility.

E. A. Feigenbaum
Feigenbaum

5P41 RR00785-11
RR00785- 11

Progress - Future
Future Plans

Summary
Summary of
of Five-!lear
Five-year Objectives
Objectives
The long-term
long-term objectives
objectives of the SUMEX-AIM
SUMEX-AIM resource nucleus during
during the follow-on
follow-on
5 year
period
(of
which
we
are
in
the
third
year)
are
summarized
year period
which
third year)
summarized below. These are
broken
resource operations,
operations, training
training and
and education,
education, and core
broken into
into three
three categories:
categories: resource

research.
research.
Resource
Resource Operations
Operations
l.
will continue
continue to encourage
1. Maintain
Ma.inta.in the vitality
vitality of the AIM
AIM community
community -- We will

and explore
explore new applications
applications of AI
AI to biomedical
biomedical research and improve
improve
mechanisms
mechanisms for interinter- and intra-group
intra-group collaborations
collaborations and communications.
communications.
While
While AI
AI is
is our
our defining
defining theme,
theme, we may
may entertain
entertain exceptional
exceptional applications
applications
justified
justified by some other
other unique
unique feature
feature of SUMEX-AIM
SUMEX-AIM essential for
for important
important
biomedical
biomedical research. To
To minimize
minimize administrative
administrative barriers
barriers to the communitycommunityoriented
oriented goals of SUMEX-AIM
SUMEX-AIM and to direct
direct our
our resources toward
toward purely
purely
scientific
scientific goals, we plan
plan to retain
retain the current
current user funding
funding arrangements
arrangements for
for
projects
User projects
projects working
working on SUMEX
SUMEX facilities.
facilities.
projects will
will fund
fund their
their own
own
manpower
manpower and local needs; will
will actively
actively contribute
contribute their
their special expertise
expertise to
the SUMEX-AIM
SUMEX-AIM community;
community; and will
will receive an allocation
allocation of
of computing
computing
resources under
under the control
control of
of the
the AIM
AIM management
mana.gement committees.
committees. There
There will
will
be no "fee
“fee for
for service"
service” charges for
for community
community members.
members. We also will
will continue
continue
to exploit
exploit community
community expertise
expertise and sharing
sharing in software
software development,
development, and to
facilitate
facilitate more effective
effective information-sharing
information-sharing among
among projects.
projects.
2. Provide
Provide effective
effective computational
computational support
support for
for AIM
AIM community
community goals -- We
We will
will
continue
intelligence
continue to expand
expand support
support for
for artificial
artificial
intelligence research and new
applications
applicat.ions work,
work, to develop
develop new computational
computational tools
tools to support
support more
more mature
mature
proj ects, and to facilitate
projects,
facilitate testing
testing and research dissemination
dissemination of
of nearly
nearly
operational
programs. We
operational programs.
We will
will continue
continue to
to operate
operate and develop
develop the existing
existing
central
cent,ral facility
facility as
as the nucleus
nucleus of
of the resource. We
We will
will acquire
acquire additional
additional
11ipment to meet
eq
equipment
meet developing
developing community
community needs for
for more
more capacit,y,
capacity, larger
larger
program address
New computing
program
a.ddress spaces,
spaces, and improved
improved interactive
interactive facilities.
facilities. New
computing
hardware
becoming available
hardware technologies
t,echnologies becoming
available now
now and in the next
next few years
years will
will
play a key
play
key role in these developments,
developments, and we expect
expect to take
take the lead in this
this
community
biomedical AI
community for
for adapting
ada.pting these new tools
tools to biomedical
AI needs.
Provide effect.ive
effective and geographically
geographically accessible communication
communication facilities
facilities to
to the
the
3. Provide
SUMEX-AIM commiinit,y
community for
for effective
effective remote
remote collaborations,
collaborations, communications
communications
SUMEX-AIM
among dist,ribut,ed
distributed computing
computing nodes, and experimental
experimental testing
testing of
of AI
AI programs
programs
among
We will
will retain
retain the
the current
current ARPANET
ARPANET and TYMNET
TYMNET connections
connections for
for at
at
-- We
the near-term
near-term and
and will
will actively
actively explore
explore other
other advantageous
advantageous connections
COIJ.nections to
to
least the
communications networks
networks and to
to dedicated
dedicated links.
links.
new communications

Training
and Education
Education
Training and
l. Assist
Assist new and
and established
established projects
projects in
in the
the effective
effective use of
of the
the SlJMEX-AIM
SUMEX-AIM
1.
resource
Collaborative
projects
continue
to
be
responsible
for the
the
resource -- Collaborative
projects continue to
responsible for
development and
and dissemination
dissemination of
of their
their own
own AI
AI programs,
programs, but
but the
the resource
resource
development
staff will
will provide
provide general
general support
support and
and will
will work
work to
to make
make resource
resource goals and
and AI
AI
staff
systems known
known and
and available
available to
to appropriate
appropriate biomedical
biomedical scientists.
scientists. We
We will
will
systems
continue to
to exploit
exploit particular
particular areas of
of expertise
expertise within
within the
the community
community for
for
continue
deve loping pilot
pilot efforts
efforts in new application
application areas.
developing

E. A.
A. Feigenbaum
Feigenbaum

Progress
Progress - Future
Future Plans
Plans

5P41 RR00785-11
RR00785-11

2.
funds to
to qualifying
qualifying new and pilot
pilot
9. Continue
Continue to
to allocate
allocate "collaborative
“collaborative linkage"
linkage” funds
projects
projects to
to provide
provide for
for communications
communications and terminal
terminal support
support pending
pending formal
formal
approval
approval and funding
funding of
01 their
their projects
projects -- These funds
funds are allocated
allocated in
cooperation
prospective user
cooperation with
with the AIM
AIM Executive
Executive Committee
Committee reviews
reviews of
of prospective
projects.
projects.
3. Continue
Cont.inue to support
support workshop
workshop activities
activities including
including collaboration
collaboration with
with the
the
Rutgers
Rutgers Computers
Computers in Biomedicine
Biomedicine resource on the AIM
AIM Community
Community
\Vorkshop
Workshop and with
with individual
individual projects
projects for
for more specialized
specialized workshops
workshops
covering
covering specific
specific application
application areas or
or program
program dissemination
dissemination
Core
Research
Core Research
1.
1. Continue
Continue to
to explore
explore basic Artificial
Artificial Intelligence
Intelligence research issues
issues for
for knowledge
knowledge

acquisition.
reasoning
representation, and utilization;
utilization;
acquisition. representation,
reasoning in the presence of
of
uncertainty;
uncertainty; strategy
strategy planning;
planning; and explanations
explanations of
of reasoning
reasoning pathways
pathways with
with
particular
emphasis on biomedical
particular
biomedical applications
applications -- SUMEX
SUMEX core research
funding
to similar
funding is complementary
complementary
similar funding
funding from
from other
other agencies and
contributes
contributes to
to the
the long-standing
long-standing interdisciplinary
interdisciplinary effort
effort at
at Stanford
Stanford in basic AI
AI
research and expert
We expect
expect this
this work
work to
to provide
provide the
expert system
system design. We
foundation
foundation for
for increasingly
increasingly effective
effective consultative
consultative programs
programs in medicine
medicine and for
for
more
practical
adaptations
of
this
work
within
emerging
more practical adaptations 01 this work within emerging microelectronic
microelectronic
technologies.
technologies.
2. Support
Support community
community efforts
efforts to
to organize
organize and generalize
generalize AI
AI tools
tools that
that have been
developed
developed in the context
context of
of individual
individual application
application projects
projects -- This
This will
will include
include
work
work t,o
to organize
organize the present
present state-of-the-art
state-of-the-art in AI
AI techniques
techniques through
through the
development
development of
of practical
practical software
software packages for
for the acquisition.
acquisition, representation.
representation,
and utilization
utiiization of
of knowledge
knowledge in AI
AI programs.
programs. The
The objective
objective is to evolve a body
body
of
of software
software tools
tools that
that can be used to
to more
more easily build
build future
future knowledge-based
knowledge-based
systems
systems and explore
explore other
other bionmedical
bionmedical AI
AI applications.
applications.

E. A. Feigenbaum
Feigenbaum

5P41
5P41 RR00785-11
RR00785-11

Progress
Progress -- Future
Future Plans
Plans

Specific
Specific Plans
Plan8 for
for Year
Year 12
12
Specific
Specific plans
plans for
for the
the next
next grant
grant year
year (12)
(12) are
are summarized
summarized in
in the
the paragraphs
paragraphs below.
below.
The
The directions
directions and
and background
background for
for much
much of
of this
this work
work were
were given
given in
in earlier
earlier progress
progress report
report
sections
sections and
and are
are not repeated in
in detail
detail here.

Professional
Workstations
fiofessional
WOrk8tatiOn8
We see
see our
our major
major development
development efforts
efforts in
in year
year 12
12 to be
be in
in the area
area of
of professional
professional
workstat.ions,
workstat,ions, and
and specifically,
specifically, to fine
fine tune the integration
integration of
of these
these workstations
workstations into
into our
our
networking
networking environment.
environment. This
This involves
involves software
software integration,
integration, support
support of network
network protocols,
protocols,
general access
access to network
network printing
printing facilities,
facilities, telnet
telnet access
access to Lisp ma.chines,
ma.chines, and overall
overall
worksta.tion
workststion ma.intenance
maintenance and support.
support.
We will
will also continue
continue to explore the use
use of low cost workstations
workstations within
wit’hin our
our
environment,
environment, both
both as
as distributed
distributed processors for
for text
text editing
editing and electronic
electronic mail,
mail, and as
as
powerful
powerful graphic
graphic terminals
terminals for
for use
use with
with sophisticated
sophisticated programs
programs running
running on our
our
mainframes.
mainframes. We also see
see the use
use of virtual
virtual graphics
graphics interfaces
interfaces running
running on remote
remote
workstations
to
be
of
continued
importance
to
our
progress
in
the
future.
workstations
continued importance
our
future.

Continued
Continued Operation
Operation of
of Existing
Existing Hardware
Hardware
The
The current
current, SUMEX-AIM
SUMEX-AIM facilities
facilities represent
represent a large existing
existing investment.
investment. 'vVe
We plan
plan
to continue
continue development
deve!opment of
of our
our main
main timesharing
timesharing machine,
machine, the
t,he DEC2050/TOPS-20
DEC2060/TOPS-20
syst.ern,
syst.em, and the SUMEX-AIM
SUMEX-AIM file server
server (SAFE),
(SAFE), and make changes as
as necessary to
improve
improve the performance
performance of
of these machines.
machines. We
We do not
not propose any
any substantial
substantial changes
to the ot.her
ot,her hardware
hardware systems (2020, shared VAX,
VAX, and Lisp Machines).
Machines). We
We expect
expect them
them
to
provide effective
t.o continne
cont,inue to provide
effective community
community support
support and serve as
as a nucleus for
for our
our
distribut,ed
di,strihut.ed resource.

Communication
Networks
Communication
Network8
Networks have been centrally
Net,works
centrally important
important to the research goals of
of SUMEX-AIM
SUhlEX-AIM and
will
increasingly distributed
computing. Communication
will become more
more so in the context
context of
of increasingly
distributed computing.
Communication
will be crucial
crucial to
to mainta.in
maintain community
community scientific
scientific contacts,
contacts, t,o
to facilitate
facilitate shared
shared system
system and
will
software maintenance
maintenance based on regional
regional expertise,
expertise, to
to allow
allow necessary information
information flow
flow and
soft,ware
access at
at all
all levels, and to
to meet
meet the technical
technical requirements
requirements of
of shared
shared equipment.
equipment.
access
vVe have had reasonable
reasonable success
success at
at meeting
meeting the geographical
geographical needs of
of the
We
community during
during the
the early
early phases of
of SUMEX-AIM
SUMEX-AIM through
through our
our ARPANET
ARPANET and
and
cornmunit,y
TY~INET connections.
connections. These have allowed
allowed users from
from many
many locations
locations within
,vithin the
the United
United
T\-hINET
States and
and abroad
abroad to
to gain
gain terminal
terminal access
access to
to the
the AIhl
AIM resources and
and through
through ARPANET
ARPANET
States
links t,o
to communicat,e
communicate much
much more
more voluminous
voluminous file
file information.
information. Since many
many of
of our
our users do
links
not have ARPANET
ARPANET access
access privileges
privileges for
for technical
technical or
or administratQve
administrative reasons, a key
key
not
problem impeding
impt'ding remote
remote use has been the
the limited
limited communications
communications facilities
facilities (speed, file
file
problem
transfer. and
and terminal
terminal handling)
handling) offered
offered currently
currently by
by commercial
commercial networks.
networks. Commercial
Commercial
transfer,
improvements a.re
are slow
slow in coming
coming but
but may
may be expected
expected to
to solve the
the file
file transfer
transfer problem
problem
improvements
the next. few years.
years. A
A number
number of
of vendors
vendors (AT&T,
(AT&T, IBM,
IBM, XEROX,
XEROX, etc.)
etc.) have yet
yet to
to
in the
announce commercially-available
commercially-available facilities,
facilities, but
but TELENET
TELENET is actively
actively working
working in
in this
this
announce
direction. We
\-Ve plan
plan to
to continue
continue experimenting
experimenting with
with improved
improved facilities
facilities as offered
offered by
by
direction.
commercial
or
government
sources
in
the
next
grant
term.
We
have
budgeted
for
commercial or government sources in the next grant term.
We have budgeted for
cont.inued TYMNET
TY~vINET service
service and
and an additional
additional amount
amount annually
annually for
for experimental
experimental network
network
cont,inued
connections.
connections.
Higil-:':peed intera.ctive
interactive terminal
terminal support
support will
will continue
continue to
to be a problem
problem since one
High-speed
cannol. expect
expect to
to serve
serve 1200 to
to 9600
9500 baud
baud terminals
terminals effectively
effectively over
over shared
shared long-distance
long-distance
ca.nnoi.
trllnk lines
lines with
with gross
gross capacities
ca.pacities of
of only
only 9600
9500 to
to 19200 baud.
baud. We
We feel
feel this
this is
is aa problem
problem
trllnk

E. A.
A. Feigenbaum
Feigenbaum
E.

50
50

Progress - Future
Future Plans

SP41
5P4 1 RR0078S-11
RR00785- 11

that
that is
is best solved by distributed
distributed machines able to effectively
effectively support
support terminal
terminal
interactions
interactions locally
locally and coupled
coupled to other
other AIM
AIM machines and facilities
facilities through
through network
network or
telephone
t,elephone links.
links. As new machine
machine resources are introduced
introduced into
into the community,
community, we will
will
allocate
advice to assure
a,llocate budgeted
budgeted funds with
with Executive
Executive Committee
Committee
assure effective
effective
communications
communicat,ions links.
links.

Resource
Resource Software
So jtware
We will
will continue
continue to maintain
maintain the existing
existing system, language, and utility
utility support
support,
software
software on our
our systems at the most
most current
current release
release levels, including
including up-to-date
up-to-date
documentation.
We also will
will be
be extending
extending the facilities
facilities available
available to users where
documentation.
appropriate,
rely
appropriate, drawing
drawing upon
upon other
other community
community developments
developments where possible. We rely
heavily
heavily on the needs
needs of the user community
community to direct
direct system
system software
softwa.re development
development efforts.
efforts.
\Vithin
Within the AIM
AIM community
community we expect to serve as
as a center
center for software-sharing
software-sharing
between various
This will
will include
include contributing
contributing locallylocallyvarious distributed
distributed computing
computing nodes. This
developed programs,
those derived
programs, distributing
distributing
derived from
from elsewhere in the communit,y,
community,
maintaining
maintaining up-t.o-date
up-to-date information
information on subsystems
subsystems available,
available, and assisting
assisting in software
software
maintenance.
ma.inth the management
management committees.
committees. This
redistribut,ed
project members
by the individual
principal investigators.
redist,ribut.ed among
among project
members as directed
directed by
individual principal
investigators.
System
project allocations
Syst’em enforcement
enforcement of
of project
a.llocations is done on a weekly
weekly basis. As the weekly
weekly file
dump is done, if
the aggregate
aggregate space in use by
by a project
project exceeds
exceeds its
its allocation,
allocation, files are
dump
if the
archived from
from associated user directories
directories which
which are over
over allocation
allocation until
until the
the project
project is
archived
within its
its authorized
aut,horized lirnits.
limits.
within
vVe are using
using the TOPS-20
TOPS-20 class scheduler
scheduler to
to attempt
attempt to
to enforce
enforce the 40:40:20
We
bala,nce in terms
terms of
of CPU
CPU utilization
utilization and
and to
to avoid
avoid system
system and
and user’
user' inefficiencies
inefficiencies under
under
bal:-mce
overload conditions.
conditions. In practice,
practice, the
the 40:40 split
split between
between Stanford
Stanford and non-Stanford
non-Stanford
over1on.d
projects is fa.irly
fairly well
well realized
realized (see
(see Figure
Figure 10 on page 34 and the tables
tables of
of recent
recent project
project
projects
llsage on page 3Gj.
:36).
usage
Our job-scheduling
job-scheduling controls
controls bias the
the allocation
allocation of
of CPU
CPU time
time based on per
per cent
cent time
time
Our
consumed relative
relative to
to the
the time
time allocated
allocated according
according to
to the
the 40:40:20
40:40:20 community
community split.
split.
consumed
Howeyer, the
the controls
controls are “soft”
"soft" in
in that
that they
they do not
not waste
waste computer
computer cycles ifif users below
below
However,
their
allocated
percentages
are
not
on
the
system
to
consume
those
cycles.
In
the
early
their allocated percentages
not on the system to
In the early
years, the
the operating
operating dispa.rity
disparity in
in CPU
CPU use reflected
reflected a substantial
substantial difference
difference in demand
demand
years,
betwren the
the Stanford
Stanford community
community and
and the
the developing
developing national
national projects,
projects, rather
rather than
than
between
inequity of
of access.
access. For
For example,
example, the
the Stanford
Stanford utilization
utilization is spread
spread over
over a large
large part
part of
of the
the
inequity
24-hour cycle,
cycle, while
while national-AIM
national-AIM users tend
tend to
to be more
more sensitive
sensitive to
to local
local prime-time
prime-time
24-hour

A. Feigenbaum
Feigenbaum
E. A.

Resource Management
Management and Allocation
Allocation

5P41
5P41 RR00785-11

constraints.
(The 3-hour
3-hour time
time zone
zone phase
phase shift
shift across
across the continent
continent is
is of substantial
substantial help
constraints. (The
in load-balancing).
During
load-balancing).
During peak times under
under the overload
overload control
control system
system reported
reported
previously,
previously, the Stanford
St,anford community
community experienced mutual
mutual contentions
contentions and delays while
while the
Alr..l
AIM group
groap had relatively
relat.ively open access
access to the system.
This
This disparity
disparity in usage
usage has disappeared
disappeared in recent
recent years with
with the growth
growth of the
national
nat)ional user community,
community, and we enabled overload
overload controls
controls for
for the national
national community
community as
as
well. For
For the present.
present, we propose to continue
continue our
our policy
policy of "soft"
“soft” allocation
allocation enforcement
enforcement
for
for the fair
fair split
split of resource capacity.
capacity.
Our
Our system
system also categorizes
categorizes users
use,rs in terms
terms of access
access privileges.
privileges. These comprise
comprise
fully-authorized
users,
pilot
projects,
associates,
guests,
and
network
fully-authorized users, pilot projects, associates,
network visitors
visitors in descending
order
order of system
syst.em capabilities.
capabilities. \Ve
We want
want to encourage bona fide medical
medical and health
health research
people to experiment
experiment with
with the various
various programs
programs available
available with
with a minimum
minimum of red t.ape,
tape,
while
users to bypass the advisory
while Dot
not allowing
allowing unauthenticated
unauthenticated
advisory group
group screening
procedures
procedures by
by coming
coming on as
as guests. So far,
far, we have had relatively
relatively little
little abuse compared
compared
to that
that experienced
experienced by
by other
other network
network sites, perhaps
perhaps because of the personal
personal attention
attent,ion
directed
direct’ed by senior
senior staff
staff to logon records,
records, and to other
other security
security measures. However,
However, the
experience of most
most other
other computer
computer managers
managers behooves us to be cautious
cautious about
about being as
as
wide
wide open as
as might
might be preferred
preferred for
for informal
informal service to pilot
pilot efforts
efforts and demonstrations.
demonstrations,
\Ve
!Ye will
will continue
continue developing
developing this
this mechanism
mechanism in conjunction
conjunction with
with management
management committee
committee
policy
decisions.
policy
We
We also have encouraged
encouraged mature
mature projects
projects to apply
apply for
for their
their own
own machine
machine resources
in order
order to preserve
preserve the SUMEX-AIM
SUMEX-AIM resource for
for research and development
development efforts
efforts and
to
justify their
t,o support
support projects
projects unable
unable to justify
their own machines. The
The Rutgers
Rutgers resource has its
own
part of
project
own 2050
2060 machine,
machine, part
of which
which is allocated
allocated for
for AIM
AIM use,
use, and the CADUCEUS
CADUCEUS project
has instalit'd
planned development
program
installed a.a. VAX
V.4X 11/780
11/780 machine
machine to
t,o support
support its
its planned
development and program
test,ing work.
“Simulation of
of Cognitive
Cognitive Processes"
Processes” Project
Project
work. Profs.
Profs. Lesgold
Lesgold and Greeno's
Greeno’s "SimUlation
has
tire ly to their
h3.s moved
moved en
ent.irely
their own
own local VAX.
VAX.

A. Feigenbaum
Feigenbaum
E. A.

5P41 RR00785-11
RR00785- 11

Dissemination
Dissemination Efforts
Efforts

I.E.
Efforts
I.E. Dissemination
Dissemination
Efforts
Throughout
Throughout its existence, SUMEX-AIM
SUMEX-AIM has expended substantial
substantial effort
effort toward
toward
disseminating
disserninating information
information about
about its activities
activities as
as aa resource and about
about the work
work of
individual
We continue
continue to make many
many presentations
presentations at
individual collaborative
collaborative projects.
projects.
professional
professional meetings,
meetings, to provide
provide services to demonstrate
demonstrate developed AI
AI programs
programs to
interested
visitors, and to work
work in organizing
organizing
interested groups
groups and individuals,
individuals, to welcome visitors,
workshops
workshops within
within the SUMEX-AIM
SUMEX-AIM community
community to introduce
introduce our
our research to collaborating
collaborating
professional
We also directed
professional communities.
communities.
directed considerable
considerable effort
effort in the past
past toward
toward
working
working with
with the Research Resources Information
Information Center
Center to produce
produce the "Seeds
“Seeds of
Artificial
Art,ificial Intelligence"
Int.elligence” monograph
monograph and other
other publications
publications and press articles
articles to address aa
broader
broader community
community of technical
technical and lay people.

Software
Software Distribution
Distribution
SUr-.IEX
SUMEX continues
continues to support
support various
various projects
projects in the distribution
distribution of
of versions
versions of
their
their sonware
soft,wa.re to requesting
requesting individuals
individuals or
or groups.
groups. Following
Following is a summary
summary of
of software
software
dissemination
dissemination this
this past
past year:
year:
EMYCIN
EMYCIN

Both
“source” versions
versions of the EMYCIN
EMYCIN
Both the "executable"
“executable” and "source"
distribution
package were restructured
for
distribution
restructured
for clarity
clarity and ease
ease of
installation.
Thirty
copies of the EMYCIN
package have been
inst,allation.
Thirty
EMYCIN
generated
generated for
for distribution
distribution of which
which about
about 6 were sources only.
only. An
An
Interlisp-VAX
version
Interlisp-VAX
version of
of EMYCIN
EMYCIN is now available,
available, thanks
thanks to Ray
Ray
Bates of
of USC-lSI,
USC-ISI, who
who did
did the conversion.
conversion. This
This runs
runs under
under UNIX
UNIX and
VMS.
VMS.

AGE
AGE

Twenty-two
Nearly
Twenty-two copies of
of the AGE
AGE system
system have been distributed.
distributed.
Nearly
half
half of
of these have been copies requested
requested in ANSI
ANSI format
format indicating
indicating they
they
were evidently
evidently going
going to
to non-Tops20
non-Tops20 sites (probably
(probably Vaxes). As with
with the
EMYCIN
EMYCIN system,
system, Ray
Ray Bates
Bates at
at USC-lSI
USC-IS1 has converted
converted AGE
AGE to
to run
run
under
AX. A
under Interlisp-V
Interlisp-VAX.
A version
version is also available
available for
for the Xerox
Xerox 1108
series Lisp
Lisp workstations.
workstations.

GENET
GENET

In conjunction
phaseout of
conjunction with
with the phaseout
of the GENET
GENET community
community on
SUMEX,
programs and databases
SUMEX, a software
software package comprised
comprised of
of programs
developed by
by researchers
researchers at
at Stanford
Stanford and elsewhere was assembled for
for
developed
distribution to
to interested
interested GENET
GENET users. Versions
Versions of
of the software
software were
distribution
provided for
for use on both
both DEC-10
DEC-lO and DEC-20
DEC-20 systems
systems operating
operating under
under
provided
TOPS-lO, TENEX.
TENEX, and TOPS-20.
TOPS-20.
Installation procedures
procedures were
were
TOPS-lo,
Installation
documented, and a substantial
substantial amount
amount of
of telephone
telephone consultation
consultation was
documented,
provided. The
The package has been well-received
well-received and appears
appears to
to be in
provided.
active use at
at many
many of
of the 21 academic
academic sites to
to which
which itit was sent. Only
Only
active
copy of
of the
the complete
complete Genet
Genet system
system was set out
out in the
the past
past year.
year.
one copy
However, several
several sets of
of Genet
Genet related
related data
data files have been distributed.
distributed.
However,
This includes
includes several
several copies of
of the
the NIH
NIH and EMBL
EMBL Sequence
Sequence Libraries.
Libraries.
This
A limited
limited amount
amount of
of operations
operations support
support has been given
given to
to Brutlag’
Brutlag's
A
s
interaction with
with Sam
Sam Karlin
Karlin of
of the
the Math
Math department
department and a variety
variety of
of
interaction
other
groups.
other groups.

MRS
MRS

Twenty-two copies of
of MRS
MRS have
have been distributed
distributed through
through Sumex.
Sumex.
Twenty-two
Several others
others have also been distributed
distributed directly
directly by
by the
the HPP.
HPP. Most
Most
Several
have been sent
sent out
out to
to VAX/Unix
V A.XjUnix sites or
or Symbolics
Symbolics Lisp
Lisp machine
machine sites.
have

A. Feigenbaum
Feigenbaum
E. A.

Dissemination
Dissemination Efforts
Efforts

5P41 RR00785-11
RR00785-11
5P41

SACON
SACON

Two copies
copies of
of SACON
SACON have
have been
been prepared
prepared and
and distributed.
distributed.
Two

GLISP
GLISP

Two copies
copies of
of GLISP
GLISP were
were distributed.
distributed.
Two

67
67

E. A.
A. Feigenbaum
Feigenbaum
E.

Comments
Comments on
on the
the Biot,echnology
Biot,echnology Resources
Resources Program
Program

5P41
5P41 RR00785-11
RR00785- 11

I.F.
on
Resources
Program
I.F. Comments
Comments
on the
the Biotechnology
Biotechnology
Resources
Program
Resource
Resource Organization
Organization
We continue
continue to believe that
that the Biotechnology
Biotechnology Resources Program
Program isis one
one of
of the
most effective
vehicles
for
developing
and
disseminating
technological
tools
for
biomedical
effective
for developing and disseminating technological
for biomedical
research.
resea.rch. The
The goals
goals and
and methods
methods of
of the program
program are
are well-designed
well-designed to encourage
encourage building
building
of
groups
of the necessary multi-disciplinary
multi-disciplinary
groups and
and merging
merging of
of appropriate
appropriate technological
technological and
and
medical
medical disciplines.
disciplines. In
In our
our experience
experience with
with the SUMEX-AIM
SUMEX-AIM resource, several
several elements
elements of
of
this
this approach
approach seem
seem to emerge as
as key to the development
development and management
management of an effective
effective
resource:
1.
1. Effect,ive
Effective Management
Management Framework
Framework -- There
There needs
needs to be
be an explicit
explicit agreement
agreement

between the
t.he BRP
BRP and the resource principal
principal investigator
investigator which
which establishes aa
clear
c1ea.r mandate
mandate for the resource and its allocation,
allocation, provides
provides worthwhile
worthwhile
incentives
incentives for the host institution
institution and investigator
investigator to invest
invest the necessary
substantial
substantial professional
professional career time
time to develop and manage the resource, and
ensures equitable
equitable distribution
distribution of
of resource services to its target
target community.
community.
2.
2. Close \Vorking
Working Relationship
Relationship with
with the NIH
NIH -- A
A resource is a major
major and often
often
long-term
long-term investment
investment of
of money
money and human
human energy. A
A close
close and mutuallymutuallysupportive
supportive working
working relationship
relationship between
between resource management,
management, its advisory
advisory
committees,
is essential to assure healthy
NIH administration
administration
healthy
committees, and the NIH
development
development of
of the resource and its relationship
relationship to its user community.
community. We
We at
SUMEX-AHvl
have benefited
SUMEX-AIhl
benefited immensely
immensely from
from such a relationship
relationship with
with Dr.
Dr.
William
William R. Baker,
Baker, Jr.,
Jr., in the evolution
evolution of
of the SUMEX-AIM
SUMEX-AIM community.
community. We
We
look
beneficial relationship
look forward
forward to a continuing
continuing mutually
mutually beneficial
relationship with
with Dr.
Dr. Baker's
Baker’s
successor at the
NIH.
the NIH.
3. Freedom
by its
Freedom to Explore
Explore Resource Potential
Potential -- A
A resource, by
its nature,
nature, operates
operates
at the
“cutting edge"
edge” in developing
t)he "cutting
developing its
its characteristic
characteristic technology
technology and learning
learning
to effectively
effectively disseminate
disseminate itit to
to the
the biomedical
biomedical community
community at
at large. The
The BRP
BRP
t,o
should not
not impose artificial
artificial constraints
constraints on the
the resource for
for commercializing
commercializing its
its
should
efforts (fees for
for service)
service) or
or developing
developing its
its potential
potential (funding
(funding duration
duration limits
limits or
or
efforts
annual budget
budget ceilings).
ceilings).
artificial policy
policy impositions
impositions can serve to
to
annual
Such artificial
lIndermine the very
very goals central
central to
to the
the BRP’
BRP's
for existence.
existence.
llndermine
s reason for
Satisfactory policies
policies in this
this regard
regard have been worked
worked out
out and
and should
should be
Satisfact.ory
retained.
retained.

Electronic Communications
Communication,Oj
Electronic
SUMEX-AIM has pioneered
pioneered in developing
developing more
more effective
effective methods
methods for
for facilitating
facilitating
SUMES-AIM
scientific communication.
commllllication. Whereas
\Vhereas face-teface
face-to-face contacts
contacts continue
continue to
to play
playa
key role,
role, in
in
scient.ific
a key
the longer-term
longer-term we feel that
that computer-based
computer-based communications
communications will
will become increasingly
increasingly
the
important to
to the
the NIH
NIH and
and the
the biomedical
biomedical community.
community. We
We would
would like
like to
to see
see the
the BRP
BRP take
take
important
more adive
active role
role in promoting
promoting these tools
tools within
within the
the NIH
NIH and
and its
its grantee
grantee community.
community.
aa more

E. A.
A. Feigenbaum
Feigenbaum
E.

68
68

sp·a RROO785-11
RR00785-11
5P-11

Description
Description of
of Scientific
Scientific Subprojects
Subprojects

ll. Description
Description of
of Scientific
Scientific Subprojects
Subprojects
II.
il.A. Scientific
Scientific Subprojects
Subprojects
ILA.
The following
following subsections
subsections report
report on the
the AIM
AIM community
community of
of projects
projects and “pilot”
"pilot"
The
efforts including
including local
local and national
national users of
of the
the SUMEX-AIM
SUMEX-AIM facility
facility at
at Stanford.
Stanford.
efforts
However, those projects
projects admitted
admitted to
to the
the National
National AIM.
AIM community
community which
which use the
the RutgersRutgersHowever,
AIM resource
resource as their
their home base are not
not explicitly
explicitly reported
reported here.
AIhf
addition to
to these det;ailed
detailed progress
progress reports,
reports, abstracts
abstracts for
for
In a.ddition
individual users are submitted
submitted on a separate
separate Scientific
Scientific Subproject
Subproject
individual
have included
included here briefer
briefer summary
summary abstracts
abstracts of
of the
the fully-authorized
fully-authorized
B on page 209.
B

project and its
its
each project
Form. However,
However, we
Form.
projects in
in Appendix
Appendix
projects

The collaborative
collaborative project
project reports
reports and comments
comments are the
the result
result of
of a solicitation
solicitation for
for
The
contributions sent
sent to
to each of
of the
the project
project Principal
Principal Investigators
Investigators requesting
requesting the
the following
following
contriblltions
information:
information:
I. SUMhIARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM

A. Project
Project rationale
rationale
A.
Medica.l relevance
relevance and collaboration
collaboration
B. Medical
Highlights of
of research
research progress
progress
C. Highlights
--Accomplishments this
this past
past year
year
--Accomplishments
--Research in progress
progress
--Research
D. List
List of
of relevant
relevant publications
publications
E. Funding
Funding support
support
II. INTERACTIONS
\\11TH
INTERACTIONS
WITH THE
THE SUMEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
A. Medical
Medical collaborations
collaborations and program
program dissemination
dissemination via
via SUMEX
SUMEX
B. Sharing
Sharing and interactions
interactions with
with other
other SUMEX-AIM
SUMEX-AIM projects
projects
(via
(via comp1lting
computing facilities,
facilities, workshops,
workshops, personal
personal contacts,
contacts, etc.)
etc.)
C. Critique
Critique of
of resource management
management
(community
(community facilitation,
facilitation, computer
computer services, communications
communications
services, capacity,
capacity, etc.)
etc.)
III.
III. RESEARCH
RESEARCH PLANS
PLANS
A. Project
Project goals and plans
--Near-term
--Near-term
--Long-range
--Long-range
B. .Justification
.Justifica.tion and requirements
requirements for
for continued
continued SUMEX
SUMEX use
use
C. Needs
Nerds and plans for
for other
other computing
computing resources beyond
beyond SUMEX-AIM
SUMEX-AIM
D. Recommendations
Recommendations for future
future community
community and resource development
development
\'Ve
We believe that
that the reports
reports of the individual
individual projects
projects speak for
for themselves
themselves as
as
case, the reports
reports are recorded
recorded as
as submitted
submitted and are
rationales
In any case,
rationales for
for participation.
participation.
The only
only exceptions
exceptions are the respective
respective
the responsibility
responsibilit#y of the indicated
indicated project
project leaders. The
lists of relevant
relevant publications
publications which
which have been uniformly
uniformly formatted
formatted for parallel
parallel reporting
reporting
on the Scientific
Scientific Subproject
Subproject Form.
Form.

69
59

Feigenbaum
E. A. Feigenbaum

Stanford Projects
Projects
St.anford

RR00785-11
5P41 RR00785-11

II.A.!. Stanford
Stanford Projects
Projects
II.A.l.
The following
following group
group of
of projects
projects is formally
formally approved
approved for
for access
access to
to the
the Stanford
Stanford
The
aliquot of
of the
the S
SUMEX-AIM
resource. Their
Their access
access is based on
on review
review by
by the
the Stanford
Stanford
alicluot
UXIEX-AIM resource.
Advisory Group
Group and
and approval
approval by
by Professor
Professor Feigenbaum
Feigenbaum as Principal
Principal Investigator.
Investigator.
,4dvisory
In addition
addition to
to the
the progress
progress reports
reports presented
presented here, abstracts
abstracts for
for each project
project and
and
In
its individual
individual users are submitted
submitted on a separate
separate Scientific
Scientific Subproject
Subproject Form.
Form.
its

E. A.
,4. Feigenbaum
Feigenbaum

EXPEX
EXPEX - Expert
Expert Explanation
Explanation Project
Project

5P41
5P41 RR00785-11

IT.A.1.1.
EXPEX
- Expert
Explanation
Project
II.A.l.l.
EXPEX
Expert
Explanation
Project

EXPEX
Project
EXPEX - Expert
Expert Explanation
Explanation
Project
Edward
M.D.,
Edward H.
H. Shortliffe,
Shortliffe,
M.D., Ph.D.
Ph.D.
Departments
of
Science
Departments
of Medicine
Medicine and
and Computer
Computer
Science
Stanford
Stanford University
University

I.
I.

SUM:MARY
OF
PROGRAM
SUMMARY
OF RESEARCH
RESEARCH
PROGRAM

A.
A. Project
Project Rationale
Rationale
EXPEX
EXPEX is
is not
not a single project
project but
but a combination
combination of
of efforts
efforts that
that are directed
directed at
basic
ba.sic issues
issues in the development
development of
of representational
representational schemes to facilitate
facilitate knowledge
knowledge
acquisition
The
The work
work includes
includes not
not only
only the study
study of
of fundamental
fundamental
acquisition and explanation.
explanation.
representational
represent,ations.! formalisms
formalisms but
but also the encoding
encoding of various
various types
types of
of knowledge,
knowledge, such as
as
causal information
information and user models. In
In addition,
addition, to complement
complement these research directions,
directions,
the project
project has served as
as the focus for
for preparing
preparing three
three books on medical
medical computing
computing
research.
vVe
We believe that
that the productivity
productivity of basic computer
computer science research tends
tends to be
heightened
heightened by experiments
experiments that
that deal with
with significant
significant real world
world problem
problem domains.
domains.
Challenges
biology have introduced
Challenges drawn
drawn from
from chemistry,
chemistry, medicine,
medicine, and molecular
molecular biology
introduced
additional
but have simultaneously
addition4 complexity
complexity to
to expert
expert systems
systems work
work at
at Stanford,
Stanford, but
simultaneously forced
forced
system
developers
to
respond
to
pragmatic
constraints
and
user
demands
system developers to respond to pragmatic constraints
demands that
that have had a
significant
significant impact
impact on the basic AI
AI techniques
techniques selected or
or developed.
developed. Thus,
Thus, we believe that
that
creative
by working
creative investigation
investigat,ion into
into symbolic
symbolic reasoning
reasoning techniques
techniques is facilitated
facilitated by
working in real
world
Much
world settings
sett,ings where
where the application
application forces us to
to avoid
avoid oversimplification.
oversimplification.
Much of
of our
our
research effort
effort therefore
therefore deals with
with medical
medical domains
domains (viz.,
(viz., endocrinology
endocrinology and renal
renal
pathophysiology).
pathophysiology).

Relevance and
B. lvfediral
Medical Relevance
and Collaboration
Collaboration
Our
Our interest
interest, in explanation
explanation derives
derives from
from the insights
insights we gained
gained in developing
developing
explanatory ca.pabi!ities
capabilities for
for the
the MYCIN
MYCIN system.
system.
In the
the case
case of
of MYCIN
MYCIN and
and its
its
explana.tory
In
descendents, we have been able to
to generate
generate intelligible
intelligible explanations
explanations by
by taking
taking advant,age
advantage
descendents,
of its
its rule-based
rule-based representation
representation scheme. Rules can be translated
translated into
into English
English for
for display
display
of
to
a
user,
their
interactions
explicitly
demonstrated.
By
adding
and
their
interactions
can
also
be
explicitly
demonstrated.
to
By adding
mechanisms for
for understanding
understanding questions
questions expressed in simple
simple English,
English, we were
were able to
to
mechanisms
create an interactive
interactive system
system that
that allowed
allowed physicians
physicians to
to convince
convince themselves
themselves tha.t
that they
they
create
with t,he
the basis for
for the
the program’
program's
recommendations.
The limit,ations
limitations of
of the
the
agreed with
s recommendations.
The
explanations generated
generated in this
this way
way have become increasingly
increasingly obvious,
obvious, however,
however, and have
expIa.nations
to improved
improved characterization
characterization of
of the
the kinds
kinds of
of explanation
explanation capabilities
capabilities that
that must
must be
led t.o
developed ifif clinical
clinical consultation
consultation systems
systems are to
to be accepted
accepted by
by physicians.
physicians. The
The potential
potential
developed
of workstation
workstation graphics
graphics as a means of
of avoiding
avoiding natural
natural language
language issues
issues in
in the
the
use of
explanation process is also an area
area of
of great
great promise
promise with
with which
which we are currently
currently
exp!a.nat.ion
experimen ting.
experimenting.
\Vith these motivations
motivations in mind,
mind, we are involved
involved in a series of
of research
research projects
projects
With
that address medical
medical knowledge
knowledge representation
representation and
and explanation.
explanation. The
The individual
individual projects
projects
that
include
the
following:
include
following:

A. Feigenbaum
Feigenbaum
E. A.

EXPEX
EXPEX - Expert
Expert Explanation
Explanation Project
Project

5P41
5P4 1 RR00785-11
RROO785-11

1.
1. Mr.
hlr.

Greg
NESTOR program
program uses
Greg Cooper's
Cooper’s NESTOR
uses a detailed
detailed knowledge
knowledge base
regarding
pathophysiologic relationships
The
program is
regarding pathophysiologic
relationships in hypercalcemia.
hypercalcemia.
The program
designed to
physician's
proposed explanation
to critique
critique a physician’
s hypothesis
hypothesis regarding
regarding a proposed
expla.nation
for
patient manifestations
for a set of
of patient
manifestations when
when an elevated
elevated serum
serum calcium
calcium has been
observed.
particular interest
Of particular
interest is the techniques
techniques Cooper
Cooper has developed
developed for
for
observed. Of
using knowledge
knowledge of
of causality
causality to
to avoid
avoid the assumption
assumption of
of conditional
conditional
independence
independence commonly
commonly used in Bayesian
Bayesian diagnosis
diagnosis systems.

2. Mr.
pathophysiology,
Mr. John Kunz
Kunz has represented
represented the
the knowledge
knowledge of
of renal
renal pathophysiology,

including
including the quantitative
quantitative relationships
relationships that
that characterize
characterize the way
way in which
which the
body manages water
body
water and electrolytes,
electrolytes, to develop
develop a consultation
consultation and analysis
analysis
system
system (AI/MM)
(AI/MM) that
that melds mathematics
mathematics and AI
AI techniques.
techniques.
3. Building
Building on his earlier
earlier experience
experience with
with developing
developing an explanation
explanation capability
capability for
for

NEOM YCIN (in
project members
NEOMYCIN
(in collaboration
collaboration with
with the GUIDON
GUIDON project
members as
as outlined
outlined
elsewhere in this
this report),
report), Dr.
Dr. Glenn
Glenn Rennels has begun to
to work
work on a new
system
syst*em that
that uses
uses knowledge
knowledge of
of medicine
medicine to help
help formulate
formulate and resolve complex
complex
decision
better
decision analyses. Convinced
Convinced that
that decision
decision analytic
analytic techniques
techniques would
would be better
accepted
physician were to
accepted in medicine
medicine if
if the physician
to interact
interact with
with a knowledge-based
knowledge-based
interface
int)erface (rather
(rather than
t,han with
with the
the decision
decision trees themselves),
themselves), Dr.
Dr. Rennels has
made use of
"influence
diagrams"
as
a
central
method
central method for
for guiding
guiding the
the
of “influence diagrams” as
interaction.
The
explanation
issues
become
especially
evident
when
interaction. The explanation issues
especially evident when an analysis
analysis
is complete
complete and his system
system needs to
to generate
generate a defense for
for the recommendation
recommendation
it
it has made.
4. Mr.
Mr. Curt
Curt Langlotz
La.nglotz has continued
continued to
to work
work on a hypothesis
hypothesis assessment module
module

for
program uses
This program
uses a critiquing
critiquing model
model which
which
for the ONCOCIN
ONCOCIN system.
system. This
inherently
inherently involves
involves advanced
advanced explanation
explanation techniques.
techniques. The
The work
work uses
uses the
the Xerox
Xerox
1108 professional
professional workstation
workstation (Dandelion)
(Dandelion) and is further
further described
described in the
ONCOCIN
portion of
ONCOCIN Project
Project portion
of this
this annual
annual report.
report.
S.
project using the Xerox
.5. During
During 1983, Ms. Shako
Shoko Tsuji
Tsuji completed
completed a project
Xerox workstation
workstation
to experiment
experiment with
with graphical
graphical techniques
techniques for
for examining,
examining, manipulating,
manipulating,
expanding,
base. Also
expanding, and editing
editing a large medical
medical knowledge
knowledge base.
Also working
working in the
context
of
ONCOCIN,
her
code
was
designed
for
use
by
knowledge
context of ONCOCIN, her
for use by knowledge engineers.
The
building an interface
The work
work has inspired
inspired subsequent
subsequent work
work in building
interface for
for the nonprogrammer clinician
protocols in the
programmer
clinician who
who wishes to
to write
write and test
t,est new protocols
ONCOCIN
The
project is described
ONCOCIN environment.
environment.
The project
described in greater
greater detail
detail in the
ONCOCIN
portion of
ONCOCIN portion
of the annual
annual report.
report.
To
prepared two
booles on
To complement
complement, these basic research activities,
activities, we have
hat-e prepared
two boo!cs
Artifidal
beginning work
Artificial Intelligence
Intelligence in Medicine
Medicine and are beginning
work on a third
third (see
(see Section
Section C for
for
details).
details).

C.
Highlights of
Research Progress
Progress
G. Highlights
o j Research

C.l
NESTOR
CT.1The NI3’
TOR System
Sydem
NESTOR is intended
patient data
NESTOR
intended to allow
allow a user to
t’o input
input patient
data plus a hypothesis,
hypothesis, and
then
then have the system
system critique
critique that
that hypothesis
hypothesis in light
light of
of the data.
data. The
The system,
system, an evolving
evolving
thesis
project that
t.hesis project
that is largely
largely the work
work of
of Mr.
Mr. Greg
Greg Cooper,
Cooper, relies on basic associational
associational
information
part from
but supplemented
information drawn
drawn in part
from the INTERNIST-J
INTERNIST-I knowledge
knowledge base but
supplemented with
with
causal and temporal
t.emporal associations.
associations.
The
behind this
physicians want
The motivation
motivation behind
this research
research is the conviction
conviction that
that physicians
want active
active

E. A. Feigenbaum
Feigenbaum

EXPEX
EXPEX - Expert
Expert Explanation
Explanation Project
Project

5P41
5P41 RR00785-11
R.R00785-11

control
cont,rol of the diagnostic
diagnostic process and that
that they
they also want
want and need aa system
system that
that explains,
explains,
in aa user-tailored
user-tailored way, its evaluation
evaluation of the physician's
physician’s hypothesis.
hypothesis. There
There may
may be
be times
times
when the user wants
wants to give complete
complete control
control to NESTOR
NESTOR and just
just be
be in aa mode of
answering
answering questions, but
but we feel
feel that
that this
this should be
be an option
option and not
not aa requirement.
requirement. It
It
is
is observations
observations such as
as these that
that have also accounted
accounted for
for the hypothesis
hypothesis assessment
assessment work
work
underway
underway in the ONCOCIN
ONCOCIN research, briefly
briefly mentioned
mentioned above and further
further described
described in
the section
section of this
this report
report dealing
dealing with
with that
that project.
project.
The
The initial
initial NESTOR
NESTOR system
system is
is now largely
largely complete
complete and is undergoing
undergoing evaluation
evaluation
at this
time.
Of
particular
interest
is
the
adequacy
of
the
techniques
Of particular interest
techniques developed for
this time.
allowing
allowing NESTOR
NESTOR to avoid
avoid the traditional
traditional assumption
assumption of conditional
conditional independence used in
Bayesian systems.
syst,ems. Also, because
because NESTOR's
NESTOR’s probabilistic
probabilistic model is
is more formal
formal than
than the
ad hoc scheme used
used in, say, INTERNIST,
INTERNIST, the assumptions
assumptions made by our
our system
system are more
explicit.
explicit.
\Ve have also developed search techniques
techniques that
that allow
allow NESTOR
NESTOR to
to explore
explore
efficiently
efficiently a very
very large search space in order
order to find
find the most
most probable
probable (multiple
(multiple disease)
disease)
hypothesis.
hypothesis. This
This technique
technique is
is general and can be applied
applied to many
many nonmedical
nonmedical problems
problems
where
where the goal is to find
find the most
most probable
probable hypothesis
hypothesis among many
many possibilities.
possibilities.

C.2
C.2 Integrating
Integrating ~Mathematical
Mathematical Models
Models with
with AI
AI Afethods
Methods
This
This research project,
project, known
known as
as AI/MM,
AI/MM, is the dissertation
dissertation research of
of Mr.
Mr. John
Kunz.
The
The system
system int,egrates
integrates AI
AI and simple
simple mathematics
mathematics to analyze a physiological
physiological
Kunz.
model.
model. In a selected medical
medical domain
domain (renal
(renal physiology),
physiology), we have built
built a computer
comput,er
program
It
program based on these techniques.
techniques.
It analyzes physiological
physiological behavior,
behavior, diagnoses
abnormality,
program fits
abnormality, and explains
explains the rationale
rationale for
for its
its analyses. The
The program
fits data
data to the
model.
model, identifies
identifies whether
whether the data
data are abnormal,
abnormal, and identifies
identifies the possible causes
causes and
effects
The
physiological model
effects of
of any
any abnormalities.
abnormalities.
The physiological
model is based on knowledge
knowledge about
about
anatomy,
physiological system,
anatomy, the hehavior
heha,vior of
of the physiological
system, and the mechanism
mechanism of
of action
action of
of the
system.
problems discussed
system. It's
It’s validity
va,lidity has been tested
tested by
by having
having it
it analyze many
many of
of the problems
in Valtin's
Renal Function.
Function.
Va.ltin‘s text
text IZellal
The
project have been to:
The specific
specific aims of
of this
this project
1.
physiological model.
1. Dev('lop
Develop a vocabulary
vocabulary for
for a physiological
model.

The
The vocabulary
vocabulary represents
represents
the "basic
physiology" of
“ba,sic physiology”
of a biological
biological system
system and appears to be adequate to
the concepts
concepts included
introductory professional-level
professional-level physiology
physiology
express Lhe
included in an introductory
text.
text.

Develop a reasoning
reasoning system
system which
which can solve problems
problems expressed
expres;~ed in the
2. Develop
vocabulary.
vocabul:try.
Demonstrate the basic necessity, appropriateness
appropriateness and limitations
limitations of
of the
3. Demonstrate
vocabulary and reasoning
reasoning procedure.
procedure.
vocabulary
C.3
Knowledge-Based Explanations
Explanations in
in a Decision
Decision Analysis
Analysis Environment
Environment
Cr.3 Kno,ufledge-Based
This new project,
project, thesis research by
by Dr.
Dr. Glenn
Glenn Rennels, is motivat,ed
motivated by
by the
This
observation that
that AI
AI techniques
techniques could
could greatly
greatly facilitate
facilitate a user’
user's
effort to
to specify
specify the
the det.ails
details
observation
s effort
of a complex
complex clinical
clinical decision
decision task
task and
and to
to seek assistance with
with that
that task.
task. Although
Although
of
decision theoretic
theoretic notions
notions have been shown
shown to
to be relevant
relevant to
to such medical
medical problems,
problems, they
they
decision
largely been unused by
by clinicians,
clinicians, even when
when computer-based
computer-based solutions
solutions have been
have largely
offered. \Ve
\Ve believe
believe that
that an intelligent
intelligent system
system should
should be able to
to guide
guide the
the definition
definition of
of the
the
offered.
decision t.ask
task and explain
explain the
the results
results of
or the
the analysis
analysis without
without requiring
requiring that
that a user be
decision
familiar wit,h
with the
the underlying
underlying decision
decision analytic
analytic techniques
techniques being
being used to
to solve the
the problem.
problem.
familiar

A. Feigenbaum
Feigenbaum
E. A.

EXPEX
EXPEX - Expert
Expert Explanation
Explanation Project
Project

5P41 RR00785-11
RR00785-11

The
The basic notion
notion is to
to use directed
directed graphs,
graphs, termed
termed "influence
“influence diagrams"
diagrams” as
as a
language for
physician at
for communication
communication with
with a physician
at a graphical
graphical display
display terminal.
terminal.
Nodes in
these graphs
by the user who
graphs are defined
defined by
who is seeking advice, and their
their structure
structure and
meaning
meaning is largely
ia.rgely intuitive.
intuitive. The
The task
task of
of converting
converting influence
influence diagrams
diagrams to
to decision
decision trees is
a knowledge-based
problem that
potentially well-suited
knowledge-based problem
that is potentially
well-suited for
for a solution
solution that
that uses
uses AI
AI
methods.
Similarly, the
the results
results of
of a decision
decision analysis,
analysis, including
including the sensitivity
sensitivity analysis,
analysis,
methods. Similarly,
will
physician user in terms
will need to
to be explained
explained to
to the physician
terms of
of influence
influence diagrams
diagrams and
knowledge
The
being
The necessary knowledge
knowledge structures
struct)ures are currently
currently being
knowledge of
of the domain.
domain.
designed, and an early
prototype system
The
The research uses
uses a 9835
9836
early prototype
system is operational.
operational.
workstation
by Hewlettworkstation donated
donated to the Medical
Medical Information
Information Sciences
Sciences Training
Training Program
Program by
HewlettPackard
Packard Company
Company and soon to
to be networked
networked to
to the SUMEX
SUMEX 2060.

C.4 Books
Books on Medical
Afedical Arti
Artificial
Intelligence and
Afedical Computing
jicial Intelligence
and Medical
Computing
\Ve
both of
published
1Ve have completed
completed two
two books, both
of which
which are in press and due to be published
in mid-1984:
mid- 1984:
W.J.
E.H.
Readings in
Medical Artificial
Artificial
W.J. and Shortliffe,
Shortiiffe,
E.H. Readings
in Medical
Intelligence: The First
First Decade. Reading,
Reading, MA:
MA: Addison-Wesley,
Addison-Wesley, 1984.
Intelligence:

•0 Clancey.
Clancey,

Rule-Based Ezpert
Expert Systems:
Rule-Based
Systems: the
.MYCIN
Experiments of
Heuristic Program.ming
Programming Project.
Project.
h,fi%IN Experiments
o j the Stanford
Stan ford Heuristic
Reading.
Reading, MA:
MA: Addison-\Vesley,
Addison-Wesley. 1984.

•l Buchanan,
Buchanan, \V.J.
W.J. and Shortliffe,
Shortiiffe, E.H.
E.H.

In addition,
just begun work
beginning to
addition, we have just
work on a textbook
textbook for
for students
students beginning
t.o study
study
medical
This
medical computing
computing and artificial
artificial intelligence.
intelligence.
This multi-authored
multi-authored volume
volume should
should be
completed
by the end of
publication date is contemplated.
completed in draft
draft form
form by
of 1984. A
A 1985 publication
contemplated.
•l Shortliffe,
An Introduction
Introduction to
Short.iiffe, E.H.,
E.H., \Viederhold,
Wiederhold, G.C.M.,
G.C.M., and Fagan,
Fagan, L.M.
L.M. An
Afedical Computer
preparation).
Rfedical
Computer Science.
Science. Reading,
Reading, MA:
MA: Addison-Wesley
Addison-Wesley (in
(in preparation).

D. Publications
Publications Since
January 1989
D.
Since January
1983
1. Shortliffe,
Shortliffe, E.H.
E.H.

Medical
In
Medical consultation
consultation systems:
systems: designing
designing for
for doctors.
doctors.
In
Dcsigning
for
Human-Computer
Communication
(M.S.
Sime
and
M.J.
Designing for Human-Computer
Communication
(M.S.
M.J.
Coombs.
Coombs. eds.),
eds.). Chapter
Chapter 8, pp. 209-238, London:
London: Academic
Academic Press, 1983.

2. Shortliffe,
Shortliffe, E. H. Medical
Medical Cybernetics:
Cybernetics: The
The Challenges
Challenges of
of Clinical
Clinica. Computing.
Computing.
In
Tcchnology
International
Stability,
and
Growth,
S.
Basheer
Ahmed
In Tech,nology International
Stability, and Growth, S.
Ahmed and
Alice
Alice P. Ahmed,
Ahmed, editors;
editors; Chapter
Chapter 12, pp. 148-165; Associated
Associated Faculty
Faculty Press,
Inc.,
New York,
Inc., Port, \Vashington,
Washington, New
York, 1984.
3. (*)
(*) Shortliffe,
Shortliffe, E.H.
E.H. and Fagan,
Fagan, L.M.
L.M. Expert
Expert systems
systems research: modeling
modeling the
medical
An Integrated
Integrated Approach
Approach to Monitoring
Afonitoring
In An
medical decision
decision making
making process. In
(J.S. Gravenstein.
Newbower, A.K.
N.T. Smith,
Gravenstein, R.S. Newbower,
.4.K. Ream, and N.T.
Smith, eds.), pp.
183-200, Woburn,
Woburn, MA:
hIA: Butterworth's,
Butterworth’s, 1983.
1983.
'1.
4. Duda,
Duda,

R.O.
R.O. and Short-Iiffe,
Shortliffe,
220:251-268
220:261-368 (1983).

E.H.
E.H.

Expert
Expert systems
systems

research.

Science,
Science,

5. (*)
(*) Langlotz,
Langiotz. C.P.
C.P. and Shortliffe,
Shortliffe, E.H.
E.H. Adapting
Adapting a consultation
consultation system
system to
to
critique
plans.
International Journal
Journal of
Studies,
critique user plans.
International
o j lI.-fan-Afachine
Man-Muchine
Studies,
19>179-495
19:479-496 (1983)
(1983)

6. Shortliffe,
Hypothesis
Shortliffe, E.H.
E.H.
Hypothesis generation
generation in medical
medical consultation
consultation systems:
systems:
art·ificial
In lI.-fEDINFO
83 (J.H.
artificial intelligence
intelligence approaches.
approaches.
MEDINFO
(J.H. van
van Bemmel,
Bemmel,
M.
North Holland,
M. Ball,
Ball, and
a.nd O.
0. Wigertz,
Wigertz, eds.),
eds.). pp. 480-483, North
Holland, Amsterdam,
Amsterdam, 1983.
E. A. Feigenbaum
Feigenbaum

EXPEX
EXPEX -- Expert
Expert Explanation
Explanation Project
Project

5P41
5P4 1 RR00785-11
RR00785- 11

7.
Graphical access
access to
to the
the knowledge
knowledge base
base of
of aa
7. (*)
(*) Tsuji,
Tsuji, S.
S. and
and Shortliffe,
Shortliffe, E.H.
E.H. Graphical
medical
consultation
system.
Proceedings
of
AA
.
A'[SI
Congress
83,
Congrese 83, pp.
pp.
Proceedings of A4MSI
medical consultation system.
551-555,
San
Francisco,
Ca
.
,
May
1983.
551-555, Ssn Francisco, Ca., May 1983.
8.
The
In
The science
science of
of biomedical computing.
computing.
In A1eeting
hleetilzg the
the
8. Shortliffe,
Shortliffe, E.H.
E.H.
and Medical
Challenge:
Challenge: Informatics
In formatics
Medical Education
Education (J.C.
(J.C. Pages,
Pages, A.H. Levy,
Levy,
F.
F. Gremy,
Gremy, and
and J.
J. Anderson, eds.),
eds.), pp. 1-10,
l-10, Amsterdam:
Amsterdam: North-Holland,
North-Holland, 1983.
1983.
To
To be
be reprinted
reprinted ih
ih Medical
Medical Informatics,
Informatics, 1984.
1984.
9.
9. (*)
(*) Kllnz.
Kunz, J.C.,
J.C., Shortliffe,
Shortliffe, E.H.,
E.H., Buchanan,
Buchanan, B.G.,
B.G., and
and Feigenbaum,
Feigenbaum, E.A.
E.A.
Comparison
Comparison of techniques of computer-assisted decision
decision making
making in medicine.
In
111Pure and Applied
.4ppfied Biostructure
Biostructure (Claudio
(Claudio Niccolini,
Niccolini, Ed.), Singapore: World
World
Press.
Press, 1983.
1983.
10.
10. (*)
(*) Kunz, J.C., Shortliffe,
Shortliffe, E.H., Buchanan, B.G.,
B.G., Feigenbaum, E.A. ComputerComputerassisted
of Philosophy
Philosophy and A-fedicine,
Medicine,
assisted decision making
making in medicine. Journal
Journal of
Summer 1984
1984 (in press).
press).
11.
ll.‘(*)(*) HasJing,
Hssling, D. \V.,
W., Clancey, W.
W. J.,
a.a diagnostic
consultation
system.
diagnostic consultation
Studies,
Studies, Spring
Spring 1984
1984 (in press).
press).

and Rennels,
Rennels, G. Strategic
Strategic explanations
explanations for
International
Journal
of
International
Journal of Afan-Afachine
Man-Machine

12.
12. Shortliffe,
Shortliffe, E.H. Reasoning methods in medical consultation
consultation systems: artificial
artificial
intelligence
intelligence approaches (tutorial).
(t,utorial). Computer
Computer Programs
Programs in
in Biomedicine,
Biomedicine,
January
January 1984
1984 (in press).
press).

E. Funding
Funding Support
Support
Grant
Grant Title:
Title: "The
“The Development
Development of Representation
Representation Methods
Methods to Facilitate
Facilitate
Knowledge
Knowledge Acquisition
Acquisition and Exposition
Exposition in Expert
Expert Systems"
Systems”
Principal
Principa.1 Investigator:
Investigator: Edward
Edward H. Shortliffe
Shortliffe
Agency:
Naval Research; ID
Number: NR
NR 049-479
A4gency: Office
Office of
of Naval
ID Number:
049-479
Term: January
January 1981
to December 1983
Term:
1981 to
Total award:
award: $~456,622
$455.622
Total
Grant Title:
Title: “Research
"Research on Introspective
Introspective Systems”
Systems"
Grant
Principal Investiga.tor:
Investigator: M
Michael
Principal
ichael R. Genesereth
Agency: Office
Office of
of Naval
Naval Research; ID
ID Number:
Number: NR
NR 049-479
Agency:
Term: Jnnunry
January 1984
1984 to
to December 1986
Term:
Total award:
award: $312,070
$:312.070
Total
Title: “Information
"Information Structure
Structure and Use in Knowledge-Based
Knowledge-Based Expert
Expert
Grant. Title:
Systems"
Systems”
Principal Investigat.or:
Investigator: Bruce
Bruce G. Buchanan
Buchanan
Principa.1
Agency: Na.tionsl
National Science
Science Foundation;
Foundation; ID
ID Number:
Number: 83-12148
83-12148
Agency:
Term: h,larch
t-.larch 1984 to
to February
February 1987
Term:
Tota,l
award: $300,000
$300.000 (includes
(includes indirect
indirect costs)
Tot
al award:

ll.
II.

INTERACTIONS WITH
WITH THE
THE SUMIZX-AIM
SUMEX-AIM RESOURCE
RESOURCE
INTERACTIONS
A.. Medical
Afedical Collaborations
Collaborations and
and Program
Program Dissemination
Dissemination via
via SUMEX
SUAfEX
A.

None of
of t,hese
these new
new programs
programs is
is yet
yet ready
ready for
for disseminat,ion.
dissemination.
None

75
75

They are
are mostly
mostly
They

E. A.
A. Feigenbaum
Feigenbaum
E.

5P41 RR00785-11
RR00785- 11

EXPEX
EXPEX - Expert
Expert Explanation
Explanation Project
Project

fundamental
research experiments
experiments with
with limited
limited clinical
clinical usefulness
usefulness other
other than
than as
funda.mental
research
demonstration
demonstrat,ion projects.
projects. Our
Our past
past experience
experience has shown,
shown, however,
however, that
that SUMEX
SUMEX provides
provides
a superb
superb vehicle
vehicle for
for demonstrating
demonstrating systems,
systems, even at a dist.ance.
dist,ance.
The
The new book
book writing
writing eCfort
effort will
will in particular
particular be facilitated
facilitated by
by SUMEX,
SUMEX, much
much as
the AI
A1 Handbook
Handbook was in the
the past.
past. A
A multi-authored
multi-authored text
text of
of this
this type,
type, particularly
particularly one for
for
which
which the
the authors
authors are spread
spread at
at numerous
numerous different
different universities
universities around
around the country,
country,
would be a nightmare
nightmare to
to compile
compile if
if it
it were not
not for
for the
the SUMEX
SUMEX resource.
resource. Many
Many of
of the
the
would
of manuscript
manuscript
contributors to
to the
the book
book have been assigned SUMEX
SUMEX accounts
accounts for
for purposes
purposes of
contributors
preparation.
preparation. Online
Online manuscript
manuscript work
work through
through the
the shared
shared facility,
facility, coupled
coupled with
with messaging
messaging
capabi\it.ies,
capabilit,ies, will
will greatly
greatly enhance the
the efficiency
efficiency and
and accuracy
accuracy oC
of the developing
developing chapters
chapters
the editing
editing process.
and the

B.
Interaction with
UM Project8
Projects
B. Sharing
Sharing and
and Interaction
with Other
Other SUMEX-..
SUMEX-AIM

Although our
our EXPEX
EXPEX work
work is young,
young, we are already
already benefiting
benefiting Cram
from interactions
interactions
Although
with ot.her
other researchers
researchers who
who use
use the SUMEX-AIM
SUMEX-AIM
resource.
The NESTOR
work in
with
resource.
The
NESTOR work
particular
knowledge
particular has depended
depended on access
access to
to the
the INTERNIST-l
INTERNIST-l
knowledge base and on Crequent
frequent
exchange of
Similarly,
of messa.ges
messa.ges with
with the researchers
researchers at
at the
the University
University of
of Pittsburgh.
Pittsburgh.
Similarly, our
our
collaboration
collaboration with
with the
the GUIDON
GUIDON research
research team
team for
for the
the implementation
implementation of
of an explanation
explanation
capability
capability would
would not
not have been possible
possible without
without the
the facilitated
facilitated communication
communication and
and shared
shared
file access
access available
available via
via SUMEX.
SUMEX.
file
C. Critique
Resource Management
Atfanagement
Critique of
of Resource
SUf\IEX
Not
SUhIES continues
continues to
to provide
provide a superb
superb environment
environment for
for research
research of
of this
this kind.
kind. Not
only
only is the
the 201)0
2060 a well
well managed
managed resource
resource under
under Ed
Ed Pattermann's
Pattermann’s leadership,
leadership, but
but the
hypothesis
hypothesis assessment and graphical
graphical query
query systems
systems are dependent
dependent upon
upon access
access to
to high
high
performance
performance professional
professional workstations,
workstations, and
and we are delighted
delighted with
with the resources that
that
SUMEX
SUMEX has provided
provided us in this
this regard.
regard.
III.
RESEARCH
PLANS
ID.
RESEARCH
PLANS

A. Project
Project Goals
Plan8
A.
Goal.9 and
and Plans
\Ve
We anticipate
ant,icipate completion
completion of
of many
many of
of these basic research
research efforts
efforts during
during the
the coming
coming
year.
NESTOR work
year. Cooper's
Cooper’s NESTOR
work is largely
largely complete,
complete, and a thesis
thesis document
document is anticipated
anticipated
in June
June 1984. Similarly,
Similarly, Kunz
Kunz has completed
completed his work
work on AI/MM,
AI/MM, and
and his dissertation
dissertation is
approaching
approaching completion.
completion. Both
Both Cooper
Cooper and
and Kunz
Kunz have completed
completed their
their oral
oral examinations
examinations
this ",,"ork.
work.
on this
The
The project
project oC
of Tsuji
Tsuji is complete
complete and
and she has now
now lett
left StanCord.
Stanford. However,
However, the
the code
she developed
developed is being
being modified
modified Cor
for ongoing
ongoing use in the
the ONCOCIN
ONCOCIN environment.
environment.
The
The project
project oC
01 Langlotz
Langlotz continues
continues to
to be an active
active research
research effort
effort within
within the
the
ONCOCIN
His plan
ONCOC’IN project.
project.
plan for
for the
the coming
coming year
year is briefly
briefly outlined
outlined in the ONCOCIN
ONCOCIN
portion of
of this
this annual
annual report.
report.
portion
The
just getting
The work
work of
of Rennels,
Rennels, which
which is just
getting underway,
underway, will
will be better
better formulated
formulated by
by
next
next year
year at
at this
this time.
time. We
We expect
expect the
the project
project to
to last
last at
at least
least two
two more
more years,
years, however.
however.
The
The textbook
textbook preparation
preparation is scheduled
scheduled for
for completion
completion in approximately
approximately one year,
year,
with publication
publication anticipated
anticipated during
during H185.
1985.
with

E. A. Feigenbaum
Feigenbaum

76
76

EXPEX
EXPEX - Expert
Expert Explanation
Explanation Project
Project

5P41
5P4 1 RR00785-11
RROO785-11

B. Requirement8
Requirement.'} for
for Continued
Continued SUAfEX
SUMEX Use
All
All the work
work we are doing
doing is largely
largely dependent
dependent on the SUMEX
SUMEX re.,ource.
resource. The
The new
work
by the Medical
work of
of Rennels
Rcnnels is using Hewlett-Packard
Hewlett-Packard 9835
9836 workstations
workstations owned
owned by
Medical
Information
program, but
but Dr.
Information Sciences training
training program,
Dr. Rennels continues
continues to be dependent
dependent upon
upon
SUMEX
for
projects, only
SUhlEX
for communication
communication and collaboration.
collaboration. Of
Of the other
other projects,
only the
t,he hypothesis
hypothesis
assessment and graphical
projects are sufficiently
justify their
graphical query
query projects
sufficiently mature
mature to justify
bheir transfer
transfer to
to
one of
personal workstations,
of the SUMEX
SUMEX personal
workstations, so the new 2060 continues
continues to
to be a key
key element
element
in our
plan.
our research plan.
In addition.
benefits of
addition, we have long appreciated
appreciated the
the benefits
of GUEST
GUEST and network
network access
access to
the progrs.ms
programs we are developing.
SUMEX
greatly
enhances
our
ability
to
obtain
feedback
developing. SUMEX greatly
our ability
obtain feedback
from
physicians and computer
programs
from interested
interested physicians
computer scientists
scientists around
around the country.
country. As our
our programs
continue
cant inue to
t,o mature,
mat,ure, it
it will
will become increasingly
increasingly important
important that
that we be able to make them
them
available
by distant
ava.ilable for
for demonstration
demonstration and for
for access
access by
distant collaborators
collaborators via
via the SUrvfEX
SUhlEX
network.
ne twnrk.

c.
Requirements for
for Additional
Additional Computing
Resources
C. Requirement8
Computing Resources
The
provide a suitable
The mainframe
ma.inframe machine
machine should
should continue
continue to
to provide
suitable environment
environment for
for
most
plans to
NESTOR, or
We have no plans
to transfer
transfer NESTOR,
or
most of
of our
our work
work in the
the months
months ahead. We
AI/~'IM
AI/MM to other
other hardware
hardware soon.

D.
Recommwdation,'J for
for Future
Future Community
Resource Development
Development
D. Recommendations
Community and
and Resource
\Ve are very
provided since the
We
very satisfied
satisfied with
with the facilities
facilities SUMEX
SUMEX has provided
the upgrade
upgrade to
the
professional workstations
t,he DEC
DEC 2050.
2060. Other
Other than
than continued
continued acquisition
acquisition of
of professional
workstations that
that can be
shared
by some of
programs in this
projects, we have no requests
shared by
of the more
more mature
mature progra.ms
this set of
of projects,
for
for additional
addit,ional acquisitions
acquisitions or
or resource development
development at this
this time.
time.

77
77

E. A.
A. Feigenbaum
Feigenbaum

GUIDON/NEOMYCIN
Project
GUIDON/NEOhIYCIN
Project

5P41
5P41 RR00785-11

II.A.1.2.
GUIDON/NEOMYCIN
Project
II.A.1.2.
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
William
J.
Ph.D.
William
J. Clancey,
Clancey,
Ph.D.
Department
Computer
Science
Department
Computer
Science
Stanford
University
Stanford
University
Bruce
Ph.D.
Bruce G.
G. Buchanan,
Buchanan,
Ph.D.
Computer
Science
Computer
Science Department
Department
Stanford
University
Stanford
University

I.
I.

SUM?vlARY
OF
PROGRAM
SUMMARY
OF RESEARCH
RESEARCH
PROGRAM
A.
A. Project
Project Rationale
Rationale

The
Project
The GUIDON/NEOMYCIN
GUIDON/NEOMYCIN
Project is a research program
program devoted
devoted to the
development
development, of
of a knowledge-based
knowledge-based tutoring
tutoring system
system for
for application
application to medicine.
medicine. This
This work
work
derived
That research gave way
way to three
three
derived from
from our
our first
first system,
system, the MYCIN
MYCIN program.
program. That
sub-projects
sub-projects (EMYCIN,
(EMYCIN, GUIDON,
GUIDON, and ONCOCIN)
ONCOCIN) described
described in previous
previous annual
annual reports.
reports.
El\l)"CIN
EMYCIN has
haz been completed
completed and its resources reallocated
reallocated to other
other projects.
projects. GUIDON
GUIDON
and ONCOCIN
ONCOCIN have become projects
projects in their
their own
own right.
right.
The
project is to
program that
The key
key issue
issue for
for the GUIDON/NEOMYCIN
GUIDON/NEOMYCIN
project
to develop
develop a program
that
can provide
provide advice
by human
advice similar
similar in quality
quality to that
that given
given by
human experts,
experts, modeling
modeling how
how they
they
structure
problem solving
procedures. The
struct,ure their
their knowledge
knowledge as
as well
well as
as their
their problem
solving procedures.
The consultation
consultation
program
using
this
knowledge
is
called
NEOMYCIN.
NEOMYCIN's
knowledge
base,
program using this knowledge
called NEOMYCIN.
NEOMYCIN’s
knowledge base,
designed for
use
in
a
teaching
application,
will
become
the
subject
material
used
by a
for use
teaching application, will
the subject material
by
family
programs referred
The
problemfamily of
of instructional
instructional programs
referred to
to collectively
collectively as GUIDON2.
GUIDON2.
The problemsolving
procedures are developed
by running
NEOMYCIN and
solving procedures
developed by
running test
test cases
cases through
through NEOMYCIN
comparing
behavior. Also,
NEOMYCIN as
Also, we are using NEOMYCIN
as a test
test bed for
for the
the
compa.ring them
them to
t,o expert
expert behavior.
explanation
part of
programs.
explana.tion capabilities
capabilities that
that will
will eventually
eventually be part
of our
our instructional
instructional programs.
The purpose
purpose of
of the
the current
current contract,
contract, now
now in its
sixth of
of six
six years,
years, is to
to construct
construct an
The
its sixth
intelligent tut,oring
tutoring system
system t.hat
that teaches diagnostic
diagnostic strategies
strategies explicitSly.
explicitly. By
By strategy,
strategy, we
intelligent
plans for
for establishing
establishing a set of
of possible diagnoses, focusing
focusing on and
and confirming
confirming
mean plans
individual diagnoses, gat.hering
gathering data,
data, and
and processing
processing new data.
data. The
The tut,orial
tutorial program
program will
will
individual
capabilities to
to recognize
recognize these plans, as well
well as to
to articulate
articulate strategies
strategies in explanat,ions
explanations
have capabilities
about how
how to do diagnosis.
diagnosis. The
The strategies
strategies represented
represented in the program,
program, modeling
modeling
a.bout
techniqlles,
and
explanat.ion
techniques
are
wholly
separate
from
the
knowledge
base, so
techniques,
explanation techniques
wholly separate from the knowledge base,
can be used wit.h
with many
many medical
medical (and
(and non-medical)
non-medical) domains.
domains. That
That is, the target
target program
program
will be able t.o
to be tested
tested with
with other
other knowledge
knowledge bases,
bases, using
using system-building
system-building tools
tools that
that we
will
provide.
provide.

B. Medical
Jvledical Relevance
Relevance and
and Collaboration
Collaboration
B.
There is a growing
growing realization
realization that
that medical
medical knowledge,
knowledge, originally
originally codified
codified for
for the
the
There
purpose of
of computer-based
computer-based consultations,
consultations, may
may be utilized
utilized in additional
additional ways
ways that
that are
purpose
medically relevant.
relevant. Using
Using the
the knowledge
knowledge to
to teach
teach medical
medical students
students is perhaps
perhaps foremost
foremost
medically
among t,hese.
these, and NEOMYCIN
NEOMYCIN continues
continues to
to focus on methods
methods for
for augmenting
augmenting clinical
clinical
among
knowledge in order
order to
to facilitate
facilitate its
its use in
in a tutorial
tutorial setting.
setting. A
A particularly
particularly important
important
knowledge

A. Feigenba.um
Feigenbaum
E. A.

GUIDON/NEOMYCIN Project
Project
GUIDON/NEOMYCIN

5P41 RR00785-11
RR00785-11
5Pll

aspect of
of this
this work
work is t,he insight
insight that
that has been gained
gained regarding
regarding the
the need to
to structure
structure
aspect
knowledge differently,
differently, and
and in
in more
more detail,
detail, when
when itit is being
being used for
for different
different purposes
purposes
knowledge
(e.g., teaching
teaching as opposed
opposed to
to clinical
clinical decision
decision making).
making). ItIt was this
this aspect
aspect of
of the
the GUIDON
GUIDON
(e.g.3
resea.rch that
t.hat led to
to the
the development
development of
of NEOMYCIN,
NEOMYCIN, which
which is an evolving
evolving computational
compu~ational
research
model of
of medical
medical diagnostic
diagnostic reasoning
reasoning that
that we hope will
will enable us to
to better
better understand
understand
model
teach diagnosis
diagnosis to
to students.
students. An
An important
important additional
additional realization
realization is that
that these
and teach
st.ructuring methods
methods are beneficial
beneficial for
for improving
improving the
the problem-solving
problem-solving performance
performance of
of
st,ructuring
consultation progra,ms,
progra.ms, providing
providing more
more detailed
detailed and
and abstract
abstract explanations
explanations to
to consultation
consultation
consultation
users, and
and making
making knowledge
knowledge bases easier to
to maintain.
maintain.
As we move
move from
from technological
technological development
development of
of explanation
explanation and
and student
student modeling
modeling
As
the next
next year
year begin
begin to
to collaborate
collaborate more
more closely
closely with
with the
the medical
medical
capabilities, we will
will in the
capabilities,
communit.y to
to design an effective,
effective, useful
useful tutoring
tutoring program.
program. Stanford
Stanford Medical
Medical School
School
community
Dr. Maffly,
Maffly, have shown
shown considerable
considerable interest
interest in
in this
this project.
project. A
A research
research
fa.culty, such as Dr.
fa.culty,
fellow associated with
with Maffly,
Maffly, Curt
Curt Kapsner,
Kapsner, MD,
MD, joined
joined the
the project
project last
last year
year to
to serve as
fellow
expert and liaison
liaison with
with medical
medical students
students at
at Stanford.
Stanford.
medical expert
medica.

C. High.lights
High.lights of
of Research
Research Progress
Progress
C.

C.l
Accomplish.ments This
This Pad
Past Ear
rear
C.l Accomplishments
C.1.i
Th.e NEOMYCIN
NEOlwYCIN Consultation
Consultation Program
Program
C.l.1 Th,e

NEOMYCIN is distinguished
distinguished from
from other
other AI
AI consultation
consultation programs
programs by
by its
its uses
uses of
of an
NEOMYCIN
explicit set of
of domain-independent
domain-independent meta-rules
meta-rules for
for controlling
controIling all
all reasoning.
reasoning. These rules
rules
explicit
constitute
the
diagnostic
procedure
that
we
want
to
teach
to
students:
the
stages
of
constitute the diagnostic procedure that
want to teach to students: the
of
diagnosis.
how
to
focus
on
new
hypotheses,
and
how
to
evaluate
hypotheses.
It
has
been
a
diagnosis, how
hypotheses, and how to evaluate hypotheses. It
major
undertaking, separa.te
problem of
major undertaking,
separate from
from the
the problem
of representing
representing disease
disease knowledge,
knowledge, to
to
design and test
procedure. Such modifications
test this
t,his diagnostic
diagnostic procedure.
modifications require
require changing
changing our
our
conception
partitioned
For example,
example, this
this year
year we partitioned
conception of
of how disease
disease knowledge
knowledge is organized.
organized. For
disease
and "red
disease findings
findings into
into "non-specific"
“non-specific”
“red flag"
flag” (those requiring
requiring explanation),
explanation),
augmenting
procedure to
augmenting the diagnostic
diagnostic procedure
to use this
this information
information for
for focusing
focusing on hypotheses.
hypotheses.
A
program reason about
A second cha.nge
change is to
to have the program
about the disease
disease process more
more generally.
generally.
By
by organ
NEOMYCIN now
primitive means to
By associating
associating symptoms
symptoms by
organ system,
system, NEOMYCIN
now has prirnitive
to infer
infer
when a disease
disease process began. It
It also makes more
more complete
complete use
use of
of severity,
severity, location,
location, and
progression information
progression
information to
to discriminate
discriminate among
among hypotheses.
hypotheses.
During
During this
this past
past year,
year, we completely
completely reworked
reworked the program's
program’s knowledge
knowledge of
of nonmeningitis
cases.
This
is
important
if
we
wish
to
teach
students
to
consider
meningitis cases. This
important if
wish
teach students
consider the
competitors
of
meningitis
and
how
to
discriminate
among
them.
The
goal
is
competitors
meningitis
discriminate among them. The
is to prepare
prepare
the program
order to test
test the
program for
for presenting
presenting these (or
(or similar)
similar) cases
cases to students.
students. In order
modeling
modeling component.
component, it
it is necessary to
to ensure that
that the program
program has sufficient
sufficient expertise
expertise to
recognize good student
student beha.vior.
behavior. All
All data
data that
that might
might be relevant
relevant to solving
solving a given
given
problem
problem must
must be
be known
known to the program.
program. The
The key
key problem
problem here is establishing
establishing a base
base of
synonyms
synonyms and knowledge
knowledge about
about classes
classes of data.
data. To
To do this,
this, we have been collecting
collect,ing
protocols
protocols of student.s
students solving
solving problems,
problems, requiring
requiring them
them to request
request all by simple
simple initial
initial case
case
information.
information. Student
Student behavior
behavior also suggests disease
disease knowledge
knowledge that
that must
must be
be added to the
knowledge
knowledge base
base that,
that an expert
expert might
might not
not consider,
consider, but
but which
which the modeling
modeling program
program must
must
recognize in a student.
In general, we find
find that
that students
students carry
carry out
out a much
much broader,
broader,
student.
inefficient
inefficient search, requesting
requesting much
much more information
information than
than an expert
expert and
a.nd drawing
drawing fewer
from
the
information
that
they
receive.
conclusions
conclusions from t,he information that they
The
Teaching diagnosis involves
involves recoglllzmg
recognizing
The Image
Ima.ge Student
Student Modeling
Modeling Program
Program -- Teaching
the intent
int,ent behind
behind a student's
student’s behavior,
behavior, so that
that missing
missing knowledge
knowledge can be distinguished
distinguished
from
from inappropriate
inappropriat,e strategies.
strategies. The
The teacher
teacher interprets
interprets behavior,
behavior, critiques
critiques it,
it, and provides
provides

E. A. Feigenbaum
Feigenbaum

5P41
5P41 RR00785-11
RR00785-11

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

advice
To do
do this
this successfully
successfully and
and efficiently
efficiently in
in aa complex
complex
advice about
about other
other approaches.
approaches. To
domain,
the
teacher
benefits
from
multiple,
complementary
modeling
strategies.
domain, the teacher benefit,s from multiple, complementary modeling strategies. IMAGE
IMAGE
isis aa. student
student modeling
modeling program
program that
that uses
uses NEOMYCIN's
NEOMYCIN’s meta-rules
meta-rules and
and disease
disease knowledge
knowledge
to
to understand
understand student
student diagnostic
diagnostic plans.
plans.
A student
student isis presented
presented with
with aa problem
problem to diagnose.
diagnose. As the student
student requests more
problem
problem data
data (Le.,
(i.e., takes aa history
history and
and physical
physical of
of the patient),
patient), IMAGE
IMAGE looks
looks for
for
regularities
regularities in
in sequences
sequences of
of his data
data requests. IMAGE
IMAGE contains
contains aa body of
of knowledge
knowledge about
about
how to map such
such sequences
sequences of
of behavior
behavior onto
onto aa strategical
strategical interpretation
interpretation of
of what
what the
student
student isis doing. The process isis heuristic
heuristic in
in nature
nature because
because the program
program will
will sometimes
sometimes
lose
lose track
track of what
what the student
student is
is doing, because
because he
he is
is being inconsistent
inconsistent or using
unexpected
unexpected strategies.
strategies.
The I~!AGE
IMAGE uses
uses aa dual search strategy.
strategy. The
The program
program first
first produces
produces multiple
multiple
predictions
Focused,
predictions of student
student behavior
behavior by aa model-driven
model-driven simulation
simulation of NEOMYCIN.
NEOMYCIN.
data-driven
By
By supplementing
supplementing each other,
other, these
dst)a-driven searches then
then explain
explain incongruities.
incongruities.
met.hods
lead
to
an
efficient
and
robust
plan
understander.
methods
efficient
robust plan understander.
A model of student
student strategies
strategies in medical
medical diagnosis must
must disambiguate
disambiguate the possible
purposes and knowledge
knowledge underlying
underlying the student's
student’s actions.
actions. The
The approaches
approaches followed
followed by
other
other plan
p1a.nrecognizers
recognizers and student
student modelers
modelers are not
not sufficient
sufficient here because:
because:
1.
1. t.he
t,he complex
complex domain
domain makes thorough
thorough searches impractical,
impractical, whether
whether top-down
top-down
or
bott.om-up;
or bottom-up;
2.
2. we are not
not modeling
modeling only
only facts and rules used in isolation,
isolation, but
but also the
procedures
for
applying
them;
procedures for applying them;
3. everyone
every one of
of the student's
student’s actions
actions must
must be monitored
monitored in case
case the teaching
teaching
module
module decides to
to interrupt;
interrupt;
4. his behavior
behavior must
just explained;
must be evaluated
evaluated and not
not just
explained; and
might not
not have any
any explicit
explicit goal statements
statements from
from the student,
student, so we expect
expect
5. we might
to rely
rely only
only on his queries for
for problem
problem data
data as evidence for
for his thinking.
thinking.
t,o
The IMAGE
IMAGE program
program is a prototype
prototype system
system which
which is now
now being
being extended.
extended. Specifically,
Specifically, a
The
more useful
useful system
system would
would examine
examine its
its own
own interpretations
interpretations and
and strive
strive for
for coherence. We
\Ve
more
designing such a such a system
system now,
now, using
using the
the “blackboard
"blackboard model”
model" for
for postding
posting
are designing
interpretations that
that. may
may change over
over time.
time. The
The levels of
of this
this blackboard
blackboard are: 1) the
the
interpretations
stUdent's
data requests,
requests, 2) a classification
classification of
of question
question type
type (e.g., triggered,
triggered, follow-up,
follow-up,
student’
s data
hypothesis-directed, general),
general), 3) a strategic
strategic interpretation
interpretation in terms
terms of
of NEOMYCIN’
NEOMYCIN's
hypothesis-directed,
s
diagnostic procedure
procedure (ta.sks
(tasks and
and meta-rules).
meta-rules).
By incorporating
incorporating a strategic
strategic level
level of
of
diagnostic
By
interpretation, this
this program
program can be expected
expected to
to make
make significant
significant contributions
contributions to
to our
our
interpretation,
understanding and
and use of
of the
the blackboard
blackboard model
model of
of interpretation.
interpretation. The
The first
first version
version of
of this
this
understanding
program will
will seek t,o
to explain
explain student
student behavior
behavior in
in terms
terms of
of deletion
deletion and
and reordering
reordering of
of
progra,m
simple
variations
of
disease
knowledge
(e.g.,
false
proC'edural
knowledge,
plus
procedural
knowledge, plus simple variations
of
knowledge (e.g.,
data/hypothesis relations).
relations). Study
Study of
of student
student protocols
protocols is now
now suggesting
suggesting what
what kinds
kinds of
of
dats.jhypothesis
variations are common
common that
that we might
might easily
easily identify
identify automatically.
automatically.
variations

C.l.2 The
The NEOMl%IN
NEOMYCIN Ezplanation
Explanation System
System
C.1.2
The initial
initial explanation
explanation system
system of
of NEOMYCIN,
NEOMYCIN, now
now completed,
completed, enables the
the user
user t,o
to
The
answer WHY
\VHY and
and HOW
HOW questions
questions during
during a consultation.
consultation. That
That is, when
when the
the program
program
answer
prompts the
the user
user for
for new
new data,
data, the
the user
user may
may ask
ask WHY
\VHY the
the data
data is
is being
being requested
requested or
or
prompts
HO\V some st,rategic
strategic task
task will
will be
be (or
(or was)
was) accomplished.
accomplished. Unlike
Unlike MYC’
MYCIN's
explanation
HO\V
IN’s explanation

E. A.
A. Feigenbnum
Feigenbaum
E.

80
80

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

5P41 RR00785-11
RR00785-11

system,
NEOMYCIN are
system, upon
upon which
which this
this kind
kind of
of capability
capability is patterned,
patterned, explanations
explanations in NEOMYCIN
in terms
plan, not
just specific
terms of
of the diagnostic
diagnostic plan,
not just
specific associations
associations between
between data
data and diagnoses.
The
program can provide
provide abstract
The program
abstract and concrete
concrete paraphrases
paraphrases of
of strategy
strategy rules (based on
canned text).
We have begun the next
next phase, which
which is to
to answer
answer WHY
WHY questions
questions by
by
text,). \Ve
condensing
condensing the entire
entire line of
of reasoning.
reasoning. The
The program
program will
will use
use models of
of the user's
user’s disease
disease
and strategic
strategic knowledge,
knowledge, plus general
general explanation
explanation heuristics,
heuristics, to select the task
task and focus
information
Prototypic
information that
that is most
most likely
likely to
to be of
of interest.
interest.
Prototypic user models are now
now
implemented.
Heuristics
Heuristics have been designed and include:
include: 1) mentioning
mentioning the last
last task
task
implemented.
whose focus (or
(or argument)
argument) changed
changed in kind
kind (e.g., from
from a disease
disease hypothesis
hypothesis to
to a finding
finding
request):
request): 2) never
never mentioning
mentioning tasks
tasks that
that are merely
merely iterating
iterating over
over a list
list of
of rules, findings,
findings,
or
or hypotheses;
hypot,heses; and 3) only
only mentioning
mentioning tasks
tasks with
with a rule
rule as
as an argument
argument to
to programmers.
programmers.
Related
Related to
t,o our
our explanation
explanation condensations
condensations is an effort
effort to
to teach the strategic
strategic
language of
For
For example,
example, we will
will have students
students annotate
annotate a
of tasks
tasks to
to students.
students.
NEO!"IYCIN typescript
NEOh,lYCIN
typescript in terms
terms of
of tasks
tasks and foci, to
to help them
them recognize good strategic
strategic
behavior.
behavior. This
This requires
requires a common
common language of
of what
what the tasks are, e.g., "grouping"
“grouping” and
"asking
Rather
just marking
Rather than
than just
marking annotating
annotating tasks, we seek the
“asking general
general question."
question.”
principles by
principles
by which
which the tasks
tasks could
could be consistently
consistently structured
structured into
into primitives
primitives and
auxiliary.
These same principles
auxiliary.
principles could
could be used by
by the
the explanation
explanation system
system for
for choosing
choosing
tasks to mention.
mention. Our
Our current
current theory
theory is that
that these primitive
primitive or
or "interesting"
“interesting” operations
operations
correspond
correspond to meta-rules
meta-rules that
that establish
establish a new focus.

C.LI
for Teaching
C.l.4 Graphics
Graphics for
Teaching
vVe
We are continuing
continuing make
make extensive
extensive use
use of
of graphics
graphics in our
our programs.
programs. For
For example,
example,
we are implementing
program that
implementing a program
that will
will mostly
mostly automatize
automatize the protocol
protocol collection
collection
process (though
(though we are cautious
cautious about
about how
how menus will
will bias student
student behavior,
behavior, even when
when
lists
lists are very
very long and full
full of
of irrelevant
irrelevant findings).
findings). As part
part of
of our
our series of
of instructional
instructional
programs, GUIDON-WATCH
is now being
progra.ms,
GUIDON-WATCH
being implemented
implemented as
as a graphic
graphic system
system for
for watching
watching
NEOMYCIN's
reasoning.
For
NEOMYCIN’
s
reasoning.
For example,
example, we can highlight
highlight the
the hypotheses
hypotheses under
under
consideration
consideration and show graphically
graphically how
how the program
program "looks
“looks up"
up” its
its hierarchies
hierarchies before
before
refining
is also mostly
refining hypotheses.
hypotheses. The
The design or
of GUIDON-ANNOTATE
GUIDON-ANNOTATE
mostly complete.
complete. It
It
will
NEOMYCIN's
will allow
allow a student
student to
to mark
mark up a typescript
typescript of
of NEOMYCIN’
s behavior
behavior using
using the same
language of
of tasks
tssks the
t.he program
program uses
uses when
when explaining
explaining its own
own behavior;
behavior; iconic
iconic menus are
very
very usef1ll
useful to
to avoid
avoid natural
natural language difficulties
difficulties (though
(though it
it is clear
clear that
that the student
student will
will
sometimes
somet,imes need to
to "talk
“talk back").
back”).

C.2
Research in
Progress
C.2 Research
in Progress
The
The following
following projects
projects are active
active as
as of
of June 1983 (see
(see also near-term
near-term plans
plans listed
listed in
Section
Section III.A):
111.‘~):
1.
NEOMYCIN's
disease
1. augmenting
augmenting NEOhlYCIN’
s
disease knowledge
knowledge so we can fairly
fairly evaluate
evaluate the

program's
program’s focussing strategies
strategies and evaluate
evaluate IMAGE;
IMAGE;
2. developing
produce
developing capability
capability to
to automatically
automatically
produce summary
summary explanations
explanations of
of
NEOMYCIN's
NEOMYCIN’
s reasoning.
reasoning.
3. development
and GUIDON-ANNOTATE
for
development of
of GUIDON-WATCH
GUIDON-WATCH
GUIDON-ANNOTATE
for teaching
teaching
NEOMYCIN's
NEOMYCIN’
s knowledge
knowledge to
to students.
students.
4. developing
developing new student
student modeling
modeling program
program based on the blackboard
blackboard model.

E. A.
A. Feigenbaum
Feigenbaum

5P41
5P41 RR00785-11

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

D.
D. Publications
Publications Since January
January 1989
198.3
D., Clancey,
Clancey, W.J.,
W.J., Rennels, G.: Strategic
Strategic
Consultation.
C’onsultation. Int
Int JJ Man-Machine
Man-Machine Studies, in press.

1.
1. Hasting,
Hasling,

explanations
explanations

an
in

2.
2. Clancey,
Clancey, W.J.:
W.J.: The advantages
advantages of
of abstract
abstract control
control knowledge
knowledge in
in expert
expert
8y8tem
system de8ign.
design. Proceedings
Proceedings of AAAI-83,
AAAI-83, pages
pages 74-78.
3. Clancey,
Clancey, \V.J.:
W.J.: Acquiring,
Acquiring, representing,
representing, and
and evaluating
evaluating a competence
competence model
model
of
OF
of diagno.'li8.
diagnosis. In Chi,
Chi, Glaser,
Glaser, and Fan
Farr (Eds.),
(Eds.), THE
THE NATURE
NATURE
OF
EXPERTISE.
EXPERTISE. In preparation.
preparation. HPP-84-2.
HPP-84-2.
4.
IN
4. Clancey,
Clancey, W.J.
W.J. and E. H. Shortliffe.:READINGS
Shortliffe.:RELaINGS
IN !vIEDICAL
MEDICAL ARTIFICIAL
ARTIFICIAL
Il'VTELLIGENCE:
THE
Reading:
IKTELLIGENCE:
THE FIRST
FIRST DECADE.
DECADE.
Reading: Addison-Wesley,
Addison-Wesley, in
press.
5. Clancey,
Clancey, W.J.:
W.J.:
AAAI-84.
AAAI-84.

Classification
Problem
Classification
Problem Solving.
Soluing. HPP-84-7.
HPP-84-7.

Submitted
Submitted to

E. Funding
Funding Support
Support
Cont,ract.
Contract Tit.le:
Title: "Explorat.ion
“Exploration of
of Tutoring
Tutoring and Problem-Solving
Problem-Solving Strategies"
Strategies”
Principal
Principal Investigator:
Investigator: Bruce
Bruce G. Buchanan,
Buchanan, Adjunct
Adjunct Prof.
Prof. Computer
Computer Science
Associate
Associate Investigator:
Investigator: William
William J. Clancey,
Clancey, Research Assoc. Computer
Computer Science
Agency:
Agency: Office
Office of Naval
Naval Research and
Army
Army Research Institute
Institute (joint)
(joint)
ID
ID number:
number: NOOOI4-79-C-0302
N00014-79-C-0302
Term:
Term: March
March 1979 to
to March
March 1985
Tot,al
Tot(a.1award:
a.ward: $583,892
$083,892

II.
WITH
THE
RESOURCE
II. INTERACTIONS
INTERACTIONS
WITH
THE SUMEX-AIM
SUMEX-AIM
RESOURCE
A. Medical
Medical Collaborations
Program Dissemination
Dissemination via
A.
Collaborations and
and Program
uia SUAfEX
SUMEX
A
NEOMYCIN has been shown
by the
‘4 great
great deal of
of iuterest
interest in GUIDON
GUIDON and NEOMYCIN
shown by
medical
mcdica.1 and
a.nd computer
computer science communities.
communit,ies. We
We are frequently
frequently asked to
to demonstrak
demonstrate
these programs
programs to
t,o Stanford
St,anford visitors
visitors or
or at meetings
meetings in this
this cOllDtry
country or
or abroad.
abroad. GUIDON
GUIDON is
available on the
the SUMEX
SUMEX 2020. Physicians
Physicians have generally
generally been enthusiastic
enthusiastic about
about these
ava.ila.ble
programs' potential
potential and what
what they
they reveal
reveal about
about current
current approaches
approaches to
to computer-based
computer-based
programs
medical decision
decision making.
making.
medical
Perhaps our
our most
most significant
significant project
project to
to disseminate
disseminate our
our research
research via
via SUMEX
SUMEX in l.he
the
Perhaps
past year
year has been the completion
completion of
of a book,
book, “Readings
"Readings in Medical
il,ledical Artificial
Artificial Intelligence:
Intelligence:
past
The First
First Decade,”
Dec:1de," edited
edited by
by Dr.
Dr. Clancey
Clancey and Dr.
Dr. Shortliffe.
Shortliffe. All
All of
of the significant
significant
The
SUMEX-AI~vl products
products of
of the
the past
past decade are described
described in this
this collection.
collection. Each
Each cha.pter
chapter is
SUMES-,41&I
preceded by
by a.
a one-age historical
historical introduction.
introduction. In addition,
addition, opening
opening and closing
closing chapters
chapters
preceded
by t)he
the editors
editors survey
survey issues
issues in the
the field
field and the
the promise
promise of
of the
the future.
future. A
A complete
complete index
index
by
should make
make t,he
t.he book
book of
of considerable
considerable educational
educational value.
value. Preparation
Preparation of
of this
this volume
volume has
should
greatly aided
aided by
by use of
of editing
editing and
and formatting
formatting programs
programs available
available on
on SUMEX-AIM.
SUMEX-AIM.
been greatly
Royalties for
for the
the book,
book, beyond
beyond production
production costs, will
will be used to
to sponsor
sponsor an invited
invited lecture
lecture
Royalties
at a major
major AI
AI nationa.
national conference,
conference, such as AAAI.
AAAI.
at
As mentioned
mentioned
disease knowledge
knowledge of
of
disease
July 1983). This
This
in July

A. Feigenbaum
Feigenbaum
E. A.

earlier, a physician
physician joined
joined our
our group
group this
this year
year to
to help
help us develop
develop the
the
earlier,
the program
program (our
(our first
first collaborator,
collaborator, Tim
Tim Beckett,
Beckett, MD,
MD. died
died of
of cancer
cancer
the
physician has found
found the
the convenience
convenience of
of accessing SUMEX
SUMEX from
from his
physician

RR00785-11
5P41 RR00785-11

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

laboratory or
or at
at home to be extremely
extremely important
important for
for finding
finding time
time to
to test
test NEOMYCIN
hboratory
NEOMYCIN and
to communicate
communicate with
with us by
by electronic
electronic mail.
mail.

Feigenbaum
E. A. Feigenbaum

5P41
5P41 RR00785-11
RR00785- 1 I

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

B.
Projects
B. Sharing
Sharing and
and Interaction
Interaction with
with Other
Other SUA/EX-AlA!
SUMEX-AIM
Projects
GUIDON/NEOMYCIN
retains
GUIDON/NEOMYCIN
retains strong
strong contact
contact with
with the
the ONCOCIN
ONCOCIN project,
project, as
as both
both
are
siblings
of
the
tvIYCIN
parent.
These
projects
regularly
share
programming
expertise
are siblings of the MYCIN parent. These projects regularly share programming expertise
and
and cont.inue
continue to
to jointly
jointly maintain
maintain large
large utility
utility modules
modules developed
developed for MYCIN.
MYCIN. In addition,
addition,
the
the centra.l
centra,l SUMEX
SUMEX development
development group
group acts
acts as
as an
an important
important clearing
clearing house
house for solving
solving
problems
problems and
and distributing
distributing new
new methods.
methods.

C.
C. Critique
Critique of
of Resource
Resource Management
Management
In the winter
winter of 1984,
1984, the SUMEX
SUMEX staff
staff efficiently
efficiently and effectively
effectively shifted
shifted our
our
operation
operation away from
from the center
center of campus to aa professional
professional office
office building
building adjoining
adjoining the
medical
medica~l center.
center. The placement
placement and installation
installation of LISP
LISP workstations
workstations proceeded smoothly.
smoothly.
After
After aa.year with
with Ed Pattermann
Pattermann as
as director
director of SUMEX,
SUMEX, we can report
report that
that the stability
stability
and
a.nd excellence of the resource we have come to expect has
has been completely
completely maintained.
maintained.
Very
Very import.ant
important to us,
us, the RAVEN
RAVEN laser printer
printer installed
installed at our
our new site not
not only
only provides
provides
excellent-quality
output,
but
as
a
machine
devoted
to
the
Heuristic
Programming
excellent,-quality out,put, but as a machine devoted
Heuristic Programming Project
Project
has
has eliminated
eliminated the dela.ys
dela.ys we were experiencing
experiencing aa year
year a.go.
a.go.
\Vith
continuing to experience a few
With t,he
the shift
shift to personal
personal machines, we are continuing
The
difficulties.
The greatest.
greatest, problem
problem appears to be inadequately
inadequately debugged software
software from
from
difficulties.
XEROX.
In particular,
SEROX.
particular, Interlisp-D
Interlisp-D relies heavily
heavily on network
network capabilities
capabilities and must
must be
compatible
compatible with
with several operating
operating systems. This
This transition
transition to new kinds
kinds of hardware
hardware and
software
softwa.re can be
be expected
expected to continue
continue for
for several years. Therefore,
Therefore, we are extremely
extremely
reliant
reliant upon
upon the availability
avai1abilit.y of experienced
experienced systems support.
support. We
We believe
believe that
that additional
additional
SU~IEX
SUhlEX staff
staff is necessary to accommodate
accommodate growing
growing community
community needs.
needs

ill.
RESEARCH
PLANS
III.
RESEARCH
PLANS

A. Project
A.
Project Goals
Goals and
and Plans
Plans
over the next
next year
year will
will continue
continue on several
fronts, leading
leading to
to several
several
Research over
several fronts,
prototype instruct,ional
instructional programs
programs by
by early
early 1985.
protot,ype
1. Continue
Continue to
to develop
develop the knowledge
knowledge base so the program
program can understand
understand and
and

anticipate any
any reasonable
reasonable approach
approach to
to the cases
cases chosen for
for teaching.
teaching.
ant.icipate
Test st,udent
student modeling
modeling program
program on these cases,
cases, collecting
collecting data
data for
for further
further
2. Test
development of
of the program,
program, as well
well as exploring
exploring about
about the
the range
range of
of student
student
development
approaches to
to diagnosis.
diagnosis.
approaches
Extend the
t.he explanation
explanation system
system to
to do full
full summaries.
summaries. Incorporate
Incorporate modeling
modeling
3. Extend
capabilities that
t.hat relate
relate inquiries
inquiries to
to a user model.
model. Provide
Provide explanations
explanations t.ailored
tailored
capabilit,ies
to this
this interpretation
interpretation of
of t,he
the motivation
motivation behind
behind the
the user’
user's
inquiry.
to
s inquiry.
4. Int,egra.te
Integrate current
current display
display capabilities
capabilities into
into running
running NEOMYCIN
NEOMYCIN consultation
consultation to
to
show
how
the
space
of
diagnoses
is
explored
and
how
diagnostic
tasks
are
show how the
of
explored and how diagnostic tasks
generated. Develop
Develop these capabilities
capabilities to
to explore
explore forms
forms of
of graphic
graphic explanation
explanation
generated.
useful
in
tutoring.
(GUIDON-WATCH)
useful
tutoring. (GUIDON-WATCH)
S. Extend
Extend student
student modeling
modeling system
system to
to include
include heuristics
heuristics for
for generating
generating tests
tests that
that
S.
will confirm
confirm and
and extend
extend the
the model.
model. Improve
Improve the
the model
model to
to include
include analysis
analYSis of
of
will
pat,tans
in
model
interpretations,
including
dependency-directed
patterns
including
in
model
interpretat*ions,
dependency-directed
backt.racking in
in the
the belief
belief system
system and
and some capability
capability to
to critique
critique the
the
” backtra.cking”
mode lint:?; rules.
rules. Relate
Relate this
this to
to knowledge
knowledge acquisition
acquisition research.
research.
rnodrling
II

E. A.
A, Feigenbaum
Feigenbaum
E.

84
84

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

5P41
5P-11 RR00785-11

6.
6. Work
Work closely with
with medical
medical students
students to package NEOMYCIN
NEOMYCIN capabilities
capabilities in aa
"workstation"
“workstation” for
for learning
learning medical
medical diagnosis, determining
determining what
what mix
mix of student
student
and program
program initiative
initiative isis desirable.
desirable.

B. Long
Programs
Long term
term plans:
plans: the GUIDON2
GUIDON2 Family
Family of
of Instructional
Instructional
Programs
"Ve
We sketch
sketch here our
our general conception
conception of the research we plan
plan for
for 1984-87,
specifically
Our
specifically the construction
construction of instructional
instructional systems that
that use
use NEOMYCIN.
NEOMYCIN.
Our ideas are
strongly
strongly based on recent
recent proposals
proposals by
by JS
JS Brown,
Brown, particularly
particularly his paper
paper "Process
“Process versus
Product
--A perspective
perspective on tools for
for communal
communal and informal
informal electronic
electronic learning"
learning” and
Product, --A
some related
related papers that
that he wrote
wrote in 1983.
1983. The
The plan
plan is
is to implement
implement at least three
three of
these programs
GUIDON-MANAGE,
and GUIDONprograms (here called GUIDON-WATCH,
GUIDON-WATCH,
GUIDON-MANAGE,
GUIDONANNOTATE).
ANNOTATE).
The key idea is that
that NEOMYCIN
NEOMYCIN provides
provides aa language
language by
by which
which aa program
program can
converse wit.h a student
student about
about strategies
strategies and knowledge
knowledge organization
organization for
for diagnosis.
NEOtvIYCIN's
NEOMYCIN’s tasks and structural
structural terms
terms provide
provide the vocabulary
vocabulary or
or parts
parts of speech; the
meta-rules
We will
will construct
construct different
different
meta-rules are the grammar
grammar of the diagnostic
diagnostic process. We
graphic,
graphic, reactive
reactive environments
environments in which
which the student
student can observe, describe, compare,
compare, and
improve
improve diagnost,ic behavior
behavior of
of himself
himself and others.
others. There
There are many
many shared, underlying
underlying
ca.pabilities
ca.pa.bi!it.ies that
that wil!
will be constructed
constructed in parallel
parallel and improved
improved over
over time.
time.
Our
Our approach
approach is
is to delineate
delineate clearly
clearly different
different kinds
kinds of
of interactions
interactions that
that a student
st.udent
might.
might, have with
wit,h a program
pr0gra.m concerning
concerning diagnostic
diagnostic strategies.
strategies. Thus,
Thus, each instructional
instructional
system
activity>,
where
syst.em (but
(but one) has a name of the form
form GUIDON-,
where the name
specifies what
what the student
student is doing
doing (e.g., watching,
watching, telling).
telling). The
The programs
programs can be made
arbitrarily
by integrating
arbitrarily complex
complex by
integrating coaches,
coaches, student
student models, and explanation
explanation systems. "Ve
We
try
try here to
to separate
separate out
out these capabilities,
capabilities, trying
trying to
to get at
at the
the minimum
minimum interesting
interesting
activities
provide for
activities we might
might provide
for a student.
student.
GUIDON-WATCH
The
NEOMYCIN
GUIDON-VVATCH
The simplest
simplest system
system allows
allows a student
student to watch
watch NEOMYCIN
solve a.a. problem,
problem. perhaps
perhaps one supplied
by the student.
supplied by
student. Graphics
Graphics display
display the evolving
evolving
search space, that
that is, how
how tasks, as
as operators,
operators, affect
affect the differential
differential (Differential
(Differential
---(Question
The
---(Quest ion X)--S)--- > Differential').
Differential’).
The student
student can step through
through slowly
slowly and replay
replay the
interaction.
prosaic explanations
program is
intcract.ion. He can ask for
for prosaic
explanations and summaries
summaries of
of what
what the program
doing.
program will
The program
will also indicate
indicate its
its task
task and focus for
for each data
data request.
request. This
This
doing. The
introduces
introduces the
t.he student
student to the idea that
that the diagnostic
diagnostic process has structllre
structure and follows
follows a
certain kind
kind of
of logic.
certain
GUIDON-MANAGE In
In this
this system
system the
the student
student solves a problem
problem by
by telling
telling
GUIDON-MANAGE
NEO!\IYCIN
what task
task to
to do at each step. Essentially,
Essentially, the
the student
student provides
provides the strategy
strategy
NEOXIY’
C’IN what
and the program
program supplies
supplies the
the tactics
tactics (meta-rules)
(meta-rules) and
and domain
domain knowledge
knowledge to carry
carry out
out the
a.nd
strategy. The
The program
program will
will in genera!
general carry
carry through
through tasks
tasks in
in a logical
logical way,
way, for
for example,
example,
strst.egy.
proCt'cdiug to
to t,est
tcst a hypothesis
hypothesis completely,
completely, and not
not “breaking”
"breaking" on FINDOUT
FINDOUT or
or
proceeding
APPLYHULES
(t.wo low-level
low-level tasks
tasks that
that mainly
mainly test
test domain
domain knowledge
knowledge and not
not strategy).
strategy).
APPLI’
RIJLES (two
ThE' program
program will
will not
not pursue
pursue new hypotheses
hypotheses automatically.
automatically. However,
However, the
the student
student will
will
The
alwa.ys see
see wha.t
what questions
questions a task
task caused the
the program
program to
to request,
request, as well
well as how
how the
alwa,ys
differential changes.
This activity
activity leads the
the student
student to
to observe the entailments
entailments of
of
differential
This
strategies, helping
helping him
him become a better
better observer
observer of
of his own
own behavior.
behavior. Here he shows that
that
strategies,
knows the structural
structural vocabulary
vocabulary that
that makes
makes a strategy
strategy appropriate.
appropriate.
he knows
GUIDON-ANNOTATE This
This system
system allows
allows the
the student
student to
to annotate
annotate a NEOMYCIN
NEOMYCIN
GUIDON-ANNOTATE
typescript, indicating
indicating the
the task
task and
and focus associated with
'with each data
data request.
request. The
The program
program
typescript,
will indicate,
indicate, upon
upon request,
request, where
where the
the student
student is incorrect
incorrect and
and which
which annotations
annotations are
will
different from
from NEOMYCIN’
NEOMYCIN's,
but still
still reasonable
reasonable interpretations.
interpretations. The
The student
student will
will be
different
s, but

A. Feigenbaum
Feigenbaum
E. A.

5P41 RR00785-11
RR00785- 11

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

able to
to- choose these tasks from
from a menu
menu of
of icons, either
either linearly
linearly or
or hierarchically
hierarchically displayed,
displayed,
as
NEOMYCIN will
as he prefers.
prefers. (Again,
(Again, NEOMYCIN
will annotate
annotate its
its own
own solutions
solutions upon
upon request
request and allow
allow
replaying.)
This
replaying.)
This activity
activity gets the
the student
student to
to think
think strategically
strategically by
by recognizing
recognizing a good
st.rategy.
st,rategy. In
In this
t,his way,
way, he learns to
to recognize how
how strategies
strategies affect
affect the
the problem
problem space.
GUIDON-APPRENTICE
This
NEOMYCIN in which
GUIDON-APPRENTICE
This is a variant
variant of
of NEOMYCIN
which the program
program
stops during
during a consultation
consultation and asks the student
student to propose the
the next
next data
data request(s).
request(s). The
The
student
student is asked to
to indicate
indicate the task
task and focus he has in mind,
mind, plus
plus the differential
differential he is
operating
NEOMYCIN would
operating upon. The
The program
program compares
compares this
this proposal
proposal to
to what
what NEOMYCIN
would do. In
In
this
this activit.y
activit,y we descend to
to the domain
domain level and require
require the student
student to instantiate
instant,iate a
st.rategy
st,rategy appropriately.
a.ppropriately.
GUIDON-DEBUG
Here the student
GUIDON-DEBUG
student is presented
presented with
with a buggy
buggy version
version of
of
NEOM'(CIN
goes through
through the steps of
of annotating
annotating the buggy
buggy
NEOMY’
CIN and must
must debug it.
it. He goes
consultation
consultation session,
session, indicating
indicating what
what questions
questions are out
out of
of order
order or
or unnecessary,
unnecessary, indicating
indicating
what
properly, and then
what tasks are not
not being
being invoked
invoked properly,
then trying
trying out
out his hypothesis
hypothesis on a
"rep:J.ired"
predict what
what will
will be different,
d.ifferent, then
then allowed
allowed to
to observe
“repaired” system.
system. He is asked to predict
what
what happens. This
This activity
activity teaches the student
student to recognize how
how a diagnostic
diagnostic solution
solut,ion
can be non-optimal,
provides him
non-optimal, further
further emphasizing
emphasizing the value
value of
of good strategy.
strategy. It
It also provides
him
with
practice for
problem behavior.
with key
key meta-cognitive
meta-cognitive practice
for criticizing
criticizing and debugging
debugging problem
behavior.
GUIDON-SOLVE
This
The
The student
student carries
carries
GUIDON-SOLVE
This is the complete
complete tutorial
tutorial system.
system.
through
plan recognizer
through diagnosis
diagnosis completely,
completely, while
while a plan
recognizer attempts
attempts to
to track
track what
what he is doing
doing
and a coach interrupts
interrupts to offer
offer advice. Here annotation,
annotation, comparison,
comparison, debugging,
debugging, and
('xplana.tion
expiana.tion are all integrated
int)egrated to illustrate
illustrate to
to the student
student how his solution
solution is non-optimal.
non-optimal.
For
For example,
example, the student
student might
might be asked to annotate
annotate his solution
solution after
after he is done; this
this
will
point out
will point,
out strategic
strategic gaps in his awareness and provide
provide a basis for
for critique
critique and
improvement.
A "curriculum"
“curriculum” based on rrequent
frequent student
student faults
faults and important
important things
things to
improvement. A
learn
In
learn will
will drive
drive the interaction.
interaction.
In this
this activity,
activity, the student
student is on his own.
own. Faced with
with
the
must exercise his diagnostic
diagnostic procedure
procedure from
from start
start to
to
t.he proverbial
proverbial "blank
“blank screen,"
screen,” he must
finish.
finish.
GUIDON-GAME
Two
GUIDON-GAME
Two or
or more
more students
students play
play this
this together
together on a single machine.
machine.
They
All
They are given
given a case
case to solve together,
together, and each student
student requests data
data in turn.
turn.
All
st.lldents
When
students receive the requested
requested information.
information.
When a student
student is ready,
ready, he makes a
di3gnosis,
diagnosis, indicated
indicat)ed secretly
secretly to
to the program
program while
while the others
others are not
not watching.
watching. He then
then
drops
drops out
out of
of the questioning
questioning sequence.
sequence. However,
However, he can re-enter
re-enter later,
later, but
but of
of course will
will
be penalized.
Afterwards, score is based on the number
number of
of questions
questions asked and use
use of
of
penalized. Afterwards,
good strategy.
strategy. The
The coach will
will indicate
indicate to
to weak
weak players
players what
what they
they could
could learn
learn from
from strong
strong
players,
one
players. encouraging
encouraging them
them to
to discuss certain
certain issues
issues among
among themselves.
themselves. Variation:
Variation:
person solves while
while one or
or more
more competing
competing students
students annotate
annotate the solution
solution and show where
where
it
Variation:
NEOMYCIN (and
Variation: one team
team introduces
introduces a bug
bug into
into NEOMYCIN
(and
it, could
could be improved.
improved.
predicts
This
predicts the effect)
effect) and the
the other
other team
team finds
finds it
it (as in SOPHIE).
SOPHIE).
This activity
activity will
will
encourage students
students to share their
their experiences and talk
talk to and learn
learn from
from each other
other about
about
the diagnostic
disgnostic process.

C. Requirements
Reqw:rements for
for Continued
Continued SUMEX
SUMEX Use
Although
NEOMYCIN work
Although most
most of
of the
the GUIDON
GUIDON and NEOMYCIN
work is shifting
shifting to
to Xerox
Xerox
Dolphins
Dolphins and Dandelions
Dandelions (D-machines),
(D-machines), the DEC
DEC 2050
2060 and 2020 continue
continue to
to be key
key
elements
elements in our
our research plan.
plan. Our
Our primary
primary use
use of
of the 2050
2060 will
will be to develop
develop the
NEOMYCIN conSUltation
access.
NEOMYCIN
consultation system,
system, possibly
possibly by
by remote
remote ARPANET
ARPANET
access. Because of
of
address
a.ddress space limitations,
limitations, the consultation
consultation program
program can be combined
combined with
with explanation
explanation or
or
student
student modeling
modeling facilities,
facilities, but
but not
not both,
both, as is required
required for
for GUIDON2
GUIDON2 programs.
programs. We
We
continue
continue to use
use the 2020 for
for demonstrating
demonstrating the original
original GUIDON
GUIDON program.
program. As always,
always,
the 2050
2060 will
will be essential
essential for
for work
work at home, writing,
writing, and electronic
electronic mail.
mail.
E. A. Feigenbaum
Feigenbaum

85
86

5P41
5P4 1 RROOi85-11
RR00785- 11

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

D. Requirements
Requirements for
for Additional
Additional Computing
Resources
D.
Computing Resources
The D-machine's
D-machine’s large address space is permitting
permitting development
development of
of the large
The
program that
that complex
complex computer-aided
computer-aided instruction
instruction requires.
requires. Graphics
Graphics will
will enable us to
program
methods for
for presenting
presenting material
material to naive
naive users. We
We also plan
plan to use
use the Ddevelop new methods
mn.chine as
as a reliable,
reliable, constant
constant "load-average"
“load-average” machine,
machine
machine, for
for running
running experiments
experiments with
with
physicians and students.
students.
The development
development of
of GUIDON2
GUIDON2 on the
the D-machine
D-machine will
will
physicians
The
demonstrate the
the feasibility
feasibility of
of running
running intelligent
demonstrate
intelligent consultation
consultation or
or tutoring
tutoring systems on
sma.11,affordable
affordable machines
machines in physicians'
physicians’ offices,
offices, schools and other
other remote
remot,e sites.
small,
We cnrrently
currentSly have access
access to
to 1 1/2
l/2 DOLPHINs.
DOLPHINS.
We
We
We expect that
that 3 full
full time
time
programmers will
will need access
access to
to two
two full
full machines.
machines. vVe
We are keeping
keeping logs so we can
ca.n begin
begin
programmers
understand patterns
patterns of
of ::t.ctivity
activity and how
how these "personal"
“personal” machines
machines can be effectively
effectively
to understand
shared.
shared.

E. A. Feigenbaum
Feigenbaum

GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project

RR00785-11
5P41 RR00785-11

E.
Community and
and Resource
E. Recommendations
Recommendations for
for Future
Future Community
Resource Development
Development
shift our
our development
development of
of systems
systems to
to personal
personal LISP
LISP machines,
machines, such as
as the
As we shift
DOLPHIN, it
it becomes more
more difficult
difficult to
to access
access these programs
programs remotely
remotely for
for access
access from
from
DOLPHIN,
our
our homes (so that
that we may
ma,y work
work conveniently
conveniently during
during the evenings and weekends)
weekends) and
from remote
remote sites for
for collaboration
collaboration and demonstration.
demonstration.
from
This
This problem
problem will
will be partly
partly
ameliorated by
by "dial-up"
“dial-up” (modem)
(modem) access
access to these machines,
machines, but
but the use
use of
of bit-mapped
bit-mapped
ameliorated
displays requiring
requiring a high-bandwidth
high-bandwidth makes the
the phone lines inadequate
inadequate for
for our
our purposes.
displays
Further
Further technological
technological development
development of
of networks,
networks, probably
probably involving
involving access
access over
over cables, will
will
be necessary.
computer resources become more distributed,
distributed, the need for
for a central
central machine
machine
As computer
does not
not diminish.
diminish. Programs
Programs and knowledge
knowledge bases
bases continue
continue to be shared,
shared, requiring
requiring highhighdoes
speed network
network connections
connections among
among computers
computers and file servers. SUMEX-AIM's
speed
SUMEX-AIM’s role will
will
shiftly slightly
slightly over
over the
the next
next few years
years to accommodate
accommodate these needs,
needs, but
but its
its identity
identity as
as a
shiftly
central
will only
only change in kind,
kind, not
not importance.
importance.
cent
ral resource will
Moreover,
Moreover, sophisticated
sophisticated
printing devices, such as
as the
the Xerox
Xerox RAVEN,
RAVEN, must
must necessarily
necessarily be shared, again using a
printing
network. Maintenance
Maint,enance of
of this
this network
network and its
its shared
shared devices will
will become a key
key activity
activity
network.
for
for the SUMEX
SUMEX staff.
staff. Thus,
Thus, while
while computing
computing resources will
will be provided
provided by
by the
the "outboard
“outboard
engines” of
of personal
personal machines,
machines, the community
community will
will remain
remain intricately
intricately
linked and
engines"
linked
dependen
dependent,t. on common,
common, but
but peripheral,
peripheral, resources.
From
From t.his
t,his perspective,
perspective, future
future resource development
development should
should focus on improving
improving the
capabilities of
of networks,
networks, file servers, and attached
attached devices to respond
respond to individua,l
individual
ca,pabilities
For example,
example, it
it is now
now common
common for
for 10%
10% of
of a user's
user’s time
time at a personal
personal machine
machine
requests. For
spent waiting
waiting for
for a file server
server or
or printer
printer to
to process a request. Multi-processing
to be spent
Multi-processing
necessity in such an environment,
environment, so a request
request can be honored,
honored, while
while the user
becomes a necessity
returns
returns to continue
continue his programming
programming or
or editing.
editing.

E. A.
i4. Feigenbaum
Feigenba,um

HPP
HPP Core
Core AI
AI Research
Research

RR00785-11
5P41 RR00785-11

II.A.1.3. HPP
HPP Core
Core AI
AI Research
Research
II.A.1.3.

Heuristic
Heuristic Programming
Programming Project
Project
Principal Investigator:
Investigator: Edward
Edward A.
A. Feigenbaum
Feigenbaum
Principal
C-Principal
Co-Principal Investigator:
Investigator: Bruce
Bruce G.
G. Buchanan
Buchanan
Department
Department of
of Computer
Computer Science
Science
Stanford
Stanford University
University

SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
I.I. SUMMARY

A.IB. Rationale
Rationale and
and Medical
Afedical Relevance
Relevance
A./B.
Medicine and the
the biological
biological sciences are knowledge-intensive
knowledge-intensive with
with an exponential
exponential
Medicine
rate of
of growth
growth in relevant
relevant knowledge.
knowledge. This
This means that
that problem
problem solving
solving of
of all
all sorts
sorts is
rate
becoming increasingly
increasingly complex
complex in these disciplines.
disciplines. Further,
Further, most
most problems
problems are symbolic
symbolic
becoming
nature rather
rather than
than amenable
amenable to
to mathematical
mathematical formulation
formulation and numerical
numerical solution.
solution.
in nature
Artificia.l Intelligence
Intelligence (AI)
(AI) methods
methods have been focused on medical
medical and biological
biological problems
problems
Artificia.1
for over
over a decade with
with considerable
considerable success.
success. This
This is because, of
of all
all the
the computing
computing methods
methods
for
known, AI
AI met,hods
met.hods are the
the only
only ones that
that deal explicitly
explicitly with
with symbolic
symbolic information
information and
known,
problem solving
solving and with
with knowledge
knowledge that
that is heurist,ic
heuristic (experiential)
(experiential) as well
well as factual.
factual.
problem
particularly fast-moving
fast-moving area of
of AI
AI is expert.
expert systems. An
An expert
expert system
system is one
One particularly
performance level rivals
rivals that
that of
of an human
human expert
expert because itit has extensive
extensive domain
domain
whose performance
knowledge (current,ly
(currently usual!y
usually derived
derived from
from an human
human expert);
expert); itit can reason
rea..<;on about
about its
its
knowledge
knowledge
knowledge to solve difficult
difficult problems
problems in the domain;
domain; it
it can explain
explain its line of
of reasoning
reasoning
much
much as
as an human
human expert
expert can; and it
it is flexible
flexible enough to incorporate
incorporate new knowledge
knowledge
without
Expert
witholit reprogramming.
reprogramming.
Expert Systems
Systems draw
draw on the current
current stock
stock of
of ideas in AI,
AI, for
for
They are adequate
adequate for
for capturing
capturing
example,
They
example, about
about representing
representing and using
using knowledge.
knowledge.
problem-solving
Numerous high-performance,
problem-solving expertise
expertise for
for many
many bounded
bounded problem
problem areas. Numerous
high-performance,
expert
expert, systems
systems have resulted
resulted from
from this
this work
work in such diverse
diverse fields
fields as
as analytical
analytical chemistry,
chemistry,
medlcal
cancer chemotherapy
chemotherapy management,
management, VLSI
VLSI design, machine
machine fault
fault
medical diagnosis,
dia,gnosis, cancer
of these programs
programs rival
rival human
human experts
experts in solving
solving
dia.gnosis.
diagnosis, and molecular
molecular biology.
biology. Some of
problems
problems in particular
particular domains
domains and some are being
being adapted
adapted for
for commercial
commercial use.
use. Other
Other
tools for
for representing
representing and utilizing
utilizing
projects
projects have developed
developed generalized
generalized software
software tools
knowledge
UNITS,
EMYCIN,
UNITS, AGE,
AGE, MRS,
MRS, GLISP)
GLISP) as
as well
well as
as comprehensive
comprehensive
knowledge (e.g., EMYCIN,
publications
Intelligence
publications such as
as the three-volume
three-volume Handbook
Handbook of
of Artificial
Artificial
Intelligence and books
summa.rizing
summarizing lessons
lessons learned
learned in the DENDRAL
DENDRAL and MYCIN
MYCIN research projects.
projects.
But
But the current
current ideas fall
fall short
short in many
many ways, necessitating
necessitating extensive
extensive further
further basic
research dforts.
efforts. Our
Our core research goals, as
as outlined
outlined in the next
next section,
section, are to analyze
the limitations
limitations of current
current techniques
techniques and to investigate
investigate the nature
nature of methods
methods for
for
overcoming
overcoming them.
them. Long-term
Long-term success
success of
of computer-based
computer-based aids in medicine
medicine and biology
biology
methods available
available for
for representing
representing and using
depend on improving
improving the programming
programming methods
domain
That
That knowledge
lcnowledge is inherently
inherently complex
complex .... it contains
contains mixtures
mixtures of
domain knowledge.
knowledge.
symbolic
symbolic and numeric
numeric facts
facts and relations,
relations, many
many of them
them uncertain;
uncertain; it
it contains
contains knowledge
knowledge
at different
different levels of
of abstraction
abstraction and in seemingly
seemingly inconsistent
inconsistent frameworks;
frameworks; and it
it links
links
examples and exception
exception clauses
clauses with
with rules of thumb
thumb as
as well
well as
as with
with theoretical
theoretical principles.
principles.
Current
Current techniques
t,echniques have been successful only
only insofar
insofar as
as they
they severely limit
limit t.his
t#his
complexity.
comp1exit.y. As the applications
applications become more far-reaching,
far-reaching, computer
computer programs
programs will
will have
to deal more effectively
effectively with
with richer
richer expressions and much
much more voluminous
voluminous amounts
amounts of
knowledge.
knowledge.
This
This report
report documents
documents progress on the basic or core research activities
activities within
within the

Feigenbaum
E. A. Feigenbaum

HPP
HPP Core AI
AI Research

5P
41 RR00785-11
5P41

Hruristic
Heurist,ic Programming
Programming Project
Project (HPP),
(HPP), funded
funded in part
part under
under the SUMEX
SUMEX resource as
as well
as
This work
work explores aa broad
broad range of basic
as by other
other federal and industrial
industrial sources. This
research
resea,rch ideas
ideas in many
many application
application settings,
settings, all of which
which contributes
contributes in the long term
term to
improved
improved knowledge
knowledge based
based systems in biomedicine.
biomedicine.
C. Highlights
Highlights of
of Research
Research Progress
Progress
In the last year,
year, we made progress on several major
major topics
topics of research. The
The style
style of
research t.hat
that we believe is
is most
most productive
productive at this
this stage of development
development of AI
AI is the
experimental
experimental style.
style. Thus,
Thus, within
within the HPP
HPP we build
build systems that
that implement
implement our
our idea.s
idea.s for
for
answering
answering (or
(or shedding
shedding some light
light on) fundamental
fundamental questions; we experiment
experiment with
with those
systems to drtermine
determine the strengths
strengths and limits
limits of the
ihe ideas;
ideas: we redesign and test more; we
attempt
attempt to generalize
generalize the ideas from
from the domain
domain of
of implementation
implementation to other
other domains;
domains; and
we publish
order to carry
carry out
out this
this style
style of research, then,
then,
publish details
details of the experiments.
experiments. In order
we select.
select specific
specific problems
problems to help focus the general questions. Many
Many of these specific
specific
problem
problem domains
domains are medical
medical or
or biological.
biological. In this
this way
way we believe the HPP
HPP has made
substantial
just to the AIM
substantial contributions
contributions to core research problems
problems of
of interest
interest not
not just
AIM
community
community but
but. to AI
AI in general.
general.
Progress
Progress is reported
reported below
below under
under each of
of the major
major topics
topics of
of our
our work.
work.
are to HPP
HPP technical
technical reports
reports listed
listed in the publications
publications section.
section.

Citations
Citations

How
How can the knowledge
knowledge necessary for
for complex
complex
problem
problem solving
solving be represented
represented for
for its
its most
most effective
effective use
use in automatic
automatic inference
inference
processes?
processes? Often,
Often, the knowledge
knowledge obtained
obtained from
from experts
experts is heuristic
heuristic knowledge,
knowledge,
gained
How can this
this knowledge,
knowledge, with
with its
its
gained from
from many
many years
years of
of experience.
experience. How
inherent
inherent vagueness and uncertainty,
uncertainty, be represented
represented and applied?
applied?

1.
n01lliedge representation:
1. 1-(
Knowledge
representation:

\Vork
NEOMYCIN systems
VVork on the logic-based
logic-ba.sed MRS
MRS and the rule-based
rule-based NEOMYCIN
systems
continues,
Numerous copies
cont.inues, attracting
attracting wide
wide interest
interest within
within the AI
AI community.
community. Numerous
of
of MRS
MRS have been sent
sent to collaborators
collaborators elsewhere who
who are experimenting
experimenting with
with
it
book on rule-based
by Buchanan
it on the own
own machines.
machines. The
The book
rule-based expert
expert systems by
Buchanan
8:
& Shortliffe
Short,liffe was completed
completed in this
this year.
year.
[Se('
[See HPP
HPP technical
t,echnical memos HPP-83-26,
HPP-83-26, HPP-83-28,
HPP-83-28, HPP-83-29,
HPP-83-29, HPP-83-34,
HPP-83-34,
HPP-84-1]
HPP-84- 1]
What
What kinds
kinds of
of software
software tools
tools and
system architectures
architectures can be constructed
constructed to
to make
make itit easier to
to implement
implement expert
expert
syst,em
programs with
with increasing
increasing complexity
complexity and high
high performance?
performance? How
How can we
programs
flexible control
control structures
structures for
for powerful
powerful problem
problem solving
solving programs?
programs?
design flexible

2. Advanced
Adt1anced architectures
architectures and
and Control:
Control:

A major
major effort,
effort in exploring
exploring and understanding
understanding the
the Blackboard
Blackboard a.rchitecture
architecture ha.s
has
A
undrrtaken. A
A new pilot
pilot project
project using
using this
this architecture
architecture was st,arted
started in
been undcrbnken.
domain of
of protein
protein chemistry
chemistry (see
(see description
description of
of Jardetzky
Jardetzky &
8[ Buchanan
Buchanan
the domain
pilot project).
project). We
\Ve have also begun
begun investigating
investigating Blackboard
Blackboard systems
systems as a way
way
pilot
of organizing
organizing expert
expert systems
systems to
to exploit
exploit concurrency.
concurrency. Initial
Initial work
work has begun
begun
of
using the HASP/AGE
HASP/AGE systems
systems as an application
application example.
example.
using
[See
HPP t,echnical
technical memos
memos HPP-83-30,
HPP-83-30, HPP-83-33,
HPP-83-33, HPP-83-38,
HPP-83-38, HPP-83-43,
HPP-83-43,
[See HPP
HPP-83-H, HPP-84-4,
HPP-84-4, HPP-84-61
HPP-84-6]
HPP-83-44,

Knowledge acquisition:
acquisition: How
How is knowledge
knowledge acquired
acquired most
most efficiently”from
efficiently"from
3. Knowledge
human expert,s,
experts, from
from observed
observed data,
data, from
from experience,
experience, and
and from
from discovery?
discovery?
human
How can a program
program discover
discover inconsistencies
inconsistencies and
and incompleteness
incompleteness in its
its
How
knowledge base?
base?
How can the
the knowledge
knowledge base be augmented
augmented without
without
knowledge
How
perturbing t,he
the established
established knowledge
knowledge base?
base?
perturbing

A. Feigenba.um
Feigenbaum
E. A.

gO
90

HPP
HPP Core
Core AI
AI Research

5P41 RR00785-11
RR00785-11

\Ve
We have continued
continued to
to make
make progress on two
two on-going
on-going projects
projects for
for learning
learning by
by
experience
experience and learning
learning by
by analogy,
analogy, and have initiated
initiated work
work on three
three new
systems
Those
Those three
three are learning
learning by
by watching,
watching,
systems for
for acquiring
acquiring knowledge.
knowledge.
learning
All
learning from
from text,
text, and learning
learning rules
rules & meta-rules
meta-rules inductively.
inductively.
All three
three of
of
the
t,he new systems
systems use
use medical
medical problems
problems as their
their test-domains.
test-domains.
[Preliminary
[Preliminary results
results have been published
published in HPP-83-27,
HPP-83-27, HPP-83-36,
HPP-83-36, HPP-84-2,
HPP-84-2,
HPP-84-8.]
HPP-84-S.]
4.

Knowledge utilization:
By
By what
what inference
inference methods
methods can many
many sources of
of
hr7owledge
utilization:
knowledge
jointly and effiCiently
knowledge of
of diverse
diverse types
types be made to
to contribute
contribute jointly
efficiently
toward
How can knowledge
knowledge be used intelligently,
intelligently, especially
especially in
toward solutions?
solutions? How
systems with
with large knowledge
knowledge bases,
bases, so that
that it
it is applied
applied in an appropriate
appropriate
manner
manner at the appropriate
appropriate time?
time?
These issues
issues are being
being explored
explored in the
the development
development of
of MRS
MRS (Meta(MetaRepresentation
Representation System)
System) where
where one of
of the roles of
of meta-knowledge
meta-knowledge is to guide
the effective
They are also central
central in the studies
studies
effective use
use of
of lower
lower level knowledge.
knowledge. They
of
of Blackboard
Blackboard control
control systems and their
their use
use in concurrent
concurrent expert
expert systems.
[See
[See HPP
HPP technical
technical memos
membs HPP-83-26,
HPP-83-26, HPP-83-28,
HPP-83-28, HPP-83-30,
HPP-83-30, HPP-83-33,
HPP-83-33,
HPP-83-38,
HPP-83-38, HPP-84-1,
HPP-84-1, HPP-84-2,
HPP-842, HPP-84-6]
HPP-84-61

5. Software
Software Tools:
Tools: How
How can specific
specific programs
programs that
that solve specific
specific problems
problems be
generalized
generalized to more
more widely
widely useful
useful tools
tools to
to aid
a.id in the development
development of
of other
other
programs
programs of
of the same class?
class?
We
We have continued
continued the development
development of
of new software
software tools
tools for
for expert
expert, system
system
construction
construction and the distribution
distribution of
of packages that
that are reliable
reliable enough and
documented
documented so that
that other
other laboratories
laboratories can use
use them.
them. These include
include the old
old
rule-based
rule-based EMYCIN
EMYCIN system,
system, MRS,
MRS, and AGE.
AGE.
[See
[See HPP
HPP technical
technical memos HPP-83-26,
HPP-83-26, HPP-83-28,
HPP-83-28, HPP-83-29,
HPP-83-29, HPP-83-33]
HPP-83-331
6.

Explanation and
How
Explanation
and Tutoring:
Tutoring:
How can the knowledge
knowledge base
base and the line of
of
reasoning
reasoning used in solving
solving a particular
particular problem
problem be explained
explained to
to users?
users? What
What
constitutes
constitutes a sufficient
sufficient or
or an acceptable
acceptable explanation
explanation for
for different
different classes
classes of
of
users?
How can
ca.n knowledge
knowledge in a system
system be transferred
transferred effectively
effectively to students
students
users? How
and trainees?
tra.inees?
The
NEOMYCIN program
comparison
The NEOMYCIN
program has undergone
undergone preliminary
preliminary
comparison with
with
medical
medical students'
students’ protocols
protocols to
to understand
understand the extent
extent to
to which
which its medical
medical
concepts
concepts match
match those of
of the students.
students. Analysis
Analysis of
of experts'
experts’ problem
problem solving
solving has
also been done. NEOMYCIN’
NEOMYCIN's
s explanation
explanation capabilities
capabilities have been improved.
improved.
NEOMYCIN in
New work
work on student
student modelling
modelling has started
started in order
order to
to test
test NEOMYCIN
the cont.ext
context of
of tutoring.
tutoring.
[See
[See HPP
HPP technical
technical memos HPP-83-41,
HPP-83-41, HPP-83-42,
HPP-83-42, HPP-84-2,
HPP-84-2, HPP-84-7]
HPP-84-71

7. Planning
Planning and
Design:
and Design:

planning
planning and design?
design?
const.raints?
const.raint,s? How
How are

What
What are reasonable and effective
effective methods
methods for
for
How
How can symbolic
symbolic knowledge
knowledge be coupled
coupled with
with numerical
numerical
constraints
const,raints propagated
propagated in design problems?
problems?

The
The Palladio
Palladia system
system for
for assisting
assisting in the design of
of VLSI
VLSI circuits
circuits has been
demonstrated
demonstrated and results
results presented
presented in major
major publications
publications and conferences.
[See
[See HPP
HPP technical
t,echnical memos HPP-83-31,
HPP-83-31, HPP-83-39,
HPP-83-39, HPP-83-45,
HPP-83-45, HPP-83-46.
HPP-83-46,
HPP-83-47,
HPP-83-47, HPP-84-3,
HPP-84-3, HPP-84-5]
HPP-84-51
8.

Diagnosis: How
Diagnosis:
How can we build
build a diagnostic
diagnostic system
system that
that reflects
reflects any
any of
of several

E. A. Feigenbaum
Feigenbaum

5P41 RR00785-11
RR00785-11

HPP
HPP Core
Core AI
AI Research

diagnostic
How
use knowledge
How can we use
knowledge at different
different levels of
of
diagnostic strategies?
strategies?
abstraction
process?
abstraction in the diagnostic
diagnostic process?
Research on using
using causal models in a medical
medical decision
decision support
support system
system
(NESTOR)
was
largely
completed
and
will
be
published
in
the
coming
(NESTOR)
largely completed
will
published
coming year.
year. A
A
~econd
program that
physiology
second medical
medical diagnosis
diagnosis program
that uses
uses causal models of
of renal
renal physiology
(AIjMM)
published soon. We
(AI/MM) was also substantially
substantially completed
completed and will
will be published
We are
investigating
investigating the process of
of diagnosis
diagnosis in electronics
electronics as
as well
well as
as in medicine.
medicine. The
The
major
major thrust
thrust of
of this
this work
work has been integrating
integrating causal models about,
about, and the
the
structure
body.
structure of, a computer
computer system
system or
or systems
systems of
of the
the human
human body.
[See
[See HPP
HPP technical
technica. reports:
reports: HPP-83-32,
HPP-83-32, HPP-83-37,
HPP-83-37, HPP-83-40,
HPP-83-40, 84-7]
84-71

D. Relevant
Relevant Publications
Publications
D.
HPP-83-28
HIT-83-20

Miehael
“MRS Casebook",
Casebook “, May
May 1983.
Michael R. Genesereth,
Genesereth, ".MRS

HPP-83-27
EDT’-83-27

Thomas
Thomas

HPP-83-28
HIT-83-28

Michael
Meta-level Representation
Representation System",
Michael R. Genesereth,"A
Genesereth, “A Meta-level
System “, May
Ma.y
1983.

HPP-83-29
HIT-83-29

M. Grinberg,
Installation Instructions
Instruction8",“, May
This
Grinberg, "MRS
“MRS Installation
May 1983.
This
report
purchased the software
report available
available only
only to
t,o those who
who have purchased
software system
system
MRS.
MRS.

HPP-83-30
HIT-83-30

Barbara
Barbara

HPP-83-31
HPP-83-31

Harold
Tong,
An
Harold Brown,
Brown, Christopher
Christopher
Tong, Gordon
Gordon Foyster,
Foyster, "Palladio:
“Palladia: An
Exploratory
Exploratory
Environment for
for IC
IC Design
Design",“, June 1983.
Environment

HPP-83-32
HFP-83-32

John Kunz,
Kunz, E.A.Feigenbaum,
E.A.Feigenbaum, Bruce
Bruce G. Buchanan,
Buchanan, E.H.
E.H. Shortliffe,
Shortliffe,
"Comparison
of
Decision hfaking
Afuking
“Comparison
of Techniques
Techniques of
of Computer-Assisted
Computer--4ssisted Decision
in
publication in the Pure
and
in Aledicine".
Medicine “. Submitted
Submitted for
for publication
Pure
and Applied
Applied
Biostructure.
World
Biostructure.
World Press, Singapore
Singapore (1983).

HPP-83-33
HIT-83-33

Nelleke Aiello,
for Expert
Expert
Nelleke
Aiello, "A
“A Comparative
Comparative Study
Study of
of Control
Control Strategies
Strategies for
SY8tems:
AGE Implementation
Implementation of
PUFF",“, June
Systems: AGE
of Three Yariations
Variations of
of PUFF
1983.

HPP-83-34
HIT-83-34

Jock Mackinlay,
Presentation: The Generation
Problem for
for
Mackinlay, "Intelligent
“Intelligent Presentation:
Generation Problem
User Interfaces
Inter faces ",“, March
March 1983.

HPP-83-36
I-ET-83-30

Russell Greiner
New'? A Semantic
Greiner and ~tichael
Michael R. Genesereth,
Genesereth, "H-1wt's
“What’s New?
Semantic
Definition of
Novelty",“, June 1983.
Definition
of Novelty

HPP-83-37
HIT-83-37

Robert
About Time-dependent
Behavior in
Robert Joyce, "Reasoning
“Reasoning About
Time-dependent Behavior
in a System
System
for Diagnosing
Diagnosing Digital
Digital Hardware
Hardware Faults
Faults",“, August
for
August 1983.
1983.

HPP-83-38
Hpp-83-38

Barbara
Blackboard Model
A/odel of
Barbara Hayes-Roth,
Hayes-Roth, "The
“The Blackboard
of Control
Control ",“, June 1983.

HPP-83-39
mP-83-39

Jerry
Expert System
for
.Jerry Yan,
Yan, Gordon
Gordon Foyster,
Foyster, Harold
Harold Brown,
Brown, "An
“An Expert
Syste,m for
Integrated Circuits",
As.'Jigning
A[ask Levels
Levels to Interconnect
Interconnect in
Assigning
Mask
in Integrated
Circuits “,
October
October 1983.

E. A. Feigenbaum
Feigcnbaum

D. Dietterich
and Ryszard
Dietterich
Ryszard S.
S. Michalski,
Michalski,
Patterns in
Patterns
in Sequences of
of Objects
Objects If,
“, May
May 1983.

"Discovering
“Discovering

Hayes-Roth,
"The
Blackboard Architecture:
Architecture: A
Hayes-Roth,
“The Blackboard
Framework for
for Problem
Problem Solvi1'g'?"
Framework
Solving?” May
May 1983.

92
92

General
General

HPP
HPP Core
Core AI
AI Research
Research

5P411 RR00785RR00785-11
5P4
11

HPP-83-40
HPP-83-40

Benoit Mulsa.nt
Mulsant and
and
Benoit
A
Daily
Activity
on
A Daily Activity on

HPP-83-41
HPP-83-41

(working paper)
paper) Diane
Diane Warner
Warner Hasling,
Hasling, “Strategic
"Strategic Explanations
Explanations for
for aa
(working
Diagnostic Consultation
Consultation System
System “,", in
in MA1
AAAI Proceedings
Proceedings 1983
1983 pp.
pp.
Diagnostic

David Servan-Schreiber,
Servan-Schreiber, “Is been improved
improved with
with a region
region based window
window system
system which
which increases both
both speed and
flrxibility.
The
The region
region based window
window system,
system, the register
register input
input devices, and the
flexibility.
formatting
formatt,ing language
la.ngua.ge interface
interface (that
(that describes how
how forms
forms should
should be presented
presented on the
display)
display) have been generalized
generalized to
to be usable by
by other
other portions
portions of
of the ONCOCIN
ONCOCIN project
project
(notably
(notably the OPAL
OPAL knowledge
knowledge acquisition
acquisition interface
interface described
described below).
below).

C.2.2
Development of
program to
physician inter
face to
C.2.2 Development
of new
new program
to connect
connect the
the physician
interface
to the
the
reasoning
portion of
program
reasoning portion
of .,the
the program
The
The ONCOCIN
ONCOCIN system
system uses
uses a special design that
that allows
allows the Interviewer
Interviewer program
program
a.nd
program to operate
and the reasoning
reasoning section
section of
of the program
operate independently.
independently. In order
order to coordinate
coordinate
the activities
programs, a special communication
a,ctivities of
of these two
two programs,
communication program,
program, called
called the
t,he
Interactor
1nteract)or was drsigned
designed and built.
built.
The
t,eractor program
program provides
provides a message
The In
Interactor
message passing facility
facility between
between two
two or
or more
more
Interlisp-D
processes (sub-programs
Interlisp-D processes
(sub-programs that
that can run
run at the same time).
time). The
The form
form of
of the
messages
The
messa.ges are specified
specified by
by the programmer.
programmer.
The system
system further
further allows
allows messages
messages to
to
processes
machines
processes running
running on different
different
machines via
via the computer
computer network
network called
called the
ETHERNET.
This
ETHERNET.
This will
will allow
allow moving
moving components
components of
of a large program
program from
from one to
to several
machines
The Interactor
Interactor also has the
machines in a way
way invisible
invisible to the programs
programs themselves. The
ability
ability to rind
find other
other Int.eractors
Interactors on the local communication
communication network.
network.

C.2.3
Detleiopment of
programs to
C.2.9 Development
of new
new programs
to improve
improve the
the efficiency
efficiency and
and capabilities
capabilities
of
NCO CIN
0f o
olvc~oc~Iiv

109

E. A.
A. Feigenbaum
Feigenbaum

ONCOCIN
ONCOCIN Project
Project

5P41 RR00785-11

In order
order to speed
speed up both
both versions
versions of the ONCOCIN
ONCOCIN system, we have written
written aa
simple
simple rule and control
control block
block compiler
compiler for
for ONCOCIN
ONCOCIN that
that converts
converts rules and control
control
blocks into
into Interlisp
Interlisp programs,
programs, and then
then into
into compiled
compiled Interlisp.
Interlisp. This
This helps to alleviate
alleviate aa
memory
version
memory space
space problem
problem we have had in the Interlisp-lO
Interlispversion of the system
system as
as well
well as
as
the
workstation
version
of
the
program.
give us increased speed
in
speed
workstation version
program.
Another
Another systems level 3.'>pect
aspect of our
our work
work is in the creation
creation and access
access of efficient
efficient
patient
patient record
record data
data files. To
To this
this end, we have implemented
implemented a machine
machine independent
independent hash
file system
system (special data
data record
record format)
format) that
that allows access
access to the data
data base
base via
via memory
memory
from
and Interlisp-D
The system
system is
is compatible
compatible with
with both
both Interlisp-lO
InterlispInterlisp-D and
from disk files. The
allows sharing
sharing of files between
between the two
two systems. Its
Its format
format is
is also machine
machine independent
independent
enough to allow
allow access
access from
from other
other lisps on other
other computers.
computers. It
It is currently
currently accepted by
XEROX
XEROX as
as a standard
standa.rd for
for the D machines
machines and has been used by them
them to bring
bring up
programs
programs of use
use by all D machine
machine users.
users. Along
Along the same lines, we have experimented
experimented with
wit,h
solutions
solutions to the problems
problems of having
having portions
portions of text
text easily
easily accessible
accessible by key
key from
from a file in
a machine
machine independent
independent way.
way.

C.2.4
Reorganization
and
programs for
for improved
C.2.4
Reorganization
and recoding
recoding of
of existing
ezisting programs
improved efficiency
efficiency
The
The reasoning
reasoning portion
portion of the
t.he ONCOCIN
ONCOCIN program
program is being reprogrammed
reprogrammed to
increase speed and to benefit
from
the
special
capabilities
of
the
Interlisp
benefit from
capabilities
Interlisp workstation.
workstation.
We
are
also
re-writing
parts
of
the
program
that
were
borrowed
We
re-writing parts of
program that
borrowed from
from other
other expert
expert
systems developed
developed by our
our group.
group.
\Ve
We have reorganized
reorganized the system
syst,em into
into logical
logical subsystems
subsystems that
that are of a manageable
manageable
size. This
This consisted
consisted of cat.egorizing
categorizing all the system
system functions
functions (portions
(portions of
of the program)
program) that
that
are necessary for
putting each in an appropriate
The
for the Reasoner to
to run
run and putting
appropriate file.
file. The
Reasoner now runs
runs in stand
stand alone mode independently
independently of
of which
which system
system it
it is on.
programming part
part for
\Ve
We are now
now in the
the process of
of cleaning
cleaning up the specific
specific programming
for each of
of
the subsystems.
both style
subsystems. This
This entails
entails making
making various
various enhancements
enhancements for
for both
style and efficiency,
efficiency,
adding
and further
breaking down
breaking
down functionally
functionally
adding comments
comments and documentation,
documentation,
further
independent
parts
of
the
system.
independent parts of
system.
We
portions of
We have transferred
transferred portions
of our
our EMYCIN
EMYCIN utilities
utilities (based on the MYCIN
MYCIN
expert
both Interlisps.
expert syst.em) and rewritten
rewritten those utilities
utilities to
to make
make them
them work
work in both
Interlisps. We
We
have removed
program that
removed from
from the stand-alone
stand-alone Reasoner sections
sections of
of the
the program
that depended on
specific hardware
hardware of
of the
the DEC-2060
DEC-2050 mainframe
mainframe computer
computer and now
now have versions
versions of
of
the specific
ONCOCIN on the 2060
2050 and D-machines
D-machines that
that are identical,
identical, being
being generated
generated from
from the same
ONCOCIN
program text,.
t.ext. This
This step also included
included the
the use of
of the
the new
new hash file syst,em
system (described
(described
program
D-machines.
above) on the D-machines.

C.2.5 System
Sy.'1tem support
support for
for the
the reorganization
reorganization
C.2.5
implemented a program
program called
called Graphcalls
Graphcalls which
which allows
allows programmers
programmers on
\Ve have implemented
the D-ma.chines
D-machines to
to visually
visually graph
graph the structure
structure of
of the
the programs
programs they
they have written.
written. One
t,he
examine the use of
of each of
of the
the functions
functions on the graph
graph as well
well as examine
examine and
can also examine
variables they
they access.
access. It
provides visual
visual tracing
tracing and
and dynamic
dynamic control
control of
of a
change the variables
It also provides
program in
in execution.
execution. ItIt has been used daily
daily since its
its creation
creation by
by both
both our
our project
project and
pr0gra.m
members of
of the
the SUMEX
SUMEX community.
community.
members

C.B Goal:
Goal: To
To modify
modify the
the 2020 Clinic
Clinic Version
Version of
of ONCOCIN
ONCOCIN in
in response
response to
to user
user
C.9
feedback
feedback
During the
the last
last year,
year, we have added
added a number
number of
of new
new options
options to
to ONCOCIN
ONCOCIN for
for use
During
by the
the fellows
fellows in the
the clinic.
clinic. These include:
include: a special
special option
option to
to request
request that
that a test
test be
by

A. Feigenbaum
Fdgenbaum
E. A.

110

ONCOCIN
ONCOCIN Project
Project

5P41
5P41 RR00785-11
RR00785- 11

ordered
for entering
entering reasons
reasons for
for treatment
treatment
ordered STAT
STAT (immediately),
(immediately), special menus for
modifications
modifications (these are used
used when there is a disagreement between the ONCOCIN
ONCOCIN
recommended therapy
physician's
therapy and the physician’
s treatment
treatment plan, in order
order to
to gather
gather data
data about
about
why
why the physician
physician has decided to override
override the system), and the option
option to
to request a copy of
a patient’
patient's
s flowsheet
flowsheet be printed
printed out
out on the clinic
clinic line-printer.
line-printer. We
We have also streamlined
streamlined
the met.hods
met,hods that
that the various
various forms
forms are created
created by
by ONCOCIN.
ONCOCIN.

C.4
for use
the
used
C.4 Goal:
Goal: To encode
encode and
and implement
implement
for
use by
by ONCOCIN
ONCOCIN
the COfnmonly
commonly
used
chemotherapy
protocols from
from our
chemotherapy
protocols,
our oncology
oncology clinic
clinic
We have pursued two
two approaches to
to increasing the number
number of
of prot.ocols
protocols known
known by
by
our
The first
first approach is to
to use
use the existing
existing software
software to
to implement
implement active
active
our system. The
protocols not
protocols
not encoded
encoded at the time
time of
of our
our last report.
report. The
The second
second approach has been
been to
develop new software
software that
that is able to
to dramatically
dramatically speed
speed up the entry
entry of
of protocols
protocols by
by
providing graphically-oriented
providing
graphically-oriented forms
forms to
to be filled
filled out
out on the computer
computer that
that follow
follow the basic
outline
protocol documents.
outline of the protocol
In t.he
protocol to
t,he past, adding
adding a new protocol
to the ONCOCIN
ONCOCIN knowledge base
base has been
been a
tedious process
process in which
programmer sit
which an oncologist
oncologist and a programmer
sit down and translate
translate the
oncologist's
All
oncologist’s knowledge about
about the protocol
protocol into
into rules accessible
accessible to ONCOCIN.
ONCOCIN.
All the
rules pertaining
pertaining to the new protocol
protocol are written
process must
written at
at that
that time,
time, and this
this process
must be
repeated for
protocol that
This
for every new protocol
that is added to ONCOCIN.
ONCOCIN.
This method
method is rather
rather
inefficient
inefficient since many
many of
of the rules are similar
similar between protocols,
protocols, differing
differing only
only in their
their
data
To
To speed
speed the process
process of knowledge acquisition,
acquisition, a program
program is being
data content.
content.
developed whereby
whereby a doctor
doctor could sit
sit down at a terminal
terminal and fill
fill in a series
series of
of forms
forms
containing
containing appropriate
appropriate questions about
about a new protocol.
protocol. The
The information
information entered would
would
take care of the large number
number of general rules pertaining
pertaining to the protocol
protocol and allow
allow the
doctor
doctor and programmer
programmer to concentrate
concentrate on the special cases.
cases.
The
The program
program will
will have two
two levels, the first
first of
of which
which is the program
program that
that will
will
interact
interact directly
directly with
with the doctor.
doctor. This
This program
program runs on Xerox
Xerox D machines which
which have
extensive graphics
Sections of the display
graphics capabilities.
capabilities.
display screen
screen (called windows)
windows) are
organized in a way
way that
that emulates the physician's
physician’s patterns
patterns of thought
thought when thinking
thinking about
about
the prot.ocoi.
protocol. Other
Other graphical
gra.phical entry
entry devises
devises have been
been used
used to
to encourage
encourage pointing
pointing at the
answer rather
rat.her than
than text
text entry.
entry. These methods are able to display
display all of the possible
choices
choices in a compact
compact and comprehensible way.
way. The
The first
first phase
phase of
of this
this program
program has been
been
complrted
complet.ed and h;JS
has been
been examined and approved
approved by our
our oncology collaborators.
collaborators.
InFormat.ion
Informat.ion entered in the top
top level program
program will
will be
be converted
converted to
t,o an intermediate
intermediat,e
data
data. structure
structure which
which will
will be
be used
used by the second
second level of the program
program to make new rules
for the ONCOCIN
ONCOCIN knowledge base.
base. Eventually,
Eventually, this
this process
process will
will also work
work in the opposite
opposit,e
direction
direclion so that
that information
informa,tion about
about a previously
previously entered protocol
protocol can be copied or
or
modified
modified by the
t,he physician
physician for the new protocol.
protocol. This
This "similar
“similar to"
to” option
option will
will also extend
t·o
t.o chemotherapies
chemot,herapies and drugs, so that
that when the doctor
doctor enters a chemotherapy
chemotherapy or
or drug
drug that
that
the system
systt=rn knows about,
about, pertinent
pertinent information
information will
will be filled
filled in for the doctor
doctor to copy or
or
modify.
modify. The
The "similar
“similar to"
to” capably
capably along with
with the use
use of
of graphical
graphical input
input devices to speed
speed
the process
process of entering
entering a new protocol
protocol and will
will also reduce errors
errors and duplications.
duplications. When
When
this
this project
project is completed
completed the total
total time
time needed
needed to
to enter
enter a new protocol
protocol should be greatly
greatly
reduced and more effort
effort will
will be concentrated
concentrated on fine tuning
tuning the rules to handle special
situations.
situations.

C.5
Evaiuat'ions of
Performance
(2.5 Eualuations
of ONCOCIN's
ONCOCIN'S
Performance
Data
Data collection
collection and analysis for all three ONCOCIN
ONCOCIN evaluations
evaluations are now complete,
complete,
results
results were presented at the annual
annual meeting
meeting of
of the Society
Society for Medical
Medical Decision Making,
Making,
and we expect to have formal
formal reports
reports published
published during
during the next
next year.
111
111

E. A. Feigenbaum
Feigenbaum

ONCOCIN
ONCOCIN Project
Project

5P41 RR00785-11
RR00785- 11

Study
Study 1, overseen by
by Dr.
Dr. Robert
Robert Carlson
Carlson of
of the Division
Division of
of Oncology,
Oncology, is an
evalua,tion of
of the program's
program’s impact
impact on the attitude
attitude of
of the
the oncology
oncology fellows
fellows towards
t,owards
evaluation
computers
All
All physicians
physicians were administered
administered
computers in general
genera.1 and ONCOCIN
ONCOCIN in particular.
particular.
questionnaires and structured
structured interviews
interviews in the Spring
Spring of
of 1981
1981 before
before ONCOCIN
ONCOCIN was
questionnaires
introduced.
intQoduced. The
The same questionnaires
questionnaires were distribut.ed
distributed to them
them again after
after they
they had used
system for
for over
over a year.
year. Follow-up
Follow-up interviews
interviews were also undertaken.
undertaken. This
This study
study was
the system
repeated again
a.ga.in during
during 1983 to
to determine
determine the trends
trends over
over time.
time. The
The results
results of
of this
this study
study
repeated
are presently
presently being
being prepared
prepared as a formal
formal report.
report.
\Ve
We are also revising
revising this
this study
study in preparation
preparation for
for the integration
integration of
of the
workst.ation version
version of
of ONCOCIN
ONCOCIN into
into the clinic.
clinic. To
To maintain
maintain some consistency
consistency in the
workstation
evaluation process, the original
original questions
questions from
from Study
Study 1 will
will be given
given and analyzed
analyzed as
as
evaluation
before along with
with new questions.
questions. Several
Several of
of these new survey
survey and interview
interview questions
questions will
will
before
serve as
as a "baseline"
“baseline” for
for evaluating
evaluating any
any perceived
perceived improvements
improvements that
that will
will come with
with the
introduction
intZroduction of
of the professional
professional workstations
workstations in the clinic.
clinic.

Study 2, overseen by
by Dr.
Dr. Daniel
Daniel Kent
Kent of
of the Division
Division of
of General
General lilternal
Internal Medicine,
Medicine,
Study
is an evaluation
evaluat.ion of
of the program's
program’s impact
impact on the
the completeness
completeness and accuracy
accuracy of
of flowsheet
flowsheet
data
Research programmers
data recorded
recorded with
with and without
without ONCOCIN.
ONCOCIN.
programmers wrote
wrote routines
routines to
to
formally analyze
a.naiyze on-line
on-line flow
flow sheets for
for completeness
completeness and accuracy.
accuracy. Pre-ONCOCIN
Pre-ONCOCIN flow
flow
formally
then entered
entered into
into the system
system exactly
esactly as
as they
they were
were originally
originally
recorded by
by the
the
sheets were then
recorded
physicinn.
The same analytic
analytic routines
routines were used to
to analyze
analyze these pre-ONCOCIN
pre-ONCOCIN flow
flow
physician. The
sheet,s. The
The pre- and postpost- ONCOCIN
ONCOCIN data
data were compared.
compared.
sheet.s.
Results
Results indicat.e that
that
ONCOCIN has had a statistically
statistically significant
significant beneficial
beneficial impact
impact on the completeness
completeness of
of data
data
ONCOCIN
recording, the ordering
ordering of
of required
required tests, and the accuracy
accuracy of
of the data
data recorded.
recorded. A
A formal
formal
recording,
report
report of
of the results
results is in preparation.
preparation.
Finally, Study
Study 3 is examining
examining the comparison
comparison between
between ONCOCIN's
ONCOCIN’s therapeutic
therapeutic
Finally,
treatment decisions made by
by oncology
oncology fellows
fellows in the same setting.
setting. The
The
advice and the treatment
study was coordinated
coordinated by
by Dr.
Dr. David
David Hickam,
Hickam, formerly
formerly of
of our
our Division
Division of
of General
General Internal
Internal
study
hledicine and now
now on the
the faculty
faculty at the University
University of
of Oregon
Oregon in Portland.
Portland.
Medicine
Expert
Expert
evaliuators rated
rated treatment
treatment plans
plans without
without knowing
knowing whether
whether the recommendation
recommendation was that
tha.t
evaluators
of ONCOCIN
ONCOCIN or
or one of
of the clinic
clinic physicians.
physicians. Over
of
Over 200 flow
flow sheets were evaluated
evaluated by
by
Stanford lymphoma
lymphoma experts,
experts, and the resulting
resulting data
data have been fully
fully analyzed
analyzed by
by Dr.
Dr.
Stanford
Hick-am. The
The results
results indicate
indicate that
that the experts
experts were unable
unable to
to fault
fault the recommendations
recommendations
Hickam.
by ONCOCIN
ONCOCIN relative
relative to those of
of experienced
experienced oncology
oncology fellows
fellows treating
treating patients
patients
made by
with
wit,h lymphoma.
lymphoma.. A
A paper
pa,per describing
describing the results
results is in preparation.
preparation.
A stlldy
study was made
ma.de of
of all
all of
of the cases
cases run
run by
by phYSicians
physicians in the clinic
clinic to determine
determine
A
st.atist.ics
when they
they chose
chose to
to override
override ONCOCIN's
ONCOC’I~N’s therapy
therapy recommendation.
recommendation. The
The
stat
istks about. when
result#s showed that
that approximately
approximately 75%
75% of
of the time
time they
they agreed completely.
completely. \Vhen
When there
there
resl1lts
disagreements, 15%
15% were about
about individual
individual drug
drug doses.
doses. This
were disagreements,
This study
study pointed
pointed out
out a
number of
of situations
situations where
where ONCOCIN
ONCOCIN needs more
more knowledge,
knowledge, and
snd where
where our
our expert
expert
nnmber
clarificntion from
from the Principal
Principal Investigators
Investigators of
of the particular
particular protocol.
protocol.
needed clarificar.ion
As a
result.. a meeting
meet,ing was
wa.s held (7/12/83)
(7/12/83) with
with some of
of the Faculty
Faculty in charge of
of the Hodgkin's
Hodgkin’s
result.
prot,ocols to discllsS
discllss issues
issues arising
arising from
from this
this study.
study.
protocols
C.6 Documentation
C.6
Documentation

An extensive
extensive effort
effort to
to document
document the ONCOCIN
ONCOCIN system
system was completed
completed during
during this
this
An
year. Many
Many aspects of
of the ONCOCIN
ONCOCIN program
program and its
its programming
programming environment
environment are
last year.
now \vritten
writ,ten and available
available for
for project
project members'
members’ use.
use. The
now
The increase in documentation
documentation has
significantly reduced
reduced the start-up
start-up time
time for
for new researchers
researchers working
working with
with the project.
project. In
significantly
addition, we have published
published several papers
papers and prepared
prepared several
several technical
technical reports
reports
addition,
describing the system.
syst,em.
describing

E. A. Feigenbaum
Feigenbaum

112
112

ONCOCIN
ONCOCIN Project
Project

5P41 RR00785-11
RR00785-11

C.7
Hypothesis Assessment
Assessment
C.7 Hypothesis
As mentioned
mentioned above, largely
largely through
through the
the efforts
efforts of
of Curtis
Curtis Langlotz,
Langlotz, we have
continued
continued to
to develop modifications
modifications to ONCOCIN
ONCOCIN that
that will
will permit
permit it
it to
to function
function as
as an
"observer"
physician's
“observer” of
of the physician’
s own
own decisions rather
rather than
than as
as a primary
primary source of
of advice. By
By
permitting the physician
physician to
plan on the flowsheet,
permitting
to enter
enter his or
or her
her own
own therapy
therapy plan
flowsheet, we can
acknowledge
acknowledge the oncologist's
oncologist’s ability
ability to
to reach
reach appropriate
appropriate therapeutic
therapeutic decisions for
for most
most
patients. ONCOCIN
physician's
plan with
patients.
ONCOCIN will
will simply
simply compare
compare the physician’
s plan
with what
what it
it believes is the
proper therapy.
If
physician, or
If the system
system agrees with
with the
the physician,
or determines
determines that
that small
small
proper
therapy.
differences
differences are clinically
clinically insignificant,
insignificant, no advice from
from the computer
computer will
will be necessary. If
If
significant
significant disagreements
disagreements occur,
occur, on the other
other hand,
hand, ONCOCIN
ONCOCIN will
will need to respond
respond with
with
warnings
plan may
warnings and explanations
explanations for
for why
why it
it feels that
that an alternate
alternate therapy
therapy plan
may be
preferable. Our
preferable.
Our experience with
with ONCOCIN
ONCOCIN since its
its clinic
clinic implementation
implementation suggests that
that
tllis
preferred by
physicians. It
this mode of
of interaction
interaction will
will be preferred
by the clinic
clinic physicians.
It will
will require
require minimal
minimal
changes
decision
chnnges to ONCOCIN's
ONCOCIN’s
decision making
making approach,
approach, but
but the
the determination
determination of
of what
what
differences
differences are clinically
clinically significant,
significant, and the optimal
opt,imal method
method for
for explaining
explaining their
their
importance
physician, are exciting
problems.
importance to the physician,
exciting challenges and important
important theoretical
theoretical problems.
An
International Journal
Journal
An initial
initial report
report, describing
describing this
this work
work appeared
appeared during
during 1983 in the International
of
Al11tl-Afachine Studies,
plan to continue
of Man-i\lachine
Studies, and we plan
continue enhancing
enhancing the
the system's
system’s critiquing
critiquing and
explanation
Mr.
Mr. Langlotz
Langlotz presented
presented this
this work
work in the 1983 Society
Society for
for
explanation capabilities.
capabilities.
Computer
Computer Applications
Applications in Medical
Medical Care
Care Conference
Conference Student
Student Paper
Paper Competition,
Competition, and was
a finalist
finalist in the competition.
competition. The
The approach
approach will
will not
not be used in the clinic,
clinic, however,
however, llDtiI
until
ONCOCIN
professional workstations,
ONCOCIN has been transferred
transferred to
to professional
workstations, hopefully
hopefully in about
about two
two
years.

C.7
Rule Analysis
Analysis
C.? Query
Query System
System and
and Rule
Shako
Shoko Tsuji
Tsuji has completed
completed her work
work on the development
development of
of a query
query system
system to
permit
permit easy access
access to the large ONCOCIN
ONCOCIN knowledge
knowledge base. Once we had encoded several
several
hundred
hundred rules.
rules, it
it became unwieldy
unwieldy for
for system
system builders
builders to work
work from
from large hard-copy
hard-copy
listings
listings of
of the
t,he knowledge
knowledge base,
base, and we anticipate
anticipate that
that physicians
physicians will
will also require
require direct
direct
access
access to the program's
progra.m’s knowledge.
knowledge. The
The query
query system
system permits
permits this
this kind
kind of
of access.
access. Rather
Rather
than
than dealing
dea.ling with
with natural
natural language understanding
understanding by
by computer,
computer, we are designing
designing ways
ways
that
that menu
menu selection
select,ion and the high-speed
high-speed interface
interface can be used to
to permit
permit access
access to the
information
information that
that is needed by
by a physician
physician or
or system
system builder.
builder. A
A paper
paper describing
describing the early
early
work
A,A.MSI Congress
work was presented
present.ed last year
year (May
(May 1983) at the A4MSI
Congress 83
8.Y in San Francisco.
Francisco.
In previous
previous reports
reports we also described
described the work
work of
of Dr.
Dr. Motoi
Motoi Suwa
Suwa who
who developed
developed
programs to 3:"sist
programs
a.?sist. in determining
determining knowledge
knowledge base
base consistency
consistency and completeness. His
paper on this
AI Magazine.
Afagazine. However,
paper
t,his subject
subject appeared
appeared in late 1982 in the AI
However, the programs
programs
that.
t,hnt, he wrote
wrote were never
never formally
formally linked
linked to
to our
our system
system for
for writing
writing rules and modifying
modifying
otller
parts of
Noble spent
ot)her p;Lrts
of tlle
t,he knowledge
knowledge base.
base. As a result,
result, Mr.
Mr. Robert
Robert Noble
spent time
time during
during the
last few years modifying
modifying Suwa's
Suwa’s code so that
that it
it would
would operate
operat,e as
3s an integral
integral part
part of
of
ONCOCIN.
These changes
ONCOCIN.
cha.nges have now
now been implemented
implemented so that
that a new rule
rule can be
dynamically
dynnmicslly cOlllpared
compared to the rest of
of the knowledge
knowledge base
base during
during the process of
of knowledge
knowledge
entry.
Noble is currently
entry. Mr.
Mr. Noble
currently considering
considering how
how such a program
program might
might be implemented
implemented on
a workstation
workstation in order
order to take
take advantage
advantage of
of the newly
newly available
available graphical
graphical capabilities
capabilities of
of
these macllines.
machines.

C.g
Encoding of
Additional Fkotocols
Protocols
C.9 Encoding
of Additional
As was indicated
to a
indicated above, we have emphasized
emphasized transfer
transfer of
of ONCOCIN
ONCOCIN
professional workstation
rather
of
professional
workstation
rather than
than the implementation
implementation
of additional
additional protocols.
protocols.
However,
However, the oncologist
oncologist in charge of
of breast
breast cancer treatment
treatment at Stanford
Stanford had expressed
great
great interest
interest in adding
adding those treatment
treatment protocols
protocols to
to the system
system as
as soon as
as possible. We
We

113

E. A. Feigenbaum
Feigenbaum

5P41 RR00785-11
RR00785-11

ONCOCIN Project
Project
ONCOCIN

accordingly encoded and
and thoroughly
thoroughly tested
tested the
the treatment
treatment plans
plans for
for adjuvant
adjuvant therapy
therapy
have accordingly
of breast
breast carcinoma
carcinoma (CMF
(CMF and
and CMFVP
CMFVP treatment
treatment plans)
plans) and
and released them
them for
for regular
regular
of
the spring
spring of
of this
this year.
year. Encoding
Encoding of
of the
the CMF
CMF treatment
treatment plan
plan required
required encoding
encoding of
of
use in the
special rule
rule types.
types. In
In order
order to
to represent
represent these treatment
treatment plans
plans special
special methods
methods were
were
special
created for
for looking
looking back
back to
to previous
previous cycles to
to compa.re current
current laboratory
laboratory results
results to
to
created
previous va.lues.
values. This
This allows
allows the
the development
development of
of treatment
treatment recommendations
recommendations based upon
upon
previous
past experience
experience with
with the
the patient.
patient. A
A number
number of
of other
other protocols
protocols were
were added
added to
to the
the
past
ONCOCIN system
system in order
order to
to keep the
the system’
system's
knowledge about
about Hodgkin’
Hodgkin's
and
ONCOCIN
s knowledge
s and
These included
Lymphoma protocols
protocols current.
current.
included new Lymphoma
Lymphoma protocols
protocols with
with very
very
Lymphoma
complex alternating
alternating chemotherapies
chemotherapies (M-HOP/B-Cepp/HD-MTX
(M-HOP jB-CeppjHD-MTX and M-BACOD/HDM-BACODjHDcomplex
MTX), and new Hodgkin’
Hodgkin's
protocols (alternating
(alternating MOPP/ABVD).
MOPP j ABVD).
hlTX),
s protocols

C.l0 Strategic
Strategic Therapy
Therapy Planning
Planning
C.10
As mentioned
mentioned above, we have begun
begun a new
new research
research project
project to
to study
study the
the therapy
therapy
As
planning process, and how
how strategies
strategies which
which are used to
to plan
plan therapy
therapy in difficult
difficult cases
cases
pla.nning
might be represented
represented on a computer.
computer. This
This project,
project, which
which we call
call the
the ONYX
ONYX project,
project, has
might
as it,s
its goals: to
to conduct
conduct basic research
research into
into the
the possible representations
representations of
of the
the therapy
therapy
planning process; to
to develop
develop a computer
computer program
program to
to represent
represent. this
this process:
process; and eventually
eventually
planning
t.o interface
interface the
the planning
planning program
program with
with ONCOCIN.
ONCOCIN. The
The project
project members
members (Fagan,
(Fagan,
to
Bischoff, Williams,
Williams, Langlotz,
Langlotz, and Rennels)
Rennels) have spent
spent many
many hours
hours meeting
meeting with
with Dr.
Dr. Sikic
Sikic
Bischoff,
trying to
to understand
underst.and how
how he plans
plans therapy
therapy for
for patients
patients whose special
special clinical
clinical situation
situation
trying
precludes following
following the
the standard
standard therapeutic
therapeutic plan
plan described
described in the
the prot.ocol
protocol document.
document. In
In
precludes
March of
of t)his
this year,
year, the
the group
group spent
spent two
two days at
at Xerox
Xerox Palo
Palo Alto
Alto Research Center
Center
binrch
(PAHe), working
working with
with Mark
Mark Stefik,
Stefik, Daniel
Daniel Bobrow
Bobrow and Sanjay
MiHal of
of PARC
PARC on
(PARC),
Sanjay Mittal
possible representations
representations for
for the knowledge
knowledge structures
structures and how
how such a program
program might
might run
run
We hope to
to have a prototype
prototype of
of this
this
using the LOOPS
LOOPS knowledge
knowledge programming
programming system.
system. We
system
system running
running this
this year.
year.

D. Publications
Publications Since
January 1989
D.
Since January
1983
Expert systems
systems research:
research: modeling
modeling the
the
Expert
In: An
to Monitoring
medical
An Integrated
Integrated Approach
Approach to
Afonitoring
medical decision
decision malcing
making process. In:
(J.S. Gravenstein,
Newbower, A.K.
N.T. Smith,
Gravenstein, R.S. Newbower,
A.K. Ream, and N.T.
Smith, eds.), pp.
183-:!OO.
183-200, \Voburn,
Woburn, MA:
MA: Butterworth's,
Butterworth’s, 1983.

1. (*)
(*) Shortiiffe,
Shortiiffe, E.H.
E.H. and Fagan,
Fagan, L.M.
L.M.

2.
2. Duda,
Dudn,

R.O.
R.O.

and Shortliffe,
Shortliffe,

E.H.
E.H.

Expert systems research.
Expert

Science,
Science,

220:251-258
220:261-268 (1983).

Adapting a consultation
consultation system
system to
3. (*)
(*) Langlotz,
La,nglot,z, C.P.
C.P. and Shortliffe,
Shortliffe, E.H.
E.H. Adapting
International
Journal
of Man-Machine
Studies,
critique
International
Journal
of
]t.lan-Afachine Studies,
critique user plans.
19:479-495
19:479-496 (1983).

4.
Graphical access
access to the knowledge
knowledge base
base of
of a
4. (*)
(*) Tsuji,
Tsuji, S.
S. and Shortliffe,
Shortliffe, E.H.
E.H. Graphical
Proceedings
of
Congress
89,
medical
medical consultation
consultation system.
Proceedings
of AAAfSI
AAMSI
Congress
83, pp
551-555. San Francisco,
Francisco, CA,
CA, May
May 1983.
1983.
5. (*)
(*) Bischoff,
Bischoff, M.B.,
M.B., Shortliffe,
Shortliffe, E.H.,
E.H., Scott,
Scott, A.C.,
A.C., Carlson,
Cnrlson, R.W.
R.W. and Jacobs,
C.D.
C.D. Integratio:l
Integration of a computer-based
computer-based consultant
consultant into
into the clinical
clinical setting.
setting.
Proceedings
7th
Symposium
on
Applications
in
Proceedings
7th Annual
Annual
Symposium
on Computer
Computer
Applications
in Afedical
Medical
Care,
October 1983,
1983, Baltimore,
Baltimore, Maryland.
Maryland.
Care. pp. 149-152. October
6. Mulsant,
Rlulsant, B. and Servan-Schreiber,
Servan-Schreiber, D.: Knowledge
Knowledge engineering:
engineering: a daily
daily activity
activity
on a hospital
and
Re.ion of confounding
confounding time
time intervals,
intervals, or
or statistical
statistical control.
control. The
The Study
Study Module
Module then
then
determines
detrrmines an appropriate
appropriate statistical
statistical method,
method, using knowledge
knowledge stored
stored as
as production
production rules.
\10st.
1lost studies
studies have used a longitudinal
longitudinal design involving
involving a multiple
multiple regression model
model applied
applied
to individual
individual pat.ient
pat,ient records.
records. Results
Results across patients
patients are combined
combined using weights
weights based
on the
t,he precision
precision of the estimat.ed
estimat,ed regression coefficient
coefficient for
for each patient.
patient.

B
\/edical Relevance
B..•,Weciical
Relevance and
and Collaboration
Collaboration
As a t.est
t,est bed for
for system
syst,em development
development our
our focus of
of attention
attention has been on the
records
records of patients
patients with
with systemic
systemic lupus
lupus erythematosus
erythematosus (SLE)
(SLE) contained
contained in the Stanford
Stanford
port.ion
port,ion of the ARAMIS
ARAMIS Da.t.a
Da,t,a Bank.
Bank. SLE
SLE is a chronic
chronic rheumatologic
rheumatologic disease
disease with
with a broad
broad
spectrum
spectrum of manifestations.
manifest.ations. Occasionally
Occasionally the disease
disease can cause
cause profound
profound renal
renal failure
failure and
lead to an early
early dt'ath.
death. \Nith
With many
many perplexing
perplexing diagnostic
diagnostic and therapeutic
therapeutic dilemmas,
dilemmas, it
it is a
disea.se
of
considerable
medical
interest.
disease of considerable medical interest.
In the fut.lIre
project users of
future we anticipate
anticipate possible collaborations
collaborations with
with other
other project
of the
TOO
National Stroke
Northern California
TOD Syst.em
System SllCh
such 3,S
as the National
Stroke Data
Data Bank,
Bank, the Northern
California Oncology
Oncology
Gro1lp,
Group, and the
t,he St,anford
St,a.nford Divisions
Divisions of
of Oncology
Oncology and of
of Radiation
Radiation Therapy.
Therapy.
\Ve
project is broadly
broadly applicable
We believe that
tha.t this
t,his research
resea.rch project
applicable to the entire
entire gamut
gamut of
of
chronic
bulk of
chronic diseases
diseases that
that constitute
constit,ute the bulk
of morbidity
m0rbidit.y and mortality
mortality in the United
United States.
States.
Consider
Consider flY\"
five major
major diagnost.ic
diagnost,ic categories
categories responsible
responsible for
for approximately
approximately two
two thirds
thirds of
of the
two
two million
million deaths
deat,hs per year
year in the United
United States: myocardial
myocardial infarction,
infarction, stroke,
stroke, cancer,
hypertension,
Therapy
hypertension. and diabetes.
diabetes.
Therapy for
for each of
of these diagnoses is fraught
fraught with
with
controversy concerning
concerning the balance
balance of
of benefits
benefits versus costs.
controversy
l. Myocardial
Myocardial Infarction:
Infarction: Indications
Indications for
for and
and efficacy
efficacy of
of coronary
coronary artery
artery bypass
1.
graft vs. medical
medical management
management a,lone.
alone.
Indications for
for long-term
long-term
grsft
Indications
antiarrhythmics ..... . long-term
long-term anticoagulants.
anticoagulants. Benefits
Benefits of
of cholesterol-lowering
cholesterol-lowering
antia.rrhythmics
diet.s, exercise, et,c.
etc.
diets,

Strokr: Efficacy
Ef'ficacy of
of long-t,erm
long-term anti-platelet
anti-platelet agents, long-term
long-term anticoa.gulation.
anticoagulation.
2. Stroke:
Indications for
for revascularization.
revascularization.
Indications
Cancer: Relative
Helative efficacy
efficacy of
of radiation
radiation therapy,
therapy, chemotherapy,
chemotherapy, surgical
surgical excision
excision
3. Cancer:
singly or
or in combination.
combination. Optimal
Optimal frequency
frequency of
of screening
screening procedures.
procedures.
- singly
Prophylactic thera.py.
therapy.
Prophylactic
4. Hypertension:
Hypertension: Indications
Indications for
for
chronic
an
tihypertensive
drugs.
chronic antihypertensive drugs.
arteriography in
in work-up.
work-up.
arteriography

therapy. Efficacy
Efficacy versus
versus
therapy.
Role of
of various
various diagnostic
diagnostic

adverse effects
effects of
of
tests
such
as
renal
tests
renal

Diabetes: Influence
Influence of
of insulin
insulin administration
administration on microvascular
microvascular complications.
complications.
5. Diabetes:
of ora.
or::!,l hypoglycemics.
hypoglycemics.
Role of
Feigenbaum
E. ‘A.
4. Feigenbsum

-..

1'>'>
132

RADIX
RADIX Project
Project

5P4l
5PA 1 RR00785-11
RR00785- 11

Despite the expenditure
expendit,ure of billions
billions of dollars
dollars over recent
recent years for randomized
randomized
controlled
controlled trials
trials (RCT's)
(RCT’s) designed to answer these and other
other questions, answers have been
been
slow in coming.
coming. ReT's
RCT’s are expensive of funds and personnel. The therapeutic
therapeutic questions in
clinical
clinical medicine
medicine are too numerous
numerous for each
each to be
be addressed by its own series
series of RCT's.
RCT’s.
On t.he
t,he other
ot,her hand.
hand, the data
data regularly
regularly gathered
gathered in patient
patient records
records in the course of
the
t.he normal
normal performance
performance of health
health care delivery
delivery are a rich
rich and largely
largely underutilized
underutilized
resource. The
of these data
The ease
ease of accessibility
accessibility and manipulation
manipulation
data afforded
afforded by
computerized
computerized clinical
clinical databases holds out
out the possibility
possibility of a major
major new resource for
for
acquiring
acquiring knowledge
knowledge on the evolution
evolution and therapy
therapy of chronic
chronic diseases.
diseases.
The
The goal of the research that
that we are pursuing
pursuing on SUMEX
SUMEX is to increase the
reliability
reliability of knowledge
knowledge derived
derived from
from clinical
clinical data
data banks with
with the hope of providing
providing a new
tool
tool for augmenting
augmenting knowledge
knowledge of diseases
diseases and therapies
therapies as
as a supplement
supplement to knowledge
knowledge
derived
the incorporation
derived from
from formal
formal prospective
prospective clinical
clinical trials.
trials. Furthermore,
Furthermore,
incorporation of
knowledge
knowledge from
from both
both clinical
clinical data
data banks
banks and other
other sources into
into a uniform
uniform knowledge
knowledge base
base
should
should iflC'rease
increase the ease
ease of
of access
access by individual
individual clinicians
clinicians to this
this knowledge
knowledge and thereby
thereby
facilitate
facilitate both
both the
t,he practice
pract.ice of
of medicine
medicine as
as well
well as
as the investigation
investigation of
of human
human disease
disease
processes.
processes.

C.
of
C. Highlights
Highlights
of Research
Research Progress
Progress
C.l
C.1 1 .May
May 1988
198.9 to
to 1 May
May 1984
198;5
Our
Our primary
primary accomplishments
accomplishments in this
this period
period have been the following:
following:
1) complete
complet,e modifications
modifications to RADIX
RADIX to accommodate
accommodate the one hundred-fold
hundred-fold increase
patients,
in the size of
of our
our database to
to 1700 patients,

2) carry
prednisone on serum
carry out
out the study
study of
of the effect
effect of
of prednisone
serum cholesterol
cholesterol on the new
database.
database.
3) publish
publish results
patient prednisonejcholesterol
prednisone/cholesterol study,
results of
of the 1700 patient
study,

4) publish
publish the description
by us to this
description of
of a two-stage
two-stage regression method
method adapted
adapted by
this

stlldy.
study,
5) complete
preparation for
complete System
System Programmer's
Programmer’s !v1:1nuals
Manuals and User's
User’s Manual
Manual in preparation
fo1
transfer
tra.u?fer to
to outside
olltside sites, and
5) begin transfer
transfer of
of RADIX
RADIX to
to Xerox
Xerox D-Machine
D-Machine personal
personal work
work stations.
stations.
6)

C.l.l Modifications
j'IIodifications to
to RADIlr’
RADIX
the enlarged
enlarged database
database
C.l.l
for for the
Extensive modifications
modifications to
to RADIX
RADIX were
were required
required to
to deal with
with the
the 100-foid
Extensive
lOO-fold
in the
the size of
of the database.
database. The
The modifications
modifications necessary to
to run
run the study
study
automatically on the prednisone/cholesterol
prednisone/cholesterol study
study were completed
completed this
this year.
year.
automatically

increase
module
module

C.i.E Prednisone/chlouterol
Prednisone/chlosterol study
study on
on enlarged
enlarged database
database
C.1.2
\Ve have carried
carried out
out the
t,he automated
automated study
study of
of the
the effect
effect of
of prednisone
prednisone on serum
serum
We
cholesterol using
using the
the new
new 1700 patient
patient database.
database. ItIt has strongly
strongly confirmed
confirmed the
the effect
effect
cholest,erol
previously observed
obst'rved in the
the 50-patient
50-patient SLE
SLE database.
da.tabase. In
In addition,
addition, we are examining
examining the
the
previously
effect in non-SLE
non-SLE patients
patients and
and in
in other
other patient
patient subsets. We
We are also examining
examining alternative
alternative
effect
pharmacokinetic models
models for
for the
the prednione
prednione effect
effect using
using the
the newly
newly available
available data.
data.
pharmscokinetic

C.1.3 Publish
Publish results
results of
of predn.isone/cholesterol
prednisone/cholesterol study
study
C.1.9

123

A. Feigenbaum
Feigenbaum
E. A.

5P41 RR00785-11
RR00785- 11

RADIX
RADIX Project
Project

The
paper reporting
The paper
reporting these results
results is in draft
draft form.
form. It
It will
will be submitted
submitted for
for
publicat.ion shortly.
publication
shortly.

C.LI
Publish description
of
method
C.1.4 Publish
description
of 2-stage
!&stage regression
regression
method
A
publicat.ion.
A description
description of
of the 2-stage
Z-sta.ge regression method
method has been submitted
submitted for
for publicat,ion.

C.l.5
Documentation
C.l.5 Document&ion
A
A two-volume
two-volume System
System Programmer's
Programmer’s Manual
Manual and a User's
User’s Manual
Manual describing
describing
implementation,
implementation, maintenance
maintenance and use of
of the
the system
system at
at Stanford
Stanford has been completed.
completed. In
addit.ion,
addit,ion, a complete
complete set of
of the files needed for
for on-line
on-line demonstrations
demonstrations has been created,
created,
separating
separating them
them from
from the working
working versions.
versions.

C.1.6
RADIX to
D-Machines
CJ.6 Tran.ser
Transer of
of RADIX
to D-Machines
Prl.'liminary
Preliminary work
work on implementing
implementing RADIX
RADIX on D-Machines
D-Machines has begun. This
This will
will
cont.inue
continue in coming
coming years.
years.

C.l.7
C. 2.7 Other
Other accomplishments
accomplishments
\Ve have present.ed the results
results of
of our
our research at
at several
several conferences during
during the
year.
publications for
publications.
year. Additional
Additional publications
for the year
year are noted
noted in the section
section on publications.
C.2
Research in
Progress
C.2 Research
in Progress
\Ve
patient
We are currently
currently completing
completing additional
additional studies
studies on subsets of
of the
the 1700 patient
database.
These
include
automated
analysis
of
the
prednisone/
cholesterol
effect
in
nondatabase.
include automated analysis of
prednisonel cholesterol effect
SLE
patients and subsets of
patients, and fitting
pharmacokinetic models
SLE pa.tients
of SLE
SLE patients,
fitting alternative
alternative pharmacokinetic
of
prednisone/cholesterol effect.
of the prednisone/cholesterol
effect. This
This work
work should
should be completed
completed shortly.
shortly. We
We will
will then
then
return
below in the
ret,urn to
to the more
more AI-oriented
AI-oriented aspects of
of RADIX,
RADIX, as
as described
described below
the section
section on
Resea.rch
Resea.rch Plans.
Plans.

D. Publication8
Publications
D.
1. Blum.
Regression: Application
Application to
Clinical
Blum. R.L.:
R.L.: Two
Two Stage
Stage Regression:
to a Time-Oriented
Time-Oriented
Clinical

Database. (Submitted
Dcltabnue.
(Submitted
Epidemiology.)
Epidemiology.)

for
for

publication
publication

to
to

the

American
American

Journal
Journal

of
of

R.L.:
Prednisone Elevates
Elevates Cholesterol:
An
Automated Study
R.L.: Prednisone
Cholesterol:
An Automated
Study of
of
Longitudinal Clinical
Data. (Manuscript
preparation.)
Longitudinal
Clinical
Data.
(Manuscript in preparation.)

2.
9. Blum,
Blum,

3.
Minimycin: AA !vfiniature
Rule-Based System
nnd Walker,
Walker, M.G.:
M.G.: Minimycin:
Miniature
Rule-Ba8ed
System
3 . Blllm,
Bliim, R.L
R.L.... and

(Submitted
publication to
(Submitted for
for publication
to M.D.Computing)
M.D.Computing)
R.L.:
A/odding and
encoding
clinical
causal
R.L.:
Modeling
and
encoding
dinid
CCJu8al relationships.
relationships.
Proceedings
Proceedings of
of SCAMC,
SCAMC, Baltimore,
Baltimore, MD,
MD, October,
October, 1983.

4. Blum,
Blum,

5. Blum,
Representation of
derived
Blum, R.L.:
R.L.: Representation
of empirically
empirically
deriued causal
causal relationships.
relationships.
IJCAI,
IJCAI, Karlsruhe,
Ksrlsruhe, \Vest
West Germany,
Germany, August,
August, 1983 .

R.L.:
representation
of
Machine
representation
0 / clinical
clinical
R.L.: !v/achine
MEDINFO
MEDINFO 83, Amsterdam,
Amsterdam, August,
August, 1983.

causal
relationships.
causal
relationships.

R.L.:
decision
making
aboard
Clinical
decision
making
aboard the
the
R.L.: Clinical
Chairman's
paper,
Session
on
Artificial
Intelligence
Chairman’s paper, Session
Artificial
Intelligence
Making,
hlaking, AAMSI,
AAMSI, San Francisco,
Francisco, May,
May, 1983.

Starship
Enterprise.
Starship
Enterprise.
and Clinical
Clinical Decision
Decision

5.
G. Blum.
Blum.
7. Blum,
Blum,

8. Blum.
hypotheses
on
Studying
hypotheses
on a time-oriented
time-oriented
Blum, R.L.
R.L. and Wiederhold,
Wiederhold, G.: Studying

E. A. Feigenbaum
Feigenbaum

124

RADIX
RADIX Project
Project

5P41 RR00785-11
RR00785-11

database:
.An
An overvie'w
of
RX project.
project. Proc.
overview
of the
the Rx
Proc. Sixth
Sixth SCAMC,
SCAMC, IEEE,
IEEE,
database:
\Vashington
D.C.,
October,
1982.
\Vashington D.C., October,
9. Blum,
Induction of
from a time-oriented
clinical
Blurn, R.L.:
R.L.: Induction
of causal
causal relationships
relationship8
from
time-oriented
clinical
database:
An overview
project. Proc.
database: An
overview of
of the
the R:'<:
RY project.
Proc. AAAI,
AAAI, Pittsburgh,
Pittsburgh, August,
August,
1982.
1982.

10. Blum,
Automated induction
of
from a timeinduction
of causal
causal relationships
relationships
from
timeBlum, R.L.:
R.L.: Automated
oriwted
RX project. Proc.
oriented clinical
clinical database:
database: The
The RXproject.
Proc. AMIA
AMIA San Francisco,
Francisco, 1982.

Discovery and
Representation of
Relationships from
from a
11.
11. Blum,
Blum, R.L.:
R.L.: Discovery
and Representation
of Causal
Causal Relationships
Clinical
Database: The
RX Project.
Project. IN
Large Time-oriented
IN D.A.B.
D.A.B. Lindberg
Lindberg
Large
Time-oriented
Clinical
Database:
The RX
and P.L.
NOTES IN
P.L. Reichert.z
Reichertz (Eds.),
(Eds.), LECTURE
LECTURE NOTES
IN MEDICAL
MEDICAL INFORMATICS,
INFORMATICS,
SpringerVerlag, 1982.
Springer-Verlag,
1982.
Discovery, confirmation,
and
incorporation
12. Blum,
Blum, R.L.:
R.L.: Discovery,
confirmation,
and
incorporation
relationships
from
a
large
time-oriented
clinical
database:
The
relationships
from
large time-oriented
clinical
database:
The
Computers
April,
Computers and Blomed.
Biomed. Res.
Res. 15(2):164-187,
15(2):161-187,
April, 1982.
1982.

of
causal
of
causal
RX
project.
RX project,

13.
Discovery and
of
from a
13. Blum,
Blum, R.L.:
R.L.: Discovery
and representation
representation
of causal
causal relationships
relationships
from
large
time-oriented
clinical
database:
The
RX project
project (Ph.D.
large
time-oriented
clinical
database:
The RX
(Ph.D. thesis).
thesis).
Computer
Computer Science and Biostatistics,
Biostatistics, St.anford
Stanford University,
University, 1982.
1982.
R.L.:
Displaying clinical
data
from a time-oriented
clinical
data
from
time-oriented
R..L.: Di8playing
Computers
1981.
Computers in BioI.
Biol. and Med. 11(4):197-210,
11(4):197-210,
1981.

14.
14. Blum.
Blum.

database.
database.

15.
Automating the
study of
hypothe,'3es
15. Blum,
Blum, R.L.:
R.L.: Automating
the study
of clinical
clinical
hypotheses on
on a time-oriented
time-oriented
databa,'le:
R.Y project.
project. Proc.
database: The
The RY
Proc. MEDINFO
MEDINFO 80, Tokyo,
Tokyo, October,
October, 1980, pp.
4,56-4130.
456-460. (Also
(‘41~0 STAN-CS-79-816)
STAN-CS-79-816)
16.
Inferring
knowledge
from clinical
data
jerring
knowledge
from
clinical
data
16. Blum,
Blum, R.L.
R.L. and Wiederhold,
Wiederhold, G.: In
utilizing
techniques
from artificial
intelligence.
Proc.
banks
technique8
from
artificial
intelligence.
Proc. Second
banks
utilizing
SCMvlC,
November, 1978.
SCAh3C:. IEEE,
IEEE, Washington,
Washington, D.C.,
D.C., November,
1978.
Blum. R.L.:
R.Y
17. .Blum.
R.L.: The
The RY
clinical
data
clinical
data banking
banking
University
University Ph.D.
Ph.D. thesis
thesis

project: AA medical
consultation
system integrating
project:
medical
consultation
system
integrating
and
intelligence
methodologies,
Stanford
and artificial
artificial
intelligence
methodologies,
Stanford
proposal, August,
proposal,
August, 1978.
1978.

18.
18. Kuhn,
Kuhn, Ingeborg,
Ingeborg, Gio
Gio Wiederhold,
Wiederhold, Jonathan
Jonathan E. Rodnick,
Rodnick, Diane
Diane M.
M. RamseyRamseyKlee,
Automated Ambulatory
Ambulatory Medical
Medical
Klee, Sanford
Sanford Brnctt,
Benet,t, and Donald
Donald D. Beck:
Beck: Automated
published by
by Springer-Verlag,
Record Sy.5tems
Springer-Verlag, 1983, in
Record
System8 in
in the
the U.S.,
U.S., to be published
Information
Inforrna.tion Systems
Systems for
for Patient
Patient Care,
Care, B. Blum
Blum (ed.), Section
Section III,
III, Chapter
Chapter 14.
19. Walker,
\Valker, M.G.,
Lisp Tutorial.
(Submitted
publication to
M.G., and Blum,
Blum, R.L.:
R.L.: AA Lisp
Tutorial.
(Submitted for
for publication
to
M.D.Computing.)
M.D.Computing.)
20. Wiederhold,
Knowledge and
Database Management,
A1anagement, IEEE
W’iederhold, Gio:
Gio: Knowledge
and Database
IEEE Software
Software
Premier
pp.63--73.
Premier IsslJe,
Issue, Ja11.1984,
Jan.1984, pp.63--73.
21. Wiederhold,Gio:
\Viederhold,Gio: Networking
Networking of
Data Information,
Information,
of Data
\Vorkshop
on
the
Role
of
Computers
Workshop
of Computers in Cancer
Cancer
Institu
tes
of
Health,
June
1983,
pp.113-11 9.
Institutes of Health,
1983, pp.L13-119.

National
National
Clinical
Clinical

Cancer
Cancer
Trials,
Trials,

Institute
Institute
National
National

22. \Viederhold,
Database Design
Design (in
Wiedrrhold, Gio:
Gio: Database
(in the Computer
Computer Science Series) McGrawMcGrawHill
NY, May
Hill Book
Book Company,
Company, New York,
York, NY,
May 1977, 678 pp. Second edition,
edition, Jan.
1983,
1983, 758
768 pp.

125
125

E. A.
A. Feigenbaum
Feigenbaum

RADIX Froject
Project
RADIX

5P41 RR00785-11
RR00785-11
5P41

23. Wiederhold,
Wiederhold, G.:
G.: IN
IN D.A.B.
D.A.B. Lindberg
Lindberg and
and P.L.
P.L. Reichertz
Reichertz (Eds.).
(Eds.), Databases
Databases lor
for
23.
Health
Cure,
Lecture
Notes
in
Medical
Informatics,
Springer-Verlag,
1981.
Health Care, Lecture Notes in Medical Informatics, Springer-Verlag, 1981.
Wiederhold, G.:
G.: Database
Database technology
technology in,
in health
health care.
care. J.
J. Medical
Medical Systems
Systems
24. Wiederhold,
5(3):175-Hlt), 1981.
5(3):175-196,

E. Funding
Funding Support
Support Statue
Status
E.
Representation and
and Use of
of Causal
Causal Knowledge
Knowledge for
for Inference
Inference from
from
1) Representation
Databases
DatabEes
Robert L.
L. Blum,
Blum, M.D.,
M.D., Ph.D.:
Ph.D.: Principal
Principal Investigator
Investigator
Robert
Total award:
award: $89,597
$89,597 (direct
(direct ++ indirect)
indirect)
Total
Term: hIarch
l\.'larch 15, 1984 through
through March
March 14, 1986
Term:
DeriVing Knowledge
Knowledge from
from Clinical
Clinical Databases
Databases
2) Deriving
Gio C.
C. M.
M. Wiederhold,
Wiederhold, Ph.D.:
Ph.D.: Principal
Principal Investigator
Investigator
Gio
National Library
Library of
of Medicine
Medicine
National
Total award:
award: $291,192
$291,192 (direct)
(direct)
Total
Term: May
May 1,
I, 1984 through
through November
November 30,
3D, 1986
Term:

II.
II.

INTERACTIONS WITH
WITH THE
THE SUMEXA.IM
SUMEX-AIM: RESOURCE
RESOURCE
INTERACTIONS
A.. Collaborations
-4.
Collaborations

During
During the
the last
last year
year we have completed
completed System
System Programmer's
Programmer’s Manuals
Manuals and
and a
User's
User’s Manual
hlsnual as
as steps
steps towards
towards making
making the
the system
system available
available to
to outside
outside collaborators.
collaborators. We
We
have had preliminary
preliminary discussions
Edward Shortliffe
Shortliffe and
and Lawrence
Lawrence Fagan
Fagan
discussions with
with Drs.
Drs. Edward
concerning
project. Once
Once the
the RADIX
RADIX
concerning use of
of components
components of
of RADIX
RADIX in the
the ONCOCIN
ONCOCIN project.
program
progrs.m is developed,
developed, we would
would anticipate
anticipate collaboration
collaboration with
with some of
of the
the ARAMIS
ARAMIS
project
project sites in the further
further development
development of
of a knowledge
knowledge base pertaining
pertaining to
to the chronic
chronic
arthritides.
arthritides. The
The ARAMIS
ARAlMIS Project
Project at
at the Stanford
Stanford Center
Center for
for Information
Information Technology
Technology is
used by
by a number
number of
of institutions
institutions around
around the
the country
country via
via commercial
commercial leased lines to store
store
and process their
their data.
data. These institutions
institutions include
include the University
University of
of California
California School
School of
of
Medicine,
hledicine, San Francisco
Francisco and Los Angeles;
Angeles; The
The Phoenix
Phoenix Arthritis
Arthritis Center,
Center, Phoenix;
Phoenix: The
The
University
University of Cincinnati
Cincinnati School of Medicine;
Medicine; The
The University
University of
of Pittsburgh
Pittsburgh School of
Medicine;
Kansas
University;
and
The
University
of
Saskatchewan.
All
All of
of the
The University
Saskatchewan.
hledicine; Kans,as University:
rheumatologists
at
these
sites
have
closely
collaborated
with
the
development
of
ARAMIS,
rheumatologists
closely collaborated with
development of ARAMIS,
and their
their interest
interest in and use
use of the RADIX
RADIX project
project is
is anticipated.
anticipated. We
We hasten
hasten to mention
mention
thaI,
we
do
not
expect
SUMEX
to
support
the
active
use
of
RADIX
as
an
on-going
that
not expect SUMEX
support
active use
RADIX as
on-going service
to this
this extensive
extensive network
network of arthritis
arthritis centers,
centers, but
but we would
would like to be able to allow
allow the
national
nationa. centers
centers to participate
participate in the development
development of the arthritis
arthritis knowledge
knowledge base
base and to
test that
that knowledge
knowledge base
base on their
their own clinical
clinical data
data banks.
banks.

B. Interactions
EX-AIM Projects
Interactions with
with Other
Other SUM
SUA4ELX-AIM
Projects
Several
Several of
of the conce!}ts
concepts incorporated
incorporated into
into the design
design of
of the RADIX
RADIX Project
Project have
been
by other
The RADIX
RADIX knowledge
knowledge base
base is
is similar
similar to
been inspired'
inspired .by
other SUMEX-AIM
SUMEX-AIM Projects.
Projects. The
the Units
The
The production
production rule inference
inference
Units Package of
of the MOLGEN
MOLGEN PROJECT.
PROJECT.
mechanism
mechanism used
used by us
us is
is similar
similar to that
that in
in the MYCIN
MYCIN Project.
Project.
Several
Several programs
programs developed
developed by
by the
the MYCIN
MYCIN group
group are
are regularly
regularly used
used by
by RADIX.
RADIX.
These
These include
include disk
disk hash
hash file
file facilities,
facilities, text
text editing
editing facilities,
facilities, and
and miscellaneous
miscellaneous LISP
LISP
functions.
functions.

E.
E. A.
,A. Feigenbaum
Feigenbaum

126
126

RADIX
RADIX Project
Project

5P41
5P4 1 RR00785-11
RR00785- 11

Regular
Regular communication
communication on programming
programming details
details isis facilitated
facilitated by the on-line
on-line mail
mail
system.

C. Critique
Critique of
of Resource
Resource Alanagement
Management
The
The DEC
DEC System
System 20 continues
continues to provide
provide accepta.ble
acceptable performance,
performance, but
but it
it is
is
frequently
frequently heavily
heavily loaded at peek hours.
The
The SUMEX
SUMEX resource management
management continues
continues to be accessible
accessible and cooperative.
cooperative.

ID.
III. RESEARCH
RESEARCH PLANS
PLANS
A. Project
Project Goals and
and Plans
The
The overall
overall goal of the RADIX
RADIX Project
Project is to develop a computerized
computerized medical
medical
information
system
capa.ble
of
accurately
extracting
medical
knowledge
information system cspa.ble
accurately extracting medical knowledge pertaining
pertaining to the
therapy
therapy and evolution
evolution of
of chronic
chronic diseases
diseases from
from a database consisting
consisting of
of a collection
collection of
of
stored
stored patient
patient records.
records.
SHORT-TERM
SHORT-TERM GOALS
GOALS -For
For the last two
two years we have concentrated
concentrated more heavily
heavily on publishing
publishing and
presen
tat ion of our
presentation
our earlier
earlier AI
AI results,
results, on acquisition
acquisition of a 1700 patient
patient database, on
medical
medical studies
studies based on the enlarged
enlarged database, and on reporting
reporting the medical
medical results
results and
statistical
statist.ica.1 techniques
techniques arising
arising from
from our
our research. This
This is
is in concert
concert with
with the long-term
long-term goal
of ensuring
ensuring that
that the work
work of
of the SUMEX
SUMEX / Artificial
Artificial Intelligence
Intelligence in Medicine
Medicine community
community
be disseminated
he
disseminated and applied
applied in the general
general medical
medical community.
community.
During
coming two
two years
years we will
will concentrate
concentrate much
much more
more on the artificial
artificial
During the coming
intelligence
intelligence aspects of
of RADIX.
RADIX. We
We were successful this
this year
year in obtaining
obtaining funding
funding from
from the
National Library
National Science Foundation
pursue this
National
Library of
of Medicine
Medicine and the National
Foundation to pursue
this work.
work. In
In
particular, we will
particular,
will be deeply
deeply concerned
concerned with
with the representation
representation of
of causal, temporal,
temporal, and
11antitative medical
qqusnt!it,st,ive
medical knowledge.
knowledge. It
It has become clear
clear that
that t,hese
these types
types of
of knowledge
knowledge are
crurial
crucia.1 for
for the RADIX
RADIX tasks of
of automated
automated discovery
discovery of
of medical
medical knowledge
knowledge and the
provision of
provision
of intelligent
intelligent, automated
aut.omsted assistance to
to clinical
clinical researchers, in addition
addition to their
t,heir
generally
perceived value
generally perceived
value in other
other medical
medical expert
expert systems
systems applications.
applications.
LONG-RANGE GOALS
GOALS -- There
There are two
two inter-related
inter-related long-range
long-range goals of
of t,he
the
LONG-RANGE
RADIX Project:
Project: 1) automatic
automatic discovery
discovery of
of knowledge
knowledge in a large time-oriented
time-oriented database
database
RAD1.X
and 2) provision
provision of
of assistance to
to a clinician
clinician who
who is interested
interested in testing
testing a specific
specific
a.nd
hypothe:ois. These t.asks
tasks overla,p
overla.p to
to the extent
extent that
that some of
of the algorithms
algorithms used for
for
hypothesis.
discovery are also used in the process of
of testing
testing an hypot,hesis.
hypothesis.
discovery
\Ve hope to
to make
make these
broad
range
of
hypotheses
broad
of hypotheses and
records.
records.

algorithms sufficiently
SUfficiently
algorithms
over
a
broad
spectrum
over broad spectrum

robust that
that they
they will
will work
work
robust
of
data
distributions
in
the
of data distributions
the

over a
over
patient
patient

B. Justification
Justification and
and Requirements
Requirements for
for Continued
Continued Use of
of SUMEX
SUMEX
B.
Computerized clinical
clinical data
data banks
banks possess
possess great
great potential
potential as tools
tools for
for assessing
assessing the
the
Computerized
effica.cy of
of new diagnostic
diagnostic and
and therapeutic
therapeutic modalities,
modalities, for
for monitoring
monitoring the
the quality
quality of
of health
health
efficacy
delivery, and for
for support
support of
of basic medical
medical research.
research. Because of
of this
this potential,
potential, many
many
care delivery.
clinical da.ta
data banks
banks have recent,ly
recently been developed
developed throughout
throughout the
the United
United Stat,es.
States. However,
However,
clinical
the init)ial
initial problems
problems of
of data
data acquisition,
acquisition, storage,
storage, and retrieval
retrieval have been dealt
dealt with,
with,
once the
there remains
remains a.
a. set of
of complex
complex problems
problems inherent
inherent in the
the task
task of
of accurately
accurately inferring
inferring
t.here

127

A. Feigenbaum
Feigenbaum
E. A.

RADIX
RADIX Project
F’roject

5P41
5P41 RR00785-11
RR00785- 11

medical
medics.1 knowledge
knowledge from
from aa collection
collection of
of observations
observations in
in patient
patient records.
records. These
These problems
problems
concern
concern the
the complexity
complexity of
of disease
disease and
and outcome
outcome definitions,
definitions, the
the complexity
complexity of
of time
time
relationships,
relationships, potential
potential biases
biases in
in compared
compared subsets,
subsets, and
and missing
missing and
and outlying
outlying data.
data. The
The
major
major problem
problem of
of medical
medical data
data banking
banking isis in
in the
the reliable
reliable inference
inference of
of medical
medical knowledge
knowledge
from
from primary
primary observational
observational data.
data.
\Ve see
see in
in the
the RADIX
RADIX Project
Project aa method
method of
of solution
solution to
to this
this problem
problem through
through the
the
utilization
of
knowledge
engineering
techniques
from
artificial
intelligence.
The
utilization of knowledge engineering techniques from artificial intelligence. The RADIX
RADIX
Project,
Project, in
in providing
providing this
this solution,
solution, will
will provide
provide an
an important
important conceptual
conceptual and
and technological
technological
link
link to aa large community
community of medical
medical research groups
groups involved
involved in the treatment
treatment and study
study
of the chronic
chronic arthritides
arthritides throughout
throughout the United
United States and Canada,
Canada, who
who are presently
presently
using the ARArvns
Data Bank
Ba.nk through
through the CIT
CIT facility
facility via
via TELENET.
TELENET.
ARAMIS Data
Beyond
Beyond the arthritis
arthritis centers which
which we have mentioned
mentioned in this
this report,
report, the TOD
TOD
(Time-Oriented
Data
(Time-Oriented
Data Base) User Group
Group involves
involves aa broad
broad range of university
university and
community
community medical
medical institutions
institutions involved
involved in the treatment
treatment of cancer, stroke,
stroke, cardiovascular
cardiovascular
disease,
disease, nephrologic
nephrologic disease,
disease, and others.
others. Through
Through the RADIX
RADIX Project,
Project, the opportunity
opportunity
will
will be
be provided
provided to foster
foster national
national collaborations
collaborations with
with these research groups
groups and to
provide
provide aa major
major arena in which
which to demonstrate
demonstrate the utility
utility of artificial
artificial intelligence
intelligence to
clinical
clinical medicine.
medicine.

C. Recommendation8
for Re80urce
Recommendations for
Resource Development
Development
The
The on-going
on-going acquisition
acquisition oi
oi personal
personal work-station
work-station Lisp
Lisp processors is a very
very positive
positive
step.
step, as
a.s these provide
provide an
a.n excellent
excellent environment
environment for
for program
program development,
development, and can serve
as
as a vehicle
vehicle for
for providing
providing programs
programs to
to collaborators
collaborators at other
other sites. Continued
Continued acquisitions
acquisitions
are very
very desirable.
desirable.
Another
Another resource that
that would
would be highly
highly desirable
desirable is a faster
faster and more
more reliable
reliable means
for
programs interactively
between SUMEX
for transferring
tra.nsferring data
data and programs
interactively between
SUMEX and the CIT
CIT IBM
IBM 370.
The
The addition
addition of
of a reliable
reliable local network
network facility
facility would
would greatly
greatly facilitate
facilitate our
our ability
ability to
to
transfer
patient files from
transfer patient
from CIT
CIT to
to SUMEX.
SUMEX.

E. A.
A. Feigenbaum
Feigenbaum
E.

128

National
National AIM
AIM Projects
Projects

5P41
51’41 RROOi85-11
RR00785-11

ll.A.2.
AIM
II.A.2. National
National
AIM Projects
Projects
The following
following group
group of projects
projects isis formally
formally approved
approved for access
access to the AIM
AIM aliquot
aliquot
of the SUMEX-AIM
Their access
access isis based
based on review
review by the AIM
AIM Advisory
Advisory
SUMKX-AIM resource. Their
Group
Group and approval
approval by the AIM
AIM Executive
Executive Committee.
Committee.
In addition
addition to the progress reports
reports presented
presented here, abstracts
abstracts
its individual
users
are
submitted
on
a
separate
Scientific
individual users
submitted
Scientific Subproject
Subproject

129

for each project
project and
Form.
Form.

A. Feigenbaum
Feigenbaum
E. A.

CADUCEUS
CADUCEUS Project
Project

5P41
5P41 RR00785-11
RR00785- 11

ll.A.2.1.
CADUCEUS
Project
II.A.2.1.
CADUCEUS
Project

CADUCEUS
CADUCEUS Project
Project
J. D.
D. Myers,
Myers, M.D.
M.D. and
and Harry
Harry E.
E. Pople,
Pople, Jr.,
Jr., Ph.D.
Ph.D.
University
University of
of Pittsburgh
Pittsburgh
Decision
Decision Systems
Systems Laboratory
Laboratory
Pittsburgh,
Pittsburgh, Pa.,
Pa., 15261
15261

I. Su.M:MARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
A.
A. Projcct
Project rationalc
rationale
The
The principal
principal objective
objective of this
this project
project is the development
development of a high-level
high-level computer
computer
diagnostic
diagnostic program
program in the broad
broad field
field of internal
internal medicine
medicine as
as an aid in the solution
solution of
complex
To be effective,
effective, the program
program must
must be
complex and complicated
complicated diagnostic
diagnostic problems.
problems. To
~apable
capa.ble of multiple
multiple diagnoses (related
(related or independent)
independent) in aa given patient.
patient.
A
A major
major achievement
achievement of this
this research undertaking
undertaking has been the design of
of a
program
along with
progra.m called INTERNIST-I,
INTERNIST-I,
with an extensive
extensive medical
medical knowledge
knowledge base.
base. This
This
program
program has been
been used
used over
over the past
past decade to analyze
analyze many
many hundreds
hundreds of
of difficult
difficult
diagnost.ic
diagnost,ic problems
problems in t.he
t,he field
field of internal
internal medicine.
medicine. These problem
problem cases
cases have included
included
cases
journals (particularly
Case Records
of the Massachusetts
Massachusetts
cases publisht"cl
published in medical
medical journals
(particularly
Records of
General
CPCs, and unusual
unusual problems
problems
General Hospital,
Hospital, in the New England
England Journal
Journal of Medicine),
Medicine), CPCs,
of
patients
in
our
Medical
Center.
In
most
instances,
but
by
no
means
all,
INTERNIST-I
of patient,s
our Medical Center. In most
but by
all, INTERNIST-I
has performed
high-lighted
performed at the level of the skilled
skilled internist,
internist, but
but the
the experience has high-lighted
several areas (or
for improvement.
improvement.

B. Medical
Afcdical Releoance
Relevancc and
and Collaboration
B.
Collaboration
The progra.m
program inherently
inherently has direct
direct and substantial
medical relevance.
relevance.
The
substantial medical
The institution
institution of
of collaborative
collaborative studies
with other
other institutions
institutions has been deferred
deferred
The
studies with
pending completion
completion of
of t,he
the programs
programs and knowledge
knowledge base enhancements
enhancements required
required for
for
pending
CADUCEUS. The
The insta.llation
installation of
of our
our own,
own, dedicated
dedicated VAX
VAX computer
computer can be expected
expected to
to
CADUCEUS.
considerably any
any future
future collaboration.
collaboration.
aid considerably
HI~ghlights of
of Research
Rcsearch Progress
Progrcss
C. Highlights

----Accomplishments this
this past
past year
year
----Accomplishments
In a previous
previous progress
progress report
report the
the concept
concept of
of “facets”
"facets" of
of diseases
diseases was
was introduced
introduced
In
and the
the need of
of CADUCEUS
CADUCEUS to
to proceed
proceed from
from broad
broad pathophysiopathophysio- logical
logical and
and pa.thopathoand
biochemical
concepts
t,o
specific
disease
processes
was
emphasized.
The
need
for
better
biochemical concepts t,o specific
emphasized. The
for better
represent,ation of
of a.natomical
anatomical information
information and
and for
for better
better time
time representation
representation were
were pointed
pointed
represent,ation
out.
out.
Drs. hliller
rvliller and
and Myers
Myers have
have continued
continued in
in the
the development
development of
of a new
new format
format for
for the
the
Drs.
CADUCEUS knowledge
knowledge base.
major goal
goal in
in making
making the
the transit,ion
transition from
from the
the
CADLiCEUS
AA major
INTEH0:IST-I knowledge
knowledge base to
to that
that of
of CADUCEUS
CADUCEUS ha.s
has been
been to
to insure
insure that
that there
there is
INTERNIST-I
continuity between
between the
the two:
two: The
The CADUCEUS
CADUCEUS knowledge
knowledge base
base will
will be
be derived
derived from
from the
the
cont.inuity
information in
in the
the INTERNIST-I
INTERNIST-I knowledge
knowledge base,
base, with
with significant
significant additions
additions made
made as
as
information
necessary.
necessary.
A. Feigenbaum
Feigenbaum
E. A.
E.

130

CADUCEUS
CADUCEUS Project
Project

5P41
5P41 RR00785-11
RRO0785-11

A screen-oriented
screen-oriented editor
editor program
program for entering
entering and manipulating
manipulating the knowledge
knowledge
base was written
writt.en in Franz
Franz Lisp. Using the editor,
editor, aa total
total of 52
52 diagnosis nodes have been
been
base
crea.ted and aa total
total of 282
282 findings
findings have been defined. Due to the more complex
complex nature
nature of
created
finding, the 282
282 findings
findings represent
represent over
over 600 old INTERNIST-I
INTERNIST-I style
style manifestations.
manifestations.
aa. finding.
In the CADUCEUS
CADUCEUS knowledge
knowledge base,
base, the basic unit
unit of observational
observational in- formation
formation is
is
Unlike an INTERNIST-I
INTERNIST-I
manifestation, aa finding
finding can be
be assigned aa
called aa. finding.
Unlike
manifestation,
finding.
status within
within aa given
given patient,
patient, either
either "normal"
“normal” or any number
number of forms
forms of
of abnormal.
abnormal. For
For
status
example,
example, the status
status of the finding
finding "heart
“heart murmur"
murmur” can be either
either absent (normal)
(normal) or
Various qualifiers
qualifiers are allowed
allowed to modify
modify aa finding.
finding.
For the finding
finding "heart
“heart
present.
For
present. Various
patient, aa user might
might specify
specify that
that it is
is heard at the second left
left
murmur",
murmur”, in aa specific patient,
interspace.
systole only,
only, that
that it
it is blowing,
blowing, that
that
interspace. that
that it
it is
is systolic,
systolic, that
that it
it is heard
heard in early
early systole
is crescendo- decrescendo. For
For findings
findings whose values vary
vary
it
it is
is grade
gra,de 2 of 6, and its shape is
numerically,
numerically, (e.g. SGOT-blood)
SGOT-blood) the units
units of measurement
measurement and the normal
normal range are
specified so
so that
that aa. user may
may simply
simply enter
enter aa number
number as
as the result
result of the test.
test,
specified
The
The concept
concept of a disease
disease profile
profile has been carried
carried over
over from
from the INTERNIST-I
INTERNIST-I
knowledge
knowledge base.
bsse. However,
However, there
there are three
three separate
separate diagnostic
diagnostic node types
types represented
represented in
the CADUCEUS
CADUCEUS knowledge
knowledge base:
base: the disease,
disease, the facet, and the subdivision.
subdivision. A
A disease
disease is
ent,ity whose presence should
should be reported
reported if
if detected
detected in a patient,
patient, and conceptually
conceptually
an entity
corresponds
correeponds to the diseases
diseases mentioned
mentioned in the separate chapters
chapters of standard
standard medical
medical
text.bnoks. A
A subdivision
subdivision is either
either a specific
specific subtype
subtype of
of a disease
disease (e.g. hepatitis
hepatitis B is a
textbooks.
subt.ype
subtype of acute viral
viral hepatitis,
hepatitis, a disease)
disease) or
or a major
major specific
specific organ
organ system
system involvement
involvement
by a multisystem
multisystem disease
disease (e.g. lupus
lupus nephritis
nephritis and lupus
lupus cerebritis
cerebritis are subdivisions
subdivisions of
of the
disease system
system lupus
lupus erythematosus).
erythematosus).
disease
The
The internal
internal organization
organization of
of disease,
disease, facet
facet and subdivision
subdivision profiles
profiles is identical.
identical.
Apart
Apart from
from links
links to other
other nodes, there
there are nine essential components
components to
to each profile:
profile:
disea.:5e
disea.se para.meters
para.meters (e.g. prevalence
prevalence of
of disease,
disease, specific
specific sites it
it effect,s);
effects); demographic
demographic
information about
about patients
patients with
with the disease;
disease; general
general predisposing
predisposing factors
factors (which
(which are
information
interciepencient.
interdependent, only
only one of
of which
which is likely
likely to
to be present);
present); independent
independent risk
risk factors
factors (which
(which
often
often co-exist
co-exist, synergist.ically);
synergistically); general
general findings
findings caused by
by the illness; specific
specific findings
findings which
which
relatively unique
unique to the disease
disease process; characteristic
characteristic findings
findings (e.g. a positive
positive throat
throat
are relatively
culture
culture for
for beta
beta hemolytic
hemolytic streptococcus
streptococcus in streptococcal
streptococcal pharyngitis);
pharyngitis); academically
aca.demically known
known
but
but clinically
clinically contraindicont,raindi- cated findings
findings (e.g. one should
should not
not do a renal
renal biopsy
biopsy in patients
patients
wit,h renal
renal leptospirosis,
lept,ospirosis, but
but we know
know what
what the biopsy
biopsy will
will show if
if it
it is done anyway);
anyway); and
with
manifestations whose presence make
make the
the diagnosis
diagnosis untenable
untenable (e.g. male sex makes
makes
manifest,ations
pregnancy
pregnancy an invalid
invalid consideration).
consideration). In
In addition
addition to
to the aforementioned
aforementioned work
work in internal
internal
medicine.
medicine. Drs.
Drs. Gordon
Gordon Banks
Banks and
a.nd John
John Vries
Vries have beeu
been working
working on the development
development of
of a
neurological diagnostic
diagnostic component
component for
for CADUCEUS.
CADUCEUS.
Dr. Banks
Banks has developed
developed a
neurological
Dr.
neurnanatomic
1,000 neuroanatomic
neuroanat.omic database
database which
which contains
contains spatial
spatial descriptors
descriptors for
for nearly
nearly 1,000
neuroanatomic
st,ructures
structures and contains
contains information
information as to
to their
their blood
blood supply,
supply, and function.
function. This
This database
database
will allow
allow a.na.tomic
anatomic loca.lization
localization of
of neurologic
neurologic lesions. Some of
of this
this work
work for
for the
the peripheral
peripheral
will
nervous
nervous system
system has been done previously
previously by
by students
students in our
our laboratory.
laboratory. The
The approach
approach to
to
central nervous
nervous syst.em
system hss
has been to
to design a set of
of “symbolic
"symbolic coordinates”.
coordinates". In
the cent.ral
const0uct
ing the neuroanat,omic
constructing
neuroanatomic database,
database, the human
human body,
body, including
including the
the nervous
nervous system,
system,
conceptually partitioned
partitioned into
into a set of
of cubes (boxes). The
The largest
largest cube, containing
containing the
the
is conceptually
ent,ire body.
body. is 2.187m
2.187m on a side. This
This cube is divided
divided into
into 27 smaller
smaller cubes, each 729mm
729mm
on a side. Each
Each of
of the
the smaller
smaller cubes is likewise
likewise subdivided
subdivided until
until finaliy
finally cubes that
that are
each lmm
1mm on a side are reached.
reached. Thus
Thus any
any cube has neighbors
neighbors (of
(of equal size) rostra],
rostral,
caudal,
caudal. vent,ral,
ventral. dorsal,
dorsal, left,
left, and right
right of
of it,
it, as well
well as a “pa,rent”
"parent" cube which
which contains
contains it,
it,
and “dsught,er”
"dallghter" cubes which
which itit contains.
contains. Each
Each of
of these cubes has the
the potential
potential for
for being
being
represented inside
inside the
the computer
computer program
program with
with a unique
unique name (known
(known as an atom
atom in LISP.
LISP.
represrnted
which the
the dat,abase
database is programmed).
programmed). Attached
Attached to
to each cube LISP
LISP atom
atom
the language in which

131

A. Feigenbaum
Feigenbaum
E. A.

5P41
5P4 1 RR00785-11
RR00785- 1 I

CADUCEUS
CADUCEUS Project
Project

a.re
are lists of all of the anatomic
anatomic structures
structures that
that are completely
completely and partially
partially contained
contained
within
within the cube, as
as well
well as
as the blood supply
supply to the region. This
This structure
structure facilitates
facilitates rapid
rapid
retrieval
retrieval of the location
location of aa given anatomic
anatomic structure
structure as
as well
well as
as rapid
rapid localization
localization of
possible areas
areas of involvement
involvement when there isis evidence of dysfunction
dysfunction of one or more neural
neural
systems.
The
The hierarchical
hierarchical arrangement
arrangement of
of the nested cubes ensures rapid
rapid convergence during
during
searches,
searches, because
because if the sought
sought object
object is
is not
not found
found in a parent
parent cube, there
there is no need to
search for
for it
it in any of the patient's
patient’s children
children cubes.
cubes. The
The addition
addition of anatomic
anatomic reasoning
reasoning
may
may allow
allow parsimonious
parsimonious explanation
explanation of multiple
multiple manifestations
manifestations arising
arising from
from a single
lesion.
lesion, or allow
3,110~the program
program to query
query the user regarding
regarding the presence of manifestations
manifestations of
involvement
involvement of areas that
that might
might be
be expected to be affected
affected by whatever
whatever clinical
clinical state
st.ate the
program
pr0gra.m has under
under current
current consideration.
consideration.
Dr. Vries
Vries has developed an imaging
imaging system
system using "octree
“octree encoding"
encoding” to reconstruct
reconstruct
n-dimensional
n-dimensional images of
of the database as
as well
well and images of
of patients
patients acquired
acquired by
by CT,
CT,
NMR,
NMR, and
a.nd other
other neuroimaging
neuroimaging techniques.
techniques. Combining
Combining the database with
with the imaging
ima.ging
system
correlation
system may
may open new areas
a,rea.s of
of research, including
including clinical-pathological
clinical-pathological
correlation of
of
imaged lesions with
with symptoms,
symptoms, signs, and affected
affected structures,
structures, automated
automated reading
reading of
of
images, etc.
Dr.
Dr. r-.mler
Miller in the last
last year
year completed
completed work
work on a sub-project
sub-project of
of CADUCEUS,
CADUCEUS, called
called
CPCS.
CPCS. He received
received support
support for
for this
this work
work from
from the National
National Library
Library of Medicine
Medicine New
Investigator
lnvest.igator Program.
Program. The
The original
original objective
objective of the project
project was to create a program,
program,
CPCS
CPCS (for
(for Computer-based
Comput,er-based Patient
Patient Case Simulator),
Simulator), to aid in the teaching
teaching of diagnosis to
medical
knowledge
medical students.
st,udents. The
The INTERNIST-I/CADUCEUS
INTERNIST-I/CADUCEUS
knowledge base
base was to be used as
as the
source of
program's
of the program’
s medical
medical expertise.
expertise. This
This overall
overall goal has been accomplished,
accomplished, and
the
program CPCS
AX-II/780 using Franz
t.he program
CPCS exists and runs
runs on our
our V
VAX-h/780
Franz Lisp.
Lisp. The
The CPCS
CPCS
project was a feasibility
project
feasibility study
study to demonstrate
demonstrate that
that it
it is possible to
to construct
construct a general
general
case
The
project has been successful in that
program has been
The project
that the CPCS
CPCS program
case simulator.
simulator.
written,
and
runs
quite
well
in
its
small
test
domain.
But
there
is
room
writt)en,
runs quite well
its small test domain. But there
room for
for the
the future
future
development
development. of
of CPCS.
CPCS. Further
Further construction
construction of
of the CADUCEUS
CADUCEUS knowledge
knowledge base, in areas
beyond the current
beyond
current set of
of liver
liver diseases,
diseases, will
will significantly
significantly improve
improve the utility
utility of
of the CPCS
CPCS
program. As additional
a.ddit.ional capabilities
capabilities are added to CADUCEUS,
CADUCEUS, the corresponding
corresponding changes
program.
will
will be made in CPCS.
CPCS.
The
medical knowledge
continued to
grow both
both in the incorporation
of
The medical
knowledge base has continued
to grow
incorporation of
modification of
of diseases
diseases already
already profiled
profiled so as to
to include
include recent
recent
diseases and the modification
new diseases
medical knowledge.
knowledge. The
The knowledge
knowledge base of
of 3/l/84
3/1/84 includes
includes 591
591 individual
individual
advances in medical
disease profiles,
profiles, 4,040 manifestations
manifestations of
of disease,
disease, and
and about
about 3,500 ”"links"
or
disease
links” or
interrelationships among
among diseases
diseases as well
well as a myriad
myriad of
of miscellaneous
miscellaneous pieces of
of
int,errelnt,ionships
informat.ion which
which are essential
essential for
for the
the correct
correct operation
operation of
of the
the system.
system. Twenty
Twenty new
information
profiled during
during the
the past
past year
year and the
the pediatrics
pediatrics knowledge
knowledge base
base has
diseases have been profiled
diseases
continued to
to grow.
grow.
continued
Recently the
the medical
medical knowledge
knowledge base (but
(but not
not yet
yet the
the diagnost.ic
diagnostic program)
program) has been
Recently
made available
ava.ilable on line for
for use of
of the medical
medical house staff
staff at
at Presbyterian-University
Presbyterian-University
ma.de
Hospit.al, our
our main
ma.in t,eaching
t.eaching hospital
hospital in internal
internal medicine,
medicine, and
and at
at an affiliated
affiliated community
community
Hospital,
hospit.al in Pit.tsburgh,
Pit.tsburgh, the
t.he Shadyside
Shadyside Hospital
Hospital which
which operates
operates a residency
residency program
program in
hospit,al
internal medicine.
medicine. Preliminary
Preliminary reports
reports indicate
indicate that
that the
the residents
residents find
find the
the knowledge
knowledge base
internal
useful.
useful.
----Research in progress
progress
----Research
There are five
five major
major components
components to
to the
the continuation
continuation of
of this
this research
research project:
project:
There

A. Feigenbaum
Feigenbaum
E. A.

132

CADUCEUS Project
Project
CADUCEUS

5P41
5Pdl RR00785-11
RR00785-11

1.
refinement and testing
testing of
of the
the extensive
extensive
1. The
The enlargement,
enlargement, continued
continued updating,
updating, refinement
medical
medical knowledge
knowledge base
base required
required for
for the operation
operation of
of INTERNIST-I.
INTERNIST-I.
2.
?. The
The completion
completion and implementation
implementation of
of the improved
improved diagnostic
diagnostic consulting
consulting
program,
which
CADUCEUS,
which has been designed to
to overcome
overcome certain
certain
program, CADUCEUS,
performance problems
performance
problems identified
identified during
during the
the past
past years
years of
of experience with
with the
the
origina.l
original INTERNIST-I
INTERNIST-I program.
program.
3. Institution
Institution of
of field
field trials
trials of
of CADUCEUS
CADUCEUS on the clinical
clinical services in internal
internal
medicine
medicine at the Health
Health Center
Center of
of the
the University
University of
of Pittsburgh.
Pittsburgh.
4. Expansion
Expansion of
of the clinical
clinical field
field trials
trials to other
other university
university health
health centers
centers which
which
ha.ve
have expressed interest
interest in working
working wit,h
with the system.
system.
5. Ad:tptation
program and data
Adaptation of
of the diagnostic
diagnostic program
data base
base of
of CADUCEUS
CADUCEUS to
SI\
bserve educational
pl1rposes and the evaluation
of clinical
clinical performance
performance and
subserve
educational purposes
evaluation of
competence.
compet,ence.
Current
Current, activity
activity is devoted
devoted mainly
mainly to
to
development
of
the
medical
knowledge
development of
medical knowledge
diagnostic
program.
diagnostic consulting
consulting program.

the first
first
base,
base, and

two
two
the

of
of these, namely,
namely,
implementation
implementation of
of

the
the

continued
continued
improved
improved

D. List
List of
publications
D.
of relevant
relevant publicationa
Harry
E.:Knowledge-based Expert
Expert Systems:
The Buy
Buy or
Build
Harry E.:Knowfedge-based
Systems:
or Build
Derision IN
Reitman
INTELLIGENCE
Derision
IN Walter
Walter
Reitman (Ed.),
(Ed.), ARTIFICIAL
ARTIFICIAL
INTELLIGENCE
APPLICATIONS
FOR
Proceeding
NYU Symposium.
APPLICATIONS
FOR BUSINESS.
BUSINESS.
Proceeding of
of the NYU
Symposium.
Ablex
Ablex Pub.
Pub. Corp.,
Corp., May
May 1983,
1983, pp. 23-40.

1.
I. Pople,

2.
Artificial Intelligence
Intelligence and
Medical Education
Education The
3. ~1yers,
Myers. J.D.:
J.D.: Artificial
and MedicaI
The Medical
Medical
Journal,
Journal, St. Joseph Hospital,
Hospital, Houston.
Houston. Vol.
Vol. 18, December
December 1983, pp. 193-202.

E. Fwnding
Funding support
support
E.

1. Clinical
Decision
Clinical
Decision Systems Research
Research Resource
Resource
Harry
Harry E. Pople,
Pople, Jr.,
Jr., Ph.D.
Associate
Associate Professor
Professor of
of Business
Business
Jack D. Myers,
Myers, M.D.
University
Professor
University
Professor (MediCine)
(Medicine)
University
Uni versi ty of
of Pittsburgh
Pittsburgh
Division
Division of
of Research
Research Resources
Resources
National Institutes
National
Institutes of
of Health
Health
5 R24
R24 RROllOI-07
RROllOl-07
07/01/80 - 06/30/85 - $1,607,717
$369,484
07/01/83 - 06/30/84 - $369,484

CADUCEUS:
CADUCEUS: A
A Computer-Based
Computer-Based Diagnostic
Diagnostic Consultant
Consultant
Harry
Harry E. Pople,
Pople, Jr.,
Jr., Ph.D.
Associate
Associate Professor
Professor of
of Business
Business
Jack D. Myers,
Myers, M.D.
H.D.
University
Professor
University
Professor (Medicine)
(Medicine)
University
of
University
of Pittsburgh
Pittsburgh
National Library
National
Library of
of Medicine
Medicine
National Institutes
of
National
Institutes
of Health
Health

133

E. A. Feigenbaum
Feigenbaum

5P41 RR00785-11
RR00785- 11

CADUCEUS
CADUCEUS Project
Project

R01 LM03710-04
5 ROI
o;r/ol/ao
06/30/85
$817,884
07/01/80
-- 06/30/85
-- $817,884
07/01/83
C6/30/84 -- $196,710
$196,710
07/01/83
-- 06/30/84

Neurologic Consultation
Computer Program
Neurologic
Consultation
Gordon E. Banks,
Banks, M.D.
Assistant
Assistant Professor
Professor of
of Medicine
Medicine
National
Library of
of Medicine
Medicine - New
New Investigator
Investigator
National Library
National
Institutes
of Health
Health
of
National Institutes
5 R23 LM03889-02
Lb+03889-02
04/01/82 - 03/31/85
03/31/85 - $107,675
$107,675
04/01/82
04/oi/a3
03/31/84
$35,975
04/01/83
-- 03/31/84
-- $35,975
04/oi/a4
03/31/85
$35,975
04/01/84
-- 03/31/85
-- $35,975

ll.
WITH
II. INTERACTIONS
INTERACTIONS
WITH THE
THE SUMEX-AIM
SUMElX-AIM RESOURCE
RESOURCE
A,B.
Collaborations and
and Program
Ka SUMEX
SUMEX
A,B. Medical
Afedical Collaborations
Program Dissemination
Dissemination Via
CADUCEUS
CADUCEUS remains
remains in a stage of
of research and development.
development. As noted
noted above, we
are continuing
cont,inuing to
to develop
develop better
better computer
computer programs
programs to operate
operate the diagnostic
diagnostic system,
system, and
a.nd
t.he
t,he knowledge
knowledge base
base cannot
cannot be used very
very effectively
effectively for
for collaborative
collaborative purposes
purposes until
until it
it has
critical stage of
of completion.
completion.
factors have stifled
stifled collaboration
collaboration via
via
reached a critical
These factors
SUI\!EX
SUMEX up to this
this point
point and will
will continue
continue to
to do so for
for the next
next year
year or
or two.
two. In
In the
the
meanwhile, through
through the SUMEX
SUMEX community
community there
there continues
continues to
to be an exchange of
of
meanwhile.
information and states
states of
of progress. Such interactions
interactions particularly
particularly take
take place at
at the
information
annual AIM
AIhI Workshop.
Workshop.
annual

Critique of
of Resource
C. Critique
Resource Management
Jvfanagement

sur-.mx
Our
SUhlEX has been an excellent
excellent resource for
for the development
development of
of CADUCEUS.
CRDUCEUS.
Our
large program
program is handled
handled efficiently,
efficiently, effectively
effectively and accurately.
accurately. The
The staff
staff at SUMEX
SUMEX have
uniformly supportive,
supportive, cooperative,
cooperative, and innovative
innovative in connection
connection with
with our
our project's
projectA’s
been uniformly
needs.
needs.
III. RESEARCH
RESEARCH PLANS
PLANS
ill.
A.
Goals and
and Plans
A. Project
Project Goals
Plan8
Continued
Continued effort
effort to complete
complete the medical
medical knowledge
knowledge bases
bases in internal
internal medicine
medicine and
pediatrics
pedintrics will
will be pursued
pursued including
including the incorporation
incorporation of
of newly
newly described
described diseases
diseases and new
or
or altt'n'd
alt,ered medical
medical information
information on "old"
“old” diseases.
diseases. The
The latter
latter two
two activities
activities have proven
proven to
to
Profiles of
of added diseases
diseases plus other
other
be more formidable
formidable than
t,ha.n originally
originally conceived.
conceived. Profiles
informat,ion
information is first
first incorporated
incorporated into
into the medical
medical knowledge
knowledge base
base at SUMEX
SUMEX before
before being
being
transferred
into
This
tratlferred
into our
our newer
newer information
information structures
structures for
for CADUCEUS
CADUCEUS on the
t,he VAX.
VAX.
This
sequence retains
retains the operative
operative capability
capability of
of INTERNIST-I
INTERNIST-I as
as a computerized
computerized "textbook
“textbook of
of
sequence
medicine"
medicine” for educational
educstionsl purposes.

Justification and
Requirements for
for Continued
B. Justification
and Requirements
Continued SUMEX
SUME. Use
Our
A..X.
Our use
use of
of SUMEX
SUMEX will
will obviously
obviously decline
decline with
with the installation
installation of
of our
our V
VAX
Nevertheless. the excellent
Nevertheless,
excellent facilities
facilities of
of SUMEX
SUMEX are expected to
to be used for
for certain
certain
developmental
at SUMEX
development,al work.
work. It
It is intended
intended for
for the present
present to keep INTERNIST-I
INTERNIST-I
SUMEX for
for
Our team
team hopes to remain
remain as
as a
compara.tive
compsra,tive use
use as
as CADUCEUS
CADUCEUS is developed here. Our
component, of
of the SUMEX
SUhlEX community
community and to
to share experiences and developments.
developments.
component.
E. A. Feigenbaum
Feigenbaum

134

CADUCEUS
CADUCEUS Project
Project

5P41 RR00785-11
RR00785-11

C. Need.~
Need.~ and
Plans for
for Other
Re80urces Beyond
Beyond SUAfEX-ALM
and Plans
Other Computing
Computing Resources
SUMEX-AIM
Our
predictable needs in this
by the dedicated
~'C computer
Our predictable
this area will
will be met
met by
dedicated V
VAX
computer
recently
recently installed.
inst,alled.

D. Recommendations
Recommendations for
for Future
Future Community
Resource Development
Development
D.
Community and
and Resource
\\llether a program
program like
better operated
Whether
like CADUCEUS,
CADUCEUS, when
when mature,
mature, will
will be better
operated from
from
centralized,
personal computers
centra.lized, larger
larger computers
computers or
or from
from the developing
developing self contained
contained personal
computers is
difficult
predict. For
difficult to
t,o predict.
For the foreseeable future
future it
it would
would seem that
that centralized,
centralized, advanced
advanced
fa,cilit·ies
like
SUtvlEX
will
be
important
in
further
program
development
fscilit,ies like SIJMEX will
important
further prograrn development and refinement.
refinement.

135
135

E. A.
A. Feigenbaum
Feigenbaum

CLIPR
CLIPR -- Hierarchical
Hierarchical Models
Models of
of Human
Human Cognition
Cognition

5P41
5P41 RR00785-11
RR00785- I I

II.A.2.2.
CLIPR
Models
II.A.2.2.
CLIPR -- Hierarchical
Hierarchical
Models of
of Human
Human Cognition
Cognition

Hierarchical
Hierarchical Models
Models or
of Human
Human Cognition
Cognition (CLIPR
(CIJPR Project)
Project)
Walter
Kintsch
and
Walter
Kintsch
and Peter
Peter G.
G. Polson
Polson
University
University or
of Colorado
Colorado
Boulder,
Colorado
Boulder,
Colorado

I.
OF
PROGRAM
I. SUM:MARY
SUMMARY
OF RESEARCH
RESEARCH
PROGRAM
A. Project
Project Rationale
Rationale
The
The two
two CLIPR
CLIPR projects
projects have made progress during
during the last year.
year. The
The prose
comprehension
project
has
completed
one
major
project,
and
is
designing
comprehension project has completed
major project,
designing aa prose
comprehension
comprehension model that
that reflects
reflects state-of-the-art
state-of-the-art knowledge
knowledge from
from psychology
psychology (van
(van Dijk
Dijk
&.
& Kintsch,
Kintsch, 1983) and artificial
artificial intelligence.
intelligence. During
During the last year, Polson,
Polson, in collaboration
collaboration
with
with Dr.
Dr. David
David Kieras
Kieras of the University
University of Arizona
Arizona have h'ave
have continued
continued work
work on a project
project
studying
st,udying the psychological
psychological factors
factors underlying
underlying device complexity
complexity and the difficulties
difficulties that
that
nontechnically
nontechnically trained
trained individuals
individuals have in learning
learning to use
use devices like
like word
word processors.
They
They have develop formal
formal representations
representations of
of aa user's
user’s knowledge
knowledge of
of how
how of
of operate
operate a
device and of
of the user-device interface
interface (Kieras
(Kieras &
& Polson,
Poison, in Press) and have completed
completed
several experiment.s
experiment.s evaluating
evaluating their
their theory.
theory.

B. Tech.nical
Goals
Tech.nical Goals
The
first, the
the text
text comprehension
comprehension
The CLIPR
CLIPR project
project consists of
of two
two subprojects.
subprojects. The
The first,
project,
is
headed
by
\Valter
Kintsch
and
is
a
continuation
of
work
on
understanding
of
project,
by Walter Kintsch
continuation of work
understanding of
connected
laboratory for
for several
years.
connected discourse that
that has been underway
underway in Kintsch's
Kintsch’s laboratory
several years.
The second, the device complexity
complexity project
project is headed by
by Peter
Peter Polson
Polson in collaboration
collaboration with
with
The
Dayid Kierss
Kieras of
of the University
University of
of Arizona,
Arizona, Tucson.
Tucson. They
They are studying
studying the
the learning
learning and
and
David
problem solving
processes involved
involved in the utilization
utilization of
of devices like
like word
word processors or
or
problem
solving processes
complex computer
computer controlled
controlled medical
medical instruments
instruments (Kieras
(Kieras &
&. Poison,
Polson, in Press)
Press)
complex
The goal of
of the prose comprehension
comprehension project,
project is to
to develop
develop a computer
computer system
system
The
capable of
of the meaningful
meaningful processing
processing of
of prose. This
This work
work has been generally
generally guided
guided by
by the
the
ca,pa,ble
comprehension model
model discussed by
by Kintsch
Kintsch and van
van Dijk
Dijk (1978)
(1978), although
although our
our
prose comprehension
programming efforts
efforts have identified
identified necessary clarifications
clarifications and
and modifications
modifications in that
that
programming
model
(Miller
&.
Kintsch,
1980,
1981;
Kintsch
&
Miller,
1981;
Miller,
1982).
In
general,
model (Miller S: Kintsch,
Kintsch
Miller,
Miller, 1983). In general,
this research
research has emphasized
emphasized the
the importance
importance of
of knowledge
knowledge and
and knowledge-based
knowledge-based processes
this
in comprehension,
compn'hension, and
and we are accordingly
accordingly working
working with
with the
the AGE
AGE and
and UNITS
UNITS groups
groups at
at
SUMEX
toward
the
development
of
a
knowledge-based,
blackboard
model
of
prose
SU~IEX toward the development of
knowledge-based, blackboard model of
comprehension. \\‘
\Ve
to be able to
to merge the
the substantial
substantial artificial
artificial intelligence
intelligence research
research
comprehension.
e hope to
systems with
with psychological
psychological interpretations
interpretations of
of prose comprehension,
comprehension, resulting
resulting in
in a
on these systems
computational model
model that
that is also psychologically
psychologically respectable.
respectable.
computational
The goal
goal of
of the
the device
device complexity
complexity project
project is to
to develop
develop explicit
explicit models
models of
of the
the useruserThe
device interactsion.
interaction. They
They model
model the
the device
device as a nested
nested automata
automata and
and the
the user
user as a
device
production system.
system. These
These models
models make
make explicit
explicit kinds
kinds of
of knowledge
knowledge that
that are
are required
required to
to
production
operate different
different kinds
kinds of
of devices
devices and
and the
the processing
processing loads
loads imposed
imposed by
by different
different
operate
implementations of
of a device.
device. We
\Ve feel that
that tools
tools being
being developed
developed at
at SUMEX--in
SUMEX--in particular
particular
implementations
AGE and
and the
the UNIT
UNIT package--will
package--will dramatically
dramatically facilitate
facilitate our
our abilities
abilities to
to generate
generate such
such
AGE
models
of
the
user-device
interface.
models of the user-device interface.
E. A.
A. Feigenbaum
Feigenbaum
E.

136

SP41
5P41 RR00785-11
RROO785-11

CLIPR
CLIPR - Hierarchical
Hierarchical Models
Models of
of Human
Human Cognition
Cognition

C. Medical
Afedical Relevance
Relevance and
and Collaboration
Collaboration
The
The text
text comprehension
comprehension project
project impacts
impacts indirectly
indirectly on medicine,
medicine, as
as the medical
medical
profession is no stranger
problems of
profession
stranger to
to the problems
of the information
information glut.
glut. By
By adding
adding to
to the
research on how
how computer
computer systems
systems might
might understand
understand and summarize
summarize texts,
texts, and
determining
deterrnining ways
ways by
by which
which the readability
readability of
of texts
texts can be improved,
improved, medicine
medicine can only
only be
helped by research on how people understand
understand prose. Development
Development of
of a more
more thorough
thorough
understanding
processes responsible
underst,anding of
of the various
various processes
responsible for
for different
different types
types of
of learning
learning problems
problems
in children
children and the
the corresponding
corresponding development
development of
of a successful remediation
remediation strategy
strategy would
would
also be facilitated
facilitat,ed by
by an explicit
explicit theory
theory of
of the normal
normal comprehension
comprehension process.
Note that
particularly relevant
Not,e
that, our
our goal of
of a blackboard
blackboard model
model is particularly
relevant to
to the
understanding
understa.nding of
of learning
learning difficulties.
difficulties. One important
important aspect of
of a blackboard
blackboard model
model is the
separation
sepa.ra.tion of
of cognitive
cognitive processes
processes into
into a set of
of interacting
interacting subprocesses. Once such
subprocesses have been identified
identified and constructed,
constructed, it
it would
would be instructive
instructive to
to observe the
model's
performance when
processes are facilitated
model’s performance
when certain
certain of
of these processes
facilitated or
or inhibited.
inhibited. Many
Many
researchers have shown
shown that
that there
there are a variety
variety of
of cognitive
cognitive deficits
deficits (insufficient
(insufficient shortshortterm
poor long-term
can lead to
that
to
term memory
memory capacity,
capacit,y, poor
long-term memory
memory retrieval,
retrieval, and such) that
reading
problems. Having
power of
resding problems.
Having a blackboard
blackboard model
model in which
which the power
of individual
individual
components
components could
could be ma.nipulated
ma,nipulated would
would be a significant
significant step in determining
determining the
the nature
nature of
of
such ren.ding
problems.
reading problems.
The
project has two
The device complexity
complexity project
two primary
primary goals: the development
development of
of a
cognitive
cognitive theory
theory of
of user-device
user-device interaction
interaction in including
including learning
learning and performance
performance models,
and the
t,he development
development of
of a theoretically
theoretically driven
driven design process that
that will
will optimize
optimize the
relationships
relationships between
between device functionality
functionality and ease
ease of
of learning
learning and other
other performance
performance
factors
factors (Polson
(Polson & Kieras,
Kieras, 1983). The
The results
results of
of this
this project
project should
should be directly
directly relevant
relevant to
to
the
t.he design of
of complex,
complex, computer
computer controlled
controlled medical
medical equipment.
equipment. We
We are currently
currently using
using
word
word processors to
t,o study
study user-device
user-device interactions,
interactions, but
but principles
principles underlying
underlying use
use of
of such
devices should
should generalize
generalize to
to medical
medical equipment.
equipment.
Both
Both the text
text comprehension
comprehension project
project and the device complexity
complexity project
project involve
involve the
t#he
development
processes; cognitive
development of
of explicit
explicit models
models of
of complex
complex cognitive
cognitive processes:
cognitive modelling
modelling is a
stated
NIMH.
stated goal of
of both
both SUMEX
SUMEX and research supported
supported by
by NIMH.
The
The on-going
on-going development
development of
of the prose comprehension
comprehension model
model would
would not
not be possible
without
without, our
our colla.boration
colla.borat*ion with
with the AGE
AGE and UNITS
UNITS research groups.
groups. We
We look
look forward
forward to
a continued
Several
continued collaboration,
colla.boratlion, with,
with, we hope, mutually
mutually beneficial
beneficial ,results.
results.
Several other
other
psyC'hologists h3\'e
psychologists
ha,ve either
eit,her used or
or shown
shown an interest
interest in using an early
early version
version of
of the prose
romprehension
comprehension model.
model, including
including Alan
Alan Lesgold
Lesgold of
of SUMEX's
SUMEX’s SCP project,
project, who
who is exporting
exporting
the system
system to the LRDC
LRDC Vax.
Vax. \Ve
We have also worked
worked with
with James Greeno
Green0 -- another
another
member
project -- on a project
project that
rnember of
of the SCP
SCP project
that will
will integrate
integrate this
this model
model with
with models of
of
problem solving
problem
solving developed
developed by
by Greeno
Green0 and others
others at the University
University of
of Pittsburgh.
Pittsburgh.
:\'eedless
Needless to sa.y,
say, all of
of this
this interaction
interaction has been greatly
greatly facilitated
facilitated by
by the local and
network-wide
network-wide communication
communication systems supported
supported by
by SUMEX.
SUMEX. There
There has
ha.s been considerable
considerable
commnnication
communica.t.ion bet.ween
bet,ween members
members of
of the prose comprehension
comprehension and AGE/UNITS
AGE/UNITS groups
groups as
as
program bugs have been discovered
program
discovered and corrected;
corrected; the presence of
of a mail
mail system
system has made
this
this process infinitely
infinitely easier than
than if
if telephone
telephone or
or surface
surface mail
mail messages
messages were required.
required. The
The
mail
professional contacts
mail system,
system, of
of course, has also enabled us to
to maintain
maintain professional
contacts established
established at
conferences
conferences and other
other meetings,
meetings, and to
to share and discuss ideas with
with these contacts.
contacts.

137
137

E. A.
A. Feigenbaum
Feigenbaum

CLIPR
CLIPR - Hierarchical
Hierarchical Models of Human
Human Cognition
Cognition

5P41 RR00785-11

D.
D. Progress
Progress Summary
Summary
The prose comprehension
comprehension project
project has compieted
compieted an initial
initial version
version of aa model of
prose comprehension
1980).
comprehension (Miller
(Miller 8l
& Kintsch,
Kin&h,
1980). This
This model has been applied
applied to aa large
number
of
texts,
and
has
yielded
quite
reasonable
predictions
of
reca.ll
number
yielded quite
predictions
recall and readability.
readability.
Psychologists
from
other
universities
have
used
this
system
to
derive
Psychologists from other universities
this system
derive reading
reading time
time and
recall
recall predictions
predictions for
for their
their own experimental
experimental materials.
materials. We
We are currently
currently using the AGE
AGE
and UNITS
UNITS packages to extend
extend this
this model
model toward
toward one that
that can make use
use of world
world
knowledge
knowledge in its analyses; this
this model
model is discussed in Miller
Miller and Kintsch
Kintsch (1981) and Miller
Miller
(1982). It
It is further
further developed in van Dijk
Dijk and Kintsch
Kintsch (1983) has been applied
applied to the
domain
domain of word
word arithmetic
arithmetic problems
problems in our
our most
most recent
recent work
work (Kintsch
(Kintsch and Greeno, in
Press).
The
The device complexity
complexity project
project is in it's
it’s third
third year.
year. "We
We have developed an explicit
explicit
model for
for the knowledge
knowledge structures
structures involved
involved in the user-device
user-device interaction,
interaction, and we are
developing
Our
Our preliminary
preliminary theoretical
theoretical results
results are described
described in
developing simulation
simulation programs.
programs.
Kieras
Kieras &,
& Polson
Polson (in
(in Press). We
We have also completed
completed several experiments
experiments evaluating
evaluating the
theory.
theory.

E. List
List of
of Relevant
Relevant Publications
Publication8
1.
1.

Kieras,
Kieras, D.E.
D.E. and Polson, P.G.:
P.G.: An
An outline
outline of
of a theory
theory of
of the
the user
u8er complexity
complezity
of
of devices and sy.'Jtems.
sy.stem.9. Working
Working Paper
Paper No.1,
No. 1, Device Complexity
Complexity Project,
Project,
Universities
Iiniversities of
of Arizona
Arizona and Colorado,
Colorado, May,
May, 1982.

2. Kieras,
formal analysis
Kieras, D.E.
D.E. and Polson,
Poison, P.G.:
P.G.: The formal
analyclis of
of user
u8er complexity.
complexity. Int.
ht.
J. Man-f\.lachine
Studies,
In
Press.
Man-Machine Studies, In
3. Kintsch,
tiint,sch, "V.
W. and
a.nd van
van Dijk,
Dijk, T.A.:
T.A.: Toward
Toward a model
model of
of text
tezt comprehension
comprehension and
and

production. Psychological
production.
Psychological Rev. 85:363-394, 1978.
4. Kintsch,
J.G.:Understanding and
solving word
word arithmetic
arithmetic
Kintsch, W.
W. and Greeno,
Greeno, J.G.:Underatanding
and solving
problems. Psychological
problems.
Psychological Review,
Review, In
In Press.
S.
Readability and
short prose
prose passages:
passages:
and recall
recall of
of short
5. l'vliller,
Miller, J.R.
J.R. and Kintsch,
Kintsch, W.:
W.: Readability
.4.
Human
-4 theoretical
theoretical analysis.
analysis. J. Experimental
Experimental Psychology:
Psychology:
Human Learning
Learning and
f\.lemory
hiemory 5:33.5-354,
61335-354, 1980.
Miller, J.R.
J.R. and
and Kint,sch,
Kintsch, W.:
"V.: Readability
Readability and
and recall
recall of
of short
short prose
prose passages.
passages.
6. h?iller,
Text 1:21S-232,
Text
1:215-232. 1981.
J.R.:
A Knowledge-based
Knowledge-based Model
Model of
of Prose
Prose Comprehension:
Comprehension:
Miller, J.R.:
7. hiiller,
A
Applications toto Expository
Expository Text.
Text. IN
IN B.K.
B.K. Britton
Britton and J.B.
J.B. Black
Black (Eds.),
(Eds.),
Appficationu
UNDERSTANDING
EXPOSITORY
TEXT.
Erlbaum,
Hillsdale,
NJ,
1982.
UNDERSTANDING
EXPOSITORY TEXT. Erlbaum, Hillsdale, NJ,
8. Polson,
Polson, P.G.
P.G. and Kieras,
Kieras, D.E.:
D.E.: Theoretical
Theoretical foundations
foundations of
of a design
design process
process
for
the
minimization
of
user
complexity.
\Vorking
Paper
No.3,
guide
guide for the minimization
of user complezity. \Vorking Paper No.
3,
Project on
ou User
User Complexity,
Complexity, Universities
Universities of
of Arizona
Arizona and
and Colorado,
Colorado, June, 1983.
Project

Polson, P.G.
P.G. and
and Kieras,
Kieras, D.E.:
D.E.: A
A formal
formal description
description of
of users’
users' knowledge
knowledge ofof
9. Poison,
how to
to operate
operate a device
device and
and user
user complexity.
complexity. Behavior
Behavior Research Methods
Methods
how
Instrumentation.
and Instrument,ation.
van Dijk,
Dijk, T.A.
T.A. and
and Kintsch,
Kintsch, W.
\v.:STRATEGIES
10. van
: STRATEG IES
COiHPREHENSION. Academic
Academic Press, New
New York,
York, 1983.
198:3.
CO~\IPREIIENSIO.V

A. Feigenbaum
Feigenbaum
E. A.

138

0OF
F

DISCOURSE
DISCOURSE

5P41 RR00785-11
RR007851 1

CLIPR
CLIPR - Hierarchical
Hierarchical Models
Models of
of Human
Human Cognition
Cognition

Funding Support
F. Funding
Support Status
Status
1. Text
Text Comprehension
Comprehension and Memory
Memory

\Valter
Walter Kintsch,
Kintsch, Professor,
Professor, University
University of
of Colorado
Colorado
National
Institute
of
Mental
Health
5
Rol
Nat,ional Institute of Mental Health
Rol MHI5872-14-16
MH15872-14-16
7/1/81
7/l/81 - 6/30/84:
6/30/84: $281,085
7/1/83
7/l/83 - 6/30/84:
6/30/84: $69,878
$69,878
2. Understand
problems
Understand and solving
solving word
word arithmetic
arithmetic problems
\Valter
Walter Kintsch,
Kintsch, Professor,
Professor, University
University of
of Colorado
Colorado
National Science Foundation
National
Foundation
8/1/83
8/l/83 - 7/31/86:
7/31/86: $200,000
$200,000
3. User Complexity
Complexity of
of Devices and Systems
David
David Kieras,
Kieras, Associate
Associate Professor,
Professor, University
University of
of Arizona
Arizona
Peter
Pet.er G. Polson,
Polson, Professor,
Professor, University
University of
of Colorado
Colorado
International
International Business Machines
Machines Corporation
Corporation
1/1/82
]
2/31/84:
$364,000
l/1/82
12/31/84: $364,000
1/1/84
l/l,‘84 - 12/31/84:
12/31/84: $145,000

IT.
WITH
RESOURCE
II. INTERACTIONS
INTERACTIONS
WITH THE
THE SUMEX-AIM
SUMEX-AIM
RESOURCE
A. Sharing
Interactions with
Projects
.Q.
Sharing and
and Interactions
with Other
Other SUAfEX-AIAf
SUMEX-AIM
Projects
Our
primary interaction
Our primary
interaction with
with the SUMEX
SUMEX community
community has been the work
work of
of the
projects at
prose comprehension
comprehension group
group with
with the AGE
AGE and UNITS
UNITS projects
at SUMEX.
SUMEX. Feigenbaum
Feigenbaum
and Nii
Nii have visited
visited Colorado,
Colorado, and one of
of us (Miller)
(Miller) attended
attended the AGE
AGE workshop
workshop at
at
sur"mx.
Both
of
these
meetings
have
been
very
valuable
in
increasing
our
understanding
SUMG.X. Both of
meetings
very valuable
increasing our understanding
of
problems might
by the various
of ilow
how our
our problems
might best be solved
solved by
various systems
systems available
available at
at SUMEX.
SUMEX.
\Ve also hope that
We
that our
our experiments
experiments with
with the AGE
AGE and UNITS
UNITS packages have been helpful
helpful
to the development
projects.
development of
of those projects.
\Ve
\Ve should
should also mention
mention theoretical
theoretical and experimental
experimental insights
insights that
that we have received
received
from
project. The
from Alan
Alan Lesgold and other
other members
members of
of the SUMEX
SUMEX SCP project.
The initial
initial
comprehension
by Dr.
comprehension model
model (Miller
(Miller &
& Kintsch,
Kintsch, 1980) has been used by
Dr. Lesgold and other
ot,her
researchers at
a.t tht'
the University
University of
of Pittsburgh,
Pittsburgh, as
as well
well as
as rest'archers
researchers at
at Carnegie-Mellon
Carnegie-Mellon
Univt'rsity,
Universit’y, the University
University of
of Manitoba,
Manitoba, Rockefeller
Rockefeller University,
University, and the University
University- of
of
Victoria.
Victoria.

Re.'3ource Management
Afanagement
B. Critique
C’ritique of
of Resource
The
The SU!\IEX-AIM
SIJ’llEX-AIM resource is clearly
clearly suitable
suitable for
for the current
current and future
future needs of
of
our
project. \Ve
our project.
We have found
found the staff
staff of
of SUMEX
SUMEX to be cooperative
cooperative and effective
effective in dea.ling
dealing
wit.h
The
wit,h special requirements
requirements and in responding
responding to
to our
our questions.
The facilities
facilities for
for
communicat.ion
on
the
ARPANET
have
also
facilitated
collaborative
communicat.ion
ARPANET
fa.cilitated collaborat.ive work
work with
with
investigators
investigators throughout
throughout the country.
country.

ill.
III. RESEARCH
RESEARCH PLANS
PLANS

A.
Long Range
Range Projects
Projects Goals
Plans
A. Long
Goals and
and Plans
The
by the prose comprehension
The use
use of
of SUMEX
SUMEX by
comprehension group
group was greatly
greatly reduced
reduced in the

139

E. A.
A. Feigenbaum
Feigenbaum

CLIPR
CLIPR - Hierarchical
Hierarchical Models
Models of
of Human
Human Cognition
Cognition

5P41 RR00785-11
RR00785-11

two years, because
because the
the focus of
of the work
work during
during that
that period
period was on experimental
experimental work
work
two
book writing,
writ.ing, rather
rather than
than computer
computer simulation.
simulation. This
This will
will change in the
the fall
fall of
of 1984,
and book
when a new research associate will
will join
project whose primary
primary responsibility
responsibility will
will be in
when
join the project
cont,inuing the modelling
modelling work
work started
started in previous
previous years
years with
with J. Miller
Miller (who
(who is no longer
longer
continuing
with us). Thus,
Thus, we expect
expect a level of
of activity
activity comparable
comparable to
to previous
previous years
years next
next
associated with
fall.
fall.

E. A. Feigrnbaum
Feigpnbaum

140

5P41
5P41 RR00785-11
RR00785-11

CLIPR
CLIPR -- Hierarchical
Hierarchical Models
Models of
of Human
Human Cognition
Cognition

The
The primary
primary goal
goal of
of the
the device
device complexity
complexity project
project isis the
the development
development of
of aa theory
theory
of
of the
the processes
processes and
and knowledge
knowledge structures
structures that
that are
are involved
involved in
in the
the performance
performance of
of routine
routine
cognitive
cognitive skills
skills making
making use
use of
of devices
devices like
like word
word processors.
processors. We
We plan
plan to
to model
model the
the useruserdevice
interaction
by
representing
the
users
processes
and
knowledge
as
a
production
device int,eraction by representing the users processes and knowledge as a production
system
system and
and the
t.he device
device as
as aa nested automata.
automata. We
We are
are also
also studying
studying the role of
of mental
mental
models in
in learning
learning how to use
use them.

B. Justification
and Requirements
Justification
Requirementa for
for Continued
Continued SUMEX
SUMEX Use
Use
The
The research of
of the prose comprehension
comprehension project
project isis clearly
clearly tied to continued
cont.inued access
access
to t.he
the AGE
AGE and UNITS
UNITS packages, which
which are simply
simply not available
available elsewhere. We hope that
that
our
our continued
continued use
use of these systems will
will be
be offset
offset by the input
input we have been
been and will
will
continue
continue to provide
provide to those projects:
projects: our
our relationship
relationship has
has been
been symbiotic,
symbiotic, and we look
forward
forward to its continuation.
continuation.

C. Needs
Keeds and
and Plans
Plans for
for Other
Other Computational
Computational Resources
Resources
\Ve currently
currently use
use two
two other
other computing
computing systems located at the University
Universit,y
Colorado.
Colorado. One is
is the Department
Department of
of Psychology's
Psychology’s VAX
VAX 11/780,
11/780, which
which is used
used primarily
primarily
run
run real-time
rea.l-t,ime experiments
experiments to be modeled on SUMEX.
SUMEX. The
The second is
is the University
University
Colorado's
Colorado’s CDC
CDC 6400,
6400. which
which is
is used
used for
for various
various types
types or
of statistical
statistical analysis.

of
to
of

When
When the ARPA-sponsored
ARPA-sponsored Vax/Interlisp
Vax/Interlisp project
project is completed,
completed, we would
would be most
most
int,erested in experimenting
experimenting with
with becoming
becoming a remote
remote AGE/UNITS
AGE/UNITS site. It
It would
would seem that
that
this
this sort
sort of dev('lopment
devciopment is the ultimate
ultimate goal of
of the package projects,
projects, and this
this type
type of
int.eract.ion,
intera,ct.ion, once it
it becomes feasible, would
would be a logical
logical extension
extension of
of our
our association
association wit.h
with
the SUMEX
SUhIEX fa.cility.
fa.cilit,y.

D. Recowlwzendations
Recommendations for
for Future
Future Community
Resource Development
Development
and Resource
D.
Community and
Our
prima.ry recommendation
Oiur primary
recommendation for
for ruture
future development
development within
within SUMEX
SUMEX involves
involves (a)
t.he
t,he cont,inued
cont,inurd support
support of
of INTERLISP,
INTERLISP, which
which is needed for
for AGE
AGE and for
for other
other work
work we
have underway
underway on SUMEX
SUMEX and (b)
(b) the continued
continued development
development of
of the
the AGE
AGE and UNITS
UNITS
projects. In
In particular,
particular, we would
would like
like to
to see
see an extension
extension of
of AGE
AGE to
to include
include a wider
wider
projects.
variety
of
cont.rol
structures
so
that
our
psychological
models
would
not
be
confined
to
variety of control structures
that our psychological
would not
confined to
one particular
part iclliar view
view of
of knowledge-based
knowledge-based processing. The
The limited
limited physical
physical capacity
capacity of
of
SUr-,fEX, both
both in terms
terms of
of address space and overloading,
overloading, is, as before,
before, a major
major problem.
problem.
SUhlES,
The prose comprehension
comprehension group
group can no longer
longer use the
the publicly
publicly released AGE/UNITS
AGE/UNITS
The
system due to
to its
its severely
severely limited
limited address space, and
and has had
had to
to build
build a personal
personal AGE
AGE
system
system from
from a stripped-down
stripped-down version
version of
of Interlisp
Interlisp and
and a selected subset
subset of
of AGE
AGE and
system
UNITS. We
We heartily
heartily endorse the
the plans
plans underway
underway to
to obtain
obtain more
more computing
computing capacity
capacity for
for
UNITS.
the SUMEX
SUl'vlEX project.
project.
the
Given our
our acquisition
acquisition of
of a VAX,
VA.X,
. we particularly
particularly support
support the
the ongoing
ongoing and
and continued
continued
Given
development of
of INTERLISP
INTERLISP for
for the
the VAX,
VAX, so that
that local
local use of
of AGE
AGE and
and UNITS
UNITS would
would be
development
possihle. Since we, as well
well as other
other psychologists,
psychologists, need the
the real-time
real-time capability
capability of
of
possible.
VAX/VMS to
to run
run on-line
on-line experiments,
experiments, we hope that
that the
the INTERLISP
INTERLISP system
system to
to be
VAX/VhIS
developed will
will be compatible
compatible with
with VMS.
VMS. Note
Note that
that this
this need for
for real-time
real-time work
work coincides
coincides
developed
with
real-world
applications
of
SUMEX
programs,
in
which
a
VAX
might
be
devoted
to
with real-world applications of SUMEX programs, in which
VAX might
devoted to
both real-t.ime
real-time patient
patient monitoring
monitoring and
and diagnostic
diagnostic systems
systems such
such as PUFF
PUFF or
or MYCIN.
MYCIN.
both

141

E. A.
A. Feigenba.um
Feigenbaum
E.

5P41
5P41 RR00785-11
RR00785- 11

Rutgers
Rut,gers Research
Research Resource
Resource

II.A.2.3.
Rutgers
II.A.2d.
Rutgers Research
Research Resource
Resource

Rutgers
in Biomedicine
Rutgers Research
Research Resource--Computers
Resource--Computers
Biomedicine
Principal
Saul
Principal Investig.ators:
Investigators:
Saul Amarel
Amarel [1982-83],
[1982-831,
Casimir
Casimir Kulikowski,
Kulikowski, Sholom
Sholom Weiss
Weiss [1983-84].
[1983-841.
Rutgers
Rutgers University,
University, New
New Brunswick,
Brunswick, New
New Jersey
Jersey

I. SUMMARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
A. Goals
Goals and
and Approach
Approach
The
The fundamental
fundamental objective
objective of the Rutgers
Rutgers Resource is to develop a computer
computer based
framework
framework for significant
significant research in the biomedical
biomedical sciences
sciences and for
for the application
application of
resea,rch
resea.rch result,s
results t.o
t.o the solution
solution of important
important problems
problems in health
health care. The
The central
central concept
concept
is
in artificial
is t.o
to introduce
int,roduce advanced
advanced methods
methods of computer
computer science
science - particularly
particularly
artificial
intelligence
int,elligence into
into specific
specific areas of biomedical
biomedical inquiry.
inquiry. The
The computer
computer is used as
as an integral
integral
part
part of the inquiry
inquiry process, both
both for
for the development
development and organization
organization of knowledge
knowledge in a
domain
domain and for its utilization
utilization in problem
problem solving
solving and in processes
processes of
of experimentation
experimentation and
theory
theory formation.
forma.tion.
At
At, present,
present., the total
total number
number of investigators
investigators who
who participate
participate in scientific
scientific activities
activities
of t.he
outside investigators
investigators
t.he Resource is 83, of
of these, 20 have Rutgers
Rutgers appointments,
appointments, 21 are outside
who
mainly located
located at
at Rutgers,
Rutgers, and
who participate
particips.te in collaborative
colla,borative research projects
projects that
that are mainly
42 are investigators
from
colla.borative
national
AIM
projects
that
are
located
different
investigators from collaborative national AIM projects that
located in different
parts
members in Administrative,
Administrative,
parts of
of the country.
country. In addition,
addition, the
the Resource has 12 other
other members
Computer
functions. Thus,
Thus,
Computer Systems/Operations
Systems/Operations and general
general programming
programming and secretarial
secretarial functions.
Rutgers Resource community
community numbers
numbers at
at present
present a total
total of
of 95 participants.
participants.
the Rutgers
activities include
include research projects
projects (collaborative
(collaborative research
research and
and core
Resource activities
res(~arch) training/dissemination
training/dissemination projects,projects, and computing
computing services in
in support
support of
of user
rese;r.rch)
projects.
projects.

B. Medical
Afedical Relevance
Reiwanre and
and Collaborations
Collaborations
B.
In 1883-84
1983-84 we continued
con tinned the
the development
development of
of several
several versatile
versatile systems
systems for
for building
building
In
and t,esting
testing consult,ation
consult.ation models
models in biomedicine.
biomedicine. The
The EXPERT
EXPERT system
system ha.s
has had
had many
many of
of
a.nd
its ca.psbilities
capabilities enhanced in the course of
of collaborative
collaborative research
research in
in the
the areas of
of
its
rheumatology, ophthalmology,
ophthalmologs, and clinical
clinical pathology.
pathology.
rheumatology,
In ophthnlmology
opht.halmology we have developed
developed a knowledge
knowledge representation
representation scheme for
for
In
treatment
pb,nning
which
is
both
natural
and
efficient
for
encoding
the
strategies
for
treatment planning which
both natural and efficient for encoding the strategies for
choo:3ing
among
competing
and
cooperating
treatment
plans.
This
involves
a
ranking
of
choosing a.mong competing
cooperating treatment plans. This involves
ranking of
treatments according
according to
to their
their characteristics
characteristics and
and desired
desired effects
effects as well
well as
treatments
contra indications. Kastner
Kastner has generalized
generalized the
the scheme so that
that itit is now
now being
being used
used for
for a
cont,raindications.
number of
of reasoning
reasoning models:
models: infectious
infectious eye disease, primary
primary eye care,
care, and
and rheumatology
rheumatology
number
m;1nagement. Our
Our main
main collaboration
collaboration continues
continues to
to be with
with Dr.
Dr. Chandler
Chandler Dawson
Dawson of
of the
the
management.
Proct.or Foucdation,
Foundation, UCSF.
UCSF.
Proctor
In rheumatology,
rheumatology, our
our collaboration
collaboration with
with Drs.
Drs. Donald
Donald Lindberg
Lindberg and
and Gordon
Gordon Sharp
Sharp
In
at the
the Universit’
University
of Missouri-Columbia
Missouri-Columbia has
has continued
continued at
at a very
very active
active level.
level. The
The model
model
at
y of
which now
now includes
includes detailed
detailed diagnostic
diagnostic criteria
criteria for
for 26
26 major
major
for rheumstologicsl
ri1ellmatological diseases which
for

E. A.
A. Feigenbaum
Feigenbaum
E.

142

Rutgers
Rutgers Research
Research Resource
Resource

5P41
5P11 RH00785-11
RROOi85- 11

diseases,
diseases, had
had the
the management
mana.gement advice
advice and
and treatment
treatment planning
planning developed
developed further.
further. Dr.
Dr.
Sha.rp's
Sha.rp’s group
group continues
continues to
to develop
develop the
the knowledge
knowledge base
base in
in this
this area,
area, with
with formalization
formalization of
of
the
the knowledge
knowledge carried
carried out
out in
in conjunction
conjunction with
with Dr.
Dr. Lindberg's
Lindberg’s group
group and
and the
the Medical
Medical
Expert
The
Expert Systems
Systems Group
Group at
at Rutgers.
Rutgers.
The Resource
Resource researchers
researchers have
have developed
developed new
new
representational
representational elements
elements for
for EXPERT
EXPERT in
in response
response to the needs
needs of
of the rheumatology
rheumatology
research, and
and Politakis
Politakis has
has developed
developed aa coordinated
coordinated system
system called
called SEEK
SEEK (System
(System for
for
Empirical
Empirical Experimentation
Experimentation with
with Expert
Expert Knowledge.)
Knowledge.) which
which provides
provides interactive
interactive assistance
assistance
to the human
human expert
expert in
in testing,
t,esting, refining
refining and
and updating
updating aa knowledge
knowledge base
haze against
against aa data
dat,a.
base
bsse of
of trial
trial cases.
ca.ses.SEEK
SEEK has
has been
been tested and
and extended
extended during
during the past year.
In clinical
Robert Galen
clinical pathology
pathology our
our main
main collaboration
collaboration has
has been with
with Dr. Robert
(Cleveland
Clinic
Foundation),
with
whom
we
have
developed
the
serum
serum protein
protein
(Cleveland Clinic Foundation), with whom
electrophoresis
electrophoresis model which
which isis incorporated
incorporated into
into an instrument
instrument - the scanning
scanning
densitometer
This
densitometer manufactured
manufactured by Helena Laboratories.
Laboratories.
This instrument
instrument with
with interpretive
interpretive
reporting
reporting capabilities
capabilities has now been
been on the market
market for over aa year, isis located at over 100
100
clinical
clinical sites, and represents
represents the first
first known
known spin-off
spin-off of AI
AI expert
expert systems research in the
t.he
field of laboratory
instrumentation.
'vVe
continue
laboratory
instrumentation.
We
continue to refine
refine the representational
representational
mecha.nisms
model.
mecha.nisms used
used for
for this
this kind
kind of model.
In biomedical
biomedical modeling
modeling applications
applications we are experimenting
experimenting with
with several prototype
prototype
models for
for giving
giving advice on the interpretation
interpretation of
of experimental
experimental results
results in the field
field of
enzyme kinetics.
kinetics. in conjunction
conjunction with
with Dr.
Dr. David
David Garfinkel.
Garfinkel. His PENNZYME
PENNZYME program
program has
been linked
linked to a model in EXPERT,
EXPERT, which
which allows
allows the user to interpret
interpret the progress of the
model analysis.

C. Highlights
of Research
Research Progress
Progress
Highlights of
Expert
Expert Medical
Medical Systems
Systems (C.
(C. Kulikowski,
Kulikowski, S. Weiss)
Weiss)
Research has continued
problems of
continued on problems
of representation,
representation, inference
inference and control
control in
expert
systems.
Emphasis
has
been
placed
this
year
on
problems
of
knowledge
expert
Emphasis
placed this year
problems of knowledge base
acquisition,
acquisition, empirical
empirical testing
testing and refinement
refinement of
of reasoning
reasoning (the
(the SEEK
SEEK system),
system), and
treat.ment planning
planning st,ra.tegies
strategies over
over time.
time. From
From a technological
technological point
point of
of view
view the
the market,
market
trea.t.ment
availability of
of the int.erpretive
interpretive reporting
reporting version
version of
of a scanning
scanning densitometer,
densitometer, and the
the
availability
development of
of modeis
models for
for eye care consultation
consultation that
that run
run on microprocessor
microprocessor systems
systems
development
(Apple IIc,
lIe, IBM-PC)
IBM-PC) represents
represents an important
important achievement
achievement for
for AIM
AIM research in showing
showing its
its
(Apple
practical impact
imp:1ct in medical
medical applications.
applications. This
This was recognized
recognized by
by the award
award of
of a scientific
scientific
practical
exhibit prize
prize at.
at, the Academy
Academy of
of Ophthalmology
Ophthalmology Annual
Annual Meeting
Meeting in November
November 1983.
exhibit
SEEK: A
A System
System for
for Empirical
Empirical Experimentation
Experimentation with
with Expert
Expert Knowiedge
Knowledge
1.1) SEEK:
SEEK is a~t system
syst.em which
which has been developed
developed t.o
to give
give interactive
interactive advice
advice about
about rule
rule
SEEK
refinement
daring
the
design
of
an
expert
system.
The
advice
takes
the
form
of
refinement during the
of
expert system.
The advice takes the form of
suggestions
for
possible
experiments
in
generalizing
and
specializing
rules
in
an
expert
suggestions for possible experiments in generalizing and specializing rules in
expert
model that
that has been specified
specified based on
on reasoning
reasoning rules
rules cited
cited by
by aa. human
human expert.
expert. Case
model
('xperience, in the
the form
form of
of stored
stored cases
cases with
with known
known conclusions,
conclusions, is used to
to interactively
interactively
experience.
guide the
the expert
expert in refining
refining the
the rules
rules of
of a model.
model. The
The clesign
design framework
framework of
of SEEK
SEEK consists
consists
gl.;idtx
of a tabular
tabular model
model for
for expressing
expressing expert-modeled
expert-modeled rules
rules and
and a general
general consultatiou
consultation system
system
of
for a.pplying
applying a model
model to
to specific
specific cases.
cases. This
This approach
approach has proven
proven particularly
particularly valuable
valuable in
in
for
assisting the
the expert
expert in
in domains
domains where
where the
the logic
logic for
for discriminating
discriminating two
two diagnoses is difficult
difficult
assisting
to specify;
specify; and
and we have
have benefited
benefited primarily
primarily from
from experience
experience in
in building
building the
the consultation
consultation
to
system in
in rheumatology.
rheumatology.
system
Treatment Planning
Planning
1.2) Trea.tment
The ranking
ranking and
and selection
selection strategies
strategies developed
developed as
as aa stand-alone
stand-alone system
system last
last year
year
The

143

E. A.
A. Feigenbaum
Feigenbaum
E.

Rutgers
Rut,gers Research Resource

5P41 RR00785-11

have been
Capabilities
Capabilities for expressing
been incorporated
incorporated into
into the EXPERT
EXPERT framework.
framework.
reasoning over time
time have been
been added, so stored
stored chart
chart reviews can be
be carried
carried out
out
automatically,
a.ut~omat.ically. summarizing
summarizing various
various patterns
patterns of findings
findings over
over time,
time, and abstracting
abstracting the
major
major features
features of interest
interest for prognostic
prognostic advice or treatment
treatment recommendations.
recommendations.
Applications
Applications have been in infectious
infectious eye
eye disease
disease modeling,
modeling, rheumatology
rheumatology treatment,
treatment, and
sequential
sequential advice in interpretation
interpretation and sequencing of cardiac
cardiac enzyme tests (e.g. CPK/LDH
CPK/LDH
isoenzymes).
1.3) Technology
Technology Transfer
Transfer
Important
Import.ant technology
technology transfer
transfer milestones
milestones have also been achieved this
this year:
year: the
instrument
EXPERT
inst.rument interpretation
interpretation
EXPERT program
program for
for serum
serum protein
protein has been widely
widely
disseminated
disseminated after
after being made available
available by Helena Laboratories,
Laboratories, based on the prototype
prototype
program
program developed by us;
us; and we have succeeded
succeeded in transferring
transferring a large knowledge
knowledge base
base
in rhel1matology
hypotheses and 1000 rules)
rules) onto
onto a
rheuma.tology (about
(about 1000 findings,
findings, 400 hypotheses
mi<'roprocessor
microprocessor (rvtotorola
(Motorola 68000) based system
system - the WICAT
WICAT - which
which is well
well within
within the
means of clinical
clinical researchers and practitioners.
practitioners. This
This system
system has been on site at the
University
University of r-.lissouri
Missouri during
during the last
last year
year for
for testing
testing and refining
refining of
of the knowledge
knowledge base.
base.
1.4)
1.4j Learning
Learning with
with Prior
Prior Structural
Structural Knowledge
Knowledge
This
This approach
approach to knowledge
knowledge acquisition
acquisition and representation
representation has as
as its goal to allow
allow
the expert
just the elements
expert to specify
specify just
elements that
that are to enter
enter into
into the reasoning
reasoning model,
model, with
with a
few causal and taxonomic
taxonomic relations.
relations. These should
should then
then be
be sufficient
sufficient to guide a learning
learning
program
which
operat,es
on
a
data
base
of
cases
with
known
program which operates
data base of cases with known end-points.
end-points. Such an approach
approach
wonld
woiiid he
be w3eful
ii,seful in situations
sit:uations where
where the expert
expert either
either has little
little time
time to explicitly
explicitly formulate
formulate
decision rules, or
program [Drastal
or finds
finds it
it difficult
difficult to do so.
so. Our
Our program
[Drastal and Kulikowski,
Kulikowski, 1982]
19821
uses
uses aa. blackboard
blackboard representation,
representation, with
with multiple
multiple knowledge
knowledge sources to
to handle
handle the different
different
conclusions,
pertain to
conclusions, and the formation
formatsion of
of rules from
from the data
data that
that pertain
to them.
them. We
We have
tested
tested this
this scheme in the areas of
of glaucoma
glaucoma and rheumatology,
rheumatology, and shown
shown that
that there
there are
some int.eresting
provided by
interesting tradeoffs
tradeoffs between
between the degree of
of a-priori
a-priori structure
structure provided
by the expert,
expert,
and
snd the
t.he complexity
complexity of
of rule
rule generation.
generation.
In relat
ion to
relation
to
alternative
alternn.tive means
mea.ns of
of
of
of additional
a.dditional medical
medical

a system
preprocessing or
system like
like SEEK,
SEEK, this
this approach
approach represents
represents a preprocessing
or
developing
prototype model. We
developing the prot.otype
We are now
now investigating
investigating the role
semantic
semantic constraints
constraints on the strategies
strategies of
of rule
rule generation.
generation.

2) Artificial
Artificial 1ntelligence:Expertise
Intelligence:Expertise Acquisition
Acquisition and Problem
Problem Reformulation
Reformulation (S.
Amarel)
Amarei)
The main
main research
research activity
activity in this
this area is concerned
concerned with
with improvements
improvements in problem
problem
The
solving expertise
expertise via
via shifts
shifts in problem
problem representation,
representation, i.e., via
via reformulation.
reformulation.
solving
this research, we have concentrated
concentrated on the developmental
developmental processes
processes that
that lead to
to
In this
formation of
of specialized
specialized high
high performance
performance procedures
procedures in sub-domains
SUb-domains of
of a problem
problem
the forma.tion
ci::lss. Theory
Theory formation
formation is a,
a key
key task
task in these processes;
processes; and we are now
now studying
studying several
several
CIXS.
approaches to this
this task
task - both
both top-down,
top-down, model
model guided,
guided, approaches
approaches and ‘'bottom-up'
approaches
bottom-up’
methods that
that are based on detailed
detailed analysis
analysis of
of individual
individual cases.
cases.
methods

D. lip-to-Date
Up-to-Date List
List ofof Publications
Publications
D.
The following
following is an update
update of
of publications
publications in the
the Rutgers
Rutgers Resource for
for the
the period
period
The
198·1 (only
(only publications
publications not
not listed
listed in previous
previous SUMEX
SUMEX annual
annual reports
reports are
1983 and 198.1
presented here).
presented
Wei5s, S.M.
S.M. and
and Kulikowski,
Kulikowski, C.A.
C.A. A
A Practbcal
Practical Guide
Guide to Designing
Designing Ezpert
Expert
1. Weiss,
Systems, Rowman
Rowman a.nd
and Allanheld,
Allanheld, 1984.
Syatemu.
Feigenbaum
A. Feigenbaum
E. A.

144

Rutgers
Rutgers Research
Research Resource
Resource

5P41
5P41 RR00785-11
RR00785- 11

2.
2. Kulikowski,
Kulikowski, C.A.
C.A. contributor
contributor to
to the
the Knowledge
Knowledge Acquisition
Acquisition chapter
chapter edited
edited by
by
B.
B. Buchanan
Buchanan in
in the
the book
book Building
Building Expert
Expert Systems
Systems (F.
(F. HayesHayes- Roth,
Roth, et
et aI.,
al.,
cds)
eds) Addison-\o\Tesley,
Addison-Wesley, 1983
1983 (in
(in press).
press).

Strategies
of
3.
3. 1'30,
l’a.0, Y.
Y. and
and Kulikowski,
Kulikowski, C.A.,
C.A., "lYfultiple
’ Multiple
Strategies
of Reasoning
Reasoning for
for
Expert
Expert Sy.'3tems",
Systems “, Proc.
Proc. Sixteenth
Sixteenth Hawaii
Hawaii International
International Conference
Conference on Systems
Sciences,
Sciences, pp.
pp. 510-514
510-514 ,, 1983.
1983.**
4.
4. Kulikowski,
Kulikowski, C.A.
C.A. "Progress
“Progress in
in Expert
Expert AI
Al Medical
Medical Consultation
Consultation Systems:
Systems:
1980
1980 -- 1983
1983 If,“, Proc.
Proc. MEDINFO
MEDINFO '83
‘83 , pp.
pp. 499-502,
499-502, Amsterdam,
Amsterdam, August
August 1983.*
1983.t
5.
5. Kastner,
Kastner, J.K.,
J.K., \Veiss,
Weiss, S.M., and Kulikowski,
Kulikowski, C.A.,
C.A., "An
“An Efficient
Efficient
Consultation
Systems",
Proc.
MEDINFO
Time-Dependent
Time-Dependent Consultation
Systems I,
MEDINFO '83,
‘83,
1983.
*
1983.’

Scheme for
for
pp.619-622,

6.
6. K1Jlikowski,
Kulikowski, C.A.
C.A. "Expert
“Ezpert Medical
Medical Consultation
Consultation Systems",
Systems “, Journal
Journal of Medical
Medical
Systems, v.7. pp. 229-234, 1983.*
7. Weiss, S.M., Kulikowski,
Kulikowski, C.A.,
C.A., and Galen, R.S., "Representing
“Representing Expertise
Expertise in
in a
Computer
Computer Program:
Program: The Serum
Serum Protein
Fbotein Diagnostic
Diagnostic Program",
Program”, Journal
Journal of
Clinical
Clinical Laboratory
Laboratory Automation,
Automation, v.3, pp. 383-387, 1983.*
8. Kastner,
for
Kastner, J.K.,
J.K., Weiss, S.M., and Kulikowski,
Kulikowski, C,A.,
C.A., "” lin
An Expert
Expert System
System for
Front-line
Front-line Health
Health Workers
Workers in
in Primary
Primary Eye Care",
Care “, Proc.
Proc. Seventeenth
Seventeenth Hawaii
Hawaii
International
1nt.ernationn.i Conference
Conference on Systems Sciences,
Sciences, pp. 162-166, 1984.
1984.**
9. Kulikowski,
Kuiikowski. C.A.
C.A. "Knowledge
“K,nowledge Acquisition
Acquisition and
and Learning
Learning in
in EXPERT",
EXPERT”, Proc.
Proc.
1983 'Workshop
IIIinois,Champaign-Urbana
Workshop on Machine
Machine Learning,
Learning, Univ.
Univ. of
of Illinois,Champaign-Urbana
1983.
credit.
Indicat.e
Indicate by
by an
a,n asterisk
asterisk (*)
(*) that
that the resource was given
given credit.

E. Funding
Funding Support
Support
Since December
December 1983, the Rutgers
Rutgers Research Resource on Artificial
Artificial Intelligence
Intelligence in
funded under
under grant
grant RR
RR 02230-01 from
from the Division
Division of
of Research Resources,
Medicine is funded
Medicine
Biotechnology Resources Program.
Program. Principal
Principal Investigators
Investigators are Casimir
Casimir A.
A. Kulikowski,
Kulikowski,
Biotechnology
Proffssor of
of Comput,er
Computer Science and Chairman
Chairman of
of the Department
Department of
of Computer
Computer Science
Profrssor
Dr. Shoiom
Sholom M.
M. Weiss,
\Veiss, Associate
Associate Research Professor
Professor of
of Computer
Computer Science.
[1984-87]' and Dr.
[1984-871,
The tota.
total direct
direct costs for
for the
the period
period 1983-87 is
The
current
period
(December
I,
1983
November
current period (December 1,
November 30, 1984)

$3,198,075, with
with the
the total
total for
for the
the
$3,198,075,
being
$
989,276.
being

The Rutgers
Rutgers Resource was funded
funded until
until December
December 1983 through
through an NIH
NIH grant
grant
The
entitled"“Rut.gers
Rut,gers Research Resource on
on Computers
Computers in
in Biomedicine”
Biomedicine" -- number
number P41RR643.
P41RR643.
entitled
The Co-Principa.1
Co-Principal Investigators
Investigators were
were Dr.
Dr. Saul
Saul Amarel,
Amarel, Professor,
Professor, Chairman
Chairman of
of the
the
The
Department of
of Computer
Computer Science, and Director
Director of
of the
the Laboratory
Laboratory for
for Computer
Computer Science
Department
Research, and Dr.
Dr. Casimir
Casimir Kuiikowski,
Kulikowski, Professor
Professor of
of Computer
Computer Science at
at R.utgers.
Rutgers.
Kesezrch,

II. INTER4CTIONS
INTERA.CTIONS WITH
WITH THE
THE SUMEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
II.

A. hledical
Aledical Collaborations
Collaborations and
and Dissemination
Dissemination
A.
The SUMEX-hIM
SUMEX-AIM facility
facility provides
provides a backup
backup node
node where
where some of
of our
our medical
medical
The
collaborators
can
access
programs
developed
at
Rutgers.
The
bulk
of
the
medical
collaborators ca.n access programs developed at Rutgers.
The bulk of the medical
collaborative work
work outlined
outlined in
in LB.
LB. above
above is centered
centered at
at the
the Rutgers
Rutgers facility
facility (the
(the RutgersRutgerscollaborative
AIM node).
node).
AIM

145

A. Feigenbaum
Feigenbaum
E. A.
E.

RR00785-11
5P41 RR00785-11

Rutgers
Rutgers Research Resource

Dissemination
Dissemination activities
activities continue
continue to
to be an important
important responsibility
responsibility of
of the Rutgers
Rutgers
Resource within
wit)hin t,he
t,he AIM
AIM community.
community. The
The following
following activities
activities took
took place in the last year:
year:
1.
Ninth AIM
1. Ninth
,4IM \Vorkshop
Workshop (1983):

Organized
Organized by
by Dr.
Dr. Casimir
Casimir Kulikowski,
Kulikowski, it
it was held in Baltimore,
Baltimore, in conjunction
conjunction
with
It
It consisted
consisted of
of a series of
of working
working group
group
with the SCAMC
SCAMC 83 meeting.
meeting.
discussions followed
followed by
by summary
summary presentations
presentations by
by members
members of
of the AIM
AIM
community
community on their
their conclusions.
conclusions.
2. Hawaii
Hawaii International
International Conference
Conference On
On Systems Sciences:
Sciences:
presented a paper
paper on the expert
Dr.
Dr. \Veiss
Weiss presented
expert system
system for
for front-line
front-line health
health
workers,
workers, and Dr.
Dr. Kulikowski
Kulikowski chaired
chaired a session
session on knowledge
knowledge based medical
medical
systems.
3. VII-pn,n-American
VII-Pan-America,n Congress on Rheumatology:
Rheumatology:
Dr.
presented the
Dr. Sharp
Sharp present,ed
the rheumatology
rheumatology knowledge
knowledge base and consultation
consultation
program at this
program
this meeting.
meeting.
4. At
Amarel was elected member
member of
of the Executive
Executive
At the AAAI-82
AAAI-82 meeting,
meeting, S. Amarel
Council
Council of
of AAAI.
AAAI. He is also General
General Chairman
Chairman of
of IJCAI-83
IJCAI-83 which
which was held
held in
Karlsruhe,
Karlsruhe, W.
W. Germany
Germany in August
August 1983. Dr.
Dr. Kulikowski
Kulikowski was the organizer
organizer for
for
an expert
expert medical
medical systems session
session at
at MEDINFO
MEDINFO 83.

B.
National .11.\1
Projects at Rutgers
Rutgers
B. National
AIM Projects
The
projects, approved·
The national
national AIM
AI&l projects,
approved. by
by the AIM
AIM Executive
Executive Committee,
Committee, that
that are
associated with
with the RutgerS-AIM
Rutgers-AIM node are the
the following:
following:
1. INTERNIST
jCADUCEUS project,
project, headed by
INTERNISTjCADUCEUS
by Dr.
Dr. Myers
Myers and Dr.
Dr. Pople
Pople from
from

the University
University of
of Pittsburgh,
Pittsburgh, has been using
using the Rutgers
Rutgers Resource as a backup
backup
system
system for
for development
development, and experimentation.
experimentation.
2. Medical
project, headed by
Medica. Knowledge
Knowledge Representation
Representation project,
by Dr.
Dr. Chandrasekaran
Chandrasekaran

from
from Ohio
Ohio State
State University,
University, is doing
doing most
most of
of its
its research on the Rutgers
Rutgers
system.
system.
3. PURSUIT
project, directed
PURSUIT project,
directed by
by Dr.
Dr. Greenes from
from Harvard
Harvard University,
University, is doing
doing

most
most of
of its
its research on a Goal-Directed
Goal-Directed Model
Model of
of Clinical
Clinical Decision-Making
Decision-Making at
Rutgers.
Rutgers.
4. Biomedical
Biomedical Modeling,
Modeling, by
by Dr.
Dr. Garfinkel
Garfinkel from
from the
the University
University of
of Pennsylvania.
Pennsylvania.
5. At.tending
At,tending Project,
Project, directed
directed by
by Dr.
Dr. Perry
Perry Miller
Miller of
of the Yale
Yale Medical
Medical Center,
Center, is
doing
physician's
plan of
doing much
much of
of the
the research on critiquing
critiquing a physician’
s plan
of management
management at
Rutgers.
Rutgers.
6. MEDSIM
project:
MEDSIM project:

This
pilot project
project designed to provide
provide resource-sharing
This is a pilot
resource-sharing
a.nd
and community
community building
building facilities
facilities for
for about
about 25 researchers
researchers in bio-mathematical
bio-mathematical
modeling
modeling and simulation.
simulation.

C. Critique
Resource Management
Management
Critique of
of SUMEX-AIAf
SUMEX-AIM
Resource
Rutgers
using
DEC-20
primarily for
SUMEX
DEC-PO system
system primarily
for
Rut,gers is currently
currently
using the SUMEX
communication
communication with
with other
other researchers in the AIM
AIM community
community and with
with SUMEX
SUMEX staff,
staff, and

E. A. Feigenbaum
Feigenbaum

146

Rutgers
Rutgers Research
Research Resource
Resource

SP41
5P41 RR0078S-11
RROO785-11

also
also for
for backup
backup computing
computing in
in demonstrations,
demonstrations, conferences
conferences and
and site
site visits.
visits. Our
Our usage
usage isis
currently
currently running
running at
at less
less than
than 50
50 connect
connect hours
hours per
per year
year at
at SUMEX,
SUMEX, with
with an
an overall
overall
connect/CPU
connect/CPU ratio
ratio of
of about
about 30.
30.
Rutgers
Rutgers isis beginning
beginning to
to place
place more
more emphasis
emphasis on
on the
the use
use of personal
personal computers,
computers, and
on
on network
network support
support needed
needed to
to make
make these
these effective.
effective. Sumex
Sumex has
has been
been help in the following
following
wa.ys:
ways :
•l The
The AIM
AIM Executive
Executive Committee
Committee allocated
allocated to
to the
the Rutgers-AIM
Rutgers-AIM node
node one
one of the
Xerox
Xerox Dolphins
Dolphins acquired
acquired by
by SUMEX,
SUMEX, to
to help
help us
us develop experience in
supporting
This machine was used
used almost
almost entirely
entirely to help
support,ing personal machines. This
us
us develop and test network
network support(We
support(We are using Ethernet
Ethernet with
with the Xerox
Xerox
PUP
PUP networking
networking protocols),
protocols), and subsequently
subsequently returned
returned to SUMEX.
SUMEX.
•l Most
Most of network
network software
software that
that we use
use
Having
this
software
available
Having t,his software availa.ble has saved

was originally
originally developed at SUMEX.
SUMEX.
us an enormous
enormous amount
amount of time.
time.

•l Initia.lly
Initinlly SUMEX
SUMEX was very
very helpful
helpful in giving
giving us advice about
about setting
setting up our
our
Ethernet
Ethernet and the Dolphins.
Dolphins.

m.
III. RESEARCH
RESEARCH PLANS
PLANS
A.
A. Project
Pro;lect Goals
Goals and
a,nd Plans
Plans
We
We are pla.nning
planning to
t,o continue
continue along the main
main lines of
of research that
that we have
est.ablished
est.n.blished in the
t,he Resource to date. Our
Our medical
medical collaborations
collaborations will
will continue
continue with
with
emphasis on development
development of expert
expert consultation
consultation systems in rheumatology,
rheumatology, ophthalmology
ophthalmology
and clinical
inference and planning
planning will
will
clinical pathology.
pathology. The
The basic AI
AI issues
issues of
of representation,
representation, inference
continue
Our
with emphasis on further
further
continue to receive attention.
attention.
Our core work
work will
will continue
continue with
development
representations and
development of
of the
t,he EXPERT
EXPERT framework
framework and also on AI
AI stUdies
studies in representations
problems
of
knowledge
and
expertise
acquisition.
We
propose
to
work
number of
of
to work on a number
problems
knowledge
expertise acquisition. We
terhnology
will be affordable
affordable by
by our
our
technology transfer
transfer experiments
experiments to micro
micro processing
processing that
that will
biomedka.l research snd
and clinical
clinical collaborators.
collaborators. We
We also plan
plan to
to continue
continue our
our participation
participation
biomedica.1
AIM dissemination
dissemination and training
training activities
activities as well
well as our
our contribution
contribution -- via
via the
the
in :\IM
RUTGERS/LCSR computer
computer -- to
to the shared
computing facilities
facilities of
of the national
national AIM
AIM
RUTGERS/LCSR
shared computing
network.
network.

B. Justification
Justification and
and Requirements
Requirements for
for Continued
SUMEX Use
B.
Continued SUMEX
Continued access
access to
to SUMEX
SUMEX is needed for:
for:
Continued
Ba.ckllP for
for demos, et,c.
et.c.
1. Rack~lp
Programs developed
developed to
to serve the National
National AIM
AIM Community
Community should
should be runnable
runnable
2. Programs
both facilit.ies.
facilities.
on both
There should
should be joint
joint development
development activities
activities between
between the
the staffs
staffs at
at Rutgers
Rutgers and
and
3. There
SUMEX in order
order to
to ensure portability,
portability, share the
the load,
load, and
and provide
provide a wider
wider
SUMEX
variety of
of inputs
inputs for
for developments.
developments.
variety

C. Needs
Needs and
and Plans
Plans for
for Other
Other Computing
Computing Resources
Resources Beyond
Beyond SUMEX-AIM
SUAIEX-AL\1
C.
Our comput,ing
computing is going
going to
to move
move in
in the
the direction
direction of
of personal
personal computers.
computers. We
vVe will
will
Our
continue to
to use Sumes
Sumex for
for backup
backup purposes,
purposes, however.
however.
continue

D. Recommendations
Recommendations for
for Future
Future Community
Community and
and Resource
Resource Development
Development
D.

147

E. A.
A. Feigenbaum
Feigenbaum
E.

Rutgers Research Resource
Rutgers

5P41 RR00785-11
RR00785- 11

Use of
of personal
personal comput,ers
compubers and minicomputers
minicomputers is continuing
continuing to
to grow
grow in the
the AIM
AIM
community.
We find
find that
that the
the biggest
biggest challenge
challenge is supporting
supporting these systems. Although
Although
community.
vVe
central computing
computing will
will continue
continue to
to be needed for
for communication
communication and coordination,
coordination,
some central
that over
over the next
next few years
years all AIM
AIM research projects
projects and even individual
individual
we believe that
collaborators
collaborators will
will come to
to have their
their own
own hardware.
hardware. However
However many
many of
of these community
community
members (particularly
(particularly the
the collaborators)
collaborators) will
will not
not be in a position
position to
to support
support hardware
hardware or
or
members
software
software on their
their own.
own. We
We would
would certainly
certainly expect
expect SUMEX
SUMEX to
t,o continue
continue to
to provide
provide expert
expert.
this area. However
However we believe it
it would
would be helpful
helpful for
for SUMEX
SUMEX to
to have a formal
formal
advice in this
program to support
support smaller
smaller computers
computers in the field.
field. We
We envision
envision this
this as including
including at least
program
following items:
it,ems:
the following

A central
central source of
of information
information on hardware
hardware and software
software that
that is likely
likely to
to be
•l A
of interest
interest to the AIM
AIM community.
community.
SUMEX might
might want
want to become a
of
SUMEX
distribution
dist,ribution point
point for
for certain
certain of
of this
this software,
software, and even help coordinate
coordinate
quantity
quantity purchase
purchase of
of hardware
hardware if
if this
this proves
proves useful.
useful.
•l Assistance
Assistance in support
support of
of hardware
hardware and software
software in the
the field.
field. Depending
Depending upon
upon
the hardware
hardware involved,
involved, this
this might
might involve
involve advice over
over the telephone
telephone or
or actual
actual
board-swapping
board-swapping by
by mail.
mail. \Vith
With our
our Dolphins
Dolphins we have found
found that
that there
there are a
number
number of
of problems
problems that
that can be resolved
resolved over
over the telephone
telephone if
if we can find
find
someone with
with appropriate
appropriate expertise.
expertise.

E. A. Feigenbaum
Feigenbaum

148

Rutgers
Rutgers Research Resource

5P41
5P4 1 RR0078S-11
RR00785- 11

ll.A.2.4.
SECS:
&
of
II.A.2.4.
SECS: Simulation
Simulation
St Evaluation
Evaluation
of Chemical
Chemical Synthesis
Synthesis
SECS
SECS - Simulation
Simulation and
and Evaluation
Evaluation or
of Chemical
Chemical Synthesis
Synthesis Project
Project
Principal
W.
Principal Investigator:
Investigator:
W. Todd
Todd Wipke
Wipke
Board
Board or
of Studies
Studies in
in Chemistry
Chemistry
University
University or
of California
California
Santa
Santa Cruz,
Cruz, CA.
CA. 95064
95064
Coworkers:
Coworkers:
l.
I. Kim
Kim
D. Rogers
J. Chou
Chou
M.
hl. Hahn
Hahn
M.
hf. Yanaka
>‘anska
l.
I. Iwataki
Iwataki

1.
I.

(Grad
(Grad student)
student)
(Grad
(Grad Student)
Student)
(Postdoctoral)
(Postdoctoral)
(Grad
(Grad Student)
Student)
(Postdoctoral)
(Postdoctoral)
(Postdoctoral)
(Postdoctoral)

SUMMARY
OF
SUMMARY
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
A. Project
Project Rationale
Rationale
-4.

\Vith the SECS project
project our
principles of
With
our long range goal is to
to develop the logical
logical principles
of
molecula.r
practical computer
programs to
molecu1a.r construction
construction and to
to use
use these in developing
developing practical
computer programs
to
assist
bio-organic molecules.
a.ssist investigators
investigators in designing
designing stereospecific
stereospecific syntheses of
of complex
complex bio-organic
Our
project, is aimed
Our second area
a.rea of
of research, the XENO
XENO project,
aimed at
at improving
improving methods
methods for
for
predicting potential
potential biologica.
biological activity
plausibility of
predictsing
activity of
of metabolites
metabolites and plausibility
of incorporation
incorporation and
excretion
excretion of metabolites.
metabolites.

B. Medical
Afedical Relevance
Relevance and
and Collaboration
Collaboration
The
biological activity
The development
development of
of new drugs
drugs and the study
study of
of drug
drug structure
structure biological
activit,y
relationships
depends
upon
the
chemist's
ability
to
synthesize
new
molecules
relstionships
chemist’s ability to synthesize
molecules as
as well
well as
as his
to
isotopic
a.hility
incorporating
isotopic labels or
or other
ot,her
a.bility
to modify
nlodify existing
existing structures,
structures, e.g., incorporating
substituents
bio-molecular substrates.
The Simulation
Simulation and Evaluation
Evaluat,ion of
of Chemical
Chemical
subst ituents into
into bio-molecular
substrates. The
Synthesis
project aims at assisting
Synt,hesis (SEeS)
(SECS) project
assisting the synthetic
synthetic chemist
chemist in designing
designing stereospecific
stereospecific
synt,heses
biologically important
The advantages
advantages of
of this
this computer
computer approach
approach
syntheses of
of biologically
important molecules. The
over
over normal
normal manual
manual approaches
a.pproa.ches are many:
many: 1) greater
greater speed in designing
designing a synthesis;
synthesis; 2)
freedom
past experience and past solutions;
freedom from
from bias of
of past
solutions; 3) thorough
thorough consideration
consideration of all
possibl(' syntheses llsing
possible
using a more extensive
extensive library
library of
of chemical
chemical reactions
reactions than
than any
any individual
individual
person can remember;
remember; 4) greater
greater capability
capability of
of the computer
computer to
to deal with
with the many
many
struct
st.rucl ilHS
ures which
which result;
result,: and 5) capability
capability of
of computer
computer to
to see
see molecules
molecules in a graph
graph
theoretical
projection.
theoretical S(,llSC.
sense. free from
from the bias of
of 2-D projection.
The
using XENO
predict the plausible
plausible
The objective
objective of
of using
XENO in metabolism
metabolism studies
studies is to
to predict
metabolites
of
a
given
xenobiotic
in
order
that
they
may
be
analyzed
for
metabolites of
given xenobiotic
order that they may
analyzed for possible
carcinogenicity.
Metabolism
carcinogenicity.
Metabolism research may
may also find
find this
this useful in the identification
identification of
of
metabolites
met,a.bolitcs in that
t,lmt it
it suggests what
what to
to look
look for.
for. Finally,
Finally, one may
may envision
envision applications
applications of
of
this
problem domains
t,his technology
technology in problem
domains where
where one wishes to
to alter
alter molecules
molecules in order
order to
to inhibit
inhibit
certain
certain types
types of
of metabolism.
metabolism.

C. Highlights
Highlights of
Research Progress
Progress
o j Research

149

E. A.
A. Feigenbaum
Feigenbaum

SECS: Simulation
Simulation 8l
62 Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

RR00785-11
5P41 RR00785-11

C.1 SECS
SECS Project
C.l
Project Developmenta
Developments
The
The majorit.y
majority of
of our
our research has been aimed
aimed at
at strategic
strategic planning
planning in chemical
chemical
synthesis.
Specific work
work has included
included the SST
SST project
project for
for recognizing
recognizing potential
potential starting
starting
synthesis. Specific
materials from
from a target,
target, the MCS
MCS project
project for
for maximal
maximal common
common subgraph
subgraph searching,
searching, and a
mat.erials
project for
for rapid
rapid substructure
substructure search using
using parallelism.
parallelism.
project

Starting Material
Strategies. The
C.l.a
Material Strategies.
C.1.a SST
SST -- Starting
The importance
importance of
of selecting
selecting good
starting materials
materials for
for a synthesis
synthesis has been known
known for
for a long time,
time, but
but only
only recently
recently has
starting
work
work st.arted
started on applying
applying computer
computer techniques
techniques to
to the
the selection
selection process. The
The selection
selection of
of
st,arting material
material for
for a synthesis
synthesis is frequently
frequent,ly the major
major discovery
discovery in a synthesis
synthesis and the
st.arting
of converting
converting the starting
starting material
material to the target
target is minor
minor by
by comparison.
comparison. Last
Last
process of
year
year we reported
reported development
development of
of the SST program
program for
for selecting
selecting starting
starting materials
materials that
that
are appropriate
appropriate for
for a given
given synthetic
synthetic target
target using a library
library of
of available
available chemicals,
chemicals, but
but
without
without reference
reference to
to reactions.
reactions. SST
SST handles problems
problems of
of classes
classes I-III
I-III given
given below:
below:

I)
I)

Target
Target =
= SM

Identical
Identical match
match
Superstructure match
match
Superstructure
Substructure match
match
Substructure
Similarity
Similarity match
match

II) Target
Target >
> SM
II)
III) Target
Target <
< SM
III)
IV)
IV) None of
of these

For a search over
over our
our abstracted
abstracted file,
file, the identical
identical match
match means that
that the target
target
For
starting materials
materials are identical
identical except
except for
for functionalization.
functionalization. The
The superstructure
superstructure match
match
and sta.rt.ing
t.he case
case where
where we must
must make
make carbon-carbon
carbon-carbon bonds during
during a synthesis.
synthesis. The
The
is the
substracture match
match is the case
case where
where the
the starting
starting material
material is larger
larger than
than the
the target,
target, so
substructure
carbon-ca,rbon bonds have to
to be broken.
broken. Finally,
Finally, the
the similarity
similarity match
match is where
where carboncarboncarbon-carbon
carbon
carbon bonds have to be both
both made
made and
and broken
broken during
during the synthesis.
synthesis.
Our
Our research
research in efficient
efficient starting
starting material
material strategies
strategies has continued
continued this
this past
past year
year in
two
two different
different areas. In the first,
first, we have explored
explored the prospect
prospect of
of llsing
using a.a parallel
parallel
computer in the graph
graph matching
matching process described
described in the following
following section
section and in the
computer
solution to
to the class IV
IV problem
problem (see
(see above) which
which is
second we have developed a solution
described in a subsequent
subsequent section.
section.
described
C.1.c Subgraph
Subgraph Search
Search Using
Using Parallelism.
C.Lc
Parallelism.

Subgraph
Subgraph matching
matching is an important
important method
method llsed
used in many
many different
different computer
computer
applications in organic
organic chemistry.
chemistry, including
including the
the recognition
recognition of
of functional
functional groups,
groups, synthesis
synthesis
applications
planning,
planning, constraint
const.raint testing
testing in structure
structure generation,
generation, selection
selection of
of starting
starting materials
materials for
for
synt,hesis, and structure
structure oriented
oriented retrieval.
retrieval. The
synthesis.
The fundamental
fundamental problem
problem is, given
given a query
query
.d.~trutttwe
(QS) and a5 candidate
candidate
superstructure
(CS), determine
determine if
if there
there exists a
superstructure
(CS),
substructure (QS)
mapping of
of the atoms
stems (nodes) of
of the substructure
substructure onto
onto the candidate
candidate superstructure
superstructure such
mapping
that the connected
connected atom
atom pairs
pairs in the query
query substructure
substructure are also connected
connected in the
that
superst,ructure, and that
that the atom
atom and bond
bond types
types also correspond.
correspond.
superstructure.

Although substructure
substructure search is a non-numerical
non-numerical problem.
problem, it
it is computationally
computationally
Although
demanding
demanding because
because ultimately
ult!imately it
it involves
involves establishing
establishing an atom
atom by
by atom
atom correspondence
correspondence
belween the QS and
a.nd the CS, and this
this problem
problem is a member
member of
of the class of
of NP-complete
between
NP-complete
problems. In a worst
worst case
case for
for N
N atoms
atoms in the QS and M
M atoms
atoms in the CS (M>N).
(M>N),
problems.
one
may have to
t,o consider
consider N!/(M-N)!
N!/(M-N)!
mappings for
for each CS. The
may
mappings
The objective
objective of
of our
our research
wa,s to
t]o explore
explore the feasibility
feasibility of
of applying
applying parallel
parallel processing
processing to this
this problem.
problem.
'was
Although the node ma.tching
ma.tching process is an NP-complete
problem, if
if we eliminate
eliminate all
a.11
Although
NP-complete problem.
backtracking, the order
order of
of the algorithm
algorithm reduces to
t,o O(N),
O(N), where
where N
number of
of
backtracking,
N is the number

Feigenba.um
E. A. Feigenbaum

150

RR00785-11
5P41 RR00785-11

SECS: Simulation
Simulation & Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

atoms
atoms in t.he
the subgraph.
suhgraph. This
This would
would represent
represent a major
major improvement.
improvement. Unfortunately,
Unfortunately, the
algorithm
NP-complete with
a.lgorithm is now NP-complete
with respect
respect to
to sequential
sequential processors.

configuration architecture;
architecture; a central
central processor with
with a
We
We proposed
proposed a "star"
“star” configuration
communication
communication line to
to a number
number of
of lower
lower processors, with
with no direct
direct communication
communication
Each processor
processor has a small
small amount
amount of
of memory
memory as
as a
allowed
a,llowed between
between the lower
lower processors. Each
which avoids
avoids the problems
problems inherent
inherent in shared
shared memory.
memory.
",vorking
space",
The
“working
space”, which
The
communication
communication packets
packets are compact
compact to reduce the storage
storage and communication
communication burden
burden on
the central
cent’ra.1processor.
The simulation
simulation algorithm
algorithm called
called MOLSIM
MOLSIM was implemented
implemented using the SIMULA
SIMULA
The
Isngua.ge. We
We studied
studied this
this algorithm
algorithm as a function
function of
of the number
number of processors and
and the
language.
nature
nature of
of the particular
particular graphs
graphs being
being matched
matched (6-31 non-H
non-H atoms).
atoms). We
We found
found an average
utiliza.tion
utiliza.tion of
of 84%
84% on a ~5
25 cpu machine
machine (figured
(figured as
as total
total processor time/real
time/real time),
time), bnt
but
only
only 50%
SO% on a 50 cpu machine
machine although
although for
for some structure
structure matching
matching questions,
questions, the
The average speed enhancement
enhancement using this
this size machine
machine (50
efficiency
efficiency rearhed
reached 97%.
97%. The
factor of
of 30. A
A real machine
machine of
of the architecture
architecture needed to
to run
run this
this
processors) was a factor
algorithm
algorithm exists
exists at Purdue
Purdue University,
University, and time
time is being requested
requested to test
test the algorithm
algorithm in
real time.
time. This
This algorithm
algorithm is a unique
unique approach
approach to
to the problem
problem of
of graph
graph matching
matching and will
will
likely become practical
pract.ical when
when parallel
pa.rallel processors are commonly
commonly available
available and inexpensive.
inexpensive.
likely
(This
.. )
C.1.b
(This work
work is submitted
submitted for
for publication
publication in J. Chern.
Chem. Inf.
Inf. Comput.
Comput. Sci
Sci..)
Alaximal Common
AICS-- Maximal
Search.
AICS-Common Subgraph
Subgraph Search.
The second area of
of starting
starting material
material strategy
strategy work
work this
this year
year has been in solving
solving the
The
Our solution
solution to
to this
this problem
problem involves
involves development
development of
of a
class IV
IV problem
problem given
given above. Our
efficient maximal
maximal common
common subgraph
subgraph matching
matching algorithm.
algorithm.
chemists represent
represent
new efficient
Since chemists
organic
organic molecules as
as graphs,
graphs, computational
computational chemists
chemists need graph
graph theoretical
t,heoretical techniques
techniques
such as
as graph
graph isomorphism,
isomorphism, subgraph
subgraph search, and maximal
maximal common
common subgraph
subgraph search. Of
Of
these three
three important
import,a.nt procedures.
procedures, maximal
maximal common
common subgraph
subgraph search (MCSS)
(MCSS) remains
remains the
rnost difficult
difficult and least utilized.
utilized.
most
The ext.ensive
ext,ensive comput.a.tional
computational demands
demands of
of MCSS
MCSS has restricted
restricted its
its possible uses.
uses. \Ve
We
The
have previollsly
previously not.ee!
not,ed that.
tha.t maximal
maximal common
common subgraph
subgraph search
sea.rch could
could be useful in our
our
starting mat.erial
ma,terial select.ioll
selection program,
program, SST, but
but that
that the computational
computational demands were too
too
st.arting
rigoro1ls:
Cone et al. who
interpretive
rigorous.*
who has used MCSS
MCSS in their
their "self-training
“self-training
interpretive and
retrieval
retrieval system"
system” (STIRS).
(STIRS), has noted
noted that
that other
other potential
potential uses
uses of
of for
for MCSS
MCSS include
include
comput.er-assisted
computer-assisted organic
organic synt.hesis
synthesis and structure
structure activity
activity studies."
studies.**
Given two
two gra.phs.
gra.phs, finding
finding a common
common subgraph
involves discovering
discovering the assignment
assignment
Given
subgraph involves
of
of some of
of the nodes ane!
and edges
edges of
of one graph
graph onto
onto the other
other graph
graph while
while preserving
preserving the
adjacency
adjacency relationships
relat,ionships of
of the nodes. The
The size of
of the common
common subgraph
subgraph is the number
number of
of
graph of
edges
edges preserved
preserved in the assignment.
assignment. If
If there
t,here exists no common
common sub
subgraph
of larger
larger size, the
common
common subgraph
subgrsph is called
called maximal.
maximal.
Our
Our approach
approach to MCSS
MCSS was to reduce redundant
redundant searching
searching and try
try to
to shrink
shrink the
size of
of the
t,he search space. We
size
We observe that
that most
most libraries
libraries of
of chemicals
chemicals have compounds
compounds
which have similarities;
simila.rities; by
by capitalizing
capitalizing on these similarities
similarities we might
might be able to
t,o reduce
which
the
t,he search space. The
The essentia.l
essentia>lprinciple
principle is that
that if
if we know
know a relationship
relationship between
between library
library
graph
graph A
A and B we can relate
relate query
query graph
graph C to
to A.
A, then
then we may
may therefore
therefore already
already know
know
somet.hing about
about) the
t.he relationship
relationship of
of C to
to B. We
We establish
establish the relationship
relationship between
between A
A and
a.nd
B in a one-time-only
one-time-only preprocessing
preprocessing of
of the library
library. .

•W.T.
Inf. Comput.
Sci.,
lC’.T. Wipke.
Wipke, D. Rogers,
Rogers, I.I. Chern.
Chem. Inf.
Corn&.
Sci., 10S4,
1984, ,(in
,(in press)
press)
•• M.M
MAI Cone.
Cone,

Venkataraghavan,
F.W. Mclafferty,
Melafferty,
I.I. Am.
Chem. Soc.,
Sot., 00,
QQ, 766S,
7688, (1077)
(1977)
R. Venkataraghavan,
F.W.
Am. Chern.

lSI
151

E. A. Feigenbaum
Feigenbaum

SEes:
SECS: Simulation
Simulation &
& Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

5P41
5P41 RR00785-11
RR00785-11

The
The objective
objective of
of our
our study
study was
was to
to demonstrate
demonstrate the
the feasibility
feasibility and
and study
study properties
properties
of
of such
such an
an algorithm.
algorithm. The
The following
following design
design features
features were
were deemed
deemed important.
important.
•l The
The processed
processed library
library should
should be
be storage
storage efficient.
efficient.
•l It
It should
should be
be possible
possible to
to add
add compounds
compounds incrementally
increlnentally to the file, to avoid the

cost
cost of
of reprocessing
reprocessing the
the file
file whenever
whenever changed.
changed.
•l The
The time
time required
required to
to store
store aa new
new compound
compound in
in the
the processed
processed library
library should be
be
minimal;
minimal: preferably,
preferably, an
an upper
upper limit
limit on
on this
this time
time requirement
requirement, should
should be
be known.
known.
•l It
It should be
be possible to create the compound
compound from
from its processed form,
form, so
so that
that

both
both processed
processed and non-processed libraries
libraries do not have to be
be kept.
kept.
\Ve
We wished the system to be
be useful for aa range of non-preselected
non-preselected queries, therefore
therefore
we ruled out
out simply
simply "training"
“training” the system
system for aa small
small class
class of query
query choices or depending
depending
on det.ailed
detailed Knowledge
knowledge of the allowed
allowed queries.
We had noticed
noticed that
that identical
identical common
common subgraph
subgraph candidates
candidates are often
often generated
generated
during
during the search of different
different compounds
compounds against
ag?inst aa query.
query. \Ve
We envisioned
envisioned collecting
collecting these
common
allow the search to be conducted
common segments into
into an intermediate
intermediate graph,
graph, to.
to.allow
conducted once
over
over the common
common featllre.
feature. If
If our
our search procedure
procedure can take
take advantage
advantage of
of the abstraction
abstraction
of the common
common graph
graph from
from two
two or more test compounds,
compounds, then
then the number
number of attempted
attempted
subgraph
suhgraph matches
mat)ches will
will be
be reduced.
reduced.
A
A non-recursive
non-recursive FORTRAN
FORTRAN algorithm
algorithm was implemented
implemented for
for the MCS-1
MCS-1 program
program
using aa, tree-st.ructured
st.orage
file.
The
tree
storage
search
algorithm
reductions
tree-struct,ured storage
The
storage
algorithm gives reductions
from
from 70 to
to 90%
90% in the search relative
relative to conventional
conventional unstructured
unstructured sequential
sequential storage
storage
systems.
Reductions
are
especially
good
for
the
case
where
library
contains smaller
Reductions
especially
for
case
where
the
library
contains
smaller
syst,ems.
graphs
The
of the search tree is
gra.phs or
or a series of
of similar
similar graphs.
graphs.
The general
general structure
structure of
det.ermined
tree are established,
established, addition
addition
det.ernlined early
en.rly in its creation;
creation; once the major
major nodes of the
the tree
of
Sorting
that the tree
tree can
of compounds
compounds rarely
rarely alters
alters it
it, significantly.
significantly.
Sorting experiments
experiments showed
showed that
that improved
improved results
results can be obtained
obtained when
when searching
searching over
over the
the
be "seeded"
“seeded” in such a wa.y that
seeded
library relative
relative to
to the unseeded library.
library.
seeded library
\Ve have a.pplied
applied this
this MCS-1
MCS-1 algorithm
algorithm to
to the Class IV
IV starting
starting material
material
We
recognition problem.
problem. The
The init.ial
initial abstraction
abstraction of
of the starting
material library
library resulted
resulted in an
recognition
starting mat,erial
abstracted library
library of
of significantly
significantly reduced
reduced size. Organizing
Organizing this
this abstracted
abstracted library
library in
in a
abstract.ed
tree-structured form
form allows
allows the
the discovery
discovery of
of starting
starting materials
materials which
which do not,
not have a
tree-structured
subgraph or
or super
super graph
graph relationship
relationship with
with the
the target.
target. A
A trial
trial run
run with
with morphine
morphine was
was
subgraph
successful in point,ing
point.ing out
out an interesting
interesting starting
starting material
material candidate
candidate not
not found
found by
by the
the SST
SST
progra.m. This
This work
work is being
being submitted
submitted to
to the
the J. Chem.
Chem. Inf.
Inr. Comput.
Comput. Sci..
program.

C.2 _YiTiVO
TEND Program
Program Developments
Developments
~7.2
The metabolic
metabolic fate
fate of
of various
various compounds
compounds in
in the
the human
human body
body is extremely
extremely complex,
complex,
The
extremely important
important for
for itit is known
known that
that through
through metabolism
metabolism certain
certain otherwise
otherwise
yet. extremely
harmless compounds
compounds are converted
converted into
into toxic
toxic and possibly
possibly carcinogenic
carcinogenic agents.
agents. Because of
of
harmless
this complexity
complexity itit is difficult,
difficult, looking
looking at
at a given
given compound,
compound, to
to forecast
forecast potential
potential biological
biological
this
activity of
of t,hat
that given
given compound.
compound. The
The objective
objective of
of this
this proposal
proposal is to
to develop
develop a practical
practical
activit,y
computer program
program by
by which
which a biochemist
biochemist or
or metabolism
metabolism expert
expert can
can explore
explore the
the
computer
metabolites of
of a given
given compound
compound and
and be alerted
alerted to
to the
the plausible
plausible biological
biological activity
activity of
of each
metabolites
rnet,abolite.
met,a.bolit,e.
This research
research aims
aims to
to explore
explore the
the degree to
to which
which current
current knowledge
knowledge of
of metabolism
metabolism
This
can be
be used
used by
by a computer
computer program
program to
to make
make reasonable
rea.sonable projections
projections of
of what
what metabolites
metabolites
can

E. A.
A. Feigenbaum
Feigenbaum
E.

152

SPH
SP41 RR00785-11
RR00785-11

SECS:
SECS: Simulat.ion
Simulation 8l
& Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

might
might result
result. from
from exposure
exposure of
of aa compound
compound to
to aa biological
biological system.
system. The
The project
project involves
involves
representing
in
a
computer
the
metabolic
processes
with
all
known
specificities
representing in a computer the metabolic processes with all known specificities and
and
applying
applying these
these processes
processes to
to aa xenobiotic
xenobiotic compound
compound to
to generate
generate aa set
set of
of plausible
plausible
metabolites
metabolites which
which may
may t.hemselves
themselves be
be further
further metabolized.
metabolized. \Ve
We also
also plan
plan an
an evaluation
evaluation
module
module to
to appraise
appraise the
the plausible
plausible biological
biological activities
activities of
of each
each metabolite
metabolite using aa rule base
base
to
Thus
Thus the attention
attention of the
to rebte
relate chemical
chemical structure
structure to
to biological
biological activity.
activity.
experimentalist
experiment,alist will
will be
be attracted
attracted to
to those
those metabolites
metabolites that
that are likely
likely to be
be biologically
biologically
active.
active.

C.2.a Atomic
Atomic Charges.
Chargeu.
During
During the past year, we implemented
implemented rapid
rapid atomic
atomic charge calculation
calculation algorithms
algorithms
to be
be llsed
used wit.h
wit,h the XENO
XENO program
program in order
order to better
better predict
predict the biological
biological activity
activity of
metabolite~.
metnbolites. The algorithms
algorithms were based
based on Gasteiger's
Gasteiger’s PEOE
PEOE (part,ial
(partrial equalization
equalization of
orbital
orbital electronegativity)·
electronegativity)* and SO-POE
SD-POE (sigma
(sigma dependent
dependent pi orbital
orbital electronegativity)··
electronegativity)”
mooels.
models. The PEOE
PEOE model has
has been
been used
used for sigma charge calculations
calculations for sigma
sigma bonded
bonded
and non-conjugated
non-conjugated pi systems. The
The SO-POE
SD-POE model has been used for
for conjugated
conjugated
aliphatic
aliphat,ic and single ring
ring aromatic
a.romatic molecules. For
For polyaromatic
polyaromatic systems, the pi charge
cha.rge
calculations
calculations are being implemented
implemented using the SO-POE
SD-POE model
model and the Longuet-Higgins
Longuet-Higgins
approximat.ion.
In the PEOE
PEOE model, the SO-POE
SD-POE model,
model, and polyaromatic
polyaromatic
approximation.***•••
hydrocarbon
pi charge calculation,
atoms
by
hydrocarbon
calculation,
atoms are characterized
characterized
by their
their orbital
orbital
e1ectronega,ti
vi ties.
electronegativities.
\Ve
We have shown that
that the charges so calculated
calculated are reasonable when
when compared
compared to
the work
Our
work of others
others in the literature.
literature.
Our purpose is then
then to
to correlate
correlate the biological
biological
centers in the
the metabolites.
metabolites.
a,ctivit,y
activit,y of metabolites
meta.bolites with
wit,h the atomic
atomic charge on electrophilic
electrophilic centers
\Ve
extent by
by atomic
atomic charges
We also think
think that
that metabolism
metabolism itself
itself can be controlled
controlled to
to some extent
so the metabolic
metabolic transforms
transforms may
may make
make use
use of this
this data
data eventually.
eventually.

C.2.b ph’
pKa
a Calculations.
Calculations.
\Ve have continued
continued work
work on the problem
problem of
of the
the estimation
estimation of
of the
the dissociation
dissociation
\Ve
constant.s for
for organic
organic acid and bases
bases to be used to
to increase the expertise
expertise of
of the
the XENO
XENO
constant,s
program. \Ve have been investigating
investigating two
two approaches
approaches to
to do this
this type
type of
of estimation:
estimation:
pr0gra.m.
LFER (linear
(linear free energy
energy relations),
relations), and theoretical
theoretical or
or quantum
quantum chemical
chemical approaches.****
approaches .. • ..
LFER
The LFER
LFER computation
computation is performed
performed automatically
automatically by
by first
first selecting
selecting appropriate
appropriate
The
skeleton st#ructures
structures from
from a library,
library, then
then recognizing
recognizing attached
attached groups
groups and
and finally
finally
skelet.on
calculating the
the pKa
pKa from
from the relevant
relevant equations
equations and group
group substituent
substituent constants.
constants. This
This is
ca.lculat,ing
first automatic
automatic pKa
pKa estimation
estimation algorithm
algorithm ever developed
developed and
and promises
promises wide
wide utility
utility on
on
the first
its own
own outside
outside of
of the XENO
XENO program.
program.
its
To determine
determine the
the most
most representative
representative pKa
pKa ifif more
more than
than one acid
acid or
or base center
center is
To
present, several
several empirica.
empirical rules
rules are followed:
followed:
present,

• acids
acids -- use the
the ionized
ionized form
form of
of the
the acid
acid center
center with
with the
the lowest
lowest pKa
pKa as a
substituent for
for the
the pKa
pKa calculation
calculation of
of the
the next
next lowest
lowest acid,
acid, and
and so on.
on .
substituent

GasteigE'r and
and M.
M. Marsili,
Marsili, Tetrahedron
Tetrahedron Iett.,
lett., 34,
84,3181,
(1981)
l•J.J. Gasteiger
3181, (1081)
Gasteiger and
and hl.
11. Ma&i,
Marsili, Tetrahedron,
Tetrahedron, 38,
88, 3219,
3210, (1080)
(1980)
l••*J.J. Gasteiger

•••H,H, C.C. Longuet-Higgins,
Longuet-Higgins, 1.
I. Chem.
Chern. Phys.,
Phy,., f3,
18, 275,
275, (1050)
(1950)

l *.

•••• D.O.

Perrin, B.
B. Dempsey,
Dempsey, and
and E.P.
E.P. Serjeant,
Serjeant, ‘·pKa
Prediction for
for Organic
Organic Acids
Acids and
and Bases.,
Bases·, Chapman
Chapman and
and
D.D. Perrin,
pKa Prediction
Hall, New
New York,
York, 1081.
}081.
Hall,

153

E. A.
A. Feigenbaum
Feigenbaum
E.

SECS: Simulation
Simulation 8[
& Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

5P41
5P41 RR00785-11
RR00785- 11

•l bases
bases - use
use the
protonated form
pKa as a
the protonated
form of
of the
the base center
center with
with the highest
highest pKa
substituent
for
the
pKa
calculation
of
the
next
highest'
base,
and
so
on.
substituent for
pKa calculation of the next highest’base,
•l strong
acids
bases - compute
pKa's
strong
acids and
and bases
compute pKa’
s for
for the acids first,
first, then
then use
use ionized
ionized
form
pKa's
bases.
form of
of the acids as substituents
subst,ituents to
to compute
compute pKa’
s of
of the
the bases.
•l strong
acids
basis - compute
pKa's
bases first,
compute pKa’
s for
for bases
first, then
then use
use
strong
acids and
and weak
weak
basis
protonated form
bases as
pKa's
protonsted
form of
of the bases
as substituents
substituents to
to compute
compute pKa’
s of
of the acids.
•l weak
basis - compute
pKa's
bases first,
compute pKa’
s for
for weak
weak bases
first, then
then use
use
weak acids
acids and
and weak
weak basis
deprot.onated
bases as substituents
pKa's
deprot,onated form
form of
of bases
substituents to compute
compute pKa’
s for
for the acids.
C.2.g Collaborative
Efforts. In
past year
program
Collaborative Efforts.
In the past
year most
most work
work was aimed
aimed at program
development
problems, however
development rather
rather than
than application
application to
to laboratory
laboratory problems,
however in the next
next year
year we
do expect
program modifications
perform several
expect after
aft*er completion
completion of
of the current
current program
modifications to
to perform
practical analyses in conjunction
NIH. and other
practical
conjunction with
with our
our collaborators
collaborators at lCl
ICI of
of UK,
UK, NIH,
other
parties that
parties
that have indicated
indicated interest.
interest.
The
project continues
pharmaceutical
The SECS project
continues to
to have collaborations
collaborations with
with the pharmaceutical
indust,ry
joint program
program development.
indust,ry who
who are adding
adding chemical
chemical transforms
transforms and doing
doing some joint
development,
for
for example.
example, Dr.
Dr. Yanaka
Yanaka continued
continued work
work started
started at Santa
Santa Cruz
Cruz after
after he returned
returned to
to
Kureha
paper has been prepared
prepared on that
Kureha. Chemical
Chemical in Japan
Japan and a paper
that work.
work.

D. List
List of
Project Publications
Publications
D.
of Current
Current Project
1.
Rapid Subgraph
Parallelism
Using Parallelism
Subgraph Search
Search Using
1. Wipke.
Wipke, W.T
W.T.,.. and Rogers, D.:
D.: Rapid

J. Chen.
Chen. Inr.
Inf. Comput.
Comput. Sci (submitted
(submitted 24 April
April 84)

2. Wipke.
\-V.T.: "An
A-Z: A
Wipke, W.T.:
“An Integrated
Integrated System
System for
for Drug
Drug Design"
Design” in Computers
Computers A-Z:

AluTwfacturer's
Hardward and
for the Pharmaceutical
Pharmaceutical
~l~untrfacturer
‘s Guide
Guide to Hardward
and Software
Software for
Industry Aster
Industry
Aster Publishing
Publishing Co., Sprinfield.
Sprinfield, Oregon.
Oregon, (in
(in press)
3. Wipke.
synthetic design.
Symmetry and
and organic
organic synthetic
design.
Wipke, W.T.
W.T. and Huber.
Huber, M.:
M.: Symmetry
Accepted
Accepted in Tetrahedron.
Tetrahedron.
4. Wipke,
Prediction of
Wipke, W.T.,
W.T., Ouchi,
Ouchi, G.I.
G.I. and Chou,
Chou, J.T.:
J.T.: Computer-Assisted
Computer-Assisted
Prediction
of
Afetabolism.
IN
L. Goldberg
(Ed.).
STRUCTURE-ACTIVITY
Goldberg
STRUCTURE-ACTIVITY
&fetnbolism.
IN
(Ed.),
CORRELATIONS
AS A
TOOL
IN
CORRELATIONS
A PREDICTIVE
PREDICTIVE
TOOL
IN TOXICOLOGY.
TOXICOLOGY.
Hemisphere
New York,
Hemisphere Publishing
Publishing Corp
Corp.,.. New
York, 1983.
1983.
5. Johnson.
A.,
Johnson, C.K..
C.K., Thiessen.
Thiessen, W.E.,
W.E., Burnett,
Burnett, M.N.,
M.N., Condran.
Condran, P. Ronlan,
Ronlan, A.,
Yanaka.
\Vipke. W.T.:
procedure.'}
Ysnaka. M. and Wipke,
W.T.: Systematic
Systematic derivation
derivation of
of chemical
chemical procedures
products, J. of
for tra7I·'Jforming
.surplus
for
transforming
surplus hazardous
hazardous chemicals
chemical8 to useful
useful products,
of
Hazardous Materials.
Alaterials. (In
Hazardous
(In press)

o.
Automated Inference
Inference in
Planning Organic
in Planning
Organic Synthesis
Synthesis
6. Dolata.
Do1a.t.a. D.P.:
D.P.: QED:
QED: Automated
(Ph.D.
(Ph.D. dissertation).
dissertation). University
University of
of California,
California, Santa
Santa Cruz,
Cruz, 1984.
1984.
7. Rogers.
Artificial Intelligence
Intelligence in
SST:
SST: Starting
Starting
Rogers, D.: Artificial
in Organic
Organic Chemistry.
Chemistry.
Alaterial Selection
(Ph.
Material
Selection Strategies
Strategies
(Ph. D. dissertation).
dissertation). University
University of
of California.
California,
Santa
Sant,a Cruz.
Cruz. 1984.
Wipke.
Artificial Intelligence
Intelligence in
SST:
in Organic
Organic Synthesis.
Synthesis.
SST:
Wipke, W.T
W.T.,.. and Rogers.
Rogers, D.: Artificial
Starting
Afaterial Selection
An Application
Application of
Search.
Starting Material
Selection Strategies.
Strategies.
An
of Superstructure
Superstructure
Search.
J. Chern.
Chem. lnf.
Inf. Cornput.
Comput. Sci..
Sci., 24:0000.
24:0000, 1984.
1984.

E. Funding
Funding Status
Status

E. A.
A. F'eigenbaurn
F‘eigenbaum

154
154

5P41
5Pfl RR00785-11
RR00785-11

SECS:
SECS: Simulation
Simulation &
& Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

1.
1. Computer-Assisted
Computer-Assisted Prediction
Prediction of
of Xenobiotic
Xenobiotic Metabolism
Metabolism

Principal
Principal Investigator:
Investigator: \V.
W. Todd
Todd Wipke,
Wipke, Professor,
Professor, UCSC
UCSC
Agency:
NIH,
Environmental
Health
Sciences;
Agency: NIH, Environmental Health Sciences; No:
No: ES02845-02
ES02845-02
4/1/82-3/31/85
$257,801
4/1,‘82-3/31/85
$257,801 TDC
TDC
4/1/84-3/31/85
$$ 89,140
4/l/84-3/31/85
89,140 TDC
TDC
2.
2. Graphical
Graphical Display
Display of
of Chemical
Chemical Inferences
Inferences and
and Molecular
Molecular
Relationships
Relationships
Principal
Principal Invest,igator:
Investigator: W. Todd
Todd Wipke,
Wipke, Professor,
Professor, UCSC
Agency: Evans and
a,nd Sutherland
Sutherland Corporation
Corporation
Gift
Gift of PS300
PS300 B/W
B/W High Performance
Performance Graphical
Graphical Display
Display System
Permanent,
Permanent, value $95,000 TDC
TDC
3.
3. Compnter
Computer Synthesis
Principal
Principal Investigator:
Investigator: W.
W. Todd
Todd Wipke,
Wipke, Professor, UCSC
Agency: Stauffer
St,auffer Chemical
Chemical Company
Company
Permanent,
Permanent, $5.000
$6,000 TDC
TDC

F.
I;. Research
Research Env£ronment
Environment
At
At the University
University of California,
California, Santa Cruz,
Cruz, we are connected to the SUMEX-AIM
SUMEX-AIM
resource by
a
4800
baud
multiplexed
leased
line.
Our
by
4800
multiplexed leased
Our video terminals
terminals consist of a Z-29,
‘Z-29.
DM-3025,
1620,
We have a PS300 graphic
DM-3025, TIi45.
TI745, CDI-I030.
CDI-1030, DIABLO
DIABLO
1620, and an ADM-3A.
ADM-3A.
graphic
display
display which
which is driven
driven by SUMEX.
SUMEX. UCSC
UCSC has only
only a small IBM
IBM 370/145,
370/145, a PDP-ll/45,
PDP-11/45,
11/70
jobs for student
11/70 and a V1\..'(
VAX 11/780,
llj780, (the l1's
11’s are restricted
restricted to running
running small jobs
student timetimesharing)
sharing) all of which
which are unsuitable
unsuitable for
for our
our current
current research. The
The SECS laboratory
laboratory is
located
locst#ed in 125
125 Thimann
Thimann Laboratories,
Laboratories, adjacent
adjacent to
to the synthetic
synthetic organic
organic laboratories
laboratories at
at
Santa
Sa.nta Cruz.
Cruz.

n.
II.

INTERACTIONS WITH
WITH THE
THE SUhdEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
INTERACTIONS
A.. Medical
AJedicaJ Collaborations
Collaborations and
and Program
Program Dissemination
Dissemination via
via SUMEX
SUAJEX
A.

available in the GUEST
GUEST area of
of SUMEX
SUMEX for
for casual users, and in the SECS
SECS is available
DE~'fO area for
for serious collaborators
collaborators who
who plan
plan to
to use
use a significant
significant amount
amount of
of time
time and
DEMO
need to
to save the synthesis
synthesis tree generated. Much
Much of
of the
the access
access by
by others
others has been through
through
graphic terminal
terminal equipment
equipment at
at Santa
Santa Cruz,
Cruz, so much
much more
more convenient
convenient for
for structure
structure
the gra.phic
input and output.
output. Demonstrations
Demonstrations and sample synthetic
synthetic analyses were generated for
for
input
numerOllS visitors
visitors from
from t,he
the US and abroad.
abroad. Demonstrations
Demonstrations of
of SECS in Sweden were
numerous
performed by
by Dr.
Dr. R. E. Carter,
Carter, University
University of
of Lund,
Lund, Sweden, at
at many
many universities
universities and
performed
companies.
Professor Wipke
vVipke has also used several SUMEX
SUMEX programs
programs such as CONGEN
CONGEN in
in his
Professor
Computers and Information
Information Processing in Chemistry.
Chemistry. Communication
Communication between
on Computers
course on
collaborators is facilitated
facilitated by
by using SUMEX
SUMEX message
message drops, especially
especially when
when time
time
SECS collaborators
differences between
between the U.S. and Europe
Europe and Australia
Australia makes normal
normal telephone
telephone
differences
communication difficult.
difficult. Testing
Testing and
and collaboration
collaboration on
on the
the XENO
XENO and
and FSECS
FSECS project
project
communication
with researchers
rese8-rchers at
at the
the NC1
NCI depend on
on having
having access
access through
through SUMEX
SUMEX and
and TYMNET.
TYMNET.
with
Collaboration with
with Lund
Lund University.
University. The
The introduction
introduction of
of SECS to
to organic
organic chemists
chemists
Collaboration
in
Sweden
was
one
of
the
seeds
that
led
to
the
establishment
of
a
computer
graphics
in
of the seeds that
to the establishment of
computer graphics
laboratory for
for organic
organic chemistry
chemistry at
at the
the University
University of
of Lund,
Lund, with
with strong
strong support
support from
from aa
laboratory
government agency,
agency, the
the National
National Swedish
Swedish Board
Board for
for Technical
Technical Development
Development (STU).
(STU).
government

155
155

E. A.
A. Feigenbaum
Feigenbaum
E.

SEes:
SECS: Simulation
Simulation &
& Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

5P41
5P41 RR00785-11
RR00785- 11

Interest
Interest in the
the applications
applications of
of computers
computers and computer
computer graphics
graphics in organic
organic chemistry
chemistry has
spread
very
rapidly
throughout
the
country,
and
chemist.s
at
all
of
the
major
spread very rapidly throughout the country,
chemists at all of
major Swedish
Swedish
universities,
universit)ies, as
as well
well as in the pharmaceutical
pharmaceutical industry,
industry, have taken
taken steps to participate
participate in
the exciting
exciting developments
developments in this
this field.
field.
Interest
Interest in the pedagogical
pedagogical value
value of
of SECS at the graduate
graduate level has led to
to its
its use to
to
illustrate
illustrate the concepts
concepts of
of retrosynthetic
retrosynthetic planning
planning and analysis
analysis in conjunction
conjunction with
with a course
"Synthetic
given
“Synthetic Organic
Organic Chemistry--Chemistry--given by
by Prof.
Prof. Paul
Paul Helquist
Helquist (SUNY,
(SUNY, Stony
Stony Brook):
Brook):
Modern
The same course was given
given at the Royal
Royal Institute
Institute of
of
Modern Methods
h4ethods and Strategy
Strategy”.II . The
Technology
Technology in Stockholm,
Stockholm, and at the
the Chemical
Chemical Center
Center in Lund,
Lund, using
using SECS as an integral
integral
part
part of
of the course.
A
A Workshop
Workshop on Computers
Computers in Organic
Organic Chemistry
Chemistry was sponsored
sponsored by
by STU
STU on May
May
17-18,1983, in Gothenburg
Gothenburg to
to help organic
organic chemists
chemists in Sweden enter
enter this
this area of
of work.
work.
Daniel
Daniel P. Dolata
Dolata from
from Prof
Prof Wipke's
Wipke’s group
group in SC was an invited
invited speaker.
speaker.
A
A chemist
chemist from
from Lund,
Lund, Dr.
Dr. Alvin
Alvin Ronlan,
Roman, spent
spent a sabbatical
sabbatical leave with
with the Wipke
Wipke
group.
group, and a graduate
graduate of
of the Wipke
Wipke group,
group, Dr.
Dr. Dolata,
Dolata, is spending
spending a postdoctoral
postdoctoral stay
stay in
Lund.
Lund.
In
In collaboration
collaboration with
with the SECS group
group at UCSC,
UCSC, Dolata
Dolata will
will inst,all
install the SECS
program
11/780 for
for the use of
of the chemists
chemists in
program and the QED
QED program
program on a new VAX
VAX 11/780
Lund.
Lund, and will
will continue
continue research with
with QED.
QED. For
For example,
example, it
it would
would be of
of interest
interest to
develop rule
rule bases
bases to
to assist the
the chemist
chemist in structure
structure elucidation,
elucidation, and structure-activity
snructure-activity
relationships.
re1a.tionships.
Another
Anot,her area of
of collaboration
collaboration involves
involves compilation
compilation of
of chemical
chemical transforms
transforms by
by the
chemists
chemists in Lund.
Lund. Some of
of the chemists
chemists in Lund
Lund work
work with
with natural
natural products
products (isolation
(isolation
and synthesis),
synthesis), with
with a view
view toward
toward the discovery
discovery and characterization
characterization of
of physiologically
physiologically
active
active substances. For
For example
example a strongly
strongly mutagenic
mutagenic compound
compound has been isolated
isolated from
from a
Swedish mushroom
(Lactarius
vellereus),
its
structure
determined,
and
a
total
synthesis
mushroom (Lact,arius vellereus), its structure determined,
total synthesis
elaborated.
other work,
work, a traditional
traditional abortifacient
abortifacient from
from Bangladesh
Bangladesh is being
being isolated
isolated
elaborat.ed. In other
from
from plant
plant material,
material, and a psychoactive
psychoactive substance
substance is being
being extracted
extracted from
from the leaves of
of a
Nigerian plant.
A collaboration
collaboration with
with a university
university in Holland
Holland is now
now developing
developing along
Nigerian
p!ant. A
similar
similar lines, and Cornell
Cornell University
University is planning
planning a similar
similar center
center for
for computer
computer applications
applications
in chemistry.
chemist,ry.

B. Examples
Examples of
with
Projects
R.
o j Cross-fertilization
Cross- fertilization
with other
other SUAfEX-A1Af
SUhfEX-,4Ihf
Projects
The
The AILIST
AILIST bulletin
bulletin board
board has been used extensively
extensively for
for interacting
interacting with
with many
many
projects
projects and locating
locating references
references for
for further
further information
information related
related to
to program
program design and AI
AI
technology.
t.echnology. There
There are no longer
longer any
any other
other chemical
chemical or
or biochemical
biochemical projects
projects on SUMEX
SUkIEX so
our
our interaction
int.eraction with
with the community
community is limited
limited to AI
AI technology
technology interchange,
interchange, attending
attending
seminars
at
Stanford,
etc.
seminars
Stanford,

C. Critique
Resource Services
Clritique of
o j Resource
Services
SUMEX-AIM
SUMES-AIM gives us at UCSC,
UCSC, a small
sma.11university,
university, the advantages
advantages of
of a larger
larger
group
group of
of colleagues, and interaction
interaction with
with scientists
scientist,s all
ail over
over the
the country.
country. Previollsly
Previously we
were provided
provided very
very good service
service by
by SUMEX-AIM,
SUMEX-AIM, but
but since 1 April
April 1984, the
the computer
computer
service has been very
National AIM
very poor.
poor. Although
Although the
the National
AIM usage
usage of
of SUMEX
SUMEX has been small,
small,
our
This is a very
very
our project.
project, has been put
put in a separate
separate class with
with a 3%
3% cpu limitation.
limit,ation. This
severe restriction
restriction which
which prevents
prevents short
short usage
usage peaks from
from being
being averaged
averaged with
with other
other users.
Our
Our project
project is the only
only project
project subjected
subjected to
to such limitations.
limitations. The
The poor
poor response time
time we
are observing
observing (load
(load averages of
of 25-50!) is significantly
significantly hindering
hindering our
our ability
abilit,y to
to perform
perform
the research NIH
NIH funded
funded to be done on SUMEX.
SUMEX. This
This is worsened
worsened by
by the fact
fact we are in
ect.
th
thee Ia.st
la,st year
year of
of the proj
project.
E. A. Feigenbaum
Feigcnbaum

156
156

RR00785-11
5P41 RR00785-11

SECS: Simulation
Simulation 8l
62 Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

D. Collaborations
Atfedical Use of
Programs
D.
Collaborations and
and Medical
o j Fkograms
via
via Computers
Computer8 other
other than
than SUMEX
SUMEX
SECS 2.9 has been installed
installed on the CompuServe
CompuServe computer
computer networks
networks for
for the past
past
three
three years
years so anyone can access
access it
it without
without having
having to convert
convert code for
for their
their machine.
machine.
This
This has proved
proved very
very useful
useful as
as a method
method of
of getting
getting people to
to experiment
experiment with
with this
this new
t.echnology.
t,echnology. Dr.
Dr. George Purvis
Purvis of
of Battelle
Battelle has accessed
accessed SECS via
via CompuServe,
CompuServe, as
as has
SECS also resides on the
Gene Dougherty
Dougherty of
of Rohm
Rohm and Haas and many
many others.
others.
Medicindat
Medicindat machine
machine at the University
University of
of Gothenborg,
Gothenborg, Sweden, and is available
available all over
over
Sweden by
Similarly in Australia,
Australia, SECS resides at the University
University of
of Western
Western
by phone. Similarly
Australia
A
Australia and is available
available throughout
throughout Australia
Australia over
over CSIRONET.
CSIRONET.
A lecture
lecture series was
given
given on SECS in Tokyo
Tokyo and
a.nd SECS has been installed
installed at
at two
two locations
locations in Japan. FSECS
FSECS
has been installed
National Laboratory
installed on a DEC-IO
DEC-10 at
at Oak
Oak Ridge
Ridge National
Laboratory and serves for
for
collaborative
collaborat.ive development
development of
of that
that approach
approach with
with Carroll
Carroll Johnson. PRXBLD
PRXBLD has been
disseminated
disseminated to over
over 30 sites on various
various types
types of
of computers
computers including
including DEC-lO,
DEC-10, DEC-20,
DEC-20,
IBf'.I.
IBRI, VAX,
VAX, PRIt-.m
PRIME and Honeywell.
Honeywell.

III.
RESEARCH
PLANS
(4/84-4/85)
ill.
RESEARCH
PLANS
(4/84-4/85)

A. Near-Term
Near-Term ProJkct
Project Goals
Plans Our
A.
Goals and
and Plans
Our research projects
projects will
will move off
off
SUMEX-AIM
SUMEX-AIM by
by 31
31 March
March 1985
1985 to some other
other as
as yet
yet unspecified
unspecified computer
computer system.
system.
Therefore
Therefore our
our research object,iYes
objectives on SUMEX-AIM
SUMEX-AIM are to complete
complete research in progress,
consolidate
consolidate programs
programs and files for
for moving
moving to
to another
another system.
system. The
The QED
QED and SST
SST projects
projects
have been completed
completed and the first
first phase
phme of
of the RXAN
R,YAN project
project outlined
outlined last year
year has been
completed. On
On the SECS
SEC’S project,
project, the reaction
reaction library
library is being extended
extended by
by Dr.
Dr. Iwataki.
Iwataki.
completed.
\Ve
\Ve will
will continue
continue to collaborate
collaborate with
with coworkers
coworkers in SECS research on other
other machines
machines but
but
on SUMEX
We
SLIMES will
will primarily
primarily be preparing
preparing SECS for
for removal
removal from
from SUMEX.
SUMEX.
We are
exploring
exploring PROLOG
PROLOG as
as a replacement
replacement for
for the QED
QED system
system and plan
plan some preliminary
preliminary
sample PROLOG
PROLOG programs
programs to compare
compare the
the capabilities
capabilities of
of PROLOG
PROLOG and QED.
QED. But
But the
majority
majority of
of our
our activities
activities will
will be aimed
aimed at completion
completion of
of the XENO
XENO program.
program.
XENO Goals
Our
dY&\rO
God3
Our objectives
objectives for
for this
this year
year follow
follow our
our pla.n
plan in the original
original proposal.
proposal.
Basically in this
this next
next year
year we plan
plan to complete
complete the implementation
implementation of
of algorithms
algorithms not
not yet
yet
Basically
completed
completed and focus on testing
testing with
with applications
applications to demonstrate
demonstrate the current
current power
power of
XENO
XENO on typical
typical laboratory
Iaborat.ory metabolism
metabolism problems.
problems. In
In this
this last
last year
year of
of this
this project
project our
our
goal is to bring
bring XENO
XENO to a relatively
relatively stable
stable finished
finished point
point which
which will
will be useful to
to other
other
will complete
complete the
the algorithms
algorithms in progress, document
document them
them and
researchers. \Ve believe we will
submit
submit, publications
publications on all of
of the work
work within
within the year.
year. The
The major
major areas of
of focus are
listed below.
below.
listed
A.1
Charge Calculations.
Calculations.
A.1 ,&ornic
Atomic Charge
We
We also plan
plan to complete
complete our
our correlations
correlations
between
between atomic
atomic charges and sites of
of metabolism,
metabolism, as
as we have already
already done with
with bond
bond
reactivity.
react,ivity. \Vhen
CZ’hen such correlations
correla.tions are established,
established, then
then they
they can be used to
to guide XENO
XENO
to
to apply
apply metabolic
metabolic transforms
transforms more
more selectively
selectively to
to the most
most active
active parts
parts of the molecules.

A.2 pKa
pJ(a Calculations.
A.2
Calculatione. We
We plan
plan to complete
complete testing
testing of
of the pKa
pKa algorithm
algorithm on
different
different. groups
groups on metabolites
metsbolites so that
that information
information may
may be used by
by XENO
XENO for
for activity
activity
evaluation,
evaluation, select,ion
selection of
of further
further possible metabolism,
metabolism, and estimation
estimat#ion of
of excretion
excretion and
t.ra.nsport.
transport.
A.S Three-Dimensional
Criteria.
We
-4.3
Three-Dimensional
Criteria.
We plan
plan to
to complete
complete our
our work
work on threethreedimensional
dimensional constraints
constraints that
tha.t apply
apply to metabolism
metabolism which
which have been obtained
obtained through
through
study
This
study of many
many metabolic
metabolic studies
studies in the literature.
literature.
This will
will require
require extensions
extensions of
of the

157

Feigenbaum
E. A. Feigenbaum

SECS: Simulation
Simulation 86
& Evaluation
Evaluation of
of Chemical
Chemical Synthesis
Synthesis

5P41 RR00785-11
RR00785-11

ALCHEM
.4LCHEM language t,o
to accommodate
accommodate new types
types of
of three-dimensional
three-dimensional relationships
relationships such as
as
overall
overall molecular
molecular size, width,
width, thickness,
thickness, ratio
ratio of
of length
length to
to width,
width, etc
etc..

.1.4
-4.4 Log P Calculation.
Calculation. In
In order
order to
to more
more accurately
accurately estimate
estimate the possibility
possibility of
of
excretion
excret,ion and
a.nd binding
binding for
for met.abolism,
met,abolism, we plan
plan to
to incorporate
incorporate a log P calculation
calculation module.
module.
This
This will
will provide
provide the partition
partition coefficient
coefficient between
between octanol
octanol and water.
water. There
There are already
already
programs
Our
programs to
to calculate
calculate log P, and these have been shown
shown to
to be very
very accurate.
accurate.
our
objective,
objective, time
time permitting,
permitting, will
will be to
to include
include such a module
module in the XENO
XENO program
program and
correlate
correlate log P with
with metabolism
metabolism transform
transform application
application and with
with excretion.
excretion.
B. Justification
Justification and
Requirements for
for Continued
B.
and Requirements
Continued Use of
of SUAIEX
SUMEX
The
The XENO
XENO project
project which
which resides on SUMEX
SUMEX is in its
its last year
year of
of support,
support,
consequently
consequently we need to complete
complete that
that research on the SUMEX
SUhlEX machine.
machine. By
By 31 March
March
1985,
plan to move XENO
1985, we
we plan
XENO and all our
our research off
off SUMEX
SUMEX onto
onto some other
other computer.
computer.
\Ve are currently
currently exploring
exploring what
what machine
machine may
may be suitable
suitable and available.
available. After
After 1 April
April
198,5
1085 we will
will not
not need SUMEX
SUMEX for
for computational
computational support,
support, but
but will
will need access
a.ccessto be able
to
respond to
to electronic
electronic mail,
mail, and continue
continue to
to
t,o retrieve
retrieve certain
cert.ain files from
from archive,
archive, respond
participate
participate in the AIM
AIM scientific
scient,ific interchange
interchange through
through electronic
electronic mail
mail and bulletin
bulletin boards.
boards.
It.
practical to
It, is not
not, pract,ical
to retrieve
retrieve every
every file
file we have ever archived,
archived, it
it would
would use
use too
too much
much
SUMEX
SUMEX operator
operator time,
time, and it
it is unnecessary
unnecessary as
as long as
as we can access
access them
them if
if we need
them
them in the future.
future. That
That access
access would
would not
not require
require significant
significant resources.
prior to 31 March
However
1985 we have obligations
obligations to
to complete
complete the research on
However prior
March 1985
t,he
NIH and need sufficient
the XENO
SENO project
project supported
supported by
by NIH
sufficient SUMEX
SUMEX cpu
CPU time
time to
to accomplish
accomplish
t.his
this goal. This
This means normal
normal editing,
editing, compile,
compile, load, and test
test executions
executions plus
plus some
runs
to
some
metabolic
problems.
It
appears
the
current
removal
applicat.ion
application runs to
metabolic problems. It
of our
our
current removal of
project from
National AIM
project,
from the
t,he National
AIM portion
portion of
of the
the SUMEX-AIM
SUMEX-AIM resource and placement
pla.cement in a
of
class restricted
restricted to 3%
3% peak utilization
utilization is hindering
hindering the research productivity
productivity
of this
this
project.. \Ve are experiencing
project,.
experiencing load averages of
of 25-50 a high
high percentage
percentage of
of the
the time.
time. We
We
request
National AIM
request to
to have om
our project
project placed back
back in the National
AIM portion
portion of
of the SUMEX-AIM
SUMEX-AIM
resource as
as we were allocated,
allocated, and we will
will carefully
carefully monitor
monitor to
to see
see that
that our
our resource
utilization
utQlizat.ion does
does not
not exceed our
our quota
quota of
of time.
time. We
We feel this
this is a reasonable request
request in light
light
of
National community
of the mission
mission of
of SUMEX-AIM
SUMEX-AIM to the National
community of
of which
which this
this project
project is a part.
part.
C. Needs
Needs Beyond
Beyond SUA-lEX-AIM
SUMEX--,;1IM As mentioned
mentioned above our
our project
project needs additional
additional
computing
computing resources and we are exploring
exploring acquiring
acquiring a computer
computer for
for installation
installation at
at UCSC
UCSC
and obt.aining
obt.aining the necessary resources to
to support
support it.
it. \Ve
We are seeking information
information about
about
comparisons
comparisons between
between machines
machines and cost effectiveness
effectiveness of
of different
different hardware
hardware combinations.
combinations.

D.
Recommendations for
for Community
and
Resource Development
Development
L?. Reconmendations
Community
and Resource
It
It, appears the
the’ SUMEX-AIM
SUMEX-AIM resource is increasingly
increasingly becoming
becoming basically
basically a Stanford
Stanford
reSOllrce
resource and
a.nd that
t,hst. there
there is a difference
difference between
between the
the portion
portion of
of the resource allocated
allocated to the
National community
National community.
Our
Nationa.
communit,y and the portion
portion actually
actually used by
by the National
community.
Our
project
National community
project, is part
pa.rt of
of the National
community and in need of
of better
better service,
service, we hope that
that can
be improved.
improved.

An
An important
important part
part of
of medicine
medicine is treatment
treatment of
of disea:-es
diseases with
with drugs,
drugs, chemicals,
chemicals,
chemicals
chemicals that
t,hat were designed and synthesized
synthesized by
by chemists.
chemists. Since the termination
termination of
of the
DENDRAL
project, there
seems to be declining
declining support
support for
for artificial
artificial intelligence
intelligence
DENDRAL
project.
there seems
applications
We
applications in chemistry.
chemistry.
We feel that
that support
support of
of this
this area is essential to the
advancement
in this
The
NIH Research
advancement of
of medicine
medicine
this country.
country.
The lack
lack of
of chemists
chemists on NIH
Resources computing
computing peer review
review 'is
.is contributing
contributing to the problem.
problem. In general
general the AIM
AIM
communit.y
community would
would benefit
benefit by
by involving
involving disciplines
disciplines other
other than
than computer
computer science.

E. A. Feigenbaum
Feigenbaum

158

SOLVER
SOLVER Project
Project

5P41
5P41 RR00785-11

II.A.2.5.
SOLVER
Project
II.A.2.5.
SOLVER
Project

SOLVER:
Problem
Solving
Expertise
SOLVER:
Problem
Solving
Expertise
Dr.
Dr. P.
P. E.
E. Johnson
Johnson
Center
for
in
Learning
Center
for Research
Research
in Human
Human
Learning
University
of
University
of Minnesota
Minnesota
Dr.
Dr. W.
W. B.
B. Thompson
Thompson
Department
of
Science
Department
of Computer
Computer
Science
University
of
University
of Minnesota
Minnesota

I.
I.

SUMMARY
OF
PROGRAM
SUMMARY
OF RESEARCH
RESEARCH
PROGRAM
A.
A. Project
Project Rationale
Rationale

This
This project
project focuses
focuses upon the development
development of strategies
strategies for
for discovering
discovering and
documenting
documenting the
t,he knowledge
knowledge and skill
skill of
of expert
expert problem
problem solvers. In the last
last fifteen
fifteen years,
considerable
considerable progress has been made in synthesizing
synthesizing the expertise
expertise required
required for
for solving
solving
extremely
Computer programs
programs exist
exist with
with competency
competency comparable
comparable to
ext’remely complex
complex problems.
problems. Computer
human
human experts
exprrt,s in diverse areas ranging
ranging from
from the analysis
analysis of mass spectrograms
spectrograms and
nuclear
nuc!ear magnetic
rnagnet,ic resonance (Dendral)
(Dendral) to the diagnosis of
of certain
certain infectious
infectious diseases
diseases
(Mycin).
(Mycin).
Design of
particular task
of an expert
expert system
system for
for a particular
task domain
domain usually
usually involves
involves the
interaction
of
two
distinct
groups
of
individuals,
"knowledge
engineers,"
interaction of two distinct groups of individuals, “knowledge engineers,” who
who are
primarily concerned
problem solving
primarily
concerned with
with the specification
specification and implementation
implementation of
of formal
formal problem
solving
t,echniques,
problem area) who
provide factual
(in the relevant
relevant problem
who provide
factual and
techniques, and "experts"
“experts” (in
heuristic
problem solving
hcurist,ic information
information of
of use
use for
for the problem
solving task
t,ask under
under consideration.
consideration. Typically
Typically
the knowledge
particular
knowledge engineer
engineer consults
consults with
with one or
or more
more experts
experts and decides on a particular
represent3tional
Next, "units"
“units” of
of factual
factual information
information
representational structure
structure and inference
inference strategy.
strategy. Next,
are specified.
properties of
problem domain
specified. That
That is, properties
of the problem
domain are decomposed into
into a set of
of
manageable
processing by
by the inference
Once this
this
msnagesble elements
elements suitable
suitable for
for processing
inference operations.
operations.
organization ha.s
has been established,
established, major
major efforts
efforts are required
required to
to refine
refine representations
representations and
organization
acquire fact.ual
factual knowledge
knowledge organized
organized in an appropriate
appropriate form.
form.
Substantial research
research
a,cquire
Substantial
prohlems exist
exist in developing
developing more
more effective
effective representations,
representations, improving
improving the
the inference
inference
problems
process, and in finding
finding better
better means of
of acquiring
acquiring information
information from
from either
either experts
experts or
or the
the
problem area itself.
itself.
problem
Programs currently
currently exist
exist for
for empirical
empirical investigation
investigation of
of some of
of these questions
questions for
for a
Programs
particular problem
problem domain
domain (e.g. AGE,
AGE, UNITS,
UNITS, RLL).
RLL). These tools
tools allow
allow the investigation
investigation
particular
of a.lt,ernnt,e
alternate organizations,
organizations, inference
inference strategies,
strategies, and rule
rule bases
bases in an efficient
efficient manner.
manner.
of
\Vhat is still
still lacking,
lacking, however,
however, is a theoretical
theoretical framework
framework capable of
of reducing
reducing dependence
\Vhst
the expert’
expert's
intuition or
or on near
near exhaustive
exhaustive testing
testing of
of possible organizations.
organizations. Despite
Despite
on the
s intuition
their successes,
successes, there
there seems to
to be a consensus that
that expert
expert systems
systems could
could be better
better than
than
their
they are.
Most expert
expert systems
systems embody
embody only
only the
the limited
limited amount
amount of
of expertise
expertise that
that
Most
they
individuals are able to
to report
report in
in a particular,
particular, constrained
constrained language (e.g. production
production rules).
rules).
individuals
current systems
systems are approximately
approximately as good
good as human
human experts,
experts, given
given that
that they
they represent
represent
IfIf current
only
a
port,ion
of
what
individual
human
experts
know,
then
improvement
in the
the
only
portsion of what individual human experts know, then improvement in
"knowledge capturing”
capturing" process should
should lead to
to systems
systems with
with considerably
considerably better
better
“knowledge
performance.
performance.

159

A. Feigenbaum
Feigenbaum
E. A.

RR00785-11
5P41 RR00785-11

SOLVER
SOLVER Project
Project

and Collaboration
Collaboration
B. Medical
AIedical Relevance
Relevance and
Collaboration
Collaboration with
with Dr.
Dr. James Moller
Moller MD
MD in the Department
Department of
of Pediatrics,
Pediatrics, Dr.
Dr.
Donald
Donald Connelly
Connelly MD
MD in the Department
Department of
of Laboratory
Laboratory Medicine,
Medicine, at the University
University of
of
r-.finnesota.
hfinnesota. Collaboration
Collaborat,ion with
with Dr.
Dr. Eugene Rich
Rich MD
MD and Dr.
Dr. Terry
Terry Crowson
Crowson MD
MD at St.
Paul
Paul Ramsey
Ramsey Medical
Medical Center.
Center.

of Research
C. Highlights
Highlights of
Research Progress
Progress
Prior research at Minnesota
Minnesota on expertise
expertise in
Accomplishments
Accomplishments of
of This
This Past
Past Year
Year -- Prior
diagnosis
diagnosis of
of congenital
congenital heart
heart disease
disease has resulted
resulted in a theory
theory of
of diagnosis
diagnosis and an
embodiment
embodiment of
of that
that theory
theory in the form
form of
of a computer
computer simulation
simulation model,
model, Galen,
Galen, which
which
diagnoses cases
cases of
of congenital
congenital heart
heart disease
disease [Thompson,
[Thompson, Johnson & Moen,
Moen, 1983].
19831.
Galen
Galen is descended from
from two
two earlier
earlier programs
programs written
written here at
at Minnesota:
Minnesota:
and
Deducer [Swanson,
a.nd Dedrrcer
(Swanson, 1977].
19771. Deducer
Deducer is a program
program that
that builds
builds hemodynamic
hemodynamic
models of
of the circulatory
circulatory system
system that
that describe specific
specific diseases.
diseases. The
The models are built
built, by
by
using
usirig knowledge
knowledge about
about how
how idealized
idealized parts
parts of
of the
the circulatory
circulatory system
system are causally
causally related.
related.
Diagnoser
program
Dia.gnoser is a recognition-driven
recognition-driven
program that
that performs
performs diagnoses by
by successively
successively
hypothesizing
hypothesizing one or
or more
more of
of these models and matching
matching them
them against
against patient
patient data.
data. The
The
models that
tha,t match
match best are used as the final
final diagnosis.
diagnosis. A
A series of
of experiments
experiments carried
carried
out at Minnesota
Minnesota have shown
shown that
that Diagnoser/Deducer
Diagnoser/Deducer performs
performs as well
well (and
(and sometimes
sometimes
out
better)
better) than
than expert
expert human
human cardiologists
cardiologists [Johnson
[Johnson et aI.,
al., 1981].
19811.
Dial]lloser
Diagnouer

Despite their
their early
early successes,
successes, Diagnoser
Diagnoser and Deducer
Deducer did
did not
not have a clear,
clear,
Despite
comprehensible
comprehensible structure
structure that
that is required
required for
for the
the kind
kind of
of experiments
experiments we wish
wish to
t,o perform.
perform.
Galen
Galen was built
built to remedy
remedy this
this problem,
problem, taking
taking advantage
advantage of
of the experience
experience gained
gained in the
of Diagnoser
Disgnoser and Deducer.
Deducer.
design of
Galen
Galen consists of
of four
four major
major components:
components: a working
working memory
memory called
called the scratchpad,
scratchpad,
a knowledge
proposer and a
knowledge base
base of
of rules
rules and hypotheses,
hypotheses, a procedure
procedure called
called the proposer
procedure called
called the reviewer.
reviewer.
procedure
The
The scratchpad
scratchpad contains
contains data
data about
about the problem
problem that
that Galen
Galen is trying
trying to
to solve and
the hypotheses
hypotheses that
that are being
being investigated
investigated to
to explain
explain that
that data.
data. In
In effect,
effect, the scratchpad
scratchpad
represents
represents Galen's
Galen’s current
current execution
execution state.
state.
Rules are pattern-action
pattern-action pairs.
pairs. The
The pattern
pattern part
part of
of a rule
rule describes a possible state
state
of the
t,he scratchpad.
scratchpad. Patterns
Patterns can contain
contain imbedded
imbedded logical
logical connectives
connectives (e.g. ANDs,
ANDs, ORs,
of
NOTs) and can be constructed
NOTsj
constructed to match
match at
at varying
varying levels of
of detail.
detail. The
The action
action part
part is a
procedure
procedure that
that is executed
executed if
if the pattern
pattern part
part matches
matches the scratchpad's
scratchpad’s contents.
contents. Each
Each
action
nct,ion part
part writes
writ,es an assertion
assertion on the
the scratchpad
scratchpad about
about a hypothesis,
hypothesis, together
together with
with the
evidence for
for making
making that
that assertion.
assertion. These assertions
assertions can express that
that a new hypothesis
hypothesis is
being considered,
considered, or
or that
that an old
old hypothesis
hypothesis has been accepted, rejected,
rejected, confirmed
confirmed or
or
disconfirmed.
Action
disconfirmed.
Action parts
parts can also assert that
that a hypothesis
hypothesis is sufficient
sufficient to
to solve the
the
current. problem,
problem, or
or that
that the problem
problem is not
not solvable.
solvable.
current.
Because the pattern
pattern parts
parts of
01 rules
rules can examine
examine anything
anything on the scratchpad,
scratchpad, it
it is
possible to express rules
rules about
about hypotheses
hypotheses as well
well as
as rules
rules about
about problem
problem data.
data. In
pa.rticular, this
this makes it
it possible to
to directly
directly examine
examine the accumulated
accumulated evidence for
for and
particular,
against
against each currently
currently contending
contending hypothesis,
hypothesis, making
making numerical
numerical measures of
of certainty
certainty
unnecessary.
unnecessary.

A hypothesis
hypothesis is simply
simply a named
named collection
collection of
of rules. The
The hypotheses
hypotheses in Galen's
Galen’s
A
knowledge
knowledge base
base can be thought
thought of
of as a directed
directed graph,
graph, in which
which vertices
vertices are hypotheses
hypotheses
and
snd edges
edges are rules. One hypothesis
hypothesis IIpoints
“points toll
to” another
another if
if the first
first hypothesis
hypothesis contains
contains a
rule
rule whose action
action part
part can assert something
something about
about the second.
E. A. Feigenbaum
Feigenbaum

160

SOLVER
SOLVER Project
Project

5P41
5P41 RR00785-11

The
The level of detail
detail of such aa knowledge
knowledge base
base leads
leads to serious problems
problems with
with the
computational
focuses its computational
computational resources
compllt,st.iona! complexit.y
comp1exit.y of search processes.
processes. Galen focuses
so
so that
that the knowledge
knowledge embodied
embodied in the graph
graph of hypotheses can be
be used
used in an efficient
efficient
manner.
manner. Successful diagnoses result
result from
from good first
first hypotheses about
about possible defects and
efficient
efficient mechanisms
mechanisms for refining
refining these hypotheses.
Galen
Gslen works
works by using the proposer
proposer and the reviewer
reviewer to investigate
investigate hypotheses
hypotheses (Le.
(i.e.
search the graph)
graph) by applying
applying rules (Le.
(i.e. following
following the edges
edges from
from one vertex
vertex to another).
another).
vVhenever
Whenever a new piece of problem
problem data
data is
is written
written on the scratchpad,
scratchpad, the proposer
proposer applies
all
a!! relevant
relevant rules specific
specific to the type
type of that
that piece of data.
data. These rules write
write assertions
assertions on
the scratchpad
scra.tchpa,d about
about new hypotheses, effectively
effectively identifying
identifying vertices
vertices in the graph
graph that
that are
worthy
Next, the reviewer
reviewer applies all relevant
relevant rules
worthy starting
starting points
point,s for further
further search. Next,
contained
contained in the hypotheses
hypotheses that
that are named in assertions on the scratchpad.
scratchpad. Successfully
Successfully
applying
applying one of these rules corresponds
corresponds to propagating
propagating the search along a specific
specific edge
edge of
the graph.
graph. The
The search is
is constrained
constrained because
because (1) only
only the most promising
promising vertices
vertices in the
graph
graph are ever used
used to initiate
initiate search; (2) only
only a small
small number
number of
of edges
edges are ever followed;
followed;
a.nd
and (:3)
(3) most
most rules in a hypothesis
hypothesis deal with
with evidence for
for and against
against the hypothesiS
hypothesis itself,
itself,
giving
giving a graph
gra.ph where
where the number
number of
of effectively
effectively outward-pointing
outward-pointing edges
edges at each vertex
vertex is
small.
small.
The
The read-propose-review
rea,d-propose-review cycle repeats
repeats in this
this way
way until
until some hypothesis
hypothesis has been
shown
shown correct,
correct, until
unt,il the
t,he problem
problem has been shown
shown unsolvable,
unsolvable, or
or until
until all the data
data has been
examined.
examined.
Currently,
Currently, data
data given
given to Galen
Galen is taken
taken from
from a (possibly
(possibly imaginary)
imaginary) patient's
patient’s
medical
Hypotheses
medical chart.
chart.
Hypotheses in the knowledge
knowledge base
base represent
represent the ten most
most commonly
commonly
occurring
physiological
occurring congenital
congenital heart
heart diseases
diseases and their
their variants,
variants, useful intermediate
intermediate physiological
findings,
findings, and classes
classes of
of hypotheses.
hypotheses. Since hypotheses
hypotheses are implemented
implemented as named
named teams
teams of
of
production rules, it
product,ion
it is also possible to
to represent
represent other
other kinds
kinds of
of hypotheses
hypotheses should
should the need
arise.
Moreover,
arise.
IMoreover, Ga.len
Ga,!en has been constructed
constructed so that
that its
its inference
inference engine does not
not contain
contain
any procedures
procedures specialized
pediatric cardiology.
specialized for
for pediatric
cardiology. It
It is therefore
therefore conceivable
conceivable to extend
extend
Galen
Gnlen to other
ot,her domains
domains if
if effective
effective knowledge
knowledge bases
bases for
for those domains
domains can be constructed.
constructed.
To
To determine
determine the generality
generality of
of our
our model
model of
of expertise
expertise in diagnostic
diagnostic reasoning,
reasoning, we
are
part of
a.re also investigating
investigating domains
domains outside
outside medicine.
medicine. As part
of this
t,his effort,
effort, we have developed
developed
a computational
program debugging
computational model of
of the fault
fault localization
localization process in program
debugging [Sedlmeyer,
[Sedlmeyer,
198:3]
with our
our work
work in congenital
congenital heart
heart disease,
disease,
19831 that
that is not
not, based directly
directly on Galen.
Galen. As with
concentrated on the design of
of mechanisms
mechanisms for
for structuring
structuring problem
problem specific
specific
we have concentrated
knowlt'dge a,nd
and for
for focusing
focusing limited
limited computational
computational resources.
knowledge
Progress -- Since human
human experts
experts are notoriously
notoriously poor
poor at
at describing
describing
Resra.rch in Progress
Rrsca.rch
their own
own knowledge,
knowledge, our
our work
work requires
requires the
the creation
creation of
of problem
problem solving
solving tasks
tasks through
through
t,heir
which expert.s
experts can reveal
reveal crit.eria
criteria for
for initiating
initiating specific
specific hypotheses
hypotheses and methods
methods for
for
which
those hypot.hcses.
hypotheses.
invest iga ting t,hose
investigating
Current techniques
t.echniques of
of representing
representing hypotheses
hypotheses and their
their expectations
expectations for
for diagnosis
diagnosis
Current
do not,
not, however,
however, provide
provide much
much detailed
detailed information
information about
about the
the control
control processes expert,s
experts
do
use to
to guide
guide their
t.heir reasoning.
reasoning. Such control
control processes typically
typically incorporate
incorporate highly
highly refined
refined
use
heuristics about
about. which
which the
the experts
experts are almost
almost wholly
wholly unaware.
unaware. To
To discover
discover the
the needed
heuristics
cont.rol knowledge,
knowledge, we ask experts
experts to
to complete
complete tasks
tasks in which
which we have systematically
systematically
cont,rol
perturbt~d aspects of
of the
the problem
problem data.
data. The
The data
data in
in these tasks
tasks are chosen so that
that
perturbed
members
of
an
overlapping
set
of
hypotheses
will
be
suggested
during
while
solving
the
members of
overlapping
of hypotheses will
during while solving the
problem.
Success
in
solving
such
problems
depends
on
the
ability
to
overcome
an
initially
problem. Success in solving
problems
on the ability to overcome
initially
plallsihle incorrect
incorrect hypot,hesis
hypothesis in favor
favor of
of a later,
later, more
more correct
correct alternative.
alternative.
plausible

161
161

A. Feigenbaum
Feigenbaum
E. A.

5P41 RR00785-11
RR00785-11

SOLVER
SOLVER Project
Project

Several
Several examples
examples illustrate
illustrate our
our approach.
approach. \Ve
We are studying
studying performance
performance of
of Galen
Galen
cases [Johnson
[Johnson &
& Thompson,
Thompson, 1981]
19811 that
that were initially
initially misdiagnosed
misdiagnosed in
on "garden
“garden path"
path” cases
hospit.al
hospit,al files. Analysis
Analysis of
of such cases
cases suggest that
that errors
errors are made because
because experts
experts rely
rely on
very efficient
efficient heuristics
heuristics that
that are not
not universally
universally correct.
correct. In
In one such example,
example, a seemingly
seemingly
very
plausible
plausible hypothesis
hypothesis is suggested early
early in the case.
case. Although
Although the hypothesis
hypothesis superficially
superficially
seems
seems to
t,o explain
explain what
what is observed
observed about
about the patient,
patient, the hypothesis
hypothesis is incorrect.
incorrect. Because
the incorrect
incorrect hypothesis
hypothesis seems
seems marginally
marginally adequate, it
it acts to prevent
prevent a more correct
correct
Success in such a case
case hinges on the ability
ability
hypothesis
hypothesis from
from being
being suggested in
in its
its place.
place. Success
to use
use the proper
proper set of
of competing
competing hypotheses
hypotheses in order
order to provide
provide more than
than one
to
Investigation of
of this
this phenomenon
phenomenon in human
human experts
experts has
explanation
explanation of
of the case
case data.
data. Investigation
linking disease
disease hypotheses
hypotheses in Galen.
Galen. It
It has
suggest.ed
suggested implementation
implementation of
of "transition
“transition rules"
rules“ linking
that watch
watch for
for potential
potential garden
garden
also suggested implementation
implementation of
of "monitor
“monitor hypotheses"
hypotheses” that
pat.h
path errors
errors and avoid
avoid them
them before
before they
they become serious.

We are also investigating
investigating several research questions
questions relevant
relevant to
to t.he
the architecture
architecture of
of
We
Galen.
Galen. \Ve
We have designed an interface
interface to
to Galen
Galen so that
that users who
who are unfamiliar
unfamiliar with
with the
inner workings
workings of
of the program
program can interactively
interactively enter
enter case
case data.
data. Designing
Designing the interface
interface
inner
questions about
about what
what forms
forms of
of data
data are necessary to adequately
adequately and completely
completely
raised questions
We are also studying
studying ways in which
which a causal reasoning
reasoning
represent
represent all possible cases.
cases. We
component.
Proposer
component, can be integrated
integrated with
with the prototypical
prototypical reasoning
reasoning components
components (the
(the Proposer
and Reviewer)
Reviewer) that
that are already
already present
present in Galen.
Galen. In
In particular,
particular, we are interested
interested in
st.udying
studying ways in which
which causal reasoning
reasoning can aid or
or replace prototypic
prototypic reasoning
reasoning when
when it
it
becomes inadequate
inadequate to reach a diagnosis.
diagnosis.
become~
In another
another project,
project, we are investigating
investigating methods
methods of
of probabilistic
probabilistic reasoning.
reasoning. Most
Most
sy:::tems
systems rely
rely on numerical
numerical schemes for
for weighting
weighting evidence or
or ranking
ranking observed
observed data.
data.
weights are often
often probabilistic
probabilistic in nature,
nature, but
but other
other schemes have also been used.
These weights
Myrin.
Mycin, for
for example.
example, uses
uses certainty
certainty factors
factors and PIP
PIP uses
uses likelihoods
likelihoods composed of
of matching
matching
scores and binding
binding scores. In
In contrast,
contrast, humans
humans do not
not seem to rely
rely upon
upon such numerical
numerical
techniques.
t,echniques. Research
Resea.rch has shown
shown that
that people are often
often quite
quite poor
poor at probabilistic
probabilistic
reasoning.
However,
reasoning.
However, experts
experts make
make decisions which
which involve
involve weighting
weighting evidence and
selecting
selecting from
from competing
compet.ing alternatives.
alternatives. They
They must
must utilize
utilize a reasoning
reasoning process which
which serves
as
as an alternative
alternative to a numerical
numerical weighting
weighting technique.
technique.

We believe the process of
of weighting
weighting alternatives
alternatives along various
various criterial
criteria] dimensions
dimensions
\Ve
is not
not a domain
domain specific
specific technique,
technique, but
but rather
rather a general process which
which is applied
applied in specific
specific
t,he coming
coming year,
year, we wiJI
will examine
examine this
t,his process in various
various domains
domains and
instances.
In the
instsnccs.
a.ttempt
a.tt,empt to ut.ilize the result·s
result,s in designing
designing more
more powerful
powerful reasoning
reasoning techniques.
techniques.
In
In the area
a.rea of
of law,
law, our
our work
work has focused
focused on the area of
of corporate
corporate law
law (the
(the problem
problem
of
a proposed
We
of structuring
structuring
proposed corporate
corporate acquisition).
acquisition).
We have collected
collected data
data from
from ~4
24
pra.ct.icing law.\"ers
lawyers and in the coming
coming year
year a PhD
PhD thesis will
will be completed
completed describing
describing this
this
practicing
work.
work. In the coming
coming year
year we will
will be also completing
completing a study
study of
of the corporate
corporate acquisition
acquisition
problem
problem in man~1!;ement
man:tgement in order
order to
to further
further refine
refine our
our knowledge
knowledge capturing
capturing tools.
tools.

D. List
Li.'3t of
Relevant Publications
Publications
D.
of Relevant

1. Connelly,
Connelly, D. and Johnson, P.E.:
P.E.: Medical
solving. Human
Human Pathology,
Pathology,
1.
fl.;fedical problem
problem solving.
11(5):41?-419,
11(5):41~-419.

1980.
1980.

2. Elstein,
Proposed Reaeatch
Research
Elstein, A.,
A., Gorry,
Gorry, A.,
A., Johnson, P. and Kassirer,
Kassirer, J.: Proposed
Efforts. IN
Efforts.
IN D.C.
D.C. Connelly,
Connelly, E. Benson and D. Burke
Burke (Eds.),
(Eds.), CLINICAL
CLINICAL
DECISION MAKING
MAKING AND
AND LABORATORY
LABORATORY
University of
of Minnesota
Minnesota
DECISION
USE. University
Press.
Press, 1982,
1982, pp. 327-334.

E. A. Feigenbaum
Feigcnbaum

162
152

SOLVER
SOLVER Project
Project

5P41 RR00785-11
RR00785-11

P.J.:
Knowledge based components
P.J.: Knowledge
components
diagnosis.
Learning
diagnosis. Learning Research and Development
Development
PDS-:!,
PDS-2, University
University of
of Pittsburgh,
Pittsburgh, September,
September, 1981.
1981.

3. Feltovich,
Feltovich,

of
in
of expertise
expertise
in
Center
Technical
Center Technical

medical
medical
Report
Report

Role and
4. Feltovich,
Feltovich, P.,I.,
P.J., Johnson,
Johnson, P.E.,
P.E., Moller,
Moller, J.H.
J.H. and Swanson, D.B.:
D.B.: The Role
and
Development of
J\Jedical Knowledge
Knowledge in
Diagnostic Ezpertise.
Expertise.
IN
IN
Development
of &fedical
in Diagnostic
W.
W’. Clancey
Clancey and E.H.
E.H. Shortliffe
Shortliffe (Eds.),
(Eds.), READINGS
READINGS IN
IN MEDICAL
MEDICAL AI.
AI. (In
(In
press)
5. Johnson,
Models of
Medical Problem
Problem Solvers.
Cognitive Models
of Medical
Solvers. IN
IN D.C.
D.C.
Johnson, P.E.:
P.E.: Cognitive
Connelly,
Connelly, E. Benson, D. Burke
Burke (Eds.),
(Eds.), CLINICAL
CLINICAL DECISION
DECISION MAKING
MAKING AND
AND
LABORATORY
USE. University
LABORATORY
University of
of Minnesota
Minnesota Press, 1982,
1982, pp. 39-51.
6. Johnson,
Johnson, P.E.:
P.E.:

}Vhat kind
should a system
system be? J. Medicine
What
kind of
of expert
ezpert should
Medicine and

Philosophy,
Philosophy, 8:77-97,
8:i7-97, 1983.

Expert Mind:
Mind: A new
for the Information
Information
7. Johnson, P.E.,
P.E., The Expert
new Challenge
Challenge for
SciWtl:st
IN
SYSTElvl
Scientist
IN Th.
Th. M.
M. A.
A. Bemelmans
Bemelmans (Ed.),
(Ed.), INFORMATION
INFORMATION
SYSTEM
DEVELOPMENT
FOR
EFFECTIVENESS,
Elsevier
DEVELOPMENT
FOR ORGANIZATIONAL
ORGANIZATIONAL
EFFECTIVENESS,
Elsevier
Science Publishers
Publishers B. V.
V. (North-Holland),
(North-Holland), 1984.
1984.
Design of
8. Johnson,
D.G. and Feltovich,
Feltovich, P.J.:
P.J.: Design
of decision
decision
Johnson, P.E.,
P.E., Severance, D.G.
support agatema
systems in
Rationale and
principles from
from the analysis
support
in medicine:
medicine: Rationale
and principles
analysis of
of
ph!Jsician expertise.
physician
expertise. Proc.
Proc. Twelfth
Twelfth Hawaii
Hawaii International
International Conference
Conference on System
System
Science, 'Western
Western Periodicals
Periodicals Co.
Co. 3:105-118,
3:105-118, 1979.
1979.
9. Johnson,
Johnson, P.E.,
P.E., Duran,
Duran, A.,
A., Hassebrock,
Hassebrock, F.,
F., Moller,
Moller, J., Prietula,
Prietula, M.,
M., Feltovich,
Feltovich,
P. and Swanson, D.: Expertise
Expertise and
in diagnostic
diagnostic reasoning.
reasoning. Cognitive
Cognitive
and error
error in
Science 5::!35-283,
5:235-283, 1981.
1981.
10. Johnson.
Jvfodels
Johnson, P.E.
P.E. and Hassebrock,
Hassebrock, F.:
F.: Validating
Validating Computer
Computer Simulation
Simulation Models
of
Expert
Reasoning.
IN
R.
Trappl
(Ed.),
CYBERNETICS
AND
SYSTEMS
of Expert Reasoning. IN
Trappl (Ed.), CYBERNETICS AND SYSTEMS
RESEARCH.
North-Holland Publishing
RESEARCH. North-Holland
Publishing Co., 19.82.
1982.
and Thompson,
down
garden path:
path:
Thompson, \V.B.:
W.B.: Strolling
Strolling
down the garden
and
from error
problem solving.
solving. Proc.
and recovery
recovery
from
error in
in expert
expert problem
Proc. Seventh
Seventh
IJCAI,
IJCAI, Vancouver,
Vancouver, B.C.,
B.C., August,
August, 1981,
1981, pp. 214-217.

11
11.. .lohnson,
Johnson,

P.E.
P.E.

Detection
Detecfion

P.E.,
Jvfultimethod study
study of
Moller. J.H.:
J.H.: bfultimethod
of
P.E., Hassebrock,
Hassebrock, F. and Moller,
clinical
judgement. Organizational
Behavior
clinical
judgement.
Organizational
Behavior and Human
Human Performance
Performa.nce
30:201-230, 1982.

12.
12. Johnson,
Johnson,

13. l\1011er,
New techniques
hfo!ler, J.H
J.H.,.. Bass, G.M.,
G.M., Jr. and Johnson, P.E.:
P.E.: New
techniques in
in the
of
patient management
problems. Medical
Education
ron.'ltrurtion
construction
of patient
management
problems.
Medical
Education
15:1.')0-153,
1981.
15:l.W153,
1981.
14.
simulation of
problem solving
solving in
14. Swanson, D.B.:
D.B.: Computer
Computer
simulation
of expert
expert problem
in medical
medical

diagnosis.
diclgnosis. Unpublished
Unpublished Ph.D.
Ph.D. dissertation,
dissertation, University
University of
of Minnesota,
Minnesota, 1978.
1978.
Psychological Analysis
Analysis of
15. Swanson, D.B.,
D.B., Feltovich,
Feltovich, P.J.
P.J. and Johnson, P.E.:
P.E.: Psychological
of
Physician
Expertise:
Implications
for
The
Design
of
Decision
Support
Physician Ecpertiae:
Implications
for
Design of Decision
Support
Systems.
In D.B.
North-Holland
Systems.
D.B. Shires and H. \Vold
\Vold (Eds.),
(Eds.), MEDINF077,
MEDINF077,
North-Holland
Publishing
Publishing Co., Amsterdam,
Amsterdam, 1977,
1977, pp. 161-164.
161-164.
15.
Recognition-based diagnostic
16. Thompson,
Thompson, W.B.,
W.B., .Johnson,
Johnson, P.E.
P.E. and Moen,
Moen, J.B.: Recognition-based
diagnostic
reasoning.
reasoning. Proc.
Proc. Eighth
Eight,h IJCAI,
IJCAI, Karlsruhe,
Karlsruhe, West
West Germany,
Germany, August,
August, 1983.
1983.

163
163

E. A.
A. Feigenbaum
Feigenbsum

5P41 RR00785-11
RR00785-11

SOLVER
SOLVER Project
Project

17.
17. Sedlmeyer,
Sedlmeyer, R.L.,
R.L., Thompson,
Thompson, W.B.
W.B. and Johnson, P.E.:
P.E.:
localization
in
debugging.
The
localization in debugging. The Journal
Journal of
of Systems
Systems and

Knowledge-based fault
fault
Knowledge-based
Software
Software (in
(in press).

18.
Diagnostic reasoning
18. Sedlmeyer,
Sedlmeyer, R.L.,
R.L., Thompson,
Thompson, W.B.
W.B. and Johnson, P.E.:
P.E.: Diagnostic
reasoning
in
software fault
fault localization.
Proc.
in software
localization.
Proc. Eighth
Eighth IJCAI,
IJCAI, Karlsruhe,
Karlsruhe, West
West
Germany,
Germany, August,
August, 1983.
19. Sedlmeyer,
Knowledge-Based Fault
Fault
Sedlmeyer, R.L.,
R.L., Thompson,
Thompson, W.B.,
W.B., 8l
& Johnson,
Johnson, P.E.:
P.E.: Kno,wledge-Baaed
Debugging. The
Localization in
Localization
in Debugging.
The Journal
Journal of
of Systems
Systems and Software
Software (in
(in press).

E. Funding
Funding and
E.
and Support
Support
\Vork
\Vork on the SOLVER
SOLVER project
project is currently
currently supported
supported by
by a grant
grant from
from the Control
Control
Data
to
Dat,a. Corporation
Corporation
to Paul
Paul Johnson
Johnson ($90,000;
($90,000; 1983-85) and by
by a grant
grant from
from the
Microelectronics
Microelectronics and Information
Information Sciences Center
Center at the University
University of
of Minnesota
Minnesota to
to Paul
Paul
Johnson, William
William Thompson
Thompson and two
two colleagues ($800,000;
($800,000; 1984-87).

II.
II.

INTERACTIONS
vVITH
THE
RESOURCE
INTERACTIONS
WITH
THE SUMEX-AIM:
SUMEX-AJM
RESOURCE
A. Medical
Afedical Collaborations
Program Dissemination
Dissemination via
A.
Collaborations and
and fiogram
via SUAIEX
SUMEX

\Vork
\Vork in medical
medical diagnosis
dia.gnosis is carried
carried out
out with
with the
the cooperation
cooperation of
of faculty
faculty and
students
Paul Ramsey
Ramsey Medical
Medical
students in tht'
the University
University of
of Minnesota
Minnesota Medical
Medical School and St. Paul
Center.
Cent,er.

B. Sharing
Interactions with
ProJects
B.
Sharing and
and Interactions
with Other
Other SUA-lEX-AIM
SUfifEX-AIM
Projects
A
A year
year ago, conversations
conversations were begun with
with \Villiam
William Clancey
Clancey at Stanford
Stanford University
University
regarding
regarding collaboration
collaboration on the study
study of
of current
current knowledge
knowledge capturing
capturing methods.
methods. We
We plan
plan
to develop
develop this
this collaboration
collaboration in the coming
coming year.
year.

C. Critique
Resource Management
Management
Critique of
o j Resource
(None)
(None)

ill.
RESEARCH
PLANS
III.
RESEARCH
PLANS
A. Project
Project Goals
Plans
A.
Goals and
and Plans
Near term
Near
term -- Our
Our research objectives
objectives in the near term
term can be divided
divided in three
three
parts. First,
parts.
First., we are committed
committed to
to the design, implementation,
implementation, and evaluation
evaluation of
of Galen,
Ga.len, as
as
described
We have completed
completed an interactive
interactive front
front end so that
that physicians
physicians can
described above. \Ve
directly
direct.& enter
enter patient
ont.ient data,
data, and Galen's
Galen’s knowledge
knowledge base
b%e is currently
currently being "tuned"
“tuned” with
with
tile
the help of Dr.
Dr. James Moller
Moller MD,
MD, an expert
expert physician
physician collaborator
collaborator from
from the University
University of
of
f\Iinnesota
r\linnesot,n Pediatric
Pediatric Cardiology
Cardiology Clinic.
Clinic. During
During the coming
coming year,
year, Galen's
Galen’s performance
performance will
will
be compareo
compwred with
wit,11that
thn.t of
of the Diagnoser
Dingnoser program
program and with
with expert
expert phYSicians.
physicians.
Our
Our second objective
objective consists of
of making
making extensions
extensions to the knowledge
knowledge capturing
capturing
strategies
loped in our
strategies deve
developed
our original
original work
work in medical
medical diagnosis. In
In the near term
term this
this work
work
will
will examine
examine descriptive
descriptive strategies
strategies in which
which experts
experts attempt
attempt to use
use a formalized
formalized language
to express what
production rules),
rules), observational
observational strategies
strategies in which
which
what they
they know
know (e.g. production
experts
perform
t;1sks
designed
to
reveal
information
from
which
a
theory
expert,s perform t.asks
reveal information from which
theory of
of task
task specific
specific
expl'rtise
expcrtize can be bnilt,
built, and intuitive
intuitive strategies
strategies in which
which either
either experts
experts behave as
as knowledge
knowledge
engineers or knowledge
knowledge engineers attempt
attempt to
to perform
perform as
as pseudo experts.
experts. In the coming
coming
year
year we ,viII
will also be attempting
attempting to develop a program
program to
to automate
automate the early
early stages of
of

E. A. Feigenbanm
Feigenbaum

154
164

SOLVER
SOLVER Project
Project

5P-11 RROO785-11

knowledge
knowledge capturing,
ca.pturing, analogous to the "prototype
“prototype stage"
stage” of design referred
referred to in software
software
engineering.
engineering.
Our
Our third
t)hird near term
term objective
objective will
will be
be to investigate
investigate one
one of the central
central problems
problems of
recognit.ion
classify problems
problems when solving
solving them.
them.
recognition ba.<;ed
based problem
problem solving,
solving, how to classify
Questions
Questions related
relat.ed to problem
problem classification
classification which
which we will
will be
be examining
examining include:
include: What
What
patterns
patterns do experts
experts and novices detect
detect in aa problem
problem that
that allows them
them to classify
classify it
it as
as an
instance
inst,ance of aa problem
problem type
type that
that is already
already known?
known? How does
does an expert
expert make an initial
initial
choice of the level of abstraction
abstraction to be
be used in solving
solving aa problem?
problem? How
How can an expert
expert
recover
recover from
from an
a.n initial
initial incorrect
incorrect choice of levels?
levels? How
How can the difference
difference between
between causal
and
a.nd prot,otypic
prot,ot,ypic modes of reasoning
reasoning be modeled as
as differences
differences in levels of abstraction,
abstraction, and
how can aa common
common model for
for these two
two types of reasoning
reasoning be
be constructed?
constructed? \Ve
We will
will be
be
pllrsuing
pllrsuing these questions in the area of physics problem
problem solving,
solving, as
as well
well as
as in medicine.
medicine.
Long range -- Our
objective is
is to improve
improve the methodology
methodology of the
Our long range objective
"knowledge
“knowledge capturing"
capturing” process that
that occurs in the early
early stages of the development
development of
expert
expert systems
systems when
when problem
problem decomposition
decomposition and solution
solution strategies
stra.tegies are being specified.
specified.
Several
Severs1 related
related questions
questions of interest
interest include:
include: What
What are the performance
performance consequences
consequences of
of
different
different approaches,
spproaches, how can these consequences
consequences be evaluated,
evaluated, and what
what tools
tools can assist
in making
making the best choice?
choice? How
How can organizations
organizations be determined
determined which
which not
not only
only perform
perform
well,
well, but,
but are structured
structured so as
as to facilitate
facilitate knowledge
knowledge acquisition
acquisition from
from human
human experts? In
t.he
t,he coming
coming year
year we will
will be exploring
exploring these questions in areas of
of design and management
management
a.'
as3 ','iell
well as
as in law, physics
physics and
a.nd medicine.
medicine.

B. Justification
for Continued
Jelati jication and
and Requirements
Requirements for
Continued SUMEX
SUMEX Use
Use
Our
Our current
current model
model development
development takes
takes advantage
advantage of
of the
the sophisticated
sophisticated Lisp
Lisp
programming environment
programming
environment, on SUMEX.
SUMEX. Although
Although much
much current
current work
work with
with Galen
Galen is done
A.,'( 11/780,
benefit from
using aa. version
version running
running on a local V
VAX
11/780, we continue
continue to benefit
from the
int.eraction
by the SUMEX
use
int,era.ct,ion with
with other
other researchers facilitated
facilitated by
SUMEX system.
system. We
We expect
expect to use
SU~IEX to allow
program. We
plan to continue
use
SU:,lEX
allow other
ot,her groups
groups access
access to t.he
the Galen
Galen program.
We also plan
continue use
of
of the knowledge
knowledge engineering
engineering tools
tools available
available on SUMEX.
SUMEX.

C. Needs
Needs and
Plans for
for Other
Resources Beyond
Beyond SUAfEX-AIA1
and Plans
Other Computing
Computing Resources
SUibfEX-Al&f
Paul
Paul .Johnson
Johnson is a member
member of
of the
the group
group of investigators
investigators who
who have recently
recently
submitt.ed
proposal to
submitt,ed a proposal
to establish
establish a national
national computer
computer network
network for
for cognitive
cognitive scientists
scientists
(COGNET). In addition,
addition. our
our current
current grant
grant will
will permit
permit the
the purchase
purchase of
of some single-user
single-user
(COGNET).
computers (we are currently
currently comparing
comparing several
several alternative
alternative machines).
machines).
SUMEX will
will
computers
SUMEX
continue to
to be used for
for collaborative
collaborative activities
activities and for
for program
program development
development requiring
requiring
cont.inue
tools not
not available
available locally.
locally.
tools

D. Recommendations
Recommendations for
for Future
Future Community
Community and
and Resource
Resource Development
Development
D.
(None)
(None)

165

A. Feigenbaum
Feigenbaum
E. A.

Pilot
Pilot Stanford
Stanford Projects
Projects

5P41 RR00785-11
RR00785- 11

II.A.3.
Pilot
Projects
II.A.3.
Pilot Stanford
Stanford
Projects
Following
pilot projects
projects currently
Following are descriptions
descriptions of
of the informal
informal pilot
currently using
using the
Stanford
portion of
resource,
pending funding,
St,anford portion
of the SUMEX-AIM
SUMEX-AIM
resource, pending
funding, full
full review,
review, and
authorization.
authorization.
In addition
progress reports
presented here, abstracts
project are
addition to the
the progress
reports presented
abstracts for
for each project
submitted
submitted on a separate
separate Scientific
Scientific Subproject
Subproject Form.
Form.

E. A.
A. Feigenbaum
Feigenbaum

156
166

CAMDA
CAMDA Project
Project

5P41
RfW0785-11
51’
41 RFt00585-11

IT.A.3.1. CAMDA
CAMDA Project
Project
II.A.3.1.
CAMDA Project
Project
CAMDA
CAMDA Research
Research Staff:
Staff:
CAMDA
Samuel Holtzman,
Holtzman, Co-PI
Co-PI
Samuel
Prof. Ronald
Ronald A.
A. Howard,
Howard, Co-PI
Co-PI
Prof.
Jack Breese
Breese
Jack
Dr. Emmet
Emmet Lamb
Lamb
Dr.
Dr. Robert
Robert Kessler
Kessler
Dr.
Dr. Frank
Frank Polansky
Polansky
Dr.

Engineering-Economic
Engineering-Economic Systems
Systems
Engineering-Economic Systems
Systems
Engineering-Economic
Engineering-Economic
Engineering-Economic Systems
Systems
School
School of
of Medicine
Medicine
School
School of
of Medicine
Medicine
School
School of
of Medicine
Medicine

Associated faculty:
faculty:
Associated
Engineering-Economic
Engineering-Economic Systems
Systems
Engineering-Economic
Engineering-Econom ic Systems
Systems

Prof. Edison
Edison Tse
Tse
Prof.
Prof. Ross
Ross Shachter
Shachter
Prof.

I.
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
I. SUMMARY
A. Project
Project Rationale
Rationale
A.
The
project is an attempt
The Computer-Aided
Comput,er-Aided Medical
hiedical Decision
Decision Analysis
Analysis (CAMDA)
(CAMDA) project
attempt to
t,o
develop
by taking
develop intelligent
int,elligeut medical
medical decision
decision systems by
taking advantage
advantage of
of the complementary
complementary
methodologies
methodologjes of
of decision
decision analysis
a.nalysis and artificial
artificial intelligence.
intelligence.

Medical Relevance
Relet'ance and
B. Medical
und Collaboration
Collaboration
The
primary effort
project during
The primary
effort of
of the CAMDA
CA>lDA project
during 1983 was focused on the design
and implementation
implement,st.ion of
of RACHEL,
RACHEL, an intelligent
intelligent decision system
system for
for infertile
infertile couples. This
This
effort
physicians and patients
patients deal with
effort is aimed
a.imed at
at helping
helping physicians
with difficult
difficult choices regarding
regarding
pertinent.
RACHEL is being developed in close
close cooperation
cooperation between
between
pertinent, medical
medical procedures.
procedures. RACHEL
t.he
depart,ment of Obstetrics
Obstetrics and
the department
depart*ment of Engineering-Economic
Engineering-Economic Systems, the department
Gynecology,
Gynecology, and the
t.he department
department of
of Surgery
Surgery (Urology
(Urology Division),
Division), all at Stanford.
Stanford.

C. Highlights
Highlights of
of Research
Research Progres8
Progress
C.l
past year
year
C.l Accomplishments
Accomplishments this
this pad
The
The CAMDA
CAbIDA project
project began in the summer
summer concentrated
concentrated our
our efforts
efforts on three
three
specific
specific tasks: the development
development of a formal
formal representation
representation for
for uncertain
uncertain decisions, the
design and implementation
implementat.ion of solution
solution algorithms
algorithms for
for formal
formal decision problems,
problems, and the
construction
constructZion of an inferential
inferential processor specifically
specifically tailored
tailored to the process of
of formalizing
formalizing
decision problems.
problems.
Most
Most of our
our research has been based on the concept
concept of an influence
influence diagram
diagram
(Howard
(Howard and Matheson,
Mstheson, 1984) which
which is
is generalization
generalization of decision trees as
as aa representation
representation
for decision problems.
Influence diagrams
diagrams (IDs)
(IDS) have several major
major features
fentures that
that make
problems. Influence
them
them attractive
attractive for use
use in intelligent
intelligent decision systems. Technically,
Technically, IDs
IDS prevent
prevent the loss
loss of
information
information often
often incurred
incurred in constructing
constructing asymmetric
asymmetric decision trees (Olmsted,
(Olmsted, 1984),

157
167

Feigenbaum
E. A. Feigenbaum

P41
P41 RR00785-11
RR00785-11

CAMDA
CAMDA Project
Project

wit,hout.
wit,hout suffering
suffering from
from the
the explicit
explicit exponential
exponential growth
growth of
of symmetric
symmetric decision
decision trees.
trees.
Furthermore,
Furt,hermore, unlike
unlike decision
decision trees,
trees, ID
ID decision
decision models
models ta.ke
take full
full advantage
advantage of
of probabilistic
probabilistic
inclependence
independence relat.ions,
relations, which
which can
can have
have aa significant
significant impact
impact on
on the
the simplicity
simplicity of
of the
the
decision
decision model,
model, and
and on
on the
the efficiency
efficiency oC
of its
its formal
formal solution.
solution.
A
A particularly
particularly useful
useful feature
feature of
of influence
influence diagrams
diagrams which
which we have recently
recently begun
investigating
investigating isis that
that fact
fact that
that they
they can
can be
be used
used to
to represent
represent deterministic
determinist.ic as
as well as
as
probabilistic
probabilistic relations
relations between
between model
model elements.
elements. In
In Cact,
fact, deterministic
deterministic relations
relations can be
be
exploited
exploited to
to describe
describe complex
complex probabilistic
probabilistic behavior.
behavior. This
This Ceature
feature allows the construction
construction
of simple knowledge
bases
(composed
primarily
of
deterministic
knowledge bases
primarily
deterministic statements)
statements) which
which can be
be
llsecl
used to create
crea.te problem-specific
problem-specific probabilistic
probabilistic decision models.
In addition
addition to their
their technical
technical advantages
advantages IDs
IDS have been
been empirically
empirically shown to be
intuitively
int.uit,ively appealing
a.ppealing to decision makers
makers (Owen,
(Owen, 1978),
1978), and to provide
provide an excellent
excellent means
of communication
communication between experts
experts in different
different fields. In the context
context of our
our own efforts,
effort,s,
we have found
found that
that the physicians
physicians who
who are participating
participating in the development
development of RACHEL
RACHEL
have ha.d
had liWe
little difficulty
difficulty using IDs
IDS as
as aa simple
simple representation
representation for
for expressing the
t,he decisions
they
they and their
their patients
pat!ients face.
face.
Another
Another important
important feature
feature of IDs
IDS is the Cact
fact that
that they
they are naturally
naturally constructed
constructed in
a backwards,
bsckwards. goal-directed
goal-directed fashion
fashion (decision
(decision trees usually
usually lead to
to a forward-reasoning
forward-reasoning
approach).
approach). Backward
Backward development
development of decision models has two
two important
importa.nt advantages
advantages for
for
our
our purposes. First,
First, it
it has a strong
strong attention-focusing
attention-focusing effect
effect since it
it encourages the
dt~ci"ion
decision maker
maker to first
first think
think of what
what he or she
she wants,
wants, and then
then about
about what
what can be done
to change the
t.he world
world according
according to the expressed preferences.
preferences. Decision
Decision trees usually
usually have
t,he
opposite
effect.
Thus,
they
often
lead
the
decision
process
along
paths
that
although
t,he opposite effect. Thus, they often
decision
along paths that although
releva.nt
attention-focusing effect
effect of
of
releva.nt to the decision
decision at
at hand,
hand, have little
little effect
effect on it.
it. The
The attention-focusing
IDs
its efficiency.
efficiency. The
The second
IDS on the
t,he decision making
making process tends to
to contribute
contribute to
to its
advantage
of intelligent
intelligent decision
decision
advantage of
of the goal-directed
goal-directed nature
nature of
of IDs
IDS for
for the
the construction
construction of
systems is that
amenable to
to computer-based
computer-based
that it
it makes the formulation
formulation of
of decision problems
problems amenable
automation
a,utomation as
as a rule-based
rule-based system.
system.
Having decided
decided on IDS
IDs as a means to
to represent
represent decision
decision problems,
problems, we have designed
Having
implemented several
algorithms to
to solve well-formed
well-formed influence
influence diagrams.
diagrams. This
This effort
effort
and implemented
several algorithms
resulted in the development
development of
of a powerful
powerful software
package which
which can generate
generate
has resulted
software package
optimal strategies
their certain
certain equivalent
equivalent directly
directly from
from an ID.
ID. This
This package is
optima.1
st.ra.tegics and their
beginning to
to be tested
tested and
a.nd augmented
augmented to
to make
make itit easier to
to use by
by researchers
researchers other
other tha.n
than
beginning
its developers.
developers. In part,
part, this
this package
package is based on the
the work
work of
of Olmsted
Olmsted (1983),
(1983), and
and on a
its
constructive proof
proof by
by Shacht.er
Shachter (1984)
(1984) that,
that, given
given certain
certain technical
technical features,
features, shows
shows that
that an
construct,ive
influence
diagram
can
always
be
solved
in
finite
time.
inflllence diagram
a.lways
solved in finite time.
An importSant
important feature
feature of
of RACHEL
RACHEL is that
that itit attempts
attempts to
to help
help its
its users in
in the
the
An
development of
of models
models for
for their
their decisions.
decisions. Thus,
Thus, unlike
unlike most
most other
other decision
decision analysis
analysis tools,
tools,
development
RACHEL is designed to
to use domain
domain knowledge.
knowledge. Therefore,
Therefore, a central
central element
element in
in the
the
RACHEL
archit,ecture of
of the
the RACHEL
RACHEL system
system is an
an algorithm
algorithm which
which performs
performs symbolic
symbolic inference.
inference.
archit.ecture
Although several
several general-purpose
general-purpose inference-engines
inference-engines exist
exist within
within our
our research
research environment,
environment,
hlt.holugh
found itit advantageous
advantageolls to
to implement
implement our
our own
own for
for reasons of
of efficiency
efficiency and
and
we have found
compatibility. Furthermore,
Furthermore, our
our inference
inference algorithms
algorithms are particularly
particularly well
well suited
suited for
for the
the
compatibility.
construction of
of decision-analytic
decision-analytic models.
models.
construcbion
Finally, from
from the
the standpoint
standpoint of
of computer
computer implementation,
implementation, we
we have
have developed
developed a
Finally,
d~tta structure
structure which
which allows
allows us to
t,o represent
represent a wide
wide class of
of multiple-entry
multiple-entry disconnected
disconnected
data
cyclical directed
directed graphs,
graphs, where
where both
both vertices
vertices and
and edges can
can be
be associated
associated with
with arbitrary
arbitrary
cyclical
data
structures
(such
as
frames).
For
short,
we
refer
to
these
graphs
as
WEBs
(as in
in aa
data structures (such
frames). For short, we refer to these graphs as WEBS (as
spider's),
and we
we have
have used
used them
them to
to represent
represent a multitude
multitude of
of small
small and
and medium-sized
medium-sized
spider’
s), and

E. A.
A. Feigenbaum
Feigenbaum
E.

158
168

5P4 1 RR00785-11
RROOi85- 11
5P41
objects such as
as
objects
tables,
help
text
tables,
text

CAMDA
CAMDA Project
Project

influence diagrams,
diagrams, medical
medical
influence
databases, and mathematical
mathematical

decision knowledge
knowledge bases,
bases, command
command parse
data
(e.g.,
vectors
and
matrices).
data
vectors
matrices).

169
159

A. Feigenbaum
Feigenbaum
E. A.

CAr-.mA
CAhlD~4 Project
Project

P41
P41 RR00785-11
RR00785-11

C.2
C.2 Research
Research in
in progress
progreso
The immediate
project
immediate goal of the
the CAMDA
CAMDA
project is
is to complete
complet,e aa pilot-level
pilot-level
implementation
implementation of the RACHEL
RACHEL system
system within
within the next few months.
months. As we define it, aa
pilot
pilot system
system isis aIle
one where the essential algorithms
algorithms work
work beth
bcth individually
individually and interactively
interactively
with
with one another,
another, operating
operating with
with knowledge
knowledge that
that is
is representative
representative of the system's
system’s domain.
domain.
Such a system
system lacks two
two important
important elements
elements that
that must
must exist
exist within
within a prototype-level
prototype-level
implementation:
implement,ation: an extensive
extensive knowledge
knowledge base,
base, and aa front
front end usable by trained
trained users who
who
may
the system.
system.
may not
not be
be familiar
fa.miliar with
with the details
details of the
To
To complete
complete aa pilot
pilot implementation
implementation of RACHEL,
RACHEL, we intend
intend to direct
direct our
our efforts
efforts
towards
towards the following
following four
four tasks: incorporating
incorporating aa medical
medical value model elicitation
elicitation facility.
facility,
strengthening
strengthening our
our influence-diagram
influence-diagram solution
solution procedure,
procedure, improving
improving the performance
performance of
RACHEL's
implementing an explanation
explanation module
module to justify
justify the
R.4C’HEL’s inference
inference engine, and implementing
deci~ion
decision model being developed. Once this
this implementation
implementation is completed,
completed, RACHEL
RACHEL will
will be
brollght
broiight. to the
t,he participating
participating physicians
physicians to begin to develop its knowledge
knowledge base.
base.

D.
D. Publication.,]
Publications
"IDEAS:
“IDEAS: Influence
Influence
Diagram
System",
Department of
of Engineering-Economic
Engineering-Economic Systems,
Diagram Elicitation
Elicitation
System “, Department
Stanford
Stanford University,
University, Stanford,
Stanford, California,
California, 1983.

1.
1. Breese, J.S., Davis,
Davis, D., Parnell,
Parnell, G.S., and Taneja,
Taneja, R.:

2. Holtzman,
Ana/ysi.,
Holtzman, S.:"A
S.: “A A/ode!
Model of
of the Decision
Decision
Analysis Process",
Process II, Department
Department of
of

Engineering-Economic
Systems, Stanford
Engineering-Economic
Stanford University,
University, Stanford,
Stanford, California,
California,
1981.
1981.
3. Holtzman,
Decision Aid
Aid for
for Patients
Patients with
End-Stage Renal
Renal Disease
Disease",“,
Holtzman, S.:"A
S.: aA Decision
with End-Stage

Department
Department of
of Engineering-Economic
Engineering-Economic Systems, Stanford
Stanford University,
University, Stanford,
Stanford,
California,
1983.
California,
4. Holtzman,
IrOn
Holtzman, S.: “On

the Use of
Formal Models
Models in
Decision Making
Making",“, Proc.
o j Formal
in Decision
Proc.

TIMS/ORSA
Nat. Mtg.,
TIhLIS/ORSA Joint
Joint Nat.
Mtg., San Francisco,
Francisco, May,
May, 1984.
5. ((*)
.. ) Holtzman,
Holt,zman,

Department
Depa.rtment
forthcoming.
fort,licoming.

S.:

of
of

"Intelligent
Decision Systems",
Dissertation,
“Intelligent
Decision
Systems “, Ph.D.
Ph.D.
Dissertation,
Engineering-Economic
Systems,
Stanford
University,
Stanford
Engineering-Economic
University,

Howard, R..4.,
R.A., and Matheson,
Matheson, J.E.:
J.E.: "Influence
Diagrams", in Howard,
Howard, R.A.,
R.A.,
6. Howard,
“Influence Diagram”“,
Matheson, J.E.
J.E. (Eds.):
(Eds.): “The
"The Principles
Principles and
and Applications
Applications of
of Decision
Decision
and Matheson,
Analysis, ”II Vol.
Vol. II,
II, Strategic
Strategic Decisions
Decisions Group,
Group, Menlo
Menlo Pa,rk,
Park, California,
California, 1984.
Analysis,
Olmsted, S.M.:
S.M.: “On
IrOn Representing
Representing and
and Solving
Solving Decision
Decision Problems”,
Problems", Ph.D.
Ph.D.
7. Olmsted,
Dissertation. Department
Department of
of Engineering-Economic
Engineering-Economic Systems,
Systems, Stanford
Stanford
Dissertation.
University, St,anford,
Stanford, California,
California, 1983.
University,
Owen, D.L.:
D.L.: "The
Influence
Diagrams in
in Structuring
Structuring Complex
Complex
8. Owen,
The Use oof
j In
jluence Diagrams
Deri.'3ion Problems
Problems",“, in Howard,
Howard, R.A.,
R.A., and Matheson.
Matheson, J.E. (Eds.):
(Eds.): “The
"The
Derision
Principles a.nd
and Applications
Applications of
of Decision
Decision Analysis,”
Analysis," Vol.
Vol. II,
Strategic Decisions
Decisions
Principles
II, Strategic
Group, Menlo
Menlo Park,
Park, California,
California, 1984.
Group,
Shachter, R.‘
R.:: “Evaluating
"Evaluating In
Influence
Diagrams",
working paper,
paper, Department
Department
9. Shacht.er.
jluence Diagrams
“, working
of Engineering-Economic
Engineering-Economic Systems,
Systems, Stanford
Stanford University,
University, Stanford,
Stanford, California,
California,
of
1984.

A. Feigenba,um
Feigenbaum
E. A.

170
170

CAMDA
CAMDA Project
Project

5P41
5Pal RROOi85-11
RR00785-11

E.
E. Funding
Funding Support
Support
The
However,
The CA~mA
CAniDA project
project does
does not
not yet
yet have
have direct
direct funding
funding support.
support.
However, in
in
addition
addition to
to SUMEX
SUAIEX computer
computer usage,
usage, the
the project
project has
has benefited
benefited from
from aa number
number of
of hardware
hardware
gifts
gifts and
and research
research support
support for
for individuals.
individuals.

E.1
E.l Stanford
Stan ford .Medical
Medical School
School
The
The dt'partment
department of
of Obstetrics
Obstetrics and
and Gynecology
Gynecology and
and the department
department of Surgery
Surgery
(Urology
Samuel
(Urology Division)
Division) have
have provided
provided various
various types
types of
of support
support to the project.
project.
Samuel
Holtzman
Holtzmsn has
has received research assistantship
assistantship awards for several quarters.
quarters. In addition,
addition, the
Inft'rtility
Infert.ility Clinic
Clinic at Stanford
Stanford has
has purchased
purchased several terminals
terminals for
for the specific
specific purpose of
developing
RACHEL
and
other
CAMDA
decision
systems.
developing RACHEL
other CAMDA

E.2
E.2 Decision
Decision Systems
Systems Laboratory
Laboratory
The CA~mA
CAhlDA project
project has access
access to the facilities
facilities of the Decision
Decision Systems Laboratory
Laboratory
(DSL)
Systems, and constitutes
(DSI,) in the Department
Department of Engineering-Economic
Engineering-Economic
constitutes the
laboratory's
laborat.ory3 most
most active
act,ive research project.
project. The DSL maintains
maintains several terminals,
terminals, printers
printers
and aa personal computer
computer for research on the development
development of computer-based
computer-based decision
systems. The majority
majority of the terminals
terminals and printers
printers were recently
recently donated
donated to the DSL
DSL by
by
Qume
MAD
Qume Corporation.
Corporation.
MAD Computer
Computer of Santa
Santa Clara
Clara has also contributed
contributed to
to the support
support
of the CAMDA
CAMDA project
project through
through the consignment
consignment of a MAD-l
MAD-l persona.l
personal computer,
computer, and
provision
provision of a.a research assistantship
assistantship for
for Samuel Holtzman.
Holtzman.

IT.
WITH
THE
RESOURCE
II. INTERACTIONS
INTERACTIONS
WITH
THE SlThtIEX-AIM
SUMEX-AIM
RESOURCE
II.A Medical
Afedical Collaborations
Dissemination Via
SUMEX
II.A
Collaborations and
and Program
Program Dissemination
Via SUMEX
Since its inception,
from an active
active relationship
relationship
inception, the CAMDA
CAMDA project
project has benefited
benefited from
between
decision
analysts,
computer
scientists,
and
members
of
the
Stanford
medical
between
analysts, computer scientists,
members of the Stanford medical
community.
In
particular,
RACHEL
is
being
developed
in
close
cooperation
with
particular, RACHEL
being developed
cooperation with
community.
physicians in the Infertility
Infertility Clinic
Clinic at
at Stanford.
Most of
of this
this cooperation
cooperation has, up
up to
to this
this
physicia,ns
Stanford. Most
point, consisted
consisted of
of an intense
intense mutual
mutual learning
learning experience for
for all
all project
project participants.
participants. The
The
point,
primary purpose
pmpose of
of this
this initial
initial effort
effort has been to
to develop
develop an effective
effective means to
to represent
represent
primary
medical decision
decision knowledge.
knowledge. As we have described
described above, this
this work
work has culminated
culminated in
in the
medical
definition of
of a representation
representation language based on influence
influence diagrams.
diagrams.
definition
\Vithin
the next
next few months,
months, RACHEL
RACHEL is expected
expected to
to attain
attain pilot-level
pilot-level
\\‘
ithin
the
performance, and its
its knowledge
knowledge base will
will begin
begin to
to be deveioped.
developed. At
At this
this point,
point, most
most of
of
performa.nce,
interaction involving
involving participating
participating physicians
physicians will
will shift
shift to
to the
the design and
and
the intera.ction
implementation of
of an infertility
infertility decision
decision knowledge
knowledge base.
This task
task will
will involve
involve
implementation
This
considerable
direct
use
of
the
SUMEX
facility
by
medical
personnel.
considerable direct, use of the SUMEX facility by medical personnel.
As an added
added benefit
benefit of
of the
the development
development of
of Rr\CHEL,
RACHEL, itit often
often occurs
occurs that
that specific
specific
,4s
subsystems become useful
useful in
in their
their own
own right.
right. For
For instance,
instance, a simple
simple program
program to
to aid
aid
subsystems
physicians in
in determining
det.ermining a course of
of action
action in
in cases
cases of
of idiopathic
idiopathic infertility
infertility has been
physicians
implemented and made available
available on SUMEX
SUMEX to
to the
the staff
staff of
of the
the Stanford
Stanford infertility
infertility clinic,
clinic,
implemented
who have used itit on an experimental
experimental basis.
who

IT.B. Sharing
Sharing and
and Interactions
Interactions with
with other
other SUMEX-AIM
SlThtIEX-AIM Projects
Projects
ILB.
II.B.l SUMEX-AIM
SUAIEX-AlN! 1988
1983 Workshop:
Workshop:
II.B.l
Samuel Holtzman
Holtzman chaired
chaired the
the working
working group
group on
on decision
decision analysis
analysis and
and artificial
artificial
Samuel
intelligence in
in medicine.
medicine. This
This group
group considered
considered the
the current
current status
status and
and future
future of
of medical
medical
intelligence

171

E. A.
A. Feigenbaum
Feigenbaum
E.

RR00785-11
P41 RR00785-11

CAMDA
CAh.IDA Project
Project

A full
full report
report of
of the
the working
working group's
group’s deliberations
deliberations and conclusions
conclusions is
decision
decision systems. A
online
SUMEX
file
< HOLTZMAN.CAMDA>AIM-DA-FINALHOLTZMAN.CAMDA
>AIM-DA-FINALavailable
at SUMEX
in tile
<
ova.ilable online
REPORT.TXT.
and should
REPORT.TXT,
should appear
appear in the
the forthcoming
forthcoming workshop
workshop report.
report.

lI.B.2
Participation in
Knowledge Representation
Representation seminar
II.B.2 Participation
in the Knowledge
seminar at Stanford
Stanford
As part
part of
of the CAMDA
CAMDA project,
project, we have made several presentations
presentations to
to the general
general
Stanford medical
medical and
and’'computer
computer science community.
community. These presentations
presentations have been made
Stanford
within
jointly by
within the context
context of
of the Knowledge
Knowledge Representation
Representation seminar,
seminar, held jointly
by the computer
computer
science department
department and the medical
medical school, and well
well attended
attended by
by other
other SUMEX
SUMEX
researchers
researchers at Stanford.
Stanford.
The
The speakers, and titles
titles of
of the
the most
most recent
recent presentations
presentations follow:
follow:
Samuel Holtzman:
HOI tzman:
Samuel

On the
the Design
Design and Implementation
Implementation of
of
On
Computer-Based
Computer-Based Decision
Decision Systems.
Systems.

Samuel
Samuel Holtzman:
Holtzman:

A Simple
Simple Representation
Representation
for Uncertain
Uncertain Knowledge
A
for

Prof.
Prof. Ross Shachter:
Shachter:

Influence Diagrams
Diagrams and their
their Use
Use in
Influence
Representing and Solving
Solving Complex DeciSion
Decision
Representing
Problems.
Problems.

II.C. Critique
Resource Management
Management
II.C.
Critique of
of Resource
The
The CAMDA
CAMDA project
project has been immeasurably
immeasurably aided by
by the availability
availability of
of the
SUt-.IEX
SUhlEX computing
computing resources. In general
general we find
find the overall
overall physical
physical facilities
facilities to
to be of
of
excellent
In
excellent quality.
quality.
In addition,
addition, we have been quite
quite impressed with
with the quality
quality of
of the
particular, we have found
found it
it to
to be a pleasure to deal with
with Ed
Ed
SUMEX
In particular,
SUMEX staff.
staff.
Pa.t.termann,
Pa.t,terma.nn, who
who has
ha.s been invariably
invariably courteous,
courteous, responsive
responsive to our
our needs,
needs, and effective
effective in
his actions.
act,ions. Pam
Pa.m Ryalls
Ryalls has also provided
provided much
much needed help in managing
managing the CAMDA
CAMDA
project.
project. in a manner
manner that
that is friendly
friendly and efficient.
efficient.

There are,
a.re, however,
however, two
two areas where
where we feel service and performance
performance could
could be
There
improved to the
t,he benefit
benefit of
of the entire
entire SUMEX
SUMEX community.
community. The
The first
first concerns
concerns the SUMEX
SUMEX
improved
facilities
facilities themselves.
themselves. the other
other refers
refers to
to our
our means of
of communicating
communicating with
with these facilities.
facilities.

lI.(U
II.C.l SU.~IEX
XrAIEX load
load
In the pt'riod
period that
that the CAMDA
CAMDA project
project has been active,
active, we have noticed
noticed a
significant
during
significa.nt increase
increa.se in the maximum
maximum machine
machine loading,
loading, particularly
particularly
during weekday
weekday
afternoons.
afternoons. Although
Although this
this is a normal
normal feature
feature of
of time-shared
time-shared computer
computer systems, the load
has become
sufficiently high
become.sufficiently
high in
in recent
recent months,
months, that
that it
it is beginning
beginning to
to be difficult
difficult to work
work
on SUMEX
SUMEX d1uing
during business hours.
hours. In addition,
addition, reliability
reliability has
ha5 been adversely
adversely affected
affected in
inst,a.nces. An
An increase in SUMEX
SUMEX computing
computing capacity.
capacity, or
or a means of
of preventing
preventing
some inst.ances.
overloading
overloading of
of the machine
machine should
should be considered.
considered. We
We believe that
that an emphasis on
distributing some of
of SUl'vlEX's
SUMEX’s computing
computing power
power away
away from
from a centralized
centralized mainframe
mainframe could
could
distributing
have a significant
significant effect
effect on reducing
reducing the system
system load.

II.C.
II.C.22 Ethernet
Ethernet
The
The CA~lDA
CALIDA project
project uses
uses SUMEX
SUMEX almost
almost exclusively
exclusively through
through Ethernet
Et.hernet software
software
hardware located
located in Terman
Terman Engineering
Engineering Center,
Center, where
and hardware
where the department
department of
of
Engineering-Economic Systems
Systems is located.
located. This
Engineering-Economic
This software
software has on occasion been extremely
extremely
unreliable for
for extended
extended periods
periods of
of time,
time, resulting
resulting in substantially
substantially reduced
reduced productivity
productivity for
for
unreliable
project. Adequate
Adequate communicat.ion
communication facilities
facilities at Stanford
Stanford are of
of critical
critical importance
importance t.o
to
the project.

E. A. Feigenbaum
Feigenbsum

172

CAMDA Project
Project
CAMDA

5P411 RR00785RR00785-11
5P4
11

the successful
successful conduct
conduct of
of our
our research.
research. Although
Although Stanford
Stanford Ethernet
Ethernet management
management is not
not
the
directly under
under the
the jurisdiction
jurisdiction of
of SUMEX,
SUMEX, in
in order
order for
for the
the SUMEX
SUMEX resource
resource to
to be utilized
utilized
directly
to the
the fullest,
fullest, the
the planning
planning and
and administration
administration of
of networking
networking at
at Stanford
Stanford needs to
to be
to
better coordinated.
coordinated. We
We have
have begun
begun to
to explore
explore several
several means to
to improve
improve the
the current
current
better
situation. and
and we believe
believe that
that explicit
explicit SUMEX
SUMEX support
support of
of our
our efforts
efforts would
would be quite
quite
situation,
beneficial.
beneficial.

III.

RESEARCH PLANS
PLANS
RESEARCH
III.A Project
Project Goals
Goals and
and Plans
Plans
III.A

For the
the near
near term
t.erm future
future the
the primary
primary goals of
of the
the CAMDA
CAMDA project
project are to
to develop
develop a
For
"pilot" and
and then
then “prototype”
"prototype" version
version 01
of the
the RACHEL
RACHEL system.
system. Over
Over an extended
extended period,
period,
“pilot”
our objective
object.ive is to
to arrive
arrive at
at useable, fully-validated
fully-vaiidated and
and documented
documented systems
systems for
for support
support
our
of medical
medical decision
decision making
making in
in infertility
infertility and
and other
other domains.
domains.
of
Implementat.ion of
of a pilot
pilot system
system is primarily
primarily an int.egrative
integrative task
task at
at this
this point,
point,
Implementation
bringing together
together the
the medical
medical knowledge
knowledge base, symbolic
symbolic inference
inference procedure,
procedure, decision
decision
bringing
problem solution
solution procedure,
procedure, and
and influence
influence diagram
diagram data
data structures.
structures.
All of
of these
All
problem
components exist
exist independently.
independently. The
The pilot
pilot system
system will
will consist
consist 01
of these systems
systems interacting
interacting
components
to provide
provide a simplified
simplified version
version of
of infertility
infertility decision
decision counseling.
counseling.
to
The prototype
prototype implementation
implementation of
of RACHEL
RACHEL will
will include
include substantially
substantially greater
greater
The
amounts of
of medical
medical knowledge
knowledge than
than the
the pilot.
pilot. The
The major
major task
task at
at this
this stage will
will be the
the
amounts
in~orporation of
of expert
expert knowledge
knowledge regarding
regarding functional
functional relations,
relations, probability
probability distributions,
distributions,
incorporation
a TILl decision
decision alt,ernat,ives
alt.ernatives in the infertility
infertility domain.
domain. At
At this
this point
point in its
its development,
development, the
the
and
system
system will
will be available
available for
for use by
by participating
participating physicians
physicians at
at the
the infertility
infertility clinic
clinic on a
"test"
justification of
“test” basis, beginning
beginning the
the critical
critical phase of
of validation
validation and justification
of the
the system.
system.
A
A major
major goal of
of the
the project
project is to
to bring
bring RACHEL
RACHEL to
to a IIdefensible
“defensible” ll level of
of
performance
domain.
A
A working
working system
system with
wit.h full
full documentation,
documentation,
performance in the
the infertility
infertility
domain.
expla.nation
Over
Over the
the long
long term,
term,
cxpla.nation of
of it,s
it,s conclusions,
conclusions, and
and user interface
interface is envisioned.
envisioned.
infcrtilit.y
infertilit,y is but,
but a single
single example
example of
of the
the range of
of medical
medical decisions
decisions amenable
amenable to
to decision
decision
analytic
After RACHEL
RACHEL has been fully
fully implemented
implemented
analytic treatment
t.reatment in an automated
automated system.
system. After
and tested.
tested, other
other systems
systems focusing
focusing on cardiology
cardiology or
or oncology,
oncology, for
for example,
example, might
might be
These
systems
would
consist
of
a
common
core
of
procedural
knowledge
developed.
developed.
systems would consist of common
of procedural knowledge based
on decision
analysis,
and
be
instantiated
with
the
medical
decision analysis,
instantiated with
medical knowledge
knowledge of
of the particular
particular
domain.
domain.

IlI.B
and
for Continued
SUlHEX
1II.B Justification
Justification
and Requirements
Requirements for
Continued
SiYMEX Use
The
It
It draws
draws on elements
elements of
of decision
decision
The CAMDA
CAMDA project
project is truly
truly interdisciplinary.
interdisciplinary.
analYSis.
The project
project has the potential
potential to
to
analysis, artificial
artificial intelligence,
intelligence, and medical
medical science. The
contribute
SUMEX-AIM provides
provides the
contribute to each of
of these disciplines
disciplines in important
important ways.
ways. SUMEX-AIM
resonrces
resources to continue
continue this
this research
research with
with the necessary access
access to
to members
members of
of the Stanford
Stanford
research community.
communit,y.
The
The development
development of
of automated
automated decision
decision systems
systems has the
the potential
potential to greatly
greatly
increase the use
In the past,
past, although
although
use and acceptance
accept,ance of decision analysis
analysis methods.
methods.
decision
decision analysis
analysis has be~n
been shown
shown to be an extremely
extremely effective
effective means of asSisting
assisting in
dccision
effort involved
involved in
sad uncertain
uncertain domains,
domains, the cost and effort
decision m'aking
making in complex
complex and
producing
producing an analysis
analysis was prohibitive
prohibitive for
for most
most individuals.
individuals. Automated
Automated decision
decision analysis
analysis
can result
result in a much
much lower
lower cost per user, allowing
allowing decision
decision theoretic
theoretic techniques
techniques to achieve
much
much wider
wider application.
application.
The
The development
development of decision
decision systems
systems owes much
much to advancements
advancements in the fields of

173
173

E.
E. A. Feigenbaum
Feigenbaum

CA~1DA Project
Project
CA>lDA

P41 RR0078511
RR0078S-11
P41

artificial intelligence,
intelligence, expert
expert systems,
systems, and
and knowledge
knowledge engineering.
engineering.
One continuing
continuing
One
artificial
challenge in
in these
these fields
fields has
has been
been representation
representation and
and reasoning
reasoning with
with probabilistic
probabilistic
challenge
knowledge. The
The representation
representation of
of knowledge
knowledge in
in influence
influence diagrams,
diagrams, and
and the
the use
use of
of
knowledge.
decision analysis
analysis in
in probabilistic
probabilistic reasoning
reasoning are
are both
both significant
significant topics
topics of
of research
research being
being
decision
pursued within
within the
the CAMDA
CAMDA project.
project.
pursued
For the
the medical
medical community
community the
the CAMDA
CAMDA project
project has
has the
the potential
potential for
for providing
providing
For
tools
and
techniques
that
greatly
improve
the
quality
of
decision
making
in
medicine.
t.ools and techniques that greatly improve the quality of decision making in medicine.
RACHEL
explicit.ly
considers
uncertainty,
decision
alternatives,
and
patient
preferences
in
RACHEL explicitly considers uncertainty, decision alternatives, and patient preferences in
developing
recommendations.
The
objective
is
to
develop
insight
and
understanding
The objective is to develop insight and understanding
developing recommendations.
regarding t,radeoffs
tradeoffs and
and alternatives,
alternatives, both
both for
for the
the patient
patient and
and the
the attending
attending physician.
physician.
rega.rding
SUMEX-AIM provides
provides aa unique
unique resource
resource for
for the
the continuation
continuation of
of the
the CAMDA
CAMDA
SUMEX-AIM
project.
The
available
computing
resources,
plus
access
to
the
Stanford
AI
and
medical
project. The available computing resources, plus
to the Stanford AI and medical
communities
are
of
critical
importance
for
the
successful
completion
of
the
research.
communities are of critical importance for the successful completion of the research.
.
III.C
Needs
and
Plans
for
other
Computing
Resources
beuond
SUAIEX-AIM
III.C Needs and Plans for other Computing Resources beyond SUMEX-AIM
\Ve are
are pursuing
pursuing the
the purchase
purchase or
or donation
donation of
of several
several computing
computing resources
resources for
for
We
the Decisions
Decisions System
System Laboratory.
Laboratory. Our
Our primary
primary need at
at present
present is for
for a
inst,allation in
in the
installation
LISP machine
machine (e.g.,
(e.g., Symbolics
Symbolics 3600),
3600), enabling
enabling us to
to perform
perform local
local processing
processing and
and increase
increase
LISP
our graphics
graphics capabilities.
capabilities.
our
At present
present the
the project
access to
to one MAD-l
MAD-l personal
personal computer
computer (IBM-PC
(IBM-PC type).
type).
At
project has access
\Ve are considering
considering various
various other
other PC/workstation
PC/workstation facilities
facilities to
to use as front
front ends for
for
We
CAMDA products.
products.
CAA4DA

IlI.D Recommendations
Recommendations for
for Future
Future Community
and Resource
Resource Development
Development
III.D
Community and
Increases in distributed
distributed computing
computing capabilities
capabilities on the
the SUMEX-AIM
SUMEX-AIM system
system is a
Increases
primary
II.C.l, distributed
distributed file
file editing
editing and
and
primary need at this
this point.
point. As we mentioned
mentioned in Section
Section II.C.l,
graphics
the mainframe.
mainframe. At
At this
this time,
time,
gra.phics capabilities
capabilities would
would simultaneously
simultaneously reduce load on the
we are pn.rtiClllarly
environment where
where a
particularly interested
interested in the possibility
possibility of designing
designing an environment
cen~ralized
interact at
at a high
high level
level with
with a
centralized processor
processor (such
(such as the
the SUMEX
SUMEX 20/60)
20/60) would
would interact
mllch
much less
less powerful
powerful dedicated
dedicated processor
processor (such as
as a SUN
SUN workstation,
workstation, or
or an Apple
Apple Lisa
Lisa or
or
Macintosh)
A?acint.osh) wit,h
with specific
specific capabilities
capabilities such as
as bit-mapped
bit-mapped graphics
graphics and special
special purpose
purpose
hardware.
hardware.

E.
E. A.
A. Feigenbaum
Frigenbsum

174
174

5P41 RR00785-11
RR00785-11
5P41

MENTOR Project
Project
MENTOR

ll.A.3.2. MENTOR
MENTOR Project
Project
II.A.3.2.
MENTOR Project
Project
MENTOR
Stuart M.
M. Speedie,
Speedie, Ph.D.
Ph.D.
Stuart
Terrence F.
F. Blaschke,
Blaschke, M.D.
M.D.
Terrence
Department of
or Medicine
Medicine
Department
Division of
of Clinical
Clinical Pharmacology
Pharmacology
Division
Stanford University
University
Stanford

SUM1viARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
I.I. SUMMARY
.'L Project
Projat Rationale
Rationale
A.
The goal of
of the
the MENTOR
MENTOR (Medical
(Medical EvaluatioN
EvaluatioN of
of Therapeutic
Therapeutic ORders)
ORders) project
project is
The
to design
design and
and develop
develop an expert
expert system
system for
for monitoring
monitoring drug
drug therapy
therapy for
for hospitalized
hospitalized
to
patients t,hat
that will
will provide
provide appropriate
appropriate advice
advice to
to physicians
physicians concerning
concerning the
the existence
existence and
and
patients
management of
of adverse
adverse drug
drug reactions.
reactions. The
The computer
computer as a recording-keeping
recording-keeping device
device is
management
becoming increasingly
increasingly common
common in hospital-based
hospital-based health
health care, but
but much
much of
of its
its potential
potential
becoming
remains unrealized.
unrealized. Furthermore,
Furthermore, this
this information
information is provided
provided to
to the
the physician
physician in
in the
the
remains
form
of
raw
data
which
is
often
difficult
to
interpret.
The
wealth
of
raw
data
may
The wealth of raw data may
form of raw data which
often difficult to interpret.
effectively
important information
the patient
patient from
from the
the physician.
physician. This
This is
effectively hide important
information about
a.bout the
part.icularly true
by
particularly
true with
with respect
respect to
to adverse reactions
reactions to
to drugs
drugs which
which can only
only be detected
detected by
simultaneous
simultaneous examinations
examinations of
of several
several different
different types
types of
of data
data including
including drug
drug data,
data,
laboratory
laboratory tests and clinical
clinical signs.
In
In order
order to
to detect
detect and appropriately
appropriately manage
manage adverse drug
drug reactions,
reactions, sophisticated
sophisticated
medical
medical knowledge
knowledge and
and problem
problem solving
solving is required.
required. Expert
Expert systems
systems offer
offer the possibility
possibility of
of
embedding
embedding this
this expertise
expertise in a computer
computer system.
system. Such a system
system could
could automatically
automatically gather
gather
the approprhte
appropriate information
information from
from existing
existing record-keeping
record-keeping systems
systems and continually
continually monitor
monitor
for
for t.lle
t,he occurrence
occurrence of adverse drug
drug reactions.
reactions. Based on a knowledge
knowledge base of
of relevant
relevant data,
data,
it
it, could
could analyze
a.nalyze incoming
incoming data
data and inform
inform physicians
physicians when
when adverse reactions
reactions are likely
likely to
occur
The MENTOR
MENTOR project
project is an attempt
attempt to explore
explore the
occur or when they
they have occurred.
occurred. The
problems
problems associated with
with the development
development and implementation
implementation of such a system
system and to
implement
implement a prototype
protot,ype of a drug
drug monitoring
monitoring system
system in a hospital
hospital setting.
setting.

B.
R. Medical
Medicnl Relevance
Relevance and
and Collaboration
Collaboration
A number
number of independent
independent studies
studies have confirmed
confirmed that
that the incidence
incidence of adverse
reactions
reactions t,o
to drugs
drugs in hospitalized
hospitalized patients
patients is
is significant
significant and that
that they
they are for the most
most
part
pa.rt preventable.
preventable. Moreover,
Moreover, such statistics
statistics do not
not include
include instances
instances of suboptimal
suboptima,l drug
drug
t.herapy
extended length-of-stay,
length-of-stay, or ineffective
ineffective
therapy which
which may
may result
result in increased costs, extended
therapy.
thera.py. Data
Data in these areas are sparse, though
though medical
medical care evaluations
evaluations carried
carried out
out as
as
part
of
hospital
quality
assurance
programs
suggest
that
suboptimal
therapy
is
common.
part of hospital quality assurance programs suggest that suboptimal therapy is common.
Other.
Other computer
computer systems
systems have been
been developed
developed to influence
influence physician
physician decision
decision
making
by
monitoring
patient
data
and
providing
feedback.
However,
most of
of these
these
making
monitoring patient data and providing feedback. However, most
~ystems
This shortcoming
shortcoming involves
involves the
systems suffer
suffer from
from aa significant
significant structural
structural shortcoming.
shortcoming. This
evaluation
In all
all cases,
cases, these criteria
criteria consist
consist of
of
evaluation rules that
that are
are used to generate
generate feedback.
feedback. In
discrete,
discrete, independent
independent rules.
rules. Yet,
Yet, medical
medical decision
decision making
making isis aa complex
complex process
process in
in which
which
many
many factors
factors are
are interrelated.
interrelated. Thus
Thus attempting
attempting to
to represent
represent medical
medical decision-making
decision-making as
as aa

175
175

E.
E. A.
A. Feigenbaum
Feigenbaum

5P41
11
5P41 RR00785RR00785-11

MENTOR Project
Project
MENTOR

discrete set
set of
of independent
independent rules,
rules, no
no matter
matter how
how complex,
complex, is a task
task that
that can,
can, at
at best,
best,
discrete
result in
in a first
first order
order approximation
approximation of
of the
the process. This
This places an inherent
inherent limitation
limitation on
on
result
the quality
quality of
of feedback
feedback that
that can
can be provided.
provided. As
As a consequence itit is extremely
extremely difficult
difficult to
to
the
develop feedback
feedback that
that explicitly
explicitly takes
takes into
into account
account all
all information
information available
available on
on the
the
develop
patient. One
One might
might speculate
speculate that
that the
the lack
lack of
of widespread
widespread acceptance
acceptance of
of such
such systems
systems may
may
patient.
be due to
to the
the fact
fact that
that their
their recommendations
recommendations are often
often rejected
rejected by
by physicians.
physicians. These
These
systems must
must be made
made more
more valid
valid ifif they
they are to
to enjoy
enjoy widespread
widespread acceptance
acceptance among
among
systems
physicians.
physicians.
The proposed
proposed MENTOR
MENTOR system
system is designed
designed to
to address the
the significant
significant problem
problem of
of
The
adverse drug
drug reactions
reactions by
by means of
of a computer-based
computer-based monitoring
monitoring and
and feedback
feedback system
system to
to
adverse
inflnence physician
physician decision-making.
decision-making. ItIt will
will employ
employ principles
principles of
of artificial
artificial intelligence
intelligence to
to
influence
create a more
more valid
valid system
system for
for evaluating
evaluating therapeutic
therapeutic decision-making.
decision-making.
create
The work
work in the
the MENTOR
MENTOR project
project is intended
intended to
to be a collaboration
collaboration between
between Dr.
Dr.
The
at Stanford
Stanford and
and Dr.
Dr. Speedie at
at the
the University
University of
of Maryland.
Maryland. Dr.
Dr. Speedie is
Blaschke 3.t
Blaschke
spending t,he
the 1983-84
academic year
year on sabbatical
sabbatical with
with Dr.
Dr. Blaschke
Blaschke in the
the Division
Division of
of
spending
1983-8-I academic
Pharmacology at
at Stanford
University. While
While at
at Stanford,
Stanford, Dr.
Dr. Speedie has been
Clinical Pharmacology
Clinical
Stanford University.
strengthening his expertise
expert.ise in the
the area
area of
of artificial
artificial intelligence
intelligence and establishing
establishing links
links in the
the
strengthening
AI community.
community. Dr.
Dr. Speedie has begun
begun work
work on the
the development
development of
of the
the MENTOR
MENTOR system
system
AI
pilot project
project on the
t.he SUMEX-AIM
SUMEX-AIM facility.
facility. Over
Over the
the past
past nine months,
months, Drs.
Drs. Blaschke
Blaschke and
and
pilot
have worked
worked closely
closely together
together to
to design the
the MENTOR
MENTOR project.
project. The
The blend
blend of
of
Speedie ha.ve
previous experience,
experience. medical
medical knowledge,
knowledge, computer
computer science knowledge
knowledge and evaluation
evaluation
previous
expertise they
they represent
represent is vital
vital to
to the
the successful
successful completion
completion of
of the
the activities
activities in the
the
design expert,ise
MENTOR project.
project.
h!ENTOR

C. Highlights
Highlights of
Research fiogrcsa
Progress
of Research
The
The MENTOR
MENTOR project
project was initiated
initiated in December
December 1983. The
The work
work to date
date has
consisted
National Center
consisted of
of preparation
preparation of
of a grant
grant proposal
proposal for
for the National
Center for
for Health
Health Services
Research and initial
initia.1 exploration
exploration of
of the problem
problem of
of designing
designing the MENTOR
MENTOR system.
system. Work
Work
has begun
begun on constructing
constructing a system
system for
for monitoring
monitoring potassium
potassium in patients
patients with
with drug
drug
therapy
therapy that
thjt can adversely
adversely affect
affect potassium
potassium levels.

E. Funding
Funding Support
E.
Support
Application
Applica.tion for
for grant
grant support
support is pending.
pending.

ll.
WITH
RESOURCE
II. INTERACTIONS
INTERACTIONS
WITH THE
THE SUMEX-AIM
SUMEX-AIM
RESOURCE
A
i\ledical Collabor.ations
via
.-I...•Medical
Collaborations and
and Program
Rogram Dissemination
Dissemination
via SUMEX
SUMEX
This
This project
project represents
represents a collaboration
collaboration between
between faculty
faculty at Stanford
Stanford University
University
Medical
hizdical Cent.er
Center and t.he
the University
University of
of Maryland
Maryland School of
of Pharmacy
Pharmacy in exploring
exploring
SUMEX, through
through its communications
communications
computer-based
SUMEX,
computer-based monitoring
monitoring of
of drug
drug therapy.
therapy.
returns to the University
University of
of
capabilities.
capabilities, will
will facilitate
facilitate this
this collaboration
collaboration when
when Dr.
Dr. Speedie returns
Maryland
Maryland in August
August of
of 1984.

B. Sharing
Projects
Sharing and
and Interactions
Inteiactions with
with Other
Other SUAIEX-AIM
SUMEX-MM
Projects
Interactions
projects
Interactions with
with other
other SUMEX-AIM
SUMEX-AIM
projects has been on an informal
informal basis.
bcasis.
Personal
Personal contacts
contacts have been made with
with individuals
individuals working
working on the ONCOCIN
ONCOCIN project
project
concerning
concerning issues
issues related
related to the formulation
formulation of the previously
previously mentioned
mentioned proposal.
proposal. We
expect
expect interactions
interactions with
with other
other projects
projects to increase significantly
significantly once the groundwork
groundwork has

E. A.
‘4. Feigenbaum
Feigenbaum

176
176

MENTOR
hlENTOR Project
Project

SPH
SF’41 RR00785-11
RRO0785-11

been
been laid
laid and
and issues
issues directly
directly related
related to
to AI
AI are
are being
being addressed.
addressed. Given
Given the
the geographic
geographic
separation
separation of
of the
the investigators,
investigators, the
the ability
ability to
to exchange
exchange mail
mail and
and programs
programs via
via the
the SUt\lEX
SUMEX
system
system as
as well
we!! as
as communicate
communicate with
with other
other SUMEX-AIM
SUMEX-AIM projects
projects isis vital
vita! to
to the
the success
success of
of
the
ect.
t’he proj
project.

C.
C. Critique
Critique of
of Resource
Resource AJanagement
A4anagement
To
To date,
dn.te, the resources of
of SUMEX
SUMEX have been
been fully
fully adequate
adequate for
for the needs
needs
project.
The staff
staff have been
been most helpful
helpful with
with any
any problems
problems we have had and
and
project. The
fully
fully sat.isfied
sat,isfied with
with the current
current resource management.
management. The
The only
only concern
concern we have
to the
t.he stat·e
st.8t.eof the documentation
documentation on the system.

of
of this
we are
are
relates

ill.
PLANS
III. RESEARCH
RESEARCH
PLANS
A. Project
Project Goals
Gouls and
and Plans
Plans
To
To accomplish
accomplish the goa.ls
goa.!s described in the Project
Project Rationale,
Rationale, aa number
number of tasks will
will
be
be underta.ken.
underta.ken. The short-term
short-term task
task is to develop an initial
initial prototype
prototype of the medical
medical
knowledge
knowledge base
base and inference mechanisms
mechanisms for arriving
arriving at appropriate
appropriate therapy
therapy monitoring
monitoring
decisions. This
This initial
init,ial work
work focuses
focuses on monitoring
monitoring for
for hyperkalemia
hyperkalemia and the decisionmaking
making process wit.h
with respect to ordering
ordering potassium
potassium levels. We will
will then
then attempt
attempt to
construct
const,ruct a system
system combining
combining frames and rules that
that will
will model
mode! this
this process. The
The purpose
purpose
of this
this initial
initial exercise is to explore
explore the problems
problems involved
involved in constructing
constructing an AI
AI system
system
that
that me('ts
meet)s the needs
needs of drug
drug therapy
therapy monitoring
monitoring and to establish
establish development
development guidelines
guidelines
for
ect.
for the larger
larger proj
project.
The
project depend on the outcome
The long-range
long-range plans for
for the
the MENTOR
MENTOR project
outcome of
of the
funding
project has the
funding decision. However,
However, assuming
assuming a favorable
favorable decision,
decision, the full
full project
following
following goals:
1.
prototype computer
patient drug
1. Implement
Implement a prototype
computer system
system to
to continuously
continuously monitor
monitor patient
drug

thernpy
t,hera.py in a hospital
hospit,al setting.
setting. This
This will
will be an expert
expert system
system that
that will
will use
use a
modular,
frame-oriented
form
of
medical
knowledge,
separate
inference
frame-orient.ed
form
of
medical
knowledge,
a
separate
inference
modular,
engin(' for
for applying
applying the
th(' knowledge
knowledge to
to specific
specific situations
situations and automated
automated
engine
collection of
of data
data from
from hospital
hospital information
information systems to
to produce
produce therapeutic
therapeutic
collectlion
ad visories.
advisories.
2. Select a small
small number
number of
of important
important and frequently
frequently occurring
occurring medical
medical settings
settings
!?.
combination therapy
therapy with
with cardiac
cardiac glycosides
glycosides and di:retics)
diuretics) that
that can lead
(e.g., combination
to therapeutic
therapeut.ic misa,dventures,
misadventures, construct
construct a comprehensive
comprehensive medical
medical knowledge
knowledge
to
base necessary to
to detect
detect these situations
situations using
using the
the information
information typically
typically found
found
b:lse
comput.erized hospital
hospital information
information system
system and generate
generate timely
timely advisories
advisories
in a comput,erized
intended to
to a!t,er
alter behavior
behavior and
and avoid
avoid preventable
preventable drug
drug reactions.
reactions.
intended
3. Seiect
Seiect and test
test several
several methods
methods of
of formulating
formulating and
and providing
providing
physicians
in
order
to
find
an
optimal
method
of
feedback
that
physicians in order to find
optimal method of feedback that
and useful
useful to
to physicians
physicians and
and is feasible to
to implement.
implement.
and

advisories to
to
advisories
is acceptable
acceptable

Design and
and begin
begin to
to implement
implement an evaluation
evaluation of
of the
the impact
impact of
of the
the prototype
prototype
4. Design
MENTOR system
system on
on physicians’
physicians' therapeutic
therapeutic decision-making
decision-making as well
well as on
on
MENTOR
outcome measures
measures related
related to
to patient
patient health
health and
and costs of
of care.
outcome

B. .Justification
Justification and
and Requirements
Requirements for
for Continued
Continued StJMEX
SUMEX Use
B.
This project
project needs
needs continued
continued use
use of
of the
the SUMEX
SUMEX facilities
facilities for
for two
two reasons.
reasons. First’
First is
is
This
that itit provides
provides access
access to
to an
an environment
environment specifically
specifically designed
designed for
for the
the development
development of
of AI
AI
that

177

E. A.
A. Feigenbaum
Feigenbaum
E.

rvtENTOR
MENTOR Project
Project

RR00785-11
5P41 RR00785-11

systems. The
The MENTOR
MENTOR project
project focuses
focuses on the development
development of
of such as
as system
system for
for drug
drug
monitoring that
t,hat will
will explore
explore some neglected
neglected aspects of
of AI
AI in medicine.
medicine. Access to
to SUMEX
SUMEX
monitoring
is necessary for
for t.imely
t,ime!y development
development of
of the MENTOR
MENTOR system,
system, as
as well
well as advice and
a£sistance
assistance in the design and development
development of
of a well-designed
well-designed and efficient
efficient system.
system. Access
to SUMEX
SUMEX is also necessary to
to support
support the collaborative
collaborative effort
effort in this
this project
project as described
described
to
previously.
previously.

and Pfans
Other Computing
Computing Resources
SUMEX-AIM
C. Needs
Needs and
Plans for
for Other
Resources Beyond
Beyond SUAIEX-AIM
A
A major
major long-range
long-range goal of
of the MENTOR
MENTOR project
project is to
to implement
implement this
this system
system on a
independent
It
independent hardware
hardware system
system of
of suitable
suitable architecture.
architecture.
It is recognized
recognized that
that the full
full
monitoring
monit.oring system
system will
will require
require a large patient
patient data
data base
base as well
well as
as a sizeable medical
medical
knowledge base
base and must
must operate
operate on a close to
to real-time
real-time basis. Ultimately,
Ultimately, the SUMEX
SUMEX
knowledge
facilil·ies
Thus
facilities will
will not
not be s11itable
suitable for
for these applications.
applications.
Thus we intend
intend to transport
transport the
the
prototype
prot.otype system
system to
to a dedicated
dedicated hardware
hardware system
system that
that can fully
fully support
support the the planned
planned
system
system and which
which can be integrated
integrated into
into the SUMe
SUMC Hospital
Hospital Information
Information System.
System.
However, no firm
firm decisions have been made about
about the requirements
requirements for
for this
this system
system since
However,
many
many specification
specification and design decisions remain
remain to
to be made.

D. Recommendations
Recommendations for
for Future
Future Community
Resource Development
Development
D.
Community and
and Resource
In the brief
brief time
time we have been associated with
with SUMEX,
SUMEX, we have been generally
generally
In
p!ca,sed with
with the facilities
facilities and services. However,
However, it
it is evident
evident that
that disk
disk space is a critical
critical
pleased
factor in the functioning
functioning of
of the facility.
facility. It
factor
It would
would seem wise to
to increase disk
disk storage
storage in
order to
to meet
meet the needs of
of the users. Our
Our experience also indicates
indicates that
that an attempt
attempt, needs
order
to
with
to be made to organize
organize and update
update the
the documentation
documentation associated
associated
with the
the various
various
SUMEX systems. Being
Being new users, we found
found that
that paths
paths to
to useful
useful software
software was somewhat
somewhat
SUMEX
longer
longer t.han
than one might
might, expect.
expect. An
An expanded
expanded introduction
introduction to the system
system that,
that, at
at least,
briefly
briefly described
described the software
software available
available on SUMEX
SUMEX would
would be useful.

E. A. Feigenbaum
Feigenbaum

178

Protein
Protein Secondary
Secondary Structure
Structure Project
Project,

5P41
5P41 RH00785-11
RR00785-11

ll.A.3.3.
Protein
Secondary
Structure
Project
lI.A.3.3.
Protein
Secondary
Structure
Project

Protein
Project
Protein Secondary
Secondary Structure
Structure
Project
Robert
M.D.
Robert M.
M. Abarbanel,
Abarbanel,
M.D.
Section
Science
Section on
on Medical
Medical Information
Information
Science
University
of
Medical
University
of California
California
Medical Center
Center
University
of
at
University
of California
California
at San
San Francisco
Francieco

I.
I.

SUM1vL<\.RY
PROGRAM
SXJMMARY OF
OF RESEARCH
RESEARCH
PROGRAM
A.
A. Projcct
Project Rationalc
Rationale

Development
Development of a protein
protein structure
structure knowledge
knowledge base
base and tools for
for manipulation
manipulation of
that
knowledge
to
aid
in
the
investigation
of
new
structures.
System
that knowledge
investigation
structures.
System to include
include
cooperating
cooperat,ing knowledge
knowledge sources that
that work
work under
under the guidance
guidance of other
other system
system drivers
drivers to
find
Evaluations of structure
structure predictions
predictions using
find solutions
solutions to protein
protein structure
st,ructure problems.
problems. Evaluations
known
known proteins
prot,eins and other
other user feedbacks available
available to aid user in developing
developing new methods
methods
of prediction.
prediction.

B. Afcdical
Medical Relcvancc
Relevance and
wad Collaboration
Collaboration
!\!any
Many important
important proteins
proteins have been sequenced
sequenced but
but have not,
not, as
as yet,
yet, had their
their
secondary
The systems
systems developed here would
would aid medical
medical
secondary or
or tertiary
tertiary structures
structures revealed.
revealed. The
scientists
particular configurations,
scientists in the search for
for particular
configurations, for
for example,
example, around
around the active
active sites
in enzymes. Predictions
Predictions of
of secondary
secondary structure
structure will
will aid in the determination
determination of
of the
the full
full
"natural"
configuration
of
important
biological
materials.
Development
of
systems
“natural” configuration of important biological materials. Development of systems such
3S
as these will
will contribute
cont,ribute to
to our
our knowledge
knowledge of
of medical
medical scientific
scientific data
data representation
representation and
retrieval.
retrieval.

Highlights of
of Research
Rcsearch Progress
Progrcss
C. Nighlights
The prediction
prediction of
of beta-alpha
beta-alpha protein
protein structures
structures is complete.
complete. The
The system
system was
The
developed on a.
a VAX
VAX 11
/750 at
at the University
University of
of California,
California, San Francisco,
Francisco, to
to allow
allow
developed
111750
rese3Tchers to
to describe
describe patterns
patterns of
of amino
amino acid
acid residues that
that will
will be sought
sought in the
the sequences
sequences
researchers
under st*udy.
study. The
The presence or
or absence of
of these “primary”
"primary" patterns
patterns are then
then combined
combined
under
with ot’
otht'r
measures of
of structure,
structure, like
like hydrophobicity,
hydrophobicity, to
to suggest possible alpha
alpha helix
helix or
or
wit,11
her measnres
beta sheet
sheel or
or turn
turn configurations.
configurations.
beta
The segments
segments of
of a sequence
sequence between
between turns
turns are then
then analyzed
analyzed to
to determine
determine the
the
The
allowable extent
extent of
of the
the possible
possible secondary
secondary structure
structure assignments.
assignments. Any
Any segments
segments remaining
remaining
allowable
to generate
generate al!
all possible
possible complete
complete structures.
structures. Only
Only two
two beta
beta strands
strands with
with the
the
are t.hen used to
character of
of sheet edges are allowed
allowed in
in any
any prediction.
prediction. This
This hierarchical
hierarchical generation
generation and
and
character
pruning result,s
r,~sults in nearly
nearly 95%
95% t,urn
turn prediction
prediction accuracy,
accuracy, and
and excellent
excellent delimiting
delimiting of
of helices
pruning
and sheets. In
In some cases,
cases, one and
and only
only one secondary
secondary structure
structure is predicted.
predicted.
and
Research in
in Progress
Progress --- At
At this
this time,
time, work
work is under
under way
way to
to extend
extend this
this a/P
a/ f3
Research
assignment work
work to
to a set
set of
of cancer
cancer causing
causing viral
viral proteases.
proteases. These
These proteins
proteins are believed
believed t,o
to
.lulsant, B. and
and Servan-Schreiber,
Servan-Schreiber, D.:Knowledge
D.:Knowledge engineering:
engineering:
daily
14.
AA daily
acit'vity on a hospital
hospital ward,
wurd, Computers
Computers in Biomedical
Biomedical Research, 1984.
activity

Spitzer, R. L.,
L., Endicot.t,
Endicott, J. and
and Robins,
Robins, E.: Research
Research diagnostic
diagnostic criteria,
criteria, (2d
(2d
15. Spitzer,
cd.) New
New York
York State
Sta.te Department
Department of
of Mental
Mental Hygiene,
Hygiene, New
New York
York Psychiatric
Psychiatric
ed.)
Institute, Biometrics
Biometrics Research
Research Division,
Division, 1975.
Institute,
Spitzer, R. L.:
L.: (Ed.).Diagnostic
(Ed.).Diugnostic and
and statistical
stutistical manual
manuul of
of mental
mental disorders,
disorders,
16. Spit,zer,
(3d ed.). Washingt,on,
·Washington, D.
D. C.:
C.: American
American Psychiatric
Psychiatric Associa.tion,
Association, 1980.
(3d
17. Vsn
Van Melle,
Melle, W.:The
W.:The EIZffCIN
EJvfYCIN Manual,
Manual, Computer
Computer Science Department,
Department,
Stanford
University,
Report
HPP-81-16,
1981.
Stanford University, Report HPP-81-16,

E. Funding
Funding Support
Support
E.
\Ne have
have
We
at
the
Univ
of
at t,he Univ of

submitted
submitted
Michigan,
hfichiga.n,

an application
application for
for support
support to
to the
the Vice-President
Vice-President for
for Research
Research
who
has
funds
for
"seed
money"
for
faculty
research
(Total
who has funds for “seed money” for faculty research (Total

199

E. A.
A. Feigenbaum
Feigenbaum
E.

5P41 RR00785-11
RR00785-11

RXDX Project
Project
RXDX

Direct
Direct Costs
Costs =
= $3215). \Ve
We have prepared
prepared a grant
grant application,
application, to
to be sent to
to the NIH
NIH
"Small
Program for
for the May
Ma.y I,
1, 1984 deadline
deadline (Total
(Total Direct
Direct Costs =
= $13,850).
$13,850).
“Small Grants"
Grants” Program
These funds
funds should
should enable us to
to gather
gather the pilot
pilot data
data we will
will need as
as part
part of
of a major
major
gran
Iication.
grsnt t app
application.

ll.
WITH
RESOURCE
II. INTERACTIONS
INTERACTIONS
WITH THE
THE SUMEX-.AIM:
SUMEX-AIM
RESOURCE
A.
Collaboration and
and Program
via SUMEX
SUMEX
A. Medicai
AJediwl Collaboration
Program Dissemination
Dissemination via
\Ve
We are collaborating
collaborating via
via SUMEX
SUMEX with
with Dr.
Dr. Jon Heiser, who
who worked
worked with
with Ruven
Ruven
Brooks
Brooks on HEADMED
HEADMED in the late 1970's.
1970’s. We
We are sharing
sharing a common
common SUMEX
SUMEX account,
account, and
Dr. Heiser
Heiser will
will write
write the section
section of
of the expert
expert
communicating
communicating using computer
computer mail.
mail. Dr.
system
system dealing
dealing with
with the treatment
treatment of
of depression (and
(and eventually
eventually of
of other
other psychiatric
psychiatric
disorders)
disorders) while
while Drs. Feinberg
Feinberg and Lindsay
Lindsay work
work on the diagnostic
diagnostic parts
parts of
of the system.
system.

B. Sharing
with
Projects
B.
Sharing and
and Collaboration
Collaloration
with other
other SUAIEX-AIAf
SUMEX-AIM
Projects
\Ve
\%‘e are also collaborating,
collaborating, although
although more
more loosely, with
with Mess~s.
Ivfessrs. Benoit
Benoit Mulsant
Mulsant and
David
David Servan-Schreiber.
Servsn-Schreiber. They
They wrote
wrote an expert
expert system
system (BLUEBOX)
(BLUEBOX) for
for the
the diagnosis
diagnosis and
treatment
treatment of
of depression
depression which
which was a first
first step in the direction
direction we are going.
going. We
We have
access to BLUEBOX
BLUEBOX through
through SUMEX,
SUMEX, and have been able to learn
learn from
from its
its successes
successes and
access
failures.
David will,
will, we expect,
expect, be able to offer
offer us many
many helpful
helpful suggestions
suggestions on
failures. Ben and David
our
our expert
expert system
system (RXDX)
(RXDX) as
as they
they pursue
pursue their
their training
training in Psychiatry
Psychiatry and continue
continue their
their
work in AI
AI in medicine.
medicine.
work

C. Critique
Resource Management
AJanagement
Critique of
of Resource
We
We have been using
using EMYCIN
EMYCIN to
to set up our
our knowledge
knowledge base, and have found
found this
this
program
pr0gra.m invaluable,
invaluable, since it
it has saved us many
many hours
hours of
of programming
programming in LISP.
LISP. There
There are
problems with
with EMYCIN,
EMYCIN, many
many of
of which
which center
center around
around discrepancies
discrepancies between
between the
some problems
the version
version of
of EMYCIN
EMYCIN described
described in the manual
manual and the version
version actually
actually running
running on
SUMEX. vVe
We would
would suggest that
that EMYCIN
EMYCIN be more
more strongly
strongly supported
supported than
than is now
now the
SUMEX.
case. if
if it
it and SUMEX
SUMEX are to
to be even more
more useful
useful to
to beginners
beginners in AI
AI in Medicine.
Medicine. This
This
cast'.
may involve
involve added expense,
expense, such as would
would be involved
involved in the purchase
purchase of
of an updated
updated
may
version
version of
of EMYCIN,
EMYCIN, but
but we would
would certainly
certainly be able to
to make
make use
use of
of the updated
updated version.
version.
We
SUivlEX
SUMEX itself
itself has been invaluable.
invaluable.
We don't
don’t have easy access
access to any
any other
other
machine
ma.chine of
of equal
qua.1 computing
computing power
power which
which also has a strongly
strongly supported
supported LISP
LISP available.
available.
Specifically,
Specifically, the Dandelion
Dandelion LISP
LISP machine
machine at Michigan
Michigan is not
not easily
easily accessible, while
while t.he
t.he
LISP compiler
compiler available
available on the Amdahl
Amdahl 5860 here differs
differs from
from those used at major
major AI
AI
LISP
centers such as
as Stanford
Stanford and MIT.
MIT. We
We have also made good use
use of
of the ARPANET
ARPANET
centers
connections
connections that
that SU~IEX
SUMEX offers.
offers. Feinberg
Feinberg will
will spend a month
month of
of his sabbatical
sabbatical working
working
with Prof.
Prof. Peter
Peter Szolovits
Szolovits at
at MIT,
MIT, learning
learning about
about AI
AI in Medicine.
Medicine. (This
(This is an
a;n obvious
obvious and
with
necessary step for
for any
any physician
physician wanting
wanting to
to begin work
work in the
the field.)
field.) This
This visit
visit was
Lindsay and Feinberg
Feinberg will
will be able to
arranged
arranged using computer
computer mail
mail through
through SUMEX.
SUMEX. Lindsay
continue
continue t,heir
their collaborative
collaborative work
work while
while the latter
latter is in Cambridge,
Cambridge, using
using the same
medium.
medium. The
The alternative
alternative would
would be days lost
lost in the mails
mails and many
many dollars
dollars spent
spent on
phone calls. vVe
We have also been able to
to get rapid
rapid help with
with problems
problems that
that arise with
with
Er-.-IYCIN
EMYCIN using computer
computer mail,
mail, saving
saving days and/or
and/or dollars.
dollars.

ill.
III. RESEARCH
RESEARCH PLAN
PLAN

A.
Goals and
and Plans
A.. Project
Project Goals
Plans
Our immediate
immediate objective
objective is to
to develop an expert
expert system
system which
which can
cau differentiate
differentiate
Our
patients
pat.ient,s with
wit,11 the various
various subtypes
subtypes of
of depressive disorder,
disorder, and prescribe
prescribe appropriate
appropriate
Feigenbaum
E. A. Feigenbaum

200

RXDX
RXDX Project
Project

5P41
SP41 RR0078.')-1l
RR0078.5-11

treatment.
This
This system
system should
should perform
perform at
at about
about the
the level
level of
of aa board-certified
board-certified
treatment.
psychiatrist,
psychia.t.rist, i.e.
i.e. better
better than
than an
an average
average resident
resident but
but not
not as
as well
well as
as aa human
human expert
expert in
in
depression.
depression. Eventually,
Eventually, we
we plan
plan to
to enlarge
enlarge the
the knowledge
knowledge base
base so
so that
that the
the expert
expert system
system
can
can diagnose
diagnose and
and prescribe
prescribe for
for aa wider
wider range
range of
of psychiatric
psychiat)ric patients,
patients, particularly
particularly those
those
with
with illnesses
illnesses which
which are
are likely
likely to
to respond
respond to
to psychopharmacological
psychopharmacological agents. We will
will
design
design the
the system
syst,em so
so that
that itit could
could be
be used
used by
by non-medical
non-medical clinicians
clinicians or by non-psychiatrist
non-psychiat.rist
MD's
MD’s as
as an
an adjunct
adjunct to
to consultation
consultation with
with aa human
human expert.
expert.

B.
. ~tification and
for continued
continued SUMEX
StJMEX use
R. Ju
Justification
and Requirements
Requirements for
This
This project
project is
is entirely
entirely dependent
dependent on access
access to SUMEX.
SUMEX. We are using the EMYCIN
EMYCIN
syst.em
on
SUMEX.
That
software
is
not
available
to
us anywhere
system
SUMEX. That software is
available
anywhere else.
else. We also make
extensive use
EX as
use of SUM
SUMEX
as aa means of communication
communication and file-sharing
file-sharing with
with our
our
consnltant"
consulta.nt3, Jon Heiser, and with
with David
David Mulsant
Mulsant and Benoit
Benoit Servan-Schreiber.
Servan-Schreiber. The
The access
access
to SUMEX
S’CJMEX resources isis essentially
essentially our
our sale
sole means of maintaining
maintaining contact
contact with
with the
community
community of researchers working
working on applications
applications of AI
AI in medicine.
medicine.
\Ve
We ant.icipate
ant,icipate that
that our
our requirements
requirements for computing
computing time
time and file space
space will
will
continue
continue to
t,o grow
grow as
as the system
system evolves.

C. Needs and
and Plans
Plans for
for Other
Other Computing
Computing Resources
Resources
As our
our project
project evolves and we run
run into
into the limitations
limitations of
of EMYCIN
EMYCIN and the timetimeshared SUMEX
SUMEX facility,
facility, we anticipate
anticipate employing
employing different
different expert
expert systems software.
software. At
At
this
this time,
time, we are not
not at a stage to say exactly
exactly what
what that
that will
will be, but
but our
our project
project is not
not
sufficiently
sufficient,ly large that
that we will
will be able to mount
mount such a software
software development
development project
project
ourselves, so we will
depend
on
development
and
support
elsewhere.
Ultimately,
when our
our
will
development
support
Ultimately, when
consultant
is
made
available
for
field
trials
and
clinical
use,
it
will
need
to
be
transported
consulta.nt
available for field trials
clinical use, it will
to
transported
to a personal
yet inexpensive
inexpensive enough
enough to
to
personal computer
computer that
that is large enough support
support the system
system yet
be widely
\Vhile current
current prices
prices of
of the
widely available.
ava.ilable. A
A LISP
LISP machine
machine is an obvious
obvious candidate.
candidate. While
necessa.ry
to drop.
drop. Our
Our design
necessa.ry hardware
hardware are too
too high,
high, computer
computer prices are continuing
continuing to
st.rategy
to that
that which
which is affordable
affordable
st,rategy is to
t,o avoid
avoid limiting
limiting ourselves and our
our aspirations
aspirations to
today; instead
inst.ead we will
will attempt
attempt to
to project
project the growth
growth of
of our
our project
project and the
the pricepricetoday:
performance curve
curve of
of computing
computing such that
that they
they meet at
at some reasonable point
point in the
performance
fut.nre.
future.

D. Recommendations
Recommendations for
for Future
Future Community
and Resource
Resource Development
Development
D.
Community and
Valuable as
a.s the
the present
present SUMEX
SUMEX facilities
facilities are to
to us, they
they are in
in many
many ways
ways limited
limited
Valuable
and awkward
awkward to
to use. The
The need for
for more
more and more
more computer
computer cycles and memory
memory
cont.inues to
to grow,
grow, of
of course. However,
However, the
the major
major limitation
limitation we feel is the
the difficulty
difficulty and
and
continues
sometimes t,hr
the impossibility
impossibility of
of making
making contact
contact with
with everyone
everyone who
who could
could be of
of value
value to
to us.
sometimes
greater emphasis will
will be put
put on internetwork
internetwork gateways.
gateways. ItIt is important
important not
not
\Vee hope that
that, greater
W’
only t,o
t.o establish
est.ablish more
more of
of these, but
but to
to develop
develop consistent
consist.ent and
and convenient
convenient standards
standards for
for
only
elert.ronic mail,
mail, electronic
electronic file
file transfers,
transfers, graphic
graphic information
information transfer,
transfer, national
national archives
archives
elect.ronic
and dsta
data ba.ses,
ha:oes, and personal
personal filing
filing and
and retrieval
retrieval (categorization)
(categorization) systems.
systems. The
The present
present
and
state of
of the
the art
art is quite
quite limiting,
limiting, now
now that
that the
the basic concepts
concepts of
of comput.er
computer networking
networking have
st.ste
available and
and have proved
proved their
their potential.
potential.
become available

201

E. A.
A. Feigenbsum
Feigenbaum
E.

Books, Papers, and Abstracts
Abstra.cts

5P41
5P41 RR00785-11
RR00785-11

II.B.
and
ILB. Books,
Books, Papers,
Papers,
and Abstracts
Abstracts
Publications
Publications for the various
various collaborative
collaborative projects
projects are summarized
summarized in their
their
respective
They also have been submitted
submitted separately
separately on the Scientific
Scientific
respective progress reports.
reports. They
Subproject
Subproject Form
Form lIB.
IIB. They
They are not
not reproduced
reproduced here to avoid
avoid redundancy.
redundancy.

IT.C.
Resource
Summary
Table
KC.
Resource
Summary
Table
Detailed
is
Detailed resource usage
usage information
information
is summarized
summarized starting
starting on page 30.
Tabulations
of
this
information
also
have
been
submitted
separately
Tabulations
this information
submitted separately on the requested
request.ed
Scientific
Subproject
Form.
These
are
not
reproduced
Scientific Subproject Form.
not reproduced here to avoid
avoid redundancy.
redundancy.

A. Fcigenbaum
Feigenbanm
E. A.

202
202

Appendix
Appendix A
A

SP41
5P41 RR0078S-11
RR00785-11

Appendix
A
Appendix
A
AIM
Committee
Membership
AIM Management
Management
Committee
Membership
Following
are the
Following
management
mana.gement committees:
committees:

current
current

membership
membership

lists
lists

of
of

ALr..J Erecutiue
Executive Committee:
AIM
Committee:
FEIGENBAUM,
FEIGENBAUM, Edward
E.dward A.,
A., Ph.D.
Ph.D. (Chairman)
(Chairman)
Principal
Principal Investigator
Investigator - SUMEX
SUMEX
Heuristic
Heuristic Programming
Programming Project
Project
Department
Department of
of Computer
Computer Science
Margaret
Margaret Jacks Hall
Hall
Stanford
Stanford University
University
Stanford.
Stanford, California
California 94305
(415) 497-4879
LEDERBERG,
LEDERBERG, Joshua, Ph.D.
Ph.D.
President
President
The
The Rockefeller
Rockefeller University
University
1230 York
York Avenue
Avenue
New York,
New York
New
York, New
York 10021
10021
(212) 570-8080, 570-8000
KULIKOWSKI,
Casimir,
KULIKOWSKI,
Casimir, Ph.D.
Ph.D.
Department
Department of
of Computer
Computer Science
Rutgers
Rutgers University
University
New Brunswick,
New Jersey 08903
New
Brunswick, New
(201) 932-2005
932-2006

203

the

various
various

SUMEX-AIM
SUMEX-AIM

Appendix AA
Appendix

5P41 RR00785RR00785-11
5P41
1I

LINDB8RG, Donald
Donald A.B.,
A.B., M.D.
M.D.
LINDBERG,
605 Lewis
Lewis Hall
Hall
605
University of
of Missouri
Missouri
University
Columbia, Missouri
Missouri 65201
65201
Columbia,
(314) 882-6966
882-6966
(314)

(Adv Grp
Grp Chrmn)
Chrmn)
(Adv

MYERS, Jack
Jack D.,
D., M.D.
M.D.
MYERS,
School
of
Medicine
School of Medicine
Scaife Hall,
Hall, 1291
1291
Scaife
University
of
Pittsburgh
University of Pittsburgh
Pittsburgh,
Pennsylvania
15261
Pittsburgh, Pennsylvania 15261
(412) 624-2649
624-2649
(412)
SHORTLIFFE, Edward
Edward H.,
H., M.D.,
M.D., Ph.D.
Ph.D.
SHORTLIFFE,
Co-Principal Investigator
Investigator -- SUMEX
SUMEX
Co-Principal
Division of
of General
General Internal
Internal Medicine,
Medicine, TC117
TC 117
Division
Stanford University
University Medical
Medical Center
Center
Stanford
Stanford, California
California 94305
Stanford,
(415) 497-6970
(415)

204
204

5P41
5P41 RR00785-11
RR00785-11

Appendix
Appendix A
A

AIM Advisory
Advisory Group:
AIM
Group:
LINDBERG,
LINDBERG, Donald
Donald A.B.,
A.B., M.D.
M.D.
505
Lewis
Hall
605 Lewis Hall
University
University of Missouri
Missouri
Columbia,
Columbia, Missouri
Missouri 55201
65201
(314)
(314) 882-5955
882-6966

(Chairman)
(Chairman)

AMABEL,
AhIAREL, Saul, Ph.D.
Ph.D.
Depart.ment
Depart,ment of Computer
Computer Science
Science
Rutgers
Rutgers Universit.y
University
New Jersey 08903
New Brunswick,
Brunswick, New
(201)
(20 1) 932-3546
932-3546
BAKER,
(Exec. Secretary)
Secretary)
BAKER, \Villiam
William R., Jr.,
Jr., Ph.D.
Ph.D.
Biotechnology
Resources
Program
Biotechnology Resources Program
National Institutes
National
Institutes of Health
Health
Building
31,
Room
Building
Room 5B43
9000 Rockville
Rockville Pike
Pike
Bethesda, Maryland
Maryland 20205
20205
(301)
(301) 496-5411
496-5411
FEIGENBAUM,
FEIGENBAUM, Edward
Edward A.,
A., Ph.D.
Ph.D. (Ex-officio)
(Ex-officio)
Principal
Principal Investigator
Investigator - SUMEX
SUMEX
Heuristic
Heuristic Programming
Programming Project
Project
Department
Department of Computer
Computer Science
Science
Margaret
Margaret Jacks Hall
Hall
Stanford
Stanford University
University
Stanford,
Stanford, California
California 94305
94305
(415)
(415) 497-4879
497-4879
KULIKOWSKI,
KULIKOWSKI, Casimir,
Casimir, Ph.D.
Ph.D.
Department
Department of Computer
Computer Science
Science
Rutgers
Rut.gers University
University
New Jersey 08903
New Brunswick,
Brunswick, New
(201) 932-2006
932-2006
LEDERBERG,
LEDERBERG, Joshua, Ph.D.
Ph.D.
President
President
The
The Rockefeller
Rockefeller University
University
1230
1230 York
York Avenue
Avenue
New York,
York, New York
York 10021
(212)
(2 12) 570-8080,
570-8080, 5iO-8000
570-8000
MIi\SKY.
MINSKY, Marvin.
Marvin, Ph.D.
Ph.D.
Artificial
Artificial Intelligence
Intelligence Laboratory
Laboratory
Massachusetts
Massachusetts Institute
Institute of
of Technology
Technology
545 Technology
Technology Square
Cambridge.
Cambridge, Massachusett.s
Massachusetts 02139
(517)
(617) 253-5854
253-5864

205

Appendix
Appendix A
A

5P41 RR00785-11
RR00785- 11

f"IOHLER,
MOHLER, William
William C., M.D.
M.D.
Associate
Associate Director
Director
Division
Division of
of Computer
Computer Research and Technology
Technology
National
Institutes
National Institutes of
of Health
Health
Building
Building 12A, Room
Room 3033
9000 Rockville
Rockville Pike
Pike
Bethesda,
Bethesda, Maryland
Maryland 20205
(30l)
(301) 496-1168
MYERS,
MYERS, Jack
Jack D., M.D.
M.D.
School of
of Medicine
Medicine
Scaife Hall,
Hall, 1291
1291
University
University of
of Pittsburgh
Pittsburgh
Pittsburgh,
Pittsburgh, Pennsylvania
Pennsylvania 15261
15261
(412) 624-2549
624-2649
PAUKER,
PAUPER, Stephen
Stephen G., M.D.
M.D.
Department
Department of
of Medicine
-Medicine - Cardiology
Cardiology
Tufts
New England
Tufts New
England Medical
Medical Center
Center Hospital
Hospital
171
171 Harrison
Harrison Avenue
Avenue
Boston,
Boston, Massachusetts
Massachusetts 02111
(517)
(617) 956-591O
956-5910
SHORTLIFFE,
(Ex-officio)
SHORTLIFFE, Edward
Edward H., M.D.,
M.D., Ph.D.
Ph.D.
(Ex-officio)
Co-Principal
Co-Principal Investigator
Investigator - SUMEX
SUMEX
Division
Division of
of General
General Internal
Internal Medicine,
Medicine, TC117
TC117
Stanford
University
Medical
Stanford University Medical Center
Center
Stanford,
Stanford, California
California 94305
(415)
(415) 497-0970
497-6970
SIMON,
SIMON, Herbert
Herbert A.,
A., Ph.D.
Ph.D.
Department
Depa.rt,ment of
of Psychology
Psychology
Baker
Baker Hall,
Hall, 339
Carnegie-Mellon
Carnegie-hlellon University
University
Schepley
Schenley Park
Park
Pit.tsburgh.
Pittsburgh. Pennsylvania
Pennsylvania 15213
(412)
(4 12) 578-2787,
578-2787, 578-2000
578-2000

205
206

Appendix
Appendix A
A

RR00785-11
5P41 RRO0785-11

Stanford
Community Advisory
Advisory Committee:
Committee:
Stan
ford Community
FEIGENBAUM, Edward
Ed'ward A.,
A., Ph.D.
Ph.D. (Chairman)
(Chairman)
FEIGENBAUM,
Heuristic Programming
Programming Project
Project
Heuristic
Department of
of Computer
Computer Science
Department
Margaret Jacks Hail
Hall
Margaret
Stanford University
University
Stanford
Stanford, California
California 94305
Stanford,
(415) 497-4879
(415)
DJERASSI, Carl,
Carl, Ph.D.
Ph.D.
DJERASSI,
Department of
of Chemistry,
Chemistry, Stauffer
Stauffer I-106
1-106
Department
Stanford University
University
Stanford
Stanford, California
California 94305
Stanford,
(415) 497-2783
(415)
~'IAFFLY, Roy
Roy H., M.D.
M.D.
MAFFLY,
of Nephrology
Nephrology
Division of
Division
Veterans Administration
Administration Hospital
Hospital
Veterans
Miranda. Avenue
Avenue
3801 hfiranda
Palo Alto,
Alto, California
California 94304
Palo
8.58-3971
(415) 858-3971

SHORTLIFFE, Edward
Edward H., M.D.,
M.D., Ph.D.
Ph.D.
SHORTLIFFE,
Co-Principal Investigator
Investigator - SUMEX
SUMEX
Co-Principal
Division of
of General
General Internal
Internal Medicine,
Medicine, TC117
TC117
Division
Stanford
Stanford University
University Medical
hjedical Center
Center
Stanford,
Stanford, California
California 94305
(415) 497-5970
497-6970

207
207

5P‘1I RR00785-11
RR00785- 11
5P41

Appendix
Appendix B

Appendix
B
Appendix
B
Community
Growth
and Project
Project
Abstracts
Community
Growth
and
Abstracts
This appendix
appendix contains
contains a graphical
graphical display
display of the development
development of the SUMEX-AIM
SUMEX-AIM
This
community over
over the years and abstracts
abstracts of
of currently
currently active
active projects.
projects. Figure
community
Figure 15
15 below
illustrat,es the substantial
substantial growth
growth in the cumulative
cumulative number
number of projects
projects in the Stanford,
Stanford,
illustrates
Nat,ional AIM,
AIhf, and Rutgers-AIM
Rutgers-AIM communities
communities since the resource began operation
operation in 1974
1974
National
until this
this past year.
year. The
The recent
recent decrease
decrease in the total
total number
number of projects
projects is due to the
up until
severa,l long time
Lime SUMEX-AIM
SUMEX-AIM projects,
projects, namely
namely Dendral,
Dendral, Puff/Vm,
Puff/Vm. Act,
Act, and
closure of several
Protein. Activity
Activity in the community
community however
however remains
remains high, as
as evidenced by the number
number of
Prot,ein.
pilot projects
projects (5
(5 Stanford
SLa.nford pilots,
pilots, 22 Aim
Aim pilots,
pilots, and 11 Rutgers
Rutgers pilot)
pilot) currently
currently active
active in the
pilot
SUMES-AIM community.
community.
SUMCX-AIM

25
25

Numbzr
of Projects
Projects
Number of

Rutgers Projects
Projects
Rutgers

p=-j
4J+------L

National AIM
AIM Projects
Projects
National

~--------~'----

r--------------------~

P974

1975

1976

1977

Stanford Projects

1978

Figure
Figure 15:
15:

1:)79

1930

"l!JS1

19B;?

1D83

198'~

1985

SUh?EX-AIM
SUMEX-AIM Growth
Growth by
by Community
Community

209
:::!09

A. Feigenbaum
Feigenbaum
E. A.

5P41 RR00785-11
RR00785-11

Appendix
Appendix B

National AIM
National
AIM Project:
Project:

CADUCEUS
CADUCEUS (formerly
(formerly INTERNIST)
INTERNIST)

Principal
Principal Investigators:
Investigators:

Jack D. Myers,
Myers, M.D.
M.D. (MYERS@SUMEX-AIM)
(MYERSBSUMEX-AIM)
Harry
Harry E. Pople,
Pople, Ph.D.
Ph.D. (POPLE@SUMEX-AIM)
(POPLEQSUMEX-AIM)
University
University of
of Pittsburgh
Pittsburgh
Pittsburgh,
15261
Pittsburgh, Pennsylvania
Pennsylva.nia 15261
Dr.
Dr. Pople:
Pople: (412) 624-3490
6243490

The major
major goal of
of the CADUCEUS
CADUCEUS Project
Project is to
to produce
produce a reliable
reliable and adequately
adequately
The
complete
complete diagnostic
diagnostic consultative
consultative program
program in the field
field of
of internal
internal medicine.
medicine. Although
Although this
this
program is intended
intended primarily
primarily to
to aid skilled
skilled internists
internists in complicated
complicated medical
medical problems,
problems,
program
the program
program may
may have spin-off
spin-off as a diagnostic
diagnostic and triage
triage aid to
to physicians'
physicians’ assistants,
assistants,
rural health
health clinics,
clinics, military
military medicine
medicine and space travel.
travel. In
In the
the design of
of CADUCEUS
CADUCEUS and
rural
its
I, we have attempted
its predecessor INTERNIST
INTERNIST
attempted to
to model
model the
the creative,
creative, problemproblemformulation
formulation aspect of
of the clinical
clinical reasoning
reasoning process. The
The program
program employs
employs a novel
novel
heuristic procedure
procedure that
that composes differential
differential diagnoses, dynamically,
dynamically, on the basis of
of
heuristic
clinical
clinical evidence. During
During the course of
of a CADUCEUS
CADUCEUS or
or INTERNIST
INTERNIST I consultation,
consultation, it
it is
not
not uncommon
uncommon for
for a number
number of
of such conjectured
conjectured problem
problem foci
foci to
to be proposed
proposed and
investigated, with
with occasional
occasional major
major shifts
shifts taking
taking place in the program's
program’s conceptualization
conceptualization
investigated,
of
of the task
task at hand.
hand.

SOFTWARE
SOFTWARE AVAILABLE
AVAILABLE ON
ON SUMEX
SUMEX

Versions of
of INTERNIST
INTERNIST
available for
for experimental
experimental use,
use, but
but the project
project
Versions
are available
continues
continues to
to be oriented
oriented primarily
primarily towards
towards research and development;
development; hence, a stable
stable
production version
version of
of the system
system is not
not yet
yet available
available for
for general
general use.
use.
production

REFERENCES
REFERENCES
Pople,
DIALOG model
The DIALOG
model of
of diagnostic
diagnostic
Pople, H.E.,
H.E., Myers,
Myers, J.D. and Miller,
Miller, R.A.:
R.A.: The
logic
logic and
and its
its use in
in internal
internal medicine.
medicine. Proc.
Proc. Fourth
Fourth IJCAI,
IJCAI, Tbilisi,
Tbilisi, USSR,
September,
September, 1975.
in diagnostic
diagnostic
problem
Pople, H.E.:
H.E.: The formation
of composite
composite hypotheses
hypotheses in
Pople,
formation of
problem
solving:
exercise in
in synthetic
synthetic reasoning.
reasoning. Proc.
Proc. Fifth
Fifth IJCAI,
IJCAI, Boston,
Boston,
An exercise
solving: An
August,
August, 1977.

Feigenbaum
E. A. Feigenbaum

210
210

5P41
5P41 RR00785-11
RR00785-11

Appendix
Appendix BB

National
National AIM
AIM Project:
Project:

SECS
SECS --- SIMULATION
SIMULATION AND
AND EVALUATION
EVALUATION
OF
OF CHEMICAL
CHEMICAL SYNTHESIS
SYNTHESIS

Principal
Principal Investigator:
Investigator:

W.
W. Todd
Todd Wipke,
Wipke, Ph.D.
Ph.D.
Department
Department ot
of Chemistry
Chemistry
University
University ot
of California
California at
at Santa
Santa Cruz
Cruz
Santa
Santa Cruz,
Cruz, California
California 95064
95064
(408)
(408) 429-2397
429-2397 (WIPKE~SUMEX-AIM)
(WIPKE@SUMEX-AIM)

The
The SECS
SECS Project
Project aims at developing
developing practical
practical computer
computer programs
programs to assist
investigators
in
designing
syntheses
of
complex
organic
molecules
of
biological
investigators
complex organic
biological interest.
interest.
Key
Key features
features of this research include
include the use
use of computer
computer graphics
graphics to allow
allow chemist
chemist and
computer
computer to work
work efficiently
efficiently as
as aa team, the development
development of knowledge
knowledge bases
bases of chemical
chemica.1
reactions,
reactions, and the formation
formation ot
of plans to reduce the search for solutions.
solutions. SECS is
is being
used
used by the pharmaceutical
pharmaceutical industry
industry for designing syntheses of drugs.
A spin-off
spin-off project,
project, XENO,
XENO, is aimed at predicting
predicting the plausible
plausible metabolites
metabolites of
foreign
foreign compounds
compounds for carcinogenicity
carcinogenicity studies.
studies. First,
First, the metabolism
metabolism is simulated;
simulated; then
then
the metabolites
met,abolites are evaluated
evaluated for
for possible carcinogenicity.
carcinogenicity.
SOFTWARE
SOFTWARE AVAILABLE
AVAILABLE ON
ON SUMEX
SUMEX
SECS--

An
An organic
organic synthesis
synthesis design prograr.1
progra,m available
available with
with a reaction
reaction library
library
over
500
reactions.
The
program
is
accessible
to
users
via
over
reactions. The program
to
via a teletype
teletype
DEC
DEC GT40
GT40 type
type graphics
graphics terminal.
terminal.

of
of
or
or

XENO-XENO--

A
of xenobiotic
xenobiotic compounds.
compounds.
A program
program for
for prediction
prediction of metabolites
metabolites of
Although
early development
development stages, this
this
Although the project
project is still
still in the early
program
testing.
program is available
available for
for preliminary
preliminary exploration
exploration and testing.

PRXBLD-PRXBLD--

A
molecular models
models
A facility
facility for
for building
building approximate
approximate 3-dimensional
3-dimensional molecular
from
The program
program employs
employs an
from their
their 2-dimensional
2-dimensional representations.
representations.
The
energy minimization
minimization approach
approach and is available
available both
both stand-alone
stand-alone and as
energy
part of
of SECS.
part

QED-QED--

A domain-independent
domain-independent inference
inference engine which
which represents
represents knowledge
knowledge in
A
first order
order predicate
predicate calculus.
calculus.
first

FSECS-FSECS--

forward-working synthesis
synthesis prototype
prototype program
program for
for finding
finding starting
starting
A forward-working
A
material oriented
oriented syntheses.
material

SST-SST--

A program
program for
for searching
searching through
through a library
library of
of possible
possible starting
starting materials
materials
A
to
suggest
potential
starting
materials
for
a
given
target
molecule.
to
potentia.1 starting materials for given target molecule.

REFERENCES
REFERENCES
\Vipke, W.T.,
W.T., Rogers,
Rogers, D.:
D.: Rapid
Rapid Subgraph
Subgraph Search
Search Using
Using Parallelism.
Parallelism.
\Vipke,
Chern. If.
If. Comput.
Comput. Sci. (Submitted
(Submitted April
April 24, 1984).
J. Chem.
W.T.: An
An Integrated
Integrated System
System for
for Drug
Drug Design,
Design, in
in COMPUTERS
COMPUTERS
\Vipke, W.T.:
Wipke,
A-Z: AA Manufacturer’
Manufacturer's
Guide to
to Hardware
Hardware and
and Software
Software for
for the
the
A-Z:
s Guide
Pharmaceutical Industry,
Industry, Aster
Aster Publishing
Publishing C.,
C., Springfield,
Springfield. Oregon.
Oregon.
Pharmaceutical
(In press)
press)
(In
Wipke, W.T.,
\V.T., and
and Rogers,
Rogers, D.:
D.: Artificial
Artificial Intelligence
Intelligence in
in Organic
Organic
Wipke,

211
211

E. A.
A. Feigenbaum
Feigenbaum
E.

5P41 RR00785-11
RR00785-11

Appendix
Appendix B

Synthesis.
An
Synthesis. SST:
SST: Starting
Starting !vfaterial
Material Selection
Selection Strategies.
Strategies. An
Application of
Application
of Superstructure
Superstructure Search.
Search. J. Chern.
Chem. Inf.
Inf. Comput.
Comput. Sci.,
24:0000, 1984.
Wipke,
prediction
Wipke, W.T.,
W.T., Ouchi,
Ouchi, G.!.
G.I. and Chou,
Chou, J.T.:
J.T.: Computer-assisted
Computer-assisted prediction
of
CORRELATIONS
of metabolism.
metabolism. IN
IN L. Goldberg
Goldberg (Ed.),
(Ed.), STRUCTURE-ACTIVITY
STRUCTURE-ACTIVITY
CORRELATIONS
AS A
PREDICTIVE.
TOOL
IN
TOXICOLOGY.
Hemisphere
Publishing
A PREDICTIVE.TOOL
IN TOXICOLOGY.
Hemisphere Publishing Corp.,
Corp.,
New York,
New
York, 1983, pp 151-169.
Wipke,
Wipke, \V.T.,
W.T., Ouchi,
Ouchi, G. and Krishnan,
Krishnan, S.: Simulation
Simulation and
and evaluation
evaluation of
of
chemical
synthesis--SECS. An
An application
intelligence
chemical synthesis--SECS.
application of
of artificial
artificial
intelligence
techniques.
techniques. Art.ificial
Artificial Intelligence
Intelligence 10:999, 1978.

E. A.
-4. Feigenbaum
Feigenbaum

212

Appendix
Appendix B

5P41
5P41 RR00785-U
RR00785-11
National
National AIM
AIM Project:
Project:

CLIPR
CLIPR -- HIERARCHICAL
HIERARCHICAL MODELS
MODELS
OF HUMAN
HUMAN COGNITION
COGNITION

Principal
Principal Investigators:
Investigators:

Walter
Walter Kintsch,
Kintsch, Ph.D. (KINTSCH@SUMEX-AIM)
(KINTSCH@SUMEX-AIM)
Peter
Peter G. Polson,
Poison, Ph.D.
Ph.D. (POLSON@SUMEX-AIM)
(POLSON@SUMEX-AIM)
Computer
Computer Laboratory
Laboratory for Instruction
Instruction
in Psychological
Psychological Research (CLIPR)
(CLIPR)
Department
Department of Psychology
Psychology
University
University of Colorado
Colorado
Boulder,
Boulder, Colorado
Colorado 80302
80302
(303) 492-6991
492-6991
Contact:
Contact: Dr.
Dr. Peter
Peter G. Polson (Polson@SUMEX-AIM)
(Polson@SUMEX-AIM)

The CLIPR
CLIPR Project
Project isis concerned with
with the modeling
modeling of complex psychological
psychological
processes.
It is comprised
comprised of two
two research groups. The
The prose comprehension group has
has
processes. It
completed
completed a project
project that
that carries out
out the microstructure
microstructure text
text analysis described by
by Miller
Miller
a.nd
and Kintsch
Kintsch (1980),
(1980) yielding
yielding predictions
predictions of the recall and readability
readability of that
that text
text by
human
More recently,
recently, this
this group
group has been
been interacting
interacting with
with the Heuristic
Heuristic
human subjects.
subjects.
Programming
Programming Project
Project at Stanford,
Stanford, using the AGE
AGE and UNITS
UNITS packages to build
build a more
complex
model
of
the
knowledge-based
processes
characteristic
of
prose
comprehension.
complex
processes characteristic
The
The planning
planning group is working
working toward
toward a model of the planning
planning processes
processes used
used by
by expert
expert
computer
comput,er software
software designers.
designers. The initial
initial processes
processes involved
involved in learning
learning to use
use computers
computers
and other
other complex devices.

SOFTWARE
SOFTWARE AVAILABLE
AVAILABLE ON
ON SUMEX
SUMEX
A
programs has been developed
perform the microstructure
A set of
of programs
developed to
to perform
microstructure text
text analysis
described in Kintsch
Kintsch and van Dijk
Dijk (Psychological
(Psychological Review, 1978) and Miller
Miller and Kintsch
Kintsch
(1980). The
program accepts a propositionalized
propositionalized text
The program
text as
as input,
input, and produces indices that
that
can be used.
based in
used to estimate
estirnate the text's
text’s recall
recall and readability.
readability. A
A more complex
complex model based
AGE
AGE and UNITS,
UNITS, which
which emphasizes
emphasizes the knowledge-based aspects of
of comprehension, is
currently
under development.
currently under
development.

REFERENCES
REFERENCES
Jeffries, R., Turner,
Turner, A.A.,
A.A., Polson, P.G.
P.G. and Atwood,
Atwood, M.A.:
M.A.: The F’
Processes
Jeffries,
rocesses
Involved in
in Designing
Des!'gning So
Software.
IN J.R. Anderson
Anderson (Ed.),
(Ed.), COGNITIVE
COGNITIVE SKILLS
SKILLS
Inuoieled
jt,ware. IN
AND THEIR
THEIR ACQUISITION.
ACQUISITION. Hillsdale,
Hillsdale, NJ,
NJ, L. Erlbaum
Erlbaum Assoc., 1981.
(Forthcoming)
AND
1981. (Forthcoming)
Kieras, D.E.
D.E. a.nd
and Poison,
Polson, P.G.:
P.G.: The formal
formal analysis
analysis oof
Kiera.s,
j
user
complexity.
Int.
J.
Man-Machine
Studies,
In Press.
user complexity. Int.
Man-Machine Studies, In
Kintsch, W.:
W.: On
On modeling
modeling comprehension.
comprehension. Educ.
Educ. Psychologist,
Psychologist, 14:3-14, 1979.
Kintsch,
fI.,tiller, J.R.
J.R. and Kintsch,
Kintsch, W.:
W.: Readability
Readability and
and recall
recall oof
short prose
prose
Miller,
j short
passages: A
A. theoretical
theoretical analysis.
analysis. J. Experimental
Experimental Psychology:
Psychology:
passages:
Human Learning
Learning and Memory,
Memory, 1980.
1980. (In
(In press)
Human

213
213

A. Feigenbaum
Feigenbaum
E. A.

5P41 RR00785-11

Appendix
Appendix B

Rutgers
Rutgers AIM
AIM Project:
Project:

RUTGERS
RUTGERS RESEARCH
RESEARCH RESOURCERESOURCE
COMPUTERS
COMPUTERS IN
IN BIOMEDICINE
BIOMEDICINE

Principal
PI aincipal Investigators:
Investigators:

Saul Amarel,
Amarel, Ph.D.[1982-83],
Ph.D.[1982-831, Casimir
Casimir Kulikowski,
Kulikowski, Ph.D.
Ph.D.
Sholom
Sholom M. Weiss, Ph.D.[H)83-84j
Ph.D.1 1983-841
Department
Department of Computer
Computer Science
Science
Rutgers
Rutgers University
University
New Brunswick,
Brunswick, New Jersey 08903
(201) 932-3546 (AMAREL@RUTGERS)
(AMARELQRUTGERS)
(201) 932-2006 (KULIKO\VSKI@RUTGERS)
(KULIKOWSKI@RUTGERS)
(201) 932-2379 (WEISS@RUTGERS)
(WEISS@IRUTGERS)

The
The Rutgers
Rutgers Research Resource provides
provides the research support
support with
with artificial
artificial
intelligence
intelligence systems, and the computing
computing support
support with
with its DEC2060
DEC2060 facility
facility to a large
number
There are currently
currently 86 investigators
investigators
number of biomedical
biomedical scientists
scientists and researchers. There
associated with
with the Resource. Research activities
activities are concentrated
concentrated in three
three major
major areas:
expert
expert medical
medical systems, models for
for planning
planning and knowledge
knowledge acquisition,
acquisition, and general
general AI
AI
systems
systems development.
development.
One of
of the most
most significant
significant achievements
achievements in bringing
bringing the work
work of
of the Resource to
to
bear
bear on clinical
clinical research and practice
practice lies in the transfer
transfer of
of technology
technology from
from our
our large
DEC20
DEC20 machine
machine to microprocessor
microprocessor compatible
compatible representations.
representations. The
The initial
initial breakthrough
breakthrough
came with
with the automatic
automatic translation
translation of
of a serum
serum protein
protein electrophoresis
electrophoresis interpretation
interpretation
model
model so that
that a version
version could
could be incorporated
incorporated in an instrument
instrument - the
t,he scanning
scanning
densitometer
densitometer (CliniScan)
(CliniScan) produced
produced by
by Helena
Helena Laboratories.
Laboratories. After
After testing,
testing, it
it was
disseminated
disseminated commercially,
commercially, marking
marking the
the first
first successful transfer
transfer of
of technology
technology from
from the
Resource to
being used in over
to general
general availability
availability in the clinical
clinical community.
community. It
It is now
now being
over one
hundred
hundred clinical
clinical locations.
locations.
During
period, we have started
project with
During the current
current period,
started a new project
with long
long term
term
implications
implications for
for the impact
impact of
of AIM
AIM technology:
technology: the development
development of
of a hand-held
hand-held
microcomputer
microcomputer version
version of
of an expert
expert consultation
consultation system
system for
for front-line
front-line health
health workers.
workers. In
collaboration
with
collaboration
with Dr.
Dr. Chandler
Chandler Dawson
Dawson (UCSF),
(UCSF), Director
Director of
of the \Vorld
World Health
Health
Organization's
Organization’s Collaborative
Collaborat,ive Centre
Centre for
for the Prevention
Prevention of
of Blindness
Blindness and Trachoma,
Trachoma, we
have developed
prototype model
primary eye care. This
developed a prototype
model for
for consultation
consultation on primary
This has been
orient.ed
problems of
oriented at problems
of injury,
injury, infection,
infection, malnutrition
malnutrition and cataract
cataract in situations
situations where
where an
opht,hamologist is unavailable.
unavailable. In
In most
most developin,
developingc nations,
nations, the incidence
incidence of
of blindness
blindness is
ophthamologist
10%
to 40%
40% higher
higher than
than in the
the USA
USA because of
of these kinds
kinds of
of problems.
problems. With
With the
the help of
of
10% to
grant from
from the
the CJSAID,
USAID, we are developing
developing the
the systems needed for
for management
management of
of eye
a gra.nt
by front-line
front-line health
health workers
workers in developing
developing nations,
nations, and outlying
outlying parts
parts of
of the
the USA.
USA.
disease by
disease
Another significant
significant technology
technology transfer
transfer experiment
experiment involves
involves a very
very large
Another
consultation model.
model. The
The rheumatology
rheumatology knowledge
knowledge base developed
developed by
by our
our collaborators
collaborators
consultation
Drs. Lindberg
Lindberg and Sharp
Sharp at
at the
the University
University of
of Missouri
Missouri has been transferred
transferred by
by us to
to the
the
Drs.
MC58000 microprocessor
microprocessor based system,
system, and
and in the
the past
past year
year testing
testing has begun
begun at
at their
their
MC68000
sit.e. This
This represents
represents a major
major step in bringing
bringing the
the results
results of
of artificial
artificial int,elligence
intelligence research
research
sit,e.
to the point
point where
where clinical
clinical researchers
researchers who
who do not
not have access
access to
to large research
research machines
machines
to
will be able to
to make
make use of
of the
the results.
results. We
We are designing
designing a specialized
specialized rheumatology
rheumatology
will
marhine
which
can
carry
out
the
same
sophisticated
reasoning
that
now
the
machine which
carry out the
sophisticated reasoning that now needs the
Resource DEC20
DEC20 , but
but will
will cost
cost little
little over
over $10,000.
$10,000. Because the
the transfer
transfer has been
Resource
accomplished we can continue
continue to
to develop
develop large scale models
models using
using the
the full
full facilities
facilities of
of the
the
accomplished
Resource DEC“0
DEC20
but wit,h
with the
the confidence
confidence that
that they
they can then
then move
move out
out into
into clinical
clinical
Resource
- 9, but
research environments
environments when
when completed.
completed.
research

A. Feigenbaum
Feigenbaum
E. A.

214

5P41
5P41 RR00785-11
RR00785-11

Appendix
Appendix BB

REFERENCES
REFERENCES
Weiss,
Weiss, S.M.
S.M. and
and Kulikowski,
Kulikowski, C.A.
C.A. A Practical
Practical Guide
Guide to
to Designing
Designing
Expert
Expert Systems,
Systems, Rowman
Rowman and
and Allanheld,
Ailanheld, H184.
1984.
Kulikowski,
Kulikowski, C.A.
C.A. contributor
contributor to the Knowledge
Knowledge Acquisition
Acquisition chapter
chapter
edited
edited by B.
B. Buchanan
Buchanan in
in the book Building
Building Expert
Expert Systems
Systems (F.
(F. HayesHayesRoth,
Roth, et
et aI.,
al., eds)
eds) Addison-Wesley,
Addison-Wesley, 1983
1983 (in
(in press).
Yao, Y. and KUlikowski,
of
Strategies
of Reasoning
Reasoning
Kulikowski, C.A.,
C.A., It-fultiple
Multiple Strategies
for
for Expert
Expert Systems,
Systems, Proc.
Proc. Sixteenth
Sixteenth Hawaii
Hawaii International
International Conference
Conference on
Systems Sciences,
Sciences, pp. 510-514,1983.*
510-514 , 1983.)
Kulikowski,
in Expert
Ezpert AI
AI Medical
Medical Consultation
Consultation Systems:
Systems:
Kuiikowski, C.A.Progress
C.A.Progresa in
1980
IQ880- 1988,
198.9, Proc.
Proc. MEDINFO
MEDINFO '83
‘83 , pp. 499-502, Amsterdam,
Amsterdam, August
August 1983.*
1983.”
Kastner,
for
Kastner, J.K.,
J.K., Weiss, S.M.,
S.M., and Kulikowski,
Kulikowski, C.A.,An
C.A.,An Efficient
Efficient Scheme for
Time-Dependent
Time-Dependent Consultation
Consultation Systems,
Systems, Proc.
Proc. MEDINFO
MEDINFO '83,
‘83, pp.619-622,
pp.619-622, 1983.*
Kulikowski,
Kuiikowski, C.A.Expert
C.A.Ezpert Medical
Medical Consultation
Consultation Systems,
Systems, Journal
Journal of
of Medical
Medical
Systems, v.7, pp. 229-234, 1983.*

in
Weiss, S.M.,
Ezpertise
in
S.M., Kulikowski,
Kulikowski, C.A.,
C.A., and Galen, R.S.,Representing
R.S.,Representing Expertise
a Computer
Computer Program:
Program: The Serum
Serum Protein
Protein Diagnostic
Diagnostic Program,
Program, Journal
Journal of
of
Clinical
Clinical Laboratory
Laboratory Automation,
Automation, v.3,
v.3, pp. 383-387, 1983.
1983.**
Kastner,
Expert System
for
Kastner, J.K.,
J.K., Weiss, S.M.,
S.M., and Kulikowski,
Kulikowski, C.A.,
C.A., An
An Ezpert
System for
Primary Eye Care,
Front-Line Health
Health Workers
Front-Line
Workers in
in Primary
Care, Proc.
Proc. Seventeenth
Seventeenth Hawaii
Hawaii
International
International Conference
Conference on Systems Sciences,
Sciences, pp. 162-166, 1984.
1984.**
Kulikowski, C.A.
C.A. Knowledge
Knowledge Acquisition
Acquisition and
and Learning
Learning in
in EXPERT,
EXPERT,
Kulikowski,
Proc. 1983 \Vorkshop
\Vorkshop on Machine
Machine Learning,
Learning, Univ.
Univ. of
of Iilinois,Champaign-Urbana
Illinois,Champaign-Urbana
Proc.
1983.
Indicate by
by an asterisk
asterisk (*)
(*) that
that the
the resource
resource was given
given credit.
credit.
Indicate

215
215

E. A.
A. Feigenbaum
Feigenbaum
E.

5P41 RR00785-11

Appendix
Appendix B

National
National AIM
AIM Project:
Project:

SOLVER
SOLVER -- PROBLEM
PROBLEM SOLVING
SOLVING EXPERTISE
EXPERTISE

Principal
Principal Investigators:
Investigators:

Paul
Paul E. Johnson, Ph.D.
Ph.D.
Center
Center for
for Research in Human
Human Learning
Learning
205 Elliott
Elliott Hall
Hall
University
University of Minnesota
Minnesota
Minneapolis,
Minneapolis, Minnesota
Minnesota 55455
(612) 373-5302 (PJOHNSON@SUMEX-AIM)
(PJOHNSON@SUMEX-AIM)
William
William B. Thompson,
Thompson, Ph.D.
Ph.D.
Department
Department of Computer
Computer Science
136 Lind
Lind Hall
Hall
University
University of Minnesota
Minnesota
Minneapolis,
Minneapolis, Minnesota
Minnesota 55455
(612) 373-0132 (THOMPSON@SUMEX-AIM)
(THOMPSON@SUMEX-AIM)

The
The Minnesota
Minnesota SOLVER
SOLVER project
project focuses
focuses upon
upon the development
development of
of strategies
strategies for
for
discovering
discovering and representing
representing the knowledge
knowledge and skill
skill of
of expert
expert problem
problem solvers. Although
Although
in the last 15 years considerable
considerable progress has been made in synthesizing
synthesizing the expertise
expertise
required
for
solving
complex
problems,
most
expert
systems
required for solving complex problems, most expert systems embody
embody only
only the limited
limited
amount
amount of
of expertise
expertise that
that individuals
individuals are able to report
report in a particular
particular constrained
constrained
lan~uage
language (e.g., production
production rules).
rules). What
What is still
still lacking
lacking is a theoretical
theoretical framework
framework capable
of reducing
reducing dependence upon
upon the
the expert's
expert’s intuition
intuition or
or on the near exhaustive
exhaustive testing
testing of
possible organizations.
organizations. Our
Our methodology
methodology consists of: (1) extensive
extensive use
use of
of verbal
verbal thinking
thinking
aloud
aloud protocols
protocols as
as a source of
of information
information from
from which
which to
to make inferences
inferences about
about
underlying
processes; (2) development
underlying cognitive
cognitive structures
structures and processes;
development of
of computer
computer models
models as
az a
means of
protocol studies;
of testing
testing the
the adequacy
adequacy of
of inferences
inferences derived
derived from
from protocol
studies; (3) testing
testing and
refinement
refinement of
of the cognitive
cognitive models
models based upon
upon the study
study of
of human
human and model
model
performance in experimental
problem-solving
performance
experimental settings.
settings. Currently,
Currently, we are investigating
investigating problem-solving
expertise
physics, engineering,
expertise in domains
domains of
of medicine,
medicine, physics,
engineering, management,
management, and law.
law.
SOFTWARE
SOFTWARE AVAILABLE
AVAILABLE ON
ON SUMEX
SUMEX
A
A redesigned
redesigned version
version of
of the Diagnoser
Diagnoser simulation
simulation model,
model, named
named Galen,
Galen, has been
implemented
implemented on SUMEX.
SUMEX.
REFERENCES
R.EFERENCES
Johnson. P.E.:
P.E.: What
kind of
of expert
expert should
should a sycltem
system be?
Johnson,
What kind
Medicine and
and Philosophy,
Philosophy. 8:77-97, 1983.
J. Medicine
Johnson. P.E.,
P.E.. Duran,
Duran. A.,
A .. Hassebrock,
Hassebrock. F.,
F .• Moller,
Moller, J., Prietula,
Prietula. M.,
M ..
Johnson,
Feltovich. P. and
and Swanson,
Swanson. D.: Expertise
Expertise and
and error
error in
in diagnostic
diagnostic
Feltovich,
reasoning. Cognitive
Cognitive Science 5:235-283,
5:235-283. 11981.
981.
reasoning.
Thompson. W.B.,
\V.B .. Johnson,
Johnson. P.E.
P.E. and Moen,
Moen. J.B.:
J.B.: Recognition-based
Recognition-based
Thompson,
diagnostic reasoning.
reasoning. Proc.
Proc. Eighth
Eighth IJCAI,
IJCAI, Karlsruhe,
Karlsruhe, West
West
diagnostic
Germany. August,
August. 1983.
1983.
Germany,
Sedlmeyer. R.L.,
R.L.. Thompson,
Thompson. W.B.
\V.B. and
and Johnson,
Johnson, P.E.:
P.E.: Diagnostic
Diagnostic
Sedlmeyer,
reasoning in
in software
software fault
fault localization.
localization. Proc.
Proc. Eighth
Eighth IJCAI,
IJCAI.
reasoning
Karlsruhe. West
West Germany,
Germany. August,
August. 1983.
1983.
Karlsruhe,

Feigenbaum
A. Feigenbaum
E. A.

216
216

5P41
5P41 RR00785-11
RR00785-1 I

Appendix
Appendix BB

Johnson,
Johnson. P.E.,
P.E., "The
“The Expert
Expert Mind:
Mind: A
A New
New Challenge
Challenge for
for the
the Information
Information
scientist."
scientist,” in
in Beyond
Beyond Productivity:
Productivity: Information
In formation System
System Development
Development
for
for Organizational
Organizational Effectiveness,
Effectivenese, Th.
Th. M.
M. A.
A. Bemelmans
Bemelmsns (editor),
(editor),
Elsevier
Elsevier Science
Science Publishers
Publishers B. V. (North-Holland),
(North-Holland), 1984.
1984.

217
217

E. A.
A. Feigenbaum
Feigenbaum
E.

Appendix
Appendix B

5P41
5P41 RR00785-11
RR00785- 11

National
National AIM
AIM Project:
Project:

Computer-Aided
Computer-Aided Diagnosis
Diagnosis of
Malignant
Malignant Lymph
Lymph Node Diseases
Diseases (PATHFINDER)
(PATHFINDER)

Principal
Principal Investigator:
Investigator:

Bharat
Bharat Nathwani,
Nathwani, M.D.
M.D.
Department
Department of Anatomical
Anatomical Pathology
Pathology
City
City of Hope National
National Medical
Medical Center
Center
Duarte,
Duarte, California
California
(213)
(213) 359-8111
359-8111 x 2456
2456 (NATHWANI@SUMEX-AIM)
(NATHWANIQSUMEX-AIM)
Lawrence
Lawrence M. Fagan, M.D.,
M.D., Ph.D.
Ph.D.
Department
Department of Medicine
Medicine
Stanford
Stanford University
University Medical
Medical Center
Center - Room
Room TC135
TC135
Stanford,
Stanford, California
California 94305
(415) 497-6979 (F
AGAN@SUMEX-AIM)
(FAGANQSUMEX-AIM)

\Ve
to assist in the
We are building
building a computer
computer program,
program, called
called PATHFINDER,
PATHFINDER,
diagnosis
dia,gnosis of lymph
lymph node pathology.
pathology. The
The project
project is based at the City
City of Hope National
National
medical
medical center
center in collaboration
collaboration with
with the Stanford
Stanford University
University Medical
Medical Computer
Computer Science
Group.
Group. A
A pilot
pilot version
version of
of the program
program provides
provides diagnostic
diagnostic advice on 45 common
common benign
benign
and malignant
malignant diseases
diseases of
of the lymph
lymph node based on 77 histologic
histologic features.
features. Our
Our research
plans are to develop a full-scale
full-scale version
version of
of the computer
computer program
program by
by substantially
substantially
increasing
increa.sing the quantity
quantity and quality
quality of
of knowledge
knowledge and to develop techniques
techniques for
for knowledge
knowledge
represent.ation
represent,ation and manipulation
manipulation appropriate
appropriate to this
this application
application area. The
The design of
of the
program
jCADUCEUS program
strongly influenced
influenced by
by the INTERNIST
INTERNIST/CADUCEUS
program
program has been strongly
developed
developed on the SUMEX
SUMEX resource.

SOFT\V
ARE AVAILABLE
S0FTWAR.E
AVAILABLE ON
ON SUMEX
SUMEX
PATHFINDER-A
program is available
for
PATHFINDER-A version
version of
of the
the PATHFINDER
PATHFINDER
program
available
for
experimentation
on
the
DEC
2060
computer.
This
version
is
a
pilot
experimentation
DEC
computer.
This version
pilot
version
program, and therefore
version of
of the program,
therefore has not
not been completely
completely tested.
tested.

A. Feigenbaum
Feigenbaum
E. A.

218
218

Appendix
Appendix B

5P41
5P41 RROOi85-U
RR00785-11

Stanford
Stanford Project:
Project:

EXPEX
EXPEX -- EXPERT
EXPERT EXPLANATION
EXPLANATION

Principal
Principal Investigator:
Investigator:

Edward
Edward H. Shortliffe,
Shortliffe, M.D.,
M.D., Ph.D.
Ph.D.
Departments
Departments of Medicine
Medicine and Computer
Computer Science
Science
Stanford
University
Medical
Center
Stanford University Medical Center - Room
Room TC135
TC135
Stanford,
Stanford, California
California 94305
(415) 49i-6979
497-6979 (SHORTLIFFE@SUMEX-AIM)
(SHORTLIFFE@SUMEX-AIM)

EXPEX
EXPEX isis a recent
recent Stanford
Stanford research project
project that
that is involved
involved with
with the development
development
of new representation
This
representation schemes
schemes to facilitate
facilitate knowledge
knowledge acquisition
acquisition and explanation.
explanation. This
includes
includes not
not only
only the study
study of fundamental
fundamental representational
representational formalisms
formalisms but
but also the
encoding
encoding of various
various types
types of knowledge,
knowledge, such as
as causal information
information and user models. The
The
research effort
effort deals with
with medical
medical domains
domains and is being undertaken
undertaken on SUMEX
SUMEX or
or on
professional
professional workstations
workstations linked
linked to the central
central resource.
Our
Our interest
interest in explanation
explanation derives
derives from
from the insights
insights we gained
gained in developing
developing
explanatory
that system
system and its descendants, we
expia,natory capabilities
ca.pabilities for
for the MYCIN
MYCIN system.
system. In that
were able to generate
generate intelligible
intelligible explanations
explanations by
by taking
taking advantage
advantage of
of a rule-based
rule-based
representation
The limitations
limitations of
of the justifications
justifications generated
generated using MYCIN's
MYCIN’s
representation scheme. The
explanation
explanat.ion capabilities
capabilities have become increasingly
increasingly obvious,
obvious, however,
however, and have led to
improved
improved characterization
characterization of
of the kinds
kinds of
of explanation
explanation capabilities
capabilities that
that must
must be developed
developed
if
EXPEX is devoted
devoted to
if clinical
clinical consultation
consultation systems are to be accepted
accepted by
by physicians.
physicians. EXPEX
the development
practical and theoretical
problem.
development of
of new practical
theoretical insights
insights into
into this
this problem.

REFERENCES
REFERENCES
Teach,
An analysis
physician att-itudes
Teach, R.L.
R.L. and Shortliffe,
Shortliffe, E.H.:
E.H.: An
analysis of
of physician
att,ltudes
systems. Comput.
regarding
regarding computer-based
computer-based clinical
clinical consultation
consultati.on systems.
Comput. Biomed.
Biomed.
Res. 14:542-558, H18l.
1981.

Explanatory power
power for
for medical
Wallis,
Wallis, J.\V.
J.W. and Shortliffe,
Shortliffe, E.H.:
E.H.: Explanatory
medical expert
expert
systems:
Studies
in
the
representation
of
causal
relationships
for clinical
clinical
systems: Studies in the representation of causal relationships
for
corlsultations. Meth.
Meth. Info.
Info. Med.
Med. 21:127-136, 1982.
consultations.
Langlotz, C.P.
C.P. and Shortliffe,
Shortliffe, E.H.:
E.H.: Adapting
Adapting a consultation
consultation
La.nglotz,
system to critique
critique user
user plans.
plans. International
International
system
Journal of
of Man-Machine
Man-Machine Studies,
Studies, 19:479-496 (1983)
Journal
Tsuji, S. and Shortliffe,
Shortliffe, E.H.:
E.H.: Graphical
Graphical access to the knowledge
knowledge base
Tsuji,
of a medical
medical consultation
consultation system.
system. Proceedings
Proceedings of
of AAMSI
AAMSI Congress
of
83, pp.
pp. 551-555, San Francisco,
Francisco, Ca.,
Ca., May
May 1983.
W., Clancey,
Clancey, W.
W. J., and
and Rennels, G.:
G.:
Hasling, D. W.,
Hasling,
Strateg£c
explanations
for
a
diagnostic
consultation
system.
Strategic explanations for
diagnostic consultation system.
International Journal
Journal of
of Man-Machine
Man-Machine Studies,
Studies, Spring
Spring 1984 (in
(in press).
International

219

E. A.
A. Feigenbaum
Feigenbaum

5P41 RR00785-11
RR00785- 11

Appendix
Appendix B

Stanford
Stanford Project:
Project:

GUIDON/NEOMYCIN
-GUIDON/NEOMYCIN
KNOWLEDGE
KNOWLEDGE ENGINEERING
ENGINEERING
FOR
FOR TEACHING
TEACHING MEDICAL
MEDICAL DIAGNOSIS
DIAGNOSIS

Prinripal
Principal Investigators:
Invest,igators:

\Villiam
William J. Clancey,
Clancey, Ph.D.
Ph.D.
701 \Velch
Welch Road
Road
Department
Department of
of Computer
Computer Science
Stanford
Stanford University
University
Palo
Palo Alto,
Alto, California
California 94304
(415) 497-1997 (CLANCEY@SUMEX-AIM)
(CLANCEY@SUMEX-AIM)
Bruce
Bruce G. Buchanan,
Buchanan, Ph.D.
Ph.D.
Computer
Computer Science Department
Department
Margaret
Margaret Jacks Hall
Hall
Stanford
Stanford University
University
Stanford,
Stanford, California
California 94305
(415) 497-0935 (BUCHANAN@SUMEX-AIM)
(BUCHANAN@SUMEX-AIM)

SOFTWARE
SOFTWARE AVAILABLE
AVAILABLE ON
ON SUMEX
SUMEX
GUIDON--A
GUIDON--A system
system developed
developed for
for intelligent
intelligent computer-aided
computer-aided instruction.
instruction. Although
Although
it
it was developed
developed in the context
context of
of MYCIN's
MYCIN’s infectious
infectious disease
disease knowledge
knowledge base,
base, the tutorial
tutorial
rules will
will operate
0perat.e upon
upon any
any EMYCIN
EMYCIN knowledge
knowledge base.
base.
NEOMYCIN--A consulation
NEOMYCIN--A
consulation system
system derived
derived from
from MYCIN,
MYCIN, with
with the knowledge
knowledge base
base
greatly
In
In contrast
contrast with
with MYCIN,
MYCIN,
greatly extended
extended and reconfigured
reconfigured for
for use in teaching.
teaching.
dia,gnostic
dia.gnostic procedures,
procedures, common
common sense
sense facts,
facts, and disease
disease hierarchies
hierarchies are factored
factored out
out of
of the
basic finding/disease
finding/disease associations.
associations. The
The diagnostic
diagnostic procedures
procedures are abstract
abstract (not
(not specific
specific to
to
any
a,ny problem
problem domain)
domain) and model
model human
human reasoning,
reasoning, unlike
unlike the exhaustive,
exhaustive, top-down
top-down
approach
approach implicit
implicit in MYCIN's
MYCIN’s medical
medical rules. This
This knowledge
knowledge base
base will
will be used in the
GUIDON2
GUIDON2 family
family of
of instructional
instructional programs.
programs.

REFERENCES
REFERENCES
Clancey,
Clancey, \V.J.:
W.J.: Overview
Overview of
of GUIDON.
GUIDON. In
In A.
A. Barr
Barr and E.A.
E.A. Feigenbaum
Feigenbaum (Eds.),
(Eds.),
THE
Vol.
THE HANDBOOK
HANDBOOK OF
OF ARTIFICIAL
ARTIFICIAL INTELLIGENCE,
INTELLIGENCE,
Vol. 2. William
William Kaufmann
Kaufmann Assoc.,
Los Altos,
Altos, CA,
CA, 1982. (Also
(Also to
to appear
appear in J. of
of Computer-based
Computer-based Instruction)
Instruction)
Clancey,
Methodology for
for building
system.
Clsncey, W.J.:
W.J.: Methodology
building an intelligent
intelligent tutoring
tutoring system.
In Kintsch,
Kintsch, Polson,
Poison, and Miller,
Miller, (Eds.),
(Eds.), METHODS
METHODS AND
AND TACTICS
TACTICS IN
IN COGNITIVE
COGNITIVE
SCIENCE.
NJ. (In
SCIENCE. L. Erlbaum
Erlbaum Assoc., Hillsdale,
Hillsdale, NJ.
(In press) (Also
(Also STAN-CS-81-894,
STAN-CS-81-894,
HPP
HPP 81-18)
81-18)
Clancey,
Acquiring, representing,
Clancey, W.J.:
W.J.: Acquiring,
representing, and
and evaluating
evaluating a competence
competence
model
of
diagnosis.
In
Chi,
Glaser,
and
Farr
NATURE
model of diagnosis. In Chi, Glaser,
Farr (Eds.),
(Eds.), THE
THE NATURE
OF
EXPERTISE.
In
preparation.
HPP-84-2.
OF E?
Source Exif Data:
File Type : PDF
File Type Extension : pdf
MIME Type : application/pdf
PDF Version : 1.6
Linearized : No
XMP Toolkit : Adobe XMP Core 4.2.1-c041 52.342996, 2008/05/07-21:37:19
Format : application/pdf
Creator : StreetI
Description :
Title :
Create Date : 2001:07:17 15:29:44Z
Creator Tool : Acrobat Capture Server 2.01
Modify Date : 2018:03:20 08:52:17-07:00
Metadata Date : 2018:03:20 08:52:17-07:00
Keywords :
Producer : Adobe Acrobat 9.0 Paper Capture Plug-in
Document ID : uuid:128fb35a-42bb-1040-ae6a-5143bd225111
Instance ID : uuid:25363a4d-6f61-1f46-a4f0-d61979dbc07d
Page Layout : SinglePage
Page Count : 233
Author : StreetI
Subject :

EXIF Metadata provided by EXIF.tools

5P41_RR00785 13_SUMEX_Annual_Report Year_11_Jun84 5P41 RR00785 13 SUMEX Annual Report Year 11 Jun84

5P41_RR00785-13_SUMEX_Annual_Report-Year_11_Jun84 5P41_RR00785-13_SUMEX_Annual_Report-Year_11_Jun84

Navigation menu

Versions of this User Manual:

Views

Navigation