5P41_RR00785 13_SUMEX_Annual_Report Year_14_Jun87 5P41 RR00785 13 SUMEX Annual Report Year 14 Jun87
5P41_RR00785-13_SUMEX_Annual_Report-Year_14_Jun87 5P41_RR00785-13_SUMEX_Annual_Report-Year_14_Jun87
User Manual: 5P41_RR00785-13_SUMEX_Annual_Report-Year_14_Jun87
Open the PDF directly: View PDF 
.
Page Count: 247
| Download | |
| Open PDF In Browser | View PDF |
SUMEX
SUMEX
STANFORD UNIVERSITY
UNIVERSITY
STANFORD
MEDICAL EXPERIMENTAL
EXPERIMENTAL COMPUTER
COMPUTER RESOURCE
RESOURCE
MEDICAL
RR-00785
ANNUAL REPORT-YEAR
ANNUAL
REPORT-YEAR 14
Submitted
Submitted to
to
BIOMEDICAL
RESEARCH
PROGRAM
BIOMEDICAL
RESEARCH TECHNOLOGY
TECHNOLOGY
PROGRAM
NATIONAL
NATIONAL INSTITUTES
INSTITUTES OF
OF HEALTH
HEALTH
June
June 1, 1987
STANFORD
STANFORD UNIVERSITY
UNIVERSITY SCHOOL
SCHOOL OF
OF MEDICINE
MEDICINE
Edward
Principal
Edward H.
H. Shortliffe,
Shortliffe,
Principal Investigator
Investigator
Co-Principal
Investigator
Edward
Edward A. Feigenbaum,
Feigenbaum,
Co-Principal
Investigator
DEPARTMENT OF
OF HEALTH
HEALTH AND
AND HUMAN
HUMAN SERVICES
SERVICES
DEPARTMENT
PUBLIC HEALTH
HEALTH SERVICE
SERVlCE
,
PUBLIC
N’ATI~NAL INSTITUTES
INSTITUTES OF
0~ HEALTH
HEALTH
NATIONAL
DIVISION OF
OF RESEARCH
RESEARCH RESOURCES
RESOURCES
DIVISION
BIOMEDICAL
RESEARCH
TECHNOLOGY
PROGRAM
BIOMEDICAL RESEARCH TECHNOLOGY PROGRAM
ANNUAL PROGRESS
PROGRESS REPORT
REPORT
ANNUAL
PART I., TITLE
TITLE PAGE
PAGE
PART
1.
1.
PHS GRANT
GRANT NUMBER:
NUMBER:
PHS
5 P41
P41 RR00785-14
RR00785-14
2.
TITLE OF
OF GRANT:
GRANT:
TITLE
SUMEX
SUMEX
Stanford
Stanford University
University Medical
Medical
Experimental
Computer
Experimental
Computer Resource
Resource
3. NAME
NAME OF
OF RECIPIENT
RECIPIENT INSTITUTION:
INSTITUTION:
3.
Stanford University
Stanford
University
4.
4.
HEALTH
HEALTH PROFESSIONAL
PROFESSIONAL SCHOOL:
SCHOOL:
School of
of Medicine
Medicine
School
5.
5.
REPORTING
REPORTING PERIOD:
PERIOD:
Sa.
5a.
5b.
5b.
FROM:
FROM:
TO:
TO:
08-01-86
08-01-86
07-31-87
07-31-87
6.
6. PRINCIPAL
PRINCIPAL INVESTIGATOR:
INVESTIGATOR:
6a.
6a.
6b.
6b.
NA,ME:
NAME:
TITLE:
TITLE:
6c.
6c.
SIGNATURE:
SIGNATURE:
7.7. DATE
DATE SIGNED:
SIGNED:
Edward H.
H. Shortliffe,
Shortliffe, M.D.,
M.D., Ph.D.
Ph.D.
Edward
Associate Professor
Professor of
of Medicine
Medicine
Associate
Computer Science
Science
and Computer
and
------~~ --------M 17
------------
/7~1-
--------------~------t----------------------8.8. TELEPHONE:
TELEPHONE:
415-723-6979
415-723-6979
SP41-RR0078S-14
5P41-RR00785-14
Table of
of Contents
Contents
Table
Title Page
Page
I.I. Title
II. Description
Description of
of Program Activities
Activities
II.
II.A. Scientific
Scientific Subprojects
Subprojects
1I.A.
II.B.
Books,
Papers,
and Abstracts
Abstracts
1I.B. Books,
and
II.C. Resource
Resource Summary
Summary Table
Table
1I.C.
III.
Narrative
Description
III. Narrative Description
IILA. Summary
Summary of
of Research
Research Progress
Progress
1II.A.
III.A.I. Resource Overview
Overview
III.A.l.
SUMEX-AIM as a Resource
Resource
III.A.I.I. SUMEX-AIM
III.A.l.l.
The Future
Future of
of SUMEX-AIM
SUMEX-AIM
III.A.l.2. The
III.A.1.2.
III.A.2. Resource
Resource Definitions
Definitions and Goals
Goals
III.A.2.
III.A.2.l. Knowledge-Based
Knowledge-Based System Research
III.A.2.1.
III.A.2.2. Resource
Resource Sharing
Sharing
III.A.2.2.
III.A.2.3.
Significance
and Impact
Impact in
in Biomedicine
Biomedicine
III.A.2.3. Significance and
III.A.2.4.
Summary
of
Current
Resource
Goals
III.A.2.4. Summary of Current Resource Goals
III.A.3. Details
Details of
of Technical
Technical Progress
III.A.3.
Highlights
III.A.3.l. Progress Highlights
III.A.3.1.
III.A.3.2. Core
Core ONCOCIN
ONCOCIN Research
III.A.3.2.
III.A.3.3.
III.A.3.3. Core
Core AI
AI Research
Development
III.A.3.4.
III.A.3.4. Core
Core System.
System -Development
III.A.3.S.
TII.A.3.5. Relevant
Relevant Core
Core Research Publications
Publications
III.A.3.6.
Resource
Equipment
III.A.3.6. Resource Equipment
III.A.3.7.
III.A.3.7. Training
Training Activities
Activities
III.A.3.8.
III.A.3.8. Resource
Resource Operations
Operations and Usage
Highlights
III.B.
111-B.Highlights
III.B.1.
III.B.l. The
The MENTOR
MENTOR Project
Project
III.B.2.
III.B.2. The
The GUIDON
GUIDON Project
Project
III.B.3.
III.B.3. The
The PROTEAN
PROTEAN Project
Project
III.B.4.
III.B.4. The
The Medical
Medical Information
Information Science Program
Program
III.B.S.
III.B.5. Remote
Remote Virtual
Virtual Graphics
Graphics
III.C.
1II.C. Administrative
Administrative Changes
I1I.D.
1II.D. Resource Management
Management and Allocation
Allocation
III.D.l.
Overall
III.D.l. Overall Management
Management Plan
Plan
IILD.2.
III.D.2. 2060 Cost
Cost Center
Center
IILE.
1II.E. Dissemination
Dissemination of
of Resource
Resource Information
Information
III.F.
1II.F. Suggestions and Comments
Comments
IV.
IV. Description
Description of
of Scientific
Scientific Subprojects
IV.A.
1V.A. Stanford
Stanford Projects
Projects
IV.A.l.
Project
IV.A.l. GUIDON/NEOMYCIN
GUIDON/NEOMYCIN
Project
IV.A.2.
IV.A.2. MOLGEN
MOLGEN Project
Project
IV.A.3.
IV.A.3. ONCOCIN
ONCOCIN Project
Project
i
1
33
33
3
33
5
5S
5S
5S
7
10
10
11
11
12
16
16
19
25
32
61
67
77
79
93
94
95
96
97
98
99
100
100
101
103
105
107
107
108
109
116
123
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
IV.A.4. PROTEAN
PROTEAN Project
Project
IV.A.4.
IV.A.5.
RADIX
Project
IV.A.5. RADIX Project
IV.B. National
National AIM
AIM Projects
Projects
1V.B.
IV.B.l.
INTERNIST-I
Project
IV.B.l. INTERNIST-I
Project
IV.B.2. CLIPR
CLIPR -- Hierarchical
Hierarchical Models
Models of
of Human
Human Cognition
Cognition
IV.B.2.
IV.B.3. MENTOR
MENTOR Project
Project
IV.B.3.
IV.B.4. SOLVER
SOLVER Project
Project
IV.B.4.
IV.B.5. ATTENDING
ATTENDING Project
Project
IV.B.5.
IV.C.
Pilot
Stanford
Projects
1V.C. Pilot Stanford Projects
IV.C.l. REFEREE
REFEREE Project
Project
IV.C.l.
IV.D.
Pilot
AIM
Projects
1V.D. Pilot AIM Projects
IV.D.l. PATHFINDER
PATHFINDER Project
Project
IV.D.l.
IV.D.2. RXDX
RXDX Project
Project
IV.D.2.
IV.D.3. Dynamic
Dynamic Systems Project
Project
IV.D.3.
IV.D.4. Knowledge
Knowledge Engineering
Engineering for
for Radiation
Radiation Therapy
Therapy
IV.D.4.
IV.D.5. Pathophysiologic
Pathophysiologic Diagnosis
Diagnosis Project
Project
IV.D.5.
Appendix A. AIM
AIM Management Committee
Committee Membership
Membership
Appendix
Scientific Subproject
Subproject Abstracts
Abstracts
Appendix B. Scientific
Appendix
References
E. H. Shortliffe
Shortliffe
ii
138
146
159
160
165
170
174
181
186
187
193
194
200
203
211
214
217
221
245
5P41- RR00785-14
5P41-RR00785-14
List of
of Figures
Figures
List
Figure
Figure 1:
Figure 2:
Figure
Figure
Figure 3:
Figure
Figure 4:
Figure 5:
Figure
Figure 6:
Figure
Figure 7:
Figure
Figure
Figure 8:
Figure 9:
Figure
Figure 10:
Figure
Figure 11:
Figure
Figure 12:
Figure
Figure 13:
Figure
Figure 14:
Figure 15:
Figure
Figure 16:
Figure
Figure 17:
AA Sample
Sample OPAL
OPAL Form
Form
TALK Session Example
Example and
and the
the Software
Software Layers
Layers Involved
Involved in
in TALK
TALK
TALK
File Server
Server Throughput
Throughput Benchmarks
Benchmarks
File
SUMEX-AIM
SUMEX-AIM DEC
DEC 2060 Configuration
Configuration
SUMEX-AIM Sun File
File Server
Server Configuration
Configuration
SUMEX-AIM
SUMEX - AIM Xerox
Xerox File
File Server
Server Configuration
Configuration
SUMEX-AIM
SUMEX
AIM
VAX
File
Server
Configuration
SUMEX-AIM
VAX File Server Configuration
SUMEX-AIM
SUMEX-AIM EtherNet
EtherNet Configuration
Configuration
Total
CPU
Time
Consumed
by Month
Month
Total CPU Time Consumed by
Monthly CPU
CPU Usage by
by Community
Community
Monthly
Monthly Terminal
Terminal Connect
Connect Time
Time by
by Community
Community
Monthly
Cumulative CPU
CPU Usage Histogram
Histogram by
by Project
Project and
and Community
Community
Cumulative
Public Data
Data Network
Network Terminal
Terminal Connect
Connect Time
Time
Public
INTERNET
INTERNET Terminal
Terminal Connect
Connect Time
Time
Downtime --- Hours
Hours per Month
Month
System Downtime
Overall
Overall System Reliability
Reliability Summary
Summary
Cost Center
Center Performance
Performance
2060 Cost
III
111
21
37
40
72
73
74
75
76
81
83
84
86
90
91
92
92
102
E.
E. H.
H. Shortliffe
Shortliffe
Description
Description of
of Program Activities
Activities
5P41-RR00785-14
5P41-RR00785-14
II.
II. Description
Description of
of Program Activities
Activities
This
This section corresponds to the predefined forms
forms required by the Division
Division of
of Research
Research
Resources
provide information
Resources to provide
information about our resource
resource activities
activities for
for their
their computerized
been submitted
retrieval
These forms
forms have
have been
submitted separately
separately and are
are not reproduced
reproduced
retrieval system.
system. These
here to avoid redundancy with
with the more extensive
extensive narrative
narrative information
information about our
resource
progress provided
provided in this report.
resource and progress
II.A.
1I.A. Scientific
Scientific Subprojects
Our
Our core research
research and
training
training activities
activities are
collaborating
collaborating projects is
II~B.
IIIB.
development actlvltles
page 16,
activities are described
described starting on page
16, our
summarized starting
page 77,
progress of
starting on page
77, and the progress
of our
detailed starting
page 107.
starting on page
107.
Books, Papers,
Papers, and Abstracts
Abstracts
The list
publications for
list of
of recent publications
for our core research
research and development work starts on
page
page 61 and those
those for
for the collaborating
collaborating projects are in the individual
individual reports starting on
page
page 107.
107.
II.C.
1I.C. Resource
Resource Summary Table
The details of
of resource
resource usage,
usage, including
including a breakdown by the various subprojects,
subprojects, is given
in the tables starting
page 79.
starting on page
79.
3
Shortliffe
E. H. Shortliffe
Narrati ve Description
Descri ption
Narrative
5P41-RR00785-14
5P41-RR00785-14
III. Narrative
Narrative Description
Description
III.
III.A. Summary of
of Research
Research Progress
1II.A.
III.A.1. Resource
Resource Overview
III.A.1.
This is an annual report
report for
for year 14 of
of the SUMEX-AIM
SUMEX-AIM resource
resource (grant
(grant RR-00785)
RR-00785),
This
first year of
of a 3-year renewal period
period to support
support further
further research
research on applications
applications of
of
the first
artificial intelligence
intelligence in
in biomedicine.
biomedicine. For
For both
both technical
technical and administrative
administrative reasons,
reasons, we
artificial
merged into
into the June 1985 SUMEX
SUMEX renewal application
application the continuation
continuation of
of work
work on
merged
dissemination of
of medical consultation
consultation systems
systems (ONCOCIN)
(ONCOCIN) that
that
the development and dissemination
as resource-related research
research under grant
grant RR-01631.
RR -01631. Progress
Progress on
been supported as
had been
as well.
ONCOCIN research
research is therefore
therefore now reported here as
core ONCOCIN
These combined efforts
efforts represent an ambitious
ambitious research
research program to:
These
Continue our long-range
long-range core research
research efforts
efforts on knowledge-based
knowledge-based systems
systems
.• Continue
needed for
for biomedical
biomedical
aimed at developing new concepts and methodologies needed
applications.
applications.
disseminating
Substantially extend ONCOCIN
ONCOCIN research
research on developing and disseminating
.• Substantially
clinical decision support
support systems.
systems.
clinical
•. Develop the core system
EX - AIM
system technology to move the national
national SUM
SUMEX-AIM
community
community from
from a dependence
dependence on the central
central SUMEX
SUMEX DEC
DEC 2060
2060 to a fully
fully
distributed,
distributed, workstation-based
workstation-based computing
computing environment.
environment.
•. Introduce
- AIM community
Introduce these
these systems
systems technologies into
into the SUMEX
SUMEX-AIM
community with
with
appropriate
phase
appropriate communications
communications and managerial assistance
assistance to responsibly phase
out
out the central resource
resource and DEC
DEC 2060
2060 mainframe
mainframe in a manner that will
will
support community
become self-sustaining
community efforts
efforts to become
self-sustaining and to continue
continue
scientific
scientific interactions
interactions through fully
fully distributed
distributed means.
means.
•. Maintain
Maintain our aggressive
aggressive efforts
efforts at training
training and dissemination
dissemination to help exploit
exploit
the research
research potential
potential of
of this field.
field.
III.A.I.1.
III.A.l.1. SUMEX-AIM
SUMEX-AIM as
as a Resource
Resource
SUMEX
SUMEX and
and the AIM
AIM Community
Community
In the fourteen
-AIM resource
fourteen years
years since the SUMEX
SUMEX-AIM
resource was
was established
established in late 1973,
1973,
computing
computing technology and biomedical
biomedical artificial
artificial intelligence
intelligence research
research have
have undergone
undergone a
remarkable evolution
evolution and SUMEX
SUMEX has
has both influenced
influenced and responded
responded to these
these changing
technologies.
technologies. It
It is widely recognized that our resource
resource has
has fostered highly
highly influential
influential
work in biomedical
work from
from which much of
of the expert systems
systems field
field emerged
emerged
biomedical AI
AI --- work
--- and that it has
has simultaneously helped
helped define the technological base
base of
of applied AI
AI
research.
research.
The focus of
of the SUMEX-AIM
SUMEX-AIM resource
resource continues to emphasize
emphasize research
research on artificial
artificial
intelligence
intelligence techniques
techniques that guide the design
design of
of computer programs that can
can help with
with
the acquisition, representation, management,
management, and utilization
utilization of
of the many forms
forms of
of
medical knowledge
knowledge in diverse
diverse biomedical
biomedical research
research and clini~al
clinical care
care settings
settings --- ranging
from
from biomolecular
biomolecular structure determination
determination and
and analysis,
analysis, to molecular biology, to clinIcal
clinical
5
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Resource Overview
Overview
Resource
decision support,
support, to medical
medical education. Nevertheless,
Nevertheless, we have long
long recognized that
that the
ultimate impact
impact of
of this
this work
work in
in biomedicine
biomedicine will
will be realized through
through its
its assimilation
assimilation
ultimate
with the full
full range of
of methodologies of
of medical informatics,
informatics, such as data bases,
bases,
with
human-computer interfaces, complex instrument
instrument control,
control, and modeling.
modeling.
biostatistics, human-computer
biostatistics,
From the start, SUMEX-AIM
SUMEX-AIM work
work has
has been
been grounded in
in real-world
real-world applications,
applications, like
like
From
systems for
for the interpretation
interpretation of
of mass
mass spectral informatibn
information about biomolecular
biomolecular
systems
synthesis, interpretation
interpretation of
of x-ray
x-ray diffraction
diffraction data on crystals,.
structures, chemical synthesis,
cognitive modeling,
modeling, infectious
infectious disease
disease diagnosis and therapy, DNA
DNA sequence
sequence analysis,
cognitive
experiment planning
planning and interpretation
interpretation in
in molecular
molecular biology,
biology, and medical instruction.
instruction.
experiment
Our
current
work
extends
this
emphasis
in
application
domains
such
as
oncology
Our current work
this emphasis in application
as oncology
protocol management,
management, clinical
clinical decision support, protein
protein structure
structure analysis, and data base
base
protocol
information retrieval
retrieval and analysis. All
All of
of these
these research
research efforts
efforts have demanded
demanded close
information
collaborations with
with diverse parts of
of the biomedical
biomedical research
research community
community and the
collaborations
integration
of
many
computational methods from
from those domains with
with knowledge-based
integration of
computational
approaches. Even though in
in the beginning
beginning the “AI-in-medicine”
"AI-in-medicine" community
community was
was quite
quite
approaches.
limited and easily-defined,
easily-defined, but
but rather
rather is spreading and is
small, itit is perforce no longer limited
inextricably linked
linked with
with the many biomedical
biomedical applications
applications communities
communities we have
inextricably
collaborated with
with over the years.
years. Driven
Driven both by the on
on-going
diffusion of
of AI
AI and by
-going diffusion
of personal computer workstations
workstations that signal the practical
practical
the development of
decentralization of
of computing
computing resources,
resources, we must develop new resource
resource communication
communication
decentralization
distributed computing
computing technologies that will
will continue
continue to facilitate
facilitate wider intraintra- and
and distributed
inter-community communication,
communication, collaboration,
collaboration, and sharing of
of biomedical
biomedical information.
information.
inter-community
SUMEX Project has
has demonstrated that itit is possible to operate a computing
computing
The SUMEX
research resource
resource with
with a national
national charter a;
a;'.’ that the services
services providable
providable over networks
research
those that facilitate
facilitate the growth
growth of
of AIAI- ii-Medicine.
ll-Medicine. SUMEX
SUM EX now has
has a reputation
reputation
were those
as
national resource,
pulling together
as a model national
resource, pulling
togerher the best available interactive
interactive computing
computing
technology, software, and computer
national
computer communications
communications in the service of
of a national
scientific
Planning
national facilities
Planning groups for
for national
facilities in cognitive
cognitive science,
science,
scientific community.
community.
computer science,
biomathematical modeling have
science, and biomathematical
have discussed
discussed and studied the
SUMEX
SUMEX model and new resources,
resources, like
like the recently instituted
instituted BIONET
BIONET resource
resource for
for
molecular biologists, are closely patterned after
after the SUMEX
SUMEX example.
The projects SUMEX
have generally required substantial computing
resources
SUMEX supports have
computing resources
with excellent interaction.
Even today though, with
with the growing, but by no means
means
with
interaction.
ubiquitous availability
ubiquitous
availability of
of workstations,
workstations, this computing
computing power is still
still hard to obtain
obtain in
all but a few universities.
universities. SUMEX
SUMEX is, in a sense,
sense, a "great
“great equalizer".
equalizer”. A
A scientist
scientist gains
access
access by virtue
virtue of
of the quality
quality of
of his/her
his/her research
research ideas,
ideas, not by the accident of
of where
where
s/he happens
happens to be situated. In other words, the resource
resource follows
follows the ethic of
of the
scientific
journal.
scientific journal.
SUMEX
SUMEX has
has demonstrated that a computer resource
resource is a useful "linking
“linking mechanism"
mechanism” for
for
bringing
bringing together
together and holding
holding together teams
teams of
of experts from
from different
different disciplines
disciplines who
share
share a common problem focus. AI
AI concepts
concepts and software are
are among the most complex
products of
of computer science.
science. Historically
Historically it
it has
has not been
been easy
easy for
for scientists in other
fields
fields to gain access
access to and mastery of
of them. Yet the collaborative
collaborative outreach and
dissemination
dissemination efforts
efforts of
of SUMEX
SUMEX have
have been
been able to bridge the gap
gap in numerous cases.
cases.
Over 36
36 biomedical
biomedical AI
AI application
application projects have
have developed
developed in our national
national community
community
and have
have been
been supported by SUMEX
SUMEX computing
computing resources
resources over the years.
years. And
And 9 of
of
these
these have
have matured to the point
point of
of now continuing
continuing their
their research
research on facilities
facilities outside
of
EX. For example,
of SUM
SUMEX.
example, the BIONET
BIONET resource
resource (named
(named GENET
GENET while at SUMEX)
SUMEX) is
being operated
operated by IntelliGenetics;
IntelliGenetics; the CADUCEUS
CADUCEUS project splits their
their research
research work
between
AX computer, and the SUMEX
between their
their own IBM
IBM PC workstations, a V
VAX
SUMEX resource;
resource:
and
and the Chemical Synthesis
Synthesis project
project now operates
operates entirely
entirely on a VAX
VAX at U.C. Santa
Santa Cruz.
The integration
integration of
of AI
AI ideas
ideas with
with other parts of
of medical informatics
informatics and their
their
dissemination into
into biomedicine
biomedicine is happening largely because
because of
of the development
deveiopment in the
the
E.
E. H. Shonliffe
Shortliffe
6
Resource
Resource Overview
Overview
5P41-RR00785-14
1970's
1970’s and early 1980's
1980’s of
of methods
methods and tools for
for the application
application of
of AI
AI concepts
concepts to
difficult
Their impact
impact was
was heightened
heightened because
because of
of
difficult professional-level
professional-level problem solving. Their
the demonstration in various areas
areas of
of medicine and other life
life sciences
sciences that these
these
methods
EX has
methods and tools really work. Here SUM
SUMEX
has played a key role, so
so much so
so that it
it
is regarded
regarded as
as "the
“the home of
of applied AL"
AI.”
SUMEX
SUMEX has
has been
been the nursery, as
as well as
as the home,
home, of
of such
such well-known
well-known AI
AI systems
systems as
as
DENDRAL
DENDRAL (chemical structure elucidation),
elucidation), MYCIN
MYCIN (infectious
(infectious disease
disease diagnosis and
therapy), INTERNIST
INTERNIST (differential
(differential diagnosis), ACT
ACT (human memory
memory organization),
organization),
ONCOCIN
protocol advice), SECS
ONCOCIN (cancer
(cancer chemotherapy
chemotherapy protocol
SECS (chemical synthesis),
synthesis), EMYCIN
EMYCIN
(rule-based expert system
system tool),
tool), and AGE
AGE (blackboard-based expert system
system tool).
tool). In the
past four
four years,
years, our community
community has
has published a dozen
dozen books that give a scholarly
scholarly
performing. These
perspective
perspective on the scientific
scientific experiments we have
have been
been performing.
These volumes, and
other work
work done at SUMEX, have
have played a seminal role in structuring
structuring modern AI
AI
paradigms
paradigms and methodology.
methodology.
III.A.1.2.
- AIM
III.A.1.2. The Future of
of SUMEX
SUMEX-AIM
Given
Given this background, what is the future
future need
need and course
course for
for SUMEX
SUMEX as
as a resource
resource
--- especially
especially in view of
of the on-going
on-goin g revolution
revolution in computer technology and costs
costs and
the emergence
emergence of
of powerful
powerful single-user workstations and local area
area networking?
networking? The
answers remain clear.
clear.
answers
Research in AI
AI in Biomedicine
Biomedicine
Basic Research
At
At the deepest
deepest research
research level, despite our considerable success
success in working
working on medical
and biological
biological applications, the problems we can
can attack are still
still sharply limited.
limited. Our
current
current ideas
ideas fall
fall short in many ways
ways against today's
today’s important
important health care and
biomedical research
research problems brought
brought on by the explosion in medical knowledge and
for
for which AI
AI should be of
of assistance.
assistance. Just as
as the research
research work
work of
of the 70's
70’s and 80's
80’s in
the SUMEX-AIM
SUMEX-AIM community
community fuels the current
current practical and commercial
commercial applications,
applications,
our work of
of the late 80's
80’s will
will be the basis
basis for
for the next decade's
decade’s systems.
systems.
The report
report of
of the panel
panel on medical informatics
informatics [12],
[12], convened late in 1985
1985 by the
National Library
National
Library of
of Medicine to review and recommend twenty-year
twenty-year goals
goals for
for the
NLM, listed among its highest priority
NLM,
priority recommendations the need
need to greatly expand and
aggressively
pursue an interdisciplinary
aggressively pursue
interdisciplinary research
research program to develop computational
computational
methods
methods for
for acquiring, representing,
representing, managing, and using biomedical
biomedical knowledge of
of all
sorts for
for health care
care and biomedical research.
research. These
These are precisely the problems which
which
the SUMEX-AIM
SUMEX-AIM community
community has
has been
been working
working on so successfully and which will
will
require work well beyond
tial
beyond the five
five year funding
funding period we have
have requested.
requested. It
It is essen
essential
that this line
line of
of research
research in the SUMEX-AIM
SUMEX-AIM community,
community, represented
represented by our core AI
AI
research,
research, the ONCOCIN
ONCOCIN research,
research, and our collaborative
collaborative research
research groups, be continued.
continued.
The Changing Role of
of the Central
Central Resource
Resource
At
At the resource
resource level, there are changing, but still
still intense, needs
needs for
for computing
computing
resources
resources for
for the active AIM
AIM research
research community
community to continue
continue its work
work over the next
five
five years.
years. The workstations to which we directed our attention
attention in 1980
1980 have
have now
potential
demonstrated
demonstrated their
their practicality
practicality as
as research
research tools and, increasingly,
increasingly, as
as potential
mechanisms for
for disseminating AI
AI systems
systems as
as cost-effective
cost-effective decision aids in clinical
clinical
mechanisms
half decade
decade we expect the era of
of highly
highly
settings.
such as
settings-such
as private offices. Over the next half
centralized general
general machines
machines for
for AI
AI research
research will
will come to an end,
end, and be
be replaced
replaced
of distributed
distributed but heterogeneous
heterogeneous single-user machines sharing
gradually by networks of
common information
paths among members
information resources
resources and communication
communication paths
members of
of the
biomedical research
research community.
community.
7
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Resource Overview
Resource
Many of
-AIM resources.
of our community
community groups are still
still dependent
dependent on the SUMEX
SUMEX-AIM
resources. For
those
those that have
have been
been able to take advantage
advantage of
of newly developed
developed local computing
computing
facilities,
facilities, SUMEX-AIM
SUMEX-AIM provides a central cross-roads
cross-roads for
for communications
communications and the
sharing of
of programs and knowledge.
knowledge. In its core research
research and development role,
SUMEX-AIM
has its sights set on the hardware
hardware and software systems
systems of
of the next
SUMEX
- AIM has
decade.
decade. We expect major
major changes
changes in the distributed
distributed computing
computing environments
environments that are
just now emerging in order to make effective
just
effective use
use of
of their
their power and to adapt them to
the development and dissemination
dissemination of
of biomedical AI
AI systems
systems for
for professional user
user
communities.
communities. In its training
training role, SUMEX
SUMEX is a crucial resource
resource for
for the education of
of
badly needed
needed new researchers
researchers and professionals
professionals to continue
continue the development of
of the
biomedical
biomedical AI
AI field.
field. The "critical
“critical mass"
mass” of
of the existing physical SUMEX
SUMEX resource,
resource, its
development staff,
staff, and its intellectual
intellectual ties with
with the Stanford
Stanford Knowledge Systems
Systems
Laboratory,
Laboratory, make this an ideal setting to integrate, experiment
experiment with,
with, and export
export these
these
for the rest of
of the AIM
AIM community.
community.
methodologies for
At
At the beginning, the SUMEX
SUMEX community
community was
was small and idea-limited,
idea-limited, and the central
SUMEX
Now the community
SUMEX computer facility
facility was
was an ideal vehicle for
for the research.
research. Now
community is
large,
progress is limited
large, and the momentum
momentum of
of the science
science is such
such that its progress
limited by
size and scientific
scientific maturity
maturity of
of the
computing
computing power and research
research manpower. The size
SUMEX community
community has
has fully
fully consumed
consumed the computing
computing resource
resource in every critical
critical
SUMEX
dimension --- CPU power, main
main memory size,
size, address
address space,
space, and file
file space
space --- and has
has
of our work
work has
has already
overflowed
overflowed to decentralized machines
machines of
of many types.
types. Much of
been focussed
focussed on developing and experimenting
experimenting with
with workstation
workstation environments
environments for
for
been
biomedical AI
AI applications.
applications. We are fully
fully committed
committed to continuing
continuing this line
line of
of research
research
biomedical
will continue
continue our experimental
experimental
for
for the future
future hardware thrust of
of the resource.
resource. We will
approach to these
these systems,
systems, rejecting articles of
of faith
faith for
for real experience.
experience. We must learn
build and exploit
exploit distributed
distributed networks of
of these
these machines
machines and to build
build and manage
manage
to build
for these
these systems.
systems. Since
Since decentralization
decentralization is central to our future,
future, we
graceful software for
must learn its technical characteristics.
The resource
resource development directions
directions we
we have
have sketched
sketched have
have received
received substantial external
impetus as
as well [12, 2, 7].
71. For
For example,
example, another of
of the key recommendations of
of the
NLM
informatics planning
planning panel [12J
[12] was
was that high-speed network
network
NLM medical informatics
communication links
links be established
established throughout
throughout the biomedical
biomedical research
research community
community so
so
communication
knowledge and information
information can be shared
shared across
across diverse research
research groups and that
that knowledge
the required interdisciplinary
interdisciplinary collaborations
collaborations can take place.
place. A
A principal
principal goal from
from the
of SUMEX
SUMEX-AIM
has been
been to experiment with
with these
these electronic
electronic links,
links, but
but SUMEX
SUMEX
start of
- AIM has
is only
Nevertheless, it
only a start toward this broad goal. Nevertheless,
it continues to be an important
important
pathfinder
pathfinder to develop the technology and community
community interaction
interaction tools needed
needed to expand
expand
community system
system and communication
communication resources.
resources.
community
Highlights
Highlights of
of Long-term
Long-term Goals
Maintain the synergIstIc
synergistic relationship
relationship between
between SUMEX
SUMEX core system
system
•. Maintain
development, core AI
AI research,
research, our experimental efforts
efforts at disseminating
disseminating
clinical
clinical decision-making
decision-making aids,
aids, and new applications
applications efforts.
efforts.
Continue to serve
serve the national
national AIM
AIM research
research community,
community, less
less and less
less as
as a
•. Continue
source of
of raw computing
computing cycles
cycles and more and more as
as a transfer
transfer point
for
source
point for
technologies important
important for
for community
community research
research and communication.
communication.
new technologies
We will
will also continue
continue our coordinating
coordinating role within
within the community
community through
electronic
electronic media and periodic
periodic AIM
AIM workshops.
workshops.
Maintain our connections to ARPANET,
ARPANET, TELENET,
TELENET, and our local Ethernet
Ethernet
•. Maintain
and assist
assist other community
community members
members to establish similar
similar links
links by example,
by integrating
integrating and providing
providing enabling software, and by offering
offering advice and
support within
within our resources.
resources.
Shortliffe
E. H. Shortliffe
8
Resource Overview
Overview
Resource
5P41-RR00785-14
5P41-RR00785-14
new computing
computing resource
resource developments on
on more
more effective
effective exploitation
exploitation
.• Focus new
of distributed
distributed workstations
workstations through
through better
better communication
communication and cooperative
cooperative
of
computing tools,
tools, using
using transparent
transparent digital
digital networking
networking schemes.
schemes.
computing
computing environments
environments of
of workstations
workstations so that
that minimal
minimal
.• Enhance the computing
dependency on
on central,.
central, general-purpose computing
computing hosts remains and these
these
mainframe
time-sharing
systems
can
be
phased
out
eventually.
Remaining
Remaining
mainframe time-sharing systems
phased out
central resources
resources will
will include
include servers for
for communications,
communications, community
community
central
information resources,
resources, and special computing
computing architectures (e.g.,
(e.g., shared- or
or
information
distributed-memory symbolic
symbolic multiprocessors)
multiprocessors) justified
justified by cost-effectiveness
distributed-memory
and unique functionality.
functionality.
of the local
Incrementally phase-in,
phase-in, disseminate, and evaluate
evaluate those aspects
aspects of
.• Incrementally
distributed computing
computing resource
resource that
that are necessary
necessary for.
for. continuing
continuing national
national
distributed
community support
support within
within this
this distributed
distributed paradigm.
This will
will
AIM community
This
AIM
point the way towards the distributed
distributed computing
computing resource
resource model
ultimately point
ultimately
that we believe will
will interlink
interlink this community
community well into
into the next decade.
decade.
that
Gradually and responsibly
responsibly phase
phase out
out the existing DEC 2060 machine as
as
.• Gradually
effective distributed
distributed computing
computing alternatives become
become widely available. We
effective
fifth years
years of
of
expect this
this to be possible sometime during
during the fourth
fourth through fifth
continuation resource.
resource.
the continuation
Continue the central
central staff
staff and management
management structure, essentially unchanged
unchanged
.• Continue
function during
during the five-year
five-year transition
transition period, except for
for the
in size and function
in
of the core part of
of the ONCOCIN
ONCOCIN research
research with
with the SUMEX
SUMEX
merging of
resource.
resource.
99
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
Resource
Resource Definitions
Definitions and Goals
III.A.2.
III.A.2. Resource
Resource Definitions
Definitions and
and- Goals
SUMEX-AIM
promoting
SUMEX-AIM is a national
national computer
computer resource
resource with
with a multiple
multiple mISSIon:
mission: a) promoting
experimental
experimental applications
applications of
of computer
computer science
science research
research in artificial
artificial intelligence
intelligence (AI)
(AI) to
biological
biological and medical problems, b) studying methodologies for
for the dissemination
dissemination of
of
biomedical
basic AI
biomedical AI
AI systems
systems into
into target user
user communities,
communities, c) supporting
supporting the basic
AI research
research
that underlies applications,
applications, and d) facilitating
facilitating network-based computer resource
resource sharing,
collaboration,
collaboration, and communication
communication among a national
national scientific
scientific community
community of
of health
EX - AIM resource
research
SUMEX-AIM
resource is located physically in the Stanford
Stanford
research projects. The SUM
University
University Medical School and serves
serves as
as a nucleus
nucleus for
for a community
community of
of medical AI
AI
projects at universities
SUMEX provides computing
computing facilities
facilities tuned
universities around the country.
country. SUMEX
to the needs
needs of
of AI
AI research
research and communication
communication tools to facilitate
facilitate remote access,
access,
interinter- and intra-group
intra-group contacts, and the demonstration
demonstration of
of developing computer
computer
programs to biomedical
biomedical research
research collaborators.
collaborators.
III.A.2.t.
III.A.2.1. Knowledge-Based
Knowledge-Based System Research
Research
The SUMEX
SUMEX Project has
has given strong
stron g impetus to the development of
of knowledge-based
knowledge-based
system
biomedicine.
Knowiedgebased system
system research
research in biomedicine.
Knowledge-based
system research
research is that part of
of
computer
processes, and the representation
computer science
science that investigates symbolic
symbolic reasoning
reasoning processes,
of
of symbolic
symbolic knowledge for
for use
use in inference
A knowledge-based
knowledge-based or expert system
system is a
inference 11.. A
procedures to solve problems that
computer program that uses
uses knowledge and inference procedures
are difficult
difficult enough
enough to require significant
significant human expertise for
for their
their solution.
solution. For some
some
perform at such
fields
fields of
of work, the knowledge necessary
necessary to perform
such a level, plus the inference
procedures used,
procedures
used, can be thought
thought of
of as
as a model of
of the expertise
expertise of
of the expert
practi
tioners of
practitioners
of that field.
field.
The knowledge of
facts
of an expert system
system consists
consists of
of facts and heuristics. The facts
constitute
publicly available, and generally
constitute a body of
of information
information that is widely shared,
shared, publicly
agreed
agreed upon by experts in a field.
field. The heuristics
heuristics are the mostly-private,
mostly-private, little-discussed
little-discussed
judgment (rules of
rules of
of good judgment
of plausible reasoning
reasoning and of
of good guessing)
guessing) that
characterize expert-level
Our work
expert-level decision-making
decision-making in the field.
field.
work views heuristic
heuristic
knowledge to be of
factual knowledge,
of equal
equal importance
importance with
with factual
knowledge, indeed
indeed to be the essence
essence
of
primarily a
of an expert system
system is primarily
of what we call expertise.
expertise. The performance level of
function
possesses.
function of
of the size
size and quality
quality of
of the knowledge base
base that it
it possesses.
-AIM community
Projects in the SUMEX
SUMEX-AIM
community are concerned
concerned in some
some way with
with the
projects
application
application of
of AI
AI to biomedical
biomedical research.
research. Brief
Brief abstracts
abstracts of
of the various .projects
currently
EX resource
currently using the SUM
SUMEX
resource can be found
found in Appendix
Appendix B and more detailed
progress summaries in Section IV.
progress
IV. The most tangible objective of
of this approach is the
development of
of computer
computer programs that will
will be more general
general and effective
effective consultative
tools for
for the clinician
clinician and medical scientist. All
All of
of these
these research
research efforts
efforts have
have
demanded
demanded close
close collaborations
collaborations with
with diverse parts of
of the biomedical research
research community
community
and the integration
integration of
of many computational
computational methods from
from those
those domains with
with
knowledge-based
knowledge-based approaches.
approaches. We have
have long recognized
recognized that the ultimate
ultimate impact
impact of
of this
work
work in biomedicine
biomedicine will
will be realized through its assimilation
assimilation with
with the full
full range
range of
of
methodologies of
of medical informatics,
informatics, including,
including, for
for example,
example, data base
base research,
research,
biostatistics,
biostatistics, decision support, complex instrument
instrument control,
control, and modeling.
There have
promising results in many application
have already been
been promising
application areas,
areas, even
even though stateof-the-art
of-the-art programs are far
far more narrowly
narrowly specialized
specialized and inflexible
inflexible than the
Needless to say,
corresponding aspects
aspects of
of human intelligence
intelligence they emulate.
emulate. Needless
say, much is yet
1Many
‘Many introductory
introductory and survey
survey texts have been written
written by now
now on Al
AI and knowledge-based
knowledge-based or
or expert
expert
systems. See
See for
for example
example [1,
[l, 11, 13, 5, 23, 4, 18].
is].
E. H. Shortliffe
Shortliffe
10
Resource
Resource Definitions
Definitions and Goals
5P41RR00785-14
5P41-RR00785-14
to be learned in the process
process of
of fashioning
fashioning a coherent scientific
scientific discipline
discipline out of
of the
experimental
procedures, and emerging theoretical structure
experimental programs, mathematical
mathematical procedures,
comprising
comprising knowledge-based
knowledge-based system
system research.
research.
III.A.2.2.
III.A.2.2. Resource
Resource Sharing
Sharing
An
EX - AIM resource
An equally important
important function
function of
of the SUM
SUMEX-AIM
resource is an exploration
exploration of
of the
use
between
use of
of computer
computer communications
communications as
as a means
means for
for interactions
interactions and sharing between
geographically
geographically remote research
research groups engaged
engaged in biomedical
biomedical computer science
science research
research
and for
This facet of
of scientific
scientific interaction
interaction is
for the dissemination
dissemination of
of AI
AI technology. This
becoming increasingly
increasingly important
important with
with the explosion of
of complex information
information sources
sources
and the regional specialization
specialization of
of groups and facilities
facilities that might
might be shared
shared by remote
researchers
And, as
as projected, we are seeing
seeing a growing decentralization
decentralization of
of
researchers [10,
[ 10, 3].
31. And,
computing
computing resources
resources with
with the emerging technology in microelectronics
microelectronics and a
correspondingly
correspondingly greater role for
for digital
digital communications
communications to facilitate
facilitate scientific
scientific exchange.
exchange.
Our community
effort is based
based upon the developing state
state of
of distributed
distributed
community building
building effort
computing
computing and communications
communications technology. While
While far
far from
from perfected, these
these capabilities
capabilities
offer
offer powerful
powerful tools for
for collaborative
collaborative linkages, both within
within a given research
research project and
among them.
A
based upon the
A number of
of the active projects on SUMEX
SUMEX are
are based
collaboration
collaboration of
of computer
computer and medical scientists at geographically separate
separate institutions,
institutions,
separate
separate both from
from each
each other and from
from the computer
computer resource
resource (see
(see for
for example,
example, the
MENTOR
MENTOR and PathFinder
PathFinder projects).
In the early 1970's,
1970’s, the initial
initial model for
for SUMEX-AIM
SUMEX-AIM as
as a centralized resource
resource was
was
based
powerful computing
based on the high cost of
of powerful
computing facilities
facilities and the infeasibility
infeasibility of
of being
able to duplicate
duplicate them readily. This
This central role has
has already evolved significantly
significantly and
continues to change
change with
with the introduction
introduction of
of more compact and inexpensive computing
computing
technology now available at many more research
research sites.
sites. At
At the same
same time, the number
of
of active groups working
working on biomedical
biomedical AI
AI problems has
has grown and the established
established
ones
ones have increased
increased in size.
size. This
This has
has led to a growth
growth in the demand
demand for
for computing
computing
resources
-AIM could reasonably
resources far
far beyond what SUMEX
SUMEX-AIM
reasonably and effectively
effectively provide on a
national
therefore turned our core systems
systems research
research to actively
national scale.
scale. We have therefore
supporting
supporting the development of
of distributed
distributed computing
computing and communications
communications resources
resources to
facilitate
facilitate collaborative
collaborative project
project research
research and continued
continued inter-group
inter-group communications.
communications.
Thus, as
become established,
balance of
as more remotely available resources
resources have become
established, the balance
of the
pilot projects
use
use of
of the SUMEX-AIM
SUMEX-AIM resource
resource has
has shifted
shifted toward supporting
supporting start-up
start-up pilot
and the growing
growing AI
AI research
research community
community at Stanford.
Stanford.
III.A.2.3.
III.A.2.3. Significance
Significance and Impact
Impact in Biomedicine
Artificial
Artificiai intelligence
intelligence is the computer
computer science
science of
of representations of
of symbolic knowledge
knowledge
and its use
problem-solving processes.
processes.
For computer
use in symbolic
symbolic inference
inference and problem-solving
applications
biology, this research
Medicine and
applications in medicine and biology,
research path is crucial. Medicine
biology
physics and engineering,
biology are not
not presently mathematically-based
mathematically-based sciences;
sciences; unlike
unlike physics
they are seldom capable
capable of
of exploiting
exploiting the mathematical characteristics of
of computation.
computation.
They are essentially inferential,
inferential, not
not calculational,
calculational, sciences.
sciences. If
If the computer revolution
revolution
is to affect
affect biomedical
biomedical scientists, computers will
will be used
used as
as inferential
inferential aids.
aids.
The growth
practitioner
growth in medical knowledge has
has far
far surpassed
surpassed the ability
ability of
of a single practitioner
to master it
processing capacity thereby
it ail,
all, and the computer's
computer’s superior
superior information
information processing
offers
processes of
offers a natural
natural appeal.
appeal. Furthermore,
Furthermore, the reasoning processes
of medical experts
experts are
are
poorly
poorly understood; attempts to model expert decision-making
decision-making necessarily
necessarily require a
degree
degree of
of introspection
introspection and a structured
structured experimentation
experimentation that may,
may, in turn, improve
improve the
physician'ss own clinical
quality
quality of
of the physician’
clinical Jecisions,
decisions, making them more reproducible and
11
11
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Resource Definitions
Definitions and Goals
Resource
insights that
that result
result may also allow
allow us more adequately to teach medical
defensible .. New insights
defensible,.
staff the techniques for
for reaching good decisions, rather
rather than merely
students and house staff
to offer
offer a collection
collection of
of facts which
which they must independently
independently learn to
to utilize
utilize coherently.
to
refinement
Perhaps the larger impact
impact on medicine
medicine and biology
biology will
will be the exposure and refinement
Perhaps
of the hitherto
hitherto largely private
private heuristic
heuristic knowledge of
of the experts of
of the various fields
fields
of
ethic of
of science
science that
that calls for
for the public
public exposure and criticism
criticism of
of
studied. The ethic
knowledge has
has traditionally
traditionally been flawed
flawed for
for want of
of a methodology
methodology to evoke and give
form to the heuristic
heuristic knowledge of
of scientists. AI
AI methodology
methodology is beginning
beginning to fill
fill that
form
need. Heuristic
Heuristic knowledge can be elicited,
elicited, studied, critiqued
critiqued by peers,
peers, and taught to
need.
students.
importance of
of AI
AI research
research and its applications
applications is increasing in
in general, without
without
The importance
for the specific
specific areas
areas of
of biomedical
biomedical interest. AI
AI is one of
of the principal
principal fronts
fronts
regard for
which university
university computer
computer science
science groups are expanding. The
The pressure
pressure from
from
along which
career-line choices
choices is great: to cite an admittedly
admittedly special case,
case, approximately
approximately
student career-line
of the students applying
applying to Stanford’
Stanford's
computer science
science Ph.D. program cite AI
AI as
as a
80% of
s computer
possible field
field of
of specialization
specialization (up from
from 30% a few years
years ago). Federal and industrial
industrial
support for
for AI
AI research
research is vigorous and growing, although support
support specifically
specifically for
for
support
biomedical applications
applications continues to be limited.
limited.
All of
of the major
major computer
computer
All
biomedical
manufacturers (e.g.,
(e.g., IBM,
IBM, DEC, TT,
TI, UNISYS,
UNISYS, HP, and others) are using and marketing
marketing
manufacturers
AI technology aggressively
aggressively and many software companies are putting
putting more and more
AI
Many other
other parts of
of industry
industry are also actively
actively pursuing AI
AI
products on the market. Many
applications in their
their own contexts, including
including defense
defense and aerospace
aerospace companies,
companies,
applications
manufacturing companies, financial
financial companies, and others.
manufacturing
Despite the limited
limited research
funding available, there is also an explosion of
research funding
of interest in
medical AI.
principal
AI. The American
American Association
Association for
for Artificial
Artificial Intelligence
Intelligence (AAAI),
(AAAI), the principal
scientific
scientific membership organization
organization for
for the AI
AI field,
field, has
has 7000
7000 members,
members, over 1000
1000 of
of
whom are members of
of the medical special
special interest group known as
as the AAAI-M.
AAAI-M.
Speakers
Speakers on medical AI
AI are prominently
prominently featured at professional medical meetings,
meetings, such
such
as
as the American
American College of
of Pathology and American
American College of
of Physicians meetings;
meetings; a
decade
intelligence
artificial
intelligence were never heard at such
such conferences.
conferences. And
And
decade ago,
ago, the words artificial
at medical computing
Computer
computing meetings,
meetings, such
such as
as the annual Symposium on Computer
Applications
Applications in Medical Care (SCAMC)
(SCAMC) and the international
international MEDINFO
MEDINFO conferences,
conferences,
the growing
papers on AI
growing interest in AI
AI and the rapid increase
increase in papers
AI and expert systems
systems
are further
further testimony
testimony to the impact
impact that the field
field is having.
AI
Such
Such diverse
diverse
AI is beginning
beginning to have
have a similar
similar effect
effect on medical education.
organizations
organizations as
as the National
National Library
Library of
of Medicine,
Medicine, the American
American College of
of Physicians,
Physicians,
the Association of
of American
American Medical Colleges,
Colleges, and
and the Medical Library
Library Association
have
have all called for
for sweeping
sweeping changes
changes in medical education, increased
increased educational use
use of
of
computing
computing technology, enhanced
enhanced research
research in medical computer science,
science, and
and career
career
development for
for people
people working
working at the interface
interface between
between medicine and computing.
computing.
They all cite evolving
-AIM) AI
evolving computing
computing technology and (SUMEX
(SUMEX-AIM)
AI research
research as
as key
motivators.
motivators. At
At Stanford,
Stanford, we have
have vigorous special
special programs for
for student training
training and
research
research in AI
AI --- a new graduate
graduate program in Medical Information
Information Sciences
Sciences and the
two-year
two-year Masters
Masters Degree
Degree in Al
AT program. All
All of
of these
these have
have many more applicants than
available slots.
slots. Demand for
for their
their graduates,
graduates, in both academic
academic and industrial
industrial settings,
settings, is
so
so high that students
students typically
typically begin to receive
receive solicitations
solicitations one or two years
years before
completing
completing their degrees.
degrees.
III.A.2.4.
III.A.2.4. Summary of Current
Current Resource
Resource Goals
The following
following outlines the specific
specific objectives of
of the SUMEX-AIM
SUMEX-AIM resource
resource during the
the
current
current three-year award
award peflod
period begun
begun in August 1986.
1986. It
It provides an
an overall research
research
E.
E. H. Shortliffe
Shortliffe
12
12
Resource Definitions
Definitions and
and Goals
Goals
Resource
5P41-RR00785-14
5P41-RR00785-14
for the
the resource
resource and
and provides
provides the
the backdrop
backdrop against
against which
which specific
specific progress
progress is
is
plan for
plan
reported. Note
Note that
that these
these objectives cover
cover only
only the
the resource
resource nucleus;
nucleus; objectives for
for
reported.
individual collaborating
collaborating projects
projects are discussed
discussed in
in their
their respective
respective reports
reports in
in Section IV.
IV.
individual
Specific aims
aims are
are broken
broken into
into five
five categories:
categories: 1)
1) Technological
Technological Research
Research and
and
Specific
Development, 2) Collaborative
Collaborative Research,
Research, 3) Service and Resource Operations,
Operations, 4)
Development,
Training and Education,
Education, and 5) Dissemination.
Dissemination.
Training
Technological Research
Research and
and Development
Development
I)1) Technological
SUMEX funding
funding and computational
computational support
support for
for core research
research is complementary
complementary to
SUMEX
similar funding
funding from
from other
other agencies
agencies (including
(including DARPA,
DARPA, NASA,
NASA, NSF, NLM,
NLM, private
private
similar
foundations, and industry)
industry) and contributes
contributes to
to the long-standing
long-standing interdisciplinary
.interdisciplinary effort
effort
foundations,
Stanford in
in basic AI
AI research
research and expert
expert system ,design.
·design. We
We expect this
this work
work to
at Stanford
provide the underpinnings
underpinnings for
for increasingly
increasingly effective
effective consultative
consultative programs in
in medicine
medicine
provide
for more practical
practical adaptations of
of this
this work
work within
within emerging microelectronic
microelectronic
and for
technologies. Specific
Specific aims include:
include:
technologies.
research on AI
AI techniques applicable
applicable to biomedical
biomedical problems. Over
.• Basic research
the next
next term we will
will emphasize
emphasize work
work on blackboard
blackboard problem-solving
problem-solving
frameworks and architectures, knowledge acquisition
acquisition or
or learning,
learning, constraint
constraint
frameworks
satisfaction, and qualitative
qualitative simulation.
simulation.
satisfaction,
for disseminating
disseminating application
application systems
systems such as
as
.• Investigate methodologies for
clinical decision-making
decision-making advisors into
into user
user groups.
This will
will include
include
This
clinical
generalized systems
systems for
for acquiring,
acquiring, representing and reasoning about complex
generalized
protocols such as
as are used
used in
in cancer chemotherapy and which
treatment protocols
might
used for
might be used
for clinical
clinical trials.
trials.
•. Support community
community efforts
efforts to organize and generalize AI
AI tools and
architectures that have
been
developed
in
the
context
have been developed
context of
of individual
individual
application
This will
will include
include retrospective evaluations of
of systems
systems
application projects. This
like
like the AGE
AGE blackboard experiment
experiment and work
work on new systems
systems such as
as BBI,
BBl,
MRS, SOAR, EONCOCIN,
EONCOCIN, EOPAL,
EOPAL, Meta-ONYX,
Meta-ONYX, and architectures for
for
concurrent
The objective is to evolve a body of
of
concurrent symbolic
symbolic computing.
computing.
software tools that can
can be used
used to more efficaciously
efficaciously build
build future
future
knowledge-based
knowledge-based systems
systems and explore other biomedical
biomedical AI
AI applications.
applications.
•. Develop more effective
effective workstation
workstation systems
systems to serve
serve as
as the basis
basis for
for
research,
seek
research, biomedical
biomedical application
application development, and dissemination.
dissemination. We seek
to coordinate basic
basic research,
research, application
application work, and system
system development so
so
that the AI
software
we
develop
for
the
next
5-10
years
AI
we develop for
years will
will be
be appropriate
appropriate
to the hardware
hardware and system
system software environments
environments we
we expect
expect to be practical
by then.
then. Our purchases
purchases of
of new hardware will
will be limited
limited to experimentation
experimentation
with state-of
-the-art workstations as
state-of-the-art
as they become
become available for
for our system
system
developments.
developments.
2)
2) Collaborative
Collaborative Research
•. Encourage
Encourage the
the exploration
exploration of new
new applications
applications of AI to biomedical
biomedical research
research
and
and improve
improve mechanisms
mechanisms for
for interinter- and
and intra-group
intra-group collaborations
collaborations and
and
communications.
While
While AI
AI isis our defining
defining theme,
theme, we
we may consider
exceptional
exceptional applications
applications justified
justified by some
some other unique feature of
of SUMEXSUMEXAIM
will continue to
AIM essential
essential for
for important
important biomedical research.
research. We will
exploit
exploit community
community expertise
expertise and
and sharing
sharing in software
software development.
•. Minimize
Minimize administrative
administrative
barriers
barriers
to
to the
the community-oriented
community-oriented
l3
13
goals
goals of
of
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Resource Definitions
Definitions and Goals
Resource
our resources
resources toward purely
purely scientific
scientific goals. We
SUMEX - AIM and direct
direct our
SUMEX-AIM
will retain
retain the current
current user funding
funding arrangements for
for projects working
working on
will
SUMEX facilities.
facilities.· User projects will
will fund
fund their
their own manpower and local
SUMEX
needs; actively
actively contribute
contribute their
their special expertise to the SUMEX-AIM
SUMEX - AIM
needs;
community; and receive an allocation
allocation of
of computing
computing resources
resources under the
community;
control of
of the AIM
AIM management committees. We will
will begin charging
charging “fees
"fees
control
for service”
service" to
to Stanford
Stanford users
users as
as DRR
DRR support
support for
for the DEC
DEC 2060 is phased
phased
for
Fees to national
national users
users will
will be delayed as
as long as
as financially
financially possible.
possible .
out. Fees
Provide effective
effective and geographically
geographically accessible
accessible communication
communication facilities
facilities to
.• Provide
community for
for remote collaborations,
collaborations, communications
communications
SUMEX-AIM community
the SUMEX-AIM
among distributed
distributed computing
computing nodes,
nodes, and experimental
~xperimental testing of
of AI
AI
will retain
retain the. current
current ARPANET
ARPANET and TELENET
TELENET
We will
programs.
for at least the near term and will
will actively
actively explore
explore other
other
connections for
advantageous connections
connections to new communications
communications networks and to dedicated
advantageous
links.
links.
3) Service
Service and Resource
Resource Operations
Operations
3)
not have the computing
computing or manpower capacity to provide
provide routine
routine
SUMEX - AIM does
does not
SUMEX-AIM
community of
of mature projects that has
has developed
developed over the years.
years.
service to the large community
their computing
computing needs
needs are better met by the appropriate
appropriate development
development of
of their
their
Rather, their
has the primary
primary focus of
of
own computing
computing resources
resources when justified.
justified. Thus, SUMEX-AIM
SUMEX -AIM has
assisting new start-up
start-up or
or pilot
pilot projects in
in biomedical
biomedical AI
AI applications
applications in addition
addition to its
assisting
research in the setting
setting of
of a sizable number
number of
of collaborative
collaborative projects. We do offer
offer
core research
continuing support
support for
for projects through the lengthy process
process of
of obtaining
obtaining funding
funding to
continuing
establish
establish their
their own computing
computing base.
base.
4)
Education
4) Training
Training and Education
•. Provide documentation
documentation and assistance
assistance to interface
interface users
users to resource
resource facilities
facilities
and systems.
systems.
•. Exploit
Exploit particular
particular areas
areas of
of expertise within
within the community
community for
for assisting In
in
the development of
pilot efforts
of pilot
efforts in new application
application areas.
areas.
•. Accept visitors
visitors in Stanford
Stanford research
research groups within
within limits
limits of
of manpower,
space,
space, and computing
computing resources.
resources.
•. Support the Medical Information
AI student programs at
Information Science
Science and MS/
MS/AI
Stanford to increase
increase the number of
of research
research personnel available to work
work on
biomedical AI
AI applications.
applications.
•. Support workshop activities
activities including
including collaboration
collaboration with
with other community
community
groups on the AIM
AIM community
community workshop and with
with individual
individual projects for
for
more specialized workshops covering specific
specific research,
research, application,
application, or system
system
dissemination
dissemination topics.
5)
5) Dissemination
Dissemination
While
While collaborating
collaborating projects are
are responsible for
for the development and dissemination
dissemination of
of
their own AI
EX resource
AI systems
systems and results,
results, the SUM
SUMEX
resource will
will work to provide
community-wide
community-wide support for
for dissemination
dissemination efforts
efforts in areas
areas such
such as:
as:
•. Encourage,
Encourage, contribute
contribute to, and
and support the on-going
on-going export of
of software
E.
E. H. Shortliffe
Shortliffe
14
14
Resource
Resource Definitions
Definitions and Goals
5P41-RR00785-14
systems
systems and tools
development.
within
within
the AIM
AIM
community
community
and for
for
commercial
commercial
•. Assist in the production
production of
of video tapes
tapes and films
films depicting
depicting aspects
aspects of
of AIM
AIM
community
community research
research..
papers, and basic research
•. Promote the publication
publication of
of books,
books, review papers,
research articles
on all aspects
- AIM research.
aspects of
of SUMEX
SUMEX-AIM
research.
15
15
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
III.A.3. Details
Details of
of Technical
Technical Progress
Progress
III.A.3.
This section gives an overview
overview of
of progress
progress for
for the nucleus
nucl€us of
of the SUMEX-AIM
SUMEX-AIM
This
resource. A
A more detailed discussion of
of our
our progress
progress in
in specific
specific areas
areas and related plans
resource.
for further
further work
work are presented
presented in
in Section III.A.3.2.
III.A.3.2.
progress for
for
Objectives and progress
for
individual collaborating
collaborating projects are discussed
discussed in
in their
their respective reports in
in Section IV.
IV.
individual
collaborative projects collectively
collectively provide
provide much of
of the scientific
scientific basis for
for
These collaborative
SUMEX as
as a resource
resource and our
our role in
in assisting
assisting them has been a continuation
continuation of
of that
that
SUMEX
in the past.
past. Collaborating
Collaborating projects are autonomous in
in their
their management and
evolved in
provide their
their own manpower and expertise for
for the development
development and dissemination
dissemination of
of
provide
their AI
AI programs.
their
III.A.3.t. Progress
Progress Highlights
Highlights
III.A.3.1.
highlights of
of SUMEX-AIM
SUMEX-AIM resource
resource activities
activities over the
In this section we summarize highlights
April 1987),
resource nucleus.
nucleus.
1987), focusing on the resource
past year (May 1986
1986 - April
made significant
significant progress
progress in
in the core ONCOCIN
ONCOCIN research
research work
work to
.• We have made
for clinical
clinical trial
trial management
management from
from the initial
initial cancer
generalize the tools for
management application.
application. We began
began examining
examining the structures
chemotherapy management
of protocols across
across several
several medical subspecialties
subspecialties other
other than cancer
of
concentrating this year on insulin
insulin diabetes
diabetes treatment.
chemotherapy, concentrating
Graphical tools are under development to facilitate
facilitate protocol
protocol definition
definition and
Graphical
knowledge base
base entry
entry and we worked on model-based reasoning to infer
infer
knowledge
protocol therapeutic actions not
not explicitly
explicitly encoded
encoded in
in the decision plan. We
protocol
have
have also
also continued
continued to examine the issues
issues of
of disseminating
disseminating the ONCOCIN
ONCOCIN
system
system into
into actual clinical
clinical settings.
settings.
•. We made
progress in
primarily in the areas
made significant
significant progress
in core AI
AI research,
research, primarily
areas of
of
knowledge
symbolic
knowledge representation, blackboard frameworks,
frameworks, parallel symbolic
computing
computing architectures, and machine learning. Work
Work has
has advanced
advanced on the
representation of
problem-solving and
of explicit
explicit strategic knowledge
knowledge for
for problem-solving
blackboard control
trade-offs
of
control knowledge,
knowledge, including
including cost/benefit
cost/benefit
trade-offs
of
increasingly complex control
control reasoning.
reasoning. The parallel architectures work
work has
has
developed
developed a flexible,
flexible, instrumented simulator
simulator of
of distributed-memory,
distributed-memory,
multiprocessor
parallel blackboard
alternative parallel
multiprocessor architectures and two alternative
frameworks for
been applied to
for expressing
expressing application
application problems.
problems. These
These have
have been
several
several signal understanding problems with
with promising
promising nearly linear
linear problemproblemsolving speedup.
The machine learning work has
has concentrated on
speedup.
explanation-based generalization and chunking
chunking work
work in the SOAR
framework,
framework, inductive
inductive rule learning, and tools for
for debugging
debugging knowledge
structures.
Work has
has also
also continued on reasoning
reasoning with
with uncertainty
uncertainty to find
find
structures. Work
ways
ways of
of combining
combining formal
formal and informal
informal approximate
approximate reasoning
reasoning methods.
methods.
We also
also continued
continued work on extending and refining
refining the BB1
BBl blackboard
blackboard
system.
system.
•. We have
have made
made excellent progress
progress on the core system
system development work
work
targeted
targeted at supporting
supporting the distributed
distributed AIM
AIM community.
community. We have
have continued
continued
implementation
implementation of
of uniform
uniform network protocol standards
standards for
for remote
workstation
workstation access,
access,redirected our virtual
virtual graphics work to take
take advantage
advantage of
of
the X window protocol being adopted
adopted by many workstation
workstation vendors,
vendors, and
implemented prototype communication
communication tools that integrate text and graphics
between
between linked
linked machines.
machines. We have
have concentrated
concentrated on the NFS protocol
protocol for
for
distributed
distributed file
file access
access and
and have
have got experimental versions of
of this and the
underlying
underlying remote procedure
procedure call facilities
facilities working
working or underway
underway for
for all of
of
E.
E. H. Shortliffe
Shortliffe
16
16
of Technical
Technical Progress
Progress
Details of
Details
5P41-RR00785-14
5P41-RR00785-14
our workstations.
workstations. An
An additional
additional service is being
being implemented
implemented to
to allow
allow
our
remote database
database queries through
through remote
remote procedure
procedure calls to
to a standard
remote
relational database.
database. We
We have
have aa prototype
prototype distributed
distributed electronic
electronic mail
mail system
relational
on
Xerox
D-machines
and
will
be
extending
and
porting
this to
to
working
working on Xerox D-machines and will be extending and porting this
other environments
environments shortly.
shortly. We
We have also made important
important progress
progress in
in
other
for text
text processing,
processing, file
file
extending the general computing
computing environments
environments for
extending
management, printing,
printing, communications,
communications, and other
other services
services on specific
specific
of 66 different
different operating
operating
workstation environments,
environments, including
including the support
support of
workstation
environments.
system environments.
We have continued
continued the dissemination
dissemination of
of SUMEX-AIM
SUMEX-AIM technology through
through
.• We
for our
our AI
AI
various media. We
We have reorganized the distribution
distribution system
system for
various
BB1) to academic,
academic,
software iools
tools (EMYCIN,
(EMYCIN, AGE,
AGE, MRS, SACON, and BBl)
software
industrial, and federal research
research laboratories, in
in order to make itit more
industrial,
efficient and require
require less
less research
research staff
staff time.
time. We have also continued
continued to
efficient
of some
some of
of our research
research projects including
including
distribute the video tapes
tapes of
distribute
of Knowledge Systems
Systems Laboratory
Laboratory work
work to
ONCOCIN, and an overview tape of
ONCOCIN,
Our group has
has continued
continued to publish actively on the results
outside groups. Our
of our
our research,
research, including
including more than 45 research
research papers
papers per year in
in the AI
AI
of
of
literature and a dozen books in
in the past 5 years
years on various aspects
aspects of
literature
SUMEX-AIM AI
AI research.
research.
SUMEX-AIM
Information Sciences
Sciences program, begun
begun at Stanford in 1983
.• The Medical Information
1983 under
Shortliffe as
as Director,
Director, has
has continued its strong development over
Professor Shortliffe
curriculum offered
offered by the MIS program
the past year. The specialized curriculum
focuses on the development of
of a new generation of
of researchers
researchers able to
focuses
support the development of
of improved
improved computer-based
computer-based solutions
solutions to
support
biomedical needs.
needs. The feasibility
resulted in large part
feasibility of
of this program resulted
biomedical
from
prior work
from the prior
work and research
research computing
computing environment
environment provided by the
SUMEX-AIM
SUMEX-AIM resource.
resource. It
It has
has recently received
received enthusiastic endorsement
from
years, has
has been
been
from the Stanford
Stanford Faculty
Faculty Senate
Senate for
for an additional
additional five
five years,
National Library
awarded
post-doctoral training
awarded renewed
renewed post-doctoral
training support from
from the National
Library
of
of Medicine
Medicine with
with high praise
praise for
for the training
training and
and contributions
contributions of
of the
SUMEX
- AIM environment
SUMEX-AIM
environment from
from the reviewing study section,
section, and has
has
received
received additional
additional industrial
industrial and foundation
foundation grants
grants for
for student support.
This
This past
past year,
year, MIS students
students have
have published many papers,
papers, including
including several
several
that have
have won conference awards.
awards.
•. While
While the SUMEX-AIM
SUMEX-AIM computing
computing resource
resource hardware
hardware has
has been
been largely
unchanged
we continue to evaluate
evaluate new workstation
workstation
unchanged this past
past year,
year, we
technologies
technologies of
of advantage
advantage to the AIM
AIM community.
community. We continue to operate
the DEC 2060
2060 mainframe
mainframe and
and file
file servers
servers for
for the
the community.
community. Because
Because of
of
the broad mix
EX-AIM community,
mix of
of research
research in the SUM
SUMEX-AIM
community, no single
computer vendor can
can meet
meet our needs
needs so
so we
we have
have undertaken
undertaken long-term
long-term
support of
of aa heterogeneous
heterogeneous computing
computing environment, incorporating
incorporating many
types
types of
of machines
machines linked
linked through multiprotocol
multiprotocol Ethernet facilities.
facilities.
•. We have
have continued to recruit
recruit new
new user
user projects
projects and
and collaborators to explore
further
these projects are
are
further biomedical areas
areas for
for applying AI. A number of these
built
built around the
the communications
communications network facilities
facilities we
we have
have assembled,
assembled,
bringing
bringing together
together medical and
and computer science
science collaborators from
from remote
institutions
institutions and
and making their research
research programs
programs available
available to still
still other
remote users.
users. At
At the
the same
same time we
we have
have encouraged
encouraged older mature projects
projects to
build
build their own computing environments thereby
thereby freeing
freeing up
up SUMEX
SUMEX
resources
resources for
for newer
newer projects.
projects.
17
17
E.
E. H.
H. Shortliffe
Shortliffe
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR0078514
SUMEX and Medical
Medical Computer
Computer Science
Science offices
offices
In June 1986, we moved the SUMEX
.• In
into the newly
newly constructed Stanford
Stanford Medical
Medical School Office
Office Building,
Building, funded
funded
into
university. This
This space
space provides us with
with almost
almost twice the area we
by the university.
previously occupied and itit is laid
laid out
out so as
as to
to promote
promote better interactions
interactions
previously
our groups and among our
our students and research
research staff.
staff.
between our
SUMEX user
user projects have made good progress
progress in
in developing and
.• SUMEX
for biomedical
biomedical
disseminating effective
effective consultative
consultative computer
computer programs for
disseminating
research. These systems
systems provide
provide expertise in
in areas
areas like
like cancer chemotherapy
research.
protocol management,
management, clinical
clinical diagnosis and decision-making,
decision-making, and molecular
molecular
protocol
biology. We have worked hard to
to meet their
their needs
needs and are grateful
grateful for
for
biology.
their expressed
expressed appreciation
appreciation (see
(see Section IV).
IV).
.• their
E.
E. H. Shortliffe
Shortliffe
18
18
5P41-RR00785-14
5P41-RR00785-14
Details
Details of
of Technical Progress
Progress
III.A.3.2.
III.A.3.2. Core ONCOCIN
ONCOCIN Research
Research
ONCOCIN
ONCOCIN is a data-management and therapy-advising
therapy-advising program for
for complex cancer
cancer
chemotherapy experiments. The development of
of the system
system began
began in 1979,
1979, following
following the
successful
successful generalization
generalization of
of MYCIN
MYCIN into
into the EMYCIN
EMYCIN expert system
system shell. The
ONCOCIN
ONCOCIN project has
has evolved over the last eight years.
years. The original
original version of
of
ONCOCIN
ONCOCIN ran on the time-shared
time-shared DEC
DEC computers, using a standard
standard terminal
terminal for
for the
time-oriented
current version uses
uses compact, single-user
time-oriented display of
of patient
patient data.
data. The current
workstations
workstations running
running on the SUMEX
SUMEX Ethernet
Ethernet network
network with
with large bit-mapped
bit-mapped displays
project has
has also expanded
expanded in scope.
scope. There are
for
for presentation of
of patient
patient data.
data. The project
three major
ONCOCIN, the therapy planning
planning program and its
major research
research components:
corn ponen ts: 1) ONCOCIN,
graphical interface;
interface; 2) OPAL,
OPAL, a graphical knowledge entry
entry system
system for
for ONCOCIN;
ONCOCIN; and 3)
ONYX,
ONYX, a strategic planning
planning program designed
designed to give advice in complex therapy
situations. Each of
of these
these research
research components has
has been
been split
split into
into two parts:
parts: continued
continued
situations.
development of
of the cancer therapy versions of
of the system,
system, and generalization of
of each
each of
of
development
This section will
will concentrate on
the components for
for use
use in other areas
areas of
of medicine. This
research topics derived from
from our applied work: 1) design
design of
of therapy
the three core research
planning
2)
planning systems
systems for
for use
use in clinical
clinical trial
trial experiments (E-ONCOCIN),
(E-ONCOCIN),
implementation
implementation of
of knowledge acquisition
acquisition systems
systems for
for clinical
clinical trials, and 3) development
continued
of
The work on continued
of general approaches
approaches to strategic therapy planning.
planning.
development of
of the ONCOCIN
ONCOCIN cancer chemotherapy advisor system
system itself
itself is described
described
separately
separately in Section IV.A.3.
IV.A.3.
1 - Overview
Overview of
of the ONCOCIN
ONCOCIN Therapy
Therapy Planning
Planning System
ONCOCIN
ONCOCIN is an advanced
advanced expert system
system for
for clinical
clinical oncology. It
It is designed
designed for
for use
use
after
after a diagnosis has
has been
been reached,
reached, focusing on assisting with
with the management
management of
of cancer
patients who are receiving chemotherapy. Because
Because anticancer agents
agents tend to be
be highly
highly
toxic, and because
because their
their tumor-killing
tumor-killing effects are routinely
routinely accompanied
accompanied by damage
damage to
toxic,
normal cells, the rules for
for monitoring
monitoring and adjusting treatment
treatment in response
response to a given
normal
patient’s course
course over time
time tend to be complex and difficult
difficult to memorize. ONCOCIN
ONCOCIN
patient's
integrates a temporal
temporal record of
of a patient's
patient’s ongoing treatment
treatment with
with an underlying
underlying
knowledge base
base of
of treatment
treatment protocols
protocols and rules for
for adjusting dosage,
dosage, delaying
treatment,
treatment, aborting
aborting cycles,
cycles, ordering
ordering special tests,
tests, and similar
similar management
management details. The
with decisions
decisions regarding the
program uses
uses such
such knowledge to help physicians with
management of
of specific
specific patients.
A major
major lesson
lesson of
of past work
work in
in clinical
clinical computing
computing has
has been
been the need
need to develop
A
methods for
for integrating
integrating a system
system smoothly
smoothly into
into the patient-care
patient-care environment
environment for
for which
it is intended. In the case
case of
of ONCOCIN,
ONCOCIN, the goal has
has been
been to provide expert
it
consultative
process, thereby
consultative advice as
as a by-product
by-product of
of the patient
patient data management
management process,
avoiding
avoiding the need
need for
for physicians to go out of
of their
their way to obtain
obtain advice.
advice. It
It is intended
that oncologists use
use ONCOCIN
ONCOCIN routinely
routinely for
for recording
recording and reviewing patient data
data on
computer’s screen,
screen, regardless
the computer's
regardless of
of whether they feel they need
need decision-making
decision-making
assistance.
assistance. This
This process
process replaces
replaces the conventional
conventional recording
recording of
of data on a paper
flowsheet and thus seeks
seeks to avoid
avoid being perceived as
as an additive
additive task.
task. In accordance
accordance
with
with its'
its’ knowledge
knowledge of
of the patient's
patient’s chemotherapy protocol,
protocol, ONCOCIN
ONCOCIN then provides
assistance
assistance by suggesting
suggesting appropriate
appropriate therapy at the time
time that
that the day's
day’s treatment
treatment is to be
maintain control
control of
of the decision, however,
however, and
recorded on the flowsheet. Physicians maintain
can override
override the computer's
computer’s recommendation
recommendation if
if they wish. ONCOCIN
ONCOCIN also
also indicates the
appropriate
appropriate interval
interval until
until the patient's
patient’s next treatment
treatment and reminds the physician of
of
radiologic and laboratory
laboratory studies required by the treatment
treatment protocol.
protocol. This
This core research
research
radiologic
report begins with
with our efforts
efforts to extend the techniques
techniques of
of ONCOCIN
ONCOCIN for
for use
use in other
report
areas
areas of
of medicine
medicine (E-ONCOCIN).
(E-ONCOCIN).
19
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
Details
Details of
of Technical
Technical Progress
Progress
2 - E-ONCOCIN:
E-ONCOCIN: Domain
Domain Independent
Independent Therapy
Therapy Planning
Planning
During
During this. past year,
year, our E-ONCOCIN
E-ONCOCIN research
research has
has concentrated on understanding
felt that the area
area of
of insulin
insulin
how protocols
protocols in medicine
medicine vary across
across subspecialties.
subspecialties. We felt
Like cancer chemotherapy,
treatment
treatment for
for diabetes
diabetes would be a good area
area to explore. Like
treatments for
for diabetes
diabetes continue
continue over long periods of
of time
time and have
have been
been the area
area of
of
intensive
Unlike cancer chemotherapy, the treatment
treatment plan must
intensive protocol
protocol development. Unlike
handle multiple
multiple doses
doses over the course
course of
of one day and deemphasizes
deemphasizes the use
use of
of drug
combinations
combinations (although
(although there are a variety
variety of
of types
types of
of insulin).
insulin). Other challenges
challenges of
of the
diabetes
diabetes area
area include
include consideration
consideration of
of multiple
multiple goals,
goals, such
such as
as finding
finding the "normal
“normal dose"
dose”
treatment plans must be
of
of. insulin
insulin versus
versus adjusting for
for short
short term trends. Diabetes treatment
flexible
flexible enough to take into
into account diet
diet and exercise
exercise patterns and their
their effects on
insulin
insulin requirements.
We performed
performed knowledge acquisition
acquisition sessions
sessions about insulin
insulin treatment
treatment of
of diabetes,
diabetes, using
the medical literature
literature and several
several internists
internists in the Medical
Medical Computer
Computer Science
Science research
research
group (Mark
(Mark Frisse, Mark
Mark Musen, and Michael
Michael Kahn).
Kahn). The proposed
proposed structure for
for the
knowledge base
base was
was implemented
implemented using the object-oriented
object-oriented programming
programming language
language upon
These experiments, like
like those
those of
of adding more
which ONCOCIN
ONCOCIN has
has been
been based.
based. These
protocols
protocols to ONCOCIN,
ONCOCIN, demonstrated the need
need for
for changes
changes in the way that the
knowledge base
base can access
access the time-oriented
time-oriented data base
base that stores
stores patient
patient data and
previous conclusions.
The relationships
relationships between
between the different
different doses
doses and types
types of
of
insulin
insulin treatments will
will also require alternative
alternative ways
ways of
of building
building treatment
treatment hierarchies.
Thus, our initial
initial experiments have
have shown that many of
of the elements of
of the ONCOCIN
ONCOCIN
design
general for
for other application
application areas,
areas, but that some
some specific
specific
design are sufficiently
sufficiently general
elements (particularly
(particularly the representation of
of temporal
temporal events) will
will have
have to be generalized.
generalized.
During
During the coming
coming year,
year, we will
will continue
continue our knowledge acquisition
acquisition experiments and
design
design a version of
of the E-ONCOCIN
E-ONCOCIN system
system that is separate
separate from
from the ongoing "clinic
“clinic
version.”
version."
OPAL: Graphical
Graphical Knowledge Acquisition
Acquisition Interface
Interface
3 - OPAL:
OPAL
OPAL is a graphical environment
environment for
for use
use by an oncologist
oncologist who wishes
wishes to enter a new
chemotherapy protocol
protocol for
for use
use by ONCOCIN
ONCOCIN or to edit
edit an existing
existing protocol. Although
Although
the system
system is designed
designed for
for use
use by oncologists who have
have been
been trained in its use,
use, it
it does
does
not
not require an understanding of
of the internal
internal representations or reasoning
reasoning strategies
strategies used
used
by ONCOCIN.
ONCOCIN. The system
system may be used
used in two interactive
interactive modes,
modes, depending on the
first permits
permits the entry
entry of
of a graphical description
description
type of
of knowledge to be entered.
entered. The first
oncologist manipulates boxes
boxes on the
of
of the overall
overall flow
flow of
of the therapy process.
process. The oncologist
screen
screen that
that stand for
for various steps
steps in the protocol.
protocol. The resulting diagram is then
translated by OPAL
OPAL into
into computer
computer code
code for
for use
use by ONCOCIN.
ONCOCIN. Thus, by drawing a
flow
flow chart
chart that
that describes
describes the protocol
protocol schematically, the physician is effectively
effectively
programming
programming the computer
computer to
to carry out
out the procedure appropriately
appropriately when
when ONCOCIN
ONCOCIN is
later used
used to guide the management of
of a patient
patient enrolled
enrolled in that protocol.
protocol.
OPAL's
OPAL’s second
second interactive
interactive mode permits
permits the oncologist
oncologist to describe
describe the details of
of the
For example, the rules for
for
individual
individual events
events specified
specified in the graphical
graphical description.
description. For
administering
patient'ss
administering a given chemotherapy will
will vary greatly depending upon the patient’
response to earlier
earlier doses,
doses, intercurrent
intercurrent illnesses
illnesses and toxicities,
toxicities, hematologic status,
status, etc.
etc.
response
Figure
Figure 1 shows
shows one of
of the forms
forms provided
provided by OPAL
OPAL for
for this type of
of specification.
specification. It
It
permits
permits the entry
entry of
of an attenuation
attenuation schedule
schedule for
for an agent based
based upon the patient's
patient’s white
such as
as this are generally
count
count and platelet
platelet count
count at the time
time of
of treatment. Tables such
found
Thus, OPAL
found in the written
written version of
of chemotherapy protocols.
OPAL permits
information using familiar
familiar forms
forms displayed on the computer's
computer’s
oncologists to enter information
of such
such forms
forms are subsequently
subsequently translated into
into rules and other
screen.
screen. The contents of
for use
use by ONCOCIN.
ONCOCIN.
knowledge structures for
Shortliffe
E. H. Shortliffe
20
5P41-RR00785-14
Orug~llcn:
en.g:
Details of
of Technical Progress
Progress
________~P~O~cc~_______
PROURtIAZINE
PROo.RBAZINE
~c~:
A
[Cta-iqa Table
Tab& Ftlt'milt11
Fccmat?j
1000nge
ItMete Table
Tati 71
IDeeta
Pfal ..lets
wet
wee
(lC
lx 1000)
low
>- 3.5
3.5
)3.a - 3.5
3.5
3.e
2.5 -- 3.B
3.8
2.5
< 2.5
2.5
<
(‘( low
(lC 1000)
>- rza
18a
189 ,;,; 158
158
. STIlI
1efrz of
of STD
ST0 m 7'j'/~ af
18f1'X.
75% of
of ST0
75'f.
Oelav
t
Delav
~
sro
Oslay
Dli
lay
Delay
Delay
Delay
Delay
Oelay
Delay
< 75
<
D
.. l .;.v
D6l;iV
DC 1 A‘/
Oelav
Delav
OS 1 ay
Deolly
Delay
Delay
Delay
DeIly
Dslay
D.alay
De
lay
Delay
ISp~if'l Delay In;ol
ISoaufy
Abort
hfai
ISoec.ify
Abort
Infot
Figure 1:
75 - l8e
10a
A Sample OPAL
OPAL Form
Form
A
Status
Status of
of the OPAL
OPAL System
System
OPAL is one of
of the few graphical knowledge acquisition
acquisition systems
systems ever designed
designed for
for
OPAL
expert systems.
systems. Even fewer are designed
designed to be used
used as
as the main
main method for
for entering
entering
as opposed
opposed to a proof
proof of
of concept implementation.
implementation. We have pursued
pursued three
knowledge as
directions in the development of
of the OPAL
OPAL system,
system, also in response
response to the large
directions
of protocols
protocols entered through this system
system during
during the last year.
year. The first
first
number of
direction is the modification
modification of
of graphical forms
forms needed
needed to allow
allow the entry
entry of
of facts that
direction
did not
not show up in the protocols
protocols used
used to test the initial
initial version of
of OPAL.
OPAL. OPAL
OPAL
did
assume that
that most of
of the knowledge to be entered will
will have very stereotyped
continues to assume
forms, e.g.,
e.g., dose
dose attenuations
attenuations for
for most treatment
treatment toxicities
toxicities are based
based on a comparison of
of
forms,
only one laboratory
laboratory measurement
me'asurement at a time, such as
as using the BUN
BUN to adjust for
for renal
only
toxicity. We sometimes need much more complex
complex ways
ways of
of stating
stating the scenarios
scenarios in
in which
which
toxicity.
This need has
has led us in
in a second
second direction,
direction, towards
dose adjustments may be necessary.
necessary. This
dose
"lower-level" rule entry
entry approaching the syntax of
of the reasoning component
component of
of
a “lower-level”
ONCOCIN, but
but using graphical
graphical input
input devices
devices where applicable.
applicable. A
A prototype
prototype version of
of
ONCOCIN,
entry system
system has
has been
been completed, and will
will soon be evaluated as
as an adjunct
adjunct to
this rule entry
the basic OPAL
OPAL system.
system.
information provided
provided on the graphical
graphical forms
forms into
into a
OPAL program maps the information
The OPAL
of the
structure (called the IDS)
IDS) that is used
used to represent the contents of
complex data structure
21
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
protocol. The
The data structure
structure is used
used for
for copying
copying information
information from
from one protocol
protocol to
protocol.
for the creation
creation of
of the ONCOCIN
ONCOCIN knowledge base.
base. Our
Our
another, and as the basis for
our intention
intention to
to generalize OPAL
OPAL for
for use
use outside of
of
experiments with
with OPAL,
OPAL, and our
suggested that
that we reorganize the OPAL
OPAL program
program to use
use a relational
relational
oncology protocols, suggested
after an existing
existing
database to store its knowledge. We have patterned the database
database after
database
relational database
database management systems
systems exist for
for the
database query syntax. Because
Because no relational
database
OPAL is based,
based, we reimplemented
reimplemented the database
database from
from its
Interlisp language
language upon which
which OPAL
Interlisp
written description.
description. The database
database structure
structure is now almost
almost complete, and we have begun
written
design a revised IDS for
for chemotherapy protocols,
protocols, and will
will be determining
determining how an
to design
for other
other areas
areas of
of medicine
medicine (e.g.,
(e.g., the insulin
insulin example being used
used
IDS would be created for
in the E-ONCOCIN
E-ONCOCIN experiments).
experiments).
in
Our ability
ability to use
use the OPAL
OPAL system
system for
for specifying
specifying oncology
oncology treatments
treatments has
has led us
us to
of a new program, named PROTEGE,
PROTEGE, that
that will
will turn
turn an interactive
interactive session
session
design of
the design
with an expert and knowledge engineer into
into the specification
specification of
of an OPAL-like
OPAL-like system
system
with
in a wide range of
of medical areas.
areas. We have implemented
implemented several
several
for clinical
clinical trials
trials in
for
prototype forms
forms for
for PROTEGE.
PROTEGE. These
These forms
forms are used
used to specify
specify a general description
description
of the application
application area.
area. Of
Of particular
particular importance
importance is the need
need to specify
specify how the
of
initial dosage
dosage of
of a drug be
planning process
process will
will take place,
place, e.g.,
e.g., how will
will the initial
therapy planning
with various adjustments of
of the dosage
dosage due to toxicities
toxicities to the treatment
treatment to
combined with
form the final
final recommended dose.
dose. Most
Most of
of this
this type of
of “procedural”
"procedural" knowledge is not
not
form
entered in the OPAL
OPAL system,
system, and must be hand-coded by the knowledge engineer.
engineer. A
A
entered
PROTEGE is in
in progress
progress by Mark
Mark Musen, M.D., and will
will be completed
thesis on PROTEGE
Ph.D. thesis
during the next year.
year.
during
ONYX: Strategic
Strategic Therapy
Therapy Planning
Planning
4 - ONYX:
Although
rich and complex, protocols seldom
Although the knowledge of
of cancer chemotherapy is rich
seldom
refer directly
underlying models of
protocol are,
directly to underlying
of drug action. The guidelines in a protocol
are,
based on the study designers'
rather, high-level
high-level composite descriptions
descriptions of
of expert advice, based
designers’
mechanisms
experience
biological models of
experience as
as well as
as biological
of the therapeutic agents
agents and their
their mechanisms
of
of action. We have observed,
observed, however, that when protocols fail
fail to cover a complex
patient, expert oncologists will
c1inicaJ
clinical situation
situation that arises
arises for
for a given patient,
will turn
turn to
process.
underlying
underlying mechanistic models and use
use them to assist
assist in
in the decision-making
decision-making process.
ONCOCIN
ONCOCIN has
has no such
such knowledge;
knowledge; it
it must therefore
therefore occasionally decline to make a
recommendation and instead refer
refer a physician to the study chairman
chairman for
for a decision
particular complex problem.
problem. It
about how to manage
manage a particular
It is accordingly
accordingly a long-range
long-range goal
to add
performance.
add model-based expert-level
expert-level reasoning to ONCOCIN's
ONCOCIN’s performance.
Our research
research in model-based reasoning is embodied in a program known as
as ONYX.
ONYX.
This system
system is based
based on the observation that creative planning
planning strategies
strategies in the oncology
domain (and many other fields)
fields) appear
appear to involve
involve a three-step process:
process: (1) heuristic
heuristic
responses
to the problem at hand,
generation of
of a small number
number of
of plans,
plans, i.e.,
i.e., plausible
responses
hand,
(2) mental simulation
simulation (also called "envisionment
“envisionment”)n ) of
of how the patient
patient would respond
over time if
if each
each of
of those
those plans were
were carried out, and (3) selection of
of a preferred plan
based
based upon the likelihood
likelihood of
of the various possible outcomes
outcomes and the value placed
placed on
those
those outcomes
outcomes by the patient
patient and physician. Step
Step 2 in this process
process involves
involves patientpatientspecific simulation
simulation of
of tumor
tumor pathophysiology
pathophysiology and drug action, but
but it
it also depends
depends on
recognition
recognition that the outcomes
outcomes .of interventions
interventions cannot be predicted with
with certainty
certainty and
and
Thus, model-based probabilistic
that probabilistic
probabilistic predictions
predictions are
are more realistic.
realistic.
probabilistic
simulations
simulations in ONYX
ONYX are
are coupled to a decision analytic
analytic module which assists
assists with the
third
third step
step in the process.
process. The work outlined
outlined here
here is preliminary.
preliminary.
Each
Each of
of the components in ONYX
ONYX may be
be generalized
generalized for
for use
use in other systems.
systems. We
have
have concentrated our work on the decision analysis
analysis component. We are
are building
building tools
that will
will allow experts
experts to frame the comparison between
between several
several possible treatments that
could be
be administered at one point
point in a patient's
patient’s course.
course. Often
Often these
these treatments V·Ill!
wi11 be
be
E.
E. H. Shortliffe
Shortliffe
22
22
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
variations on the standard treatment,
treatment, but
but with
with reduced dosages
dosages or
or delayed
delayed time
time of
of
variations
treatment.
An important
important part
part of
of the
the treatment
treatment decision
decision concerns
concerns the
the patient’
patient's
treatment.
An
s
evaluation of
of the
the possible
possible outcomes and their
their likelihood,
likelihood, as
as represented
represented in
in the
the utility
utility of
of
evaluation
various plans. The
The program we have built
built carries out
out aa dialog
dialog with
with patient
patient to
to assess
assess
the various
prints out
out the “best”
"best" choice. AA graphical
graphical
utilities, builds
builds a decision tree, and prints
the utilities,
representation of
of the decision problem
problem is build
build on the computer
computer display
display as the
the dialog
dialog
takes place.
place.
major problem
problem with
with decision analysis programs have been the way that
that the choice is
AA major
to the user.
user. Often,
Often, the answer is in
in the form
form of
of one utility
utility number
number for
for each
explained to
Most computer systems
systems for
for decision trees
trees allow
allow the user to
to see
see how
how much
much the
choice. Most
utilities will
will change
change as
as the probabilities
probabilities of
of the expected
expected events are modified.
modified. What
What is
utilities
not available, is an explanation,
explanation, in
in English,
English, of
of why one choice is better than another.
not
part of
of his Ph.D. research,
research, Curtis
Curtis Langlotz
Langlotz has
has built
built a system
system that
that can create a
As part
rationale for
for the selection. The program compares
compares various parts of
of the decision tree,
rationale
for differences in
in the problem
problem structure that account for
for the variation
variation in
in the
looking for
looking
final utilities
utilities for
for the problem. This
This explanation
explanation program has
has been tested
tested with
with several
several
final
from different
different areas
areas of
of medicine: treatment
treatment of
of heart disease,
disease,
decision problems from
antibiotic selection, and cancer treatment.
antibiotic
5 - Implementation
Implementation of
of the ONCOCTN
ONCOCIN Workstation
Workstation in the Stanford
Stanford Clinic
Clinic
mid-1986, we
we placed
placed the workstation
workstation version of
of ONCOCIN
ONCOCIN into
into the Oncology Day
In mid-1986,
This version is a completely
completely different
different program from
from the version of
of
Care clinic.
clinic. This
ONCOCIN
that
was
available
in
the
clinic
from
1981-1985
-using
protocols
entered
ONCOCIN that was
clinic from
-protocols
OPAL program, with
with a new graphical data entry
entry interface,
interface, and revised
through the OPAL
person in
in the clinic
clinic (Andy
(Andy
knowledge
knowledge representation
representation and reasoning component. One person
Zelenetz) became
that our design
design goals
goals for
for this
became primarily
primarily responsible for
for making sure
sure that
addition of
of key protocols
protocols
version of
of ONCOCIN
ONCOCIN were
were met. His suggestions
suggestions included the addition
and the ability
as a data management tool
ability to have
have the program be useful for
for clinicians
clinicians as
if
into the system.
system. Both of
of
if the complete treatment protocol
protocol had not
not yet been
been entered
entered into
these
has achieved
achieved wider
these suggestions
suggestions were
were carried out during
during this year,
year, and the program has
use
use in the clinic
clinic setting.
setting. In addition,
addition, laser-printed
laser-printed flowsheets
flowsheets and progress
progress notes
notes have
have
been
been added
added to the clinic
clinic system.
system.
The process
process of
of entering a large number of
of treatment protocols in a short
short period of
of
time led to other research
research topics including:
including: design
design of
of an automated system
system for
for
producing meaningful
meaningful test
test cases
casesfor
for each
each knowledge
knowledge base,
base, modification
modification of
of the design
design of
of
the time-oriented
time-oriented database
database and the methods
methods for
for accessing
accessing the database,
database, and the
development of methods
methods for
for graphically
graphically viewing mUltiple
multiple protocols that are
are combined
into
into one large
large knowledge
knowledge base.
base. These
These research
research efforts
efforts will
will continue
continue into
into the next year.
year.
In addition,
addition, some
some of the
the treatment regimens
regimens developed
developed for
for the original
original mainframe
mainframe
version are
are still
still in use
use and
and can
can be
be transferred to the
the new version of
of ONCOCIN.
ONCOCIN. The
process
process of converting this knowledge
knowledge will
will also
also be
be undertaken
undertaken in the next year.
year. As the
knowledge
knowledge base
base grows,
grows, additional
additional mechanisms
mechanisms will
will be
be needed
needed for
for the incremental
incremental update
update
and
and retraction of protocols.
protocols.
We also
of the
also developed
developed new
new insights about the
the design
design of the internal
internal structures .of
knowledge
knowledge base
base (e.g.,
(e.g., the
the relationship
relationship between
between the
the way
way we
we refer to chemotherapies,
chemotherapies,
drugs,
drugs, and
and treatment
treatment visits).
visits). We
We will
will continue
continue to
to optimize
optimize the
the question-asking
question-asking
procedure,
procedure, improve
improve the
the method
method for
for traversing
traversing the
the plan structure in the knowledge
knowledge base,
base,
and
and consider
consider alternative
alternative arrangements
arrangements used
used to
to represent
represent the
the structure of chemotherapy
plans.
plans. Although
Although we
we have
have concentrated
concentrated our
our review
review of
of the
the ONCOCIN
ONCOCIN design
design primarily
primarily
on
on the
the data
data provided
provided by
by additional
additional protocols,
protocols, we
we know that non-cancer therapy
problems
The E-ONCOCIN
E-ONCOCIN effort
effort isis designed
designed to produce
produce
problems may
may also
also raise
raise similar
similar issues.
issues. The
aa domain-independent
domain-independent therapy
therapy planning
planning system
system that
that includes
includes the
the lessons
lessons learned
learned from
from
our
our oncology
oncology research.
research.
23
23
E.
E. H.
H. Shortliffe
Shortliffe
of Technical Progress
Progress
Details of
5P41-RR00785-14
5P41-RR00785-14
6 - Personnel
The development of
of the generalized
generalized version of
of each
each of
of the ONCOCIN
ONCOCIN components has
has
been
been undertaken by a large group of
of computer scientists and physicians. Samson
Samson Tu
Tu
has
has had primary
primary responsibility
responsibility for
for the extensions
extensions to the design
design of
of the knowledge base,
base,
Clifford
primary responsibility
Clifford Wulfman
Wulfman has
has had primary
responsibility for
for extensions to the data entry
entry
interface.
interface. David
David Combs has
has had
had primary
primary responsibility
responsibility for
for the knowledge acquisition
acquisition
interface.
interface. Janice
Janice Rohn has
has been
been involved
involved with
with protocol
protocol and data management,
management, and has
has
primary
primary responsibility
responsibility for
for the implementation
implementation of
of the program that
that sets
sets up the
ONCOCIN
Christopher
Christopher Lane has
has developed
developed the object-oriented
object-oriented
ONCOCIN user
user environment.
environment.
systems
systems software upon which the entire
entire ONCOCIN
ONCOCIN system
system is designed.
designed.
E.
E. H. Shortliffe
Shortliffe
24
of Technical Progress
Progress
Details of
5P41-RR00785-14
5P41-RR00785-14
III.A.3.3. Core AI
AI Research
Research
III.A.3.3.
Rationale
1 - Rationale
Artificial Intelligence
Intelligence (AI)
(AI) methods
methods are particularly
particularly appropriate
appropriate for
for aiding
aiding in the
Artificial
management and application
application of
of knowledge because
because they apply to information
information
represented
judgmental rules
represented symbolically,
symbolically, as
as well as
as numerically,
numerically, and to reasoning
reasoning with
with judgmental
have been
been focused
focused on medical and biological
biological problems for
for
as
as well as
as logical ones.
ones. They have
This is because,
because,of
of all the computing
computing methods
over a decade
decade with
with considerable success.
success. This
known, AI
AI methods are the only
only ones
ones that deal
deal explicitly
explicitly with
with symbolic
symbolic information
information
and problem
problem solving
solving and with
with knowledge that is heuristic (experiential)
(experiential) as
as well as
as
factual.
Expert
Expert systems
systems are one important
important class
class of
of applications of
of AI
AI to complex problems
system is one whose
whose
--- in medicine, science,
science, engineering, and elsewhere.
elsewhere. An expert system
performance
performance level rivals
rivals that
that of
of an human expert because
because it
it has
has extensive domain
knowledge
knowledge (usually derived from
from an human expert); it
it can
can reason
reason about its knowledge to
solve difficult
difficult problems in the domain; it
it can explain its line
line of
of reasoning
reasoning much as
as an
human expert can; and it
it is flexible
flexible enough
enough to incorporate new
new knowledge without
without
reprogramming.
reprogramming. Expert
Expert Systems
Systems draw on the current
current stock of
of ideas
ideas in AI,
AI, for
for example,
about representing and using knowledge.
problemknowledge. They are adequate
adequate for
for capturing
capturing problemsolving
solving expertise for
for many bounded problem areas.
areas. Numerous high-performance,
high-performance, expert
systems
systems have
have resulted from
from this work
work in such
such diverse
diverse fields as
as analytical
analytical chemistry,
medical diagnosis,
diagnosis, cancer
cancer chemotherapy management,
management, VLSI
VLSI design,
design, machine fault
fault
diagnosis,
diagnosis, and molecular biology. Some
Some of
of these
these programs
programs rival
rival human experts in
solving
solving problems in particular
particular domains and some
some are being adapted
adapted for
for commercial
commercial use.
use.
Other projects have developed
developed generalized software tools for
for representing and utilizing
utilizing
knowledge
EMYCIN, UNITS,
UNITS, AGE,
AGE, MRS, BBI,
BBl, and GLISP)
GLISP) as
as well as
as
knowledge (e.g.,
(e.g., EMYCIN,
publications such
such as
as the three-volume
three-volume Handbook
of Artificial
comprehensive publications
Handbook of
Artificial
Intelligence and books summarizing
summarizing lessons
lessons learned in the DENDRAL
DENDRAL and MYCIN
MYCIN
Intelligence
research
research projects.
There is considerable power in the current
current stock of
of techniques,
techniques, as
as exemplified
exemplified by the
rate of
of transfer
transfer of
of ideas
ideas from
from the research
research laboratory
laboratory to commercial practice. But we
also
also believe that
that today's
today’s technology needs
needs to be augmented
augmented to deal
deal with
with the complexity
complexity
of
of medical information
information processing.
processing.
Our core research
research goals,
goals, as
as outlined
outlined in the next section, are
are to analyze
analyze the limitations
limitations of
of
current
current techniques
techniques and to investigate the nature of
of methods
methods for
for overcoming
overcoming them.
Long-term
Long-term success
success of
of computer-based aids
aids in medicine and biology
biology depend
depend on
improving
improving the programming
programming methods available for
for representing and using domain
domain
knowledge. That
That knowledge is inherently
inherently complex: it
it contains mixtures
mixtures of
of symbolic
symbolic and
knowledge.
numeric
numeric facts and relations, many of
of them uncertain; it
it contains knowledge
knowledge at different
different
levels of
of abstraction and in seemingly inconsistent
inconsistent frameworks; and it
it links
links examples
examples
and exception clauses
clauses with
with rules of
of thumb
thumb as
as well as
as with
with theoretical principles.
principles.
Current
Current techniques have
have been
been successful
successful only
only insofar
insofar as
as they severely
severely limit
limit this
complexity.
complexity. . As the applications
applications become
become more far-reaching,
far-reaching, computer programs will
will
have
have to deal
deal more effectively
effectively with
with richer
richer expressions
expressions and much more voluminous
voluminous
of knowledge.
knowledge.
amounts of
This report
report documents progress
progress on the basic or core research
research activities
activities within
within the
This
Knowledge Systems
Systems Laboratory
Laboratory (KSL),
(KSL), funded in part under the SUMEX
SUMEX resource
resource as
as
well as
as by other federal and industrial
industrial sources.
sources. This work
work explores a broad range
range of
of
basic research
research ideas
ideas in many application
application settings,
settings, all of
of which contribute
contribute in the long
term to improved
improved knowledge
knowledge based
based systems
systems in biomedicine.
25
E. H. Shortliffe
Shortliffe
Details of
of Technical
Technical Progress
Progress
Details
5P4l-RR00785-l4
5P41-RR00785-14
Highlights of
of Progress
Progress
2 - Highlights
In
progressed on several
AI. As in
In the last year, research
research has
has progressed
several fundamental
fundamental issues
issues of
of AI.
in the
past, our
our research
research methodology
methodology is experimental;
experimental; we believe it
it is most fruitful
fruitful at this
stage of
of AI
AI research
research to raise questions, examine issues,
issues, and test hypotheses
in the
hypotheses in
stage
context of
of specific
specific problems, such as
as management of
of patients with
Hodgkin’s disease.
disease.
with Hodgkin's
context
within the KSL
KSL we build
systems that implement
implement our
our ideas
ideas for
for answering (or
(or
build systems
Thus, within
light on)
on) fundamental
fundamental questions;
questions: we experiment
experiment with
with those systems
systems to
shedding some light
determine the strengths and limits
limits of
of the ideas;
ideas; we redesign
redesign and test more;
more: we attempt
attempt
determine
ideas from
from the domain
domain of
of implementation
implementation to other
other domains; and we
to generalize the ideas
publish
of the experiments. Many
publish details of
problem domains are
Many of
of these
these specific
specific problem
made substantial
medical or biological.
biological.
In this
this way we believe the KSL
KSL has
has made
substantial
contributions
not just
just to the AIM
contributions to core research
research problems of
of interest
interest not
AIM community
community but
to AI
AI in general.
Progress is reported
reported below under each
each of
of the major
major topics of
of our work. Citations
Citations are to
Progress
KSL
publications section.
KSL technical
technical reports listed in the publications
2.1 - Knowledge
Knowledge Representation
Representation
How can the knowledge necessary
necessary for
for complex
complex problem
solving be represented
for its
How
problem solving
represented for
effective use
automatic inference
inference processes?
most effective
use in automatic
processes? Often,
Often, the knowledge obtained
obtained
from experts is heuristic
heuristic knowledge, gained from
from many years
years of
of experience.
experience. How
How can
from
with its inherent
inherent vagueness
vagueness and uncertainty,
uncertainty, be represented
represented and applied?
this knowledge, with
Work continues
continues on BBl,
BBl, with
with its explicit
explicit representation of
of control
control knowledge, as
as
Work
reported last year (see
(see the summary of
of Blackboard Architectures
Architectures below). In addition,
addition,
part
of our
our research
research on NEOMYGIN
flexible, rich
rich representation
part of
NEOMYGIN is focused on using a flexible,
of control
control knowledge so that we can model problem
solving at the strategic level as
as well
of
problem solving
as
as at the tactical
tactical level.
[See KSL
KSL technical
technical reports KSL-87-0l
KSL-87-01 and KSL-87-32]
KSL-87-321
Control
2.2 - Blackboard
Blackboard Architectures
Architectures and Control
How
powerful problem
problem solving
How can we design flexible
flexible control
control structures for
for powerful
solving programs?
We have continued
blackboard architecture
reason
continued to develop the BBI
BBl blackboard
architecture for
for systems
systems that reason
-- control,
-- their
their own actions. In the area
area of
of control,
control,
about -control, explain,
explain, and learn about -we have
have developed two new domain-independent
domain-independent control
control capabilities.
capabilities. One generic
control
parameters of
control knowledge source refines
refines specified
specified parameters
of abstract control
control plans by
control knowledge source
generating legal values
network. The other control
values from
from a semantic network.
performs opportunistic
performs
opportunistic goal-directed
goal-directed reasoning whenever actions recommended by other
other
control
control decisions are not
not executable.
executable. In the area
area of
of explanation,
explanation, we have developed the
ExAct
ExAct program. It
It provides a flexible,
flexible, menu-driven
menu-driven set of
of explanation
explanation alternatives,
alternatives. as
as
well as
as a graphical
graphical display
display of
of the comparative
comparative advantages
advantages of
of alternative
alternative actions. In the
WATCH program
area of
of learning,
learning, we have developed two new capabilities.
capabilities. The WATCH
observes
observes domain
domain experts solving
solving problems and attempts to abstract from
from their
their actions
It automatically
automatically programs new control
control knowledge
the underlying
underlying control
control strategy. It
The
sources
sources to generate
generate tJ1e
the hypothesized strategy on subsequent
subsequent problems.
TRANALOGY
problems in
TRANALOGY
program notices when problems
in a new domain
domain are analogous
analogous to
problems in a known
known domain.
domain. It
It hypothesizes that analogous
analogous reasoning methods will
will
work
work in the new domain
domain as
as well and automatically
automatically programs appropriate
appropriate knowledge
sources.
sources.
We have
have begun conducting
conducting various experiments
experiments on the costs
costs and .benefits
.benefits of
of control
control
protein structure
reasoning. In the context
context of
of the PROTEAN
PROTEAN system
system for
for protein
structure modeling,
modeling, we
E. H. Shortliffe
Shortliffe
26
Details of
of Technical
Technical Progress
Progress
Details
SP41- RR0078S-14
5P41-RR00785-14
of different
different kinds
kinds of
of control
control knowledge and strategies
strategies to
to
are investigating
investigating the power of
computational efficiency.
efficiency. Early
Early results suggest
suggest that
that a small computational
computational
produce computational
investment in
in control
control reasoning can produce substantial
substantial computational
computational savings in
in
investment
problem-solving operations.
operations.
exploring differences
differences among alternative
alternative
We also are exploring
problem-solving
architectural realizations
realizations of
of a particular
particular control
control strategy.
architectural
continued to develop the ACCORD
ACCORD framework
framework for
for the class
class of
of arrangement
We have continued
exemplified by PROTEAN:
PROTEAN: arrange a set of
of objects to satisfy
satisfy constraints.
problems exemplified
ACCORD substantially
substantially enhances
enhances BBl’
BBl's
capabilities for
for control,
control, explanation,
explanation, and
ACCORD
s general capabilities
learning.
In addition
addition to PROTEAN,
PROTEAN, we have applied
applied BBl-ACCORD
BB1-ACCORD in
in the
learning.
In
SIGHTPLAN system
system for
for designing construction
construction site layouts.
SIGHTPLAN
In order
order to accommodate ACCORD
ACCORD and other
other task-specific
task-specific frameworks,
frameworks, we have
have
In
developed a set of
of generic framework
framework interpretation
interpretation procedures
procedures for:
for: parsing framework
framework
developed
sentences, matching
matching and rating
rating sentences,
sentences, generating legal parameter values
values for
for sentences,
sentences,
sentences,
translating sentences
sentences into
into the lower-level
lower-level language
language of
of BBl.
BBl. These
These procedures
procedures apply
and translating
user-specified framework
framework that satisfies the standards
standards of
of knowledge and
to any user-specified
laid down in
in ACCORD.
ACCORD. We refer
refer to this growing collection
collection of
of systems
systems
representation laid
and knowledge modules as
as the BB* environment.
environment.
[See
KSL technical reports KSL-86-38,
KSL-86-38, KSL-87-8,
KSL-87-8, and KSL-87-10
KSL-87-10 and “other
"other outside
[See KSL
publications" in
in Section III.A.3.51
III.A.3.S]
publications”
Advanced Architectures
Architectures
2.3 - Advanced
goals and technical approach of
of this project, largely supported by DARPA
DARPA under
The goals
the Strategic Computing
Computing Program, have been
been discussed
discussed in previous annual reports. To
summarize briefly,
briefly, we seek
seek to achieve
achieve two to three orders of
of magnitude speedup
speedup in the
execution of
identifying and exploiting
exploiting sources
sources of
of
of knowledge-based systems,
systems, by identifying
concurrency at all levels of
of system
system design:
design: the application
application level, the problem solving
framework
programming language
framework level, the programming
language level and the hardware systems
systems architecture
level.
Due to the inherent
inherent complexity
complexity of
of the task and the lack of
of theoretical
parallel computation
have taken an
foundations
foundations for
for parallel
computation with
with ill-structured
ill-structured problems, we have
empirical
phase of
empirical approach. During
During the first
first phase
of the project, which will
will be concluded in
July, 1987,
1987, we have
have made
made specific
specific choices
choices at each
each of
of the system
system levels,
levels, i.e.
i.e. taken a
"vertical
“vertical slice"
slice” through
through the design
design space,
space, and have
have conducted several
several experiments to
investigate the effects of
parameters on performance.
of a wide variety
variety of
of parameters
Some
Some highlights
highlights of
of our accomplishments thus far
far (most of
of which occurred during
during the
past year) include:
•. Based
Based on a careful and systematic
systematic study of
of potential
potential hardware system
system
architectures, we
we have
have established
established an architectural
architectural framework
framework for
for the
underlying
underlying machine as
as a multicomputer
multicomputer array. The study ranged
ranged over the
full
full spectrum of
of possibilities,
possibilities, from
from shared
shared memory multiprocessors to shared
shared
memory multicomputer
multicomputer networks to distributed
distributed memory multicomputer
multicomputer
networks, taking
taking into
into account the VLSI
VLSI opportunities
opportunities of
of the 1990's.
1990’s.
•. We have
have designed
designed and constructed a complex, fully
fully instrumented simulator
simulator
to realize the above
The simulated class
above architectural
architectural framework.
framework.
class of
of
machines,
machines, called CARE,
CARE, permits full
full manipulation
manipulation of
of the parameters
parameters which
specify the hardware system,
system, e.g.
e.g. communication
communication topology, memory size,
size, etc.
etc.
CARE
CARE is written
written in Zetalisp, and runs on standard
standard Lisp workstations (TI
(TI
Explorer,
Explorer, Symbolics 36xx).
36~~).
•. We have
have studied and implemented basic
basic additions
additions to the Lisp language
language to
accomplish distributed
distributed Lisp processing
processing on CARE
CARE class
class machines.
machines. These
These
additions are
are now incorporated
incorporated into
into the basic
basic simulation
simulation language.
language.
27
27
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
created an initial,
initial, experimental
experimental operating
operating system
system for
for CARE
CARE class
class
•. We created
machines,
machines, called CAOS. CAOS was
was used
used to produce our first
first experimental
experimental
results,
results, an end-to-end
end-to-end experiment
experiment using the EUNT
ELINT application,
application, using
replicated knowledge sources
sources and pipelining
pipelining for
for achieving parallel activity.
activity.
•. The results of
of these
these early experiments were encouraging. Linear
Linear speedup,
speedup,
close
close to the 45 degree
degree line, was
was achieved
achieved up to the intrinsic
intrinsic limits
limits of
of the
application.
application.
•. We generalized the traditional
traditional blackboard problem
problem solving
solving concept, and
developed
developed two new blackboard frameworks.
frameworks. These
These two frameworks, CAGE
CAGE
and POUGON,
POLIGON, take opposite points
points of
of view with
with respect
respect to the locus of
of
computing
CAGE
CAGE uses
uses knowledge sources
sources as
as the active agents,
agents,
computing activity.
activity.
whereas
whereas POUGON
POLIGON takes
takes a view that is oriented
oriented more towards
towards dataflow,
dataflow, in
which the blackboard nodes
nodes are the active agents.
agents.
•. We evaluated
evaluated a variety
variety of
of real-world
real-world applications
applications as
as drivers of
of the
underlying
underlying system
system levels, discarding several
several candidates
candidates which initially
initially looked
promising
promising but turned out not
not to be,
be, for
for various reasons.
reasons. Consequently,
Consequently, we
we
decided
decided to build
build our
our own application,
application, AIRTRAC.
AIRTRAC. As we programmed this
application
application in different
different problem
problem solving
solving frameworks
frameworks we began
began to learn
initiated experiments to study the
techniques
techniques for
for parallel
parallel programming.
programming. We initiated
performance
performance of
of AIRTRAC
AIRTRAC in both blackboard frameworks.
of the performance
performance achieved
achieved in the EUNT
ELINT/CAOS
•. Detailed studies of
ICAOS
experiments·
experiments led to drastic simplification
simplification of
of the pipelining
pipelining scheme,
scheme, an
orientation
orientation toward implementing
implementing blackboard nodes
nodes as
as active agents,
agents, and
using parallel object oriented
oriented programming
programming as
as a low level implementation
implementation
technique. An
An environment,
environment, called LAMINA,
LAMINA, grew out of
of this analysis.
analysis.
Experiments
Experiments are in
in progress
progress to compare the performance of
of AIRTRAC
AIRTRAC
implemented
implemented in LAMINA
LAMINA with
with AIRTRAC
AIRTRAC implemented
implemented in the blackboard
frameworks.
experiments, using part
frameworks. The first
first set of
of AIRTRAC/LAMINA
AIRTRACYLAMINA
of
of the knowledge base
base that
that can be used
used in a data-driven
data-driven manner, exhibited
exhibited
linear
linear speedup
speedup close
close to the limit
limit of
of the concurrency inherent
inherent in the task.
task.
By the end of
of 1987
1987 we will
will have
have completed five
five sets
sets of
of vertical
vertical slice experiments. It
It is
already clear that
that these
these experiments could have significant
significant impacts on both the
communities. Specifically:
Specifically:
hardware and software communities.
•. One important
important impact
impact of
of our research
research will
will be to shift
shift the emphasis
emphasis in
parallel architectures for
for knowledge-based
knowledge-based systems
systems from
from (probably
(probably
premature) building
building of
of hardware to the development of
of software systems,
systems,
techniques
techniques and tools for
for the encoding of
of knowledge-based
knowledge-based applications.
applications.
Hardware can certainly
certainly be built.
built. The real difficulty
difficulty is in developing a firm,
firm,
quantitative
quantitative understanding of
of what hardware actually matters and what
hardware may actually
actually hurt
hurt (e.g.,
(e.g., building
building hardware based
based upon incompletely
incompletely
thought-out
thought-out policy
policy decisions in the software design).
•. We will
will have demonstrated that the distributed
distributed memory paradigm is not
only
only a viable
viable alternative
alternative to shared
shared memory
memory architectures, but
but perhaps
perhaps
superior
provide evidence
superior in important
important ways.
ways. The vertical
vertical slice experiments provide
evidence
that
that implementing
implementing a relatively
relatively complex application,
application, using a non-shared
without the
address
address space
space with
with message
message passing,
passing, can be accomplished without
will have
have
complexities
complexities of
of managing shared
shared address
address spaces.
spaces. Moreover, we will
demonstrated that distributed-memory
distributed-memory multicomputers
multicomputers can
can be programmed to
achieve significant
significant (ten to one hundred times) speed-up for
for nontrivial
nontrivial
Furthermore, such
such multicomputer
multicomputer
symbolic
symbolic problem
problem solving
solving applications.
applications. Furthermore,
E. H. Shortliffe
Shortliffe
28
28
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR00785-14
fit to
to the
the (forecasted)
(forecasted) technology
technology for
for ULSI
ULSI of
of
systems will
will provide
provide aa better
better fit
systems
1990'ss than
than the
the shared memory
memory architectures.
the 1990’
of power”
power" in
in parallel
parallel
We will
will have
have demonstrated
demonstrated that
that the
the major
major “source
"source of
.• We
computing is the ability
ability to
to allow
allow the user to
to express
express and manipulate
manipulate parallel
parallel
computing
of the application.
application.
return on
Thus, the best return
constructs at the level of
appropriate tools
tools to
to support
support parallelism
parallelism at this
this
investment is to
to develop appropriate
investment
to support
support the development
development of
of the underlying
underlying languages
languages or
or
level, rather
rather than to
compilers.
The speedup
speedup obtainable
obtainable by
by only
only parallelizing
parallelizing programming
programming
The
compilers.
language constructs in
in a “programmer
"programmer transparent”
transparent" manner
manner (e.g.,
(e.g., parallel
parallel
language
or parallel
parallel production
production systems)
systems) is very limited.
limited.
Prolog or
Prolog
An important
important lesson
lesson learned from
from the success
success of
of our
our simulator
simulator is that
that real
.• An
environment,
applications can be carefully
carefully analyzed in
in an instrumented
instrumented environment,
applications
thereby permitting
permitting experimentation
experimentation with
with alternate
alternate architectures.
The
The
community would do well to stress
stress simulation
simulation over hardware building:
bui.lding:
community
will have demonstrated the need
need for
for fast
fast process
process creation
creation and process
process
.• We will
switching mechanisms.
mechanisms.
switching
[See KSL
KSL technical memos
KSL-87-34,
KSL-87-35.]
KSL-87-34, KSL-87-35.1
KSL-86-36,
KSL-86-36,
KSL-86-69,
KSL-86-69,
KSL-87-02,
KSL-87-02,
KSL-87-07,
KSL-87-07,
2.4 - Knowledge
Knowledge Acquisition
Acquisition and
and Machine
Machine Learning
Learning
Our research
research in machine learning
learning has
has focused on several
several distinct
distinct problem
problem domains
Our
and biochemical
biochemical (PROTEAN)
in
including
(PROTEAN)
including medical (NEOMYCIN/HERACLES)
(NEOMYCIN/HERACLES)
addition
need for
motivated by the need
for
addition to domain-independent
domain-independent investigations.
investigations. We also are motivated
effective
bases
of knowledge bases
effective tools for
for knowledge acquisition
acquisition and maintenance of
(IMPULSE
(IMPULSE and STROBE for
for FRM,
FRM, BBEDIT,
BBEDIT, KSEDIT
KSEDIT with
with BB1).
BBl).
Several
papers by researchers
presented at AAAI-86
Several papers
researchers in the KSL
KSL were presented
AAAI-86 in
in Philadelphia
Philadelphia in
August. Wilkins
Wilkins and Buchanan
Buchanan describe
describe a method of
of debugging
debugging rule sets
sets (see
(see below).
Rosenbloom and Laird
Laird [14]
[14] present
present a mapping between
between the SOAR architecture and
explanation-based generalization (EBG),
justifiable concept definition
(EBG), in which a justifiable
definition is
acquired from
from a single training
training example and an underlying
underlying theory of
of how the example
is an instance of
of the concept. SOAR is an architecture
architecture that supports general
general learning
learning
through chunking, which
addition, the
which is similar
similar to but
but not the same
same as
as EBG. In addition,
authors suggest
suggest answers
answers to some
some of
of the outstanding issues
issues in explanation-based
generalization.
Chunking
Chunking is a learning mechanism
mechanism that acquires
acquires rules from
from goal-based
goal-based experience.
experience.
SOAR is a general
general problem-solving
problem-solving architecture with
with a rule-based memory that can
can use
use
the learning capabilities
capabilities of
of chunking
chunking for
for the acquisition
acquisition and use
use of
of macro-operators.
Rosenbloom et al.
al. are
are investigating chunking
chunking in SOAR and
and find
find that chunking
chunking obtains
extra scope
scope and
and generality from
from its intimate
intimate connection with
with the
the sophisticated problem
solver (SOAR) and
and the
the memory organization
organization of
of the production
production system.
system.
In their
their AAAI-86
AAAI-86 paper,
paper, Horvitz,
Horvitz, Heckerman, and
and Langlotz present
present aa framework
framework for
for
comparing alternate
alternate formalisms
formalisms for
for plausible reasoning
reasoning [6].
[6]. They demonstrate
demonstrate aa logical
relationship
relationship between
between several
several intuitive
intuitive properties for
for measures
measures of belief
belief and
and the
the axioms
axioms
of
of probability
probability and
and discuss
discuss its relevance
relevance to research
research on reasoning
reasoning under
under uncertainty
uncertainty in
artificial
artificial intelligence.
intelligence.
29
29
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
of Technical
Technical Progress
Progress
Details of
Inductive Rule Learning
Learning
Inductive
Buchanan, et al. present an empirical
empirical study of
of the incremental
incremental learning
learning process
process using a
Buchanan,
of counter
counter examples in
in concept formation
formation with
with the rule-learning
rule-learning
careful selection of
system RL
RL (described in
in last year’
year'ss SUMEX
SUMEX report).
report). They
They find
find that
that “near
"near misses”,
misses",
system
that are similar
similar to
to acceptable
acceptable cases,
cases, are particularly
particularly effective
effective in
in
negative examples
examples that
of possible theories that
that explain
explain the examples observed.
observed. They
shrinking the space
space of
shrinking
define a metric
metric for
for the distance of
of each
each example from
from the target theory and measure
measure
define
of examples
exam pies related to the distance measured,
measured,
efficiency of
the effectiveness and efficiency
that the power of
of near misses
misses to restrict
restrict the space
space of
of possible theories
demonstrating that
their small distance from
from the target. They
They also find
find that
that intelligent
intelligent
results from
from their
of instances based
based upon knowledge of
of the state of
of the evolving
evolving theory
theory results
selection of
of an evolving
evolving theory
theory toward
toward the target concept, requiring
requiring many
in a faster convergence
convergence of
in
for learning.
learning.
cases for
fewer cases
Debugging Knowledge Structures
systems, the performance
performance of
of the system
system is strongly
strongly dependent on the
In large rule-based systems,
of the system
system is “debugged”
"debugged" and refined,
refined, i.e.,
Le., erroneous
degree to which
which the knowledge of
degree
combined, missing rules are added,
added,
identified and removed, redundant rules are combined,
rules are identified
of rules are found
found that give good results over many cases.
cases. Such
and certainty
certainty factors
factors of
restructuring of
of knowledge is an important
important type of
of learning
learning and can be
evaluation and restructuring
work in the debugging and
automated to some extent.
automated
Here we describe recent work
of knowledge bases
bases using several
several techniques.
techniques.
refinement of
refinement
problem with
with the rule sets
sets of
of rule-based systems
systems
Wilkins and Buchanan [19]
Wilkins
[19] analyze a problem
that use
use certainty
certainty factors, i.e.,
i.e., better individual
individual rules do not
not necessarily
necessarily lead
lead to a better
overall set
less-than-certain rules contribute
contribute evidence towards
set of
of rules. Since all less-than-certain
problem instances,
erroneous
erroneous conclusions for
for some problem
instances, the distribution
distribution of
of these
these erroneous
conclusions is not
This has
has
not necessarily
necessarily related to the quality
quality of
of individual
individual rules. This
important
important consequences
consequences for
for automatic
automatic machine learning
learning of
of rules, since rule selection is
usually based
based on measures
measures of
of quality
quality of
of individual
individual rules. The authors present a method
performs a model-directed
using
using a new Antidote
Antidote Algorithm
Algorithm that performs
model-directed search
search of
of the rule
space
space to find
find an improved
improved rule set.
set. They report
report that the application
application of
of this method
significantly
significantly reduces
reduces the number
number of
of misdiagnoses
misdiagnoses when
when applied to a rule set generated
generated
presented at the AAAI-86
from
from 104 training
training instances.
instances. This
This work
work was
was also presented
AAAI-86 Conference
in August.
August.
Debugging
synthetic agent
Debugging the knowledge structures of
of a problem solving agent is the synthetic
method [20]
performance upper bound for
[20] determines a performance
for debugging
debugging a knowledge base.
base.
The synthetic agent
agent systematically explores the space
space of
of near miss training
training instances
instances and
expresses
expressesthe limits
limits of
of debugging
debugging in terms of
of the knowledge
knowledge representation and control
control
language
This paper presents
presents the framework
framework for
for
language constructs of
of the expert system.
system. This
evaluating a differential
differential modeling system.
system.
Wilkins
Wilkins describes
describes the ODYSSEUS apprenticeship
apprenticeship learning
learning program [21],
[21], designed
designed to
refine
refine and debug
debug knowledge bases
bases for
for the HERACLES
HERACLES expert system
system shell. ODYSSEUS
analyzes
analyzes the behavior of
of a human specialist using two underlying
underlying domain theories, a
strategy
strategy theory
theory for
for the problem solving
solving method (heuristic
(heuristic classification),
classification), and an
inductive
inductive theory
theory based
based on past
past problem solving sessions.
sessions. ODYSSEUS improves the
knowledge
knowledge base
base for
for the expert system
system shell, identifying
identifying bugs
bugs in the system's
system’s knowledge
knowledge
in the process
process of
of following
following the line-of-reasoning
line-of-reasoning of
of an
an expert, serving as
as a knowledge
knowledge
acquisition subsystem.
subsystem. ODYSSEUS
ODYSSEUS can
can also
also be used
used as
as part of
of an intelligent
intelligent tutor,
tutor,
identifying
identifying problems in a novice's
novice’s understanding and
and serving as
as student modeler for
for
tutoring
tutoring systems.
systems.
E.
E. H. Shortliffe
Shortliffe
30
30
Details
Details of
of Technical
Technical Progress
Progress
5P41- RR00785-14
5P41-RR0078514
Wilkins, et al. illustrate
illustrate that
that an explicit
explicit representation of
of the problem
problem solving
solving method
Wilkins,
underlying theories of
of the problem
problem domain
domain provide
provide a powerful
powerful basis for
for automating
automating
and underlying
learning for
for expert system
system shells [22].
[22]. By using domain-independent
domain-independent task
task procedures
procedures
learning
task procedure
procedure metarules,
melarules, domain
domain knowledge can be located and applied
applied to achieve
and task
However, these
problem solving
solving subgoals.
subgoals.
these rules are often
often limited
limited in
in use
use due to
problem
insufficient domain knowledge.
knowledge. This
This paper describes
describes the use
use of
of metarule
metarule critics
critics in
in
insufficient
for automating
automating the acquisition
acquisition of
of domain
domain knowledge,
knowledge, illustrating
illustrating a powerful
powerful
ODYSSEUS for
form of
of failure-driven
failure-driven learning
learning at the level of
of subgoals
subgoals as
as well
well as
as at the level of
of
form
solving the entire
entire problem.
solving
31
31
E. H. Shortliffe
Shnrtliffe
E.
of Technical Progress
Progress
Details of
5P41-RR00785-14
5P41-RR00785-14
III.A.3.4.
III.A.3.4. Core System
System Development
Development
1 - Introduction
Introduction
In this section we describe
describe progress
progress on our core system
system development and work
work toward a
distributed
Before launching
distributed AIM
AIM community.
community.
launching into
into the technical details, the
motivations and plans for
for core system
system work
work are first
first summarized along four
four
motivations
dimensions: 1) the motivation
-AIM community
motivation for
for the shift
shift of
of the SUMEX
SUMEX-AIM
community from
from a
central mainframe-based model of
of computing
computing resources
resources to a largely distributed
distributed
workstation-based model; 2) the prospects
prospects for
for workstation
workstation technology and vendor
support
- AIM systems
support for
for a diverse distributed
distributed AIM
AIM community;
community; 3) the core SUMEX
SUMEX-AIM
systems
tasks
tasks needed
needed to complement
complement vendor developments
developments to realize distributed
distributed community
community
operation;
operation; and 4) the integration,
integration, dissemination, and management of
of the shift
shift of
of the
AIM
AIM community
community from
from a centralized to a more distributed
distributed operation,
operation, including
including the
remaining
remaining central resource
resource functions:
functions:
•. Motivation
Motivation for
for a Distributed
Distributed Resource: The motivations
motivations for
for supporting
supporting and
and
AIM community
community as
as a distributed
distributed community
community are manifest.
manifest.
managing the AIM
First the cost/performance
cost/performance trade-offs
trade-offs between
between centralized shared
shared computing
computing
First
facilities and personal
personal workstations have
have shifted
shifted dramatically
dramatically toward
facilities
area of
of interactive
interactive symbolic
symbolic computation
computation
workstations, especially in the area
While the technology is still
still quite young, the very best
resources.
resources. While
environments
environments for
for developing knowledge-based
knowledge-based systems
systems for
for biomedicine
biomedicine are
arguably already on personal
personal workstations. Various
Various kinds
kinds of
of workstations
workstations
rapidly decreasing
decreasing in cost and increasing in performance
are rapidly
performance so that
appropriate
appropriate models
models can be selected
selected for
for cost-effective
cost-effective research
research support
support or
system dissemination into
into practical settings like
like health care clinics
clinics or
system
application
application laboratories.
Second,
Second, the AIM
AIM community,
community, with
with its growing ties into
into other diverse areas
areas
of
of biomedical informatics,
informatics, has
has long been
been too large to effectively
effectively support
support
A number of
of AIM
AIM groups have
from
from a single central node like
like SUMEX. A
mainframe computing
computing resources
resources (such as
as at Rutgers
already moved to local mainframe
University, the University
University of
of Pittsburgh, the University
University of
of California
California at
University,
Santa
Santa Cruz, the University
University of
of Minnesota, and Ohio
Ohio State
State University).
University). Only
Only
some of
of these
these have
have been
been able to establish network
network connections for
for their
their
some
machines
machines to date,
date, without
without which low-speed terminal
terminal connections must still
still
be made
EX resource
made to the central SUM
SUMEX
resource for
for mail
mail exchange,
exchange, software sharing,
information
information access.
access. As workstation prices fall,
fall, this trend toward
decentralization
decentralization will
will accelerate
accelerate and the need
need for
for uniform
uniform network
network access,
access,
information
information services,
services, and systems/software support
support will
will increase.
increase. The
challenge
challenge will
will be to provide responsive
responsive central resource
resource services
services that
encourage
encourage and facilitate
facilitate effective
effective communication,
communication, collaboration,
collaboration, and
information sharing in the new
new distributed
distributed environment.
environment.
information
•. Prospects
Prospects for
for Workstation
Workstation Technology:
Technology: Computer
Computer workstations
workstations have already
demonstrated
demonstrated remarkably
remarkably high performance and low cost for
for symbolic
symbolic
computing
The prospects
computing applications.
prospects for
for future
future generations of
of
workstations promise an even
even fuller
fuller spectrum of
of price/performance
price/performance
alternatives.
Even with
with the trend toward more effective
effective personal
workstations, however,
puting
however, there are
are still
still aspects
aspects of
of an overall
overall com
computing
environment
environment most effectively
effectively implemented and supported through central
resources. These
These include services
services like
like large-volume
large-volume information
information and file
file
resources.
storage,
storage, special
special parallel computing architectures, multi-vendor
multi-vendor systems
systems
expertise,
expertise, and experimentation
experimentation with
with integrating
integrating new computing
computing technologies
for
for community
community deployment. But hardware
hardware is only
only a small part of
of the
Shortliffe
E. H. Shortliffe
32
32
of Technical
Technical Progress
Progress
Details of
5P41-RR00785-14
5P41-RR00785-14
-- software
software represents
represents the larger challenge in
in the effective
effective
picture -picture
of workstations
workstations with
with shared
shared resources
resources --- and here is where a
integration of
integration
community systems
systems integration
integration effort
effort is required.
Most vendors are
Most
community
of their
their own products, whereas
whereas a
motivated to
to maximize
maximize the sales
sales of
motivated
community of
of the size and scope
scope of
of the AIM
AIM community
community must be prepared
community
from diverse vendors in order to maximize
maximize its
to integrate technologies from
of rapidly
rapidly developing new capabilities.
capabilities. The
The
productivity and to keep
keep abreast
abreast of
productivity
of SUMEX-AIM
SUMEX-AIM in
in this new era is to integrate what is available from
from
role of
diverse vendors with
with core system
system development efforts
efforts to facilitate
facilitate
community research
research and communications
communications and the smooth evolution
evolution of
of the
community
AIM distributed
distributed computing
computing environment.
environment.
AIM
Systems Development
Development Tasks:
Tasks: In
In order for
for workstations to support
support AIM
AIM
.• Core Systems
community activities
activities with
with minimum
minimum dependence
dependence on expensive,
expensive, central
central
community
not only
only outstanding knowledgeknowledgemainframes, they must be able to supply not
mainframes,
also general
general computing
computing
based system.
system. development environments
environments but also
based
for tasks
tasks like
like electronic
electronic communications,
communications, text processing,
processing,
environments for
environments
information and file
file management,
management, and utilities
utilities like
like spreadsheet
spreadsheet systems.
systems.
information
Many workstation
workstation environments
environments do not have
have fully
fully developed
developed facilities
facilities in
in all
Many
of core system
system
these areas
areas and must be augmented.
augmented. Another
Another major area
area of
these
effort will
will be in
in the development of
of tools to facilitate
facilitate effective
effective workstation
workstation
effort
workstation interactions.
interactions. These
These tools include being able to access
access remote
to workstation
of
workstation and central computing
computing resources,
resources, linking
linking the graphics displays of
workstation
workstations with
with each
each other over communication
communication networks,
remote workstations
establishing and managing cooperative computing
computing tasks,
tasks, and enabling remote
of files
files and information.
information. Finally
Finally we must stay abreast
abreast
transfer and sharing of
transfer
of
of the rapidly
rapidly changing workstati'on
workstation technology and have
have allocated a small
amount
purchase appropriate
amount of
of funding
funding each
each year
year to purchase
appropriate examples
examples of
of systems
systems
important
important to AIM
AIM community
community research
research for
for testing, evaluation,.
evaluation, and
development.
•. Managing
Managing the Community
Community Transition:
Transition: As system
system research
research and development
progresses, much will
progresses,
will remain to be done to integrate and disseminate these
these
new workstation
national AIM
workstation tools throughout
throughout the national
AIM community
community --- so that
the central DEC
phased out while maintaining
DEC 2060
2060 resource
resource can
can be phased
maintaining support
support
of
be tested,
of community
community activities.
activities. System
System tools must be
tested, evaluated,
evaluated, and refined
refined
in the broad context of
of the AIM
AIM community;
community: community
community groups must fund,
fund,
use suitable workstation
network
acquire,
install, and learn to use
workstation and network
acquire, install,
communications
communications equipment; residual central services
services must be
be developed
developed and
made
made accessible
accessible to support sharing software tools, user
user consulting, and
information
information resources;
resources; and AIM
AIM workshop and
and other management
management tools for
for
coordinating,
coordinating, integrating,
integrating, and extending community
community activities
activities must be
evolved.
evolved. We will
will use
use a small group of
of Stanford and AIM
AIM community
community AI
AI
researchers
researchers and students
students to guide the development and
and testing of
of distributed
distributed
subsystems
Initially, these
these will
will come mainly
mainly
subsystems throughout
throughout the research
research period. Initially,
from
from the Stanford community
community which is easily accessible
accessible and has
has a long
experience
experience in experimenting
experimenting with
with the development and use
use of
of workstation
workstation
technologies
technologies for
for AI
AI research.
research. After
After the early years
years of
of development and
experimental dissemination, we
we will
will begin
begin to introduce these
these tools more
extensi
vely for
extensively
for general
general AIM
AIM community
community use.
use. Our estimate
estimate is that these
these tasks
tasks
will
will require the full
full five-year
five-year research
research period in order to carry out the
necessary
necessary development, make an
an orderly
orderly and
and smooth transition,
transition, and evaluate
evaluate
the results,
results, without
without disrupting
disrupting communications
communications or inter-group
inter-group collaborations.
collaborations.
33
33
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
2 - Remote
Remote Workstation
Workstation Access,
Access, Virtual
Virtual Graphics,
Graphics, and
and Windows
Windows
2.1 - Remote Access
Lisp
Lisp workstations
workstations of
of various types
types have
have proven extremely powerful.
powerful, both as
as
development
development environments
environments for
for artificial
artificial intelligence research
research and as
as vehicles for
for
addition to the compact.
compact,
disseminating AI
AI systems
systems into
into user
user communities.
disseminating
In addition
inexpensive computing
computing resources
resources workstations
workstations provide.
provide, high-quality
high-quality graphics play a key
systems have become
for
role in their
Such graphics systems
become indispensable for
their power.
understanding the complex data structures involved
involved in developing and debugging
debugging large
AI
AI systems
systems and are important
important in facilitating
facilitating user
user access
access to working
working programs (e.g.,
(e.g., for
for
ONCOCIN
ONCOCIN and PROTEAN).
PROTEAN). However, as
as we
we move towards a distributed
distributed workstation
workstation
computing
computing environment
environment for
for AI
AI research
research in the SUMEX-AIM
SUMEX-AIM community
community (and move
away
away from
from the centralized, shared
shared DEC
DEC 2060).
2060), a number of
of technical obstacles
obstacles must be
of the most important
important is to eliminate
eliminate the need
need for
for the user
user display to
overcome. One of
close to the workstation
workstation computing
computing engine.
engine.
be situated close
This is important
important in order to allow
allow users
users to work on workstations over networks from
from
This
-- at work.
work, at home.
home, or across
across the country. The first
first step
step has
has been
been getting
getting
any location
location -All workstations now have
reliable
reliable terminal
terminal access
access operational
operational on all workstations. All
TCP/IP
based terminal
terminal servers,
servers, and TCP
TCP/IP
SUMEX network
network
TCP
lIP based
/IP is being installed in the SUMEX
This allows primitive
primitive (non-graphical)
(non-graphical) access
access to the
terminal
This
terminal concentrators.
workstation’s abilities.
abilities.
A more comprehensive access
access will
will be provided through our
workstation's
A
remote graphics work.
Virtual Graphics
Graphics
2.2 - Virtual
In the past,
past, members
members of
of the SUMEX-AIM
SUMEX-AIM community
community have
have often
often watched
watched each
each others
programs work
work by linking
linking their
their CRT
CRT terminals
terminals to the text output"
output-ofof a running
running program
SUMEX 2060.
2060. In the case
case of
of workstations, though, it
it is much more difficult
difficult to
on the SUMEX
link
link across
across several
several networks to view the complex graphics output
output of
of a program. Even
locally, it
it is important
important to make graphical interaction
interaction with
with workstations across
across campus
campus or
locally,
from
from home possible.
possible. One would like
like to be able to provide the same
same powerful
powerful graphical
tools and programming
programming environment
environment that are available to a user
user sitting
sitting in front
front of
of the
workstation
workstation to the remote user
user if
if that user
user has
has a low-cost
low-cost bit-mapped
bit-mapped display and
mouse.
it is necessary
necessary to capture and encode
encode the many
mouse. In order to accomplish this, it
graphics operations involved
involved so that they can be sent over a relatively
relatively low-speed
network
network connection
connection with
with the same
same interactive
interactive facility
facility as
as if
if one had
had the display
connected through the dedicated high-speed (30 Mhz)
Mhz) native vendor display/workstation
display/workstation
connection.
connection.
As reported last year.
year, we studied the feasibility
feasibility of
of remote access
access to workstations
workstations by
experimenting
experimenting with
with a virtual
virtual graphics protocol, the Virtual
Virtual Graphics Terminal
Terminal Service
Service
(VGTS),
(VGTS), which
which was
was developed
developed at Stanford
Stanford in the Computer Science
Science distributed
distributed systems
systems
group [9,
[9, 8].
81. The VGTS provides tools to define objects like
like windows, lines, rectangles.
rectangles,
circles, bitmaps, ellipses, splines, and graphics events
events like
like mouse
mouse clicks independently
independently of
of
operating systems.
systems.
This encoding minimizes
minimizes the
the graphics hardware and operating
This
communication
communication bandwidth
bandwidth required between
between cooperating hosts,
hosts, to remotely draw a line,
line,
for
for example.
We also reported that an implementation
implementation of
of this protocol
protocol was
was developed
developed and installed
installed
operating system
system of
of a Xerox
Xerox 1186
1186 Lisp
Lisp workstation
workstation so
so that its presence
presence would be
in the operating
transparent to the programmer. This
This means
means that if
if one connects
connects to such
such a LISP
transparent
workstation
workstation from
from a SUN workstation
workstation (running
(running suitable VGTS software), the Lisp
Lisp
machine graphics will
will be sent over the net and reconstructed
reconstructed on the SUN workstation
workstation
This implementation
implementation has
has worked
without
without changes
changes to the application
application program running.
running. This
Shortliffe
E. H. Shortliffe
34
34
Details of
of Technical
Technical Progress
Progress
5P41-RR00785-14
5P41-RR00785-14
well in
in early
early experiments
experiments so that
that over an Ethernet,
Ethernet, the remote response
response time
time is
very well
time on ‘the Lisp
Lisp machine itself.
itself.
quite close to the response
response time
quite
of this
this work,
work, we had demonstrated the feasibility
feasibility of
of remotely
remotely using
consequence of
As a consequence
Ethernet to take advantage
advantage of
of their
their graphics programming
programming
workstations over an Ethernet
LISP workstations
environment.
environment.
During the past year, two
two new contenders for
for a virtual
virtual graphics standard protocol
protocol
During
appeared.
MIT Project Athena
Athena X
X window system
system [15],
[15J, and Sun
These were the MIT
appeared.
Network Extensible
Extensible Window
Window System
System [17],
[17J, referred to as
as X
X and
Inc.'ss Network
Microsystems, Inc.‘
NeWS, respectively. We spent several
several months studying both X
X and NeWS and met with
with
NeWS,
of each
each group supporting
supporting these
these protocols.
representatives of
that has
has been
been developed
developed over the past several
several years
years at
complete protocol
protocol that
XX is a very complete
MITI. X
X operates
operates at a somewhat lower level than VGP, and as
as a result can be more
MITt.
bandwidth-intensive. ItIt also assumes
assumes a static allocation
allocation of
of computation,
computation, display, and
bandwidth-intensive.
interaction responsibilities
responsibilities between
between server and client.
client. On the other hand, itit more fully
fully
interaction
implements the event mechanisms
mechanisms necessary
necessary to track mouse/window
mouse/window interactions
interactions and
implements
mouse motion
motion histories, and supports color. The protocol has
has been
been quite carefully
carefully
mouse
flexibility for
for implementing
implementing reasonable
reasonable emulations
emulations of
of
thought out, and provides more flexibility
thought
the variety
variety of
of window
window systems
systems that exist within
within our environment.
environment. For
For example, TI
TI
X
Explorers have mouse-sensitive regions within
within windows called “active
"active regions,”
regions," and X
Explorers
support for
for such a region by defining
defining an Input
Input Only
Only window with
with its own
allows the support
mouse moves
moves into
into such a window, the cursor changes
changes to show the
cursor. When the mouse
that he has
has entered an active region, and at the same
same time
time sends
sends an enter-window
enter-window
user that
user
client. The client
client can then take the appropriate action for
for that
that active
event to the client.
(for instance, scroll text). This
This is impossible
impossible to do in VGP.
region (for
NeWS is unique in the sense
sense that it
it uses
uses a programming
programming language
language to define
define its protocol.
protocol.
This
This programming
programming language
language is an extension of
of Adobe's
Adobe’s PostScript page
page layout language
language
for
for laser
laser printers.
printers. This
This feature gives NeWS its extensibility,
extensibility, for
for if
if one wishes
wishes to add a
new function
function to the server,
server, one simply
simply sends
sends the PostScript procedure
procedure implementing
implementing it
it
to the server,
server, and remotely
remotely executes
executes that new procedure.
procedure. This
This gives
gives the client
client a great
deal
deal more control
control over what a window
window looks like;
like; for
for example,
example, one could implement
implement
also allows a client
client to interact
interact with
with
round or elliptical
elliptical windows with
with NeWS. NeWS also
mouse
mouse motion
motion histories and mouse/window
mouse/window events.
events. Thus, it
it was
was very difficult
difficult to choose
choose
between
between these
these two protocols.
Ultimately,
Ultimately, we
we chose
chose X
X as
as the remote graphics protocol
protocol standard
standard for
for our work. This
decision was
was pragmatic, since we
we have
have limited
limited staff
staff resources,
resources, and X
X is receiving wide
X client
client
support
An X
support from
from both vendors and the Common Lisp community.
community.
implementation
implementation is being written
written for
for Texas
Texas Instruments
Instruments Explorers here
here at SUMEX-AIM2.
SUMEX-AIM*.
It is being written
written in Common Lisp and
Our TI
TI Explorer
Explorer X
X client
client is well underway.
underway. It
flavors, the Explorer
uses
uses flavors,
Explorer object system,
system, to represent
represent instances
instances of
of X
X windows.
windows. We are
are
currently
currently beta-testing
beta-testing Xerox
Xerox Common Lisp, and will
will port the Explorer
Explorer X
X client
client to our
Xerox Lisp
Lisp Machines later this year.
year.
Currently.
Currently, TI
TI in conjunction
conjunction with
with MIT
MIT is developing a server
server implementation
implementation for
for
Explorers. DEC
DEC is a major supporter of
of X.
X, and there
there are
are implementations
implementations under
development for
for their
their Vax line
line of
of equipment. Sun
Sun Microsystems
Microsystems is also
also doing an
an X
X
1
IThe
The X
X protocol
protocol has been completely
completely redefined
redefined this
this past
past year.
year.
all
all of
of the
the discussion
discussion that
that follows.
follows.
Its most
most recent
recent version,
version, x.n,
X.11, is assumed
assumed in
Tts
,.,
~The
‘The client
client software
software runs
runs on the
the Lisp
Lisp machine
machine and
and sends
sends the graphics
graphics protocol
protocol commands
commands to the remote
remote
user
ient is the X
user display
display system.
system. The
The dual
dual l)f
of the cl
client
X server
server software
software which
which runs
runs on the user display
display system
system
and
and translates
translates the X
X protocol
protocol sent
sent by a client
client Lisp
Lisp machine
machine into
Into real graphics
graphics pictures
pictures and
and mouse
mouse actions.
actions.
35
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
implementation
implementation beneath
beneath NeWS, as
as well as
as porting
porting X
X to run directly
directly on their
their equipment.
We are an alpha test site for
This
for the SUN implementation.
implementation.
This will
will provide
provide us
us with
with
preproduction
preproduction X
X server software
software that
that we can run on our SUN workstations
workstations to aid in
anticipate implementations
implementations for
for workstations
debugging
debugging our own client
client software. We anticipate
like
like MacIntosh
Macintosh II's
II’s when a production
production version of
of X
X is released
released this Fall.
The X
X window
window protocol
protocol is more bandwidth
bandwidth intensive
intensive than some other protocols. It
It is
our feeling
protocol can be
feeling that
that even
even with
with this limitation,
limitation, a suitable subset
subset of
of the X
X protocol
used
used in cross-country
cross-country connections
connections where slower communications
communications speeds
speeds and longer
delays
delays are common. We will
will have to determine
determine empirically
empirically what this subset
subset is. One,
for
for example, would not
not want to track a mouse
mouse in such
such a situation,
situation, but
but could reasonably
reasonably
expect to use
EnterWindow or LeaveWindow, to manage
use mouse/window
mouse/window events,
events, such as
as EnterWindow
manage a
case, more work
work needs
needs to be
remote display over long connection
connection distances.
distances. In any case,
done in this area
area to fully
fully develop and integrate these
these capabilities
capabilities into
into Lisp machine
systems
systems and to insure that cross-country
cross-country connections will
will indeed give usable
usable response
response
time. Success
Success of
of this
this work
work will
will mean that one can use
use LISP machine systems
systems from
from
TELENET,
TELENET, ARPANET,
ARPANET, or an Ether
Ether TIP
TIP connection
connection throughout
throughout the SUMEX-AIM
SUMEX-AIM
community.
community.
2.3 - Remote Graphics
Applications
Graphics Applications
As an example of
been
of applying
applying the remote graphics ideas,
ideas, a TALK
TALK program has
has been
implemented which facilitates
facilitates interactive,
interactive, electronic
electronic communication
communication between
users on
implemented
between users
workstation’s native editor,
editor, the program
independent workstations.
workstations. Layered on the workstation's
full use
use of
of all editing
editing capabilities
capabilities in the process
process of
of communication,
communication, including
including
allows the full
corrections and insertions,
insertions, font
font changes,
changes, underlining,
underlining, paragraph
formatting,
deletions, corrections
paragraph formatting,
etc. Since the workstation's
workstation’s editor
editor also supports both lowlow- and high-level
high-level graphics, the
not only
only facilitates
facilitates textual exchanges
exchanges among users,
users, but
but also allows the sending
program not
as well as
as
of
of screen
screen images
images (back traces
traces of
of program breaks,
breaks, code
code fragments, etc.) as
structured graphics images
images (which
(which can be modified
modified on the destination
destination workstation
workstation and
returned), all interactively.
An
interactively.
An example of
of a TALK
TALK session
session and an illustration
illustration of
of
TALK's
TALK’s relationship
relationship to other
other subsystems
subsystems in the workstation
workstation software environment
environment are
2.
shown in Figure 2.
The TALK
TALK program allows the use
use of
of different
different user
user interfaces, the workstation's
workstation’s
simpler terminal
terminal
document editor
just one possibility.
editor being just
possibility. We also implemented a simpler
mode for
for compatibility
compatibility with
with similar
similar programs on other similar
similar and dissimilar
dissimilar
workstations. The program was
was implemented
implemented initially
initially using the Xerox
Xerox XNS family
family of
of
Ethernet
Ethernet protocols
protocols for
for convenience and speed
speed of
of development to tryout
try out the ideas.
ideas.
Future
Future extensions will
will include
include allowing
allowing use
use of
of different
different Ethernet
Ethernet (and possibly nonnonEthernet)
Ethernet) protocols, since the program only
only requires a reliable
reliable byte-stream to operate.
operate.
We expect the IP/TCP
IP/TCP protocols
protocols will
will be added
added next in order to be able to use
use the
program over the ARPA
ARPA network.
network.
The TALK
TALK program was
was released
released gradually
gradually to increasing numbers of
of users
users in order to
get real users'
users’feedback and make changes
changes accordingly. The Medical Computer
Computer Science
Science
group did an extensive test of
of the system,
system, where for
for a period, they used
used it
it in place
place of
of
their normal
normal electronic
electronic and non-electronic
non-electronic communication
communication methods whenever
whenever possible.
their
possible.
This
This was
was both a test of
of the program and an exploration
exploration into
into what people want in the
next generation of
of electronic
electronic communication.
communication. The TALK
TALK program has
has been
been released
released to
the Xerox Lisp
Lisp workstation
workstation community
community as
as a whole and researchers
researchers at Xerox PARC
successfully used
used the program to hold an interactive,
interactive, graphic, electronic
electronic conversation
between users
users at the PARC
PARC facility
facility (in
(in California)
California) and Xerox's
Xerox’s EuroPARC
EuroPARC in England.
between
Shortliffe
E. H. Shortliffe
36
36
Details
Details of
of Technical
Technical Progress
Progress
5P41-RR0078514
5P41-RR00785-14
The files you w~nt ~re
don't delete ~ny until
rev 1 e':! them!
Talk from Guest MCS: Stanford
H~ve
you
h~d
~
chance to
gener~te
~ny
of the
..11el'; i.JI'1.3.-y ft7es J'et'~
01<., th.;nk3!
I'll
le~ve
them In:
....... O,u,TA
I'{;:, -';UEST :::', OlCTDIi'It\'
.... SOURCE
When I'm through.
I
Local
Local Host
Host
~rw .. ~'I" r.1I II; ("MC.. H' pI''''I'I'"
ofJ'il';«:h1ftll")' ",.,.. ...
rr~ ·If ' ....
u
'I",
u",r•• ,
I'll
1", ...
no'!. ,,..
'--"'1fI'T:"'"~
'I""'" J', '·'''·I'''lr.. _
\
*I ..,I %,
(
-&&*
,
m
j
Remote Host
Host
Remote
User Interlace
GraDl1ICS Interlace
GAP
Remore PrOCedure Call
................
] ..........
..................
................
............
]
F'Jtufe
E.-;.[enslon
(Rehable! Byte Stream
Transport ProtOCOl
Transm,SSIOIl Medoa
Figure
Figure 2:2:
Ethernet
TALK Session
Session Example
Example and
and the
the Software
Software Layers
Layers Involved
Involved inIn TALK
TALK
TALK
37
37
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
Application-level Window
Window System
System Standards
Standards
2.4 - Application-level
Modern programs need to
to utilize
utilize the multiple
multiple presentations, non-textual
non-textual images,
images, and
Modern
non-keyboard inputs
inputs available
available on all
all the systems
systems in
in use
use by
by SUMEX.
SUMEX. However, up until
until
non-keyboard
each machine’
machine'ss window
window system has
has been idiosyncratic
idiosyncratic to that
that machine. There is
now, each
providing a powerful,
powerful, flexible
flexible window
window system
system that
research now aimed at providing
considerable research
implemented on a wide variety
variety of
of hardware, and utilized
utilized by many forms
forms of
of
can be implemented
software. However, most of
of this
this research
research is directed at the primitive
primitive operations
operations needed
needed
to do basic graphics, windowing,
windowing, and interaction
interaction (as in
in the discussion of
of X
X protocols
protocols
interface to a standard windowing
windowing
above). We are also working
working to develop a high
high level interface
above).
writer of
of AI
AI applications
applications programs. This
This system
system is not
not being
system targeted
targeted at the writer
system
entire man/machine
man/machine interface,
interface, but
but to provide
provide a simple, easy
easy to
designed to specify
specify the entire
designed
understand and useful way for
for program
program authors to provide
provide sophisticated interfaces
interfaces
of their
their time
time working
working only
only on the interface.
interface. We
without spending a large percentage
percentage of
without
currently in
in the midst
midst of
of analyzing
analyzing current
current applications
applications in
in order
order to develop a model
are currently
for this system
system based
based on real-world
real-world experience.
for
File Access
Access and Management
3 - File
stable, efficient
efficient mechanism for
for storing
storing and organizing
orgamzmg data is central to any
AA stable,
computing environment,
environment, and is one of
of the most challenging
challenging issues
issues in the move to
computing
distributed, workstation-based
workstation-based computing.
computing. ItIt is necessary
necessary to provide
provide standard services,
services,
distributed,
file backup, archival,
archival, a flexible,
flexible, intuitive
intuitive naming
naming facility,
facility, and data interchange
such as
as file
such
services (e.g.,
(e.g., software distribution).
distribution). We also feel that, as
as the amount
amount of
of data being
services
will become more and more important
important to have powerfpl
powerful tools for
for
manipulated grows, itit will
of files.
files. We plan to support
support the community
community with
with a number
number of
of
managing hierarchies of
UNIX-based
use at SUMEX
years
UNIX-based file
file servers,
servers, like
like the VAX-based
VAX-based servers
servers in
in use
SUMEX for'several
for’several years
(see
These will
will require
(see Figure 7) and the new SUN-based server (see
(see Figure
Figure 5). These
continued SUMEX-AIM
of servers
servers
SUMEX-AIM development, however. By keeping the number of
small, the distributed
namespace
problem
should
be
manageable
in
the
near
term.
distributed namespace problem
manageable
Current
UNIX has
needed
Current UNIX
UNIX file
file servers
servers are relatively
relatively cheap
cheap and fast. UNIX
has many of
of the needed
property
facilities,
names, hierarchical
facilities, e.g.,
e.g., backup, long names,
hierarchical directory
directory structure, some
some file
file property
attributes, data conversion, and limited
limited archival
archival tools. However, while
while general
general issues
issues of
of
networking,
have received
networking, remote memory
memory paging services,
services, and flexible
flexible file
file access
access have
considerable attention
attention in
in both the academic and commercial
commercial development of
of file
file
needs. For
servers,
servers, there seems
seems to be little
little attention
attention given to other critical
critical operational
operational needs.
For
instance,
instance, the much-used file
file archiving
archiving system
system of
of the DEC
DEC 2060 (sometimes called offoffline
UNIX systems.
line cataloged
cataloged storage)
storage) has
has no analog service in the UNIX
systems. Perhaps
Perhaps this is the
result of
of UNIX
UNIX having its origin
origin in
in the small computer
computer world
world where the number of
of
users
users and volume of
of data has
has traditionally
traditionally been
been quite low. Our efforts
efforts are going into
into
improving
improving the archival
archival facilities
facilities and providing
providing case
case independence
independence and multiple
multiple
generations by adding SUM
EX software between
SUMEX
between the file
file system
system and the network. This
This
should temporarily
temporarily solve these
these problems without
without substantial loss
loss of
of performance or
maintainability.
maintainability.
For the long-term
long-term use
use of
of the distributed
distributed community,
community, we plan to develop
develop an optical
optical
disk-based
disk-based backup and archival
archival system
system and to use
use enhanced
enhanced tools on workstations
workstations to do
file
file management.
management. We are currently
currently investigating
investigating hardware options
options for
for optical
optical disk
systems.
systems. As better techniques
techniques for
for managing a distributed
distributed file
file system
system come out of
of the
early research
research stages,
stages, we
we will
will use
use them to improve
improve the distributed
distributed file
file service
service facilities.
facilities.
3.1 - Remote
Remote File
File Access
During
During the past
past year,
year, there has
has been
been a welcomed
welcomed progress
progress in vendors'
vendors’ attempts to
standardize
standardize file
file access
access protocols. Previously, each
each vendor had
had addressed
addressed the file
file storage
storage
needs
needs of
of their
their particular
particular workstation
workstation in a way
way that was
was incompatible
incompatible with most other
E. H. Shortliffe
Shortliffe
38
of Technical
Technical Progress
Progress
Details of
Details
5P41- RR00785-14
5P41-RR00785-14
workstations, making
making shared file
file access
access and
and support
support difficult
difficult in
in aa highly
highly heterogeneous
heterogeneous
workstations,
environment such as the
the SUMEX-AIM
SUMEX-AIM community.
community. Also,
Also, the
the resources
resources required
required to
to
environment
maintain many
many distinct
distinct families
families of
of filing
filing conventions
conventions and protocols
protocols on
on specialized
maintain
performance needs
needs of
of aa demanding
demanding research
research community,
community, are
are
hardware, all
all meeting
meeting the
the performance
hardware,
prohibitive. Thus, last year we proposed to adopt a variant
variant of
of the NFILE
NFILE file
file access
access
prohibitive.
developed by
by Symbolics, Inc.
Inc. ItIt now
now appears,
appeaTs, however, that
that Sun Microsystems,
protocol 1 developed
protocol1
File System
System (NFS)
(NFS) is becoming
becoming a more
more prevalent
prevalent industry
industry
Inc.'ss (SMI)
(SMI) Network
Network File
Inc.‘
fact that
that itit does not
not support
support extensible file
file attributes
attributes and file
file
standard, despite the fact
In order
order to
to encourage
encourage the porting
porting of
of NFS
NFS to
to other
other vendors’
vendors' workstations,
workstations,
generations. In
SMI has placed NFS
NFS in
in the public
public domain,
domain, and has
has a special group dedicated to aiding
aiding
SMI
requisite software. This
This group is also willing
willing to make
interested parties in
in writing
writing the requisite
to support
support non-UNIX
non-UNIX file
file systems
systems (for
(for example, they
changes to the protocol
protocol to
some changes
change so that
that NFS
NFS could be ported to
to a CRAY
CRA Y computer).
computer). We
We are
recently made a change
Instruments implementation
implementation of
of NFS
NFS on our
our Explorers,
Explorers, and are
ar~
now beta-testing
beta-testing a Texas Instruments
ourselves
engaged
in
implementing
NFS
on
Xerox
Lisp
workstations.
ourselves engaged in implementing NFS
Xerox Lisp workstations.
experimental SUN
SUN file
file server this year, and that
that NFS
NFS is
Given that we have acquired an experimental
Given
in the Kernel
Kernel of
of the 4.3 release
release of
of Berkeley UNIX,
UNIX, this path for
for unified
unified file
file
supported in
our mix
mix of
of workstations
workstations appears
appears to be the best solution
solution available. Our
Our
access across
across our
access
anticipated move to 4.3
4.3 UNIX
UNIX on our
our VAX
VAX file
file servers
servers this
this summer, and the
anticipated
of the NFS port
port to the Xerox
Xerox Lisp
Lisp machines will
will give us
us a single file
file access
access
completion of
completion
of our systems
systems with
with the exception of
of the Symbolics
protocol that is supported by all of
protocol
3600·s.
appears that a third
third party is working
working on an NFS implementation
implementation for
for
3600’
s.
ItIt appears
Symbolics machines
machines and
an~ we will
will test this in the coming
coming year.
year.
3.2 - File
File Server
Server Throughput
Throughput
At
present, a number of
within the
At present,
of file
file service strategies
strategies are employed among and within
various workstation
has its merits
merits and
workstation and time-sharing
time-sharing communities.
communities. Each strategy has
drawbacks
needs of
users.
drawbacks and only
only in
in their
their aggregate
aggregate do they address
address all the needs
of the users.
utility is the maximum
One yardstick of
of utility
maximum speed
speed of
of data transfer.
transfer. Speed
Speed of
of data transfer
transfer
is affected by the speeds
speeds of
of the processors,
processors, disks,
disks, I/O
I/O circuitry,
circuitry, file
file system
system design,
design,
network transport
transport protocols, file
file service
service protocols, software efficiency,
efficiency, system
system loading,
and other operational parameters.
throughput measurements
measurements suggest
suggest that for
for the
parameters. Simple throughput
immediate future, the mixed-vendor
mixed-vendor file
file service
service strategy
strategy still
still has
has advantages
advantages from
from the
point
point of
of view of
of data
data transfer
transfer speed.
speed. (See
(See Figure 3.)
For the Xerox workstations, the Xerox 8037
8037 file
file server
server (using the NS Filing
Filing protocol)
protocol)
provided the greatest
greatest measured
measured throughput
throughput (roughly
(roughly 37%
37% faster than the
the Sun
Sun 3/180 and
Vax 111750
111750 file
file servers,
servers, using TCP FTP). For the TI
TI workstations, the fastest
fastest server
was
was another TI
TI Explorer
Explorer (using the Chaos
Chaos FILE
FILE protocol)
protocol) providing
providing throughput
throughput 91%
91%
greater
greater than the nearest
nearest contender (a vax
vax using
using the
the Chaos
Chaos FILE
FILE protocol),
protocol), and 269%
269%
faster than the
Sun workstation
workstation provides aa virtual
virtual file
file
the closest
closest IP/TCP
IP/TCP contender. The Sun
system
system interface only for
for the Sun
Sun NFS protocol, and
and hence
hence was
was not benchmarked
benchmarked
against
against alternative servers
servers because
because we
we are
are still
still working
working on optimized
optimized NFS facilities
facilities for
for
other workstations and servers.
servers.
None of
of the client/server
client/server configurations
configurations tested
tested approached
approached the theoretical maximum
maximum
throughputs projected
by
disk
speeds,
network
speeds,
projected
speeds,
speeds, and
and other system
system design
design
considerations.
considerations. Therefore, we
we believe that through more effective
effective software
software engineering
it will
will be
be possible
possible to simultaneously
simultaneously improve data
data transfer speed
speed and
and to reduce
reduce the
the
number of
of server
server implementations
implementations necessary
necessary to support
support the
the present
present level
level of
of service.
service.
For example,
example, the
the potential for
for software
software improvement
improvement was
was illustrated
illustrated this year
year by finefine1 file access protocol is intermediate between a remole file system and a file transfer protocol.
lA
A file access protocol is intermediate
betwken a remote file syslem and a file transfer prolocol.
39
39
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
of Technical
Technical Progress
Progress
Details of
of the Xerox
Xerox implementation
implementation of
of TCP, which
which yielded improved
improved Sun file
file server
tuning of
tuning
throughput by a factor
factor of
of 30. In
In the immediate
immediate future,
future, our
our experiments
experiments in
in this
this area
throughput
will focus on the new implementations
implementations of
of the NFS
NFS file
file service protocol.
protocol.
will
Reading Throughput
Throughput
Reading
Client
Client
Server
Server
Protocol
Protocol
DEC 2060
DEC
Sun 31180
Sun 3175
3175
31180
Sun 31180
DEC 2060
DEC
Sun 3/180
31180
Sun
FTP
TCP FTP
FTP
TCP FTP
FTP
TCP FTP
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
1186
DEC 2060
DEC
DEC 780 (VMS)
(VMS)
DEC
Xerox IFS
Xerox
DEC 750 (UNIX)
(UNIX)
DEC
DEC 2060
2060
DEC
31180
Sun 31180
DEC 2060
2060
DEC
DEC 750 (UNIX)
(UNIX)
DEC
Xerox IFS
8037
Xerox 8037
Xerox 8033
8033
Leaf
PUP Leaf
TCP FIP
FTP
TCP
Leaf
PUP Leaf
Leaf
PUP Leaf
PUP FTP
TCP FTP
TCP FTP
FTP
FTP
TCP FTP
PUP FTP
Filing
NS Filing
NS Filing
Filing
18,181 baud
18,181
33,402 baud
33,402
52,526 baud
52,526
53,036 baud
53,036
67,001 baud
67,001
71,192 baud
71,192
72,207 baud
72,207
72,412 baud
72,412
84,125 baud
84,125
103,519
103,519 baud
105,486
105,486 baud
(loaded)
(loaded)
(unloaded)
(loaded)
(loaded)
(unloaded)
(loaded)
(loaded)
(unloaded)
(loaded)
(unloaded)
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
Xerox
1132
1132
1132.
1132,
1132
1132
1132
1132
1132
1132
1132
1132
1132
1132
1132
1132
DEC
DEC
DEC
DEC
DEC
DEC
DEC
DEC
DEC
DEC
DEC
FTP
TCP FTP
Leaf
PUP Leaf
PUP FTP
PUP Leaf
TCP FTP
PUP FTP
TCP FTP
NS Filing
Filing
3,228 haud
baud
3,228
18,737
18,737 baud
75,361 baud
75,361
81,711 baud
81,711
121,163
121,163 baud
baud
167,687
167,687 baud
215,000
215,000 baud
234,154
234,154 baud
(loaded)
(loaded)
(loaded)
(loaded)
(loaded)
(loaded)
(loaded)
(loaded)
TI
TI
TI
TI
TI
TI
TI
TI
TI
TI
TI
TI
Explorer
Explorer
Explorer
Explorer
Explorer
Explorer
Explorer
Explorer
Explorer
Explorer
Explorer
Explorer
2060
2060
2060
2060
750 (UNIX)
(UNIX)
(UNIX)
750 (UNIX)
750
750 (UNIX)
(UNIX)
2060
2060
Sun
Sun 3/180
31180
Xerox 8037
8037
DEC
DEC 750
750 (UNIX)
(UNIX)
Sun
Sun 31180
3080
TI
TI Explorer
Explorer
DEC
DEC 2060
2060
DEC
DEC 750
750 (UNIX)
(UNIX)
TI
TI Explorer
Explorer
Figure 3:
E.
E. H. Shortliffe
Shortliffe
.(loaded)
7,000 baud (loaded)
17,000 baud (loaded)
17,000
55,000 baud (unloaded)
55,000
TCP FTP
FI-P
TCP FTP
FfP
TCP FTP
TCP FTP
Chaos
Chaos FILE
FILE
Chaos
Chaos FILE
FILE
Reading Throughput
Reading
Throughput
Writing
Writing Throughput
Throughput
36,952
36,952 baud
58,888
58,888 baud
61,376
61,376 baud
63,320
63,320 baud
122,136
122,136 baud
233,008
233,008 baud
96,000
96,000 baud
135,208
135,208 baud
121,512
121,512 baud
110,592
110,592 baud
129,376
129,376 baud
221,192
221,192 baud
File
File Server
Server Throughput
Throughput Benchmarks
40
(was 2,412
2,412 baud)
(was
(was 2,850 baud)
(was
(was 9,096
9,096 baud)
(was
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
Electronic Mail
Mail
44 -- Electronic
Electronic mail
mail has become a primary
primary means of
of communication
communication for
for the
the widely
widely spread
spread
Electronic
SUMEX-AIM community.
community.
The advent of
of distributed
distributed workstations
workstations is
is forcing
forcing aa
SUMEX-AIM
The
significant rethinking
rethinking of
of the mechanisms
mechanisms employed to
to manage
manage such mail.
mail.
With
With
significant
mainframes,
each
user
tends
to
receive
and
process
mail
at
the
computer
he
uses
most
mainframes, each user
to
and process mail at the computer he uses most
of the time,
time, his primary
primary host.
host.
first inclination
inclination of
of many users
users when an
The first
of
independent workstation
workstation is piaced
placed in
in front
front of
of them is to
to begin receiving
receiving mail
mail at the
independent
workstation, and, in
in fact,
fact, many vendors have implemented
implemented facilities
facilities to
to do this.
this.
workstation,
However, this
this approach has
has several
several disrd\l:ntzges:
diS l1 av:mtages:
Workstations (especis!ly
(especiHlly Lisp
Lisp workstations)
workstations) have a software
software design that
that gives
.• Workstations
full
control
of
all
aspects
of
the
system
to
the
user
at
the
console.
As a
full control of all aspects of
system to
at
kept
from
result.
tasks,
like
mail,
background
tasks,
like
receiving
mail,
could
well
be
kepl
from
result,
running for
for long
long periods of
of time
time either because
because the user is asking to
to use
use all
all
running
of the
the machine’
machine'ss resources,
resources, or
or because,
because, in
in the
the course
course of
of working,
working, the
the user
user has
has
of
(perhaps accidentally)
accidentally) manipulated
manipulated the
the environment
environment in
in such
such aa way
way as
as to
to
(perhaps
This could lead to repeated
repeated failed
failed delivery
delivery attempts
pre-vent mail
mail reception. This
prevent
agents.
by outside agents.
failure of
of a single workstation
workstation could keep its user
user “off
"off the
.• The hardware failure
air" for
for a considerable time, since repair of
of individual
individual workstation
workstation units
air”
might be delayed.
delayed.
Given the growing number
number of
of workstations
workstations spread
spread
Given
might
throughout
office
environments,
quick
repair
would
not
be
assured,
whereas
throughout office
quick repair
not
assured, whereas
mainframe is generally repaired very soon after
after failure.
failure.
a centralized mainframe
difficult to keep
keep track of
of mailing
mailing addresses
addresses when each
each person
person is
•. It
It is more difficult
associated
difficulty in keeping track
associated with
with a distinct
distinct machine.
machine. Consider the difficulty
of
particularly ifif
of a large number of
of postal
postal addresses
addresses or phone numbers, particularly
there was
organization though
was no single address
address or phone number for
for an organization
which you could reach
Traditionally,
Traditionally,
reach any person
person in that organization.
organization.
electronic
of
electronic mail
mail on the ARPANET
ARPANET involved
involved remembering a name and one of
several
several "hosts"
“hosts” (machines) whose
whose name
name reflected the organization
organization in which
the individual
individual worked. This was
was suitable at a time when most organizations
organizations
had
only
one
central
"host."
It is less
less satisfactory today unless
unless the concept
had only
“host.” It
of
of a "host"
“host” is changed
changed to refer to an
an organizational
organizational entity
entity and not a
particular
particular machine.
•. It
It is very difficult
difficult to keep
keep aa multitude
multitude of heterogeneous
heterogeneous workstations
workstations
working
working properly
properly with complex mailing
mailing protocols, making it
it difficult
difficult to
move forward
forward as
as progress
progress is
is made
made in electronic communication
communication and as
as new
about
receiving
incoming
standards
emerge.
Each
system
has
to
worry
standards emerge. Each system has
incoming mail,
roming
rou[ing and
and delivering
delivering outgoing mail, formatting,
formatting, storing, and providing
providing for
for
the stability
stability of mailboxes over aa variety of possible
possible filing
filing and mailing
mailing
protocols.
Thus, we
we are
are investigating the alternative strategy
strategy of
of having aa mail
mail server
server machine
which handles
handles mail
mail transactions.
transactions. Because
Because this machine would be
be isolated
isolated from
from direct
direct
user
user manipulation,
manipulation, it
it could achieve
achieve high software reliability
reliability easily,
easily, and,
and, as
as aa shared
shared
resource,
resource, itit could
could achieve
achieve high
high hardware
hardware reliability,
reliability, perhaps
perhaps through redundancy.
redundancy. The
mail
mail server
server could
could be
be used
used from
from arbitrary
arbitrary locations,
locations, allowing
allowing users
users to read
read mail across
across
campus,
campus, town,
town, or
or country
country using
using more
more and
and more
more commonly
commonly available workstations.
The
The mail
mail server
server acts
acts as
as an
an interface
interface among
among users,
users, data
data storClge,
stornge, and
and other mailers.
mailers.
Users
Users employ
employ aa mail
mail access
access protocol
protocol (MAP)
(MAP) to
to retrieve
retrieve messages,
messages, access
access and
and change
change
properties
This protocol should
should be
be
properties of
of messages,
messages,manage
manage mailboxes,
mailboxes, and
and send
send mail.
mail. This
simple
simple enough
enough to
to implement
implement on
on relatively
relatively uncomplicated,
uncomplicated, inexpensive
inexpensive machines
machines so
so that
that
41
41
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
Details
Details of
of Technical
Technical Progress
Progress
mail
mail can
can be easily read remotely.
remotely, This
This is distinct
distinct from
from some
some previous approaches
approaches since
the mail
protocol is used
user from
mail access
access protocol
used for
for all message
message manipulations,
manipulations, isolating
isolating the user
from
all knowledge of
This means
means the the mail
mail server
server can
of how the data storage
storage is used.
used. This
utilize
utilize the data storage
storage in
in whatever way is most efficient
efficient to organize the mail.
mail. The
data storage
storage could be anything
anything from
from conventional
conventional magnetic disk file
file system
system to a highly
highly
specialized mail
mail filing
filing system
system built
built on optical
optical disks,
disks, since
since it
it is abstracted
abstracted from
from other
elements in
in the mail
mail system.
system. The other mailers constitute the mail
mail server's
server’s (and thus
the users')
use various mail
mail server
server would use
maif transport
transport
users’) link
link to the outside world.
world. The mail
protocols (e.g.,
protocols
(e.g., SMTP)
SMTP) to exchange
exchange mail
mail with
with other mail
mail hosts.
hosts.
We have
have been
been investigating
investigating user/mail
user/mail interface
interface issues
issues for
for workstations, as
as well as
as issues
issues
for
protocol itself.
for the mail
mail access
access protocol
itself. We are examining
examining several
several related projects, including
including
MIT's
public parts of
MIT’s PCMAIL
PCMAIL (Mark
(Mark Lambert,
Lambert, MIT
MIT Distributed
Distributed Systems
Systems Group),
Group), the public
of
Xerox's
NSMail, and work on Stanford's
Xerox’s Grapevine
Grapevine and NSMail,
Stanford’s V
V system.
system.
We have
have implemented
implemented an Interim
Interim Mail
Mail Access
Access Protocol (lMAP)
(IMAP) server
server on the 2060
2060 and
a client
client implementation
implementation in
in Interlisp
Interlisp on Xerox D-machines. The resulting beta-test mail
mail
environment
environment proved to be quite usable;
usable; some
some D-machine
D-machine users
users use
use it
it as
as an alternative
alternative
to the 2060 mail
mail environment
environment in
in their
their daily
daily mail
mail work.
The IMAP
user'ss incoming
IMAP server manipulates the actual file
file store copy of
of the user’
incoming
electronic
electronic mail
mail under direction
direction from
from the IMAP
IMAP client.
client. As noted above,
above, the client
client has
has
no knowledge of
of the (possibly operating system- dependent)
dependent) format
format of
of mail
mail on the
server's
protocol provides its own representation of
server’s file
file store; the IMAP
IMAP protocol
of mail
mail and the
server translates
translates between
between this and its host system
system file
file store conventions.
The IMAP
fetch commands
IMAP client
client issues
issues a series
series of
of fetch
commands to retrieve data from
from the server.
server. A
A
fetch command has
sequence and the name of
has two arguments:
arguments: a message sequence
of the data item
to be fetched. A
A message
message sequence
sequence can be a single message
messagenumber, a range
range of
of message
message
numbers, or a list
list of
of numbers or ranges.
ranges. For
For example,
example, a typical fetch command might
might
be "fetch
“fetch 2:7,10
2:7,10 flags",
flags”, meaning "fetch
“fetch the status
status flags for
for messages
messages2 through
through 7 and
message
message 10"
10” (status flags include
include "new
“new message",
message”, "deleted
“deleted message",
message”, "message
“message has
has been
been
read",
read”, etc. as
as well as
as user-defined
user-defined flags).
In IMAP,
IMAP, the actual message
message data is identified
identified by names
names such
such as
as "RFC822.Header"
“RFC822.Header” and
"RFC822.Body"
“RFC822.Body” referring
referring to the text-based mail representation used
used on the DoD
DOD
Internet
Internet standard (RFC
(RFC 822). This
This is intended to be a temporary solution
solution only,
only, since
RFC
RFC 822
822 lacks structure
structure and the capability
capability to deal
deal with
with non-text
non-text mail.
mail. We plan on
extensions to IMAP
IMAP (lMAP
(IMAP II,
II, see
see below) that will
will introduce a canonical and structured
representation of
of an electronic
electronic mail
mail message.
message. In such
such a structured form,
form, an electronic
electronic
mail
mail message
messagewould consist of
of a set of
of named
named properties and property values.
values.
During
protocol had
During implementation
implementation of
of the user
user interface
interface we
we observed
observed that the IMAP
IMAP protocol
several
several deficiencies
deficiencies which
which made
made certain mail
mail concepts
concepts difficult
difficult or impossible to
implement.
For example, there is no way in
in IMAP
IMAP to notify
notify the client
client of
of newly
implement. For
arrived
arrived mail
mail during
during an IMAP
IMAP session.
session. Other IMAP
IMAP deficiencies were
were observed
observed in
in the
design
design of
of a Common
Common Lisp
Lisp implementation
implementation for
for Texas
Texas Instruments Explorers;
Explorers; in
particular,
protocol with
particular, IMAP
IMAP is a "lock-step"
“lock-step” protocol
with no mechanism
mechanism for
for multiplexed
multiplexed
operation.
operation. This
This means
means that
that IMAP
IMAP is vulnerable to synchronization
synchronization problems in
in which a
client
client interprets
interprets part
part of
of a previous response
response as
as the answer
answer to the current
current query.
To
To address
address these
these concerns,
concerns, a new Interim
Interim Mail
Mail Access
Access Protocol (lMAP
(IMAP II)
II) was
was designed
designed
after
after extensive review. IMAP
IMAP II
II is heavily influenced
influenced by IMAP,
IMAP, although with
with a greater
degree
degree of
of formality
formality in
in the specification
specification and quite a bit
bit more extensibility.
extensibility. Instead of
of
the lock-step
lock-step query/response model of
of IMAP,
IMAP, IMAp·II
IMAP‘II uses
uses tagged
tagged commands and data
and explicitly
explicitly allows unsolicited
unsolicited data to be sent from
from the server
server to the client.
client. IMAP
IMAP II
II
introduces a more formal
formal structure to server-to-client
server-to-client path; all data
data is now identified
identified
unambiguously. This
This is especially important
important for
for extensibility
extensibility and unsolicited
unsolicited data.
data.
E. H. Shortliffe
Shortliffe
42
Details
Details of
of Technical Progress
Progress .
5P41-RR00785-14
5P41-RR00785-14
In addition,
addition, IMAP
IMAP II
II makes
makes it
it possible to fetch more than one item of
of data at a time.
This
This is an important
important performance
performance issue
issue since often
often the client
client needs
needs to fetch a set
set of
of
data items for
IMAP model of
of fetching
fetching a single data item at a
for a set of
of messages.
messages. The IMAP
time
time resulted in the client
client having to make several
several consecutive
consecutive requests
requests with
with much longer
turnaround
turnaround than a single request
request that specifies everything the client
client wants.
wants.
A
A large subset
subset of
of IMAP
IMAP II
II has
has been
been implemented
implemented on the 2060
2060 by modifying
modifying the existing
existing
IMAP
IMAP implementation.
implementation. Both the 2060
2060 implementation
implementation and the specification
specification have
have been
been
left
progress to
left open-ended to allow
allow for
for extensions as
as the need
need arises.
arises. Work
Work is now in progress
Since the interface
interface is no longer
modify
modify the Interlisp
Interlisp user
user interface
interface to use
use IMAP
IMAP II. Since
limited
limited to the model of
of IMAP
IMAP a general
general restructuring
restructuring of
of the Interlisp
Interlisp client
client is being
done to take advantage
advantage of
of the new facilities
facilities offered
offered by IMAP
IMAP II. A
A Common Lisp
implementation,
implementation, based
based on IMAP
IMAP II,
II, is also
also in progress.
progress.
5 - Text
Text Editing
Editing
All
All workstation
workstation systems
systems have
have text editing
editing facilities,
facilities, some
some adaptations of
of systems
systems in use
use
on mainframes
EMACS-like editors) and some
some specialized
specialized What-You-See-IsWhat-You-See-Ismainframes (e.g.,
(e.g., EMACS-like
What-You-Get
What-You-Get (WYSIWYG)
(WYSIWYG) editors (e.g.,
(e.g., TEdit
TEdit for
for Xerox workstations or InterLeaf
InterLeaf
for
for UNIX
UNIX workstations).
workstations). We are currently
currently making use
use of
of each
each workstation's
workstation’s facilities,
facilities,
making
making extensions where needed
needed to bring
bring compatibility
compatibility among the various workstations
(in
(in both
both user
user interface
interface and document format)
format) without
without detracting from
from the powerful,
powerful, but
but
idiosyncratic,
idiosyncratic, features.
features. Text
Text formatting,
formatting, to produce printable
printable or displayable forms
forms of
of
documents, is another area
is expended.
area where
where considerable vendor effort
effort
expended.
Implementations
Implementations of
of SCRIBE or TEX
TEX systems
systems are
are available for
for some
some workstations
directly.
directly. Also, since these
these formatting
formatting processes
processes are essentially batch operations, we
These can
can be fed a raw
expect to provide
provide servers
servers that offer
offer formatting
formatting services.
services. These
manuscript
printing
manuscript and will
will return
return a formatted
formatted version, suitable for
for one of
of the several
several printing
device standards
standards in use.
use. WYSIWYG
WYSIWYG editors are able to combine the editing
editing and
formatting
formatting processes
processes into
into the document preparation
preparation system.
system. We will
will concentrate on
PostScript and ImPress
printers, allowing
printers to fade from
Impress printers,
allowing Press
Press printers
from use.
use. The 2060
2060
also provides for
for printer
printer spooling, based
based on a first-come-first-serve
first-come-first-serve algorithm
algorithm with
with
priorities
priorities determined by submission time
time and estimated pages
pages of
of output. This
This spooling
Given adequate
adequate printing
resources, a
printing resources,
is not
not available among workstations
workstations currently.
currently. Given
laissez-faire
laissez-faire access
access policy
policy without
without spooling
spooling can
can work adequately.
adequately. If
If there is a problem,
an arbitration
arbitration scheme
scheme will
will need
need to be
be worked out, but this should be a relatively
relatively
straightforward
straightforward task.
task. Finally,
Finally, we will
will need
need to augment vendor products to provide
provide
essential
essential text processing
processing aids for
for functions
functions like
like spelling correction,
correction, document merging
and segmenting,
segmenting, and document analysis.
analysis.
Text Processing
5.1 - Text
Processing for
Jor Xerox
Xerox D-machines
D-machines
TEdit
TEdit is the text editing
editing and formatting
formatting package
package on the Xerox D-machines
D-machines (i.e.,
(i.e., the
llxx series)
series) and we have
have continued
continued our work
work to extend this environment
environment to displace
llxx
text processing
processing from
from the DEC
DEC 2060.
2060. Almost
Almost all efforts
efforts during
during the past
past year
year were
were
TMAX stands
stands for
for Tedit
Tedit Macros And
And
directed towards the Interlisp
Interlisp package
package TMAX.
TMAX. TMAX
extensions and it
it gives
gives TEdit
TEdit the ability
ability to do things that hitherto
hitherto could only
only be
be done
eXtensions
with Scribe. Scribe is a powerful
language, but
but it
it consumes
consumes far
far
with
powerful document preparation language,
too many mainframe
mainframe cycles.
cycles. Furthermore,
Furthermore, with
with Scribe you must hardcopy your output
output
see what it
it looks like.
like. TEdit
TEdit is a WYSIWYG
WYSIWYG text editing
editing and formatting
formatting system,
system,
to see
which means
means that you can
can see
see what your output
output will
will look
look like
like while
while you are creating it.
TMAX
TMAX makes
makes no attempt
attempt to mimic
mimic Scribe in TEdit.
TEdit. This
This would be a Herculean task
task
Instead TMAX
TMAX implements some
some of
of the more
given the power and flexibility
flexibility of
of Scribe.
Scribe. Instead
commonly
commonly used
used features of
of Scribe, including
including indexing, numbering, end notes,
notes, and
forward/backward referencing. All
All of
of these
these features
features are implemented through menus.
menus.
forward/backward
43
43
E. H. Shortliffe
Shortliffe
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR00785-14
For example, to
to include
include an index
index request
request in
in a document,
document, the
the user
user simply
simply buttons
buttons the
the
For
command (with
(with the
the mouse)
mouse) and
and then
then types
types the
the text
text to
to be
be indexed.
indexed. TMAX
TMAX takes
takes
Index command
Index
of all
all the rest
rest (e.g.,
(e.g., associating the page
page number
number with
with the
the indexed text,
text, creating
creating a
care of
of the indices, etc.). These TMAX
TMAX features plus the
the editing
editing and formatting
formatting
list of
sorted list
TEdit make the TMAX/TEdit
TMAX/TEdit package
package an attractive
attractive
available in
in TEdit
features already available
alternative to
to Scribe.
alternative
The following
following is’
is'aa quick
quick overview
overview of
of the major
major TMAX
TMAX features:
The
-- users
users can insert
insert both
both simple
simple and extended index requests,
requests, create
Indexing -.• Indexing
a sorted file
file of
of the indices
indices and their
their page
page numbers, and even specify
specify that
that
printed in
in manual format
format (e.g.
(e.g. 111:25.7
111:25.7 for
for chapter 3,
page numbers be printed
the page
page 7). A
A simple
simple index
index is just
just the text
text to index. An
An extended
section 25, page
font to print
print itit in,
in, and
index takes the text
text to
t9 sort
sort on, the text
text to print,
print, the font
index
a page
page number
number option.
option. This
This option
option allows the user to
to specify
specify the normal
normal
page
number
in
the
index
file,
no
page
number,
or
a
user
specified
fixed
page number in
index file, no page
or
user
fixed
command that
that pops up a menu of
of the simple
simple
page number. There is also a command
page
specified so far
far and users
users can insert
insert additional
additional index
and extended indices specified
requests by simply
simply buttoning
buttoning the corresponding
corresponding item
item in
in this
this menu.
requests
Numbering --- users
users specify
specify the names
names and order
order of
of “number
"number markers”
markers" and
.• Numbering
insert these
these markers wherever they want something numbered. Users
Users
then insert
as many different
different number
number markers as
as they like
like and some
some can be
can create as
layered' (i.e. chapter, section, etc.) while
while others are disjoint.
disjoint. When a marker
layered’
or deleted,
deleted, TMAX
TMAX automatically
automatically adjusts
adjusts all the related numbers.
is inserted or
font and format
format of
of each
each number. The format
format
Users can even specify
specify the font
Users
defines how the number
will
be
displayed
(i.e.
an
Arabic
or
Roman
numeral
number will
Arabic
or a letter),
letter), the delimiter
delimiter following
following the number, and the starting
starting value.
value.
There is also a facility
-contents file.
facility to create
create a standard table-of
table-of-contents
file.
•. End Notes -just like
-- these
these are just
like footnotes
footnotes except end notes are
are
the end of
bottom of
page. A
of the text
text rather than the bottom
of the page.
A future
future
TMAX
TMAX will
will support
support footnotes. When an end note is inserted
TMAX
TMAX automatically
automatically adjusts
adjusts the other end note numbers.
numbers.
inserted at
version of
of
or deleted,
deleted,
•. References
References --- users
users can
can refer
refer to specific
specific numbering
numbering markers or end note
numbers by their
their numeric
numeric value. It
It does
does not matter if
if the number is before
or after
after the reference to it. Also, should a number change
change because
because a
number marker or end note was
was deleted
deleted or inserted,
inserted, the reference
reference to that
number will
will be
be automatically
automatically updated
updated (as
(as well as
as the number itself).
itself).
There are
are many more features
features and options
options in TMAX
TMAX and still
still more in the planning
stages.
stages. For example, one can
can edit the text of
of an end note by pointing
pointing the
the mouse
mouse to the
the
end
end note number and
and pressing
pressing the middle
middle button. Another
Another TEdit
TEdit window will
will appear
appear
containing
Some
containing the end note text.
Some of
of the features
features planned are
are footnotes,
bibliographies,
bibliographies, and
and appendices.
appendices. The TMAX
TMAX User's
User’s Guide describes
describes all the
the features
features of
this package.
package.
5.2
5.2 - Remote Editing
Editing
Currently,
Currently, the
the mainframe
mainframe editor
editor of choice among
among our users
users isis EMACS. EMACS, like
Scribe,
Scribe, is very powerful
powerful but it also
also places
places aa heavy
heavy load
load on our mainframe.
mainframe. In an
an effort
effort
to reduce
reduce the
the mainframe
mainframe load (and
(and ease
ease users
users into
into using
using TEdit),
TEdit), we
we have
have written
WEDIT
WEDIT
WEDIT (Workstation
(Workstation EDITor).
EDITor).
WEDIT provides aa convenient way
way for mainframe
EMACS
EMACS users
users to
to edit
edit their
their files
files on
on aa Xerox
Xerox D-machine
D-machine using
using TEdit.
TEdit. Note
Note that
that WEDIT
WEDIT
itself
itself isis not
not an
an editor.
editor. Itit simply
simply opens
opens aa connection
connection to
to the
the workstation
workstation and
and sends
sends aa
E.
E. H.
H. Shortliffe
Shortliffe
44
of Technical
Technical Progress
Progress
Details of
5P41-RR00785-14
5P41-RR00785-14
containing the name of
of the file
file to edit. The workstation
workstation does
does all the rest.
rest.
packet containing
When the user
user is done editing,
editing, the workstation
workstation sends
sends the updated
updated file
file back to the
mainframe. From
From the mainframe's
mainframe’s point
point of
of view WEDIT
WEDIT is an editor
editor in the sense
sense that
mainframe.
Because of
of this, it
it is easy
easy to install
install as
as the
given a file,
file, it
it returns an updated
updated version. Because
default
default mainframe
mainframe editor.
editor. A
A simple
simple change
change in the user's
user’s login command file
file is all that
is necessary.
necessary. From
From that point
point on, each
each time
time the user
user edits a file,
file, the editing
editing will
will be
EMACS users
users can experience
experience long
done by TEdit
TEdit on the user's
user’s personal workstation.
workstation. EMACS
delays
delays when the 2060 is heavily
heavily loaded.
loaded. With
With WEDIT
WEDIT (i.e. TEdit)
TEdit) there are no delays
delays
editing is done on the user's
user’s personal
personal workstation.
workstation.
since the editing
5.3 - Special
Document Types
Special Document
In
In last year's
year’s report,
report, some
some TEdit
TEdit extensions to facilitate
facilitate simple document types
types like
like
memos were mentioned.
These extensions proved to be
be very useful although this
mentioned. These
concepts developed
developed in this package,
package, we have
have
package
package was
was only
only a prototype.
prototype. Using the concepts
package allows users
users to
written
Letterhead. The Letterhead package
written a new TEdit
TEdit package
package called Letterhead.
All
create standard letterheads,
letterheads, for
for example, for
for Stanford
Stanford University
University correspondence.
correspondence. All
create
user starts the
the options
options in this Letterhead package
package are menu driven.
driven. When a user
package, a TEdit
TEdit window
window appears
appears on his workstation
workstation and the user
user is
Letterhead package,
prompted
prompted for
for several
several different
different fields.
fields. First
First a menu of
of the possible Stanford
Stanford logos
logos pops
pops
up and the user
user must select
select one of
of these
these logos.
logos. The logo is placed
placed in the upper left
left
of the window. Next
Next a menu of
of the return addresses
addresses pops up. The user
user
hand corner of
may select
Next the Letterhead package
select one of
of the known addresses
addresses or create
create his own. Next
package
asks
justified. This
asks the user
user how the address
address should be justified.
This is done though a menu and the
possible ways
justified address
ways are left,
left, right,
right, or centered.
centered. The justified
address is then inserted in the
upper right
right hand corner
corner of
of the window. Finally,
Finally, the current
current date is automatically
automatically
inserted just
just below the logo. Now
Now the letterhead is complete and TEdit
TEdit is ready
ready to
accept input
input from
from the user.
user. The user
accept
user can change
change either the logo, address,
address, or date by
pointing
pointing the mouse
mouse at the appropriate
appropriate field
field and pressing
pressing the middle
middle button.
button. If
If the logo
is buttoned, the logo menu will
will pop up and the user
user can
can select
select a different
different logo. If
If the
address
address is buttoned, the return
return address
address menu will
will pop up and the user
user can
can either select
select
a known address,
address, create
create his own address,
address, or edit the address
address already in the document.
If the date is buttoned,
buttoned, a date menu pops up allowing
allowing the user
user to display the date in
If
of several
several different
different formats.
formats. When this window
window is hardcopied, it
it will
will look
look just
like
one of
just like
a standard Stanford
Stanford University
University letter
letter
6 - System
System Building
Building Tools
Traditionally,
Traditionally, a large set of
of languages
languages and programming
programming environments
environments have
have been
been
supported on the 2060 in order to encourage
encourage experimentation
experimentation and development. We
now believe that the experience gained in those
those years
years of
of broad experimentation
experimentation can
can be
distilled
distilled into
into a fairly
fairly small set
set of
of languages
languages and tools, relieving
relieving the researcher
researcher of
of the
need
need to learn many programming
programming languages,
languages, while
while still
still providing
providing the needed
needed facilities
facilities to
allow the experimentation
experimentation to move further
further into
into the higher levels of
of knowledge
knowledge
allow
systems and problem
problem solving
solving architectures.
representation systems
As we move to the
workstation
workstation based
based environment,
environment, we plan to phase
phase out support for
for many of
of the languages
languages
we have
have offered
offered in the past and concentrate on the most relevant languages
languages for
for AI
AI
research
research and applications:
applications: C, FORTRAN,
FORTRAN, InterLisp-D,
InterLisp-D, ZetaLisp, and Common
Common Lisp.
Common Lisp
Lisp has
has already achieved
achieved popularity
popularity as
as a standard (see
(see page
page 54), and many
Common
projects are already using it. We expect to press
press for
for further
further adoption
adoption of
of Common Lisp
as a community
community symbolic
symbolic computing
computing standard, consistent with
with prior
prior investments in large
as
software systeUls
systems such
such as
as those
those which exist for
for on-going
on-going AIM
AIM projects. In addition,
addition, we
will support
support important
important higher-level
higher-level knowledge representation and problem solving
will
architectures (e.g.,
(e.g., S.l,
S.1, KEE,
KEE, Strobe, and others) as
as appropriate
appropriate for
for community
community research
research
dissemination activities.
activities.
and dissemination
45
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical Progress
Progress
Distributed Information
Information Resources
Resources and
and Access
Access
7 - Distributed
user needs
needs for
for getting
getting information
information from
from and about the computing
computing
There are many user
environment, ranging from
from help with
with command syntax to sophisticated database
database queries.
queries.
environment,
A
A distributed
distributed computing
computing environment
environment adds
adds new complexities
complexities in making
making such
such
information
information accessible
accessible and also new requirements for
for information
information about the distributed
distributed
environment
environment itself.
itself. We are adapting the many workstation-specific
workstation-specific information
information tools
to include distributed
distributed environment
environment information
information such
such as
as workstation
workstation and server
availability,
availability, "finger"
“finger” information
information about user
user locations and system
system loads,
loads, network
network
connectivity,
connectivity, and other information
information of
of interest to users
users in designing approaches
approaches to
addition, we will
will have
have to develop general systems
systems
carrying
carrying our their
their research
research tasks.
tasks. In addition,
tools for
performance to identify
for monitoring
monitoring and debugging
debugging distributed
distributed system
system performance
identify
Finally, we must adapt and develop distributed
distributed
workstation
workstation and network
network problems. Finally,
system
system tools for
for remote database
database queries
queries and organize the diverse sources
sources of
of
information
information of
of interest to AIM
AIM community
community members to facilitate
facilitate remote workstation
workstation
access
access to community,
community, project, and personal information
information that
that has
has traditionally
traditionally existed
existed in
ad
ad hoc files
files on mainframe
mainframe systems.
systems.
In conjunction
conjunction with
with the SUN file
file server we have
have been
been integrating,
integrating, we have mounted an
database system
system for
for remote information
information access
access using the commercial
commercial
experimental database
Our goal is to make access
access to the database
database information
information
UNIFY
UNIFY database
database product. Our
possible
distributed workstation
workstation environment
environment through
through network
network query
possible from
from a distributed
transactions, as
as opposed
opposed to asking the user
user to log into
into the database
database system
system as
as a separate
separate
This will
will facilitate
facilitate remote information
information access
access from
from
job and type in queries
job
queries directly.
directly. This
including expert systems,
systems, where
where the information
information can be filtered,
filtered,
within
programs, including
within programs,
system will
will provide
provide
integrated with
with other information,
information, and presented
presented to the user.
user. The system
multi-user,
multi-user, multi-database
multi-database access
access capability;
capability; that is, several
several users
users will
will be able to have
have
access
access to a single database
database at the same
same time, and a single user
user will
will be able to have
have
access
accessto several
several databases
databases at the same
same time.
The initial
initial implementation
implementation of
of the remote query system
system was
was done on a TI
TI Explorer.
Explorer.
The query interface
interface on the Explorer
Explorer communicates with
with the Sun
Sun UNIFY
UNIFY database
database
system via the Remote Procedure
Procedure Call
Call (RPC)
(RPC) mechanism which underlays the NFS
system
file access
access system.
system. The Explorer
Explorer calls a server on the SUN
SUN and sends
sends an
remote file
SQL/DML query command as
as an argument to a remote query procedure, and receives
receives
SQL/DML
SUN UNIFY
UNIFY can
can
the retrieved data and/or
and/or a message
message sent back from
from the server.
server. SUN
already manage
manage multiple
multiple databases,
databases, so a client
client can have several
several databases
databases open at the
same time. The operations on the database
database are transaction-oriented,
transaction-oriented, and therefore
therefore the
same
access functions
functions currently
currently
concept of
of a database
database access
access session
session is applicable. The access
implemented are open a database,
from a database,
database, close
close a database,
database, retrieve
retrieve data
data from
database,
from a database,
insert
insert records
records into
into a database,
database, delete
delete records
records from
database, update
update the database,
database,
unlock a database.
database.
lock a database,
database, and unlock
lock
This facility
facility can be easily converted to run in Lisp
Lisp environments
environments on machines with
with
This
SUN RPC services
package for
services implemented. Currently,
Currently, there is no RPC package
for the Xerox
Xerox DDmachines,
machines, so we undertook
undertook implementing
implementing one. This
This should be done by early summer.
8 - Distributed
Distributed system
system operation and
and management
management
The primary
primary requirements in this area
area are user
user accounting (including
(including authorization
authorization and
billing),
billing), data backup, resource
resource allocations
allocations (including
(including disk space,
space, console time, printing
printing
access, CPU time, etc.), and maintenance of
of community
community data bases
bases about users
users and
access,
projects.
NIH reporting
projects. Our accounting needs
needs are a function
function of
of NIH
reporting and cost recovery
requirements. The distributed
distributed environment
environment presents
presents additional
additional problems for
for tracking
tracking
reqnirements.
resource
resource usage
usage and will
will require developing protocols for
for recording
recording various kinds of
of
usage
usage in central data
data base
base logs
logs and programs for
for analyzing and extracting
extracting appropriate
appropriate
reports and billing
We are now involved
billing information.
information.
involved in analyzing the kinds of
of
Shortliffe
E. H. Shortliffe
46
of Technical
Technical Progress
Progress
Details of
5P41-RR00785-14
resource
resource usage
usage that can be reasonably_
reasonably. accounted
accounted for
for in a distributed
distributed environment
environment (e.g.,
(e.g.,
printing,
printing, file
file storage,
storage, network
network usage,
usage, console
console time, processor
processor usage,
usage, server access),
access), and
investigating what facilities
facilities vendors have
have provided
provided for
for keeping such
such accounts.
accounts. Data
backup
backup is, of
of course,
course, closely related to the filing
filing issue.
issue. We continue
continue to use
use and
improve
improve network based
based file
file backup for
for many of
of our file
file servers.
servers.
Mainframe and
and Workstation
Workstation System
System Environments
9 - Mainframe
The various parts of
of the SUMEX-AIM
SUMEX-AIM computing
computing environment
environment require development and
support of
of the operating
operating systems
systems that provide the interface
interface between
between user
user software and
the raw computing
computing capacity. This
This includes the mainframe
mainframe systems
systems and the workstation
workstation
systems.
Following are some highlights
highlights of
of recent system
system software environment
environment
systems. Following
developments.
developments.
9.1 - TOPS-20
TOPS-20
Our long-term
long-term plan to phase
phase out
out the 2060
2060 mainframe
mainframe system
system has
has continued as
as
scheduled.
scheduled. Development
Development efforts
efforts on the 2060
2060 have ceased,
ceased, except where
where needed
needed to keep
keep
This involves
involves
This
the machine operational
operational in the evolving
evolving distributed
distributed environment.
environment.
periodic
considerable
considerable work
work in areas
areas such
such as
as file
file system
system archiving,
archiving, retrieval,
retrieval, and backups;
backups; periodic
updating, checkout,
checkout, and installation
installation of
of new versions of
of system
system software; the regular
maintenance and updating of
of system
system host and network
network tables;
tables; and monitoring
monitoring of
of and
year, the
past year,
recovery from
from system
system failures,
failures, both
both hardware and software. Over the past
main areas
areas of
of activity
activity include:
-- The SUMEX
SUMEX 2060
2060 has
has experienced
experienced relatively
relatively
•. Network
Network service
-service reliability
reliability
frequent software crashes
crashes resulting
resulting from
from system
system problems in the handling
handling of
of
free space
ITCP network
space by the IP
IP/TCP
network software. During
During periods of
of heavy
heavy use,
use,
the entire
entire system
system would suffer
suffer an unscheduled
unscheduled restart approximately
approximately every
After a considerable amount
amount of
of investigation
investigation of
of crash
crash
eighteen
eighteen hours. After
dumps,
dumps, we isolated a cause.
cause. The problem
problem was
was introduced
introduced over a year
year before
in a modification
modification made by another site in an attempt
attempt to improve
improve network
network
After fixing
fixing this illusive
illusive bug,
bug, the 2060
2060 reliability
reliability has
has
After
performance.
improved markedly and the system
system regularly
regularly runs for
for over a week
week between
between
reloads.
reloads.
-- The Internet
Internet community
community is in the process
process of
of
•. Network
Network naming
naming domains
domains -converting to a domain naming scheme,
scheme, to replace
replace the flat
flat address
address space
space of
of
converting
the old exhaustive
Network Information
exhaustive host tables prepared
prepared by the Network
Information Center.
Although we have converted to using
using only
only fully
fully qualified
qualified names,
names, we
we are not
Although
yet running
running the domain
domain system
system on the 2060.
2060. This
This is due
due in part to the
20'ss at
unreliability
unreliability and incompleteness of
of the domain software for
for TOPSTOPS-20’
this point.
point. We expect to move to full
full domain support this coming
coming year.
year.
-- A
A significant
significant portion
portion of
of work on the 2060
2060 is
•. Dial-up
Dial-up communications
communications -During the past
past year we
we
carried on via dialup
dialup modems from
from homes.
homes. During
rearranged and consolidated our incoming
incoming modem lines. We combined
rearranged
several
several inside and outside phone number hunting
hunting sequences
sequences serving several
several
different
different modem types and speeds,
speeds, into
into well defined
defined groups for
for old-style
old-style
Vadic 1200
1200 modems,
modems, local versions of
of split
split speed
speed modems,
modems, and other
other types.
types.
This last group serves
serves any Bell/CCITT
BellKCITT modem :11
at any speed
speed from
from 300 baud
baud
to 2400
2400 baud.
baud. During
During this process
process we removed all the outside phone lines,
and now operate exclusively through Stanford-operated
Stanford-operated SL100
SLlOO lines. In
addition
addition to these
these mainframe
mainframe modems,
modems, we have
have installed
installed 10
10 modems
modems on an
Ethernet TIP, allowing
allowing users,
users, once dialed in, to connect to the host of
of their
their
choice.
choice.
4:
47
E. H. Shortliffe
Shortiiffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical Progress
Progress
Details
Center accounting
accounting --- During
During the past year, the 2060 accounting
accounting
.• Cost Center
to reflect
reflect the new Cost Center structure
structure (see
(see Section
programs were updated to
I1I.D.2). All
All the various users
users and projects were organized according to their
their
III.D.2).
monthly reports are generated
generated to reflect
reflect
cost center account numbers, and monthly
usage. As part
part of
of this conversion, a concerted effort
effort was
was made to
this usage.
of the SUMEX
SUM EX accounts,
accounts, and remove those that
that were otherwise no
review all of
longer appropriate.
9.2 - UNIX
UNIX
UNIX on our
our shared
shared VAX
VAX 11/750
111750 file
file servers.
servers. This
This system
system has
has been
been used
used
We run UNIX
pretty much as
as distributed
distributed by the University
University of
of California
California at Berkeley, except for
for local
pretty
as for
for ChaosNet
Chaos Net protocols. The local
local VAX
VAX user
network support modifications,
modifications, such as
network
not expended
expended much system
system effort
effort beyond staying
community is small, so we have not
community
current with
with operating system
system releases
releases and with
with useful UNIX
UNIX community
community developments.
current
9.3 - Xerox
Xerox D-Machines
D-Machines
of the SUMEX-AIM
SUMEX-AIM community
community continues to use
use InterLisp,
InterLisp, including
including many
Much of
Dandelion (1108).
(1108), Dandetiger (1109), and DayBreak (1186) machines,
machines, in addition
addition to the
Dandelion
Dorado (1132). We have
have used
used the Xerox implementation
implementation of.
of. the TCP network
network protocol
protocol
Dorado
(in cooperation with
with Xerox)
Xerox) extensively this past year and saw
saw its performance
performance and
(in
of Sun NFS
reliability improve
improve a great deal.
deal.
began a Lisp
Lisp implementation
implementation of
We began
reliability
(Network File
File System). The ARPA
ARPA protocol
protocol suite, which
which is seeing
seeing increasing usage,
usage,
(Network
The Sun
for random file
file access
access or
or attribute
attribute manipulation.
manipulation.
mechanism for
lacks a mechanism
specification
partially fills
specification partially
fills this void and appears
appears to be a standard whose
whose acceptance
acceptance is
growing.
The Interlisp
Interlisp software remained stable this year and almost no user
user time
time was
was wasted
wasted on
software revision problems. A
A number of
of new utilities
utilities were written
written locally
locally or acquired
from
from other sites
sites with
with whom we exchange
exchange expertise on the ARPA
ARPA Internet.
Internet.
We are among
among the first
first users
users of
of Xerox Common
Common Lisp
Lisp for
for the Xerox Lisp
Lisp machines.
The advantages
advantages to our community
community are early availability
availability of
of this widely-recognized
widely-recognized dialect
of
of the Lisp
Lisp language
language and the ability
ability to specially direct
direct the implementers'
implementers’ attention
attention to
the problems of
of greatest
greatest concern to us.
us.
The Info-ll00
Info-1100 discussion
discussion list
list which we
we sponsor
sponsor saw
saw another year
year of
of growth
growth of
of
readership
readership and participation
participation on the ARPA
ARPA Internet,
Internet, Usenet,
Usenet, Bitnet,
Bitnet, and CSNet. Among
Among
the beneficiaries are other NIH-sponsored
NIH-sponsored projects at Ohio
Ohio State
State University
University and the
University
University of
of Maryland.
In conjunction
conjunction with
with the Info-1100
Info-1100 mailing
mailing list, a library
library of
of user-written
user-written software is
-AIM 2060
made
made available to the Internet
Internet community
community on the SUMEX
SUMEX-AIM
2060 computer. Over
60 packages
Additionally, the source
source code
code to
packagesand supplements
supplements were
were distributed
distributed this way.
way. Additionally,
many of
of these
these packages
packages was
was mailed to the Info-ll00
Info-1100 mailing
mailing list
list in order to reach
reach an
even
even wider group.
We have
have worked closely with
with many other sites,
sites, including
including the Center for
for Study of
of
Language
Language and
and Information
Information at Stanford, the
the Stanford Campus Networking
Networking group, Rutgers
Rutgers
University,
University, Ohio State
State University,
University, the
the University
University of
of Pittsburgh,
Pittsburgh, Cornell,
Cornell, Maryland,
Maryland, and
and
industrial
industrial research
research groups
groups such
such as
as Xerox Palo Alto
Alto Research
Research Center, SRI, Teknowledge,
Teknowledge,
IntelliCorp,
IntelliCorp, and
and Schlumberger-Doll
Schlumberger-Doll Research.
Research. We have
have been
been the maintainers
maintainers for
for the
the
international
international electronic mail network of
of users
users for
for research
research D-machines,
D-machines, which have
have
upwards
upwards of
of 300
300 readers,
readers, and
and the interchange
interchange of
of ideas
ideas and problems among this group
has
has been
been of
of great
great service
service to all users.
users.
E.
E. H. Shortliffe
Shortliffe
48
48
5P41-RR00785-14
5P41-RR00785-14
of Technical
Technical Progress
Progress
Details of
Details
Although numerous
numerous Xerox
Xerox Lisp
Lisp machine
machine sites are able to
to obtain
obtain software
software from
from
Although
SUMEX-AIM via
via anonymous FTP
FIP over
over the
the ARPA
ARPA network,
network, itit became
became increasingly
increasingly clear
SUMEX-AIM
of the community
community did
did not
not have
have such access
access even though
though there
there is
that a large part
part of
that
To experiment
experiment with
with distributing
distributing software
software to
to these
these sites,
sites, we
we
electronic mail
mail connectivity.
connectivity. To
electronic
put together a simple
simple ASCII
ASCII encoder for
for binary
binary files,
files, BMENCODE.
BMENCODE. This
This program
put
to mail
mail binary
binary files
files (TEdit
(TEdit editor
editor files
files and *COM
*COM files
files from
from the
makes itit possible to
inherent ability
ability to
to encode bitmaps
bitmaps into
into
compiler) to
to isolated sites, exploiting
exploiting Interlisp’
Interlisp's
compiler)
s inherent
ASCII files.
files. Numerous files
files were successfully transferred
transferred around using this
this program.
program.
ASCII
of such a utility,
utility, more
more efficient
efficient
As the user community
community has begun to see
see the value of
As
of the program have been
been developed
developed elsewhere.
elsewhere.
versions of
our XNS
XNS boot
boot service (which
(which provides installation
installation and diagnostic
diagnostic programs
In extending our
In
for our
our workstations)
workstations) to
to work
work with
with the new 1186
ll86 Hardware, we ran into
i,nto trouble
trouble· as
as the
for
ll86's
hastily
written
initial
Ethernet
microcode.
booting
sequence
violated
The
booting
sequence
violated
1186’s hastily written initial Ethernet
Ethernet layering
layering principles
principles which
which prevented itit from
from routing
routing beyond the local
local network.
network.
Ethernet
After nearly a year of
of exchanging letters, packet traces
traces and software with
with Xerox,
Xerox, the
After
problem is still
still unresolved.
unresolved. This
This led to our adding a second
second Xerox
Xerox 8000-based
8000-based XNS
XNS
problem
boot
server
(using
a
spare
ll08
processor)
to
our
other
major
network
with
ll86
boot
1108 processor)
our other major network with 1186
This additional
additional server provided a suitable work-around
work-around to the problem
problem and
hardware. This
boot services.
services.
only a single workstation
workstation is still
still unable to access
access network
network boot
only
de-installation and
building this past year involved
involved the de-installation
Our move to a new building
reinstallation of
of nearly thirty
thirty workstations plus
pi us several
several printers
printers and other
other servers.
servers. In
In
reinstallation
of the move, diagnostics were
were run on all of
of the Xerox University
University Grant
Grant
anticipation of
anticipation
ll08s in
in order to get any existing
existing problems fixed
fixed under warranty. The diagnostics were
1108s
run again after
after the machines were installed in the new facility.
facility. All
All the equipment
equipment was
was
successfully
successfully relocated without
without major incident.
incident.
Instruments Explorers
Explorers
9.4 - Texas Instruments
The twenty Texas
popularity as
Texas Instruments
Instruments Explorers have
have enjoyed
enjoyed an increasing popularity
as more
projects have
have developed
developed a need
need for
for the combination
combination of
of execution speed,
speed, full
full Common
Common
Lisp, and sophisticated development facilities
facilities offered
offered by the Explorer.
Explorer. Explorers have
have
come into
into use
use in other parts of
of the national biomedical community
community as
as well, such
such as
as
Ohio
Ohio State
State University
University and the University
University of
of Maryland. However, the Explorer
Explorer is still
still
maturing
maturing as
as an AI
AI workstation.
workstation. Thus, our efforts
efforts have
have been
been directed at improving
improving the
environment
environment of
of the Explorer
Explorer by developing software, organizing
organizing user
user interest activities,
activities,
and advising Texas
Texas Instruments.
Previous experience
experience has
has shown
shown that the greatest
greatest source
source of
of advancement
advancement for
for a particular
particular
computing
computing environment
environment is the
the user
user community.
community. They are
are the most in touch with
with the
deficiencies of
of the system,
system, and
and thus
thus uniquely positioned to address
address them, as
as well as
as to
utilize
utilize the
the strengths
strengths of
of the system.
system. The product developers
developers of
of the system
system are
are frequently
frequently
too involved in the lower levels
levels of
of detail to produce
produce general,
general, effective
effective solutions to
problems, as
as well as
as being hampered
hampered by limited
limited manpower resources.
resources. However, aa
significant
significant amount of
of time and
and effort
effort is required to organize
organize this effort.
effort. This task
task has
has
traditionally
traditionally fallen
fallen to aa user-run organization, such
such as
as DECUS or Usenix.
We are
are spearheading
spearheading the
the effort
effort to organize
organize aa national or international
international users'
users’group for
for
the
the Explorer. The goals
goals of this undertaking are
are to:
to:
•. facilitate
facilitate dissemination of information
information by orgamzmg
organizing meetings
meetings where
where
presentations
presentations and
and discussions
discussions can
can be
be used
used to
to make
make little-known
little-known techniques
techniques
and
and facilities
facilities more widely known, as
as well as
as feeding back information
information on
needs
needs and
and wants
wants to developers,
developers,
•. allow
allow more
more immediate
immediate communication
communication via
via electronic
electronic mailing
mailing lists,
lists, which
which are
are
49
49
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
used for
for distribution
distribution of
of important
important software
software fixes
fixes and discussion of
of items of
of
used
software tools, or
or proposed
proposed changes
changes to
to the
general interest, such as new software
system,
system,
publish a periodic
periodic newsletter containing
containing usage
usage tips, salient extracts from
from the
.• publish
electronic mailing
mailing lists, and announcements,
announcements,
electronic
importantly, establish and maintain
maintain a library
library of
of public
public
.• and, perhaps most importantly,
domain, user
user supplied software.
domain,
was held at AAAI
AAAI ‘'86,
second meeting is being planned
preliminary meeting was
AA preliminary
86, and a second
for AAAI
AAAI ‘'87.
of those who have expressed
expressed interest in
in th?
the users’
users' group are
for
87. Over 80% of
members of
of the Info-TI-Explorer
Info-TI-Explorer and Bug-TI-Explorer
Bug-TI-Explorer mailing
mailing" lists, currently
currently
maintained on the SUMEX-AIM
SUMEX-AIM 2060.
2060. Negotiations
Negotiations with
with Texas Instruments
Instruments over the
maintained
The format
ramifications of
of the user
user library
library are in
in the final
final stages.
stages.
format and
legal ramifications
of the library
library have been
been mapped out,
out. and are currently
currently undergoing peer
procedures of
procedures
Online copies of
of the library
library will
will be maintained
maintained at Texas
Texas Instruments, and on
review. Online
2060, to facilitate
facilitate ARPANET
ARPANET access
access to the software.
the SUMEX-AIM
SUMEX-AIM 2060,
for the library,
library, most of
of which have
have been
been
There are already many entries ready for
maintained the software tools that were produced previously
previously
locally. We have maintained
developed locally.
porting to new releases.
releases. Some
Some of
of these
these have
fixing bugs,
bugs, making
making improvements, and porting
by fixing
same, including:
including:
remained essentially the same,
Explorer compatibility
compatibility package
package
.• The Symbolics 36xx to Explorer
Controller (was
(was known as
as the System
System Manager)
.• The Source
Source Code Controller
Net Imagen)
•. Imagen Via
Via TCP (was
(was Net
•. Finger
Finger Via
Via TCP (was
(was TCP Finger)
Finger)
•. Vertically
Vertically Ordered Menu Columns
•. General Named Structure Message
MessageHandler
Handler
DEFSTRUCT
•. DEFSTR
UCT Type Checking
Processor
•. Batch Processor
•. Choice Facility
Facility Enhancements
Enhancements (was
(was Choose
Choose Variable Values
Values Macros)
File System
System (was
(was FS To FS Backup)
Backup)
•. Backup To File
Many
Many of
of the tools have
have been
been enhanced
enhanced or newly written
written this year,
year, including:
including:
A number of
of pieces
pieces that allow the user
user to exploit
exploit a "desk
“desk top"
top” usage
usage
•. A
metaphor, where
where several
several applications
applications can
can be active, or semi-active at once,
once,
of each
each application
application potentially
potentially
with
with the interaction
interaction area,
area, or "window"
“window” of
overlapping
overlapping others. These
These pieces
pieces include:
provides a replacement to
0 WINDOW-MANAGER-SYSTEM-MENU
WINDOW-MANAGER-SYSTEM-MENU
the standard system
system menu that allows for
for easy
easy manipulation
manipulation of
of the
placement and shape
shape of
of windows on the screen,
screen, as
as well as
as other
common display management
management operations.
o
precise
which
allows
easy,
o RUBBER-BAND-RECTANGLES
RUBBER-BAND-RECTANGLES
easy, precise
specification
specification of a rectangle
rectangle on the screen
screen by providing
providing a constantly
o
E. H. Shortliffe
Shortliffe
50
Details of
of Technical
Technical Progress
Progress
Details
5P41- RR00785-14
5P41-RR00785-14
of the
the rectangle
rectangle (like
(like a rubber
rubber band
band attached to
to
updated “ghost”
"ghost" image of
updated
well as the
the ability
ability to
to change corners, and specify
specify a
mouse), as well
the mouse),
minimum size. Previously,
Previously, this
this was
was done
done by
by placing
placing only
only the
the upper
upper
minimum
left
and
lower
right
corners,
with
no
ghost
box,
and
only
a
beep
to
left and lower right corners, with no ghost box, and only a beep to
was too
too small.
indicate that
that the box
box was
indicate
BACKGROUNDS providing
providing a “curtain”
"curtain" between windows representing
oo BACKGROUNDS
active applications,
applications, and temporarily
temporarily inactive
inactive ones.
ones. In
In the desk-top
desk-top
active
metaphor, this
this adds a drawer to
to the desk.
desk. AA menu of
of background
metaphor,
operations, as well as eye-pleasing images,
images, are also provided.
provided.
operations,
DEEXPOSED-MOUSE which
which allows windows to handle mouse
mouse clicks
clicks
0o DEEXPOSED-MOUSE
documentation even when they are not
not completely
completely exposed.
exposed.
and documentation
SNAPSHOT-WINDOWS which
which allow
allow the user to copy a portion
portion of
of the
0o SNAPSHOT-WINDOWS
screen, thus saving itit for
for later use.
use.
screen,
TRANSPARENT-WINDOW which
which allows the image under a window
window
0o TRANSPARENT-WINDOW
of non-rectangular
non-rectangular windows.
to bleed through, providing
providing the illusion
illusion of
An on screen
screen round
round analog clock, with
with sweep
sweep second
second hand.
.• An
Development tool
tool consistency enhancements,
enhancements, including:
including:
.• Development
in the debugger,
debugger, data structure inspectors (regular and
oo commands in
flavor), editor
editor so that they can call each
each other
flavor),
not have
0 commands in tools that do not
have them to call the Lisp evaluator,
obtain
obtain argument lists, obtain
obtain macro expansions,
expansions, call the compiler,
compiler, trace
function
function invocation,
invocation, and obtain
obtain programmer supplied documentation.
o
buffer reading status
0 showing editor
editor buffer
status in a fashion similar
similar to file
file
reading status
status at the bottom of
of the screen
screen
o
0 the ability
ability to call the debugger
debugger on stack
stack groups'
groups‘from
from many different
different
contexts
o
o the ability
ability to modify
modify entries in inspect panes
panes in applications
applications other
than the Inspector
o
•. Facilities
Facilities to display a graph of
of time-varying
time-varying quantItIes.
quantities. This facility
facility isis
useful for
for monitoring
monitoring system
system performance parameters,
parameters, such
such as
as the
the number of
of
network
network packets
packets sent or received,
received, the number of
of disk operations per second,
second,
or the amount storage
storage allocated.
allocated.
•. A Screen
Screen Saver
Saver that shuts
shuts the
the display video off
off after about twenty minutes
of
of keyboard idleness
idleness to reduce
reduce display phosphor deterioration.
deterioration.
•. A version of the terminal
terminal emulator program that does
does not take
take up the
the entire
screen,
and
can
have
user
configurable
fonts.
screen, and can have user configurable
•. A facility
facility for
for attaching functions
functions to arbitrary
arbitrary keyboard
keyboard keys,
keys, most
commonly
commonly used
used to cause
cause aa particular
particular instantiation
instantiation of an
an application to be
be
selected
selected when
when aa key
key isis pressed,
pressed,allowing rapid movement among
among applications.
•. A number
number of
of editor
editor commands,
commands, including
including Tags
Tags Compile Macro Calls,
Calls, Macro
Expand
Evpand Into
Into Buffer,
Buffer, Rotate
Rotate Buffer,
Buffer, Rotate
Rotate Buffer
Buffer Backwards,
Backwards, ,\dd
.4dd File
File To
To
T:g
Tug Table,
Table, Remove
Remove File
File From
From Tag
Tag Table,
Table, and
and Evaluate
EvaIuate And
And Insert
Insert Into
Into
Buffer.
Buffer.
51
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
of files,
files, applying
applying a function
function
Functions that
that allow
allow the user to map over a set of
.• Functions
to each file.
file. The
The set of
of files
files can be specified
specified very generally, and values
values
to
in various
various ways from
from the mapped function.
function.
accumulated in
flavor inspector
inspector allowing
allowing itit to function
function in
in many situations
situations
.• Extensions to the flavor
would previously
previously have failed.
failed.
where itit would
for displaying
displaying data organized hierarchically
hierarchically in
in trees,
trees, or
or as
as graphs,
graphs,
.• AA tools for
featuring
featuring
full cycle detection
detection and handling,
handling,
oo full
edges,
0o mouse sensitive nodes and edges,
dynamic editing
editing of
of the graph display,
oo dynamic
horizontal and vertical
vertical scrolling,
scrolling,
oo horizontal
"overview" mode to facilitate
facilitate moving
moving the view port
port around in a
0o an “overview”
large graph
Introductory documents which
which have been
been used
used as
as models by a number of
of
.• Introductory
sites.
sites.
Of course,
course, all of
of these
these will
will be provided
provided to the user’
user'ss library,
library, and many of
of them have
have
Of
been given to other
other sites, including
including Intellicorp,
Intellicorp, Berkeley, ISI,
lSI, University
University of
of
already been
Maryland, and Ohio
Ohio State.
State.
Maryland,
producing and maintaining
provide
In addition
addition to producing
maintaining these
these software tools, we attempt
attempt to provide
extensive testing and evaluation
evaluation of
of Explorer
Explorer hardware and software products in a
sophisticated university
work more
university research
research environment
environment in order that these
these products work
effectively
effectively when they are distributed
distributed to the national
national community.
community. This
This testing is critical
critical
to the development of
of the computing
computing environment
environment since the combination
combination of
of
concentrated in-house
product developers
in-house expertise and close
close links
links to the product
developers allows a
turnaround
problem fixes
unavailable in the broader scope.
turnaround on problem
fixes unavailable
scope.
This
participated in testing TI's
Network File
This year we
we have participated
TI’s implementation
implementation of
of the Network
File
System
protocol, Release
System protocol,
Release 3.0
3.0 of
of the Explorer
Explorer System
System Software, and Release
Release 2.0
2.0 of
of
TCP/IP.
besides uncovering the usual
bugs, has
TCP/IP. Our testing of
of NFS, besides
usual set
set of
of bugs,
has allowed us
us
to make suggestions
suggestions to TI
TI that have
have led to an order of
of magnitude increase
increase in the data
Similarly,
throughput
throughput of
of the implementation.
implementation.
Similarly, our experience with
with DARPA
DARPA Internet
Internet
protocols
protocols has
has allowed us
us to make many suggestions
suggestions for
for improving
improving the Release
Release 3.0
3.0
Namespace
Namespace System
System which
which TI
TI has
has claimed to be invaluable
invaluable in making the system
system
acceptable
acceptable to the many Arpanet
Arpanet users
users in the national
national community.
community.
We served
served as
as a test site for
for several
several hardware revisions, as
as well, and further
further plan to
perform
perform extensive testing of
of the Explorer
Explorer II
II VLSI-based machine when
when it
it becomes
becomes
available in late spring or early summer.
Third
Third party
party software is less
less utilized,
utilized, but we
we stay
stay abreast
abreast of
of the latest releases
releases of
of the
expert system
system shell KEE,
KEE, and will
will be evaluating the Scribe text formatting
formatting system
system on
the Explorer
Explorer in a matter of
of weeks.
weeks.
In addition
addition to specific
specific testing and
and evaluation, we
we are constantly
constantly finding,
finding, tracking,
fixing,
fixing, and reporting
reporting software bugs.
bugs. This year
year we
we submitted
submitted thirty-two
thirty-two new
new bug
bug reports
on Release
Release 2.1,
2.1, twenty-one
twenty-one of
of which had
had fixes included. All
All of
of these
these fixes have
have been
been
made
made available to the national
national community
community in a patch file.
file.
E.
E. H. Shortliffe
Shortliffe
52
52
of Technical Progress
Progress
Details of
5P41-RR00785-14
5P41-RR00785-14
There have
have been
been fifteen
fifteen formal
formal reports, with
with ten fixes on release
release.3 beta in the current
current
four
were forty-two
forty-two reports returned with
with the TI
TI representative
four weeks
weeks of
of testing. There were
who brought
brought the initial
initial software after
after the first
first week,
week, most of
of which have
have been
been fixed
fixed by
TI.
TI.
We have
have also
also worked extensively on the operational
operational issues
issues involved
involved in keeping the
Texas Instruments
Instruments had no hardware or
machines running
running and useful from
from day to day.
day. Texas
software maintenance plans for
for large university
university installations
installations in place,
place, so we worked
quite hard to engineer fair
fair and serviceable
serviceable plans for
for maintenance, resulting in the
current offerings
offerings from
from TI
TI for
for all university
university sites.
sites.
current
As well as
as working
working on these
these specific
specific problems, we have had many meetings
meetings with
with Texas
Texas
Instruments
Instruments representatives
representatives wherein we have attempted to present the needs
needs of
of the
national
national community"
community. for
for shortshort- and long-term
long-term AI
AI workstation
workstation products, covering issues
issues
including
programming
including the desirability
desirability of
of specialized
specialized hardware, address
address space,
space, programming
environment
environment versus
versus execution speed,
speed, and the ability
ability to utilize
utilize the AI
AI workstation's
workstation’s
for routine
routine tasks.
tasks.
power for
Of
Of course,
course, there is also a large number of
of day-to-day
day-to-day activities
activities needed
needed to keep
keep the
computing
including resource
resource management
management (e.g.,
(e.g., disk space
space
computing environment
environment pleasant,
pleasant, including
allocation,
allocation, printer
printer management),
management), assistance
assistance with
with file
file backup and magnetic tape usage,
have produced documents
documents targeted
targeted at
and introducing
introducing new users
users to the system.
system. We have
complete novice users,
users, users
users of
of InterLisp-D
InterLisp-D machines,
machines, and users
users of
of Symbolics machines
machines
These documents have
have been
been used
used as
as examples
examples at
in order to facilitate
facilitate user
user education. These
various places
places in the national
national community.
community.
For
For the coming
coming year we plan to continue
continue development and maintenance of
of the software
processing
tools, perhaps
perhaps adding tools such
such as
as a DARPA
DARPA Internet
Internet Domain
Domain Resolver,
Resolver, text processing
facilities
facilities such
such as
as TeX,
TeX, LaTeX,
LaTeX, and document previewing
previewing tools, as well as aiding the
growth of
of the users'
users’group.
9.5 - Symbolics
Symbolics
Symbolics
Symbolics
Our
Our work
work with
with Symbolics equipment
equipment has
has been
been slowed
slowed pending resolution
resolution of
of longlongstanding maintenance issues.
has been
been stated
stated previously, in order for
for workstations
issues. As has
to be competitive
competitive with
with time-shared
time-shared mainframe
mainframe computing
computing resources,
resources, they must not only
only
This goal
goal is
This
have a low purchase
purchase price, but
but must be cost-effective
cost-effective to maintain.
maintain.
normally
normally achieved
achieved due to the economies
economies of
of scale
scale associated
associated with
with having a large
large number
as well as
as amortizing
amortizing the cost of
of software
of
of identical
identical parts in an installation,
installation, as
reasonable agreements
agreements with
with all of
of
development over many machines.
machines. We have come to reasonable
costs of
of service,
service, the
the workstation
The high costs
workstation vendors except for
for Symbolics.
exceptionally
exceptionally high price of
of mail-in
mail-in board repair, and the lack of
of a reasonable
reasonable selfselfservice alternative
justify continued
alternative has
has left
left us
us unable to justify
continued support
support of
of these
these machines
machines
unless
unless a workable agreement
agreement can be reached.
reached. We have negotiated a tentative hardware
maintenance contract,
contract, involving
involving parts from
from Symbolics, and in-house labor, and are in
negotiations for
for software maintenance. If
If we can reach
reach consensus,
consensus, we
we will
will be able to
increase
increase support
support of
of Symbolics machines
machines once again.
While
While there have been
been no appreciable system
system development activities
activities with
with the Symbolics
machines,
been maintained
machines, they have
have been
maintained in good working
working order, with
with up-to-date
up-to-date software.
We have
have not, however,
however, moved from
from Release
Release 6.1
6.1 to Genera 7.0
7.0 as
as the user
user community
community
felt
felt that the disadvantages
disadvantages of
of the transition
transition overwhelmed the advantages,
advantages, since the new
new
software was
was quite incompatible
incompatible with
with existing
existing code,
code, was
was slower, and seemed
seemed to introduce
many new problems. We will
will re-evaluate Genera 7.1
7.1 when
when it
it is released.
released. KEE
KEE and
Fortran
Fortran have
have been
been kept current,
current, and patches
patches in bulletins
bulletins from
from Symbolics have
have been
been
applied.
53
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
SUN
9.6 - SUN
just now
now bringing
bringing up several SUN
SUN workstations
workstations configured
configured for
for Lisp
Lisp research
research
We are just
work. Several
Several SUMEX
SUMEX projects have been able to experiment
experiment with
with SUN
SUN workstations
workstations
work.
through collaborations
collaborations with
with other
other groups and the Lisp
Lisp programming
programming and debugging
through
environments of
of these
these machines is still
still rather
rather primitive
primitive as
as compared to the InterLisp
InterLisp
environments
ZetaLisp machine environments.
environments. Also, SUN’
SUN'ss need
need to be configured
configured with
with relatively
relatively
and ZetaLisp
Lisp systems
systems (because
(because of
of limited
limited garbage
garbage collection
collection
large memories to accommodate Lisp
facilities currently)
currently) and this
this has
has required using third
third part
part memory.
memory. More
More standard
facilities
configurations should be available
available from
from vendors shortly
shortly and we expect to have
configurations
additional information
information to report
report next
next year.
additional
Workstation Standards
StaOndards and Access
10 - Workstation
10.1 - Computing
Environment Standards
Standards
I0.I
Computing Environment
In a heterogeneous
heterogeneous computing
computing environment,
environment, such
such as
as AI
AI research
research inevitably
inevitably involves, the
In
issue of
of cross-system
cross-system compatibility
compatibility is a central one. Users
Users of
of various machines want to
issue
share software, as
as well as
as be able to use various machines with
with a minimum
minimum
be able to share
of overhead in learning
learning the operating
operating procedures
procedures and programming
programming languages
languages of
of new
of
systems. Thus, itit is crucial
crucial to specify and propagate powerful,
powerful, flexible
flexible standards for
for
systems.
aspects of
of the computing
computing environment
environment so that
that itit is possible to transfer
transfer both
various aspects
skills and information
information among machines.
skills
inter-machine compatibility
compatibility of
of our
our software, we have been
been
In order to improve
improve the inter-machine
In
users to use
use the CommonLisp
CommonLisp programming
programming language
language [16J,
as well as
as
encouraging all users
[16], as
pressing
pressing vendors to provide more complete and efficient
efficient implementations
implementations of
of this
language.
language. We have
have already served
served as
as beta test sites
sites for
for Xerox, Texas
Texas Instruments, and
Lucid
Lucid CommonLisp
CommonLisp implementations.
implementations.
The Common
Lisp language,
CommonLisp
language, however, is only
only a subset
subset of
of the software needed
needed for
for our
research.
Research projects need
need higher-level
higher-level powerful
powerful facilities,
facilities, such
such as
as an objectobjectresearch. Research
oriented
oriented programming
programming system
system and sophisticated error
error handling.
handling. Therefore
Therefore we have
have been
been
supporting
Lisp Object System
supporting and following
following the development
development of
of the Common
CommonLisp
System (CLOS)
via membership in the electronic
electronic discussion group, technical contributions,
contributions, and porting
porting
of
of Portable Common
Common Loops (PCL),
(PCL), a predecessor
predecessor of
of CLOS, to the TI
TI Explorer.
Explorer. We are
now encouraging vendors to produce efficient
efficient implementations
implementations of
of the system,
system, and users
users
to familiarize
also encouraging vendors to adopt the
familiarize themselves
themselves with
with it. We are also
proposed
Lisp error
proposed Common
CommonLisp
error system.
system.
Other features of
of the computing
computing environment
environment also
also need
need to be standardized to be useful
on more than one machine at a time. Another
Another of
of the most important
important of
of these
these is the
See the
keyboard and display interface, often
often referred to as
as the "window
“window system".
system”. See
virtual
virtual graphics section (page
(page 34) for
for further
further discussion
discussion of
of window
window systems.
systems.
There are also
also many other areas
areas which could benefit
benefit greatly from
from standardization,
standardization,
including
including document page
page description
description languages,
languages, text and graphics representations, and
it is important
important that standards
standards not be entered
entered into
into
more networking
networking protocols. However, it
hastily, as
as an insufficient
insufficient standard can
can often
often be worse
worse than no standard at all. We
intend to continue working to develop standards
standards for
for these
these and other computing
computing needs
needs as
as
the understanding of
of the issues
issues involved
involved matures.
matures.
E.
E. H. Shortliffe
Shortliffe
54
54
of Technical
Technical Progress
Progress
Details of
Details
5P41-RR00785-14
5P41-RR00785-14
10.2 -- Protocol
Protocol Standards
Standards
IO.2
In addition
addition to
to various
various portions
portions of
of the,
the AI
AI research
research computing
computing environment,
environment, the
the most
most
In
of standardization
standardization· has
has been
been inter-machine
inter-machine communication,
communication, or
or
highly visible
visible area of
highly
networking. Underlying
Underlying all
all network
network I/O
I/O must
must be a network
network protocol
protocol for
for packet transfer
transfer
networking.
At SUMEX
SUMEX we have had long
long term
term experience with
with several
several
between cooperating hosts.
hosts. At
between
PUP/BSP, PUP/EFTP,
PUP/EFIP, IP/TCP,
IP/TCP, IP/TFTP,
IP/TFTP, IP/UDP,
IP/UDP, IP/SMTP,
IP/SMTP, and
such protocols; PUP/BSP,
most commonly
commonly used
used on SUNet. PUP/BSP
PUP/BSP and IP/TCP
IP/TCP have been
NS/SPP are those most
used to
to implement
implement both
both FTP
FIP and TELNET,
TELNET, PUP/EFTP
PUP/EFTP is an “Easy
"Easy File
File Transfer
Transfer
used
Protocol"
on
top
of
PUP
used
for
boot
like
services.
IP
ITFTP
is
a
"Trivial
File
Protocol” on top of PUP used for boot like services. IP/TFTP
“Trivial File
Transfer Protocol”
Protocol" which
which uses
uses IP/UDP
IP/UDP datagrams.
datagrams. IP/SMTP
IP/SMTP is the “Simple
"Simple Mail
Mail
Transfer
for sending mail,
mail, and runs on top
top of
of IP/TCP.
IP/TCP.
NS/SPP is a
Transfer Protocol”
Protocol" for
NS/SPP
Transfer
similar to
to PUP/BSP
PUP/BSP and is used
used for
for FTP
FTP and TELNET.
TELNET.
"Sequenced Packet Protocol”
Protocol" similar
“Sequenced
In
the
past
we
have
elected
to
write
servers
for
each
new
protocol
in
order to
to
In
write servers for each
protocol in order
systems software
software requirements.
This was
was
accommodate both
both vendor
vendor hardware and systems
This
accommodate
protocol has
has been
been supported on all
all such systems.
systems.
necessary because
because no one protocol
necessary
With others in
in the computer
computer science
science research
research community,
community, we have pressed
pressed vendors to
With
of the DARPA
DARPA standard TCP/IP
TCP/IP communications
communications protocols.
supply implementations
implementations of
that the IP protocol
protocol family
family is now supported on all
all hardware and
pleased that
We are pleased
system configurations
configurations currently
currently at SUMEX.
SUMEX. And
And we expect to
to have IP
operating system
purchase in the future.
future. Similarly,
Similarly, IP is supported on
systems we purchase
support on any new systems
of our UNIX
UNIX based
based file
file servers,
servers, and the SUNet
SUNet gateways
gateways route all IP datagrams.
datagrams.
all of
has been
been a great deal of
of deliberate effort
effort at Stanford
Stanford and SUMEX
SUMEX to enforce IP
There has
as a standard protocol
protocol for
for new software development. This
This was
was motivated
motivated by its broad
as
number implementations
implementations throughout
throughout the networking
networking and
acceptance and the growing
growing number
acceptance
not imply
imply that we will
will abandon the other protocols
protocols but
but
vendor communities.
communities. This
This does
does not
rather, since
proposed
uniformity across
across all
all vendors
vendors with
with the proposed
since we are seeking
seeking to have
have uniformity
Stanford distributed
distributed environment,
environment, we are choosing to limit
limit new implementations
implementations to the
IP protocol family.
provide improved
currently working
working to provide
improved support
support for
for
family. We are also currently
TCP/IP
TCP/IP in our Terminal
Terminal Interface
Interface Processors
Processors (TIP's),
(TIP’s), having already implemented
implemented
lIP routing
TCP
TCP/IP
routing service.
service.
As an
an example
example of
of the power of
of using uniform
uniform communication
communication protocols, we
we set
set up a
Xerox 1186
Philadelphia
1186 workstation
workstation for
for use
use by Dr. Shortliffe
Shortliffe during
durin,0 his sabbatical in Philadelphia
at the University
This university
university has
has a different
different network
network environment
environment
University of
of Pennsylvania. This
than Stanford's,
Stanford’s, although it
it is probably
probably more typical
typical of
of common
common Ethernet
Ethernet installations.
installations.
The Pennsylvania
Pennsylvania network
network provides only
only Class-B IP/TCP
IP/TCP services
services for
for VAX-based
VAX-based
VMS/Unix
VMS/Unix systems
systems over "thin"
“thin” Ethernet. The 1186
1186 was
was the only
only piece
piece of
of Xerox
hardware
on
the
network
so
the
disk
was
pre-loaded
at
Stanford.
We
successfully
hardware
so
was
successfully used
used
the
the Pennsylvania
Pennsylvania VMS VAX's
VAX’s as
as file
file servers
servers and
and time servers
servers (after
(after writing
writing appropriate
appropriate
software to interface the workstation
Using their
their
workstation to the RFC868 time protocol).
protocol).
Ethernet to ARPANET
ARPANET gateway,
gateway, we
we were
were able
able to connect to SUMEX-AIM
SUMEX-AIM directly
directly from
from
the
the Xerox workstation as
as well as
as access
access our print
print servers
servers at Stanford. Unfortunately,
Unfortunately,
hardware
hardware problems
problems with the workstation
workstation later in the year
year prevented
prevented us
us from
from attempting
attempting
any
any more complex experiments with
with distributed
distributed computing
computing and
and remote hosts.
hosts.
Such
Such standardization has
has aa
speeds
speedsare
are uniformly
uniformly higher
For aa discussion
discussion of
of network
price, however,
however, in
between
between equipment
file
file access
access protocol
that observed
observed network communications
communications
"tuned"
“tuned” to individual
individual vendor protocols.
benchmarks see
see page
page 40.
40.
11
11 -- Network
Network Services
Services
A highly important
EX system
important aspect
aspect of
of the
the SUM
SUMEX
system isis effective
effective communication
communication within
within our
growing distributed computing
computing environment
environment and
and with remote users.
users. In addition
addition to the
the
economic
economic arguments
arguments for
for terminal
terminal access,
access,networking
networking offers
offers other
other advantages
advantages for
for shared
shared
computing.
computing. These
These include
include improved
improved inter-user
inter-user communications,
communications, more
more effective
effective software
software
55
55
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
uniform user
user access
access to multiple
multiple machines and special purpose resources,
resources,
sharing, uniform
convenient file
file transfers, more effective
effective backup, and co-processing between
between remote
convenient
for maintaining
maintaining the collaborative
collaborative scientific
scientific and
machines.
Networks are crucial
crucial for
Networks
within the SUMEX-AIM
SUMEX-AIM community.
community.
software contacts within
Networks
11.1 -- Remote Networks
11.1.1 - Commercial
Commercial Network
Network Link
Link
11.1.1
At the beginning
beginning of
of this grant
grant year, SUMEX
SUMEX had just
just begun switching
switching
At
(PDN) vendors (from
(from TYMNET
TYMNET to UNINET)
UNINET) in
in an attempt
attempt
networks (PDN)
for our
our users.
users. As the result of
of a corporate merger, our
our connection
connection
service for
became a connection to TELENET.
TELENET.
became
public data
public
to improve
improve
UNINET
to UNINET
11.1.2
X.25/Ethernet Link
Link
11.1.2 - X.251Ethernet
use of
of Ethernet
Ethernet has
has prompted interest in
in a suitable
SUMEX and Stanford’
Stanford'ss heavy
heavy use
SUMEX
Ethernet system
system and the Public
Public Data Networks
Networks (specifically
(specifically
connection between
between our Ethernet
connection
TELENET). Commercial groups provide
provide a wide variety
variety of
of equipment
equipment connecting
connecting these
these
TELENET).
but lack of
of standards
standards for
for terminology
terminology make itit difficult
difficult to
X.25 networks to Ethernets, but
determine their
their function.
function.
Because our interest involves connection
connection to other X.25 based
based hosts
hosts and Packet
Because
Assemblers/Disassemblers
Assemblers/Disassemblers (PADs),
(PADS), sometimes
sometimes called Terminal
Terminal Interface
Interface Processors,
Processors, as
as
we need
need a device that provides
opposed
opposed to connecting two Ethernets via an X.25 net, we
use a SUN processor
processor for
protocol translation.
protocol
translation. One alternative
alternative we are considering is to use
for
this
processor will
normal TCP/IP
this task.
task. This
This processor
will have its normal
TCP/IP Ethernet
Ethernet capability
capability supplemented
package provided by SUN. Such
package will
will provide
provide SUMEX
with both
Such a package
SUMEX with
by an X.25 package
Users on SUMEX
will be
an inbound
inbound and outbound capability
capability relative to TELENET.
TELENET. Users
SUMEX will
NLM and
able to access
access the large variety
variety of
of hosts
hosts and services
services on the PDNs (such
(such as
as NLM
Dialog)
for file
file
Dialog) in a simple and reliable
reliable manner. Though the high level protocols for
transfer
transfer and mail exchange
exchange are developing slowly in the X.25 environment,
environment, some
some
progress is being made,
progress
made, so a general
general purpose interface
interface to these
these networks is an important
important
asset.
asset.
11.1.3
11.1.3 - ARPANET
ARPANET Link
Link
We also
also continue our extremely advantageous
advantageous connection to the Department
Department of
of
Defense's
Defense’s ARPANET,
ARPANET, managed
managed by the Defense
Defense Communications
Communications Agency (DCA).
(DCA). This
This
connection has
has been
been possible
possible because
because of
of the long-standing
long-standing basic
basic research
research effort
effort in AI
AI
within
within the Knowledge Systems
Systems Laboratory
Laboratory that is funded by DARPA.
DARPA. ARPANET
ARPANET is the
primary
primary link
link between
between SUMEX
SUMEX and other university
university and AIM
AIM machine resources,
resources,
including
including the large
large AI
AI computer science
science community
community supported by DARPA.
DARPA. We are also
also
attempting
tate the
attempting to establish
establish a link
link to the DARPA
DARPA wideband satellite network
network to facili
facilitate
rapid transfer
transfer of
of large amounts of
of data
data such
such as
as are involved
involved with
with projects like
like our
Concurrent
Concurrent Symbolic Computing
Computing Architectures
Architectures project.
As a member of
of the ARPAnet
ARPAnet group, we
we have
have an obligation
obligation to help with
with certain
network operations tasks.
tasks. For instance,
instance, we
we participated in the upgrading (to 56
56 Kbs) of
of
the connection which Advanced
Advanced Decision Systems,
Systems, Inc. (Mt.
(Mt. View, CA)
CA) has
has to our IMP.
We also
also have
have a minor
minor role in certain mail routing
routing functions
functions for
for the
the ARPANET
ARPANET
community.
community.
As part of
of an
an overall increase
increase in ARPANET
ARPANET capacity a third
third 56
56 Kbs trunk
trunk line is
being added
added to our IMP
IMP by the Defense
Defense Communications
Communications Agency (DCA).
(DCA).
E.
E. H. Shortliffe
Shortliffe
56
56
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR00785-14
Microcomputer Networks
Networks
11.2 -- Microcomputer
11.2
We connected
connected our
our Apple
Apple Macintosh
Macintosh computers
computers in
in 22 buildings
buildings with
with Appletalk
Appletalk and
and
We
Phonenet
network
products.
More
significantly,
we
integrated
them
with
the
rest
of
our
Phonenet network products. More significantly, we integrated them with the rest of our
equipment by connecting
connecting the microcomputer
microcomputer networks to
to the campus Ethernet
Ethernet networks
networks
equipment
Kinetics FastPath
FastPath gateways,
gateways, a commercial
commercial spinoff
spinoff resulting
resulting from
from the SUMEX
SUMEX
using Kinetics
on the SEAGATE
SEAGATE gateway,
gateway.
work on
work
Columbia University,
University, Stanford,
Stanford, and elsewhere,
elsewhere, makes itit possible for
for
Software written
written at Columbia
Macintosh to
to share a VAX
VAX file
file server with
with the Lisp
Lisp machines and to
to access
access hosts on
on
a Macintosh
ARPA internet
internet as
as a first-class
first-class workstation.
workstation.
the ARPA
Usage of
of a centralized VAX
VAX file
file server makes
makes nightly
nightly backup and data sharing
sharing
Usage
of usage
usage is a great improvement
improvement over
over the isolated stand-alone
stand-alone
automatic. This
This mode of
automatic.
that most people think
think of
of when they think
think of
of microcomputers.
microcomputers.
machines that
Local Area
Area Networks
N etwor ks
11.3 -- Local
For many years
years now, we have
have been
been developing our
our local area networking
networking systems
systems to
For
of this
this work
work has
has centered on the
enhance the facilities
facilities available to researchers.
researchers. Much of
enhance
of distributed
distributed computing
computing resources
resources in
in the form
form of
of mainframes,
mainframes,
effective integration
integration of
effective
workstations, and servers.
servers. Network
Network gateways
gateways and terminal
terminal interface
interface processors
processors (TIP’
(TIP's)
s)
environment together. We are developing
developed and extended
extended to link
link our environment
were developed
as needed
needed too. A
A diagram of
of our local area
gateways to interface
interface other equipment as
gateways
LAN-related
network system
system is shown in
in Figure 8 and the following
following summarizes our LAN-related
network
development work.
11.3.1
11.3.1 - Ethernet Gateways
Gateways
provide workstation
In our heterogeneous
heterogeneous network
network environment,
environment, in order to provide
workstation access
access to
necessary to able
file
file servers,
servers, mail
mail servers,
servers,.and other computers within
within the network, it
it is necessary
to route multiple
multiple networking
networking protocols through the network
network gateways.
gateways. As reported last
year,
year, the SUMEX
SUMEX gateways
gateways support PUP,
PUP, Xerox NS, Symbolics/Texas-Instrument
Symbolics/Texas-Instrument
CHAOSNET,
ITCP protocols. This
This support not only
only provides the routers
CHAOSNET, and the IP
IP/TCP
necessary
necessary to move such
such packets
packets among
among the subnetworks,
subnetworks, but also other miscellaneous
miscellaneous
services
services such
such as
as time, nameladdress
name/address lookup, host statistics, boot strap support, address
address
resolution,
resolution, and routing
routing table broadcast
broadcast and query information.
information.
This year,
year, with
with the
the acquisition of
of a SMI SUN 31180
3/180 file
file server
server and
and three SUN 3175
3175
workstations, it
it was
was necessary
necessary to add
add special
special boot-protocol
boot-protocol support for
for SMI's
SMI’s Net Disk
Disk
and NFS protocols to allow the
the SUN workstations to boot their
their Unix
Unix kernel, and
runnable programs
programs while residing on a network that is distinct
distinct from
from the one on which
the file
file server
server resides.
resides. SUN's
SUN’s convention is that each
each subnet must have
have its own file
file
server
server that can
can provide boot support. But this is too expensive
expensive for
for complex network
environments
environments such
such as
as ours.
ours. Given this broadened
broadened capability,
capability, we
we can
can now place
place our
"diskless"
“diskless” SUN workstations anywhere
anywhere within
within the
the KSL network topology, rather than on
the
the same
same network that the server
server resides.
resides.
Also, to improve
improve the
the throughput of
of our highly loaded
loaded gateways,
gateways, portions
portions of
of Ethernet
interface drivers and
and protocol routers
routers were
were rewritten. The drivers now look for
for the
the
arrival
arrival of additional
additional packets
packets while processing
processing those
those packets
packets that initiated
initiated the
the interrupt.
interrupt.
Now, each
each router can
can process
process up
up to six
six packets
packets before relinquishing
relinquishing control
control to the
the
gateways
gateways process
process scheduler.
scheduler. Previously,
Previously, each
each router would process
process only one
one such
such packet
packet
per
These two changes
changes more
more than
than doubled
doubled the
the maximum observed
observed packets
packets per
per
per call. These
second,
second, as
as well
well as
as the
the maximum
maximum throughput
throughput bandwidth
bandwidth which
which isis now
now about
about 2.5
2.5 megabits
megabits
per
per second,
second, and
and minimized
minimized the
the dropping
dropping of
of back-ta-back
back-to-back packets
packets by
by the
the Ethernet
Ethernet
interface
interface itself.
itself.
57
57
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Over the past year our network
network topology
topology grew in complexity
complexity and extent so
so that we
we now
have
have redundant
redundant routes to several
several networks within
within the KSL and Stanford LAN.
LAN. Within
Within
this more complex environment,
environment, the old
old routing
routing table management
management schemes
schemes broke down
and had to be redesigned
redesigned and changed
changed to adequately
adequately deal
deal with
with the network
network interactions
interactions
that arose.
particular network
arose. In particular,
particular, we had to ensure
ensure that when
when a route to a particular
network no
hardware or software failure,
failure, that this
longer was
was available because
because of
of electrical,
electrical, hardware
information
information was
was propagated
propagated throughout
throughout the topology in a manner that maintained
maintained
have solved this problem and our gateways
gateways now recover
routing
routing table equilibrium.
equilibrium. We have
gracefully from
from these
these failures.
failures.
gracefully
A
A second
second kind
kind of
of failure
failure occurs
occurs when a path between
between two networks fails
fails but
but the
gateways
gateways involved
involved are not
not aware
aware of
of this fact, and as
as a consequence
consequencecontinue
continue to advertise
have had
had two examples
examples of
of this over the past
routes using paths that are partitioned.
partitioned. We have
year caused
caused by the failure
failure of
of a repeater in one case
case and a transceiver in the other.
When we detect such
such a situation,
situation, we can now remove the route from
from the gateway
gateway
generating it
it using software, make the repair, and then replace
replace the route, without
without
perturbing
perturbing the connectivity
connectivity of
of our topology
topology if
if there are redundant routes around the
partition caused
caused by hardware failure.
failure.
partition
Finally,
Finally, a minor
minor change
change in the gateways'
gateways’routing
routing table update
update algorithm
algorithm when
when multiple
multiple
routes to a network
balance the load between
between these
network are available has
has managed
managed to balance
these
alternative
paths, and increase
alternative paths,
increase the throughput
throughput at the gateways
gateways involved. Such
Such gateways
gateways
are usually focal
focal points for
for high traffic
traffic volume, and the change
change was
was immediately
immediately noted
of the routing
routing
by staff
staff members sensitive to network
network throughput. The old version of
paths of
update
update protocol
protocol would hold onto
onto a route even
even if
if alternative
alternative paths
of equal
equal cost were
were
available. The new version will
will always
always update
update a route if
if a path of
of equal
equal cost arises.
arises.
When n redundant
redundant paths are available, the route changes
changes approximately
approximately every 30/n
30/n
seconds.
seconds.
These
These services
services are still
still unique
University
University network,
network, and give our
of
of high bandwidth,
bandwidth, and extremely
within
-AIM portion
portion of
within the SUMEX
SUMEX-AIM
of the Stanford
Stanford
researchers
researchers a networking
networking environment
environment that is flexible,
flexible,
dependable.
dependable.
11.3.2
11.3.2 - Terminal
Terminal Interface
Interface Processors
Processors
With
With the advent of
of reliable
reliable multiple
multiple speed
speed (300, 1200
1200 and 2400
2400 baud) modems,
modems, we
placed
placed ten such
such devices
devices on our TIPs
TIPS for
for dial-in
dial-in access,
access,and added
added autobaud recognition
recognition
to the TIP
TIP software. 2400
2400 baud
baud dial-in
dial-in connections have
have shown
shown themselves
themselves to be highly
highly
responsive
placed on the TIP
responsive in such
such a configuration,
configuration, and have
have the advantage
advantage when
when placed
TIP of
of
giving
giving the user
user access
access to any host on the Stanford local area
area network. Autobaud
recognition
recognition has
has also
also been
been added
added to the directly
directly attached
attached tty ports to simplify
simplify user/TIP
user/TIP
interaction.
interaction. If
If a user
user changes
changes his terminal's
terminal’s baud rate,
rate, the TIP
TIP will
will still
still be responsive,
responsive,
but
but at a different
different speed.
speed. Previously, such
such a line's
line’s baud rate was
was fixed,
fixed, and this often
often
deal of
of user
user frustration.
frustration.
led to a great deal
NTT ELlS
Also, the experimental
experimental NTT
ELIS Lisp
Lisp Machines used
used in the KSL
KSL currently
currently do not have
have
Ethernet
Ethernet connections. To
To accommodate
accommodate remote access
access to these
these systems,
systems, they were
were
attached
attached to TIP
TIP ports so
so that a user
user could connect to the TIP
TIP from
from the Ethernet, and
transparently connect to the ELlS
ELIS machines
machines via a TIP
TIP command. Once this
then transparently
connection
connection is established,
established, the user
user appears
appears to have
have a terminal
terminal directly
directly attached
attached to the
ELlS
ELIS itself.
itself. Currently,
Currently, there are six such
such ELlS
ELIS ports in use
use on one of
of our TIPs.
TIPS.
Incidentally,
Incidentally, the same
same code
code is currently
currently being generalized
generalized for
for use
use as
as a dial-out
dial-out module.
E. H. Shortliffe
Shortliffe
58
58
5P41-RR00785-14
5P41-RR00785-14
Details of Technical Progress
Progress
12 - Printing
Printing Services
Services
12
Laser
Laser printers
printers have
have become
become essential
essential components of the
the work environment
environment of the
the
from scientific
scientific publications
publications to
SUMEX-AI\f
SUMEX-AI>f community
community with applications
applications ranging from
we
hardcopy graphics
gaphics output
output for
for ONCOCIN
ONCOCIN chemotherapy protocol patient
patient charts.
charts. We
have
have done much systems
systems work to integrate
integate laser
laser printers
printers into the
the SLiMEX
SUMEX network
environment
environment so
so they would
would be
be routinely
routinely accessible
accessible from
from hosts
hosts and workstations alike.
This
This software has
has been
been widely shared
shared with other user
user groups
groups in the AIM
AIM community
community
and beyond.
SUMEX
SUMEX operates
operates 7 medium-speed (8-20
(S-20 pages
pages per
per minute)
minute) Imagen
Imagen laser
laser printers, 2 lowlowspeed
speed (-3
(-3 ppm) Xerox laser
laser printers.
printers, and 1 low-speed
low-speed (-3
(-3 ppm) Apple laser
laser printer.
printer.
Each
Each of
of the Imagen
Imatgen printers
printers possesses
possessesan emulator for
for a line printer.
printer, a daisy
daisy wheel
wheel
printer.
printer, a Tektronix
Tektronix plotter.
plotter. and a typesetter
typesetter (using the
the Impress language).
language). The last 3
fnterpress. and
typesetter
printers
printers render the special-purpose Press.
Press, Interpress,
and Postscript
Postscript
typesetter
languages. In total.
languages.
total, the laser
laser printers
printers printed
printed about half
half a million
million pages
pages of
of output
output
during the year.
year. Most of
during
of the
the printout
printout was
was simple text.
text, followed
followed in quantity
quantity by
impress format.
format, Impress-format
Impress-format drawings.
drawings, and screen
screen dumps.
dumps. Lastly.
Lastly,
formatted text in Impress
formatted
about 2000
2000 pages
pages each
each of
of Postscript-format
Postscript-format drawings and formatted
formatted text were
were printed
on the Apple
Apple Laser
Laser Writer.
Writer. Although
Although the Postscript
Postscript language
language isis probably the
the most
popular
popular typesetting language
language among commercial
commercial applications
applications developers
developers at the
the present
present
time (and one which we
with the Laser
Laser Writer).
Writer), the overwhelming
overwhelming
we support with
preponderance of
of reJdily-renderable
readily-renderable line printer
printer and Impress
Impress jobs in our printing
printins mix
provides the basis
basis for
for our
our decision to emphasize
emphasize the
the rel.1tively
relatively high-speed
high-speed Imagen
Ima,oen l:J.ser
laser
printers. Because
printers.
B~tcnuseof
of the increasing usage
usage of
of Postscript among vendors.
vendors, however,
however, we
we
purchased an additional
additional Apple
Apple L.1ser
Laser Writer
Writer for
for use
use in the Medical
School Office
Office
have purchased
Nfedical School
Building.
Building.
In order
order to finally
finally obtain
obtain families
families of
of fonts
fonts in common between
between our Press,
Press, Impress
Impress and
Interpress printers,
printers, we used
used the TypeFounder
TypeFounder software that we
we beta-tested for
for Xerox to
extract
extract font
font width
width information
information (for
(for use
use by our workstations)
workstations) from
from our existing
page per minute,
printer based
Interpress printer
printer fonts
fonts (a 12
12 page
minute, 300
300 dpi printer
based on the Xerox
Xercs 8000
8000
processor) and also made new fonts
fonts using character splines from
from an
an eo.rlier
earlier Xerox gro.nt
grant
Having an overlap in fonts
fonts among the printers
helps to relieve the problems
program. Having
printers helps
inherent in trying
trying to print
inherent
print the same
printer
same complex document on different
different printer
technologies. Some
patches for
Some of
of the font
font additions
additions required software patches
for the Interpress
driver
was further
further modified
modified to
driver software
software on the workstations.
workstations. The Interpress driver
driver was
provide
provide rotated fonts
fonts in order to print
print our
our specialized medical forms.
forms.
13 -- General
General User
User Software
Software
continued to assemble
assemble (develop where necessary)
necessary) and maintain
m.1intain a broad range
range
We have continued
of
of user
user support
support software.
These include
include such tools as
as language
language systems,
systems, statistics
These
packages,
packages, vendor-supplied
vendor-supplied programs, text
text editors,
editors. text search
search programs, file
file space
space
programs. graphics support,
support. a batch program execution monitor,
monitor, text
management programs,
formatting
formatting and justification
justific.1tion assistance,
assistance. magnetic tape
tape conversion aids,
aids, and user
user
information/help assistance
assistance programs.
information/help
important area of
of user software
software for
for our
our community
community effort
effort is a set of
of
particularly important
AA particularly
for inter-user
inter-user communications.
communications. We have
have built
built up a group of
of programs to
tools for
facilitate many
many aspects
aspects of
of communications
communications including
including interpersonal
interpersonal electronic
electronic mail,
mail, a
facilitate
between
"bulletin board”
board" system
system for
for various special interest
interest groups to bridge the gap between
“bulletin
m.1il and formal
formal system documents,
documents. and tools for
for terminal
terminal connections and file
file
private mail
private
transfers between
between SU;LIEX
SUMEX and various external hosts.
hosts. Examples
Examoles of
of work
work on these
these sorts
transfers
of programs have already
alre.1dy been
been mentioned
mentioned in earlier
earlier sections,
sections: particuinrly
particularly as
as they relate
of
to
to extensions
extensions for
for a distributed
distributed computing
computing environment.
environment.
59
1-1. Shortliifz
Shortlir'r'e
E. l-1.
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR00785-14
At SUMEX-AIM
SUMEX-AIM we are committed
committed to importing
importing rather
rather than reinventing
reinventing software
software where
At
number of
of the packages
packages we have brought
brought up are from
from
possible. As noted above, a number
Many avenues
avenues exist
exist for
for sharing
sharing between
between the system staff,
staff, various user
user
outside groups. Many
other facilities,
facilities, and vendors.
availability of
of fast and convenient
convenient
projects, other
The availability
communication facilities
facilities coupling
coupling communities
communities of
of computer
computer facilities
facilities has
has made possible
communication
effective intergroup
intergroup cooperation
cooperation and decentralized maintenance of
of software packages.
packages.
effective
The
many
operating
system
and
system
software
interest
groups
(e.g.,
TOPS-20,
UNIX,
The
operating system
system software interest
(e.g.,
UNIX,
D-Machines, network
network protocols,
protocols, etc.) that
that have grown up by means
means of
of the ARPANET
ARPANET
D-Machines,
been a good model for
for this
this kind
kind of
of exchange.
exchange. The other
other major
major advantage
advantage is that
have been
as a by-product
by-product of
of the constant
constant communication
communication about particular
particular software, personal
as
connections between
between staff
staff members of
of the various
various sites develop. These
These connections
connections
pass general information
information about software
software tools and to encourage
encourage the exchange
exchange of
of
serve to pass
ideas among the sites and even vendors as
as appropriate
appropriate to our
our research
research mission. We
ideas
continue to
to import
import significant
significant amounts of
of system
system software
software from
from other
other ARPANET
ARPANET sites,
sites,
continue
reciprocating with
with our
our own local developments.
developments.
Interactions have included
included mutual
mutual
reciprocating
Interactions
with various hardware configurations,
configurations, experience with
with new
backup support, experience with
of computers and operating
operating systems,
systems, designs
designs for
for local networks, operating system
system
types of
enhancements, utility
utility or language
language software, and user
user project
project collaborations.
collaborations. We have
have
enhancements,
that have interacted with
with SUMEX
SUMEX user
user projects get access
access to software
assisted groups that
assisted
community (for
(for more details, see
see the section on Dissemination
Dissemination on page
page
available in our community
103).
E.
E. H. Shortliffe
Shortliffe
60
Details
Details of
of Technical
Technical Progress
Progress
5P41-RR00785-14
5P41-RR00785-14
III.A.3.S. Relevant
Relevant Core
Core Research
Research Publications
Publications
III.A.3.5.
following is a list
list of
of new publications
publications and reports that
that have come out
out of
of our
our core
The following
efforts over the past year:
research and development
development efforts
research
KSL 85-57
85-57
KSL
Horvitz and D. Heckerman; The
The Inconsistent
Inconsistent Use of
of Measures of
of
(Journal Memo)
Memo) E. Horvitz
(Journal
To appear in:
in: Uncertainty
Uncertainty
Certainty in
in Artificial
Artificial Intelligence
Intelligence Research, August 1985.
1985. To
Certainty
Artificial Intelligence
Intelligence 15 pages
pages
in Artificial
in
KSL 85-58
85-58
KSL
(Journal Memo)
Memo) C.D. Lane, M.E. Frisse, L.M.
L.M. Fagan,
Fagan, and E.H. Shortliffe;
Shortliffe; ObjectObject(Journal
To
Medical Interface
Interface Design,
Design, December 1985.
1985.
To appear in:
Oriented Graphics
Graphics in Medical
Oriented
AAMSI-86 5 pages
pages
AAMSI-86
KSL 85-59
85-59
KSL
(Working Paper) A
Allan
Terry; Using
Using Explicit
Explicit Strategic
Strategic Knowledge to Control
Control Expert
Expert
.llan Terry;
(Working
for publication
publication in: Artificial
Artificial Intelligence
Intelligence 51
Submitted for
1985. Submitted
Systems, December 1985.
pages
paw
KSL 85-60
85-60
KSL
(Working Paper) Jean-Luc
Jean- Luc Bonnetain;
Bonnetain; FLOWER:
FLOWER: A
A First
First Cut
Cut at Designing
Designing a Budget
(Working
Proposal, September 1985.
pages
Proposal,
1985. 28 pages
KSL 86-18
86-18
KSL
appear 10:
AI
STAN-CS-86-1123. H.
Nii; Blackboard
Blackboard Systems, June 1986.
STAN-CS-86-1123.
H. Penny Nii;
1986. To appear
in: AI
Magazine Vols.
7-2 and 7-3.
7-3. 86 pages
pages
Magazine
Vols. 7-2
KSL
KSL 86-24
86-24
(Journal
M.A. Musen, L.M.
L.M. Fagan,
Fagan, D.M. Combs, and E.H. Shortliffe;
Shortliffe; Using
Using a
(Journal Memo)
Memo) M.A.
Domain
Domain Model
Model to Drive
Drive An Interactive
Interactive Knowledge Editing
Editing Tool,
Tool, September
September 1986.
1986. To
appear
Proceedings of
AAAI Workshop
Knowledge Acquisition,
Acquisition, 1986 12 pages
of AAAI
Workshop on Knowledge
pages
appear in: Proceedings
KSL
KSL 86-25
86-25
(Journal
(Journal Memo)
Memo) EJ.
E.J. Horvitz,
Horvitz, D.E. Heckerman, and c.P.
C.P. Langlotz; A
A Framework
Framework for
for
Comparing
Comparing Alternative
Alternative Formalisms
Formalisms for
for Plausible
Plausible Reasoning, May
May 1986.
1986. 5 pages
pages
KSL
KSL 86-28
86-28
(Working
Cornelius, Russ
Russ Altman,
Altman, Barbara Hayes-Roth,
(Working Paper) James
James Brinkley,
Brinkley, Craig
Crai,0 Cornelius,
Olivier
Olivier Lichtarge, Bruce Duncan, Bruce
Bruce Buchanan,
Buchanan, Oleg
Oleg Jardetzky;
Jardetzky; Application
Application of
of
Constraint Satisfaction
Satisfaction Techniques to the Determination
Determination of
of Protein
Protein Tertiary
Tertiary Structure,
Structure,
Constraint
March 1986.
1986. 14
14 pages
pages
KSL
KSL 86-29
86-29
(Working
(Working Paper) Matthew L. Ginsberg; Multi-valued
Multi-valued logics, April
April 1986.
1986. To appear
appear in:
AAAI
AAAI - 86
86 13 pages
pages
KSL
KSL 86-33
86-33
(Journal
(Journal Memo)
Memo) David
David E. Heckerman and Eric
Eric J.
J. Horvitz;
Horvitz; The Myth
Myth of
of Modularity
Modularity in
Rule-Based
Rule-Based Systems, May
May 1986.
1986. 7 pages
pages
KSL
KSL 86-36
86-36
STAN-CS-87-1148.
STAN-CS-87-1148, Bruce A. Delagi, Nakul
Nakul Saraiya,
Saraiya, Sayuri Nishimura,
Nishimura, and Greg Byrd;
An Instrumented
Instrumented Architectural
Architectural Simulation
Simulation System, January
January 1987.
1987. 21
21 pages
pages
KSL
KSL 86-37
86-37
(Working
(Working Paper) Matthew L. Ginsberg; Possible Worlds
Worlds Planning,
Planning,
Submitted for
for publication
publication to: 1986 Planning
Planning Workshop
Workshop 13 pages
pages
61
April
April
1986.
1986.
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
of Technical
Technical Progress
Progress
Details of
KSL 86-38
86-38
KSL
M. Vaughan Johnson Jr., Alan
Alan Garvey, and
STAN-CS-87-1147. Barbara Hayes-Roth,
Hayes-Roth, M.
STAN-CS-87-1147.
Michael Hewett; AA Modular
Modular and Layered Environment
Environment for
for Reasoning about
about Action,
Action,
Michael
April 1987.
1987. To
To appear in:
in: The Journal
Journal of
of Artificial
Artificial Intelligence
Intelligence in
in Engineering,
Engineering,
April
Special Issue
Issue on Blackboard
Blackboard Systems,
Systems, October
October 1986. 63 pages
pages
Special
KSL 86-39
86-39
KSL
Management Decisions:
Decisions:
(Journal Memo)
Memo) E.H. Shortliffe;
Shortliffe; Artificial
Artificial Intelligence
Intelligence in Management
(Journal
To appear in: Proceedings
Proceedings of
of a Conference
Conference on Medical
Medical
ONCOCIN, April
April 1986.
1986. To
ONCOCIN,
Information Sciences,
Sciences, University
University of
of Texas
Texas Health
Health Sciences
Sciences Center
Center at San Antonio,
Antonio,
Information
July 1985. Also
Also in
in Frontiers
Frontiers of
of Medical
Medical Information
Information Sciences,
Sciences, Praeger
Praeger Publishing,
Publishing,
July
pages
1986. 14 pages
KSL 86-40
86-40
KSL
(Journal Memo)
Memo) Christopher
Christopher Lane; The
The Ozone Manual,
Manual, July
July 1986. 34 pages
pages
(Journal
KSL 86-42
86-42
KSL
(Working Paper) Oleg Jardetzky, Andrew
Andrew Lane, Jean-Francois
Jean-Francois Lefevre,
Lefevre, Olivier
Olivier
(Working
Lichtarge, Barbara Hayes-Roth,
Hayes-Roth, Russ
Russ Altman,
Altman, Bruce Buchanan; A
A New Method
Method for
for the
Determination of
of Protein
Protein Structures
Structures in Solution
Solution from
from NMR,
NMR, May 1986.
Submitted for
for
Determination
1986. Submitted
publication in: Proc.
Proc. XXIII
XXIII Congress Ampere,
Ampere, Rome, Italy,
Italy, Sept. 1986 6 pages
pages
publication
KSL 86-43
KSL
(Journal Memo)
Memo) Edward H.
Update on Oncocin:
Shortliffe; Update
Oncocin: A
A Chemotherapy
Chemotherapy Advisor
Advisor
(Journal
H. Shortliffe;
Medical
Submitted for
for Clincal
Clincal Oncology,
Oncology, August
August 1986.
Submitted
for publication
publication in:
Medical
for
Informatics 4 pages
pages
Informatics
KSL
KSL 86-44
(Thesis) Stephen
A Program for
for Automated
Automated Summarization
Summarization of
of On-Line
On-Line
Stephen M.
M. Downs; A
Medical
Medical Records, June 1986.
1986. 27 pages
pages
KSL
KSL 86-46
86-46
STAN-CS-86-1111.
STAN-CS-86-1111. Paul Rosenbloom and John Laird;
Laird; Mapping
Mapping Explanation-Based
Explanation-Based
Generalization
AAAI-86 18 pages
Generalization onto Soar, June 1986.
1986. To
To appear in: MI-86
pages
KSL
KSL 86-47
STAN-CS-86-1124.
Daniel
Matching
STAN-CS-86-1124.
Daniel J.
J. Scales;
Scales; Efficient
Efficient
Matching
SOAR/OPS5
SOAR/OPSS Production
Production System, June 1986.
1986. 50 pages
pages
Algorithms
Algorithms
for
for
the
KSL
KSL 86-48
(Working
William J.
J. Clancey;
Clancey; Review of
of Winograd
Winograd and Flores'
Flares’ "Understanding
“Understanding
(Working Paper) William
Computers and
and Cognition:
Cognition: A
A New Foundation
Foundation for
for Design",
Design”, July 1986.
1986. 13 pages
pages
KSL
KSL 86-49
(Journal
(Journal Memo)
Memo) M.A.
M.A. Musen,
Musen, D.M.
D.M. Combs, J.D. Walton,
Walton, E.H. Shortliffe,
Shortliffe, L.M. Fagan;
Fagan;
OPAL:
OPAL: Toward the Computer-Aided
Computer-Aided Design of
of Oncology
Oncology Advice
Advice Systems, July 1986.
1986.
Submitted for
for publication
publication to: Proceedings
Proceedings of
of the Tenth Annual
Annual Symposium
Symposium on
Computer Applications
Applications in Medical
Medical Care. 10 pages
pages
KSL
KSL 86-50
(Working
(Working Paper) Ross
Ross D. Shacter
Shatter and David
David E. Heckerman; A
A Backwards
Backwards View
View for
for
Assessment,
Assessment, July 1986.
1986. 6 pages
pages
KSL
KSL 86-51
Barbara
Barbara Hayes-Roth, Bruce Buchanan,
Buchanan, Olivier
Olivier Lichtarge, Michael Hewett, Russ
Russ Altman,
Altman,
James
James Brinkley,
Brinkley, Craig Cornelius, Bruce
Bruce Duncan, and Oleg Jardetzky; PROTEAN:
PROTEAN:
Deriving
Deriving protein structure
structure from constraints,
constraints, March 1986.
1986. To appear
appear in: Proceedings
Proceedings
of
of AAAI
AAAI 1986
1986 21
21 pages
pages
E.
E. H. Shortliffe
Shortliffe
62
Details
Details of
of Technical
Technical Progress
Progress
5P41-RR00785-14
5P41-RR00785-14
KSL 86-52
86-52
KSL
(Working Paper)
Paper) Edward H.
H. Shortliffe,
Shortliffe, M.D.,
M.D., Ph.D; Medical
Medical Expert
Expert Systems: Knowledge
Knowledge
(Working
Tools for
for Physicians,
Physicians, September 1986. 24 pages
pages
Tools
86-53
KSL 86-53
KSL
(Working Paper)
Paper) Edward H. Shortliffe,
Shortliffe, M.D.,
M.D., Ph.D; Medical
Medical Expert
Expert Systems Research
(Working
at Stanford
Stanford University,
University, September
September 1986. 13 pages
pages
at
86-56
KSL 86-56
KSL
(Working Paper)
Paper) Nakul
Nakul P. Saraiya;
Saraiya; AIDE:
AIDE:
(Working
1986. 25 pages
pages
Simulation June 1986.
Simulation
Distributed Environment
Environment for
for Design and
AA Distributed
86-57
KSL 86-57
KSL
(Working Paper) Curtis
Curtis P. Langlotz, Edward H. Shortliffe,
Shortliffe, and Lawrence M. Fagan; AA
(Working
of Decision
Decision Analysis,
Analysis, November
November 1986.
1986. 21
Methodology for
for Computer-Based
Computer-Based Explanation
Explanation of
Methodology
pages
paw
KSL 86-58
86-58
KSL
William J. Clancey; Intelligent
Intelligent Tutoring
Tutoring Systems: A
A Tutorial
Tutorial Survey, September 1986.
1986.
William
Submitted for
for publication
publication in:
in: Collected
Collected papers
papers of
of the International
International Professorship
Professorship in
in
Submitted
Computer Science (Expert
(Expert Systems)
Systems) Universite
Universite de L’
DEtat,
Belgium 43 pages
pages
Computer
Etat, Belgium
KSL 86-60
86-60
KSL
(Working Paper) Alan
Alan Garvey, Michael
Michael Hewett, M. Vaughan Johnson, Robert
Robert
(Working
Schulman, Barbara Hayes-Roth; BBl
BB1 User Manual
Manual - Interlisp
Interlisp Version,
Version, October 1986.
1986.
pages
68 pages
KSL 86-61
86-61
KSL
(Working
Paper) Alan Garvey, Michael
(Working
Michael Hewett, M. Vaughan
Vaughan Johnson,
Johnson, Robert
Schulman, Barbara Hayes-Roth; BB1
BBl User Manual
Manual - Common Lisp
Lisp Version,
Version, October
1986.
1986. 72 pages
pages
KSL
KSL 86-62
86-62
(Working
Limits of
of Debugging via Differential
Differential
(Working Paper) David C. Wilkins;
Wilkins; On the Limits
Modeling,
Modeling, October 1986.
1986. 15
15 pages
pages
KSL
KSL 86-63
86-63
(Working
(Working Paper) David C. Wilkins;
Wilkins; Knowledge Base
Base Debugging Using
Using Apprenticeship
Apprenticeship
Learning
1986. 15
15 pages
pages
Learning Techniques, October 1986.
KSL
KSL 86-64
86-64
(Working
Window-Driven
(Working Paper) Donald E. Henager;
Henager; Window-Driven
March 1986.
1986. 56
56 pages
pages
Object-Oriented
Object-Oriented
Calculator,
Calculator,
KSL
KSL 86-65
86-65
Matthew L. Ginsberg, David E. Smith; Reasoning
Reasoning About Action
Action I: A
4 Possible Worlds
Worlds
Approach, May 1987.
1987. 25
25 pages
pages
KSL
KSL 86-66
86-66
Matthew L. Ginsberg, David E. Smith; Reasoning
Reasoning About Action
Action II:
11: The Qualification
Qualification
Problem, May 1987.
1987. 28
28 pages
pages
KSL
KSL 86-68
86-68
David E. Smith;
Smith; Controlling
Controlling Backward Inference,
Inference, March 1987.
1987. 67
67 pages
pages
KSL
KSL 86-69
86-69
STAN-CS-86-1136.
STAN-CS-86-1136. Harold Brown, Eric Schoen,
Schoen, and
and Bruce
Bruce Delagi; An Experiment
Experiment in
in
KnowledgeBase Signal
Knowledge-Base
Signal Understanding Using Parallel
Parallel Architectures,
Architectures, October
October 1986.
1986. To
To
appear
appear in:
in: Parallel
Parallel Computation
Computation and
and Computers
Computers for
for AI,
AI, I.S.
J.S. Kowalik
Kowalik Editor,
Editor, Kluwer
Kluwer
Publishers.
Publishers. 39
39 pages
pages
63
63
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Details
Details of
of Technical Progress
Progress
KSL
KSL 86-70
86-70
STAN-CS-86-1140.
STAN-CS-86-1140. John E. Laird,
Laird, Allen
Allen Newell, and Paul S.
S. Rosenbloom; Soar: An
An
Artificial
appear in: Artificial
Architecture for
for General Intelligence,
Intelligence, December
December 1986.
1986. To appear
Architecture
Intelligence. 66 pages
Intelligence.
pages
KSL
KSL 86-74
86-74
(Thesis)
(Thesis) Glenn Douglas
Douglas Rennels;
Rennels; A
A Computational
Computational Model
Model of
of Reasoning from
from the
Clinical Literature,
Literature, June
June 1986.
1986. 244 pages
pages
Clinical
KSL
KSL 86-75
86-75
(Journal
(Journal Memo)
Memo) Eric
Eric J.
J. Horvitz;
Horvitz; Toward a Science of
of Expert
Expert Systems, March
March 1986.
1986. 8
pages
paw
KSL
KSL 86-76
86-76
Integrating Diverse Reasoning
M. Vaughan Johnson
Johnson Jr. and Barbara Hayes-Roth; Integrating
Methods
Methods in the BBI
BBl Blackboard
Blackboard Control
Control Architecture,
Architecture, December 1986.
1986. 17 pages
pages
KSL
KSL 87-01
87-01
(Working
David C. Wilkins,
Wilkins, William
William J.
J. Clancey,
Clancey, and Bruce G. Buchanan;
Buchanan;
(Working Paper) David
Knowledge Base
Base Refinement
Refinement Using
Using Abstract
Abstract Control
Control Knowledge, January 1987.
1987. 9 pages
pages
KSL
KSL 87-02
87-02
STAN-CS-87-1146.
STAN-CS-87-1146. Gregory T. Byrd, Russell
Russell Nakano, and Bruce A.
A. Delagi; A
A PointPointto-Point
to-Point Multicast
Multicast Communications
Communications Protocol,
Protocol, January 1987.
1987. 30 pages
pages
KSL
KSL 87-03
87-03
Bruce G. Buchanan;
Buchanan; Artificial
Artificial Intelligence
Intelligence As An Experimental
Experimental Science, January 1987.
1987.
To
To appear
appear in: Synthese 41 pages
pages
KSL
KSL 87-05
87-05
STAN-CS-87-1142.
James
STAN-CS-87-1142.
James F. Brinkley,
Brinkley, Bruce
Bruce G. Buchanan,
Buchanan, Russ
Russ B.
B. Altman,
Altman, Bruce
S.
A Heuristic
Heuristic Refinement
Refinement Method
Method for
for Spatial
Spatial Constraint
Constraint
S. Duncan, Craig W. Cornelius;
Cornelius: A
Satisfaction
Satisfaction Problems, January 1987.
1987. 15 pages
pages
KSL
KSL 87-06
87-06
(Journal
Rennels; A
A Computational
Computational Model
Model of
of Reasoning from
from the
the
(Journal Memo)
Memo) Glenn D. Rennels;
SCAMC Proceedings,
Washington
Proceedings, Washington
Clinical
Clinical Literature,
Literature, January 1987.
1987. To appear
appear in: SCAMC
D.C. 1986
D.C.
I986 8 pages
pages
KSL
KSL 87-07
87-07
STAN-CS-87-1144.
Gregory T. Byrd and Bruce
STAN-CS-87-1144.
Multiprocessor
Multiprocessor Topologies, January 1987.
1987. 6 pages
pages
KSL
KSL 87-08
87-08
Robert
Robert Schulman
Schulman and Barbara Hayes-Roth; ExAct:
ExAct:
January 1987.
1987. 15
15 pages
pages
A. Delagi;
Considerations
Considerations
for
for
A
A Module
Module for
for Explaining
Explaining Actions,
Actions,
KSL
KSL 87-09
87-09
(Working
(Working Paper) Peter D. Karp
Karp and Peter
Peter Friedland;
Friedland; Coordinating
Coordinating the Use of
of
Qualitative
Knowledge in Declarative
Qualitative and Quantitative
Quantitative
Declarative Device
Device Modeling,
Modeling, January 1987.
1987.
20
20 pages
pages
KSL
KSL 87-11
87-11
Alan
Alan Garvey, Craig Cornelius, and Barbara
Barbara Hayes-Roth;
Hayes-Roth: Computational
Computational Costs versus
versus
Benefits of
of Control
Control Reasoning, February 1987.
1987. 13
13 pages
pages
Benefits
KSL
KSL 87-12
87-12
(Working
Paper) William
J.
(Working
William
J. Clancey;
Clancey; The Knowledge Engineer
Engineer as Student:
Student:
Learning
Metacognitive
Metacognitive bases
bases for
for asking good
good questions, January
January 1987.
1987. To
To appear
appear in Learning
Issues for
for intelligent
Heinz Mandl
Mandl and Alan
Alan Lesgold,
Lesgold, editors.
editors.
Issues
Inteiligent Tutoring
Tutoring Systems, Heinz
Springer-Verlag:
Springer-Veriag:
New York
York 30 pages
pages
E. H. Shortliffe
Shortliffe
64
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR00785-14
KSL 87-16
87-16
KSL
(Journal Memo)
Memo) Eric
Eric J. Horvitz;
Horvitz; Inference
Inference under
under Varying
Varying Resource
Resource Limitations,
Limitations,
(Journal
February 1987. 16 pages
pages
February
87-18
KSL 87-18
KSL
Isabelle de Zegher-Geets, Andy
Andy Freeman, Mike
Mike Walker,
Walker, Bob Blum,
Blum, Gio
Gio Wiederhold;
Wiederhold;
Computer-Aided Summarization
Summarization of
of a Time-Oriented
Time-Oriented Medical
Medical Database, February
February 1987. 6
Computer-Aided
pages
pages
KSL 87-19
87-19
KSL
Curt P. Langlotz
Langlotz and Edward H. Shortliffe;
Shortliffe; The Relationship
Relationship between
(Working Paper) Curt
(Working
Decision
Theory
and
Default
Reasoning,
February
1987.
16
pages
Decision Theory
Default
February 1987.
pages
KSL 87-20
87-20
KSL
(Working Paper) Michael
Michael G. Kahn;
Kahn; Model-Based
Model-Based Interpretation
Interpretation of
of Time-Ordered
Time-Ordered Data,
Data,
(Working
pages
March 1987.
1987. 18 pages
March
KSL 87-21
87-21
KSL
(Working Paper)
Paper) Gregory F. Cooper; An
An Algorithm
Algorithm for
for Computing
Computing Probabilistic
Probabilistic
(Working
Propositions, March
March 1987. 5 pages
pages
Propositions,
KSL 87-22
87-22
KSL
(Journal
Memo) Homer
Homer L. Chin
Chin and Gregory
Gregory F. Cooper; Stochastic
Stochastic Simulation
Simulation of
of
(Journal Memo)
Models, March 1987.
pages
1987. 11 pages
Casual Bayesian Models,
KSL 87-23
87-23
KSL
(Working Paper) Thierry
Thierry
(Working
an Immunology
Immunology Database,
Barsalou and
March
March 1987.
1987.
Gio Wiederhold;
Wiederhold; Applying
Applying a Semantic
Semantic Model
Model to
Gio
18 pages
pages
KSL
KSL 87-24
87-24
(Journal
(Journal Memo)
Memo) Homer
Homer L. Chin
Chin and Gregory F. Cooper;
Cooper; Knowledge-Based Patient
Patient
pages
Simulation,
1987. 11
11 pages
Simulation, March 1987.
KSL
KSL 87-25
87-25
(Journal
(Journal Memo)
Memo) Edward H. Shortliffe;
Shortliffe; Computers in Support of
of Clinical
Clinical Decision
Decision
Making,
Making, March 1987.
1987. 12 pages
pages
KSL
KSL 87-32
87-32
(Working
(Working Paper) William
William J.
J. Clancey;
Clancey; Diagnosis,
Diagnosis, Teaching, and
and Learning: An Overview
of
April 1987.
1987. 12
12 Pages
Pages
of GUIDON2
GUIDON2 Research, April
KSL
KSL 87-34
87-34
(Working
(Working Paper) Russell
Russell Nakano; Experiments
Experiments with
with a Knowledge-Based
Knowledge-Based System on a
Multiprocessor:
Qualitative
Multiprocessor: Preliminary
Preliminary AIRTRAC-LAMINA
AIRTRAC-LAMTNA
Qualitative Results. June,
June, 1987.
1987.
KSL-87-35
KSL-87-35
(Working
(Working Paper) Masafumi Minami;
Minami; [Experiments
[Experiments with aa Knowledge-Based
Knowledge-Based System
System on
Quantitative
aa Multiprocessor:
Multiprocessor: Preliminary
Preliminary AIRTRAC-LAMINA
AIRTRAC-LAMINA
Quantitative ReSUlts.]
Results.] June,
June, 1987.
1987.
Other
Other Outside Articles:
Articles:
Hayes-Roth, B.,
and Hewett,
Hewett, M. Applications
Applications of
of BB1
BBl to
B., Johnson,
Johnson, M.V., Garvey,
Garvey, A., and
arrangement-assembly
arrangement-assembly tasks.
tasks. ArtifiCial
Artificial Intelligence
Intelligence in
in Engineering,
Engineering, October,
October, 1986.
1986.
Hayes-Roth, B.,
and Hewett,
Hewett, M. The BB*
BB* environment.
environment. To
B., Johnson,
Johnson, M.V.,
M-V., Garvey,
Garvey, A., and
appear
appear in:
in: R.
R. Engelmore and
and A. Morgan (Eds.),
(Eds.), Blackboard
Blackboard Systems. Addison-Wesley,
London: 1987.
1987.
Garvey,
Garvey, A.,
A., Cornelius,
Cornelius, c.,
C., and
and Hayes-Roth,
Hayes-Roth, B.
E. Computational
Computational costs
costs versus
versus benefits
benefits of
of
65
65
E.
E. H.
H. Shortliffe
Shortliffe
Details of
of Technical Progress
Progress
5P41-RR00785-14
Intelligence:
control
Proceedings of
of the American
American Association
Association for
for Artificial
Artificial
Intelligence:
control reasoning.
reasoning. Proceedings
AAA187,
AAAI87, 1987.
1987.
M. V. Johnson,
Johnson, and
and Hayes-Roth, B.
B. Integrating
Integrating diverse
diverse reasoning
reasoning methods
methods in BBl.
BBl.
Proceedings
for Artificial
Intelligence:
AAA187,
Proceedings of
of the American
American Association
Association for
Artificial
Intelligence:
AAAI87, 1987.
1987.
Hayes-Roth, B.
Encyclopedia of
of Artificial
Artificial
B. Blackboard systems.
systems. In Eckroth,
Eckroth, D. (Ed.), Encyclopedia
Intelligence.
Intelligence. New York:
York: John Wiley
Wiley & Sons,
Sons, 1987.
1987.
Garvey, A., Hewett, M., Johnson,
BBI User's
User’s
Johnson, M.Y.,
M.V., Schulman, R.,
R., and
and Hayes-Roth, B.
B. BBI
Manual.
Stanford
Manual.
Stanford University.
University. 1987.
1987.
Hayes-Roth, B.,
B., Buchanan,
Buchanan, B.G.,
B.G., Lichtarge, 0.,
O., Hewett, M., Altman,
Altman, R.,
R., Brinkley,
Brinkley, J.,
J.,
Cornelius, c.,
C., Duncan, B.,
B., and Jardetzky, O.
0. Elucidating
Elucidating protein
protein structure from
from
constraints
constraints in PROTEAN.
PROTEAN. To be reprinted
reprinted in: R. Engelmore and
and A. Morgan (Eds.),
(Eds.),
Blackboard
Blackboard Systems. Addison-Wesley,
Addison-Wesley, London:
London: 1987.
1987.
Duncan, B.,
B., Buchanan,
Buchanan, B.
B. G., Hayes-Roth, B.,
B., Lichtarge, 0.,
O., Altman,
Altman, R., Brinkley,
Brinkley, J.,
J.,
Hewett, M., Cornelius,
Cornelius, c.,
C., and Jardetzky, O.
0. PROTEAN:
PROTEAN: A
A new method of
of deriving
deriving
solution
Bulletin of
of Magnetic
Magnetic Resonance, 1987.
1987.
solution structures of
of proteins. Bulletin
Hayes-Roth, B.,
B., Buchanan,
Buchanan, B.G.,
B.G., Lichtarge, 0.,
O., Hewett, M., Altman,
Altman, R.,
R., Brinkley,
Brinkley, 1.,
J.,
Cornelius, c.,
C., Duncan, B.,
B., and Jardetzky, O.
0. PROTEAN:
PROTEAN: Deriving
Deriving protein
protein structure
from
for Artificial
Intelligence:
from constraints. Proceedings
Proceedings of
of the American
American Association
Association for
Artificial
Intelligence:
AAAI86, 1986.
1986.
Jardetzky, 0.,
O., Lane, A., Lefevre, J.,
J., Lichtarge, 0.,
O., Hayes-Roth, B.,
B., Altman,
Altman, R., and
and
Buchanan,
Buchanan, B.
B. A
A new method for
for the determination
determination of
of protein
protein structures in solution
solution
from
NMR. In Proceedings
Proceedings of
XXIII Congress Ampere,
Ampere, Rome, Italy,
from NMR.
of the XXIIi
Italy, 1986.
1986.
Tommelein,
Tommelein, 1.I. D., Johnson, M. Y.,
V., Hayes-Roth,
Hayes-Roth, B.,
B., and Levitt,
Levitt, R.E., SIGHTPLAN:
SIGHTPLAN: A
A
Blackboard Expert
Expert System
System for
for the Layout
Layout of
of Temporary
Temporary Facilities
Facilities on a Construction
Construction
Proceedings of
I FI P WG5.2
Expert Systems in ComputerSite. Proceedings
of the IF/P
WGS.2 Conference
Conference on Expert
ComputerAided Design,
Design, Sydney,
Aided
Sydney, Australia,
Australia, 1987.
1987.
Tommelein,
Tommelein, 1.I. D., Levitt,
Levitt, R. E., and Hayes-Roth,
Hayes-Roth, B., Using Expert
Expert Systems
Systems for
for the
Proceedings of
Layout
Layout of
of Temporary
Temporary Facilities
Facilities on Construction
Construction Sites."
Sites.” Proceedings
of the CI
C/BB W-65
W-6.5
Symposium
Management of
Symposium on Organization
Organization and Management
of Construction,
Construction, Birkshire,
Birkshire, U.K,
UK, 1987.
1987.
Shortliffe
E. H. Shortliffe
66
of Technical
Technical Progress
Progress
Details of
Details
5P41-RR00785-14
5P41-RR00785-14
III.A.3.6. Resource
Resource Equipment
Equipment
III.A.3.6.
SUMEX-AIM core facility,
facility, started in
in March
March 1974,
1974, was
was built
built around
around a Digital
Digital
The SUMEX-AIM
Equipment Corporation
Corporation (DEC)
(DEC) KI-10
KI-10 computer
computer and the TENEX
TENEX operating
operating system
system and
Equipment
continued through
through the 1970’
1970'ss with
with a mainframe
mainframe focus for
for the resource.
resource. The interactive
interactive
continued
computing environment
environment of
of this
this facility,
facility, with
with its AI
AI program development
development tools and its
computing
network and interpersonal
interpersonal communication
communication media, was
was unsurpassed
unsurpassed in
in other
other machine
network
environments.
Biomedical scientists found
found SUMEX
SUMEX easy
easy to use
use in
in exploring
exploring
environments.
Biomedical
of developing
developing artificial
artificial intelligence
intelligence programs for
for their
their own work
work and in
in
applications of
applications
stimulating more effective
effective scientific
scientific exchanges
exchanges with
with colleagues
colleagues across
across the country.
country.
stimulating
us access
access to a
through wide-reaching
wide-reaching network
network facilities,
facilities, these
these tools provided
provided us
Coupled through
computer science research
research community,
community, including
including active artificial
artificial intelligence
intelligence and
large computer
system development
development research
research groups.
system
computer system
system research
research on early microprocessors
In the late 1970’
1970'ss and early 1980’
1980's,s, computer
In
minicomputers suggested
suggested that
that large mainframe
mainframe computers would not be
and compact minicomputers
even the dominant
dominant source
source of
of computing
computing power for
for AI
AI research
research and AI
AI
essential or even
essential
began to implement
implement a strategy for
for computing
computing
program dissemination.
dissemination.
Thus, we began
resources marked by the integration
integration of
of heterogeneous
heterogeneous systems
systems --- mainframes,
mainframes, Lisp
resources
systems (e.g.,
(e.g., for
for file
file storage
storage and printing)
printing) all linked
linked together
workstations, and service systems
have configured
configured the optimal
optimal resource
resource
area networks.
years, we have
Over the years,
by local area
computing environment
environment around shared
shared central machines coupled through
through a highhighcomputing
performance network
network to growing
growing clusters of
of personal workstations.
.
performance
of the individual
individual workstation,
workstation, especially with
with the high-bandwidth
high-bandwidth graphics
The concept of
facilities for
for expert
interface, proved ideal.
Both program development tools and facilities
interface,
with a central
system user
user interactions
interactions were substantially
substantially improved
improved over what is possible with
system
time-shared system.
main shortcomings of
of early workstation
workstation systems
systems were
were their
their
system. The main
limited
processing speed
years since our first
first
limited processing
speed and high cost.
cost. But in the few years
experimental systems,
processing power has
systems, processing
has increased
increased by more than a factor
factor of
of 10 and
the cost has
has decreased
decreased by a comparable factor.
factor.
EX resource
Today the SUM
SUMEX
resource is a complex, integrated facility
facility comprised of
of machines,
machines,
networks, and servers
servers illustrated
illustrated in Figures 4 - 8. A
A key role of
of the SUMEX-AIM
SUMEX-AIM
resource
resource is to continue
continue to evaluate
evaluate workstations
workstations as
as the technology is changing rapidly.
rapidly.
This
This evaluation includes new hardware and software, 1) to provide superior
superior development
platforms for
and execution platforms
for AI
AI research,
research, and 2) to support the ancillary
ancillary "office
“office
environment"
phased out).
environment” (presently
(presently carried out
out on the DEC
DEC 2060,
2060, which is being phased
Thus far
far no single workstation
workstation has
has materialized
materialized that provides all the services
services we
we would
like
like to see
see in support
support of
of either or both of
of these
these missions.
missions. This means
means that for
for the
foreseeable
foreseeable future,
future, we
we will
will utilize
utilize a multiplicity
multiplicity of
of machines
machines and
and software to address
address the
the
needs
needs of
of the projects.
Systems
Apple Macintosh
Systems based
based on the Motorola
Motorola 68020
68020 chip
chip (e.g.,
(e.g., SUN Microsystems or Apple
II
II workstations), the Intel
Intel 80286
80286 and 80387
80387 chips (e.g.,
(e.g., IBM
IBM PS/1-4
PS/l-4 machines),
machines), and
other newer architectures, such
such as
as reduced
reduced instruction
instruction set
set computer (RISC) chips, have
have
Lisp benchmark data rivaling
rivaling the performance
performance of
of existing, specially microcoded Lisp
machines
machines (e.g.,
(e.g., those
those from
from Xerox, Symbolics, and TI).
TI). But these
these Lisp machine vendors
vendors
are
are producing substantially
substantially faster machines
machines as
as well, using
using VLSI technology. It
It is still
still
too early to predict
predict how this "race"
“race” will
will ultimately
ultimately turn out and software environments
environments
will
will play an
an equally important
important role to raw hardware
hardware speed
speed in the decision. For now,
the Lisp software environments
environments on the "stock"
“stock” machines
machines are
are not nearly so
so extensively
developed
developed as
as on Lisp machines
machines and conversely,
conversely, the routine
routine computing
computing environments
environments of
of
Lisp machines
machines (text processing,
processing, mail, spreadsheets,
spreadsheets,etc.)
etc.) lag
lag the tools available on stock
UNIX
UNIX machines.
machines.
In the past
past year
year we
we experimentally
experimentally tried increasing usage
usage of
of TI
TI and
and Xerox Lisp
67
67
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
of Technical
Technical Progress
Progress
Details of
as AI
AI research
research platforms)
platforms) for
for text
text editing
editing and document
document
machines (purchased as
formatting, but
but their
their functionality
functionality and speed
speed do not
not approach that
that of
of the TeX
TeX and
formatting,
The Lisp
Lisp machines do not
not yet provide
provide
Scribe formatters
formatters when executed
executed on the 2060. The
Scribe
of the 2060 (e.g.
(e.g. mail,
mail, database,
database, spreadsheet,
spreadsheet, dictionary),
dictionary), but
but we and
myriad tools of
the myriad
to rewrite
rewrite mainframe
mainframe tools to address
address the most pressing
undertaking to
other groups are undertaking
of office
office tools on these
these machines
shortcomings. Another
Another problem
problem is that
that execution of
as research
research tools. Improvements
Improvements in
in processor
processor speed,
speed, memory
memory size,
their utility
utility as
impacts their
size, and window
window systems
systems may address
address this
this problem
problem in
in the near future.
future. We look
look
display size,
of the TI
TI Explorer
Explorer I1
Il and Xerox
Xerox Tamarind
Tamarind
forward to the introduction
introduction and testing of
forward
with this in
in mind.
mind.
with
Some community
community members tried
tried increasing their
their usage
usage of
of Macintosh
Macintosh applications
applications as
as a
Some
of reducing dependence
dependence on the 2060, but
but except for
for some drawing
drawing tools, they
means of
means
not up to the job,
job, hampered by the small display
display and incapacity
incapacity to cope with
with large
were not
or complex documents. We look
look forward
forward to investigating
investigating the much more
more powerful
powerful
or
as an office
office system
system and possibly Lisp
Lisp delivery
delivery vehicle. Early
Early indications
indications
Macintosh IIII as
for Lisp
Lisp development, but perhaps
perhaps mass
mass availability
availability will
will
suggest limited
limited potential
potential for
suggest
encourage improvement
improvement in this area.
area.
encourage
see an integration
integration of
of both the Lisp
Lisp machine and
In the long term, we may hope to see
In
of the present single-vendor
single-vendor offerings,
offerings,
stock machine worlds. Despite the inadequacy
inadequacy of
of Lisp
Lisp machine technology for
for office
office systems
systems ancillary
ancillary to
potential leverage
leverage of
the potential
of combining
combining the two as
as attractive
attractive as
as ever, and we intend
intend to
research makes
makes the pursuit
pursuit of
research
of new hardware opportunities
opportunities as
as they arise.
arise.
advantage of
take advantage
1 - Purchases
Purchases This
This Past Year
The core resource
resource hardware continues to be stable and the relatively
relatively small amount
amount of
of
SUMEX-AIM
purchases has
been concentrated on experimental
SUMEX-AIM money for
for new purchases
has been
experimental
workstations and server equipment needed
These
needed for
for distributed
distributed system
system development. These
purchases are paced
paced carefully
performing, more
purchases
carefully with
with the developments of
of higher performing,
purchases this past
past year
compact, and lower cost systems.
year are summarized below.
systems. The purchases
purchases in many cases
It
It should be noted that these
these purchases
cases complement
complement hardware acquired
with
including 3 SUN 3175
3/75 workstations,
workstations, a SUN 31180
31180 file
file server,
server,
with non-NIH
non-NIH funding,
funding, including
printer upgrades.
and numerous laser
laser printer
upgrades.
1. SUN X-50lB
X-501B 75 Megabyte
Megabyte Disk
Disk Drives
Drives (3 each,
each, for
for Lisp
Lisp workstations)
workstations)
2.
2. Sun
Sun 6250
6250 BPI
BP1 Tape Drive
Drive (for
(for file
file server
server backup)
3.
3. Parity
Parity 24-Megabyte memory boards
boards (3 each,
each, for
for Lisp Workstations)
Workstations)
4.
4. Apple
Apple Macintosh SE
SE computer (for
(for text processing
processing and graphics)
5.
5. Apple
Apple Mac II
II computer (40 Megabyte
Megabyte disk, 7 MB memory upgrade,
upgrade, and
video card/monitor;
card/monitor; for
for a Lisp workstation)
workstation)
6. Imagen
Imagen 3320-3 laser
laser printer
printer (for
(for higher volume printing)
printing)
7.
7. Ricoh 4120
4120 laser
laser printer
printer (used,
(used, for
for spare
spare parts)
8.
8. Toshiba TllOO
TllOO Plus
Plus Portable Computer (as
(as a portable travel computer)
computer)
9.
9. Ethernet (10MB bits) Multibus
Multibus Interface
Interface Boards
Boards (4 each,
each, for
for
expansions)
expansions)
network
network
10.
10. U.S.
U.S. Robotics 9600
9600 baud
baud modems
modems (2 each,
each, for
for higher speed
speed serial line
connections)
E.
E. H. Shortliffe
Shortliffe
68
68
of Technical
Technical Progress
Progress
Details of
5P41-RR00785-14
1.1 - Workstation
Hardware
Workstation Hardware
Using non-DRR
non-DRR funding,
funding, the KSL
KSL has
has taken delivery
delivery of
of 20 new Xerox
Xerox 1186
1186 LISP
workstations, and has
has upgraded
upgraded 4 Xerox
Xerox 1108
1108 machines to 1109
1109 (Dandetigers) with
with
used by many projects
memory expansion
expansion and floating
floating point
point support. The machines are used
in the KSL, including
NEOMYCIN efforts,
including the GUIDON
GUIDON and NEOMYCIN
efforts, BB1,
BBl, PROTEAN,
PROTEAN, and
These machines increase
increase our research
research
Financial
These
Financial Resources
Resources Management (FRM).
(FRM).
capabilities and complement
complement the Texas
Texas Instrument
Instrument Explorers
Explorers and Symbolics 36XX
36XX
facilities of
of the KSL.
facilities
justified our
Our Xerox workstations proved to be very reliable
reliable again
again this year
year and justified
strategy
strategy of
of saving money by not
not purchasing service
service contracts. Also to save
save money, we
we
arranged
arranged with
with third
third parties to repair/replace
repair/replace some
some components that did fail.
fail.
(Exception: We purchased
purchased a third-party
third-party service contract
contract on one Xerox 1132
1132 disk drive
drive
since the particular
particular device has
has failed
failed more than once.)
The basic
basic components of
of the three Sun
Sun 3175
3175 workstations
workstations were
were purchased
purchased with
with DARPA
DARPA
funding
funding for
for evaluation as
as AI
AI development engines
engines and/or
and/or office
office systems.
systems. Although
Although Sun
Sun
recommends
recommends these
these machines
machines as
as general
general purpose
purpose workstations,
workstations, experience
experience indicated that
memory and disk upgrades
upgrades to the basic systems
systems are necessary
necessary to consider their
their use
use as
as
Lisp engines.
engines. These
These upgrades
upgrades are on-order
on-order and evaluation
evaluation is still
still in the early stages.
stages.
Server Hardware
1.2 - File
File Server
Hardware
Because
Because our Lisp
Lisp workstations
workstations have
have only
only limited
limited local file
file space,
space, the development of
of
SUMEX now has
has
effective
effective shared
shared file
file servers
servers is essential
essential to our resource
resource operation.
operation. SUMEX
-based file
AXI7 50's
three UNIX
UNIX-based
file servers.
servers. Two of
of them, as
as reported in the past,
past, use
use V
VAX/750’
s as
as
the processors:
processors: the SAFE has
has four,
four, 470 Megabyte,
Megabyte, Fujitsu
Fujitsu Eagle
Eagle disk drives and the
ARDVAX
ARDVAX has
has one such
such disk drive. The SAFE also is equipped with
with a 300
300 megabyte
megabyte
CDC, removable media, disk drive
drive and a 80011600
800/1600 BPI
BP1 Kennedy tape
tape drive. The CDC
unit
unit is used
used for
for incremental
incremental backup dumps and the tape drive
drive is used
used for
for both
incremental and full
been established
A procedure has
has been
established whereby the
full backup dumps. A
ARDVAX
ARDVAX is able to use
use this equipment
equipment for
for its incremental
incremental and full
full dumps over the
network. The configurations
configurations of
of these
these systems
systems are shown in Figure 7.
7.
With
With DARPA
DARPA funding
funding this past year we bought a system
system called the KNIFE,
KNIFE, a file
file
server
processor. It
It is equipped with
with two of
of the 470 Megabyte
Megabyte
server based
based on a SUN 31180
3/180 processor.
Fujitsu
Fujitsu disk drives and a cartridge
cartridge tape drive
drive (see
(see Figure 5). We are in the process
process of
of
relatively new,
new,
adding a Fujitsu
Fujitsu 1600/6250 BPI
BP1 tape drive
drive for
for backup dumping.
dumping. Being relatively
the performance of
of this equipment
equipment in an operational
operational environment
environment has
has not yet been
been
thoroughly checked
checked out at SUMEX.
SUMEX.
The Xerox XNS Ethernet-based file
file server
server (donated by Xerox
Xerox in 1985) has
has increased
increased in
This server
server is based
based on the Xerox 8000
8000 processor
capacity and usage
processor
usage in the past year.
year. This
(identical
1108 Lisp
Lisp workstation
workstation but
but running
running more conventional
conventional
(identical hardware
har d ware to the Xerox n08
microcode) and the Century Data Systems
Systems T-305
T-305 removable media disk drive. With
With the
addition
potential storage
addition of
of two additional
additional disk drives (also donated), the total potential
storage capacity
of
of the server
server has
has increased
increased to approximately
approximately 900 MB
MB (of
(of which
which 600 MB is currently
currently
available from
from the network).
The user
user base
base for
for this server has
has grown to over sixty
sixty regular, registered
registered users
users and
This server is the primary
system software
numerous
primary system
numerous infrequent
infrequent guest
guest and project users.
users. This
year, the server
server software has
has been
been
resource
resource for
for over fifty
fifty Lisp
Lisp workstations. In the past year,
upgraded
upgraded twice, the most recent upgrade
upgrade introduced
introduced random access
access to the content
content of
of
files
files which, when
when interfaced
interfaced to Interlisp's
Interlisp’s paged
paged file
file mechanisms,
mechanisms, should improve
improve both
flexibility and effective
effective speed
speed of
of the server.
server.
the flexibility
their earlier-announced
earlier-announced potential,
Though optical
potential,
optical disks have
have been
been slow in realizing
realizing their
69
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
suitably packaged
packaged products are now appearing in
in the marketplace. ItIt is possible that
that
suitably
this technology used
used in
in place of
of (or
(or in
in conjunction
conjunction with)
with) conventional
conventional magnetic tapes
tapes
this
might provide
provide an excellent medium for
for implementing
implementing a responsive offline
offline storage
might
for data. ItIt is fair
fair to expect that
that even a small
small laboratory
laboratory could have reasonable
system for
system
access to hundreds of
of gigabytes
gigabytes of
of storage.
storage.
access
Printer Hardware
Hardware
1.3 - Printer
year, we purchased
purchased 2 new Imagen 12/300’
12/300's,
upgraded an 8/300
8/300 to a
Over the past year,
s, upgraded
12/300, and converted an old Hewlett-Packard
Hewlett-Packard 2688A to a 12/300
12/300 laser printer
printer for
for the
121300,
SUMEX-AIM community.
community. These
These enhancements
enhancements were funded by DARPA.
DARPA. The move to
SUMEX-AIM
12/300'ss was
was motivated primarily
primarily by the ruggedness
ruggedness of
of the Ricoh
Ricoh LP-4120
LP-4120 print
print engine
12/300’
used in
in those printers.
printers. Whereas
Whereas the Canon LBP-CX
LBP-CX print
print engine used
used in
in the 8/300
8/300 has
has
used
of 70,000
70,000 pages,
pages, the Ricoh
Ricoh LP-4120
LP-4120 has
has an expected
expected lifetime
lifetime of
of
expected lifetime
lifetime of
an expected
700,000 pages.
pages. Other beneficial
beneficial side-effects
side-effects of
of the upgrade
upgrade were:
were: (1) higher print
print rate
700,000
(half a ream), (3) blacker and more solid
solid
pages-per-minute), (2) bigger paper tray (half
(12 pages-per-minute),
cheaper supplies (half
(half the price per page
page compared to
print, (4) crisper print,
print, and (5) cheaper
print,
the 81300).
8/300).
Apple Laser Writer
Writer which interprets
interprets the PostScript page
page
We have also acquired an Apple
Within a few months of
of its introduction,
introduction, the Apple
Apple Laser Writer
Writer
description language.
language. Within
description
has become
become the most common laser
laser printer
printer on campus
campus and around the world.
world.
has
of scale
scale have
have made
made itit possible for
for us
us to acquire this printer
printer for
for under $4000.
$4000.
Economies of
AppleNet/Ethernet expertise will
will make itit possible for
for us
us to attach the Laser
SUMEX AppleNet/Ethernet
SUMEX
Writer to the high-bandwidth
high-bandwidth campus
campus internet
internet and operate the printer
printer at the high-end
high-end
Writer
of its 8 page-per-minute
page-per-minute capacity. (The vast majority
majority of
of laboratory-owned
laboratory-owned Laser
of
Writers
pagesWriters in the U.S.
U.S. are driven over a low-bandwidth
low-bandwidth RS-232 line
line yielding
yielding only
only 3 pagesper-minute
per-minute throughput
throughput and typically
typically greater
greater latency.) The PostScript page
page description
description
language
language is already the standard of
of choice at university
university and DARPA
DARPA sites
sites (judging
(judging by
It is generally agreed
agreed upon in these
these
traffic
traffic on the Laser-Lovers discussion
discussion group). It
communities
communities that PostScript is among the easiest-to-generate
easiest-to-generate and most expressive
expressive of
of the
page
page description
description languages
languages in use
use today and reconciles these
these traits much more
effectively
effectively than other languages
languages do.
At
printer
At present,
present, most of
of our printers
printers image at 300
300 dots per inch (dpi)
(dpi) and our finest
finest printer
exploit the special
is the aging Xerox AItoRaven which images
Alto-Raven
images at 384
384 dpi. To exploit
capabilities
capabilities of
of much higher quality,
quality, camera-ready printers
printers and to take advantage
advantage of
of the
economical Apple
Apple Laser
Laser Writer,
Writer, we have
have begun
begun an Interlisp
Interlisp implementation
implementation of
of an
"image
UNILOGIC has
has already added
added Postscript support
support
“image stream"
stream” driver
driver for
for PostScript. UNILOGIC
to Scribe and Adobe has
has implemented Postscript support for
for TeX.
1.4 - Network
Network Hardware
Hardware
As we
we evolved a more complex network topology and decided
decided to compartmentalize
compartmentalize the
overall Stanford
Stanford internet
internet to avoid electrical interactions
interactions during
during development and to
facilitate
facilitate different
different administrative
administrative conventions for
for the use
use of
of the various networks, we
we
developed
developed gateways
gateways to couple subnetworks
subnetworks together using
using Motorola
Motorola MC-68000
MC-68000 systems.
systems.
Given
Given the heterogeneity
heterogeneity of
of our environment,
environment, these
these gateways
gateways continually
continually need
need to provide
provide
additional
additional services
services to support the influx
influx of
of new
new workstations. To accommodate
accommodate current
current
and
and anticipated gateway
gateway software growth, we
we have
have increased
increased the memory capacity of
of the
MC-68000 cpu board from
from 256
256 kilobytes to 1 megabyte.
megabyte.
We also
also developed
developed a MC-68000 terminal
terminal interface processor
processor (TIP)
(TIP) to provide terminal
terminal
access
It is basically a machine that has
has a number of
of
access to network hosts
hosts and
and facilities.
facilities. It
terminal
terminal lines and
and a network interface and
and software to manage
manage the
the establishment of
of
connections for
for each
each line and
and the flow
flow of
of characters
characters between
between the
the terminal
terminal and
and host.
host. In
E. H. Shortliffe
Shortliffe
70
70
5P41-RR00785-14
Details of
of Technical Progress
Progress
the past,
past, 32
32 lines
lines per TIP
TIP was
was sufficient,
sufficient, but our transltlOn
transition plan for
for moving
moving users
users off
off
2060 includes moving
moving both the dial-in
dial-in and dial-out
dial-out functionality
functionality of
of the 2060
2060 to
the 2060
TIPS, and this year
year we upgraded
upgraded one of
of our TIPs
TIPS to support 10 such
such ports. Thus, the 32
32
TIPs,
line upper bound is no longer feasible, and there
there is now the need
need to configure
configure TIPs
TIPS
line
with
with at least 48, and perhaps
perhaps 64 lines. As with
with the gateways,
gateways, we
we have
have quadrupled the
size of
of the TIPs'
TIPS’ MC-68000
MC-68000
megabyte. This will
memory size
cpu board to 1 megabyte.
will adequately
adequately
future expansion of
of these
these servers.
servers. We have
have also
also improved
improved the Dial
Dial
handle any future
IN/OUT service for
for both the 2060
2060 and Tips for
for faster operation (2400 baud service
IN/OUT
maximum).
maximum).
SUMEX-AIM is continuing
continuing its efforts
efforts in improving
improving the networking
networking environment
environment for
for
SUMEX-AIM
unified data communications. In this report
report period, several
several
faster and more unified
reconfigurations towards
towards this endeavor
endeavor have
have been
been completed. The SUMEX-AIM
SUMEX-AIM facility
faciIity
reconfigurations
has been
been relocated
relocated to a new building.
building. This move necessitated
has
necessitated the relocation
relocation of
of all
offices as
as well as
as all associated
associated computer equipment. A
A network
network in the new building
building
offices
designed and implemented and coupled
coupled into
into the old one which connects
connects with
with
had to be designed
remaining KSL groups as
as well as
as the Stanford campus
campus proper. This
This modification
modification
the remaining
gave us
us the opportunity
opportunity to upgrade
upgrade several
several portions
portions of
of the network in
in. a manner that
gave
will provide
provide redundancy as
as well as
as future
future expansion
expansion capabilities
capabilities to the Medical
Medical Center
will
adjacent buildings. The new
and all other planned adjacent
new facility
facility was
was wired to provide
provide every
sitting space
space with
with a flexible
flexible network connect capability
capability similar
similar to a telephone type
sitting
connection. The entire scheme
scheme was
was successfully
successfully implemented with
with very little
little downtime.
downtime.
connection.
After almost a year
year in operation this scheme
scheme seems
seems to be very reliable.
After
71
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
SP41-RR00785-14
Details of
of Technical Progress
Progress
DEC
Central
Central
2M words
words of
DI820
Dl820
1/0
I10 bus
bus
I
I
RH20
RH20
1
I
MassBus
MassBus
I
RH20
RH20
RH20
RH20
I
MassBus
MassSus
DEC RP07
Disk Drive
and Controller
Controller
2 DEC
DEC TU-78
TU-78
Tape
Tape Drives
Drives
and
and Controller
Controller
r-
UNIBUS
MassBus
MassBus
-
DEC RP06
Disk Drive
and
and Controller
Controller
Console TTY
-
KUNIK Line
-
r-
Logging TTY
DEC
DEC LP-26
l-P-26
line
Line Printer
Printer
lc
11
140 Front
1 l/40
Front End
RH20
MassBus
MassBus
DEC
DEC RP07
RP07
Disk
Disk Drive
Drive
and Controller
Controller
-
KL10-E
KLlO-E
Processor
Processor
memory,
memory, Cache
Cache
El-
-
-
-
6 DEC DH-11
OH-1 1
line
Line Scanners
Scanners
96
96 lines
lines total
total
lr
Telenet
Interface
Mbit
33 Mbit
I-
-
DEC AN20
ARPAnet
Interface
t:?
MEIS
MassBus
Ethernet
Interface
~
'"
Figure 4:
4:
Figure
E. H. Shortliffe
Shortliffe
SUMEX -AIM DEC
DEC 2060 Configuration
Configuration
SUMEX-AIM
72
72
8 lines
lines
9.6 Kbit
Details
Details of
of Technical
Technical Progress
Progress’
5P41-RR00785-14
5P41-RR00785-14
-
Sun
Sun 3/180
3/180
Central Processor
Processor
4 Mbytes
Mbytes Memory
Memory
~
Console
TTY
Ethernet
Interface
~
10 Mbit
VMEbus
VMEbus
~
-
SCSI
SCSI
Controller
Controller
Disk
Controller
Figure 5:
5:
Figure
Cartridge
Cartridge
Tape
Tape Drive
Drive
Fujitsu
Eagle
Fujitsu Eagle
414 Mbyte
Mbyte
414
Disk Drive
Drive
Disk
/
'"
Fujitsu Eagle
Eagle
Fujitsu
414 Mbyte
Mbyte
414
Disk Drive
Drive
Disk
SUM EX-AIM Sun
Sun File
File Server
Server Configuration
Configuration
SUMEX-AIM
73
73
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Details
Details of
of Technical
Technical Progress
Progress
Xerox
Xerox 8000
8000
Central
Central Processor
Processor
1.5
1.5 Mbytes
Mbytes Memory
Memory
X,Y
X,Y buss
busseses
lOP
IOP bus
bus
,....jpiiqe
,....-
Ethernet
Interface
Intel
Intel 8085A
8085A
1/0
I/O Processor
Processor
1
166 Kbytes
Kbytes Memory
Memory
HSIO-L
Disk
Disk Controller
Controller
Figure
Figure 6:
6:
E.
E. H.
H. Shortliffe
Shortliffe
~
10 Mbit
~
Console
Console
TTY
-KY
V
'"
V
~
Shugart
SA1000
Floppy
Floppy Disk
Disk Drive
Drive
,
CDS
CDS Trident
Trident
300
300 Mbyte
Mbyte
Disk
Disk Drive
Drive
II
CDS
CDS Trident
Trident
300
300 Mbyte
Mbyte
Disk
Disk Drive
Drive
1I
CDS
CDS Trident
Trident
300
300 Mbyte
Mbyte
Disk
Disk Drive
Drive
I
CDS
CDS Trident
Trident
300
300 Mbyte
Mbyte
Disk
Disk Drive
Drive
SUMEX-AIM
SUMEX-AIM Xerox
Xerox File
File Server
Server Configuration
Configuration
74
Details
Details of
of _Technical
-Technical Progress
Progress
SP41RR0078S-14
5P41-RR0078514
DEC
DEC VAX
VAX 111750
1 l/750
Central
Central Processor
Processor
22 Mbytes
Mbytes Memory
Memory
Fujitsu Eagle
414 Mbyte
Disk Drive
Fujitsu Eagle
414 Mbyte
Disk
Disk Drive
Drive
_
~
V
Emulex
Disk Controller
-
TU 58
Tape Drive
TTY
Emulex
Emulex
Controller
Controller
~
Fujitsu
Fujitsu Eagle
Eagle
414 Mbyte
Disk Drive
Console
UNI8US
UNIBUS
Mass8us
Mass&s
Fujitsu Eagle
414 Mbyte
Disk
Disk Drive
Drive
......
~
DEC RK07
Disk
Disk Controller
Controller
and Drive
~
Ethernet
UNIBUS
Interface
t-
DZ-11
Line Scanner
8 lines
r-
CDC
CDC 256
256 Mbyte
Mbyte
Removable-Media
Removable-Media
Disk
Disk Drive
Drive
~
Kennedy
Kennedy
800/1600
800/l 600 bpi
bpi
Tape
Tape Drive
Drive
~
Console
TIY
DEC VAX
VAX 11/750
111750
DEC
Central Processor
Processor
Central
Mbytes Memory
Memory
22 Mbytes
Mass8us
MassEus
I
~
Fujitsu Eagle
Eagle
Fuiitsu
414 Mbyte
Mbyte
414
Disk Drive
Drive
Dlsk
Figure 7:
7:
Figure
Emulex
Emulex
Disk Controller
Controller
Disk
TU 58
Tape Drive
UNIBUS
UNIEUS
DEC RK07
Disk Controller
and Drive
DZ-11
Line Scanner
8 Lines
SUMEX-AIM VAX
VAX File
File Server
Server Configuration
Configuration
SUMEX-AIM
75
75
E. H.
H. Shortliffe
Shortliffe
E.
I
P
5P41RR0078S-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
Margaret Jacks
Jacks Hall
Margaret
Logic
Logic Group
Group
Electrical
Electrical
Engineering
Englneermg
Pine
Plne Hall
CSU
Score 2060
2060
Score
Symbolics
Symbolics 3600'5
3600’s
Xerox 1100's
1100’s
Xerox
Ether
Ether TlP
TIP
Xerox
Xerox laser
laser printers
printers
Other CSD
CSD EQUipment
Equipment
Other
Medical School
School Office
Office Building
Building
Medical
Symbolic Systems
Systems Resources
Resources Group
Group
Symbolic
Medical Computer
Computer Science
Science Group
Group
Medical
Xerox
Xerox 1100'8
1100’3
H-P
H-P 9836's
9836’s
Tl Explorers
Explorers
Imagen
lmaqen laser
laser printers
printers
Xerox
Xerox laser
laser printer
printer
Suns
Suns
Sun file
file server
server
Sun
Apple
Aoole Macintoshes
Macintoshes
Apple
Aoole laser
laser printer
printer
Ether TIPs
TIPS
Ether
Medical Center
Center
Medical
SUMEX
SUMEX Machine
Machine Room
Room
SUMEX
SUMEX 2060
2060
SUMEX 2020
2020
SUMEX
Xerox '108
1108
Xerox
Xerox 6037
6037 file
tile server
senfer
Xerox
Vax 750
750 file
file server
server
Vax
Whekm Building
Building (Welch
(Welch Road)
Road)
Whelan
and HEUX
HEW(
HPP and
Xerox 1185
1185
Xerox
printer
lmagen laser
laser printer
Imagen
oI
R
Repeater
Repeater
@]
clG
Gateway
Gateway
I
Xerox 1100's
TI Explorers
Symbolics 3600's
Silicon Grapnics
Grapnics Iris
Iris
Silicon
Sun
Sun
Vax 750
750 file
file server
server
Vax
Imagen
lmagen laser
laser printers
printers
Apple
Apple Macintoshes
Macintoshes
Apple
Apple laser
laser printer
printer
Ether
Ether TIPs
TIPS
Xerox Alto
Xerox 1132
Xerox laser printer
Figure 3:
E. H. Shortliffe
Shortliffe
SUMEX-AI:v1
SUMEX-AIM EtherNet
EtherYet Configur2.tion
Configuration
76
7n
5P41-RR00785-14
5P41-RR0078514
Details of
of Technical
Technical Progress
Progress
Details
III.A.3.7. Training
Training Activities
Activities
III.A.3.7.
SUMEX resource exists to
to facilitate
facilitate biomedical
biomedical artificial
artificial intelligence
intelligence applications.
applications.
The SUMEX
part of
of the
the facility
facility and staff
staff has
has been
been aa unique
unique feature
feature of
of
This user
user orientation
orientation on
on the part
This
our
resource
and
is
responsible
in
large
part
for
our
success
in
community
building.
our resource
in
part for our success in community building.
staff has spent significant
significant effort
effort in
in assisting users
users gain
gain access
access to
to the
resource staff
The resource
resource system
system and use
use itit effectively
effectively as well
well as in
in assisting AIM
AIM projects in
in
central resource
SUMEX experience. We
We have
their own local
local computing
computing resources
resources based
based on SUMEX
designing their
effort to
to develop,
develop, maintain,
maintain, and
and facilitate
facilitate access
access to
to documentation
documentation
also spent
spent substantial
substantial effort
also
and interactive
interactive help facilities.
facilities. The HELP
HELP and Bulletin
Bulletin Board subsystems
subsystems have been
effort to help users
users get familiar
familiar with
with the computing
computing environment.
environment.
important in
in this
this effort
important
regularly accepted
accepted a number
number of
of scientific
scientific visitors
visitors for
for periods of
of several
We have regularly
us to
to learn
learn the
the techniques
techniques of
of expert
expert system
system definition
definition
months to
to aa year,
year, to
to work
work with
with us
months
and building
building and to collaborate
collaborate with
with us on specific
specific projects.
Our ability
ability to
Our
accommodate such visitors
visitors is severely limited
limited by space,
space, computing,
computing, and manpower
accommodate
of our
our on-going
on-going research.
research.
resources to support
support such visitors
visitors within
within the demands
demands of
resources
of graduate students
students is an essential part
part of
of the research
research and
Finally, the training
training of
Finally,
of the KSL.
KSL. Based
Based largely on the SUMEX-AIM
SUMEX-AIM community
community
educational activities
activities of
environment, we have initiated
initiated two unique, special academic degree
degree programs at
environment,
Stanford, the Medical Information
Information Science
of Science
Science in
in AI,
AI, to
Science program and the Masters of
increase the number of
of students we produce for
for research
research and industry.
industry. A
A number
number of
of
increase
students
are
pursuing
interdisciplinary
programs
and
come
from
the
Departments
of
stildents
interdisciplinary
from
Departments of
Education, and’
and'Medicine.
Engineering, Mathematics, Education,
Medicine.
Medical Information
Information Sciences
Sciences (MIS)
of the most
The Medical
(MIS) program continues to be one of
resource. The MIS
obvious signs
signs of
of the local academic
academic impact
impact of
of the SUMEX-AIM
SUMEX-AIM resource.
program received
as an innovative
innovative
received recent University
University approval (in
(in October 1982) as
individuals with
with a career
career
training
training program that offers
offers MS and PhD degrees
degrees to individuals
of medicine. In
commitment
commitment to applying
applying computers and decision sciences
sciences in the field
field of
Spring 1987,
1987, a University-appointed
University-appointed review group unanimously
unanimously recommended that the
for
another
five
years.
degree
program
be
continued
training program is
degree
for
five years. The MIS training
based
based in the School
School of
of Medicine, directed by Dr. Shortliffe,
Shortliffe, co-directed
co-directed by Dr. Fagan,
Fagan,
and
and overseen
overseen by a group of
of six University
University faculty
faculty that includes two faculty
faculty from
from the
Knowledge
Knowledge Systems
Systems Laboratory
Laboratory (Profs. Shortliffe
Shortliffe and Buchanan). It
It was
was Stanford's
Stanford’s
active on-going
research
in
medical
computer
science,
plus
a
world-wide
on-going research
science,
world-wide reputation
reputation for
for
the excellence
excellence and rigor
rigor of
of those
those research
research efforts,
efforts, that persuaded
persuaded the University
University that the
field
field warranted
warranted a new academic
academic degree
degree program in the area.
area. A
A group of
of faculty
faculty from
from
the medical
medical school
school and
and the computer science
science department argued
argued that research
research in medical
computing has
has historically
historically been
been constrained by a lack of
of talented individuals
individuals who have
have
a solid footing
footing in both the
the medical and computer science
science fields. The specialized
specialized
curriculum
curriculum offered by the new
new program is intended to overcome the limitations
limitations of
of
previous
previous training
training options. It focuses
focuses on the development of
of a new
new generation of
of
researchers
researcherswith aa commitment
commitment to developing new
new knowledge
knowledge about optimal
optimal methods
methods for
for
developing
developing practical computer-based solutions to biomedical
biomedical needs.
needs.
The program
program accepted
accepted its first
first class
class of four
four trainees
trainees in the summer of
of 1983
1983 and
and has
has
now reached
reached its steady-state
steady-state size
size of approximately
approximately twenty graduate
graduate students.
students. We do not
wish
wish to
to provide too
too narrow
narrow aa definition
definition of
of what
what kinds
kinds of prior
prior training
training are
are pertinent
pertinent
because
because of
of the
the interdisciplinary
interdisciplinary nature of
of the
the field.
field. The program has
has accordingly
encouraged
encouragedapplications
applications from
from any
any of
of the
the following:
following:
•. medical
medical students
students who
who wish
wish to
to combine
combine MD
MD training
training with
with formal
formal degree
degree work
and
and research
research experience
experience in
in MIS;
MIS;
•. physicians
physicians who
who wish
wish to
to obtain
obtain formal
formal MIS
MIS training
training after
after their
their MD
MD or
or their
their
77
77
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
residency, perhaps
perhaps in
in conjunction
conjunction with
with a clinical
clinical fellowship
fellowship at
at Stanford
Stanford
residency,
Medical Center;
Medical
or BS graduates
graduates who
who have
have decided on
on a career applying
applying computer
computer
recent BA
BA or
.• recent
science in
in the medical world;
world;
science
current Stanford
Stanford undergraduates
undergraduates who wish to
to extend their
their Stanford
Stanford training
training
.• current
extra year in
in order
order to
to obtain
obtain a “co-terminus”
"co-terminus" MS
MS in
in the MIS
MIS program:
program;
an extra
PhD graduates
graduates who wish post-doctoral
post-doctoral training,
training, perhaps with
with the
.• recent PhD
formal MS credential,
credential, to
to complement
complement their
their primary
primary field
field of
of training.
training.
formal
In addition,
addition, a special one-year
one-year MS program is available
available for
for established academic
In
their computing
computing and statistical
statistical skills
skills
researchers who may wish to augment their
medical researchers
during a sabbatical break. As of
of Spring
Spring 1987, half
half our
our trainees have previously
previously received
during
MD degrees
degrees and another quarter are medical students enrolled
enrolled in
in joint
joint degree
degree programs.
MD
One-third are candidates
candidates for
for the MS degree,
degree, while
while the rest are doctoral
doctoral students. The
The
One-third
has three graduates
graduates to date,
date, with
with several
several more expecting to complete degrees
degrees
program has
of 1987.
1987.
before the end of
for the special one-year MS mentioned
mentioned above,
above, all
all students spend a minimum
minimum of
of
Except for
two years
years at Stanford
Stanford (four
(four years
years for
for PhD students) and are expected
expected to undertake
significant research
research projects for
for either
either degree.
degree. Research
Research opportunities
opportunities abound, however,
significant
of course
course include the several
several Stanford
Stanford AIM
AIM projects as
as well as
as research
research in
and they of
formal statistical
statistical approaches
approaches to medical decision making, applied
applied
psychological and formal
instrumentation, large medical databases,
databases, and a variety
variety of
of other applications
applications projects at
instrumentation,
campus. Several
Several students are already contributing
contributing
the medical center and on the main campus.
in
research described
in major ways
ways to the AIM
AIM projects and core research
described elsewhere
elsewhere in this annual
report.
We are pleased
pleased that the program already has
reputation and is attracting
has an excellent reputation
attracting
superb candidates
program'ss visibility
reputation is due
visibility and reputation
candidates for
for training
training positions. The program’
to a number of
of factors:
•. high quality
quality students,
students, many of
of whom publish their
their work in conference
proceedings
journals even
their degrees;
degrees;
proceedings and refereed
refereed journals
even before receiving their
Stanford MIS students
students have
have won first
first prize in the student paper competition
competition
at the Symposium on Computer Applications
Applications in Medical Care (SCAMC)
(SCAMC) in
1985
1985 and
and 1986,
1986, and
and have
have also
also received
received awards
awards for
for their
their work
work at annual
meetings
meetings of
of organizations such
such as
as the Society for
for Medical Decision Making,
Making,
the American Association for
for Medical Systems
Systems and Informatics
Informatics (AAMSI),
(AAMSI),
and
and the American Association for
for Artificial
Artificial Intelligence
Intelligence (AAAI);
(AAAI);
•. a rigorous curriculum
curriculum that includes newly-developed course
course offerings
offerings that are
are
available to the
the University'S
University‘s medical students,
students, undergraduates,
undergraduates, and
and computer
science
science students
students as
as well as
as to the
the program's
program’s trainees;
trainees;
•. excellent computing facilities
facilities combined with
with ample and
and diverse
diverse opportunities
opportunities
for
for medical computer science
science and
and medical decision science
science research;
research;
•. the
the program's
program’s great
great potential for
for aa beneficial
beneficial impact upon
upon health care
care
delivery in the
the highly
highly technologic but cost-sensitive era
era that lies
lies ahead.
ahead.
The program has
has been
been successful
successful in raising financial
financial and
and equipment support
support from
from
industry
industry and
and foundations.
foundations. It isis also
also recipient
recipient of
of aa training
training grant from
from the
the National
National
Library
Library of
of Medicine. The latter grant
grant was
was recently renewed
renewed for
for another five
five years
years with
aa study
study section
section review
review that
that praised
praised both
both the
the training
training and
and the
the positive
positive contribution
contribution of
of
the
.
the SUMEX-AIM
SUMEX-AIM environment.
environment.
E.
E. H.
H. Shortliffe
Shortliffe
78
78
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR00785-14
III.A.3.S. Resource
Resource Operations and Usage
III.A.3.8.
Operations and
and Support
Support
11 -- Operations
The diverse computing
computing environment
environment that
that SUMEX-AIM
SUMEX-AIM provides requires a significant
significant
The
effort at
at operations
operations and support
support to
to keep the resource responsive to community
community project
project
effort
This includes the planning
planning and management of
of physical facilities
facilities such as
needs. This
needs.
machine rooms and communications,
communications, system
system operations
operations routine
routine to
to backup and retrieve
retrieve
machine
in a timely
timely manner, and user support
support for
for communications,
communications, systems,
systems, and
files in
user files
software advice. Of
Of course,
course, the move of
of our
our groups to
to new space
space in
in the Medical
Medical School
software
Office Building
Building has
has required major
major planning
planning and care to
to ensure
ensure minimum
minimum downtime
downtime for
for
Office
our
computing
environment
and
much
systems
and
electronics
work
to
outfit
the
new
our computing environment
systems
work
outfit
space.
space.
Our active participation
participation in the planning
planning of
of the SUMEX/MCS
SUMEX/MCS facility
facility in
in the MSOB
Our
for twenty-three
twenty-three staff
staff members and thirty-five
thirty-five
resulted in
in a coordinated
coordinated environment
environment for
student workstations, and included 1000
1000 sq.
sq. ft.
ft. of
of computer
computer room
room space
space and three
for easily adding more equipment
equipment and
areas.
Provisions were made
made for
conference areas.
Provisions
networking support. The close
close interaction
interaction with
with the building
building designers
designers had the additional
additional
networking
effect of
of increasing the designers’
designers' interest and knowledge about planning
planning for
for computer
computer
effect
equipment and networking.
networking. We have
have already seen
seen our
our insight
insight spread
spread to other
other building
building
equipment
firms will
will quite likely
likely spread
spread the insight
insight
campus and the architectural firms
projects on campus
implementation of
of standards.
standards. We
further. Building
Building design
design appears
appears to be very much an implementation
further.
moving towards the development of
of more modern standards;
standards;
have had a part in moving
certainly here on the campus
campus and perhaps
perhaps elsewhere.
elsewhere.
certainly
for much of
of our operations and related systems
systems programming
programming work.
use students
students for
We use
We spend
spend significant
significant time
time on new product review and evaluation such
such as
as Lisp
workstations, terminals, communications
network equipment, microprocessor
communications equipment, network
systems,
We also pay close
close
systems, mainframe
mainframe developments,
developments, and peripheral equipment.
attention
production and projection
projection equipment, which has
has proved so
attention to available video production
so
useful in our dissemination efforts
efforts involving
involving video tapes
tapes of
of our work.
SUMEX
machine room. Our primary
SUMEX continues to operate
operate with a generally unattended.
unattended ,machine
primary
operations staff
This provides a coststaff consists
consists of
of three part-time
part-time student workers. This
effective
effective approach and gives
gives these
these undergraduate
undergraduate students
students an opportunity
opportunity to participate
participate
in the SUMEX
SUMEX project. The major use
use of
of this staff
staff is for
for moving
moving data
data files to off-line
off-line
media and to provide data
data file
file backup
backup in case
case of
of equipment failure.
failure. Though we
we have
have
had
had nothing
nothing that could be
be classified as
as a catastrophic failure
failure in the four
four years
years of
of
operating our current
current 2060
2060 equipment,
equipment, we
we have
have had
had several
several failures
failures of
of drives on the
SAFE file
file server.
server. There have
have been
been two cases
cases of
of "soft"
“soft” failures
failures of
of disks on the 2060
2060
system.
system. Though these
these incidents have
have consumed
consumed substantial staff
staff time to deal
deal with, they
have
have not involved
involved significant
significant time loss
loss to the users.
users.
2 - Resource
Resource Usage
Usage Details
Details
The following
following data
data give an
an overview of various aspects
aspects of SUMEX-AIM
SUMEX-AIM central
resource
resource usage.
usage.There are
are 5 subsections
subsections containing
containing data
data respectively
respectively for:
1.
1. Overall
Overall resource
resource loading data
data (page
(page 81).
81).
2.
2. Relative system
system loading by community
community (page
(page 82).
82).
3.
3. Individual
Individual project and
and community usage
usage(page
(page 85).
85).
4.
1. Network
Network usage
usagedata
data (page
(page 90).
90).
79
79
E.
E. H.
H. Shortliffe
Shortliffe
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR00785-14
5. System
System reliability
reliability data (page 92).
92).
For the most
most part, the data used
used for
for these
these plots
plots cover the entire
entire span of
of the
tbe SUMEXSUM EX For
AIM project.
project. This
This includes data from
from both
both the KI-TENEX
KI - TENEX system
system and the current
current
AIM
DECsystem 2060. At
At the point
point where the SUMEX-AIM
SUMEX-AIM community
community switched over to the
(February, 1983), you will
will notice sharp changes
changes in
in most of
of the graphs. This
This is due
2060 (February,
differences in
in scheduling, accounting, and processor
processor speed
speed calculations
calculations between
between the
to differences
systems.
systems.
E.
E. H. Shortliffe
Shortliffe
80
80
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical Progress
Progress
Overall Resource Loading
Loading Data
Data
2.1 - Overall
following plot
plot displays total CPU time delivered per
per month.
The following
usage of
of the
the KI-TENEX
KI-TENEX system
system and
and the
the current
current DECsystem
DECsystem 2060.
2060.
usage
data includes
includes
This data
800 ,
800~-----------------------------------------------------.
:2
C
a
600 -
:E
....
...en
::J
a
::I:
ID
C'l
ca
400
400 -
en
::J
::J
a..
()
:9a
200
200 -
f-
0O~--~--~--~--~--~~--r---~--~--~--~--~--~--~~~
I
I
I
I
I
I
I
1974
1974
1976
1976
1978
1978
Figure
Figure 9:
1980
1980
1982
1982
1984
1984
1986
1986
1988
1988
Total CPU Time
Time Consumed by Month
Month
Total
81
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
Relative System
System Loading
Loading by Community
Community
2.2 -- Relative
The SUMEX
SUMEX resource
resource is divided,
divided, for
for administrative
administrative purposes,
purposes, into
into three major
major
The
Stanford Medical
Medical School (Stanford
(Stanford Projects),
Projects),
communities: user projects based
based at the Stanford
communities:
user projects based
based outside of
of Stanford
Stanford (National
(National AIM
AIM Projects),
Projects), and common
common system
system
user
development efforts
efforts (System
(System Staffl.
Staff). As defined
defined in
in the resource
resource management plan
development
start of
of the project, the available system
system CPU capacity and
approved by the BRP at the start
file space
space resources
resources are nominally
nominally divided
divided between
between these
these communities
communities as
as follows:
follows:
file
Stanford
Stanford
AIM
AIM
Staff
Staff
40%
40%
20%
"available" resources
resources to be divided
divided up between
between these
these communities
commumtles are those
The “available”
remaining after
after various monitor
monitor and community-wide
community-wide functions
functions are accounted for.
for.
remaining
include such things as
as job
job scheduling, overhead,
overhead, network
network service, file
file space
space for
for
These include
subsystems, documentation,
documentation, etc.
subsystems,
monthly usage
usage of
of CPU resources
resources and terminal
terminal connect time
time for
for each
each of
of these
these three
The monthly
communities relative
relative to their
their respective aliquots is shown in the plots in Figure 10 and
communities
page 80,
80, these
these plots include both KI-10
KI-IO and 2060 usage
usage
Figure 11.
11. As mentioned on page
data.
E.
E. H. Shortliffe
Shortliffe
82
82
5P41-RR00785-14
5P41-RR00785-14
Details
Details of
of Technical
Technical Progress
Progress
50
50~------------------------------------------------~
National
National Projects
Projects
40 30 -
20 10 04-~T---~--r-~~~--~--~--~--~--.---~--.---~~
1974
1974
1976
1976
1978
1978
1980
1980
1982
1982
1984
1984
1986
1986
1988
1988
100~-----------------------------------------------------,
100
80 -
Stanford
Stanford Projects
Projects
"0
-0
Q)
%
3
::>
::>
::>
()
System Staff
~
o
20
20 10 04---~--?---r-~~~--~---T---r--~--~--~--~--~~
1974
1974
1976
1976
Figure 10:
1978
1978
1980
1980
1982
1982
1984
1984
1986
1986
1988
1988
Monthly
Monthly CPU Usage
Usage by Community
Community
83
83
E. H.
H. Shortliffe
Shortliffe
Details ofof Technical
Technical Progress
Progress
Details
4000,---------------____________
2'
4000
c:a
:i:
~
:J
a
T
~
5P41-RR00785-14
5P41-RR00785-14
________________________
~
National Projects
Projects
National
3000
3000
~
Q)
E
1=
2000
2000
~c:
c:
<3
1000
1000
0O~--~--~--r---r-~~~--~--~--~--~--~--~--~~
1I
I
I
I
I
I
1974
1976
1978
1974
1980
1976
1982
1978
1984
1980
1982
1984
1986
19'86
1988
1988
20000
20000
£'
c:
Stanford Projects
a
:i:
--
~
:J
a
:::c
~
Q)
E
10000
10000
t=
t)
Q)
c:
c:
a
(.)
0
1974
1974
1976
1976
1978
1978
1980
1980
1982
1982
1984
1984
1986
1986
1988
1988
12000
2'
c:a
:i:
10000
10000 -
System
System Staff
Staff
en
~
:J
a
~
Q)
E
1=
8000
8000 6000
6000 -
t)
Q)
c:
c:
4000
a
(.)
2000
00 :
1974
1974
I
1976
1976
Figure
Figure 11:
11:
E.E. H.
H. Shortliffe
Shortliffe
I
1978
1978
I
I
1980
1980
1982
1982
I
1984
1984
I
1986
1986
Monthly
Monthly Terminal
Terminal Connect
Connect Time
Time by
by Community
Community
84
84
1988
1988
Details of
of Technical
Technical Progress
Progress
Details
5P41-RR00785-14
5P41-RR00785-14
2.3 -- Individual
Individual Project
Project and
and Community
Community Usage
2.3
The following
following histogram
histogram and
and table
table show cumulative
cumulative resource
resource usage
usage by
by collaborative
collaborative
The
community
during
the
past
grant
year.
The
histogram
project
project
and
The
histogram
displays
the project
project and community during the past grant year.
distribution of
of the total
total CPU
CPU time
time consumed between May
May 1, 1986 and April
April 30, 1987,
distribution
SUM EX - AIM DECsystem 2060 system.
system.
on the SUMEX-AIM
CPU consumption
consumption by project
project (Hours),
(Hours), total
total
In the table following,
following, entries include
include total
total CPU
In
average file
file space
space in
in use
use by project
project
terminal connect time
time by
by project
project (Hours),
(Hours), and average
terminal
computer words). These data were accumulated for
for each project
project
(Pages, 1 page
page == 512 computer
(Pages,
for the months between
between May
May 1986
1986 and April
April 1987.
for
85
85
E. H.
H. Shortliffe
Shortliffe
E.
SP41-RR00785-14
5P41- RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
National AIM
AIM (12.62%
(12.62% total)
total)
National
AIM Administration
Administration
AIM
AIM Pilots
Pilots
AIM
AIM Users
Users
AIM
Attending
Attending
Caduceus
Caduceus
Clipr
Clipr
Mentor
Mentor
Solver
Solver
•
Stanford (43.892
(43.89% total)
total)
Stanford
Core Research
Research
Core
Guidon
Guidon
MIS
MIS
Molgen
Molgen
Oncocin
Oncocin
Protean
Protean
Radix
Radix
Stanford
Pilots
Stanford Pilots
Stanford Assoc.
Assoc.
Stanford
Adv.
Adv. Architectures
Architectures
Intelligent
Agents
Intelligent
Agents
Able
Able
KSL
Management
KSL Management
DART
DART
MRS
MRS
Helix
Helix
rI
=-
KSL (20.112
(20.11% total)
total)
KSL
•
Staff
Staff (23.38% total)
total)
Staff
Staff
System
System Assoc.
Assoc.
•
o0
55
10
10
Percent
Percent of
of total
totalCPU
CPU used
used
Figure
Figure 12:
12:
E.
E. H.
H. Shortliffe
Shortliffe
Cumul:::tive
Cumulztive CPU
CPU Usage
Usage Histogram
Histogram by
Sy Projec['1nd
Project 2nd Community
Community
86
56
15
Details
Details of
of Technical
Technical Progress
Progress
5P41-RR00785-14
5P41-RR00785-14
Individual Project
Project -- 5/86
5/86 through
through 4/87
4/87
Resource Use by Individual
CPU
(Hours)
(Hours)
Connect
(Hours)
(Hours)
File
File Space
Space
(Pages)
(Paw)
1) CADUCEUS
CADUCEUS
"Clinical Decision
Decision Systems
Systems
“Clinical
Research Resource”
Resource"
Research
M.D.
Jack D. Myers, M.D.
Harry E. Pople, Jr., Ph.D.
Harry
Randolph A. Miller,
Miller, M.D.
M.D.
Randolph
University of
of Pittsburgh
Pittsburgh
University
2l.63
21.63
562.85
562.85
1066
CLIPR Project
CLIPR
"Hierarchical Models
“Hierarchical
of Human Cognition”
Cognition"
of
Walter Kintsch,
Kintsch, Ph.D.
Walter
Peter G. Polson, Ph.D.
University of
of Colorado
University
0.56
0.56
144.73
144.73
176
0.93
0.93
133.87
133.87
567
567
16.06
16.06
6607.36
6607.36
1044
1044
0.16
0.16
24.91
24.91
3
92.49
92.49
2813.31
2813.31
836
836
0.15
0.15
23.82
23.82
172
172
40.22
40.22
4640.05
4640.05
2308
2308
-. ----------------
-------------------
-------------
National AIM
AIM Community
Community
National
2)
2)
3) SOLVER
SOL VER Project
3)
"Problem
“Problem Solving
Expertise"
Expertise”
Paul E. Johnson,
Johnson, Ph.D.
William
William B. Thompson, Ph.D.
University
University of
of Minnesota
Minnesota
4)
MENTOR Project
4) MENTOR
"Medical
“Medical Evaluation
Evaluation of
of Therapeutic
Orders"
Orders”
Stuart M. Speed
ie, Ph.D.
Speedie,
University
University of
of Maryland
Maryland
Terrence F. Blaschke,
Blaschke, M.D.
Stanford
Stanford University
University
5)
ATTENDING
5) ATTENDING
"A
“A Critiquing
Critiquing Approach to
Expert
Expert Computer Advice"
Advice”
Perry L. Miller,
Miller, M.D., Ph.D.
Ph.D.
Yale University
University School
School of
of Medicine
6)
AIM Pilot
Pilot Projects
Projects
6) AIM
7)
AIM Administration
Administration
7) AIM
8)
AIM Users
Users
8) AIM
Community
Community Totals
172.27
172.27
87
87
14967.35
14967.35
6073
6073
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
Details
CPU
(Hours)
(Hours)
Connect
(Hours)
(Hours)
File Space
Space
File
(Pages)
Pages)
1) GUIDON-NEOMYCIN
GUIDON-NEOMYCIN Project
Project
Bruce G. Buchanan, Ph.D.
William J. Clancey, Ph.D.
William
Dept. Computer
Computer Science
Science
95.67
10725.08
10725.08
1980
2) MOLGEN
MOLGEN Project
of Artificial
Artificial Intelligence
Intelligence
"Applications of
“Applications
to Molecular
Molecular Biology:
Biology: Research
Research in
in
Theory Formation,
Formation, Testing
Testing and
Theory
Modification"
Modification”
Edward A. Feigenbaum, Ph.D.
Peter Friedland,
Friedland, Ph.D.
Charles Yanofsky,
Yanofsky, Ph.D.
Computer Science/
Depts. Computer
Biology
Biology
34.46
34.46
7540.02
7540.02
3109
131.04
131.04
24884.82
24884.82
2871
2871
4)
PROTEAN PROJECT
4) PROTEAN
Oleg
Oleg Jardetzky
Jardetzky
School of
of Medicine
Medicine
Bruce Buchanan
Buchanan
Computer
Computer Science
Science Department
Department
88.64
88.64
12876.18
12876.18
3050
3050
5)
RADIX Project
5) RADIX
Robert
Robert L. Blum, M.D.
Gio
Gio C.M. Wiederhold,
Wiederhold, Ph.D.
Depts.
Depts. Computer Science/
Medicine
Medicine
27.01
27.01
4356.19
4356.19
828
828
6)
Stanford Pilot
Pilot Projects
6) Stanford
10.48
10.48
1368.01
1368.01
1690
1690
7)
AI Research
Research
7) Core AI
76.10
76.10
16019.67
16019.67
2732
2732
6.74
6.74
2421.81
2421.81
179
179
129.09
129.09
16857.16
16857.16
2060
2060
Stanford Community
Community
Stanford
3) ONCOCIN
ONCOCIN Project
3)
"Knowledge Engineering
Engineering
“Knowledge
for Med. Consultation”
Consultation"
for
Edward H. Shortliffe,
Shortliffe, M.D.,
M.D.. Ph.D.
Dept. Medicine
Medicine
8)
Stanford Associates
Associates
8) Stanford
9)
Information Sciences
Sciences
9 Medical Information
-----------------
Community
Community Totals
E.
E. H. Shortliffe
Shortliffe
599.24
599.24
88
88
------------------97048.94
97048.94
------------18499
18499
Details
Details of
of Technical
Technical Progress
Progress
5P41-RR00785-14
5P41-RRO0785-14
KSL-AI Community
Community
KSL-AI
CPU
CPU
(Hours)
(Hours)
Connect
Connect
(Hours)
(Hours)
File Space
Space
File
(Pages)
(Pages)
1)
Architectures
Advanced Architectures
90.41
3643.33
3643.33
2999
2)
FOL
FOL
21.80
2275.61
2275.61
0
3)
Intelligent Agent
Agent
Intelligent
5.01
824.73
824.73
720
4)
Administration
KSL Administration
KSL
70.26
13298.97
13298.97
2897
5)
DART
DART
12.99
12.99
3227.10
3227.10
1577
6)
MRS
33.54
33.54
9642.75
9642.75
2205
2205
7)
Helix
Helix
50.72
50.72
13664.49
13664.49
802
802
8))
8‘
ABLE
ABLE
6.12
6.12
3643.33
3643.33
233
233
-----------------
-------------------
-------------
274.49
74378.94
12512
12512
CPU
(Hours)
(Hours)
Connect
(Hours)
(Hours)
File Space
Space
File
(Pages)
(Pages)
288.63
288.63
38743.11
38743.11
12992
12992
5.41
5.41
681.22
681.22
471
471
-----------------
-------------------
Community totals
Community
SUMEX
SiJMEX Staff
Staff
1) Staff
Staff
2) System
System Associates
Associates
Community
Community Totals
System Operations
Operations
1) Operations
319.20
319.20
42391.73
42391.73
13471
13471
CPU
(Hours)
(HOUTS)
Connect
(Hours)
(Hoursj
File Space
Space
(Pages)
Wws)
918.73
918.73
95782.32
95782.32
24016
24016
--------=========
Resource
Resource Totals
-------
-------
2283.94
2283.94
89
89
---------==========
324569.30
324569.30
_-----------=======
74571
74571
E. H.
H. Shortliffe
Shortliffe
E.
Details
Details of
of Technical
Technical Progress
Progress
5P41-RR00785-14
5P41-RR00785-14
Statistics
2.4 - Network
Network Usage Statistics
The plots in Figure
Figure 13 and Figure 14 show the monthly
monthly network
network terminal
terminal connect time
time
for the public
public data networks and the TNTER
INTERNET
usage. The INTERNET
INTERNET is a broader
for
NET usage.
term for
for what was
was previously
previously referred to as
as Arpanet
Arpanet usage.
usage. Since
Since many vendors now
support the INTERNET
INTERNET
(TP/TCP) in addition
addition to the Arpanet,
Arpanet, which
which
support
protocols (TP/TCP)
IP/TCP in January of
of 1983.
1983, it
it is no longer possible to distinguish
distinguish between
converted to TP/TCP
between
Arpanet usage
usage and Internet
Internet usage
usage on our 2060
2060 system.
system.
Arpanet
1400~--------------------------------------------------------'
1400 ,
:2
C
o
~
Ul
....~
o
~
1200 1200
1000
1000 800
800 -
(/)
.:.::
o
3:
a;
z
600 -
rn
n;
Cl
400 400
.g
:0
~
a..
200
0
,
I
I
O~--~--~----~--~--~---T--~~--~--~---r--~----r---~--~
I
I
I
I
I
1974
1974
13:
Figure 13:
E. H. Shortliffe
Shortliffe
1976
1976
i 978
1978
i 980
1980
1982
1982
i 984
1984
Public Data Network
Network Terminal
Terminal Connect Time
Time
90
1986
1986
1988
i 988
5P41RR00785-14
5P41-RR00785-14
Details
Details of
of Technical
Technical Progress
Progress
8000~------------------------------------------------------,
8000 ,
2'
C
0
:::::;::
6000
6000
--...
rJ)
~
0
~
a>
E
i=
'5
4000
a>
c
c
0
()
Q)
...ca>
2000
2000
C
04---~--~--~--~~~--~---T--~--~~~~~~--r---~~
0
1974
1974
1976
1976
Figure 14:
14:
1980
1980
1978
I 978
1982
1982
1984
i 984
1986
1986
INTERNET
INTERNET Terminal
Terminal Connect
Connect Time
Time
91
91
E. H.
H. Shortliffe
Shortliffe
E.
1988
1988
5P41-RR00785-14
5P41-RR00785-14
Details of
of Technical
Technical Progress
Progress
2.5 - System Reliability
Reliability
System reliability
reliability for
for the DECsystem 2060 has
has remained quite high in this past period.
System
We have had very few periods of
of particular
particular hardware or software problems other than
while
while tracking
tracking down the internet
internet free space
space software bug. The data below covers
covers the
period of
of May 1,
1, 1986
1986 to April
April 30,
30, 1987.
1987. The actual downtime
downtime was
was rounded to the
nearest
nearest hour.
April 1987:
1987:
May 1986
1986 - April
May
May
Jun
Jul
Aug
Sep
Sep
Oct
Ott
10
28
28
13
13
3
2
2
Figure
Figure 15:
May 1986
1986 - April
April 1987:
1987:
Reporting
Reporting period
Total
Total UpTime
Up Time
PM Downtime
Down time
Actual Downtime
Downtime
Actual
Total Downtime
Downtime
Total
Mtbf
Mtbf
Uptime
Uptime Percentage
Percentage
Dec
Dee
Jan
Feb
Mar
Mar
Apr
Apr
2
2
1
9
11
System
System Downtime
Downtime --- Hours per Month
Month
°
365 days,
days, 0 hours, 12 minutes, and 49 seconds
seconds
365
359
359 days,
days, 23
23 hours, 8 minutes, and 11
11 seconds
seconds
o0 days,
days, 18 hours, 35
35 minutes, and 5 seconds
seconds
days, 6 hours, 29 minutes, and 33
33 seconds
seconds
4 days,
5 days,
days, 1 hour, 4 minutes, and 38
38 seconds
seconds
2 days,
days, 13
13 hours, 42 minutes, and 29 seconds
seconds
98.83
98.83
Figure
Figure 16:
E. H. Shortliffe
Shortliffe
Nov
Nov
20
Overall
Overall System
System Reliability
Reliability Summary
92
Highlights
Highlights
5P41-RR00785-14
III.B.
1II.B. Highlights
Highlights
In this section we describe
describe several
several research
research highlights
highlights fr·om
from the past year's
year’s actIvItIes.
activities.
These
These include notes
notes on existing
existing projects that have
have passed
passed important
important milestones, new
pilot
pilot projects that have shown progress
progress in their
their initial
initial stages,
stages, and other core research
research
special activities
activities that reflect
reflect the progress,
progress, impact, and influence
influence the SUMEX-AIM
SUMEX-AIM
and special
resource has
has had in the scientific
scientific and educational communities.
communities.
resource
93
93
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Highlights
Highlights
III.B.1. The MENTOR
MENTOR Project
Project
III.B.l.
The MENTOR
MENTOR (Medical
(Medical EvaluatioN
EvaluatioN of
of Therapeutic
Therapeutic ORders) project,
project, under Drs.
The
Stanford University,
University, Stuart
Stuart Speedie
Speedie at the University
University of
of Maryland,
Maryland,
Blaschke at Stanford
Terrence Blaschke
University of
of North
North Carolina,
Carolina, seeks
seeks to design and develop
Friedman at the University
and Charles Friedman
for monitoring
monitoring drug therapy for
for hospitalized
hospitalized patients. The
The purpose of
of
an expert system
system for
system is to provide
provide appropriate
appropriate advice to
to physicians concerning
concerning the existence and
the system
management of
of adverse
adverse drug reactions.
management
as a record-keeping
record-keeping device is becoming increasingly
increasingly common
common in
in hospital
hospital
The computer as
but much of
of its potential
potential remains unrealized. Often,
Often, information
information is
care, but
health care,
form of
of raw data. The wealth of
of such data may
in the form
provided to the physician in
important information
information about the patient.
patient. This
This is particularly
particularly true with
with
effectively hide important
effectively
respect to adverse
adverse reactions to drugs which
which can only
only be detected
detected by simultaneous
respect
of several
several different
different types
types of
of data including
including drug data, laboratory
laboratory tests,
tests, and
examinations of
clinical signs
signs using sophisticated medical knowledge and problem
problem solving.
solving.
Expert
Expert
clinical
systems offer
offer the possibility
possibility of
of embedding this expertise in
in a computer
computer system
system which
systems
information and monitor
monitor for
for the prospect or
automatically gather the appropriate
appropriate information
would automatically
of adverse
adverse drug reactions.
actual occurrence of
was initiated
initiated in December 1983.
project has
has been
been funded
The MENTOR
MENTOR project was
1983. The project
for Health Services
Services Research
Research since January 1,
of June
National Center for
by the National
1, 1985.
1985. As of
1, 1987,
working prototype
prototype system
system has
has been
been developed
developed and is undergoing evaluation.
evaluation.
1987, a working
of a Patient Data Base,
Base, an Inference Engine, an Advisory
Advisory
prototype consists
consists of
The prototype
The Medical Knowledge Base
Base currently
currently
Module, and a Medical Knowledge Base.
Base.
information related to Aminoglycoside
Aminoglycoside Therapy, Digoxin
Digoxin therapy, Surgical
contains information
Prophylaxis, and Microbiology
system is currently
currently implemented
implemented on a
Microbiology Lab reports. The system
been developed
Xerox workstation.
workstation. Another
Another version of
of the Patient
Patient Data Base
Base has
has been
developed for
for a
mainframe
for the interconnection
interconnection of
of the
mainframe and is currently
currently being tested.
tested. Plans call for
mainframe
mainframe and the workstation
workstation running
running the inference
inference engine.
engine. The mainframe
mainframe will
will then
be connected
connected to a Hospital Information
Information System
System for
for data acquisition.
acquisition.
E.
E. H. Shortliffe
Shortliffe
94
94
Highlights
Highlights
5P41-RR00785-14
5P41-RR00785-14
III.B.2. The GUIDON
GUIDON Project
Project
III.B.2.
GUIDON/NEOMYCIN Project, under Drs. William
William J. Clancey and. Bruce
The GUIDON/NEOMYCIN
of Stanford
Stanford University,
University, is a research
research program to
to develop a knowledgeknowledgeG. Buchanan of
based tutoring
tutoring system
system for
for application
application to medicine.
primary goal for
for the
The primary
based
GUIDON/NEOMYCIN project
project is to develop a program that
that can provide
provide advice similar
similar
GUIDON/NEOMYCIN
in quality
quality to that
that given by human experts,
experts, modeling
modeling how they structure
structure their
their knowledge
in
as well as
as their
their problem-solving
problem-solving procedures.
procedures. The consultation
consultation program using this
this
as
NEOMYCIN. The problem-solving
problem-solving procedures
procedures are developed by
knowledge is called NEOMYCIN.
running test cases
cases through
through NEOMYCIN
NEOMYCIN and comparing
comparing them to expert behavior.
behavior. Also,
running
use NEOMYCIN
NEOMYCIN as
as a test bed for
for the explanation
explanation capabilities
capabilities incorporated
incorporated in
in our
our
we use
instructional programs.
instructional
Our current
current emphasis is to construct a knowledge-based
knowledge-based tutoring
tutoring system
system that
that teaches
teaches
Our
strategy, we mean
mean plans for
for establishing a set of
of
diagnostic strategies
strategies explicitly.
explicitly. By strategy,
diagnoses, focusing on and confirming
confirming individual
individual diagnoses,
diagnoses, gathering data,
data, and
possible diagnoses,
processing new data.
data. The tutorial
tutorial program has
has capabilities
capabilities to recognize these
these plans, as
as
processing
in explanations about how to do diagnosis.
diagnosis. The strategies
strategies
as to articulate
articulate strategies
strategies in
well as
wholly
represented in the program, modeling techniques,
techniques, and explanation
explanation techniques are wholly
represented
separate from
from the knowledge base,
base, so that they can be used
used with
with many medical (and
separate
non-medical) domains.
non-medical)
felt that medical knowledge,
knowledge, initially
initially codified
codified for
for the purpose of
of
has long been
been felt
ItIt has
computer-assisted consultations, may also be used
used to teach
teach medical students.
studen ts.
The
basis of
of the system
system has
has matured enough
enough that we are now collaborating
collaborating closely
technical basis
with medical students and physicians to design
design a useful tutoring
tutoring program. The system
system
with
implements a three-step tutorial
tutorial process
process in which
which the student will
will solve a problem,
problem,
implements
watch the system
system solve it, and then explain his solution
solution and seek
seek explanations about the
system's
system’s solution.
solution. In this way,
way, the program will
will serve
serve as
as a model
model that the student can
study and compare to his own reasoning.
reasoning.
Another
Another tutorial
tutorial project involves development of
of a modeling program (ODYSSEUS)
aimed at discovering discrepancies
discrepancies between
between an expert system
system knowledge
knowledge base
base and that of
of
a student or expert problem
problem solver. When ODYSSEUS
ODYSSEUS watches
watches a student, it
it functions
functions
as
as a student modeling program and when
when it
it watches
watches an
an expert, it
it functions
functions as
as a
knowledge
knowledge acquisition
acquisition program.
The final
final major
major effort
effort involves generalizing our expert system
system tool, HERACLES,
HERACLES, so
that it
it can
can be made
made available to other research
research groups
groups wishing to develop knowledge
knowledge
bases
bases that can
can be used
used for
for tutoring.
tutoring.
In our current
current work, we
we are
are focusing on the modeling, explanation, and knowledge
knowledge
acquisition
acquisition capabilities
capabilities that will
will allow the tutor
tutor to articulate how a diagnostic solution
solution
is flawed and how it
it can
can be improved using
using specific domain knowledge.
knowledge. Thus, we
we are
are
teaching the students
students what constraints a good
good solution
solution must respect
respect and giving
giving them a
language
language for
for articulating
articulating which medical facts are
are relevant to the case
case at hand.
Physicians
Physicians have
have generally been
been enthusiastic about the potential of
of these
these programs and
what they reveal
reveal about current
current approaches
approaches to computer-based medical decision making.
95
95
E. H. Shortliffe
Shortliffe
5P4l-RR00785-l4
5P41-RR00785-14
Highlights
Highlights
III.B.3.
III.B.3. The PROTEAN
PROTEAN. Project
Project
The
The PROTEAN
PROTEAN project,
project, under
under Professors
Professors Oleg
Oleg lardetzky
Jardetzky and
and Bruce
Bruce Buchanan
Buchanan at
at
Stanford
Stanford University,
University, is concerned
concerned with using
using artificial
artificial intelligence methods
methods to aid in the
the
determination
determination of the
the 3-dimensional
3-dimensional structure
structure of proteins in solution
solution (as
(as opposed
opposed to
crystallized proteins). The molecular structure
structure of proteins is essential
essential for
for understanding
many problems of
of medicine at the
the molecular level,
level, such
such as
as the
the mechanisms
mechanisms of
of drug
action. Using NMR
NMR data
data from
from proteins in solution
solution will
will allow the
the study of
of proteins
whose
whose structure
structure cannot be
be determined with other techniques,
techniques, and
and will
will decrease
decrease the
the time
needed
needed for
for the determination.
determination. It is hoped
hoped that empirical
empirical data
data from
from nuclear magnetic
resonance
resonance (NMR)
(NMR) and
and other sources
sources may provide enough
enough constraints on structural
structural
descriptions to allow protein chemists
chemists to bypass
bypass the
the laborious methods
methods of
of crystallizing
crystallizing a
-ray crystallography to determine its structure.
protein
protein and
and using
using X
X-ray
During
we have
have extended
extended our initial
initial prototype program, called
During the
the past
past year,
year, we
It is implemented in BBl,
PROTEAN,
PROTEAN, designed
designed using a blackboard
blackboard model.
BBl, a
framework
framework system
system for
for building
building blackboard systems
systems that control
control their
their own problemproblemsolving behavior. The reasoning
reasoning component of
of PROTEAN
PROTEAN directs the actions of
of the
Geometry System
System (GS), a set
set of
of programs
programs that performs the computationally
computationally intensive
task of
of positioning
positioning portions
portions of
of a molecule with respect
respect to each
each other in three
dimensions. The GS runs in the UNIX
UNIX environment
environment on a Silicon
Silicon Graphics IRIS 3020
3020
graphics workstation.
workstation. The reasoning
reasoning program (in
(in LISP in BBl)
BBl) is coupled to the GS by
a local area
area computer network
network developed
developed by SUMEX.
SUMEX.
Pictures of
of the results
results of
of GS computations
computations are displayed
displayed on the graphics screen
screen of
of the
IRIS workstation,
workstation, using a locally
locally developed
developed program called DISPLAY
DISPLAY to draw the
evolving
protein structures at several
evolving protein
several levels of
of detail. The DISPLAY
DISPLAY program can be
user
used
used to view structures generated
generated by the GS either under the direct
direct control
control of
of the user
or
or as
as directed by the reasoning system
system running
running in BBl.
BBl. MIDAS
MIDAS and MMS
MMS are two
other
protein structures,
other molecular modeling and display systems
systems to manipulate protein
particularly
particularly those
those obtained from
from crystallographic
crystallographic techniques
techniques as
as found
found in the Protein
Protein
Data Bank. The ability
ability to observe
observe structures in three dimensions is essential
essential to
of the PROTEAN’
PROTEAN's
systems and
behavior of
understanding the behavior
s reasoning and geometry systems
essential insights on the problem solving
solving process.
process.
provides essential
addition to the Lac-repressor headpiece
headpiece protein,
protein, we have
have applied PROTEAN
PROTEAN to
In addition
of these
these latter
latter proteins
proteins
sperm whale myoglobin,
myoglobin, T4
T4 lysozyme,
lysozyme, and cytochrome B. Each of
of the protein
protein and
has a known crystal structure. In
In each
each case,
case. we extracted features of
has
for PROTEAN.
PROTEAN. We then applied
applied the PROTEAN
PROTEAN
constraints to build
build data sets
sets for
distance constraints
of the system
system with
with different
different
system to the resulting
resulting data sets
sets to determine
determine the behavior of
system
kinds of
of input.
input.
kinds
To determine
determine the correctness
correctness and capabilities
capabilities of
of the PROTEAN
PROTEAN method, we applied
applied
To
PROTEAN to sperm whale myoglobin,
myoglobin, a molecule whose
whose crystal structure
structure is known.
known.
PROTEAN
of the precision and accuracy of
of the
systematically explored the dependence
dependence of
We systematically
of the input
input data available. In
In all
all cases,
cases, the sets
sets solutions
solutions
solutions on the quality
quality of
solutions
from PROTEAN
PROTEAN include
include the actual structure
structure of
of the molecule,
molecule. with
with the best
obtained from
obtained
from data representing many
many short
short range constraints.
coming from
results coming
Work is proceeding on several aspects
aspects of
of the protein
protein structure
structure problem,
problem. including
including
Work
of several
several partial
partial arrangements and integration
integration of
of these
these pieces
pieces of
of solution
solution into
into
assembly of
atomic level volume
volume exclusion of
of atoms and information
information on
on
larger structures, using atomic
packing to
to produce more
more precise atomic
atomic level solutions,
solutions. and developing
developing more
more
sidechain packing
for unstructured
unstructured coil
coil sections of
of proteins.
appropriate representations for
appropriate
H. Shortliffe
Shortliffe
E. H.
96
5P41-RR00785-14
5P41-RR00785-14
Highlights
Highlights
III.B.4. The
The Medical
Medical Information
Information Science
Science Program
Program
III.B.4.
The Medical
Medical Information
Information Sciences
Sciences (MIS)
(MIS) program
program continues
continues to
to be
be one
one of
of the
the most
most
The
obvious signs
signs of
of the
the academic
academic impact
impact of
of the
the SUMEX-AIM
SUMEX-AIM resource
resource on
on Stanford
Stanford
obvious
University. The
The MIS
MIS program
program received
received recent
recent University
University approval
approval (in
(in October
October 1982)
1982) as
as
University.
an
innovative
training
program
that
offers
MS
and
PhD
degrees
to
individuals
with
an innovative training program that offers MS and PhD degrees to individuals with aa
career commitment
commitment to
to applying
applying computers
computers and
and decision
decision sciences
sciences in
in the
the field
field of
of
career
medicine.
In Spring
Spring 1987,
1987, aa University-appointed
University-appointed review
review group
group unanimously
unanimously
medicine.
In
recommended that
that the
the degree
degree program
program be
be continued
continued for
for another
another five
five years.
years. The
The MIS
MIS
recommended
training
program
is
based
in
the
School
of
Medicine,
directed
by
Dr.
Shortliffe,
cotraining program is based in the School of Medicine, directed by Dr. Shortliffe, codirected by
by Dr.
Dr. Fagan,
Fagan, and
and overseen
overseen by
by aa group
group of
of six
six University
University faculty
faculty that
that includes
includes
directed
from the Knowledge Systems
Systems Laboratory
Laboratory (Profs.
(Profs. Shortliffe
Shortliffe and
and Buchanan).
Buchanan).
two faculty
faculty from
two
It was
was Stanford’
Stanford's
active on-going
on-going research
research in
in medical
medical computer
computer science,
science, plus
plus aa worldworldTt
s active
wide
reputation
for
the
excellence
and
rigor
of
those
research
efforts,
that
persuaded
the
wide reputation for the
rigor of those research efforts, that persuaded the
University that
that the field
field warranted a new academic degree
degree program
program in
in the
the area.
area. AA
University
group of
of faculty
faculty from
from the medical school and the computer
computer science
science department
department argued
argued
group
that research
research in
in medical computing
computing has
has historically
historically been
been constrained
constrained by
by a lack
lack of
of
that
talented individuals
individuals who have a solid
solid footin
footingg in
in both
both the medical
medical and computer
computer science
science
fields. The specialized curriculum
curriculum offered
offered by the new program is intended
intended to overcome
fields.
limitations of
of previous training
training options.
options. ItIt focuses
focuses on the development of
of a new
the limitations
of researchers
researchers with
with a commitment
commitment to developing new knowledge about
about
generation of
optimal methods for
for developing practical
practical computer-based solutions
solutions to
to biomedical
biomedical needs.
needs.
optimal
accepted its first
first class
class of
of four
four trainees in
in the summer of
of 1983
1983 and has
has
The program accepted
reached its steady-state size of
of approximately
approximately 20 graduate students. We do not
now reached
provide too narrow
narrow a definition
definition of
of what kinds
kinds of
of prior
prior training
training are pertinent
pertinent
wish to provide
because of
of the interdisciplinary
interdisciplinary nature o-.0'.- *he
'he field.
field. The program has
has accordingly
accordingly
because
applications from
from any of
of the fc
fc .ving:
Ning:
encouraged applications
encouraged
•. medical students
students who wish
and research
research experience
experience in
to combine
combine MD
MD training
training with
with formal
formal degree
degree work
work
MIS;
after their
their ivlD
MD or their
their
•. physicians who wish to obtain
obtain formal
formal MIS training
training after
residency,
fellowship at Stanford
residency, perhaps
perhaps in conjunction
conjunction with
with a clinical
clinical fellowship
Stanford
Medical Center;
•. recent BA or BS
BS graduates
graduates who have
have decided
decided on a career
career applying
applying computer
science
science in the medical world;
•. current
current Stanford undergraduates
undergraduates who wish to extend
extend their
their Stanford
Stanford training
training
an
an extra year
year in order to obtam
obtain aa "co-terminus"
“co-terminus” MS in the MIS program;
program:
•. recent
recent PhD graduates
graduates who
who wish
wish post-doctoral
post-doctoral training,
training, perhaps
perhaps with the
formal
formal MS
MS credential, to
to complement their primary
primary field
field of training.
training.
In addition,
addition, aa special
special one-year MS
MS program isis available for
for established
established academic
academic
medical
medical researchers
researchers who
who may
may wish
wish to
to augment
augment their
their computing and
and statistical skills
during
during aa sabbatical
sabbatical break.
break. As
As of
of Spring
Spring 1987,
1987, half
half our
our trainees
trainees have
have previously received
received
MD
MD degrees
degrees and
and another
another quarter
quarter are
are medical
medical students
students enrolled in joint
joint degree
degree programs.
programs.
One-third
One-third are
are candidates
candidates for
for the
the \t1S
MS degree,
degree, while
while the
the rest
rest are
are doctoral
doctoral students.
students. The
program
program has
has three
three graduates
graduates to
to date,
date, with
with several
several more
more expecting
expecting to
to complete
complete degrees
degrees
before
before the
the end
end of
of 1987.
1987. Research
Research opportunities
opportunities for
for students
students include
include the
the several
several
Stanford
Stanford AIM
AIM projects
projects as
as well
well as
as research
research in
in psychological
psychological and
and formal
formal statistical
statistical
approaches
approaches to
to medical
medical decision
decision making,
making, applied
appiied instrumentation,
instrumentation, large
large medical
medical databases,
databases,
and
and aa variety
variety of
of other
other applications
applications projects
projects at
at the
the medical
medical center
center and
and on
on the
the main
main
campus.
campus. Several
Several students
students are
are already
already contributing
contributing in
in major
major ways
ways to
to the
the AIM
AIM projects
projects
and
and core
core research
research described
described elsewhere
elsewhere in
in this
this annual
annual report.
report.
97
97
E.
E. H.
H. Shortliffe
Shortliffe
5P41RR00785-14
5P41-RR00785-14
Highlights
Highlights
III.B.s.
III.B.5. Remote Virtual
Virtual Graphics
Lisp workstations
powerful, both as
workstations of
of various types
types have proven extremely powerful,
as
development environments
environments for
for artificial
artificial intelligence
intelligence research
research and as
as vehicles
vehicles for
for
disseminating AI
In
AI systems
systems into
into user
user communities.
communities.
In addition
addition to the compact,
inexpensive computing
playaa key
computing resources
resources workstations
workstations provide, high-quality
high-quality graphics play
role in their
Such
systems have
have become
become indispensable for
for
their power.
Such graphics systems
understanding the complex
complex data structures involved
involved in developing and debugging
debugging large
large
AI
AI systems
systems and are important
important in facilitating
facilitating user
user access
access to working
working programs (e.g.,
(e.g., for
for
ONCOCIN
ONCOCTN and PROTEAN).
PROTEAN).
In the past,
past, members of
of the SUMEX-AIM
SUMEX-AIM community
community have
have often
often watched
watched each
each others'
others’
programs work
work by linking
linking their
their CRT
CRT terminals
terminals to the text output
output of
of a running
running program
on the SUMEX
SUMEX 2060.
2060. With
With workstations,
workstations, though, it
it is much more difficult
difficult to connect
to a remote machine and be
be able to view the complex graphics output
output of
of a program.
One would like
provide the same
powerful graphical tools and
like to be able to provide
same powerful
programming
user sitting
programming environment
environment that are available to a user
sitting in front
front of
of the
workstation
bit-mapped display and
workstation to the remote user
user if
if that user
user has
has a low-cost
low-cost bit-mapped
mouse.
mouse.
During
During this past year,
year, we developed
developed a program called TALK
TALK to facilitate
facilitate interactive,
interactive,
electronic
electronic communication
communication between
between users
users on independent
independent workstations.
workstations. Layered on the
the
workstation's
workstation’s native
native editor,
editor, the program allows the full
full use
use of
of all
ail editing
editing capabilities
capabilities in
the process
process of
of communication,
communication, including
including deletions, corrections
corrections and insertions,
insertions, font
font
changes,
changes, underlining,
underlining, paragraph formatting,
formatting, etc.
etc. Since the workstation's
workstation’s editor
editor also
also
supports both lowlow- and high-level
high-level graphics, the program not only
only facilitates
facilitates textual
exchanges
exchanges among users,
users, but also
also allows the sending of
of screen
screen images
images (ONCOCIN
(ONCOCIN flow
flow
sheet
breaks, code
sheet segments,
segments, back traces
traces of
of program error
error breaks,
code fragments, etc.) as
as well as
as
structured images
images (which
(which can be modified
modified on the destination
destination workstation
workstation and returned),
all
a11interactively.
interactively.
The TALK
user interfaces, the workstation's
TALK program allows the use
use of
of different
different user
workstation’s
document editor
just one possibility.
possibility. We implemented
editor being just
implemented a simpler
simpler terminal
terminal mode
mode
for
for compatibility
compatibility with
with similar
similar programs on other workstations.
workstations.
The TALK
been released
users in order
TALK program has
has been
released gradually
gradually to increasing numbers of
of users
to get feedback and make changes
changes accordingly. The Medical Computer
Computer Science
Science group
did an extensive test of
place of
of the system
system where for
for a period they used
used it
it in place
of their
their
normal
possible. This
normal electronic
electronic and non-electronic
non-electronic communication
communication methods whenever
whenever possible.
was
was both a test of
of the program and an exploration
exploration into
into what people want in the
ihe next
generation of
been released
released to the
of electronic
electronic communication.
communication. The TALK
TALK program has
has been
Xerox Lisp workstation
workstation community
community as
as a whole and researchers
researchers at Xerox PA.RC
PARC
successfully
successfully used
used the program to hold an interactive,
interactive, graphic, electronic
electronic conversation
between
between users
users at the PARC facility
facility (in
(in California)
California) and Xerox's
Xerox’s EuroPARC
EuroPARC facility
facility (in
iin
England).
E.
E. H. Shortliffe
Shortliffe
98
98
Administrative
Administrative Changes
Changes
5P41-RR00785-14
5P41-RR00785-14
III.C. Administrative
Administrative Changes
Changes
1II.C.
few administrative
administrative changes
changes within
within the project
project this
this past year. Professor
There have been few
Shortliffe has been on sabbatical at the University
University of
of Pennsylvania as projected last
Shortliffe
but has
has stayed
stayed in
in very close contact
contact with
with SUMEX
SUMEX and the Medical
Medical Computer
Computer
year but
Science group at Stanford
Stanford through
through network
network connections. During
During this
this time, Professor
Science
in the formal
formal role of
of principal
principal investigator.
investigator. Professor Shortliffe
Shortliffe
has acted in
Feigenbaum has
expected back at Stanford
Stanford in
in mid-July.
mid-July.
is expected
Medical Computer
Computer Science
Science and SUMEX
SUMEX offices
offices into
into the newly
The move the Medical
Stanford Medical
Medical School Office
Office Building
Building was
was completed in
in June 1986.
1986. We
constructed Stanford
approximately 6500 square
square feet
feet has
has almost doubled the space
space available to
now occupy approximately
us.
of this
this space
space has
has worked out
out exceedingly well to improve
improve the
The design of
us.
interactions within
within our
our groups.
interactions
implemented a cost recovery system
system as
as part
part of
of phasing out
have also designed
designed and implemented
We have
BRTP subsidy of
of the DEC
DEC 2060 facility.
facility. The details of
of this system
system are discussed
discussed on
of 2060
page 101.
In summary, we are successfully recovering the projected 20% of
page
101. In
from Stanford
Stanford users,
users, with
with the continuing
continuing component
component
costs this year ($71,376)
($71,376) from
operations costs
of NIH
NIH support
support used
used to protect
protect national
national users
users from
from fees
fees for
for service, including
including
of
communications. This
This additional
additional burden on Stanford
Stanford projects was
was absorbed
absorbed almost
communications.
forthcoming from
from
entirely in existing
existing direct
direct cost budgets
budgets since no supplements were forthcoming
entirely
funding agencies
agencies in
in the middle
middle of
of on-going
on-going grant and contract
contract awards
awards for
for new
other funding
computing costs.
costs. This
This has
has affected
affected staffing
staffing and student support
support directly
directly in our laborlaborcomputing
intensive research
research efforts.
efforts. All
All of
of our
our new support
support applications
applications are being written
written with
with
intensive
requests for
for funds to cover computing
computing charges.
charges.
requests
This
This next year we
we will
will increase
increase the cost recovery goal to 40% of
of projected 2060
2060
operations costs
costs as
as scheduled
scheduled in our grant application
application of
of June
June 1985.
1985.
99
99
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Resource Management and Allocation
Allocation
Resource
III.D. Resource
Resource Management and Allocation
Allocation
1II.D.
III.D.l. Overall
Overall Management Plan
Plan
III.D.l.
Early in
in the design
design of
of the SUMEX-AIM
SUMEX-AIM resource,
resource, an effective
effective management plan was
was
Early
with the Biotechnology
Biotechnology Resources
Resources Program (now
(now Biomedical
Biomedical Research
Research
worked out with
NIH to assure
assure fair
fair administration
administration of
of the resource
resource for
for both
both
Technology Program) at NIH
Stanford and national
national users
users and to
to provide
provide a framework
framework for
for recruitment
recruitment and
Stanford
of a scientifically
scientifically meritorious
meritorious community
community of
of application
application projects. This
This
development of
structure has
has been
been described
described in
in some detail
detail in
in earlier
earlier reports and is documented in
in our
our
application. ItIt has
has continued
continued to function
function effectively
effectively as
as summarized
recent renewal application.
below.
Committee meets
meets regularly
regularly by teleconference to advise
AIM Executive Committee
.• The AIM
project applications,
applications, discuss
discuss resource
resource management policies, plan
on new project
other community
community b.usiness.
b_usiness. The Advisory
Advisory
activities, and conduct other
workshop activities,
meets together at the annual AIM
AIM workshop to discuss
discuss general
Group meets
resource business
business and individual
individual members are contacted much more
resource
frequently to review project applications.
applications. (See
(See Appendix
Appendix A
A on page
page 217 for
for
frequently
current listing
listing of
of AIM
AIM committee
committee membership).
membership).
a current
have actively
actively recruited new application
application projects and disseminated
.• We have
information about the resource.
resource. The number of
of formal
formal projects in
in the
information
community still
still runs at the capacity of
of our
our computing
computing
SUMEX-AIM community
SUMEX-AIM
resources.
With the development of
of more decentralized computing
computing resources
resources
resources. With
within
within the AIM
AIM community
community outside of
of Stanford
Stanford (see
(see below), the center of
of
mass
mass of
of our community
community has
has naturally
naturally shifted
shifted toward the growing
growing number
number of
of
Stanford applications
applications and core research
research projects. We still,
still, however, actively
actively
support new
new applications
applications in the national
national community
community where these
these are not
not
able to gain access
access to suitable computing
computing resources
resources on their
their own.
•. With
pilot project
With the advice of
of the Executive Committee,
Committee, we have
have awarded
awarded pilot
status
promising new application
status to promising
application projects and investigators
investigators and where
appropriate, offered
offered guidance
guidance for
for the more effective
effective formulation
formulation of
of research
research
between
plans and for
for the establishment of
of research
research collaborations
collaborations between
biomedical and computer science
This past year
year we have
have
science investigators. This
admitted projects under Professors
Professors Perry Miller
Miller at Yale University,
University, Larry
Larry
Widman
Kalet at the University
University of
of
Widman at the University
University of
of Texas,
Texas, Ira Kalet
Washington, and Robert Beck at Dartmouth
The latter
latter two
Dartmouth University.
University.
sought
sought access
access primarily
primarily for
for communication
communication with
with the AIM
AIM community
community as
as they
have
have research
research computing
computing resources
resources of
of their
their own.
•. We have
have carefully
carefully reviewed
reviewed on-going
on-going projects with
with our management
committees to maintain
maintain a high scientific
scientific quality
quality and relevance
relevance to our
biomedical AI
AI goals
goals and to maximize
maximize the resources
resources available for
for newly
developing applications
applications projects.
projects. Several
Several fully
fully authorized and pilot
pilot projects
have
have been
been encouraged
encouraged to develop their
their own computing
computing resources
resources separate
separate
from
from SUMEX
SUMEX or have
have been
been phased
phased off
off of
of SUMEX
SUMEX as
as a result and more
productive collaborative
collaborative ties
ties established
established for
for others.
others.
•. We continue to provide active support for
for the AIM
AIM workshops.
workshops. The next
one will
will be
be held
held at the
the University
University of
of Washington in conjunction
conjunction with
with the
American Association for
for Artificial
Artificial Intelligence
Intelligence meeting in July 1987.
1987. It
It is
being organized
organized jointly
jointly by Drs. Ira Kalet of
of Washington and
and Larry
Larry Fagan
Fagan of
of
Stanford.
E.
E. H. Shortliffe
Shortliffe
100
100
5P41-RR00785-14
5P41-RR00785-14
Resource
Resource Management
Management and Allocation
Allocation
•. We
We have tailored
tailored resource policies
policies to aid users whenever
whenever possible
possible within
within our
our
research mandate
Our
mandate and available
available facilities.
facilities.
Our approach
approach to system
scheduling,
scheduling, overload
overload control,
control, file
file space management,
management, etc. all
all attempt
attempt to give
give
users the greatest latitude
latitude possible
possible to pursue their
their research goals consistent
consistent
users
with fairly
fairly meeting
meeting our
our responsibilities
responsibilities
in administering
administering
SUMEX as
as a
with
in
SUMEX
national
national resource.
III.D.2. 2060 Cost Center
III.D.2.
General
General Cost Center
Center Structure
Structure
Our renewal
renewal proposal
proposal for
for the five-year
five-year period
period 8/1/86-7/31/91,
8/l/86-7/31/91,
submitted to the
Our
submitted
Division of
of Research Resources in
in June 1985, called
called for
for phasing
phasing out
out NIH
support for
for
Division
NIH support
DEC 2060 mainframe
mainframe operations
operations over
over the course of
of the grant
grant period
period and the
DEC
establishment of
of a cost center
center at Stanford
Stanford to recover
recover the unsubsidized
unsubsidized costs of
of 2060
establishment
operations
This
operations from
from the user community.
community.
This phasing-out
phasing-out process is taking
taking place linearly
linearly
over
over five
five years, with
with 20% of
of the 2060 costs being
being recovered
recovered in renewal
renewal year 1 (Grant
(Grant
Year 14), 40% in
in year 2, 60% in
in year 3, 80% in
in year 4, and 100% starting
starting in year 5.
5.
Year
In this
this process,
process, we are attempting
attempting to minimize
minimize the barriers
barriers for
for national
national projects
projects by
In
using the continuing
continuing partial
partial BRTP
BRTP subsidy
subsidy to cover
cover their
their costs for
for as
as long
long as
as posslble.
possible.
In this
this past year, use
use of
of the 2060 by members
members of
of the national
national AIM
AIM community
community has
has been
In
free of
of charge. Thus,
free
Thus, the Stanford
Stanford user projects
projects are bearing
bearing the entire
entire brunt
brunt of
of cost
recovery during
during the first
first few
few years. Our
recovery
that we
we are
Our plan
plan is conservative,
conservative, however,
however, in that
doing
doing this
this gradually
gradually and responsibly
responsibly so that
that our
our users
users can secure the funding
funding resources
software changes necessary to allow
allow them
them to relocate
relocate to other
other facilities
facilities or
or
and make software
move to workstation
workstation environments
environments for
for their
their research.
move
To implement
implement this
this plan,
plan, during
during the summer
summer of
of 1986, we requested and received
received
To
approval from
from the Government
Government Cost
Cost and Rate Studies section
section of
of Stanford's
Stanford’s Controller's
Controller’s
approval
Office to establish
establish a 2060 cost center
center effective
effective August
August 1, 1986. We set up the cost
Office
center with
with the simplest
simplest possible
possible charge structure
structure in
in order
order to minimize
minimize the accounting
accounting
center
administrative overhead,
overhead, establishing
establishing a charge rate per CPU
CPU hour
hour based
based on our
our
and administrative
projections of
of 2060 operations
operations costs and anticipated
anticipated billable
billable Stanford
Stanford project
project CPu
CPL;
projections
usage. The initial
initial rate was
was established
established at $95 per CPU
CPU hour.
hour.
usage.
We closely
closely monitored
monitored the cost center
center expenses and revenues during
during the year. A
A midmidanalysis of
of cost center
center performance
performance indicated
indicated that
that expenses would
would be somewhat
somewhat
year analysis
lower and billable
billable CPU
CPU usage
usage somewhat
somewhat higher
higher than originally
originally projected.
projected. To
To produce
produce :2"
lower
year-end
year-end (July
(July 31, 1987) break-even
break-even condition
condition for
for the cost center,
center, we lowered
lowered the
as of
of February
February 1 to $75 per CPU
CPU hour.
hour. Figure
charge rate as
Figure 17 shows the clImulative
cumuiative
collected by month
month for
for the period
period August
August 1986 through
through April
April 1987 as
:is well
well
user revenues collected
as the ideal
ideal (linear)
(linear) cost center
center recovery
recovery line.
line.
as
The
The cost center
center rate for
for Stanford
Stanford users
users is expected to increase substantially
substantially at
zt the
beginning
NIH subsidy
beginning of
of each succeeding grant
grant year through
through renewal
renewai year 5, as
as NIH
subsidy of
of 2060
costs is incrementally
incrementally withdrawn.
withdrawn.
Remote Network
Network Costs
Until this
this year, the costs associated with
with networking
networkin g were supported
Until
NIH through
supported by NIH
through
University.
Beginning this
this grant
grant year, however,
however, NTH is funding
funding our
our networking
networking
Rutgers University.
Beginning
directly as
as part
part of
of our
our 2060 operations
operations budget, and
2nd we have entered
entered into
into a
costs directly
contract with
with TELENET
TELENET
Communications Corporation
Corporation for
for networking
networking services. To
To
contract
Communications
underscore our
our commitment
commitment to subsidize
subsidize the national
national AIM
AIM community'S
community’s 2060 usage
usage as
as
underscore
long as
as possible, we have been paying
payin g for
long
for TELENET
TELENET services direc[ly
direcrly from
from the
[he SU;vTEX
SUMEX
101
Shortliffe
E. H. Shortliffe
Management and Allocation
Allocation
Resource Management
5P41-RR00785-14
5P41-RR00785-13
grant
grant this
this year on the assumption
assumption that national
national community
community members
members would
would represent
represent
majority of
of TELENET
TELENET users.
users. However,
the vast majority
However, all other
other 2060-related
2060-related expenses are
charged directly
directly to the cost center and then charged out
out to Stanford
Stanford users
users according
according to
their CPU
CPU usage
usage and to the SUMEX
SUMEX grant
grant in keeping
keeping with
with its level of
of subsidy
subsidy of
of 2060
their
operations.
operations.
This
services directly
This early
early practice
practice of
of paying
paying for
for TELENET
TELENET
directly from
from the grant
grant has
has
complicated
complicated our
our accounting
accounting procedures, since networking
networking expenses
expenses must ultimately
ultimately be
into consideration
consideration in allocating
allocating total
total annual
annual 2060 operations
operations costs in correct
correct
taken into
proportions
proportions to the resource budget and to Stanford
Stanford users.
users. Also,
Also, a recent
recent analysis
analysis of
of our
our
networking
usage
by Stanford
networking
usage indicated
indicated that
that the use
use of
of TELENET
TELENET
Stanford groups
groups is
considerably
considerably higher
higher than expected. Therefore,
Therefore, since networking
networking services are not
not being
being
used exclusively
exclusively by the national
national user
user community
community as
as originally
originally believed,
believed, we
we plan
plan to
used
change our
our procedure
procedure and charge TELENET
TELENET costs directly
directly to the cost center
center in future
future
years.
1986-l 987
1986-1987
REVENUE
REVENUE vs
vs GOAL
GOAL
SUMEX 2060
2060
SUMEX
80
70 70
60 60
r
I!!
.!!
B
<;
8
Cl
Z
'0
La
Pc
.
"
5050
HMrn
40 40
::l
""
3::J
0
2
.J:::
I>-
30 JO
20 20
10
//
..- "
;'
."
"
o
-.
- - - -.
•
Currenl Total
Talal =~ $61.474
$61.474
Currenl
Figure
Figure 17:
E. H. Shortliffe
Shortliffe
Year-End
Year-End Goal
Goal z.
~ $71.376
$71.376
Cost Center
Center Performance
Performance
2060 Cost
102
Goal
Goal
Revenue
5P41-RR00785-l4
5P41-RR00785-14
Dissemination of
of Resource
Resource Information
Information
Dissemination
III.E. Dissemination
Dissemination of
of Resource Information
Information
1II.E.
We are
are continuing
continuing our
our past
past practice
practice of
of making
making a substantial
substantial effort
effort to
to disseminate
disseminate the
the
We
This has
has taken
taken the
the form
form of
of many
many publications
publications ---- over
over
AI technology
technology developed
developed here.
here. This
AI
forty-five combined
combined books
books and
and papers
papers are published
published per
per year
year by
by the
the KSL;
KSL; wide
wide
forty-five
distribution of
of our
our software
software including
including systems software
software and
and AI
AI application
application and
and tool
tool
distribution
software,
both
to
other
research
laboratories
and
for
commercial
development;
software, both to other
laboratories and for commercial development;
production of
of films
films and
and video
video tapes depicting
depicting aspects of
of our
our work;
work; and
and significant
significant
production
project efforts
efforts at
at studying
studying the
the dissemination
dissemination of
of individual
individual applications
applications systems
systems such
such as
project
the GENET
GENET community
community (DNA
(DNA sequence analysis
analysis software)
software) and
and the
the ONCOCIN
ONCOCIN resourceresourcethe
related research project
project (see 123).
123).
related
Software Distribution
Distribution
Software
We have widely
widely distributed
distributed both
both our
our system software
software and
and our
our AI
AI tool
tool software.
software. Since
Since
We
much
of
our
general
system-level
software
is
distributed
via
the
ARPANET
we
do
not
much of our general system-level software
distributed via the ARPANET
do not
complete records
records of
of the
the extent
extent of
of the distribution.
distribution. Software
Software such as TOPS-20
TOPS-20
have complete
monitor enhancements,
enhancements, the Ethernet
Ethernet gateway and TIP
TIP programs,
programs, the
the SEAGATE
SEAGATE
monitor
AppleBus to
to Ethernet
Ethernet gateway, the PUP
PUP Leaf
Leaf server, the
the SUMACC
SUMACC development
development system
AppleBus
for Macintosh
Macintosh workstations,
workstations, and our
our Lisp
Lisp workstation
workstation programs
programs are frequently
frequently
for
distributed in
in this
this manner
manner to the ARPANET
ARPANET community
community and
and beyond.
beyond. Since
Since our
our
distributed
SUMACC
development system for
for Macintosh
Macintosh workstations
workstations is considered
considered to
to be
SUMACC software
software development
domain", we have turned
turned itit over
over to Information
Information Analysis
Analysis Associates,
Associates,
in the "public
in
“public domain”,
Mountain
View,
CA.
for
distribution
(for
a
minimum
charge)
to
groups
not
Mountain View, CA. for distribution (for
minimum charge) to groups not associated
ARPANET.
with the ARPANET.
with
Our
effort
AI tool
tool material.
material. In
In recent
recent
Our primary
primary distribution
distribution
effort is directed
directed )wards
awards our
our AI
years, the volume
for tapes has been
volume of
of inquiries
inquiries for
for this
this type 0.'
0;‘ software
software and requests for
over the past three
three years there
there
a substantial
substantial burden
burden on the staff.
staff. Records indicate
indicate that
that over
in the distribution
distribution of
of written
written
have been about
about 1,050 inquiries
inquiries that
that have resulted
resulted in
material
number
material about
about our
our software
software systems. It
It is likely
likely that
that there
there have been a similar
similar number
of
of the staff.
It
therefore
It was therefore
of unrecorded
unrecorded or
or informal
informal interactions
interactions on the part
part of
staff.
decided to turn
turn over
over most
most of
of this
this type of
of software
software distribution
distribution to Stanford's
Stanford’s Office
Office of
of
Technology
Technology Licensing
Licensing (OTL).
(OTL).
This
This organization
organization handles
handles software
software distribution
distribution and technology
technology licensing
licensing matters
matters for
for
much
Since there are several OTL
much of
of the Stanford
Stanford community.
community.
OTL staff
staff members
members assigned
to the distribution
distribution of
of Stanford
Stanford software,
software, requests for
for information
information and tapes are handled
handled
quickly
Also,
quickly and efficiently.
efficiently.
Also, OTL's
OTL’s staff
staff has
has the expertise
expertise needed to handle
handle the legal
of
questions
and an established
established
questions that
that frequently
frequently arise in the distribution
distribution
of software,
software, and
computerized
computerized record-keeping
record-keeping scheme. SUMEX
SUMEX staff
staff continues
continues to be available
available as
as needed
to assist OTL
OTL with
with special administrative
administrative and technical
technical matters.
Unfortunately,
Unfortunately, start-up
start-up delays in the transfer
transfer of
of software
software distribution
distribution to the Office
Office of
of
Technology
Technology Licensing
Licensing and the preparation
preparation of
of new versions
versions of
of MRS
MRS and BB1
BBl have
temporarily
temporarily reduced our
our distribution
distribution volume.
volume. During
During this
this report
report period
period we distributed
distributed
eleven copies of
MRS,
eight
copies
of
BBI
and
one
each
of
AGE,
of MRS. eight
of BBl
each of AGE, EMYCIN,
EMYCIN, GENOA,
GENOA,
and CONGEN.
CONGEN. During
During the past year the reconstruction
reconstruction of
of the distribution
distribution packages
packages for
for
the DENDRAL
DENDRAL Project
Project (GENOA
(GENOA and CONGEN)
CONGEN) has
has been completed.
completed. In December
December of
of
this
this year,
year, aa five-year
five-year exclusive
exclusive licensing
licensing agreement
agreement (with
(with Molecular
Molecular Designs, Ltd.)
Ltd.) for
for
the
material
in
the DENDRAL
DENDRAL
material will
will expire,
expire, and
and we
we will
will therefore
therefore have more
more flexibility
flexibility
distributing
distributing this
this material.
material.
We
We continue
continue to
to make
make aa special
special effort
effort to
to assist
assist other
other members of
of the SUMEX-AIM
SUMEX-AIM
community
community in
in integrating
integrating the
the technologies
technologies needed
needed for
for biomedical
biomedical AI
AI research. This
This is
often
achieved
through
direct
contact
with
staff
members
at
these
institutions
at
often achieved through direct contact with staff members at these institutions
at
meetings
meetings and
and workshops
workshops or
or via
via electronic
electronic mailing
mailing lists.
lists. For
For example,
example, the
the Info-ll00
Info-1100
103
103
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Dissemination of
of Resource
Resource Information
Information
Dissemination
mailing list,
list, which
which is maintained
maintained at SUMEX-AIM,
SUMEX-AIM,
hundred members
members (users
mailing
has several hundred
of
of Xerox
Xerox 1100 Series equipment)
equipment) and is monitored
monitored by our
our staff.
staff. This
This list
list is used
used to
distribute things
things like
like hardware
hardware and software
software bug reports
reports and fixes
fixes and system tools
tools and
distribute
is very
very valuable
valuable to the AIM
AIM community
community Interlisp
Interlisp users.
users.
Video
Video Tapes and Films
Films
The
The KSL
KSL and the ONCOCIN
ONCOCIN project
project have prepared
prepared several video
video tapes that
that provide
provide an
overview
overview of
of the research and research methodologies
methodologies underlying
underlying our
our work
work and that
that
available through
through our
our
demonstrate
demonstrate the capabilities
capabilities of
of particular
particular systems. These tapes are available
groups, the Fleischmann
Fleischmann Learning
Learning Center
Center at the Stanford
Stanford Medical
Medical Center,
Center, and the
groups,
Stanford
Stanford Computer
Computer Forum,
Forum, and copies
copies have been mailed
mailed to program
program offices
offices of
of our
our
various
various funding
funding sponsors. The
The three
three tapes include:
include:
•. Knowledge
Knowledge Engineering
Engineering in the Heuristic
Heuristic Programming
Programming Project
Project --- This
This 2020minute
minute film/tape
film/tape illustrates
illustrates key ideas in
in knowledge-based
knowledge-based system design and
ONCOCIN,
PROTEAN,
implementation,
using
ONCOCIN,
PROTEAN,
and
implementation,
using examples
examples from
from
It describes
describes the research environment
environment
knowledge-based
knowledge-based VLSI
VLSI design systems. It
of
of the KSL
KSL and lays out
out the methodologies
methodologies of
of our
our work
work and the long-term
long-term
research goals that
that guide
guide it.
it.
ONCOCIN Overview
Overview --- This
This is a 30-minute
30-minute tape providing
providing an overview
overview of
of
•. ONCOCIN
the ONCOCTN
ONCOCIN project.
project. It
It gives
gives an historical
historical context
context for
for the work,
work, discusses
discusses
the clinical
clinical problem
problem and the setting
setting in
in which
which the prototype
prototype system is being
being
used,
used, and outlines
outlines the plans
plans for
for transferring
transferring the system to run
run on single-user
single-user
workstations.
Brief
workstations.
Brief illustrations
illustrations of
of the graphics
graphics capabilities
capabilities of
of ONCOCIN
ONCOCIN
on a Lisp
Lisp workstation
workstation are also provided.
provided.
•. ONCOCIN
Demonstration -ONCOCIN Demonstration
-- This
This I-hour
l-hour tape provides
provides detailed
detailed examples of
of
It begins with
with a
the key components
system.
It
components of
of the ONCOCIN
ONCOCIN
demonstration
of
demonstration
of the prototype
prototype system's
system’s performance
performance on a time-shared
time-shared
mainframe
mainframe computer
computer and then
then shows each of
of the elements
elements involved
involved in
transferring
transferring the system to Lisp
Lisp workstations.
workstations.
E. H. Shortliffe
Shortliffe
104
Suggestions and
and Comments
Comments
5P41- RR00785-14
5P41-RR00785-14
III.F. Suggestions and Comments
1II.F.
Resource Organization
Organization
Resource
We continue
continue to
to believe
believe that
that the
the Biomedical
Biomedical Research Technology
Technology Program
Program is one of
of the
We
most effective
effective vehicles
vehicles for
for developing
developing and
and disseminating
disseminating technological
technological tools
tools for
for
most
biomedical research.
The goals and methods
methods of
of the program
program are well-designed
well-designed to
to
The
biomedical
encourage building
building of
of the
the necessary multi-disciplinary
multi-disciplinary groups
groups and merging
merging of
of the
encourage
appropriate technological
technological and
and medical
medical disciplines.
disciplines.
appropriate
Electronic Communications
Communications
Electronic
SUMEX-AIM has pioneered
pioneered in
in developing
developing more
more effective
effective methods
methods for
for facilitating
facilitating
SUMEX-AIM
scientific communication.
communication. Whereas
Whereas face-to-face
face-to-face contacts
contacts continue
continue to play
playaa key role,
role, in
in
scientific
longer-term we feel
feel that
that computer-based
computer-based communications
communications will
will become increasingly
increasingly
the longer-term
important to the NIH
NIH and the distributed
distributed resources of
of the biomedical
biomedical community.
community. We
important
promoting these tools
tools within
within the
would like
like to see
see the BRTP
BRTP take a more
more active
active role
role in promoting
would
NIH and its
its grantee
grantee community.
community.
NTH
105
105
Shortliffe
E. H. Shortliffe
Description
Description of
of Scientific
Scientific Subprojects
Subprojects
SP41-RR0078S-14
5P41-RR00785-14
IV. Description
Description of
of Scientific
Scientific Subprojects
Subprojects
IV.
The following
following subsections
subsections report
report on
on the
the AIM
AIM community
community of
of projects
projects and
and “pilot”
"pilot" efforts
efforts
The
including local
local and
and national
national users of
of the
the SUMEX-AIM
SUMEX-AIM facility
facility at
at Stanford.
Stanford. However,
However,
including
admitted to the
the National
National AIM
AIM community
community which
which use the
the Rutgers-AIM
Rutgers-AIM
those projects
projects admitted
those
resource as their
their home
home base are not
not explicitly
explicitly reported
reported here.
resource
In addition
addition to
to these detailed
detailed progress
progress reports,
reports, abstracts
abstracts for
for each project
project and
and its
its
In
individual users are submitted
submitted on
on a separate Scientific
Scientific Subproject
Subproject Form.
Form. However,
However, we
individual
included here briefer
briefer summary
summary abstracts
abstracts of
of the
the fully-authorized
fully-authorized projects
projects in
In
have included
Appendix B
B on
on page 221.
22l.
Appendix
The collaborative
collaborative project
project reports
reports and
and comments
comments are the result
result of
of a solicitation
solicitation for
for
The
contributions sent
sent to
to each of
of the
the project
project Principal
Principal Investigators
Investigators requesting
requesting the following
following
contributions
information:
information:
I. SUMMARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
A. Project
Project rationale
rationale
A.
Medical relevance
relevance and collaboration
collaboration
B. Medical
of research progress
C. Highlights
Highlights of
--Accomplishments this
this past year
--Accomplishments
-- Research in
in progress
progress
--Research
List of
of relevant
relevant publications
publications
D. List
Funding support
support
E. Funding
II. INTERACTIONS
INTERACTIONS WITH
WITH THE
THE SUMEX-AIM
RESOURCE
II.
SUMEX-AIM
RESOURCE
A. Medical
Medical collaborations
collaborations and program
program dissemination
dissemination via
via SUMEX
A.
SUMEX
B. Sharing
projects
Sharing and interactions
interactions with
with other
other SUMEX-AIM
SUMEX-AIM
projects
(via
(via computing
computing facilities,
facilities, workshops,
workshops, personal
personal contacts,
contacts, etc.)
C. Critique
Critique of
of resource
resource management
management
(community
(community facilitation,
facilitation, computer
computer services, communications
communications
services, capacity,
capacity, etc.)
etc.)
III.
III. RESEARCH
RESEARCH PLANS
PLANS
A. Project
Project goals and plans
plans
---Near- N ear- term
term
--Long-range
--Long-range
B. Justification
Justification and requirements
requirements for
for continued
continued SUMEX
SUMEX use
use
C. Needs and plans for
- AIM
for other
other computing
computing resources beyond
beyond SUMEX
SUMEX-AIM
D. Recommendations
Recommendations for
for future
future community
community and resource development
development
We believe
believe that
that the reports
reports of
of the individual
individual projects
projects speak for
for themselves as
as rationales
rationales
for
In any case,
case, the reports
reports are recorded
recorded as
as submitted
submitted and are the
for participation.
participation.
The only
only exceptions
exceptions are the respective
respective
responsibility
responsibility of
of the indicated
indicated project
project leaders. The
which
formatted
lists
lists of
of relevant
relevant publications
publications
which have been uniformly
uniformly
formatted for
for parallel
parallel
reporting
reporting on the Scientific
Scientific Subproject
Subproject Form.
Form.
107
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Stanford Projects
Projects
Stanford
IV.A. Stanford
Stanford Projects
Projects
1V.A.
The following
following group
group of
of projects
projects is formally
formally approved
approved for
for access
access to
to the
the Stanford
Stanford aliquot
aliquot
The
of the
the SUMEX-AIM
SUMEX -AIM resource.
resource. Their
Their access
access is
is based on
on review
review by
by the
the Stanford
Stanford
of
Advisory Group
Group and approval
approval by
by Professor
Professor Feigenbaum
Feigenbaum as Principal
Principal Investigator.
Investigator.
Advisory
In addition
addition to
to the progress
progress reports
reports presented
presented here, abstracts
abstracts for
for each project
project and
and its
its
In
individual users are submitted
submitted on
on a separate
separate Scientific
Scientific Subproject
Subproject Form.
Form.
individual
E.
E. H. Shortliffe
Shortliffe
108
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
5P41-RR00785-14
5P41-RR00785-14
IV.A.l.
Project
IV.A.l. GUIDON/NEOMYCIN
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
William J. Clancey,
Clancey, Ph.D.
Ph.D.
William
Department
Department Computer
Computer Science
Stanford
Stanford University
University
Bruce
Bruce G. Buchanan,
Buchanan, Ph.D.
Ph.D.
Computer
Computer Science Department
Department
Stanford University
University
Stanford
I.
SUMMARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
A. Project
Project Rationale
Rationale
The
Project
The GUIDON/NEOMYCIN
GUIDON/NEOMYCIN
Project is a research program
program devoted
devoted to the development
development
The key issue for
for
of
for application
application to medicine.
medicine. The
of a knowledge-based
knowledge-based tutoring
tutorin g system for
the GUlDON/NEOMYCIN
project
GUIDON/NEOMYCIN
project is to develop
develop a program
program that
that can provide
provide advice
advice
similar
similar in
in quality
quality to that
that given
given by human
human experts, modeling
modeling how
how they structure
structure their
their
knowledge
knowledge as
as well
well as
as their
their problem-solving
problem-solving procedures.
procedures. The
The consultation
consultation program
program using
NEOMYCIN. NEOMYCIN’
NEOMYCIN's
knowledge
this
this knowledge
knowledge is called
called NEOMYCIN.
s
knowledge base,
base, designed for
for use
use
in
in a teaching
teaching application,
application, is the subject
subject material
material used by a family
family of
of instructional
instructional
programs
The
procedures
programs referred
referred to collectively
collectively as GUIDON2.
GUIDON2.
The problem-solving
problem-solving
procedures are
developed
NEOMYCIN and comparing
developed by running
running test cases
cases through
through NEOMYCIN
comparing them
them to expert
expert
behavior.
Also,
NEOMYCIN as
Also, we use
use NEOMYCIN
as a test bed for
for the explanation
explanation capabilities
capabilities
behavior.
incorporated in
in our
our instructional
instructional programs.
programs.
incorporated
The
The purpose of
of the current
current contracts
contracts is to construct
construct a knowledge-based
knowledge-based tutoring
tutoring system
that
By strategy,
that teaches
teaches diagnostic
diagnostic strategies
strategies explicitly.
explicitly.
strategy, we mean plans for
for establishing
establishing
a set of
of possible diagnoses, focusing
focusing on and confirming
confirming individual
individual diagnoses, gathering
gathering
data, and processing new data. The
The tutorial
tutorial program
program has capabilities
capabilities to recognIze
recognize these
rhese
plans, as
as well
well as
as to articulate
articulate strategies
strategies in
in explanations
explanations about
about how to do diagnosis.
diagnosis. The
The
strategies represented in the program,
program, modeling
modeling techniques,
techniques, and explanation
explanation techniques
techniques
are wholly
wholly separate from
from the knowledge
knowledge base,
base, so that
that they can be used
used with
with many
many
That is, the target
target program
program will
will be able to be
medical
medical (and non-medical)
non-medical) domains.
domains. That
tested with
with other
other knowledge
knowledge bases.
bases. using system-building
system-building [Ools
tools that
that we provide.
provide.
Medical Relevance and
B. Medical
and Collaboration
Collaboration
There
There is a growing
growing realization
realization that
that medical
medical knowledge,
knowledge, originally
originally codified
codified for
for the
purpose
purpose of
of computer-based
computer-based consultations,
consultations, may
may be used
used in
in additional
additional ways that
that are
medically
medically relevant.
relevant. Using
Using the knowledge
knowledge to teach medical
medical students
students is perhaps foremost
foremost
among
among these,
these, and GUIDON2
GUIDON2 focuses on methods
methods for
for augmenting
augmenting clinical
clinical knowledge
knowledge in
A particularly
particularly important
important aspect of
of this
this
order
order to facilitate
facilitate its use
use in
in a tutorial
tutorial setting.
setting. A
work is the insight
insight that
that has
has been gained
gained regarding
regarding the need to structure
structure knowledge
knowledge
work
differently,
differently, and in
in more
more detail,
detail, when it
it is being
being used for
for different
different purposes (e.g.,
(e.g.,
It
teaching
teaching as
as opposed to clinical
clinical decision
decision making).
making).
It was this
this aspect of
of the GUIDON
GUIDON
research that
NEOMYCIN, which
that led to the development
development of
of NEOMYCIN,
which is an evolving
evolving
computational
computational model
model of
of medical
medical diagnostic
diagnostic reasoning
reasoning that
that we hope will
will enable
enable us
us to
An important
important additIOnal
additional realization
realization
better
better understand
understand and teach diagnosis
diagnosis to students. An
the problem-solving
is that
that these
these structuring
structuring methods
methods are beneficial
beneficial for
for improving
improving
problem-solving
programs, providing
providing
more detailed
detailed
md
abstract
performance
of
programs,
more
~1I1d
~lbstract
performance
of consultation
consultation
explanations
explanations to consultation
consultation users,
users, and making
making knowledge
knowiedge bases
bases easier to m::unLain.
mnlnrain.
109
E. H. Shortliffe
Shortliffe
5P41RR00785-14
5P41-RR00785-14
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
As we move
move from
from technological
technological development
development of
of explanation
explanation and student
student modeling
modeling
capabili
ties, we are now
capabilities,
now collaborating
collaborating closely
closely with
with medical
medical students
students and physicians
physicians to
In particular,
particular, medical
medical students
students have served
design an effective,
effective, useful
useful tutoring
tutoring program.
program. In
as
as research assistants.
assistants, and a current
current MSAI
MSAI student
student is an experienced
experienced physician.
physician, John
John
Sotos.
The
The project
project also collaborates
collaborates with
with a community
community of
of
Sotos, from
from Johns
Johns Hopkins.
Hopkins.
focusing on medical
medical education,
education, funded
funded by the Josiah Macy.
Macy, Jr. Foundation.
Foundation.
researchers focusing
C. Highlights
Highlights of
Research Progress
Progress
of Research
C.I
Accomplishments This
C.1 Accomplishments
This Past Year
C.l.I The GUIDON-DEBUG
GUIDON-DEBUG Tutoring
Tutoring Program
C.l.l
Program
We
We began 1986 with
with a concerted
concerted effort
effort to construct
construct a tutorial
tutorial program
program called
called
GUIDON-DEBUG. The
The idea behind
behind this
this system is to have a student
student debug a faulty
faulty
GUIDON-DEBUG.
A prototype
prototype was
knowledge
A
knowledge base by using graphic
graphic explanation
explanation and editing
editing tools.
tools.
demonstrated at the annual
annual ONR
ONR conference
conference in March.
March.
However, after
after trials
trials with
with
demonstrated
However.
medical
medical students
students we realized
realized that
that 1) it
it was difficult
difficult to choose a fault
fault at the right
right level
level
of
of difficulty
difficulty for
for a student,
student, and 2) the program
program lacked
lacked ability
ability to help the students
students and
evaluate their
their debugging
debugging because
because it
it lacked
lacked an internal
internal model
model of
of how to debug. We
evaluate
GUIDON-DEBUG development
development should
should be deferred
deferred until
until the proposed
proposed
concluded that
that GUIDON-DEBUG
concluded
knowledge
knowledge acquisition
acquisition module
module (see
(see below)
below) is completed.
completed.
C.I.2
Program
C.1.2 The GUIDON-MANAGE
GUIDON-MANAGE Tutoring
Tutoring Program
At this
this point
point we returned
returned to an alternative
alternative conception
conception described
described in our
our original
original
At
proposal,
This
proposal, a program
program called GUIDON-MANAGE.
GUIDON-MANAGE.
This program
program teaches
teaches a student
student the
of diagnosis
diagnosis by having
having him
him or
or her enter
enter all requests for
for patient
patient information
information as
as
language of
abstraction.
Thus, the student
student issues
issues "strategic
“strategic commands"
commands” such as
as "test
“test the
an abstraction.
Thus,
question
hypothesis
hypothesis meningitis"
meningitis” or
or "ask
“ask a follow-up
follow-up
question about
about the headache,"
headache,” and the
program
program (NEOMYCIN)
(NEOMYCIN) carries
carries out
out the tactics.
tactics. By year end, this
this program
program was well
well along.
along,
with
NEOMYCIN to generate help.
help, a feedback
feedback
with a complex
complex interpreter
interpreter for
for simulating
simulating NEOMYCIN
window
window to indicate
indicate what
what NEOMYCIN
NEOMYCIN did
did when it
it carried
carried out
out the commands,
commands, and many
many
continues to focus
focus on
menus for
Research continues
for making
making input
input to the program
program convenient.
convenient.
feedback components
components of
of the program.
program.
the assistance and feedback
conceived to be the first
first step in a three-step
three-step tutorial
tutorial program
program
is now conceived
which will
will include
include GUIDON-WATCH
GUIDON-WATCH (which
(which we previously
previously developed)
developed) and a yet to be
which
named
named tutorial
tutorial module.
module. In
In these three steps, the student
student will
will solve a problem,
problem, watch
watch
NEOMYCIN
NEOMYCIN solve
solve a problem,
problem, and then explain
explain his solution
solution and seek explanations
explanations about
about
NEOMYCIN'S solution.
solution. In this way, we use
use the program
program as
as a model
model that
that the student
student can
NEOMYCIN'S
study and compare
compare to his own reasoning.
reasoning.
study
GUIDON-MANAGE
GUIDON-MANAGE
C.1.3 The GUIDON-MANAGE
Program
GUIDON-MANAGE Tutoring
Tutoring Program
in explanation
explanation is another
another major
major area. This
This year we completed
completed some difficult
difficult
Research in
programming that
that allows
allows us to examine
examine a history
history in
in detail
detail of
of everything
everything NEOMYCIN
NEOMYCIN did
did
programming
solving a problem.
problem. With
With this
this foundation,
foundation, we can now
now go back and summarize
summarize
when solving
lines of
of reasoning
reasoning for
for any point
point during
during the previous
previous consultation.
consultation. In our
our first
first program,
program,
lines
completed
completed in
in 1984, we "translated"
“translated” steps (metarules)
(metarules) using text
text strings
strings built
built into
into the
program. Now
program.
Now we seek
seek to generate these strings
strings automatically
automatically by having
having the program
program
metarules and select statements
statements to mention.
mention.
This project
project makes significant
significant
read the metarules
This
contributions to text
text generation
generation research, a somewhat
somewhat ignored
ignored area of
of natural
natural language
contributions
research.
C.IA
Modeling Program
Program
C.l.4 The ODYSSEUS
ODYSSEUS Modeling
Our third
third tutorial-related
tutorial-related
project involves
involves continued
continued development
development of
of a modeling
modeling
Our
project
The
program,
program, ODYSSEUS.
ODYSSEUS.
The purpose of
of ODYSSEUS
ODYSSEUS is to discover
discover domain
domain knowledge
knowledge
E. H. Shortliffe
Shortliffe
no
110
5P41-RR00785-14
5P41-RR00785-14
GUIDON/NEOMYCIN Project
Project
GUIDON/NEOMYCIN
discrepancies between
between an application
application domain
domain knowledge
knowledge base (e.g. the
the Neomycin
Neomycin medical
medical
discrepancies
or expert
expert problem
problem solver.
solver. IMAGE,
TMAGE, an earlier
earlier modeling
modeling
knowledge base) and
and a student
student or
knowledge
not address this
this problem.
problem. The
The input
input to
to ODYSSEUS
ODYSSEUS is the
the
program developed
developed in
in 1982, did
did not
program
ODYSSEUS watches a student
student itit functions
functions
problem solver’
solver's
patient data
data requests. When
When ODYSSEUS
problem
s patient
student modeling
modeling program
program for
for GUIDON2
GUTDON2 and when
when itit watches an expert
expert itit functions
functions
as a student
as a knowledge
knowledge acquisition
acquisition program
program for
for HERACLES.
The approach
approach used by
by ODYSSEUS
ODYSSEUS to detect
detect domain-level
domain-level discrepancies
discrepancies may
may be
The
failure-driven learning
learning by
by completing
completing explanations.
explanations. An
An explanation
explanation
characterized as failure-driven
characterized
failure occurs
occurs when
when ODYSSEUS
ODYSSEUS is unable
unable to create a proof
proof tree consisting
consisting of
of instantiated
instantiated
failure
In creating
creating
metarules that
that links
links an observable
observable student
student action
action to a high-level
high-level task
task goal.
goal. In
metarules
proof trees, a top-down
top-down simulation
simulation first
first produces
produces a set of
of plausible
plausible high-level
high-level
these proof
information is used by
by a
and updates problem
problem solving
solving state information;
information; then
then this
this information
goals and
bottom-up generation
generation from
from the
the observable
observable action
action to
to these high
high level
level goals.
constrained bottom-up
constrained
proof tree can be generated
generated for
for an action
action and
and
explanation failures
failures occurs
occurs when
when no
no proof
AA explanation
this suggests
suggests a domain
domain level
level discrepancy.
discrepancy.
this
ODYSSEUS resolves this
this failures
failures in
in two
two steps. First,
First, the constraints
constraints on
on proof
proof tree
ODYSSEUS
of
generation are relaxed;
relaxed; this
this identifies
identifies relations
relations in
in metarules
metarules that
that might
might be the source of
generation
discrepancy and produces
produces a set of
of instantiations
instantiations for
for each of
of these relation
relation that
that are
the discrepancy
confirmation theory
theory tests these
candidate domain-level
domain-level discrepancies.
discrepancies. Second, a confirmation
the candidate
for plausibility.
plausibility.
candidate discrepancies
discrepancies for
candidate
ODYSSEUS has been enhanced to operate
operate directly
directly off
off an arbitrary
arbitrary
During the
the last
last year, ODYSSEUS
During
of Heracles
Heracles control
control metarules;
metarules; previously
previously the modeling
modeling program
program incorporated
incorporated
set of
knowledge about
about the particular
particular metarules
metarules that
that were used
used in
in Neomycin.
Neomycin. This
This increases
knowledge
of the program
program at the cost of
of a large
large increase
increase in
in the
generality and applicability
applicability of
the generality
search space. Initial
Initial validation
validation tests of
of Odysseus
Odysseus have been conducted
conducted and this
this has
revealed that
that following
following the strategic
strategic reasoning
reasoning of
of human
human problem
problem solvers
solvers is crucially
crucially
dependent
dependent on having
having a very
very good domain
domain knowledge
knowledge base.
base. Besides these tests on human
human
problem solvers,
problem
solvers, a validation
validation methodology
methodology called
called the synthetic
synthetic agent method
method has been
performance bound.
designed that
that allows
allows determination
determination of
of an upper
upper performance
bound. During
During the next
next
year, ODYSSEUS
ODYSSEUS will
will be completed,
completed, integrated
integrated with
with all
all parts of
of Guidon
Guidon including
including the
explanation
program,
explanation and Guidon-Manage
Guidon-Manage
program, and validated.
validated. A
A case
case library
library for
for the
Neomycin domain
Neomycin
domain will
will be constructed
constructed since this
this plays a crucial
crucial role
role in
in validation
validation and
assessment
assessment of
of the ODYSSEUS
ODYSSEUS approach.
approach.
C.l.5
HERACLES Expert
Expert System
C.1.S The HERACLES
System Shell
Shell
The
The final
final major
major effort
effort involves
involves generalizing
generalizing our
our expert
expert system tool,
tool, HERACLES, so that
that it
it
can be made available
available to other
other research groups who wish
wish to develop
develop knowledge
knowledge bases
bases
which
which can be tutored
tutored by GUTDON2.
GUTDON2. This
This project
project involves
involves a great deal of
of basic systems
programming,
programming, including
including partitioning
partitioning of
of files
files and regrouping
regrouping of
of general
general and knowledgeknowledgebase-specific
base-specific constructs.
constructs. By year end, we were ready to reconfigure
reconfigure a second program
program
built
tools
during 1985, called
called CASTER, to test out
out the system-building
system-building
tools
built in
in HERACLES during
developed
developed to date.
A
A host of
of smaller
smaller projects
projects included:
included:
. Maintenance
IMaintenance of
of
performance
performance. .
our
our
patient
patient library
library and
records
of
of
proper
proper
program
program
•. Development
Development of
of a graphics
graphics editor
editor for
for modifying
modifying the knowledge
knowledge base
base by
"correcting"
“correcting” the program's
program’s diagnosis
diagnosis of
of a particular
particular case
case..
•. Development
This
This
Development of
of menu-based
menu-based knowledge-base
knowledge-base retrieval
retrieval capability.
capability.
program
program constructs
constructs menus that
that bring
bring together
together details
details related to some fact
fact the
user
user has
has just
just asked
asked a question
question about.
III
111
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
•. More
More consistent
consistent storage and convenient
convenient access
access to "normal
“normal values"
values” for
for patient
patient
tests and findings
findings. .
•. Development
Development of
of a package for
for creating,
creating, editing,
editing, and replaying
replaying "scripts"
“scripts”
which
which take the viewer
viewer on a tour
tour of
of some aspect of
of NEOMYCIN.
NEOMYCIN. Useful
Useful for
for
documentation,
simple
of
documentation,
simple lectures,
lectures, and automatic
automatic demonstrations
demonstrations
of the
program.
program.
C.l.6
Model of
Learning
C.1.6 Model
of Learning
Finally,
Finally, in
in a paper
paper described
described below,
below, we developed
developed a theory
theory of
of learning
learning by
by debugging
debugging
This
using
using knowledge
knowledge of
of diagnostic
diagnostic strategy
strategy and organization
organization of
of disease
disease knowledge.
knowledge. This
theory now
now forms
forms the foundation
foundation for
for design of
of GUIDON2.
GUlDON2. In
In our
our current
current work,
work, we are
theory
focusing
focusing on the modeling,
modeling, explanation,
explanation, and knowledge
knowledge acquisition
acquisition capabilities
capabilities that
that will
will
allow
allow the tutor
tutor to articulate
articulate how
how a diagnostic
diagnostic solution
solution is flawed
flawed and how
how it
it can be
improved
improved using
using specific
specific domain
domain knowledge.
knowledge. Thus,
Thus, we are teaching
teaching the constraints
constraints a
good solution
solution must
must respect, plus
plus giving
giving the students
students a language for
for articulating
articulating what
what
good
medical facts
facts are relevant
relevant to the case
case at hand.
medical
C.i.7 Dissemination
of results
results
C.l.7
Dissemination of
There
There were many
many conferences
conferences relating
relating to our
our work
work this
this year. Most
Most notable
notable were the
"Tutoring
“Tutoring system workshop"
workshop” in
in Windermere,
Windermere, England
England (travel
(travel support
support from
from the AAAI)
AAAI)
“Knowledge acquisition
acquisition workshop"
workshop” in
in Banff,
Banff, British
British Columbia.
Columbia.
Other useful
useful
and the "Knowledge
Other
workshops
tools"
Clancey
workshops concerned
concerned "Higher-level
“Higher-level
tools” and "Knowledge
“Knowledge compilation."
compilation.”
Clancey
presented prominent
prominent papers at each of
of these workshops
workshops and helped organize
organize the middle
middle
presented
two.
Clancey also presented
presented Guidon/Neomycin
Guidon/Neomycin
work at additional
additional conferences
conferences in
two.
Clancey
work
Milan,
New Mexico,
Milan, London,
London, New
Mexico, Arizona,
Arizona, and Florida.
Florida.
The
The Macy
Macy Foundation
Foundation Symposium
Symposium on
on Cognitive
Cognitive Science and Medical
Medical Education
Education in
in
Montreal,
Montreal, run
run by John
John Bruer,
Bruer, was extremely
extremely valuable
valuable for
for the grantees. Researchers
working
Feltovich,
working on medical
medical instruction
instruction included:
included:
Feltovich, Evans, Hammond,
Hammond, Elstein,
Elstein, and
Patel. Small
Small meetings
meetings are unusual
unusual in
in this
this field
field (AAAI
(AAAI has more
more than
than 5000 attendees);
attendees):
discussions were detailed
detailed and illuminating.
illuminating.
the discussions
Guidon/Neomycin
work
Guidon/Neomycin
work will
will be represented
represented in
in 1987 at Clancey's
Clancey’s tutorial
tutorial on "Evaluating
“Evaluating
expert system tools"
tools” and his tutorial
tutorial on
on tutoring
tutoring systems at IJCAI
IJCAI in
in Milan.
Milan.
expert
C.2 Research
Research in
Progress
in Progress
The
The following
following projects
projects are active
active as
as of
of May
May 1987 (see
(see also near-term
near-term plans listed
listed in
Section IILA):
1II.A):
Section
1. Developing
Developing additional
additional instructional
instructional programs
programs based
based on NEOMYCIN;
NEOMYCIN;
2. Studying
Studying learning
learning in the setting
setting of
of debugging
debugging a knowledge
knowledge base;
base;
3. Re-implementing
the explanation
Re-implementing
explanation program
program to use
use the logic-encoding
logic-encoding of
of the
metarules
metarules (stating
(stating this
this program
program in
in the same task/metarule
taskimetarule language so that
that it
it
might reason about
about its
its own
own explanations);
explanations);
might
Developing
graphic methods
4. Developing
new graphic
methods for
for making
making presentations
presentations from
from the
knowledge
knowledge base,
base, including
including tour-like
tour-like lectures
lectures and "dynamic
“dynamic menus"
menus” which
which
bring together
together items
items relevant
relevant to previous
previous user inquiries;
inquiries;
bring
5. Applying
the student
Applying
student
acquisition;
acquisition; and
modeling
modeling
program,
program,
ODYSSEUS,
ODYSSEUS,
to
knowledge
knowledge
Preparing HERACLES,
HERACLES, the generalization
generalization of
of NEOMYCIN,
for use
use by other
other
6. Preparing
NEOMYCIN, for
people.
people.
Shortliffe
E. H. Shortliffe
112
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
SP41-RR0078S-14
5P41-RR00785-14
Publications Since
January 1986
D. Publications
Since January
I986
1. Clancey,
D.C., Barnhouse,
Clancey, W.J., Richer,
Richer, M.,
M., Wilkins,
Wilkins,
Barnhouse, S.,
S., Kapsner,
Kapsner, C.,
Leserman,
Leserman, D., Macias,
Macias, J., Merchant,
Merchant, A., and Rodolitz,
Rodolitz, N.: Guidon-Debug:
Guidon-Debug:
The student
student as knowledge
knowledge engineer.
engineer. KSL
KSL Working
Working paper
paper 86-34.
86-34.
2. Clancey,
Student
Models. Annual
Clancey, W.J.: Qualitative
Qualitative
Student Models.
Annual Review of
of Computer
Computer
Science. Palo
Palo Alto:
Alto: Annual
Annual Reviews,
Reviews, Report
Report KSL-86-11,
KSL-86-11, Computer
Computer Science
Dept.,
Dept., May
May 1986.
3. Clancey,
From GUIDON
NEOMYCIN and
HERACLES in
Clancey, W.J.: From
GUIDON to NEOMYCIN
and HERACLES
in twenty
twenty
short lessons:
Final Report,
Report, 1979-1985.
AI Magazine,
Magazine, 7(3):40-60,
short
lessons: ONR
ONR Final
1979-1985. The AI
7(3):40-60,
Conference,
Conference, 1986.
4. Wilkins,
Wilkins, D.C.,
D.C., Clancey,
Clancey, W.J., Buchanan,
Buchanan. B.G. An
An overview
overview of
of the ODYSSEUS
ODYSSEUS
learning
learning apprentice.
apprentice. In Machine
Machine Learning:
Learning: A
A Guide to Current
Current Research,
Research, eds.
eds.
T.M.
T.M. Mitchell,
Mitchell, 1.G.,
J.G., Carbonell,
Carbonell, and R.S. Michalski.
Michalski.
New York,
York, Academic
Academic
Press,
pages 369-373.
Press, pages
369-373. Also
Also KSL-8S-_26.
KSL-85-26.
S.
5. Clancey,
Clancey, W.J. Intelligent
Intelligent tutoring
tutoring systems: A
A tutorial
tutorial survey. International
International
Professorship
Professorship Series, 1985 Academic
Academic Press,
Press, Inc.,
Inc., London,
London, in press.
press.
6. Wilkins,
Wilkins, D.C.,
D.C., Clancey,
Clancey, W.J., and Buchanan,
Buchanan, B.G. On Using
Using and Evaluating
Evaluating
Differential
Modeling
Tutoring
Intelligent
Tutoring and Apprentice
Apprentice Learning
Learning
Differential
Modeling in
in Intelligent
Systems. In
Lessons Learned, eds.
eds. J.
J. Psotka,
Psotka?
In Intelligent
Intelligent Tutoring
Tutoring Systems: Lessons
D. Massey, and
and S.
S. Mutter.
Mutter. Lawrence
Lawrence Erlbaum
Erlbaum Publishers,
Publishers, in
in preparation.
preparation. Also
Also
KSL-86-62.
KSL-86-62.
7. Clancey,
Clancey, W.J. The
The knowledge
knowledge engineer
engineer as
as student:
student: Metacognitive
Metacognitive bases
bases for
for
asking
asking good questions.
questions. In
In Learning
Learning Issues in Intelligent
Intelligent Tutoring
Tutoring Systems, eds.
eds.
A.
A. Lesgold
Lesgold and H.
H. Mandl,
Mandl, in
in preparation.
preparation. Also
Also KSL
KSL 87-12.
87-12.
8. Clancey,
Clancey, W.J. Viewing
Viewing knowledge
knowledge bases as qualitative
qualitative models.
models. KSL
KSL Working
Working
paper 86-27.
86-27.
Know-how vs. knowledge
9. Clancey,
knowledge representation
representation (extended
(extended abstract).
abstract).
Clancey, W.J. Know-how
Proceedings of
Knowledge Compilation,
Proceedings
of the Workshop
Workshop on Knowledge
Compilation, Oregon State
Technical
pages 1-2.
Technical report,
report, September
September 1986, pages
1-2.
10. Wilkins,
D.C., Clancey,
Wilkins,
Clancey, W.J., and Buchanan,
Buchanan, B.G., Knowledge
Knowledge
Refinement
Refinement Using
Using Abstract
Abstract Control
Control Knowledge.
Knowledge. January,
January, KSL-87-01.
KSL-87-01.
Base
Base
11. Wilkins,
D.C., Buchanan,
Wilkins,
Buchanan, B.G., and Clancey,
Clancey, W.J., The Global
Globai Credit
Credit
Assignment Problem
Problem and
Apprenticeship Learning.
Learning. January,
Assignment
and Apprenticeship
January, KSL-87-04.
KSL-87-94.
12. Clancey,
Clancey, W.J. Review
Review of
of Winograd
Winograd and Flores's
Flores’s "Understanding
“Understanding Computers
Computers
and Cognition":
A
interpretation.
Artificial intelligence,
Intelligence,
interpretation.
Artificial
Cognition”:
A favorable
favorable
December,
December, 1986.
N. S.
13. Dietterich,
Dietterich, T. G., Flann,
Flann, N.
S. and Wilkins,
Wilkins, D. c.,
C., Machine
Machine Learning
Learning at
IJCAI-8S,
Machine Learning,
Learning, Volume
No.2,
IJCAI-85, in Machine
Volume 1, No.
2, 1986, 227-242.
227-242.
D. C., An
14. Karp,
An Analysis
Analysis of
of the Deep/Shallow
Deep/Shallow
Karp, P. D. and Wilkins,
Wilkins,
Distinction
Distinction For
For Expert
Expert Systems, KSL-86-32,
KSL-86-32, April
April 1986, 18
18 pp.
IS. Wilkins,
15.
Wilkins, D. C. and Buchanan,
Buchanan, B. G., Debugging
Debugging Rule
Rule Sets
Sets When
When Reasoning
Reasoning
Under
Proceedings of
Fifth National
National Conference
of’ the Fifth
Conference on
Under Uncertainty,
Uncertainty, in Proceedings
Artificial [mefligence,
August
Intelligence,
August 1986, ~48-4S4.
448-154. Also,
Also, extended version.
version.
&tificiai
KSL-86KSL-86-30,30, 20 pp.
113
E. H. Shonliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
GUIDON/NEOMYCIN
Project
GUIDON/NEOMYCIN
Project
16. Wilkins,
Learning
Wilkins, D.
D. C.,Knowledge
C.,Knowledge Base Debugging
Debugging Using
Using Apprenticeship
Apprenticeship
Learning
Techniques,
Proceedings of
Knowledge Acquisition
Acquisition for
for KnowledgeKnowledgeTechniques, in
in Proceedings
of the Knowledge
Based Systems
November 1986, 40. 0--40.
Based
Systems Workshop,
Workshop, November
O--40. 14. Also,
Also, revised
revised
version,
version, KSL-86-63,
KSL-86-63, 20 pp.
D. C., Clancey,
17. Wilkins,
Wilkins,
Clancey, W.
W. J. and Buchanan,
Buchanan, B. J.,
J., Knowledge
Knowledge Base
Base
Refinement
Knowledge
Refinement Using
Using Abstract
Abstract Control
Control Knowledge,
Knowledge, to appear
appear in
in Knowledge
Acquisition for
for Knowledge
Knowledge Based
Based Systems,
Acquisition
Systems, edited
edited by J. Boose and B. Gaines,
Gaines,
Academic Press.
Press. Also
Also to appear
appear in
in International
of Man-Machine
International Journal
Journal of
Man-Machine
Academic
Studies.
Studies. Also
Also KSL-87-0l,
KSL-87-01, Dec
Dee 1986, 12 pp.
Wilkins, D.
D. c.,
C., Cognitive
Cognitive Diagnosis
Diagnosis of
of Heuristic
Heuristic Classification
Classification
Problem
Problem
18. Wilkins,
Solving,
I nternational Conference
Artificial Intelligence
Intelligence and
Solving, Third
Third International
Conference on Artificial
and
Education, May
Education,
May 1987, pp 57. Also,
Also, KSL-86-71,
KSL-86-71, Dec
Dee 1986, 2 pp.
Funding Support
E. Funding
Support
Contract
Contract Title:
Title: "A
“A Family
Family of
of Intelligent
Intelligent Tutoring
Tutoring Programs
Programs for
for Medical
Medical
Diagnosis"
Diagnosis”
Principal Investigator:
Investigator: Bruce G. Buchanan,
Buchanan, Prof.
Prof. Computer
Computer Science, Research
Principal
Associate Investigator:
Investigator: William
William J. Clancey,
Clancey, Research Assoc. Computer
Computer Science
Associate
Agency: Josiah Macy,
Macy, JT.
Jr. Foundation
Foundation
Agency:
Term: March
March 1985 to March
March 1988
Term:
Total
Total award: $503,415 direct
direct costs
Contract
Contract Title:
Title: "Computer-Based
“Computer-Based Tutors
Tutors for
for Explaining
Explaining and Managing
Managing the
of Diagnostic
Diagnostic Reasoning"
Reasoning”
Process of
Principal
Principal Investigator:
Investigator: Bruce
Bruce G. Buchanan,
Buchanan, Prof.
Prof. Computer
Computer Science, Research
Associate
Associate Investigator:
Investigator: William
William J. Clancey,
Clancey, Research Assoc. Computer
Computer Science
Naval Research
Agency:
Agency: Office
Office of
of Naval
ID
N00014-85-K -0305
ID number:
number: N00014-85-K-0305
Total
Total award:
award: $510,311 total
total
II.
II.
INTERACTIONS
WITH
RESOURCE
INTERACTIONS
WITH THE
THE SUMEX-AIM
SUMEX-AIM
RESOURCE
Medical Collaborations
Program Dissemination
Dissemination via SUMEX
A. Medical
Collaborations and
and Program
SUMEX
We are frequently
GUIDON-WATCH,
and
frequently asked to demonstrate
demonstrate GUIDON-MANAGE,
GUIDON-MANAGE,
GUIDON-WATCH,
NEOMYCIN to Stanford
NEOMYCIN
Stanford visitors
visitors or
or at meetings
meetings in
in this
this country
country or
or abroad.
abroad. Physicians
Physicians
have generally
generally been enthusiastic
enthusiastic about
about the potential
potential of
of these programs
programs and what
what they
about current
current approaches
approaches to computer-based
computer-based medical
medical decision
decision making.
making. We use
use
reveal about
network e-mail
e-mail through
through SUMEX
SUMEX to communicate
communicate with
with other
other researchers worldwide.
worldwide.
network
Interaction with
Projects
B. Sharing
Sharing and
and Interaction
with Other
Other SUMEX-AIM
SUMEX-AIM
Projects
GUIDON/NEOMYCIN
retains strong
strong contact
contact with
with the ONCOCIN
ONCOCIN project,
project, as
as both
both are
GUIDON/NEOMYCIN
retains
siblings
siblings of
of the MYCIN
MYCIN parent.
parent. These projects
projects share programming
programming expertise
expertise and utility
utility
routines.
In
routines.
In addition,
addition, the central
central SUMEX
SUMEX development
development group
group acts as
as an important
important
clearing
and distributing
distributing new methods.
methods.
clearing house for
for solving
solving problems
problems and
C. Critique
Resource Management
Management
Critique of
of Resource
The SUMEX
SUMEX resources group
group has provided
provided exemplary
exemplary service.
service.
The
or
or suggestions whatsoever.
whatsoever.
Shortliffe
E. H. Shortliffe
114
We have no complaints
complaints
We
GUIDON/NEOMYCIN Project
Project
GUIDON/NEOMYCIN
5P41-RR00785-14
5P41-RR00785-14
III. RESEARCH
RESEARCH PLANS
PLANS
III.
A. Project
Project Goals
Goals and
and Plans
Plans
A.
Research over
over the
the next
next year
year will
will continue
continue on
on several
several fronts,
fronts, including
including one
one or
or more
more
Research
prototype instructional
instructional programs.
programs.
prototype
1. Use
Use GUIDON-MANAGE
GUIDON-MANAGE by
by medical
medical students
students to
to empirically
empirically develop
develop the
the
1.
interface and
and teaching
teaching scenario.
scenario.
interface
2. Integrate
Integrate the
the new
new explanation
explanation program
program into
into the
the GUIDON-MANAGE
GUIDON-MANAGE
program in
in order
order to
to provide
provide explanations
explanations of
of the
the operations
operations of
of tasks invoked
invoked
program
by the
the student.
student.
by
Develop the
the
3. Develop
incorporate its
its
incorporate
patient-specific
patient-specific
GUIDON-DEBUG knowledge
knowledge acquisition
acqulSltlOn program
program and
and
GUIDON-DEBUG
for manipulating
manipulating the
the
perspective on
on diagnosis
diagnosis (operators
(operators for
perspective
model) in
in feedback
feedback provided
provided within
within GUIDON-MANAGE.
GUIDON-MANAGE.
model)
Long-term plans
plans
B. Long-term
Plans beyond
beyond 1988 are uncertain
uncertain at this
this time.
time.
We expect
expect to
to make
make
We
Plans
for
routine
use
by
people
outside
of
Stanford
and
explore
available
available for routine
by people outside of Stanford and explore
our understanding
understanding of
of diagnosis
diagnosis and heuristic
heuristic
applications to broaden
broaden our
applications
problem solving.
problem
solving.
HERACLES
HERACLES
non-medical
non-medical
classification
classification
Requirements for
for Continued
SUMEX Use
C. Requirements
Continued SUMEX
SUMEX
remains the central
central communications
communications facility
facility
SUMEX
remains
communication
bye-mail
communication
by e-mail and for
for preparing
preparing publications.
publications.
-supported Lisp
SUMEX
SUMEX-supported
Lisp workstations.
workstations.
for our
our project--for
project--for
for
done on
Research is done
D. Requirements
for Additional
Additional Computing
Requirements for
Computing Resources
Resources
Within
upgrade existing
eXIstmg workstations
workstations
Within eighteen
eighteen months,
months, we believe
believe that
that we will
will need to upgrade
purchased in
in the past few
few years to incorporate
incorporate new memory
memory sizes and faster
faster processors.
Our
Our experience
experience with
with color
color monitors
monitors on IBM
IBM PC's
PC’s indicates
indicates that
that the research world
world must
must
convert
convert to color
color to fully
fully exploit
exploit the potential
potential of
of computer
computer graphics,
graphics, especially
especially for
for
knowledge
There
labs
There is some question
question whether
whether academic
academic labs
knowledge base
base browsing
browsing and editing.
editing.
will
will be left
left behind
behind by industrial
industrial efforts
efforts in
in this
this respect. We also find
find that
that the existing
existing
These must
printers
must be replaced
replaced in
in the near
printers are unreliable
unreliable and of
of uneven quality.
quality.
future,
future, perhaps at a higher
higher cost for
for durability.
durability.
E. Recommendations
Recommendations for
for Future
Future Community
Community and
and Resource Development
Development
With
of
With the proliferation
proliferation
of machine
machine types and the availability
availability of
of stand-alone
stand-alone machines
machines
such as
as the Macintosh,
Macintosh, it
it is important
important that
that the machine
machine be linked
linked for
for convenient
convenient
communication
communication bye-mail
by e-mail and conventions
conventions be established
established for
for automatically
automatically translating
translating
into
new
standard
formats.
old
publication
files
old publication files into
standard formats.
ll5
115
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
MOLGEN
MOLGEN Project
Project
IV.A.2.
0 LG EN Project
IV.A.2. M
MOLGEN
Project
MOLGEN
MOLGEN - Applications
Applications of
of Artificial
Artificial Intelligence
Intelligence to Molecular
Molecular
Biology:
Biology: Research in Theory
Theory Formation,
Formation, Testing,
Testing, and Modification
Modification
Prof. E. Feigenbaum and Dr.
Dr. P. Friedland
Friedland
Prof.
Department
Department of
of Computer
Computer Science
Stanford
Stanford University
University
Prof. Charles
Charles Yanofsky
Yanofsky
Prof.
Department
Department of
of Biology
Biology
Stanford
Stanford University
University
I.
I.
SUMMARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
Project Rationale
Rationale
A. Project
The
project
The MOLGEN
MOLGEN
project has focused
focused on research into
into the applications
applications of
of symbolic
symbolic
This has taken
taken the
computation
This
computation and inference
inference to the field
field of
of molecular
molecular biology.
biology.
specific
specific form
form of
of systems which
which provide
provide assistance to the experimental
experimental scientist
scientist in
various
various tasks, the most
most important
important of
of which
which have been the design of
of complex
complex experiment
experiment
Our current
current research concentrates
concentrates on
plans
plans and the analysis
analysis of
of nucleic
nucleic acid
acid sequences.
sequences. Our
scientific
scientific discovery
discovery within
within the subdomain
subdomain of
of regulatory
regulatory genetics.
genetics. We
We desire to explore
explore
the methodologies
methodologies scientists
scientists use
use to modify,
modify, extend,
extend, and test theories
theories of
of genetic
genetic
regulation,
regulation, and then emulate
emulate that
that process within
within a computational
computational system.
Theory
Theory or
or model
model formation
formation is a fundamental
fundamental part
part of
of scientific
scientific research. Scientists
Scientists both
both
use
They
use and form
form such models
models dynamically.
dynamically.
They are used
used to
to predict
predict results
results (and
(and therefore
therefore
to
to suggest
suggest experiments
experiments to test the model)
model) and also to explain
explain experimental
experimental results.
Models
Models are extended
extended and revised
revised both
both as a result
result of
of logical
logical conclusions
conclusions from
from existing
existing
premises
premises and as
as a result
result of
of new experimental
experimental evidence.
evidence.
Theory
Theory formation
formation is a difficult
difficult cognitive
cognitive task, and one in
in which
which there
there is substantial
substantial
Our research is toward
toward building
building a system
scope for
for intelligent
intelligent computational
computational assistance. Our
which
which can form
form theories
theories to explain
explain experimental
experimental evidence,
evidence, can interact
interact with
with a scientist
scientist
to help
help to suggest experiments
experiments to discriminate
discriminate among
among competing
competing hypotheses,
hypotheses, and can
then revise
revise and extend
extend the growing
growing model
model based upon
upon the results
results of
of the experiments.
experiments.
The
The MOLGEN
MOLGEN project
project has
has continuing
continuing computer
computer science goals of
of exploring
exploring issues
issues of
of
knowledge
discovery,
knowledge representation,
representation, problem-solving,
problem-solving,
discovery, and planning
planning within
within a real and
complex
The
complex domain.
domain.
The project
project operates in
in a framework
framework of
of collaboration
collaboration between the
Heuristic
Heuristic Programming
Programming Project
Project (HPP)
(HPP) in
in the Computer
Computer Science Department
Department and various
various
domain
It
domain experts
experts in
in the departments
departments of
of Biochemistry,
Biochemistry, Medicine,
Medicine, and Biology.
Biology.
It draws
from
from the experience
experience of
of several other
other projects
projects in
in the HPP
HPP which
which deal with
with applications
applications
of
of artificial
artificial intelligence
intelligence to medicine.
medicine, organic
organic chemistry,
chemistry, and engineering.
engineering.
B. Medical
Collaboration
Medical Relevance
Relevance and Collaboration
The
The field
field of
of molecular
molecular biology
biology is nearing
nearing the point
point where
where the results
results of
of current
current research
will
will have immediate
immediate and important
important application
application to the pharmaceutical
pharmaceutical and chemical
chemical
industries.
Already,
Already, clinical
clinical testing
testing has begun with
with synthetic
synthetic interferon
interferon and human
human
industries.
Governmental reports
reports
growth
Governmental
growth hormone
hormone produced
produced by recombinant
recombinant DNA
DNA technology.
technology.
estimate
estimate that
that there are more
more than
than two
two hundred
hundred new and established
established industrial
industrial firms
firms
already
already undertaking
undertaking product
product development
development using
using these new genetic
genetic tools.
tools.
E. H. Shortliffe
Shortliffe
11,6
11.6
MOLGEN Project
Project
MOLGEN
5P41-RR00785-14
5P41-RR0078514
The
The programs
programs being
being developed
developed in
in the MOLGEN
MOLGEN project
project have already
already proven
proven useful
useful and
Currently several
several dozen
important
Currently
important to a considerable
considerable number
number of
of molecular
molecular biologists.
biologists.
researchers in
in various
various laboratories
laboratories at Stanford
Stanford (Prof.
(Prof. Paul Berg's,
Berg’s, Prof.
Prof. Stanley
Stanley Cohen's,
Cohen’s,
Prof.
Douglas
Douglas Brutlag's,
Brutlag’s, Prof.
Prof. Henry
Henry Kaplan's,
Kaplan’s, and Prof.
Prof.
Prof. Laurence
Laurence Kedes',
Kedes’, Prof.
Prof.
Douglas
Douglas Wallace's)
Wallace’s) and over
over four
four hundred
hundred others
others throughout
throughout the country
country have used
used
We have exported
exported some of
of our
our
MOLGEN
facility.
We
MOLGEN programs
programs over
over the SUMEX-AIM
SUMEX-AIM
facility.
programs
programs to users
users outside
outside the range of
of our
our computer
computer network
network (University
(University of
of Geneva
Geneva
[Switzerland],
Imperial
and European
[Switzerland],
Imperial Cancer
Cancer Research Fund
Fund [England],
[England],
European Molecular
Molecular
Biology
Biology Institute
Institute [Heidelberg]
[Heidelberg] are examples).
examples). The
The pioneering
pioneering work
work on SUMEX
SUMEX has led
NIH-supported facility,
BlONET,
to the establishment
establishment of
of a separate NIH-supported
facility,
BIONET, to serve the
academic
with
software.
academic molecular
molecular biology
biology research community
community
with MOLGEN-like
MOLGEN-like
software.
needs of
BlONET
BIONET is now
now serving
serving many
many of
of the computational
computational
of over
over two
two thousand
thousand
academic
academic molecular
molecular biologists
biologists in
in the United
United States.
More
More generally,
generally, our
our work
work in
in qualitative
qualitative simulation
simulation as
as applied
applied to molecular
molecular biology
biology is
For
also relevant
relevant to building
building models
models of
of many
many other
other medical
medical and biological
biological systems. For
example,
Intelligence
example, one Artificial
Artificial
Intelligence researcher (Kuipers)
(Kuipers) has been applying
applying these
Other researchers within
within the KSL
KSL are
Other
techniques
techniques to the domain
domain of
of renal
renal physiology.
physiology.
considering
models
considering applying
applying these techniques
techniques to building
building
models of
of cardio-pulmonary
cardio-pulmonary
physiology.
physiology.
C. Highlights
Highlights of
Research Progress
Progress
of Research
C.l
Accomplishments
C.i Accomplishments
During
During the past year we have concentrated
concentrated on the qualitative
qualitative modeling
modeling and simulation
simulation
aspects
multi-level
model
aspects of
of the research. Our
Our view
view is that
that a well-formulated,
well-formulated,
multi-level
model of
of a
scientific
scientific theory
theory is a necessary first
first step to automated
automated discovery.
discovery. In
In addition,
addition, we have
and on
worked
worked on knowledge
knowledge acquisition
acquisition and graphical
graphical display
display of
of process information
information
We have
the description
description and understanding
understanding of
of the results
results of
of laboratory
laboratory experiments.
experiments.
also prepared an in-depth
in-depth conceptual
conceptual reconstruction
reconstruction of
of the biological
biological research which
which led
The
to the current
current detailed
detailed understanding
understanding of
of the mechanism
mechanism of
of attenuation.
attenuation.
The highlights
highlights
of
of this
this work
work are summarized
summarized in
in several categories
categories below.
below.
C.l.l
Modeling and Simulation
C.I.1 Qualitative
Qualitative Modeling
Simulation
Our
Our work
work in
in qualitative
qualitative simulation
simulation has
has been directed
directed towards
towards building
building a program
program which
which
We have built
built one model
model of
of the system
embodies
embodies a theory
theory of
of the tryptophan
tryptophan system. We
and we are designing
designing a second model
model based on the successes
successes and failures
failures of
of the first.
first.
The
The first
first model
model is organized
organized around
around a set of
of twenty
twenty important
important state variables
variables of
of the
In addition,
addition, it
it contains
contains descriptions
descriptions of
of the
tryptophan
In
tryptophan system which
which we have identified.
identified.
The novel
novel properties
properties of
of this
this model
model
causal interactions
interactions between these state variables.
variables. The
results from
from the novel
novel representations
representations used
used for
for the state variables
variables and the interactions
interactions
between them.
Our
Our approach
approach to the representation
representation of
of the values of
of state variables
variables results
results from
from two
two
biologists
observations.
First,
observations.
First, the amount
amount of
of information
information
biologists have about
about the values of
of
different amounts
amounts of
of information
information about
about
different
different state variables
variables varies
varies widely.
widely. Second, different
a given
given variable
variable may be available,
available, and of
of interest,
interest, for
for different
different problems.
problems. Thus,
Thus, our
our
representation
representation is designed to capture
capture a variety
variety of
of types of
of statements
statements about
about the value of
of
a variable.
variable. For
For example,
example, we can record
record quantitative
quantitative information
information about
about a variable
variable (x
(x =
.05),
.05), inequality
inequality information
information (x
(x >
> 10),
lo), or
or relative
relative information
information (x
(x =
= 2*y).
2*y).
Just as
as there is a range in
in the degree of
of precision
precision with
with which
which we might
might know
know the value
of
of a given
given variable,
variable, there
there is an analogous
analogous range within
within which
which we might
might know
know the causal
Consider that
that there
there does exist
exist some function
function which
which
relationship
relationship between two
two variables.
variables. Consider
describes the interactions
interactions among
among any set of
of variables
variables in
in our
our system. Biologists
Biologists may not
not
117
Shortliffe
E. H. Shortliffe
MOLGEN Project
Project
MOLGEN
5P41-RR00785-14
5P41-RR00785-14
of this
this function,
function, and
and hence
hence cannot
cannot
have been
been able
able to
to determine
determine the
the exact
exact behavior
behavior of
have
describe itit exactly.
exactly. Or,
Or, we
we may
may know
know its
its exact
exact behavior,
behavior, but
but itit may
may be
be so
so complex
complex that
that
describe
we
wish
to
describe
it
more
simply.
we wish to describe it more simply.
we require
require aa set
set of
of representations
representations which
which allows
allows us
us to
to represent
represent the
the exact
exact form
form of
of
Thus, we
Thus,
function ifif we
we have
have it,
it, or
or approximations
approximations ifif we
we do
do not
not have
have itit or
or itit isis too
too complex.
complex.
aa function
are'.represented
represented with
with several
several frames,
frames,
Relationships among
among variables
variables are
are concepts
concepts which
which are
Relationships
of
within
which
all
or
only
some
slots
may
be
filled.
Relationships
between
each
pair of
Relationships
between
each
pair
within which all or only some slots may be filled.
interacting variables
variables are
are represented
represented with
with frames
frames called
called Relations,
Relations, which
which describe
describe aa
interacting
we can
can record
record
unidirectional causal
causal relationship
relationship between
between two
two variables.
variables. For
For example,
example, we
unidirectional
any of:
of:
any
of a relationship
relationship
the sign
sign of
the
is a monotonic
monotonic relationship
relationship
whether itit is
whether
of the
the relationship
relationship is,
is, e.g.,
e.g., linear,
linear, higher
higher
what the
the functional
functional form
form of
what
polynomial,
exponential,
or
unknown
polynomial, exponential, or unknown
of the
the exponent
exponent on
on the
the input
input variable
variable
the sign
sign of
.• the
for the
the relationship
relationship
one or
or more
more quantitative
quantitative coefficients
coe{ficients for
.• one
.•
.•
.•
thus express precisely
precisely that
that (possibly
(possibly incomplete)
incomplete)
Using these representations
representations we can thus
Using
knowledge that
that biologists
biologists have about
about the
the trp
trp system.
system. We
We can
can then
then define
define experimental
experimental
knowledge
to the
the degree of
of
conditions and
and ask the
the simulation
simulation system to
to make
make predictions
predictions as to
conditions
expression of
of the
the genes in
in the
the tryptophan
tryptophan operon.
operon. For
For example,
example, we can
can ask
ask how
how much
much
expression
starved of
of tryptophan,
tryptophan, or
or when
when tryptophan
tryptophan is in
in
expression occurs
occurs when
when the cell
cell is starved
expression
excess. The
The simulation
simulation system propagates
propagates the
the initial
initial experimental
experimental conditions
conditions through
through
excess.
of the
the operon
operon varies
varies over
over
the model
model in
in a cyclic
cyclic fashion
fashion to
to predict
predict how
how the
the expression
expression of
the
time.
time.
C.l.2
Process Description
Description and
and Graphical
Display
C.I.2 Process
Graphical Display
A system has been built
built which
which generalizes
generalizes our
our experience
experience in
in process description
description by
by
A
description and animation
animation
providing
providing a simplified
simplified interface
interface for
for the domain-independent
domain-independent
description
of
broken down
down into
into component
component
of process knowledge.
knowledge. The
The system allows
allows processes
processes to be broken
sub-processes
of the subprocesses to be
sub-processes and the causal and time-oriented
time-oriented relationships
relationships of
described
specified.
tion, objects
specified. In addi
addition,
objects utilized
utilized by the processes
processes can be convenien
convenientlytly described
and "drawn"
“drawn” with
with modes and points
points of
of interaction
interaction among
among the objects
objects given
given by the user.
All
All knowledge
knowledge about
about processes
processes and objects is automatically
automatically stored
stored in
in the framework
framework of
of a
KEE
KEE knowledge
knowledge base.
base.
After
After process
process and object
object description,
description, the system automatically
automatically animates
animates the process by
displaying
displaying one of
of several primitive
primitive types of
of interactions
interactions among
among objects
objects in the proper
proper
time
This system has
has been tested on the
time order
order dictated
dictated by the process
process knowledge
knowledge base.
base. This
tryptophan
tryptophan operon
operon domain
domain and its utility
utility is currently
currently being explored
explored in a medical
medical
simulation
simulation domain.
domain.
C.l.3
of the Discovery
Discovery of
of Attenuation
Attenuation
C.I.3 A Conceptual
Conceptual Reconstruction
Reconstruction of
Scientific
The construction
construction of
of a computer
computer
Scientific theory
theory formation
formation is
is aa complicated
complicated process.
process. The
program
program to reproduce
reproduce scientific
scientific discoveries
discoveries is
is one
one way to study this process. Another
Another
way to study
study the process
process is
is by studying
studying the work
work of
of actual
actual scientists.
scientists.
In
In the
the past
past year
year we
we have
have prepared
prepared an
an in-depth
in-depth study
study of
of the discovery
discovery of
of attenuation
attenuation by
Charles
We have
have studied
studied the biological
biological literature
literature
Charles Yanofsky
Yanofsky and
and other
other researchers.
researchers. We
extensively
extensively and
and interviewed
interviewed many
many scientists
scientists involved
involved in
in the research in
in order
order to
reconstruct
reconstruct the
the different
different conceptual
conceptual states
states of
of knowledge
knowledge through
through which
which the
the scientists
scientists
passed
passed in
in their
their understanding
understanding of
of the
the tryptophan
tryptophan operon.
operon. By
By analyzing
analyzing these
these states
states of
of
knowledge
we have
have elucidated
elucidated aa number
number of
of the
the
knowledge and
and the
the transitions
transitions between
between them,
them, we
strategies
strategies and
and heuristics
heuristics which
which these
these biologists
biologists used
used to
to generate
generate and
and choose
choose between
between
E.
E. H.
H. Shortliffe
Shortliffe
ll8
118
MOLGEN Project
Project
MOLGEN
5P41-RR00785-14
5P41-RR00785-14
theories of
of the
the tryptophan
tryptophan operon.
operon. We
We have related
related these strategies
strategies to
to both
both the
the ideas of
of
theories
different philosophers
philosophers .of
of science, and
and to
to the
the diagnostic
diagnostic strategies
strategies of
of the
the Internist
Internist
different
medical expert
expert system.
system.
medical
D. Publications
Publications
D.
Bach, R., Friedland,
Friedland, P., Brutlag,
Brutlag, D.,
D., and Kedes, L.:
L.: MAXIMIZE,
MAXIMIZE, a DNA
DNA
1. Bach,
strategy advisor.
advisor. Nucleic
Nucleic Acids
Acids Res.
10(1):295-304, January,
January,
sequencing strategy
10(1):295-304,
sequencing
1982
2. Bach, R., Friedland,
Friedland, P., and Iwasaki,
Iwasaki, Y.:
Y.: Intelligent
Intelligent computational
computational assistance
assistance
for experiment
experiment design.
design. Nucleic
Nucleic Acids
Acids Res. 12(1):11-29,
12(1):11-29, January,
January, 1984.
for
nucleotide
Brutlag, D.,
D., Clayton,
Clayton, J., Friedland,
Friedland, P. and
and Kedes, L.:
L.: SE&:
SEQ: AA nucleotide
3. Brutlag,
Acids Res.
analysis and
and recombination
recombination system. Nucleic
Nucleic Acids
sequence analysis
10(1):279-294, January,
January, 1982.
10(1):279-294,
Clayton, J. and Kedes, L.: GEL,
GEL, a DNA
DNA sequencing
sequencing project
project management
management
4. Clayton,
Nucleic Acids
Acids Res.
Res. 10(1):305-321,
January, 1982.
system. Nucleic
10(1):305-321, January,
Feitelson, J.
5. Feitelson,
experiment.
experiment.
paper), May,
May,
paper),
Stefik, M.J.: A case study
study of
of the reasoning
reasoning in
in a genetics
genetics
and Stefik,
Heuristic Programming
Programming Project
Project Report
Report HPP-77-18
HPP-77-18 (working
(working
Heuristic
1977.
Frie~land, P.: Knowledge-based
Knowledge-based experiment
experiment design
design in
in molecular
molecular genetics.
genetics.
6. Friedland,
Proc. Sixth
Sixth IJCAI,
IlCAI, August,
August, 1979, pp. 285-287.
285-287.
Proc.
7. Friedland
Friedland P.: Knowledge-based
Knowledge-based experiment
design in
genetics.
experiment design
in molecular
molecular genetics.
Stanford
Computer
Science Report
(Ph.D.
Stanford
Computer
Report STAN-CS-79-760
STAN-CS-79-760
(Ph.D. thesis),
thesis),
December,
December, 1979.
8. Friedland,
MOLGEN--Applications of
symbolic
Friedland, P., Kedes, L. and BrutIag
Brutlag D.: MOLGEN--Applications
of symbolic
computation
intelligence
computation and
and artificial
artificial
intelligence to molecular
molecular biology.
biology. Proc.
Proc. Battelle
Battelle
Conference
Conference on Genetic
Genetic Engineering,
Engineering, April,
April, 1981.
9. Friedland,
Acquisition of
procedural knowledge
from domain
Friedland, P.: Acquisition
of procedural
knowledge from
domain experts.
experts.
Proc.
Proc. Seventh IlCAI,
IJCAI, August,
August, 1981, pp. 856-861.
856-861.
10. Friedland,
Friedland, P., Kedes, L., Brutlag,
Brutlag, D., Iwasaki,
Iwasaki, Y. and Bach R.: GENESIS,
GENESIS, a
knowledge-based
for representation
knowledge-based genetic
genetic engineering
engineering simulation
simulation system for
representation of
of
genetic
planning. Nucleic
genetic data
data and
and experiment
experiment planning.
Nucleic Acids
Acids Res.
Res. 10(1):323-340,
10(1):323-340,
January,
January, 1982.
P., and Kedes, L.: Discovering
the secrets
11. Friedland,
Friedland,
Discovering
secrets
Communications
of
Communications
of the ACM,
ACM, 28(11):1164-1186,
28(11):1164-1186, November,
November,
IEEE/Computer,
18(11):49:69,
IEEE/Computer,
18(11):49:69, November,
November, 1985.
of
DNA.
of
DNA.
1985, and
12. Friedland,
Friedland, P. and Iwasaki
Iwasaki Y.: The concept and
and implementation
implementation of
of skeletal
skeletal
plans. Journal
plans.
Journal of
of Automated
Automated Reasoning,
Reasoning, 1(2):
l(2): 161-208,
161-208, 1985.
13. Friedland,
Friedland, P.,
P., Armstrong,
Armstrong, P.,
P., and Kehler,
Kehler, T.: The role
role of
of computers
computers in
TECHNOLOGY 565-575,
biotechnology.
biotechnology. BIO\
BIO\TECHNOLOGY
565-575, September,
September, 1983.
14.
14. Iwasaki,
Iwasaki, Y. and Friedland,
Friedland, P.:
P.: SPEX:
SPEX: A second-generation
second-generation experiment
experiment design
design
system. Proc. of
Intelligence,
of Second National
National Conference
Conference on Artificial
Artificial
Intelligence,
August,
August, 1982, pp. 341-344.
341-344.
119
E. H. Shortliffe
Shortiiffe
5P41-RR00785-14
5P41-RR00785-14
MOLGEN
MOLGEN Project
Project
15. Martin,
N., Friedland,
P., King,
Knowledge base
Martin,
N..
Friedland,
King, 1.
J. and Stefik,
Stefik, M.1.:
M.J.: Knowledge
Fifth
management
for experiment
planning in molecular
genetics. Proc.
Fifth
management for
experiment planning
molecular genetics.
IlCAI,
IJCAI, August,
August, 1977, pp. 882-887.
882-887.
Knowledge-based simulation
16. Meyers,
Meyers, S.
S. and Friedland,
Friedland, P.: Knowledge-based
simulation of
of regulatory
regulatory
genetics in bacteriophage
Lambda. Nucleic
Nucleic Acids
genetics
bacteriophdge Lambda.
Acids Res. 12(1):1-9,
12(1):1-9, January,
January,
1984.
17. Stefik,
Machine inference
for molecular
genetics:
Stefik, M.
M. and Friedland,
Friedland, P.: Machine
inference for
molecular genetics:
Methods and
NCC, June, 1978.
Methods
and applications.
applications. Proc.
Proc. of
of NCC,
N.: A review
problem solving
18. Stefik,
Stefik, M.1.
M.J. and Martin
Martin N.:
review of
of knowledge
knowledge based problem
solving as
a basis
for a genetics
genetics experiment
basis for
experiment designing
designing system. Stanford
Stanford Computer
Computer
Science Report
March,
Report STAN-CS-77-596,
STAN-CS-77-596,
March, 1977.
DNA structures
from segmentation
19. Stefik,
M.: inferring
Inferring DNA
structures from
segmentation data:
data: A case study.
study.
Stefik, M.:
Artificial
Intelligence 11:85-114,
11:85-114, December,
December, 1977.
Artificial
Intelligence
frame-structured representation
20. Stefik,
Stefik, M.:
M.: An examination
examination of
of a frame-structured
representation system.
Proc. Sixth
Sixth IlCAI,
IJCAI, August,
August, 1979, pp. 844-852.
844-852.
Stanford Computer
Computer Science Report
Report
Planning with
21. Stefik,
Stefik, M.: Planning
with constraints.
constraints. Stanford
ST
AN -CS-80-784 (Ph.D.
STAN-CS-SO-784
(Ph.D. thesis),
thesis), March,
March, 1980.
Analysis of
Deep/Shallow Distinction
Distinction for
for
22. Karp,
of the Deep/Shallow
Karp, P., and D.
D. Wilkins:
Wilkins: An Analysis
Expert Systems.
Expert
Systems. Stanford
Stanford University
University Knowledge
Knowledge Systems Laboratory
Laboratory Report
Report
KSL-86KSL-86-32.32, 1986.
the Use of
and
23. Karp,
of Qualitative
Qualitative
and
Karp, P., and P. Friedland:
Friedland: Coordinating
Coordinating
Quantitative
Knowledge in
Declarative Device
Device Modeling.
Modeling. Stanford
Quantitative Knowledge
in Declarative
Stanford University
University
Knowledge
Knowledge Systems Laboratory
Laboratory Report
Report KSL-87-09,
KSL-87-09, 1987.
Environment for
for the Qualitative
24. Round,
Round, A.: QSOPS:
QSOPS: A Workbench
Workbench Environment
Qualitative
Simulation
Physical Processes.
Processes. Stanford
Simulation of
of Physical
Stanford University
University Knowledge
Knowledge Systems
Laboratory
Laboratory Report
Report KSL-87-37,
KSL-87-37, 1987.
Reconstruction of
Discovery of
25. Karp,
of the Discovery
of
Karp, P.,
P., and P. Friedland,
Friedland, A Conceptual
Conceptual Reconstruction
Attenuation. In Preparation.
Attenuation.
Preparation.
E. Funding
Support
Funding Support
The
grant,
The MOLGEN
MOLGEN
grant, which
which has supported
supported the bulk
bulk of
of this
this research, is titled:
titled:
MOLGEN:
Intelligence
MOLGEN: Applications
Appiications of
of Artificial
Artificial
Intelligence to Molecular
:Molecular Biology:
Biology: Research in
in
This NSF
Grant number
number MCS-8310236,
MCS-8310236.
Theory
This
NSF Grant
Theory Formation,
Formation, Testing,
Testing, and Modification.
Modification.
expired
The
Investigators
expired on 10/31/86.
10/31/86.
The Principal
Principal
Investigators were Edward
Edward A. Feigenbaum,
Feigenbaum,
Professor
Professor of
of Computer
Computer Science and Charles
Charles Yanofsky,
Yanofsky, Professor
Professor of
of Biology.
Bioiogy. Additional
Additional
support
support for
for this
this research has been provided
provided by the Defense
Defense Advanced
Advanced Research Projects
Projects
Agency,
N00039-86C-0033.
Agency, under
under contract
contract N00039-86C-0033.
II.
II.
INTERACTIONS
WITH
RESOURCE
INTERACTIONS
WITH THE
THE SUMEX-AIM
SUMEX-AIM
RESOURCE
SUMEX-AIM
continues to serve as the nucleus
nucleus of
of our
our computing
computing resources. The
The
SUMEX-AIM
continues
facility
facility has not
not only
only provided
provided excellent
excellent support
support for
for our
our programming
programming efforts
efforts but
but has
Systerns
link
Systems
served as
as a major
major communication
communication
link among
among members
members of
of the project.
project.
available
such as
available on SUMEX-AIM
SUMEX-AIM
as EMACS,
EMACS, MM,
MM, Scribe
Scribe and BULLETIN
BULLETIN BOARD
BOARD have
made possible
possible the project's
project’s documentation
documentation and communication
communication efforts.
efforts. The
The interactive
interactive
environment
environment of
of the facility
facility is especially
especially important
important in
in this
this type of
of project
project development.
development.
E. H. Shortliffe
Shortliffe
120
120
MOLGEN Project
Project
MOLGEN
5P41-RR00785-14
5P41-RR00785-14
We strongly
strongly approve
approve of
of the
the network-oriented
network-oriented approach
approach to
to a programming
programming environment
environment
We
into which
which SUMEX
SUMEX has evolved.
evolved. The
The ability
ability to
to utilize
utilize Lisp
Lisp workstations
workstations for
for intensive
intensive
into
computing while
while still
still communicate
communicate with
with all
all of
of the
the other
other SUMEX
SUMEX resources has been
computing
our work.
work. We
We currently
currently have
have a satisfactory
satisfactory mode
mode of
of operation
operation where
where
very valuable
valuable to
to our
very
essentially all
all programming
programming takes place
place on
on the workstations
workstations and most
most electronic
electronic
essentially
communications, information
information sharing,
sharing, and document
document preparation
preparation takes place
place within
within the
the
communications,
mature TOPS-20
TOPS-20 environment.
environment. The
The evolution
evolution of
of SUMEX
SUMEX has alleviated
alleviated most
most of
of our
our
mature
previous problems
problems with
with resource
resource loading
loading and file
file space. Our
Our current
current workstations
workstations are
previous
not
quite
fast
nor
sophisticated
enough,
but
we
are
encouraged
by
the
progress
that has
not quite fast nor sophisticated enough, but
encouraged by
that
been made.
We have taken
taken advantage
advantage of
of the
the collective
collective expertise
expertise on
on medically-oriented
medically-oriented knowledgeknowledgeWe
based systems of
of the
the other
other SUMEX-AIM
SUM EX -AIM projects.
projects. In
In addition
addition to
to especially
especially close ties
ties
with other
other projects
projects at Stanford,
Stanford, we have greatly
greatly benefited
benefited by
by interaction
interaction with
with other
other
with
projects at
at yearly
yearly meetings
meetings and
and through
through exchange
exchange of
of working
working papers and ideas over
over the
projects
system.
The ability
ability for
for instant
instant communication
communication with
with a large
large number
number of
of experts
experts in
in this
this field
field has
The
determining factor
factor in
in the success
success of
of the MOLGEN
MOLGEN project.
project. ItIt has made possible
possible
been a determining
the near-instantaneous
near-instantaneous dissemination
dissemination of
of MOLGEN
MOLGEN systems to a host
host of
of experimental
experimental
the
in laboratories
laboratories across the country.
country. The
The wide-ranging
wide-ranging input
input from
from these users has
users in
greatly improved
improved the general
general utility
utility of
of our
our project.
project.
greatly
We find
find itit very
very difficult
difficult to find
find fault
fault with
with any
any aspect of
of the SUMEX
SUMEX
We
management.
It
for us to expand
expand our
our user group,
group,
management.
It has made itit easy for
demonstrations to
to colleagues
colleagues and
and to
to disseminate
disseminate software
software to non-SUMEX
non-SUMEX users
demonstrations
III.
III.
resource
give
to give
overseas.
RESEARCH
RESEARCH PLANS
PLANS
A. Project
Project Goals
And Plans
Plans
Goals And
Our
Our current
current work
work has the following
following major
major goals:
1. We
We will
will continue
continue our
our work
work in
in qualitative
qualitative simulation,
simulation, modeling,
modeling, and process
description.
description. We
We will
will continue
continue testing
testing the existing
existing state-variable-based
state-variable-based model
model
of
and more
more
of the tryptophan
tryptophan operon.
operon. In
In addition,
addition, we will
will construct
construct a new and
general
This
This model
model will
will be centered around
around the
general model
model of
of the
the operon.
operon.
objects
proteins) and the
objects within
within this
this domain
domain (e.g., enzymes, DNA,
DNA, repressor
repressor proteins)
interactions
between them.
The
model
The current
current state-variable
state-variable
model makes
interactions
them.
assumptions
assumptions about
about the presence of
of different
different objects
objects and the functions
functions of
of
these objects
tain no mutations)
objects (e.g.,
(e.g., that
that they
they con
contain
mutations) which
which the new model
model will
will
make both
Essentially, the new model
model will
will
both explicit
explicit and allow
allow us
us to change. Essentially,
allow
us
to
dynamically
construct
new
state-variable
models
based
allow us
dynamically construct
state-variable
based on the
presence of
of different
different objects
objects and different
different interactions
interactions between these objects.
Changing
Changing these assumptions
assumptions is crucial
crucial to the discovery
discovery process,
process, which
which
involves
involves the postulation
postulation of
of new classes
classes of
of objects
objects and new classes
classes of
of
interactions
interactions between objects.
objects.
2. Build
Build a mechanism
mechanism for
for postulating
postulating extensions
extensions or
or corrections
corrections to the current
current
theory:
Our conceptual
conceptual reconstruction
reconstruction of
of the
theory: a constrained
constrained theory
theory generator.
generator. Our
discovery
discovery of
of attenuation
attenuation should
should be of
of critical
critical help in
in both
both this
this phase
phase and
the phases
phases which
which follow.
follow.
3.
3. Build
Build a mechanism
mechanism for
for evaluating
evaluating alternative
alternative theories.
theories. This
This would
would include
include
rating
selectability,
completeness,
rating the theories
theories based
based on plausibility,
plausibility,
selectability,
completeness,
significance,
We hope the evaluation
evaluation process
process produces
significance, and so on.
information
information useful in
in discriminating
discriminating among
among the possible
possible theories.
theories.
121
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
MOLGEN Project
Project
MOLGEN
Test the
the entire
entire structure
structure on
on the
the evolving
evolving trp
trp operon
operon regulatory
regulatory system.
system.
4. Test
Experiment with
with different
different initial
initial knowledge
knowledge bases to
to see
see how
how the
the discovery
discovery
Experiment
is altered
altered by
by the
the availability
availability of
of new
new techniques,
techniques, analogous
analogous systems,
process is
and so forth.
forth.
and
B. Justification
Justification and
and Requirements
Requirements for
for Continued
Continued SUMEX
SUMEX Use
B.
The MOLGEN
MOLGEN project
project depends heavily
heavily on
on the
the SUMEX
SUMEX facility.
facility. We
We have already
already
The
developed several
several useful
useful tools
tools on
on the
the facility
facility and
and are continuing
continuing research toward
toward
developed
applying the
the methods
methods of
of artificial
artificial intelligence
intelligence to
to the
the field
field of
of molecular
molecular biology.
biology. The
The
applying
community of
of potential
potential users is growing
growing nearly
nearly exponentially
exponentially as researchers from
from most
most
community
of the
the biomedical-medical
biomedical-medical fields
fields become
become interested
interested in
in the
the technology
technology of
of recombinant
recombinant
of
DNA. We
We believe
believe the MOLGEN
MOLGEN work
work is already
already important
important to
to this
this growing
growing community
community
DNA.
and will
will continue
continue to
to be important.
important. The
The evidence
evidence for
for this
this is an already
already large
large list
list of
of
and
pilot exo-MOLGEN
exo-MOLGEN users on
on SUMEX.
SUM EX.
pilot
of satellite
satellite computers
computers for
for technology
technology
We support
support with
with great
great enthusiasm
enthusiasm the acquisition
acquisition of
We
transfer and hope that
that the SUMEX
SUMEX staff
staff continues
continues to
to develop
develop and support
support these
transfer
of the oft-mentioned
oft-mentioned problems
problems of
of artificial
artificial intelligence
intelligence research is
One of
systems.
exactly the
the problem
problem of
of taking
taking prototypical
prototypical systems and applying
applying them
them to real problems.
problems.
exactly
SUMEX gives
gives the MOLGEN
MOLGEN project
project a chance to conquer
conquer that
that problem
problem and potentially
potentially
SUMEX
supply scientific
scientific computing
computing resources to a national
national audience
audience of
of biomedical-medical
biomedical-medical
supply
research scientists.
scientists.
research
E. H. Shortliffe
Shortliffe
122
ONCOCIN
ONCOCIN Project
Project
5P41-RR00785-14
5P41-RR00785-14
IV.A.3. ONCOCIN
ONCOCIN Project
Project
IV.A.3.
ONCOCIN
ONCOCIN Project
Project
Edward
Edward H.
H. Shortliffe,
Shortliffe, M.D.,
M.D., Ph.D.
Ph.D.
Departments
Departments of
of Medicine
Medicine and Computer
Computer Science
Science
Stanford University
University
Stanford
I.I.
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
SUMMARY
A. Project
Project Rationale
Rationale
A.
The ONCOCIN
ONCOCIN Project
Project is one of
of many
many Stanford
Stanford research programs
programs devoted
devoted to the
The
development of
of knowledge-based
knowledge-based expert
expert systems for
for application
application to medicine
medicine and the
development
allied sciences. The
The central
central issue in
in this
this work
work has been to develop
develop a program
program that
that can
allied
provide advice
advice similar
similar in
in quality
quality to that
that given
given by human
human experts,
experts, and to ensure that
that the
provide
use and acceptable
acceptable to
to physicians.
physicians. The
The work
work seeks
seeks to improve
improve the
system is easy to use
interactive process, both
both for
for the developer
developer of
of a knowledge-based
knowledge-based system, and for
for the
interactive
intended end user. In
In addition,
addition, we have emphasized
emphasized clinical
clinical implementation
implementation of
of the
intended
developing· tool
tool so that
that we can ascertain
ascertain the effectiveness
effectiveness of
of the program’
program's
interactive
developing.
s interactive
capabilities when itit is used by
by physicians
physicians who
who are caring
caring for
for patients
patients and are
capabilities
uninvolved in
in the computer-based
computer-based research activity.
activity.
uninvolved
B. Medical
Medical Relevance and Collaboration
Collaboration
The
The lessons
lessons learned
learned in
in building
building prior
prior production
production rule
rule systems have allowed
allowed us to create
a large oncology
oncology protocol
protocol management
management system much
much more
more rapidly
rapidly than was the case
case
We introduced
introduced ONCOCIN
ONCOCIN for
for use
use by Stanford
Stanford
when we started
We
started to build
build MYCIN.
MYCIN.
This would
would not
not have been possible
possible without
without the active
active
oncologists
oncologists in
in May
May 1981. This
of
of
collaboration
collaboration
of Stanford
Stanford oncologists
oncologists who
who helped
helped with
with the construction
construction
of the
knowledge
2nd
knowledge base
base and also kept
kept project
project computer
computer scientists
scientists aware of
of the psychological
psychological and
logistical
logistical issues
issues related
related to the operation
operation of
of a busy outpatient
outpatient clinic.
clinic.
Highlights of
Research Progress
Progress
C. Highlights
of Research
C.l
C.1 Background
Background and Overview
Overview of
of Accomplishments
Accomplishments
The
effort
The ONCOCIN
ONCOCIN Project
Project is a large interdisciplinary
interdisciplinary
effort that
that has
has involved
involved over
over 35
35
The work
work is currently
currently in its
individuals
individuals since the project's
project’s inception
inception in
in July
July 1979. The
eighth
eighth year; we summarize
summarize here the milestones
milestones that
that have occurred
occurred in the research to
date:
•. Year I:
I: The
The project
project began with
with two
two programmers
programmers (Carli
(Carli Scott
Scott and Miriam
Miriam
Bischoff),
Bischoff), a Clinical
Clinical Specialist
Specialist (Dr.
(Dr. Bruce Campbell)
Campbell) and students
students under
under the
direction
direction of
of Dr.
Dr. Shortliffe
Shortliffe and Dr.
Dr. Charlotte
Charlotte Jacobs from
from the Division
Division of
of
Oncology.
Oncology. During
During the first
first year of
of this
this research (1979-1980),
(1979-1980), we developed
developed
a prototype
prototype of
of the ONCOCIN
ONCOCIN consultation
consultation system, drawing
drawing from
from programs
programs
and
and capabilities
capabilities developed for
for the EMYCIN
EMYCIN system-building
system-building project.
project. During
During
that year, we also undertook
undertook a detailed
detailed analysis
analysis of
of the day-to-day
day-to-day activities
activities
of
of the Stanford
Stanford Oncology
Oncology Clinic
Clinic in order
order to determine
determine how to introduce
introduce
ONCOCIN
ONCOCIN with
with minimal
minimal disruption
disruption of
of an operation
operation which
which is already
already
running
running smoothly.
smoothly. We also spent much
much of
of our
our time
time in the first
first year giving
giving
careful
careful consideration
consideration to the most
most appropriate
appropriate mode of
of interaction
interaction with
with
chances for
for ONCOCIN
ONCOCIN to
LO become a
physicians
physicians in order
order to optimize
optimize the chances
useful and accepted tool in this specialized
specialized clinical
clinical environment.
environment.
123
Shortliffe
E. H. Shortliffe
ONCOCIN
ONCOCIN Project
Project
5P41-RR00785-14
5P41-RR00785-14
•. Year 2: The
The following
following year (1980-1981)
(1980-1981) we completed
completed the development
development of
of a
special
special interface
interface program
program that
that responds to commands
commands from
from a customized
customized
keypad.
keypad. We
We also encoded the rules for
for one more
more chemotherapy
chemotherapy protocol
protocol (oat
(oat
cell
cell carcinoma
carcinoma of
of the lung)
lung) and updated the Hodgkin's
Hodgkin’s disease
disease protocols
protocols
when new versions
versions of
of the documents
documents were released late
late in
in 1980; these
when
exercises demonstrated
of
demonstrated the generality
generality and flexibility
flexibility
of the representation
representation
Software protocols
protocols were developed
developed for
for achieving
achieving
scheme we had devised. Software
communication
communication between the interface
interface program
program and the reasoning
reasoning program,
program,
and we coordinated
coordinated the printing
printing routines
routines needed to produce
produce hard
hard copy
copy flow
flow
sheets, patient
patient summaries,
summaries, and encounter
encounter sheets.
sheets. Finally,
Finally, lines
lines were installed
installed
in
in the Stanford
Stanford Oncology
Oncology Day
Day Care Center,
Center, and, beginning
beginning in
in May
May 1981,
eight
eight fellows
fellows in
in oncology
oncology began using the system three
three mornings
mornings per week
for
for management
management of
of their
their patients
patients enrolled
enrolled in
in lymphoma
lymphoma chemotherapy
chemotherapy
protocols.
protocols.
Year 3: During
During our
our third
third year (1981-1982)
(1981-1982) the results
results of
of our
our early
early
•. Year
experience
experience with
with physician
physician users
users guided both
both our
our basic and applied
applied work.
work. We
We
collect data for
for three formal
formal studies
studies to evaluate
evaluate the
designed and began to collect
impact of
of ONCOCIN
ONCOCIN in
in the clinic.
clinic. This
This latter
latter task required
required special
special software
software
impact
development
development to generate
generate special
special flow
flow sheets
sheets and to maintain
maintain the records
records
for the data analysis.
analysis. Towards
needed for
Towards the end of
of 1982 we also began new
into a critiquing
critiquing model
model for
for ONCOCIN
ONCOCIN that
that involves
involves "hypothesis
“hypothesis
research into
assessment"
assessment” rather
rather than
than formal
formal advice giving.
giving. Finally,
Finally, in
in 1982 we began to
develop
end users
users to
develop a query
query system to allow
allow system builders
builders as
as well
well as
as end
examine
examine the growing
growing complex
complex knowledge
knowledge base
base of
of the program.
program.
4: Our
Our fourth
fourth year (1982-1983)
(1982-1983) saw the departure
departure of
of Carli
Carli Scott,
Scott, a key
•. Year 4:
figure
design and implementation
of
the
figure in
in the initial
initial
implementation
of ONCOCIN,
ONCOCIN,
promotion of
of Miriam
Miriam Bischoff
Bischoff to Chief
Chief Programmer,
Programmer, and the arrival
arrival of
of
promotion
Christopher Lane
Lane as our
our second scientific
scientific programmer.
programmer.
At this
this time
time we
Christopher
At
exploring the possibility
possibility of
of running
running ONCOCIN
ONCOCIN
single-user
on a single-user
began exploring
professional
professional workstation
workstation and experimented
experimented with
with different
different options
options for
for datadataentry
entry using a "mouse"
“mouse” pointing
pointing device. Christopher
Christopher Lane
Lane became an expert
expert
on the Xerox
Xerox workstations
workstations that
that we are using. In
In addition,
addition, since ONCOCIN
ONCOCIN
had grown
grown to such a large
large program
program with
with many
many different
different facets. we spent
spent
much
much of
of our
our fourth
fourth year documenting
documenting the system. During
During that
that year we also
aiso
modified
modified the clinic
clinic system based upon feedback
feedback from
from the physician-users,
physician-users,
modifications
for Hodgkin's
Hodgkin’s disease
disease based
based upon
upon
made some modifications
to the rules for
protocols, and completed
completed several evaluation
evaluation studies.
changes to the protocols,
The project's
project’s fifth
fifth year (1983-1984)
(1983-1984) was characterized
characterized by growth
growth in
•. Year 5: The
of our
our staff
staff (three
(three new full-time
fuil-time
staff members
members and a new
staff
the size of
oncologist
joined the group).
oncologist joined
group). The
The increased size resulted
resulted from
from a ORR
DRR grant
grant
that permitted
permitted us to begin
begin a major
major effort
effort to rewrite
rewrite ONCOCIN
ONCOCIN to run
run on
that
professional workstations.
workstations.
Dr. Robert
Robert Carlson,
Carlson, who
who had been our
our Clinical
Clinical
professional
Dr.
Specialist for
for the previous
previous two
two years,
years, was
was replaced
replaced by Dr.
Dr. Joel Bernstein,
Bernstein,
Specialist
while Dr.
Dr. Carlson
Carlson assumed a position
position with
with the nearby
nearby Northern
C::lifornia
Northern California
while
Oncology
I iation
Oncology Group;
Group: this
this appointment
appointment permitted
permitted him
him to continue
continue his affi
affiliation
both with
with Stanford
Stanford and with
with our
our research group.
group. In
In August
August of
of 1983,
1983, Larry
Larry
both
joined the project
Fagan joined
project to take over
over the duties
duties of
of the ONCOCIN
ONCOCIN Project
Project
Director
Director while
while also becoming
becoming the Co-Director
Co-Director of
of the newly
newly formed
formed Medical
Medical
Information
he
Information Sciences Program.
Program. Dr.
Dr. Fagan continues
continues to be in charge of
of l;ie
day-to-day
efforts
An
programmer,
jetY
day-to-day
efforts of
of our
our research.
An additional
additional
programmer, ;ii)Ferguson,
joined the group
Ferguson, joined
group in the fall
fall to assist with
with the effoH
effort required
required LO
io
transfer ONCOCIN
ONCOCIN from
from SUMEX
SUMEX to the llOS
1108 workstation.
workstation.
A fourth
fourth
transfer
A
programmer, Joan Differding,
Differding,
programmer,
joined the staff
joined
staff to work
work on our
our protocol
protocol
acquisition
acquisition effort
effort (OPAL).
(OPAL).
!
Shortliffe
E. H. Shortliffe
124
ONCOCIN Project
Project
ONCOCIN
5P41-RR00785-14
5P41-RR00785-14
Year 6:
6: During
During our
our sixth
sixth year (1984-1985)
(1984-1985) we further
further increased
increased the size of
of
.• Year
our programming
programming staffstaff to
to help
help in
in the
the major
major workstation
workstation conversion
conversion effort.
effort.
our
The ONCOCIN
ONCOCIN and
and OPAL
OPAL efforts
efforts were greatly
greatly facilitated
facilitated by
by a successful
successful
The
for an
an equipment
equipment grant
grant from
from Xerox
Xerox Corporation.
Corporation. With
With a total
total
application for
application
of 15 Xerox
Xerox LISP
LISP machines
machines now
now available
available for
for our
our group’
group's
all fullfullof
s research, all
time programmers
programmers have dedicated
dedicated machines,
machines, as do several of
of the
the senior
senior
time
on the project.
project. Christopher
Christopher Lane
Lane took
took on
on fullfullgraduate students
students working
working on
graduate
time responsibility
responsibility for
for the
the integration
integration and
and maintenance
maintenance of
of the group’
group's
time
s
equipment and
and associated software.
software. Two
Two of
of our
our programming
programming staff
staff moved
moved
equipment
on to
to jobs
jobs in
in industry
industry (Bischoff
(Bischoff and Ferguson)
Ferguson) and three
three new
new programmers
programmers
on
Cliff Wutfman,
Wulfman, and Samson Tu)
Tu) were hired
hired to
to fill
fill the
the void
void
(David Combs,
Combs, Cliff
(David
of Christopher
Christopher Lane.
Lane.
created by
by their
their departure
departure and
and by
by the reassignment
reassignment of
created
funding from
from DRR
ORR for
for the workstation
workstation conversion
conversion effort,
effort, we
In addition
addition to
to funding
In
have support
support from
from the National
National Library
Library of
of Medicine
Medicine which
which supports
supports our
our
more basic
basic research activities
activities regarding
regarding biomedical
biomedical knowledge
knowledge representation,
representation,
more
knowledge acquisition,
acquisition, therapy
therapy planning,
planning, and explanation
explanation as itit relates
relates to the
knowledge
ONCOCIN task
task domain.
domain. We have continued
continued to study
study the therapy
therapy planning
planning
ONCOCIN
support from
from the NLM.
NLM. This
This research is led by Dr.
Dr. Fagan
process under
under support
concentrated on how to represent
represent the therapy-planning
therapy-planning strategies
strategies
and has concentrated
for patients
patients who
who run
run into
into serious
serious problems
problems while
while
used to
to decide
decide treatment
treatment for
used
on protocol-described
protocol-described treatment.
treatment. The
The physicians
physicians who
who treat
treat these patients
patients
on
often seek out
out a consultation
consultation with
with the protocol
protocol study
study chairman.
chairman.
Dr.
Dr.
often
Branimir Sikic,
Sikic, a faculty
faculty member
member from
from the Stanford
Stanford University
University Department
Department
Branimir
of Medicine,
Medicine, and the Study
Study Chairman
Chairman for
for the oat
oat cell
cell protocol,
protocol, collaborated
collaborated
of
as data manager
manager
this project.
project. Janice
Janice Rohn
Rohn joined
joined the ONCOCIN
ONCOCIN project
project as
on this
process .
and to
to assist in
in the knowledge
knowledge entry
entry process.
•. Year 7: The
marked several milestones
milestones in
in our
our
The seventh
seventh year (1985-86)
(1985-86) marked
research on workstation-based
programming.
The
The OPAL
OPAL knowledge
knowledge
workstation-based
programming.
acquisition
acquisition system became operational,
operational, and several new oncology
oncology protocols
protocols
were entered
entered using
using this
this system. David
David Combs
Combs was
was primarily
primarily responsible
responsible for
for
creating
creating the operational
operational version
version of
of OPAL
OPAL (based on the initial
initial prototype
prototype by
As anticipated,
Joan Differding
Differding Walton).
Walton).
anticipated, we increased the speed
speed and ease
with
with which
which protocols
protocols can be added to the ONCOCIN
ONCOCIN knowledge
knowledge base.
base.
Based
Based on the protocols
protocols entered
entered through
through OPAL,
OPAL, we began experimental
experimental testing
testing
of
of the workstation
workstation version
version of
of ONCOCIN
ONCOCIN in
in the Stanford
Stanford oncology
oncology clinic.
clinic.
Clifford
Wulfman
developed
Clifford
Wulfman
developed the user interface
interface (based on an initial
initial
prototype
Samson Tu
Tu developed
developed the
prototype designed by Christopher
Christopher Lane).
Lane).
reasoning
Much of
of their
their
reasoning component
component (designed originally
originally by Jay Ferguson).
Ferguson). Much
work
work is built
built upon
upon an object-oriented
object-oriented system developed for
for our
our group
group by
We connected
Christopher
connected the various
various parts of
of the system, and
and
Christopher Lane.
demonstrated
demonstrated that
that we have the capability
capability to run
run ONCOCIN
ONCOCIN with
with the
reasoning
reasoning program
program and interface
interface program
program on different
different machines
machines in
in the
communication
communication network.
network. The
The current
current version
version of
of the program
program is currently
currently
run
run on a single
single workstation,
workstation, but
but future
future versions
versions may take
take advantage of
of the
multiple
multiple machine
machine option.
option. To
To increase the speed
speed at which
which we are able to test
protocols
protocols entered
entered into
into ONCOCIN,
ONCOCIN, we developed additional
additional programs
programs to test
real and synthetic
synthetic cases
cases without
without user interaction;
interaction; these
these are then reviewed
reviewed by
our
our collaborating
collaborating clinicians.
clinicians.
We also developed
developed a workstation-based
workstation-based program,
program, OPUS, to help clinicians
clinicians
determine
determine which
which protocols
protocols are appropriate
appropriate for
for specific
specific patients.
patients. OPUS was
was
designed and implemented
implemented by Janice Rohn
Rohn with
with the assistance of
of Christopher
Christopher
Lme.
Lane. We have been using it
ic in the clinic
clinic setting
setting since the end of
of 1985.
125
E. H. Shortliffe
Shortliffe
ONCOCIN Project
Project
ONCOCIN
5P41-RR00785-14
5P41-RR00785-14
addition to
to providing
providing an
an information
information resource
resource about
about protocols,
protocols, the
the
Thus, inin addition
Thus,
of aa graphically-oriented
graphically-oriented program
program provided
provided aa way
way to
to learn
learn about
about the
the
use of
use
of
ONCOCIN.
software
style
and
hardware
used
in
the
workstation
version
software style and hardware used in the workstation version of ONCOCIN.
mainframe version
version of
of ONCOCIN,
ONCOCIN, and
and began
began using
using the
the
We discontinued
discontinued the
the mainframe
We
The
performance
of
the
mainframe
version
workstation
version
exclusively.
The
performance
of
the
mainframe
version
workstation version exclusively.
of ONCOCIN
ONCOCIN was
was documented
documented in
in two
two evaluation
evaluation papers
papers that
that appeared
appeared inin
of
clinical journals
journals (see
(see Hickam
Hickam and
and Kent’
Kent's
papers).
clinical
s papers).
our basic
basic research
research in
in the
the design
design of
of advanced
advanced therapy-planning
therapy-planning
We continued
continued our
We
ONYX project.
project. We
We developed
developed aa mode1
model for
for planning
planning which
which
programs: the
the ONYX
programs:
from the
the fields
fields of
of artificial
artificial intelligence,
intelligence, simulation,
simulation, and
and
includes techniques
techniques from
includes
decision analysis.
analysis. Artificial
Artificial intelligence
intelligence techniques
techniques are
are used
used to
to create
create aa
decision
number of
of possible
possible plans
plans given
given the
the ideal
ideal therapy
therapy and
and the
the patient’
patient's
past
small number
small
s past
treatment history.
history. Simulation
Simulation techniques
techniques and
and decision
decision analysis
analysis are
are used
used to
to
treatment
order the most
most promising
promising plans.
Our goal
goal is
is to
to allow
allow
examine and order
Our
examine
of situations:
situations; in
in particular,
particular, the
the
ONCOCIN to
to give
give advice
advice in
in aa wider
wider range
range of
ONCOCIN
for patients
patients who
who have
have an
an unusual
unusual
system should
should be
be able to
to recommend
recommend plans for
response to
to chemotherapy.
chemotherapy.
response
During this
this year, Stephen Rappaport,
Rappaport, M.D.
M.D. joined
joined us as a programmer
programmer on
on the
the
During
planning. research.
research. Clinical
Clinical expertise for
for ONCOCIN
ONCOCIN was
was provided
provided
therapy planning.
therapy
Robert Carlson, M.D.
M.D.
by Richard
Richard Lenon,
Lenon, M.D.
M.D. and Robert
by
8: This
This year (1986-87)
(1986-87) concentrated on two diverse tasks:
tasks: 1) scaling UP
up
.• Year S:
use of
of the workstation
workstation version
version of
of ONCOCIN
ONCOCIN in
in the clinic,
clinic, and 2)
the use
generalization of
of each
each of
of the components. The latter task is described in
in
generalization
of this report(see page
page 19).
the core research
research sections of
into the Oncology
In
In 1986, we placed the workstation
workstation version of
of ONCOCIN
ONCOCIN into
This
version
is
a
completely
different
program
from the
Day Care clinic.
clinic. This
completely different
from
20--using protocols
version of
of ONCOCIN
ONCOCIN that ran on the DECsystem 20--using
entered through
with a new graphical data entry
through the OPAL
OPAL program, with
interface,
interface, and a revised knowledge
knowledge representation and reasoning component.
One of
of the Oncology Clinical
Clinical Fellows (Andy
(Andy Zelenetz) became
became responsible
for
for verifying
verifying how well our design
design goals
goals for
for ONCOCIN
ONCOCIN had
had been
been
accomplished. His suggestions
suggestions have
have included the addition
addition of
of key protocols
and the
the ability
ability to have
have the program used
used as
as aa data
data management
management tool if
if the
complete treatment
treatment protocol
protocol had
had not yet been
been entered
entered into
into the system.
system. Both
of
of these
these suggestions
suggestions were
were carried
carried out
out during
during this
this year,
year, and
and the
the program
program has
has
achieved
In addition,
addition, laser-printed
laser-printed flowsheets
flowsheets
achieved wider use
use in
in the clinic
clinic setting.
setting. In
and
and progress
progress notes
notes have
have been
been added
added to the
the clinic
clinic system.
system.
The
The process
process of
of entering
entering aa large
large number
number of
of treatment
treatment protocols in
in aa short
short
period
period of
of time
time led
led to
to other
other research
research topics
topics including:
including: design
design of
of an
an automated
automated
system
system for
for producing
producing meaningful
meaningful test
test cases
cases for
for each
each knowledge
knowledge base,
base,
modification
methods for
for the
the time-oriented
time-oriented
modification of
of the
the design
design and
and access
access methods
database,
database, and
and the
the development
development of
of methods
methods for
for graphically
graphically viewing
viewing multiple
multiple
protocols
protocols that
that are
are combined
combined into
into one
one large
large knowledge
knowledge base.
base. These
These research
research
efforts
into the
In addition,
addition, some
some of
of the
the treatment
treatment
efforts will
will continue"
continue’into
the next
next year.
year. In
regimens
regimens developed
developed for
for the
the original
original mainframe
mainframe version
version are
are still
still in
in use
use and
and
of
ONCOCIN.
The
process
of
can
be
transferred
to
the
new
version
The
process
of
can be transferred to the new version of ONCOCIN.
converting
converting this
this knowledge
knowledge will
will also
also be
be undertaken
undertaken in
in the
the next
next year.
year. As
As the
the
knowledge
additional mechanisms
mechanisms will
will be
be needed
needed for
for the
the
knowledge base
base grows,
grows, additional
incremen
tal update
Additional changes
changes in
in the
the
incremental
update and
and retraction
retraction of
of protocols.
protocols. Additional
reasoning
reasoning and
and interface
interface components
components of
of the
the system
system are
are described
described below.
below.
E.
E. H.
H. Shortliffe
Shortliffe
126
126
5P41-RR00785-14
5P41-RR00785-14
ONCOCIN Project
ONCOCIN
research project
project related to
to ONCOCIN
ONCOCIN was
was started this
this last year.
year. We
AA new research
exploring the use
use of
of continuous
continuous speech
speech recognition
recognition as
as an alternate entry
entry
are exploring
method for
for communicating
communicating with
with ONCOCIN.
ONCOCIN.
This project
project requires the
method
This
connection of
of speech
speech recognition
recognition equipment
equipment produced by
by Speech
Speech Systems,
Systems,
connection
Inc. of
of Tarzana to the ONCOCIN
ONCOCIN interface
interface module. Christopher
Christopher Lane has
has
prototype network
network connection
connection and command interpreter
interpreter
already developed a prototype
between the speech
speech module
module (running
(running on a Sun with
with special hardware added)
added)
between
Xerox 1186 computer
computer that
that runs ONCOCIN.
ONCOCIN. Clifford
Clifford Wulfman
Wulfman has
has
and the Xerox
designed a series
series of
of modifications
modifications to the ONCOCIN
ONCOCIN user
user interface
interface to allow
allow
designed
for verbal commands. Graduate student Danielle
Danielle Fafchamps has
has helped to
for
experiments to elicit
elicit how clinicians
clinicians would like
like to phrase
phrase their
their
design experiments
requests to ONCOCIN.
ONCOCIN.
requests
creating a new version of
of the Librarian
Librarian program which
Janice Rohn is creating
facilitates the physician’
physician'ss initial
initial communication
communication with
with the ONCOCIN
ONCOCIN system
system
facilitates
(based on the original
original version by Cliff
Cliff Wulfman).
Wulfman).
We continue
continue to
(based
collaborate with
with Andy
Andy Zelenetz, Richard
Richard Lenon, Robert Carlson, and
collaborate
Charlotte Jacobs
Jacobs on the design and implementation
implementation of
of ONCOCIN
ONCOCIN in the
Charlotte
clinic. Stephen
Stephen Rappaport
Rappaport has
has started a residency program to continue
continue his
clinic.
medical education.
Research in
in Progress
Progress
C.2 Research
research in the ONCOCIN
ONCOCIN project
project over the last year comprised three major
Our research
categories: (1) conversion of
of ONCOCIN
ONCOCIN to the workstation
workstation version, (2) development of
of
categories:
acquisition interface
interface (OPAL)
(OPAL) for
for entering
entering new protocols, and (3) modeling
a knowledge acquisition
of
process (ONYX).
ways to
of the strategic therapy selection process
(ONYX). We are
are now able to explore ways
test the system
system beyond the Stanford
Stanford environment.
environment.
A
A summary of
of our current
current research
research endeavors
endeavors follows.
follows.
C.2.]
system from
from the DEC-20
DEC-20 to the Xerox
Xerox 1100
C.2.1 Transfer
Transfer of
of the ONCOCIN
ONCOClN system
II00 Series
Series
machines
machines
During
process of
workstation version of
we
During the process
of converting
converting to the workstation
of ONCOCIN,
ONCOCIN, we
redesigned
have completed the major porclon
portion of
redesigned segments
segments of
of the program. We have
of that
work, and our
with the new version has
has suggested
our experience with
suggested additional
additional areas
areas for
for
improving
improving the reasoning techniques
techniques and knowledge representation of
of ONCOCIN
ONCOCIN. .
•. RedeSign
Redesign of
of the reasoning
reasoning component. A major impetus for
for the redesign
redesign of
of
the system
system was
was to develop more efficient
efficient methods
methods to search
search the knowledge
knowledge
base
have implemented a reasoning
reasoning
base during
during the running
running of
of a case.
case. We have
program that uses
uses a discrimination
discrimination network
network to process
process the cancer
cancer protocols.
This
This network
network provides for
for a compact representation of
of information
information which isis
common to many protocols but does
does not require the program to consider
and then disregard information
information related to protocols that are
are irrelevant
irrelevant to a
particular
particular patient. We continue
continue to improve
improve portions of
of the reasoning
reasoning
component
component that are associated
associated with
with reasoning
reasoning over time; e.g.,
e.g., modeling the
the
appropriate
appropriate timing
timing for
for ordering
ordering (ests
tests and identifying
identifying the information
information which
needs
needs to be gathered
gathered before the
rhe next clinic
clinic visit. In generaL
general, we
we :ue
3re
concentrating
concentrating on improving
improving the representation of
of the
the knowledge
knowledge regarding
sequences
sequencesof
of therapy actions specified by the protoco\'
protocol.
Our experience
experience with
with adding a large
large number of
of protocols has
has led
led to the
the
evaluation of
of the design
design of
of the internal
internal structure of the knowledge
knowledge base
base (e.g.,
(e.?.,
the way
way we
we describe
describe the relationships
relationships between
between chemotherapies,
chemotherapies, drugs,
drugs, and
and
treatment visits). We will
will continue :0
;o improve
improve the method for
for traversing
127
127
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
ONCOCIN Project
Project
ONCOCIN
plan structure
structure in
in the knowledge
knowledge base,
base, and consider
consider alternative
alternative
the plan
the
for representing
representing the
the structure
structure of
of chemotherapy
chemotherapy plans:
plans.
arrangements for
of treatment
treatment guidelines
guidelines and
and the
the patient
patient
Currently, the
the knowledge
knowledge base
base of
Currently,
database are separated.
separated.
We propose to
to tie
tie these two
two structures closer
database
We
on turning
turning ONCOCIN
ONCOCIN into
into a
Additional work
work is anticipated
anticipated on
together. Additional
the physician
physician enters their
their therapy
therapy and ONCOCIN
ONCOCIN
critiquing system,
system, where the
critiquing
provides suggestions
suggestions about
about possible alternatives
alternatives to
to the entered therapy.
provides
Although we have concentrated
concentrated our
our review
review of
of the ONCOCIN
ONCOCIN design
Although
primarily on the data provided
provided by additional
additional protocols,
protocols, we know
know that
that nonnonprimarily
problems may also raise similar
similar issues.
issues. The E-ONCOCIN
E-ONCOCIN
cancer therapy problems
effort is designed
designed to produce a domain-independent
domain-independent therapy planning
planning system
system
effort
that includes the lessons
lessons learned from
from our
our oncology
oncology research.
research. Samson
Samson Tu
Tu is
that
for continued
continued improvement
improvement of
of the reasoning
primarily responsible for
primarily
component of
of ONCOCIN.
ONCOCIN .
component
Development of
of a temporal
temporal network.
network. The ability
ability to represent temporal
temporal
.• Development
information is a key element of
of programs that
that must reason
reason about treatment
treatment
information
earlier version of
of the ONCOCIN
ONCOCIN system
system did
did not
not have an
protocols. The earlier
for reasoning about
about time-oriented
time-oriented events.
events.
We are
explicit structure
structure for
explicit
different configurations
configurations of
of the temporal network,
network, and
experimenting with
with different
experimenting
for querying the network.
with the syntax for
with
We are also adapting this
interface with
with the ONYX
ONYX therapy-planning
therapy-planning systems.
systems.
network so that itit can interface
network
part of
of Michael Kahn’
Kahn'ss Ph.D. thesis.
thesis.
This research
research on temporal
temporal reasonin
reasoningg is part
This
Information Sciences
Sciences Program at
3.t
Michael is a student in
in the Medical
Medical information
University of
of California
California at San
San Francisco.
Francisco .
University
•. Extensions
Extensions to the user
We continue
user interface.
interface.
continue to experiment
experiment with
with various
configurations
user interface.
been in
Many of
of the changes
changes have
have been
configurations of
of the user
interface. Many
response
response to requests
requests for
for a more flexible
flexible data management
management environment.
environment. We
becomes available corresponding to a
are occasionally faced with
with data that
that becomes
time
time before the current
current visit.
visit. This
This can
can happen if
if a laboratory
laboratory result is
delayed,
delayed, or a patient's
patient’s electronic
electronic flowsheet is started in the
the middle
middle of
of the
treatment. We have
have added
added the ability
ability to
co create
create new columns of
of data,
data, and
are designing the changes
changes to the temporal processing
processing components of
of
ONCOCIN
to
allow
for
data
that
is
inserted
out
of
ONCOCIN
for
of order. We have
have also
also
extended
extended the flowsheet to allow for
for patient
patient specific
specific parameters
parameters (e.g.,
(e.g., special
special
test
test results
results or symptoms) that the physician wishes
wishes to follow
follow over time. The
flowsheet layouts have
have been
been modified
modified to create
create protocol specific flowsheets,
flowsheets,
e.g.,
lymphoma
flowsheets
have
a
different
e.g., lymphoma flowsheets have different configuration
configuration than lung cancer
cancer
flowsheets.
flowsheets. The basic
basic structure of
of the interface
interface has
has been
been modified
modified to use
use
object-oriented
object-oriented methods,
methods, which allows for
for more flexible
flexible interaction
interaction between
between
different
different components of
of the flowsheet and the operations performed
performed on the
the
flowsheet.
flowsheet.
A continuing
continuing area
area of research
research concerns
concerns how to guide
guide the
the user
user to the most
appropriate items to enter (based
(based on the needs
needs of the
the reasoning
reasoning program)
programj
without
without disrupting
disrupting the
the fixed
fixed layout of
of the f1owsheet.
flowsheet. The mainframe
mainframe
version of
of ONCOCIN
ONCOCIN modified
modified the
the order of
of items on the
the flowsheet to
extract
extract necessary
necessary information
information from
from the user.
user. In the
the workstation version,
version, we
we
have
have developed
developed aa guidance
guidance mechanism
mechanism which alerts
alerts the
the user
user to items that are
are
needed
needed by the
the reasoning
reasoning program. The user
user isis not required to deviate
deviate from
from
aa preferred order of
of entry
entry nor required to respond
respond to
to aa question
question for which
no current
current answer
answer isis available. Cliff
Cliff Wulfman
Wulfman isis primarily
primarily responsible
responsible for
for
improvements
improvements to
to the
the user
user interface
interface of
of ONCOCIN
ONCOCIN. .
•. System
System support
support for
for the
the reorganization.
reor,Tunizution. The
The USP
LISP language.
language. which
which we
we used
used to
to
E.
E. H.
H. Shortliffe
Shortliffe
128
128
5P41-RR00785-14
5P41-RR00785-14
ONCOCIN Project
Project
ONCOCIN
build the first
first version of
of ONCOCIN,
ONCOCIN, does
does not
not explicitly
explicitly support
support basic
build
manipulation techniques (such as
as message
message passing,
passing, inheritance
inheritance
knowledge manipulation
or other
other object-oriented
object-oriented programming
programming structures).
These
techniques, or
but none of
of the
facilities are available
available in
in some commercial
commercial products, but
facilities
implementations provide
provide the reliability,
reliability, speed,
speed, size,
size, or
or
existing commercial
commercial implementations
existing
memory-manipulation techniques that
that are needed
needed for
for our
our project.
special memory-manipulation
"minimal" object-oriented
object-oriented system
system to meet our
our
therefore developed a “minimal”
We have therefore
specifications. The
The object
object system
system is currently
currently in
in use
use by each
each component
component of
of
specifications.
the new version of
of ONCOCIN
ONCOCIN and in
in the software
software used
used to connect these
these
based on
In addition,
addition, all
all ONCOCIN
ONCOCIN student projects are now based
components. In
this programming
programming environment.
environment. Christopher
Christopher Lane created and is responsible
this
for modifications
modifications to the object-oriented
object-oriented system.
system.
for
Interactive Entry
Entry of
of Chemotherapy
Chemotherapy Protocols
Protocols by Oncologists
Oncologists (OPAL)
(OPAL)
C.2.2 Interactive
major effort
effort in
in this
this grant
grant year has
has been
been the continued
continued development
development and testing of
of
AA major
software (the OPAL
OPAL system) that will
will permit
permit physicians who are not
not computer
protocol information
information on a structured
structured set of
of forms
forms presented
presented on a
programmers to enter protocol
entry of
of the system’
system'ss knowledge.
knowledge.
Most expert systems
systems require tedious entry
graphics display. Most
of knowledge is transferred
transferred from
from
systems, each
each segment
segment of
In many other medical expert systems,
into the expert system.
system.
the physician to the programmer,
programmer, who then enters the knowledge into
of the generally well-structured
well-structured nature of
of cancer treatment
have taken advantage
advantage of
We have
that can be used
used directly
directly by clinicians.
clinicians. The
design a knowledge entry
entry program that
plans to design
of cancer treatment
treatment plans includes:
structure of
multiple protocols (that
(that may be related to each
each other);
other);
.• choosing among multiple
protocol;
•. describing experimental
experimental research
research arms in each
each protocol;
•. specifying
specifying individual
individual drugs and drug combinations;
combinations;
•. setting the drug dosage
dosage level;
•. and modifying
modifying either
either the choice of
of drugs or their
their dosage.
dosage.
presented to the user
user as
Using the graphics-oriented
graphics-oriented workstations,
workstations, this information
information is presented
as
computer-generated forms
user fills
forms which
which appear on the screen.
screen. After
After the user
fills in the
blanks on the forms,
used to drive
forms, the program generates
generates the rules used
drive the reasoning
reasoning
process.
process. As the user
user describes
describes more detailed aspects
aspects of
of the protocol,
protocol, new forms are
are
added
added to the computer
computer display; these
these allow the user
user to specify the special
special cases
cases that
make
make the protocols so
so complicated. Although
Although the user
user is unaware
unaware of
of the creation of
of the
the
knowledge
knowledge base
base from
from the interaction
interaction with
with OPAL,
OPAL, a complex set
set of
of translations are
are
taking place.
place. The user's
user’s entries are
are mapped
mapped into
into an intermediate
intermediate data
data structure (IDS)
that is common for
for all protocols. From the IDS, a translation
translation program generates
generates rules
rules
for
for creating and modifying
modifying treatment, and integrates them with
with the existing ONCOCIN
ONCOCIN
knowledge
knowledge base.
base. Improving
Improving the design
design of
of the IDS and the rule translation
translation programs
programs
will
will be
be a major research
research effort
effort of
of this year.
year.
Although
Although the "forms"
“forms” were
were specifically
specifically designed
designed for
for cancer
cancer treatment
treatment plans,
plans, the
techniques
techniques used
used to organize data
data can
can be extended
extended to other clinical
clinical trials, and eventually
to other structured decision tasks.
tasks. The key factor
factor is to exploit
exploit the regularities in the
structure of
of the task
task (e.g.,
(e.g., this interface
interface has
has an
an extensive notion
notion of
of how chemotherapy
chemotherapy
regimens
regimens are
are constructed) rather than to try to build
build a knowledge-entry
knowledge-entry system
system that can
can
accept
The OPAL
possible problem specifiCation.
specification.
accept any possible
OPAL program is based
based upon
upon a
domain-independent
domain-independent forms
forms creation package
package designed
designed and implemented
implemented by
by David
Combs.
Corn bs. This
This program will
will provide the basis
basis for
for our extension of
of OPAL
OPAL to other
application
application areas.
areas.
129
129
E.
E. H. Shortliffe
Shortliffe
5P41- RR00785-14
5P41-RR00785-14
ONCOCIN Project
Project
ONCOCIN
thirty-five protocols
protocols covering
covering many different
different organ systems
systems and
We have now entered thirty-five
of protocol
protocol design (increased from
from 6 in
in last year’
year'ss annual report).
report). Based
Based on this
this
styles of
modifying OPAL
OPAL to
to increase the percentage
percentage of
of the protocol
protocol that
that can
experience, we are modifying.
experience,
our clinical
clinical collaborators.
collaborators. One direction
direction in
in which
which we have
directly by our
be entered directly
of nodes and arcs to
extended the OPAL
OPAL program is in
in providing
providing a graphical
graphical interface
interface of
extended
specify the procedural knowledge about the order
order of
of treatments and important
important decision
specify
This work
work is described in
in several
several papers by Musen.
M usen.
points within
within the treatments. This
points
C.2.3 Strategic
Strategic Therapy
Therapy Planning
Planning (ONYX)
(ONYX)
mentioned above,
above, we have continued
continued our
our research
research project
project (ONYX)
(ONYX) to study the
As mentioned
therapy-planning process
process and to determine
determine how clinical
clinical strategies are used
used to plan
therapy-planning
therapy in
in unusual
unusual situations.
situations. Our
Our goals for
for ONYX
ONYX are: (1) to conduct
conduct basic research
research
into the possible representations of
of the therapy-planning
therapy-planning process,
process, (2) to develop a
into
process, and (3) eventually
eventually to interface
interface the planning
planning
computer program to represent this process,
with ONCOCIN.
ONCOCIN. We have worked with
with our clinical
clinical collaborators
collaborators to determine
determine
program with
how to create
create therapy plans for
for patients whose
whose special clinical
clinical situation
situation preclude
following the standard therapeutic plan described in
in the protocol
protocol document.
following
prototype program design
design has
has four
four components: (1) to review the patient’
patient'ss past
The prototype
formulate a small number
number of
of revised
record and recognize emerging problems, (2) to formulate
of the generated
generated
therapy plans based
based on existing
existing problems, (3) to determine
determine the results of
simulation, and (4) to weight the results of
of the simulation
simulation and rank
plans by using simulation,
performing decision analysis. This
This model is described in the papers
papers
order the plans by performing
by Langlotz.
We have
built an expert system
based on decision analytic
have built
system based
analytic techniques
techniques as
as part of
of the
planning problem. The program carries out a
solution
solution to the fourth
fourth step
step of
of the ONYX
ONYX planning
dialogue with
user concerning the particular
particular treatment
with the user
treatment choices to be compared,
patient-specific utilities
utilities corresponding
potential problems with
with the treatments, and the patient-specific
potential
corresponding
to the possible
possible outcomes.
A decision tree is automatically
automatically created,
created, displayed on the
outcomes. A
screen,
presented to the user,
user, and is compatible
with a
screen, and solved.
solved. The solution
solution is presented
compatible with
research of
explanation program for
for decision trees
trees being developed
developed as
as part of
of the Ph.D. research
of
Curtis
Curtis Langlotz.
C.2A
Documentation
C.2.4 Documentation
In 1986,
1986, we
we videotaped a lecture and demonstration
demonstration of
of the ONCOCIN
ONCOCIN and OPAL
OPAL
systems
systems at the XEROX
XEROX Palo Alto
Alto Research
Research Center. This
This videotape is available for
for loan
from
from our offices. Our previous videotapes
videotapes have
have been
been shown at scientific
scientific meetings and
The publications
have
have been
been distributed
distributed to many researchers
researchers in other countries.
publications
described
described below further
further document our recent work on ONCOCIN.
ONCOCTN.
C.2.5
C.2.S Dissemination
Dissemination
We are
are planning
planning experimental installation
installation of
of ONCOCIN
ONCOCIN workstations
workstations in private
private
oncology offices
offices in San
San Jose
Jose and San
San Francisco. An application
application proposing this project
is currently
currently under review.
D. Publications
Publications Since January,
January, 1986
1986
1.
1. Musen,
Musen, M.A., Rohn, J.A.,
J.A., Fagan,
Fagan, L.M., and
and Shortliffe,
Shortliffe, E.H. Knowledge
engineering for
for a clinical
clinical trial
trial advice system:
system: Uncovering
Uncovering errors in protocol
protocol
specification
specification (Memo KSL-85-51).
KSL-85-51). Proceedings
Proceedings of
of AAMSI
AAMSI Congress
Congress 86
56 (A.
Levy and
and B.
B. Williams,
Williams, eds.),
eds.), pp.
pp. 24-27, Anaheim, 8-10
S-10 May 1986.
1986.
2.
2. Langlotz, c.P.,
C.P., Fagan,
Fagan, L.M., and
and Shortliffe,
Shortliffe, E.H. Overcoming
Overcoming limitations
limitations of
of
E.
E. H. Shortliffe
Shortliffe
130
130
ONCOCIN
ONCOCIN Project
Project
SP41-RR0078S-14
5P41-RR00785-14
planning techniques.
techniques. Memo
Memo KSL-85-52.
KSL-8S-S2. Proceedings
Proceedings
artificial intelligence
intelligence planning
artificial
92-96,
of AAMSI
AAMSI Congress
Congress 86 (A.
CA. Levy and B. Williams,
Williams, eds.),
eds.), pp.
92-96,
of
1986.
Anaheim, 8-10 May 1986.
Musen, M.A., Fagan.
Fagan, L.M., and Shortliffe,
Shortliffe, E.H. Graphical
Graphical specification
specification of
of
3. Musen,
knowledge for
for an expert system.
system. Memo KSL-85-53.
KSL-8S-S3. Presented
Presented at
procedural knowledge
Second IEEE
IEEE Computer
Computer Society Workshop
Workshop on Visual
Visual Languages,
Languages, pp.
the Second
TX, June 1986. Reprinted
Reprinted in
in Expert
Expert Systems:
Systems: The User
167-178, Dallas, TX,
Interface (J.
(1. Hendler, ed.).
ed.). Norwood,
Norwood, NJ: Ablex
Ablex Publishing
Publishing Company, 1987.
1987.
Interface
C.P., Fagan,
Fagan, L.M.,
L.M., Tu, S.W.,
S.W., Sikic,
Sikic, B.I.,
B.L, and Shortliffe,
Shortliffe, E.H. A
A
4. Langlotz, C.P.,
planning architecture
architecture that
that combines decision theory
theory and artificial
artificial
therapy planning
techniques. KSL-85-55.
KSL-8S-SS. Submitted
Submitted for
for publication,
publication, November
November
intelligence techniques.
1986.
1986.
S. Combs,
Combs, D.M., Musen,
Musen, M.A., Fagan,
Fagan, L.M., and Shortliffe,
Shortliffe, E.H. Graphical
Graphical
5.
Memo KSL-85-56.
entry of
of procedural and inferential
inferential knowledge.
knowledge.
KSL-8S-S6.
Proceedings of
of AAMSI
AAMSI Congress
Congress 86 (A. Levy and B. Williams,
Williams, eds.),
eds.), pp.
Proceedings
298-302, Anaheim, 8-10 May 1986.
1986.
C.D., Frisse,
Frisse, M.E., Fagan,
Fagan, L.M., and Shortliffe,
Shortliffe, E.H. Object-oriented
Object-oriented
6. Lane, CD.,
interface design.
design. Memo KSL-85-58.
KSL-8S-S8. Proceedings
Proceedings of
of
graphics in medical interface
AAMSI Congress
Congress 86 (A. Levy and B. Williams,
Williams, eds.),
eds.), pp. 293-297, Anaheim,
Anaheim,
AAMSI
8-10 May 1986.
1986.
Musen, M.A., Fagan,
Fagan, L.M., Combs, D.M., and Shortliffe,
Shortliffe, E.H. Facilitating
Facilitating
7. Musen,
knowledge entry for
for an
knowledge
an oncology therapy advisor using a model of
of the
Proceedings of
of MEDINFO-86,
MEDINFO-86, pp.
Proceedings
application
application area.
area. Memo KSL-86-1.
KSL-86-1.
46-S0,
46-50, Washington, D.C., October 1986.
1986.
8.
Fagan, L.M., Tu, S.W.,
S.W., Sikic, B.I., and Shortliffe,
Shortliffe, E.H.
C.P., Fagan,
8. Langlotz, c.P.,
Combining
Combining artificial
artificial intelligence
intelligence and decision analysis for
for automated therapy
Proceedings of
of MEDINFO-86,
MEDINFO-86, pp.
planning assistance.
assistance. Memo KSL-86-3.
KSL-86-3. Proceedings
794-798,
794-798, Washington, D.C.,
D.C., October 1986.
1986.
Context-specific
9.
E.H.
Context-specific
Fagan, L.M., and Shortliffe,
Shortliffe,
9. Kahn, M.G., Fagan,
interpretation
interpretation of
of patient records
records for
for a therapy advice system.
system. Memo
KSL-86-4.
KSL-86-4. Proceedings
Proceedings of
of MEDINFO-86,
MEDINFO-86, pp. 175-179, Washington,
Washington, D.C.,
October 1986.
1986.
Use of
of a
10.
Musen, M.A., Fagan,
Fagan, L.M., Combs,
Combs, D.M., and Shortliffe,
Shortliffe, E.H. Use
10. Yfusen,
Memo0
domain model
'\:fern
model to drive an
an interactive
interactive knowledge-editing
knowledge-editing tool.
appear in the International
International Journal of
of :Yfan-Machine
Man-Machine
KSL-86-24.
To appear
KSL-86-24.
Studies,
Studies, 1987.
1987.
11.
C-P., Shortliffe,
Shortliffe, E.H.,
E.H., and Fagan,
Fagan, L.M. Using decision theory to
11 Langlotz, C.P.,
Proceedings of
of AAAI-86,
AAAI-86, pp.
pp.
justify
Memo KSL-86-26.
Proceedings
KSL-86-26.
justify heuristics.
heuristics.
21S-219,
215-219, Philadelphia, August 1986.
1986.
Artificial
Tntelligence in Management Decisions:
Decisions:
12.
Artificial
Intelligence
Shortliffe, E.H.
12. Shortliffe,
ONCOCIN.
KSL-86-39. Proceedings
Proceedings of
of aa Conference on Medical
ONCOCIN. Memo KSL-86-39.
[nformation
lnformation Sciences,
Sciences, University
University of
of Texas
Texas Health Sciences
Sciences Center at San
San
appear in Frontiers
Frontiers of
of Medical Information
Information
Antonio,
Antonio, July 1985.
1985. To appear
Sciences,
Sciences,Praeger
Praeger Publishing, 1986.
1986.
13.
Lane, C.
C. The Ozone
Ozone (0
(0;)3) Reference
Reference Manual. KSL-86-40,
KSL-86-40, July 1986.
1986.
13. Lane,
131
131
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
ONCOCIN Project
Project
ONCOCIN
M.A., Combs, D.M.,
D.M., Walton,
Walton, J.D., Shortliffe,
Shortliffe, E.H.,
E.H., and Fagan, L.M.
L.M.
14. Musen, M.A.,
computer-aided design of
of oncology
oncology advice systems.
systems. Memo
Memo
OPAL: Toward
Toward the
the computer-aided
OPAL:
of the
the Tenth
Tenth Annual
Annual Symposium on
on Computer
Computer
KSL-86-49. Proceedings of
KSL-86-49.
Applications in
in Medical
Medical Care, pp. 43-52, Washington,
Washington, D.C., October
October 1986.
Applications
Reprinted in
in Topics
Topics in
in Medical
Medical Artificial
Artificial Intelligence
Intelligence (P.L.
(P.L. Miller,
Miller, ed.), New
New
Reprinted
York: Springer-Verlag,
Springer-Verlag, 1987.
York:
for physicians.
Shortliffe, E.H. Medical
Medical expert
expert systems:
systems: Knowledge tools
tools for
15. Shortliffe,
Memo KSL-86-52.
KSL-86-52. Special issue
issue on Medical
Medical Informatics,
Informatics, West. J. Med.
Memo
145:830-839, 1986.
1986.
145:830-839,
Medical expert systems
systems research
research at Stanford
Stanford University.
University.
Shortliffe, E.H. Medical
16. Shortliffe,
Memo KSL-86-53.
KSL-86-53.
Presented at the Twentieth
Twentieth IBM
IBM Computer
Computer Science
Science
Memo
Presented
Symposium, Shizuoka, Japan,
Japan, October 1986.
1986.
Fagan, L.M.
L.M.
17. Langlotz,
Langlotz, C.P.,
c.P., Shortliffe,
Shortliffe, E.H., and Fagan,
for
AA methodology for
of decision analysis.
analysis.
Working
paper,
computer-based explanation
explanation of
Working
KSL-86-57, November
November 1986.
1986.
KSL-86-57,
18. Shortliffe,
Shortliffe, E.H. Computers in support
support of
of clinical
clinical decision making. Memo
forthcoming Textbook
Textbook of
of
KSL-87-25, 1986.
1986.
To appear in Lippincott’
Lippincott's
KSL-87-25,
To
s forthcoming
Internal Medicine
Medicine (W.N.
(W.N. Kelley, ed.).
ed.).
Internal
19. Langlotz, C.P.
c.P. and Shortliffe,
Shortliffe, E.H. The relationship
relationship between
between decision theory
theory
default reasoning. Working
Working paper KSL-87-17,
KSL-87-17, 1987.
and default
1987.
20.
20. Shortliffe,
Shortliffe, E.H. Computer
Computer programs to support clinical
clinical decision making.
Memo KSL-87-30.
KSL-87-30. To
To appear
appear in JAMA,
JAMA, July 1987.
1987.
Funding Support
E. Funding
Support
Grant
Grant Title:
Title: "Therapy-planning
“Therapy-planning strategies
strategies for
for consultation
consultation by computer"
computer”
Principal
Principal Investigator: Edward H. Shortliffe
Shortliffe
Project Management:
Management: Lawrence M. Fagan
Fagan
Agency:
Agency: National
National Library
Library of
of Medicine
Medicine
ID
ID Number:
Number: LM-04136
LM-04136
Term: April
April 1987
1987 to March 1990
1990
Total
Total award:
award: $380,123
$380,123
Grant
Grant Title:
Title: "Knowledge
“Knowledge Management
Management for
for Clinical
Clinical Trial
Trial Advice Systems"
Systems”
Principal
Principai Investigator: Edward H. Shortliffe
Shortliffe
Project Management:
Management: Lawrence
Lawrence M. Fagan
Fagan
Agency:
Agency: National Library
Library of Medicine
Medicine
ID Number: 11 ROI
ROl LM04420-01
LMO4420-01
Term: September
September 1985
1985 through August 1988
1988
Total
Total award:
award: $314,707
$314,707
Grant
Grant Title:
Title: Postdoctoral Training
Training in Medical Information
Information Science
Science
Principal
Principal Investigator: Edward
Edward H. Shortliffe
Shortliffe
Project Management:
Management: Edward
Edward H.
H. Shortliffe
Shortliffe
Agency:
Agency: National Library
Library of
of Medicine
ID Number: 11 T32 LM07033
Term: July 1,
1, 1984
1984 -- June
June 30,
30, 1989
1989
Total
Total award:
award: $903,718
$903,718
Grant
Grant Title:
Title: Henry
Henry J.3. Kaiser
Kaiser Faculty
Faculty Scholar
Scholar in
in General
General Internal
Internal Medicine
Medicine
E.
E. H.
H. Shortliffe
Shortliffe
132
132
ONCOCIN Project
Project
ONCOCIN
5P41- RR00785-14
5P41-RR00785-14
Principal Investigator:
Investigator: Edward
Edward H.
H. Shortliffe
Shortliffe
Principal
Henry
J.
Kaiser
Family
Foundation
Agency:
Agency: Henry J. Kaiser Family Foundation
Term: July
July 1983 to
to June 1988
Term:
Total award: $250,000
$250,000 ($50,000 annually).
annually).
Total
Grant Title:
Title: Explanation
Explanation of
of Computer-assisted therapy plans
Grant
Principal Investigator:
Investigator: Lawrence M.
M. Fagan
Principal
National Institutes
Institutes of
of Health
Health
Agency: National
ID Number:
Number: 1 R23 LM04316
LM04316
ID
Term: 2/1985-l/1988
211985-111988
Term:
Total award: $107,441
$107,441
Total
WITH THE
THE SUMEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
II. INTERACTIONS
INTERACTIONS WITH
II.
A. Medical
Medical Collaborations
Collaborations and
and Program
Program Dissemination
Dissemination via
via SUMEX
SUMEX
A.
of interest
interest in
in ONCOCIN
ONCOCIN has
has been shown by the medical, computer
computer science,
science,
AA great deal of
and lay communities.
communities. We are frequently
frequently asked
asked to demonstrate the program to Stanford
Stanford
visitors. We also demonstrated our developing workstation
workstation code
code in
in the Xerox
Xerox exhibit
exhibit in
in
visitors.
associated with
with AAAI-84
AAAI-84 in Austin,
Austin, Texas,
Texas, IJCAI-85
IJCAI-85 in Los Angeles,
Angeles,
the trade show associated
AAAI-86 in Philadelphia,
Philadelphia, and Medinfo
Medinfo 86.
86. Physicians
Physicians have
have generally been
been enthusiastic
AAAI-86
ONCOCIN's
potential. The interest of
of the lay community
community is reflected
reflected in the
about ONCOCIN’
s potential.
frequent requests
requests for
for magazine interviews
interviews and television coverage
coverage of
of the work. Articles
Articles
frequent
MYCIN and ONCOCIN
ONCOCIN have
have appeared
appeared in such
such diverse publications
publications as
as Time
about MYCIN
Titne and
Fortune, and ONCOCIN
been featured on the “NBC
"NBC Nightly
Nightly News,”
News," the PBS
Fortune,
ONCOCIN has
has been
PBS
"Health
Notes" series,
“Health Notes”
series, and "The
“The MacNeil-Lehrer
MacNeil-Lehrer Report."
Report.” Most recently it
it appeared
appeared in
a special on Artificial
Artificial Intelligence
Intelligence for
for TV
TV Ontario
Ontario (Canadian PBS
PBS station).
station). Due to the
frequent
produced a videotape about
frequent requests
requests for
for ONCOCIN
ONCOCIN demonstrations, we have
have produced
the ONCOCIN
ONCOCIN research
research which
which includes demonstrations of
of our professional workstation
workstation
been shown at several
research
tape has
has been
several
research projects and the 2020-based
2020-based clinic
clinic system.
system. The tape
national
national meetings,
meetings, including
including the 1984
1984 Workshop on Artificial
Artificial Intelligence in Medicine,
Medicine,
the 1984
1984 meeting of
of the Society for
for Medical Decision Making.
Making, and
and the 1985
1985 meeting of
of
the Society for
for Research
Research and Education in Primary
Primary Care
Care [nternal
lnternal Medicme.
Medicine. The tape
tape
has
has also
also been
been shown to both national
national and
and international
international researchers
researchers 10
in biomedical
biomedical
computing.
computing. We have
have also completed an updated
updated tape.
tape.
Our group also
also continues to oversee
oversee the MYCIN
MYCIN program (not an
an active research
research project
since
since 1978)
1978) and the
the EMYCIN
EMYCIN program. Both systems
systems continue to be
be in demand
demand as
as
demonstrations of
of expert systems
systems technology.
technology. MYCIN
MYCIN has
has been
been demonstrated via
networks at both national
national and international
international meetings
meetings in the past.
past. and
and several
several medical
school and
and computer science
science teachers
teachers continue to use
use the program in their computer
science
Researcherswho visit
visit our laboratory
laboratory often begin
begin
science or
OT medical computing
computing courses.
courses. Researchers
their
their introduction
introduction by experimenting
experimenting with the
the MYCIN/EMYCIN
MYCIN/EMYCIN systems.
systems. We
We also
also have
have
made
made the
the MYCIN
MYCIN program available to researchers
researchers around the
the world who access
access
SUMEX
SUMEX using
using the GUEST account.
account. EMYCIN
EMYCIN has
has been
been made
made available to interested
researchers
researchers developing expert systems
systems who access
access SUMEX via the
the CONSULT
CONSULT account.
account.
One such
such consultation system
system for
for psychopharmacological
psychopharmacological treatment of depression,
depression, called
Bl
ue- Box (developed
ts, Benoit M
ulsan t and
Blue-Box
(developed by two French
French medical
medical studen
students,
Mulsant
and David
Servan-Schreiber),
Servan-Schreiber), was
was reported in July of
of 1983
1983 in Computers
Computers and Biomedical
Biomedical Research.
B. Sharing
EX-AIM Projects
Sharing and Interaction
Interaction with Other
Other SUM
SUMEX-AIM
Projects
The community
community created
created on the
the SUMEX
SUMEX resource
resource has
has other
other benefits which go
go beyond
beyond
we
are
able
to
experiment
with
other
developing
actual
shared
computing.
Because
Because
we
are
able
to
experiment
with
other
developing
actual shared computing.
systems,
systems, such
such as
as INTERNIST/CADUCEUS,
INTERT\iIST/CADUCEUS, and
and because
because we
we frequently
frequently interact
interact with
with
133
133
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
ONCOCIN Project
Project
ONCOCIN
other workers
workers (at
(at AIM
AIM Workshops
Workshops or
or at
at other
other meetings),
meetings), many
many of
of us
us have
have found
found the
the
other
of us
us have
have visited
visited workers
workers
scientific exchange and stimulation
stimulation to
to be heightened. Several of
scientific
other sites, sometimes for
for extended periods, in
in order
order to
to pursue
pursue further
further issues
issues which
which
at other
or workshop-based interactions.
interactions. In
In this
this regard,
regard, the
the
have arisen through
through SUMEXSUMEX - or
other
workers,
both
on
SUMEX
and
at
other
sites,
has
ability
to
exchange
messages
with
ability to exchange messageswith other
both
SUMEX
other
has
rapid and efficient
efficient dissemination
dissemination of
of ideas.
ideas. Certainly
Certainly itit is unusual for
for a
crucial to
to rapid
been crucial
community of
of researchers
researchers with
with similar
similar scholarly
scholarly interests to
to have at their
their disposal
small community
small
and efficient
efficient communication
communication mechanisms,
mechanisms, even
even among
among those
those researchers
researchers
such powerful
powerful and
such
on
opposite
coasts
of
the
country.
on opposite coasts of
country.
During this
this past two years,
years, we have had extensive interactions
interactions with
with Randy Miller
Miller at
at
During
Pittsburgh. Via
Via floppy
floppy disks and SUMEX,
SUMEX, we have experimented with
with several
several versions
Pittsburgh.
of the
the QMR
QMR program.
program. The
The interaction
interaction was
was very
very much
much facilitated
facilitated by
by the
the availability
availability of
of
of
SUMEX
for
communication
and
data
transmission.
SUMEX for communication
Critique of
of Resource
Resource Management
Management
C. Critique
Our community
community of
of researchers
researchers has
has been extremely fortunate
fortunate to work
work on a facility
facility that
Our
continued to
to maintain
maintain the
the high
high standards
standards that
that we
we have
have praised
praised in
in the
the past.
past. The
The
has continued
has
staff
members
are
always
helpful
and
friendly,
and
work
as
diligently
to
please
the
staff members are always helpful and friendly, and work as diligently to please the
SUMEX community
community as
as to please
please themselves.
themselves. As a result, the computer
computer is as
as accessible
accessible
SUMEX
easy-to-use as
as they can make it.
it. More
More importantly,
importantly, itit is a reliable
reliable and convenient
convenient
and easy-to-use
research tool. We extend special thanks to Tom
Tom Rindfleisch
Rindfleisch for
for maintaining
maintaining such high
research
our computing
computing needs
needs grow, we have
have increased
increased our
our dependence
dependence
profeSSional standards.
professional
standards. As our
as networking
networking and communication
communication protocols.
on special SUMEX
SUMEX skills
skills such as
III.
III. RESEARCH
RESEARCH PLANS
PLANS
A. Project
Project Goals and Plans
Plans
In the coming
efforts on
coming year,
year, there are several
several areas
areas in which we
we expect to expend our efforts
the ONCOCIN
ONCOCIN System:
System:
L
for cost-effective
dissemination
1. Development
Development of
of a workstation
workstation model
model for
cost-effective
dissemination of
of
clinical
''.:et this specific
specific aim we
we will
will continue
continue
clinical consultation
consultation systems. To ::---?et
the basic
basic and applied programmin:
programmin. dforts
Ifforts (ONCOCIN,
(ONCOCIN, OPAL,
OPAL, and ONYX)
ONYX)
described
described earlier in this report.
2.
for use by ONCOCIN
2. To encode and implement
implement for
ONCOCIN the commonly
commonly used
chemotherapy
chemotherapy protocols
protocols from
from our oncology clinic.
clinic. In the upcoming year,
year, we
we
will:
will:
•. Extend the OPAL
OPAL protocol
protocol entry system
system
•. Continue
of additional
additional protocols
Continue entry of
protocol/month
protocol/month (including
(including testing)
at
the
rate
of
of
one
one
3.
3. To continue
continue testing
testing of
of the workstation
workstation version of
of ONCOCIN.
ONCOCIN.
4.
4. To
To generalize
generalize the
the reasoning
reasoning and
and interaction
interaction components
components of
of the ONCOCIN
ONCOCIN
system for
for other
other applications.
applications.
B. Justification
Justification and
and Requirements
Requirements for
,for Continued
Continued SUMEX
SUMEX Use
Use
All
All the
the work
work we
we are
are doing
doing (ONCOCIN
(ONCOCIN plus
plus continued
continued use
use of
of the
the original
original MYCIN
MYCIN
program)
program) continues
continues to
to be
be dependent
dependent on
on daily
daily use
use of
of the
the SUMEX
SUMEX resource.
resource. Although
much
much of
of the
the ONCOCIN
O?dCOCIN work
work has
has shifted
shifted to
to Xerox
Xerox workstations,
workstations, the
the ::;mvlEX
SUMEX 2060
2060 and
and
E.
E. H.
H. Shortliffe
Shortliffe
134
134
ONCOCIN
ONCOCIN Project
5P41-RR00785-14
5P41-RR00785-14
programs all make
the 2020
2020 continue
continue to be key elements in our research
research plan. The programs
assumptions regarding the computing
computing environment
environment in which
which they operate.
operate.
In addition,
addition, we have
have long appreciated the benefits
benefits of
of GUEST
GUEST and network access
access to the
programs we are developing. SUMEX
SUMEX greatly enhances
enhances our ability
ability to obtain
obtain feedback
from
Network access
from interested physicians and computer
computer scientists around the country. Network
access
has
has also
also permitted
permitted high quality
quality formal
formal demonstrations
demonstrations of
of our work
work both from
from around
the United
United States
States and from
from sites
sites abroad (e.g.,
(e.g., Finland,
Finland, Japan,
Japan, Sweden,
Sweden, Switzerland).
The main
main development of
of our project
project will
will continue
continue to take place
place on LISP machines
which we have
have purchased
purchased or which have
have been
been donated by the XEROX
XEROX Corporation.
Corporation.
Requirements for
for Additional
Additional Computing
C. Requirements
Computing Resources
The acquisition
acquisition of
of the DEC
DEC 2020
2020 by SUMEX
SUMEX was
was crucial
crucial to the growth of
of our research
research
work. It
It ensured
ensured high quality
quality demonstrations
demonstrations and has
has enabled
enabled us
us to develop
develop a system
system
have begun
begun to develop
(ONCOCIN)
(ONCOCIN) for
for real-world
real-world use
use in a clinical
clinical setting. As we have
systems
systems that
that are potentially
potentially useful as
as stand-alone packages
packages (i.e.,
(i.e., an exportable
ONCOCIN),
ONCOCIN), the addition
addition of
of personal workstations
workstations has
has provided particularly
particularly valuable
new resources.
resources. We have
have made
made a commitment
commitment to the smaller Interlisp-D
Interlisp-D machines
machines ("0(“Dmachines")
machines”) produced by Xerox, and our work
work will
will increasingly transfer to them over the
next several
several years.
years. Our
Our current
current funding
funding supports our effort
effort to implement
implement ONCOCIN
ONCOCIN
on workstations
workstations in the Stanford
Stanford oncology clinic
clinic (and eventually to move the program to
non-Stanford
non-Stanford environments),
environments), but we will
will simultaneously continue
continue to require access
access to
lnterlisp
Interlisp on upgraded
upgraded workstations
workstations for
for extremely CPU-intensive
CPU-intensive tasks.
tasks. Although
Although our
dependence
dependence on SUMEX
SUMEX for
for workstations
workstations has
has decreased
decreased due
due to a recent gift
gift from
from
XEROX,
XEROX, our requirements for
for network
network support
support of
of the machines
machines has
has drastically
increased.
Individual machines do not
not provide
provide sufficient
sufficient space
space to store all of
of the
increased. Individual
software used
used in our project, nor
nor to provide
provide backup or long-term
long-term storage
storage of
of work
work in
progress.
It is the networks, file
file storage
storage devices,
devices, protocol
protocol converters, and other parts of
of
progress. It
the SUMEX
SUMEX network
network that
that hold our project together. In addition,
addition, with
with a research
research group
of
of about 20 people,
people, we are taking
taking advantage
advantage of
of file
file sharing, electronic mail, and other
information
information coordinating
coordinating activities
activities provided by the DEC
DEC 2060.
2060. We hope
hope that with
with
systems
systems support
support and research
research by SUMEX
SUMEX staff,
staff, we
we will
will be able to gradually move away
away
from
from a need
need for
for the central coordinating
coordinating machine over the next five
five years.
years.
The acquisition
acquisition of
of the DEC
DEC 2060,
2060, coupled with
with our increasing use
use of
of workstations.
workstations, has
has
greatly helped with
with the problems in SUMEX
SUMEX response
response time
time that we
we had
had described
described in
previous annual reports. We are extremely grateful
grateful for
for access
access both to the central
machine and to the research
research workstations
workstations on which we are currently
currently building
building the new
new
ONCOCIN
permitting development
D-machine’s greater address
address space
space is permitting
ONCOCIN prototype.
prototype. The O-machine's
capabiiicy of
of the
of
of the large knowledge
knowledge base
base that ONCOCIN
ONCOCIN requires.
requires. The graphics capability
workstations
workstations has
has also
also enabled
enabled us
us to develop new methods
methods for
for presenting material to
naive users.
users. [n
ln addition,
addition, the workstations have
have provided a reliable, constant "Ioad“loadaverage"
average” machine for
for running
running experiments with
with physicians and for
for development work.
The development of
of ONCOCIN
ONCOCIN on the D-machine
D-machine will
will demonstrate
demonstrate the feasibility
feasibility of
of
running
running intelligent
intelligent consultation
consultation systems
systems on small, affordable
affordable machines
machines in physicians'
physicians’
offices and other remote sites.
sites.
offices
Recommendations for
for Future
Future Community
D. Recommendations
Community and Resource Development
SUMEX
SUMEX is providing
providing an excellent research
research environment
environment and we
we are delighted with
with the
help that SUMEX
SUMEX staff
staff have
have provided implementing
implementing enhanced
enhanced system
system features
features on the
2060
2060 and on the workstations. We feel that we
we have
have a highly
highly acceptable
acceptable research
research
Workstation availability
availability is becoming
environment
environment in which to undertake our work. Workstation
increasingly
increasingly crucial
crucial to our research,
research, and we
we have
have found
found over the past year
year that
SUMEX staff
staff has
has been
been very helpful
heipful and
workstation
workstation access
access is at a premium.
premium. The SUMEX
allowing us
us O-machllle
D-machine Jse
use
understanding about our needs
needs for
for workstation
workstation access.
access, allowing
135
135
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
ONCOCIN Project
Project
ONCOCTN
wherever possible,
possible, and
and providing
providing us
us with
with systems-level
systems-level support
support when
when needed.
needed. We
We look
look
wherever
forward to
to the
the arrival
arrival of
of additional
additional advanced
advanced workstations
workstations and
and the
the development
development of
of aa
forward
more distributed
distributed computing
'computing environment
environment through
through SUMEX-AIM.
SUMEX -AIM.
more
E. Responses
Responses to
to Questions
Questions Regarding
Regarding Resource
Resource Future
Future
E.
"What do
do you
you think
think the
the role
role of
of the
the SUMEX-AIM
SUMEX-AIM resource
resource should
should be
be for
for the
the
1. “What
period after
after 7/86,
7/86, e.g.,
e.g., continue
continue like
like itit is,
is, discontinue
discontinue support
support of
of the
the central
central
period
machine, act
act as
as aa communications
communications crossroads,
crossroads, develop
develop software
software for
for user
user
machine,
community
workstations,
etc.?"
community workstations, etc.?”
We believe
believe that
that the
the trend
trend towards
towards distributed
distributed computing
computing that
that characterized
characterized
We
continue
during
the
second
half
of
the decade.
decade.
the
early
1980's
will
the early 1980’s will continue during the
half of the
Although we have begun this
this process
process by
by moving
moving much
much of
of our
our research
research
Although
activity to LTSP
LISP machines, the SUMEX
SUMEX DEC-20
DEC-20 continues
continues to
to be a major
major
activity
source of
of support
support for
for all
all communication,
communication, collaboration,
collaboration, and administrative
administrative
functions. ItIt also continues
continues to provide
provide a quality
quality LISP environment
environment for
for
functions.
rapid prototyping,
prototyping, student projects in
in the early
early stages
stages before
before workstations
workstations are
rapid
for demonstrating
demonstrating system
system features to people at a
made available, and for
distance. These latter
latter functions
functions are still
still not
not well
well handled by
by distributed
distributed
machines, and we believe that
that a logical
logical role
role for
for the resource
resource in
in the future
future is
software and communications
communications techniques that
that will
will allow
allow us
us to
to develop software
further decrease
decrease our dependence
dependence on the large central
central machine.
further
"Will you require continued
continued access
access to the SUMEX-AIM
SUMEX-AIM 2060 and ifif so,
so, for
for
2. “Will
long?"
how long?”
needs could still
with a gradual phaseout
phaseout of
of
above, our needs
As indicated above,
still be met with
the 2060
2060 over the next 3-5 years,
current services
services such as
as file
file
years, provided
provided that current
handling
advanced network
network
handling and backup, mail,
mail, document preparation,
preparation, and advanced
support
file server plus the
support are
are available from
from other machines (e.g.,
(e.g., SAFE
SAFE file
of an
Medical Computer
This implies
implies maintenance of
Computer Science
Science file
file server). This
ARPANET
ARPANET connection,
connection, connections to other campus
campus machines,
machines, and faCIlities
facilities
for
for linking
linking together the heterogeneous
heterogeneous collection
collection of
of computing
computing equipment
upon which our research
research group depends.
depends. SUMEX
SUMEX would need
need to concentrate
on providing
providing software support for
for networks and systems
systems software for
for
workstations if
if it
it were
were to provide the
the same
same level of
of service
service we
we now
experience
experience while moving to aa fully
fully distributed
distributed environmenl.
environment.
3.
3. "What
“What would be
be the
the effect
effect of
of imposing
imposing fees
fees for
for using
using SUMEX
SUMEX resources
resources
(computing
and
communications)
if
NIH
were
to
require
this?"
(computing and communications) if NIH were
this?”
Since
Since all our research
research isis NIH-supported,
NTH-supported, we
we see
see nothing
nothing but administrative
administrative
headaches
headaches without
without benefits if
if there were
were to be
be aa move to require fee-forfee-forservice
service billing
billing for
for access
access to
to shared
shared SUMEX
SUMEX resources.
resources. The net effect
effect would
simply
simply be
be aa transfer
transfer of
of funds
funds from
from one
one arm
arm of
of NIH
NIH to another (assuming
(assuming
that
that the
the agencies
agencies that
that currently
currently fund
fund our
our work
work could
could supplement our grants
grants
to
to cover
cover SUMEX
SUMEX charges),
charges), and
and there
there would
would be
be aa simultaneous
simultaneous restraining
restraining
effect
The
effect on
on the
the research
research environment.
environment.
The current
current scheme
scheme permits
experimentation
experimentation and
and flexibility
flexibility in
in use
use that
that would
would be
be severely
severely inhibited
inhibited if
if all
all
access
accessincurred
incurred an
an incremental
incremental charge.
charge.
4.
4. "Do
“Do you
you have
have plans
plans to
to move
move
if
so,
when
and
to
what
if so, when and to what kind
kind
E.
E. H.
H. Shortliffe
Shortliffe
your
your work
work to
to another
another machine
machine workstation
workstation and
and
of
system?"
of system?”
136
136
5P41-RR00785-14
5P41-RR0078514
ONCOCIN
ONCOCIN Project
As mentioned above,
above, and described
described in
in greater detail in
in our annual report, we
are making
making a major
major effort
effort to move much of
of our research
research activity
activity to LISP
machines (currently
(currently Xerox 1108's,
1108’s, 1186's
1186’s and HP-9836's).
HP-9836’s). Our familiarity
familiarity
with
with this technology, and our commitment
commitment to it, have
have resulted solely from
from
the foresight
foresight of
of the SUMEX
SUMEX resource
resource in
in anticipating
anticipating the technology and
providing for
providing
for it
it at the time
time of
of their
their last renewal. However, for
for the reasons
reasons
mentioned
mentioned above,
above, we continue
continue to depend
depend upon the central communication
communication
node for
for many aspects
aspects of
of our
our activities
activities and could effectively
effectively adapt to its
demise
phaseout were gradual and accompanied by improved
demise only
only if
if the phaseout
improved
support for
for a totally
totally distributed
distributed computing
computing environment.
environment.
137
137
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
ONCOCIN Project
Project
ONCOCIN
IV.A.4. PROTEAN
PROTEAN Project
Project
IV.A.4.
PROTEAN Project
Project
PROTEAN
Oleg Jardetzky
Jardetzky
Oleg
Nuclear
Nuclear Magnetic
Magnetic Resonance Lab, School of
of Medicine
Medicine
Stanford
Stanford University
University
Bruce
Bruce Buchanan,
Buchanan, Ph.D.
Ph.D.
Computer
Computer Science Department
Department
Stanford
Stanford University
University
I.
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
I. SUMMARY
Project Rationale
Rationale
A. Project
goals of
of this project
project are related both
both to biochemistry
biochemistry and artificial
artificial intelligence:
intelligence: (aj
(a)
The goals
use existing AI
AI methods to aid in the determination
determination of
of the 3-dimensional
3-dimensional structure of
of
use
in solution
solution (not
(not from
from x-ray
x-ray crystallography
crystallography proteins),
proteins), and (b) use
use protein
protein
proteins in
determination as
as a test problem
problem for
for experiments with
with the AI
AI problem
problem solving
solving
structure determination
Empirical
as the Blackboard Model.
Empirical data from
from nuclear magnetic
structure known as
resonance (NMR)
(NMR) and other
other sources
sources may provide
provide enough
enough constraints
constraints on structural
structural
resonance
allow protein
protein chemists to bypass
bypass the laborious
laborious methods of
of crystallizing
crystallizing a
descriptions to allow
This problem
protein
-ray crystallography
problem
protein and using X
X-ray
crystallography tc determine its structure. This
exhibits
exhibits considerable complexity,
complexity, yet there is reason
reason to believe that AI
AI programs can be
written
written that reason
reason much as
as experts do to resolve
resolve these
these difficulties
difficulties [12].
[12].
Medical Relevance
B. Medical
The molecular structure of
of proteins
proteins is essential
essential for
for understanding many problems of
of
medicine at the molecular level, such
NMR
such as
as the mechanisms
mechanisms of
of drug action. Using NMR
data from
from proteins in solution
solution will
will allow
allow the study of
of proteins whose
whose structure cannot
will decrease
decrease the time
time needed
needed for
for the
be determined with
with other
other techniques,
techniques, and will
determination.
determination.
C. Highlights
Highlights of
of Progress
Progress
During
we have
have expanded
expanded our initial
initial prototype
prototype program, called
During the past
past year,
year, we
It is implemented
implemented in BBl
BBl (discussed
(discussed in
PROTEAN,
PROTEAN, designed
designed on the blackboard model. It
the Care
Core AI
AI Research
Research section of
of this report),
report), a framework
framework system
system for
for building
building
blackboard systems
systems that control
control their
their own problem-solving
problem-solving behavior.
The reasoning component
component of
of PROTEAN
PROTEAN directs the actions of
of the Geometry System
System
(GS), a set
set of
of programs that performs
performs the computationally
computationally intensive task of
of positioning
positioning
portions
portions of
of a molecule with
with respect
respect to each
each other in three dimensions. The GS runs in
the UNIX
UNIX environment
environment on a Silicon
Silicon Graphics IRIS 3020
3020 graphics workstation,
workstation, which
provides
provides computing
computing performance comparable to aa VAX
VAX ll1780
111780 for
for our task.
task. The
reasoning
reasoning program (in Lisp in BB1)
BBl) is coupled
coupled to the GS
GS by aa local area
area computer
network, maintained
EX.
maintained by SUM
SUMEX.
Pictures
Pictures of
of the results
results of
of GS
GS computations
computations are
are displayed
displayed on the graphics screen
screen of
of the
IRIS workstation,
workstation, using
using aa locally
locally developed
developed program called DISPLAY
DISPLAY to draw the
DISPLAY program can
can be
be
evolving protein structures at several
several levels
levels of
of detail. The DISPLAY
used
used to view structures generated
generated by the GS
GS either under the direct
direct control
control of
of the
the user
user
MIDAS and
and MMS are
are two
or as
as directed by the
the reasoning
reasoning system
system running
running in BBl.
BBl. MIDAS
E.
E. H. Shortliffe
Shortliffe
138
138
ONCOCIN Project
Project
ONCOCIN
5P41-RR00785-14
5P41-RR0078514
molecular modeling
modeling and display systems
systems to
to manipulate
manipulate protein
protein structures,
other molecular
found in
in the Protein
Protein
particularly those obtained from
from crystallographic
crystallographic techniques as
as found
particularly
in three dimensions
dimensions is essential to
ability to observe structures in
Data Bank. The ability
of the PROTEAN’
PROTEAN's
systems and
understanding the behavior of
s reasoning and geometry systems
essential insights on the problem
problem solving
solving process.
process.
provides essential
PROTEAN embodies
embodies the following
following experimental
experimental techniques for
for coping
coping with
with the
PROTEAN
complexities of
of constraint
constraint satisfaction:
complexities
problem-solver partitions
partitions each
each problem
problem into
into a network
network of
of looselyloosely1. The problem-solver
PROTEAN first
first positions
positions individual
individual pieces
pieces of
of
coupled sub-problems. PROTEAN
their immediate neighbors within
within local coordinate
coordinate systems.
systems. ItIt
structures and their
subsequently composes
composes the most constrained partial
partial solutions
solutions developed for
for
subsequently
these sub-problems in
in a complete solution
solution for
for the entire
entire protein.
protein. This
This
these
combinatorics of
of search.
search.
partitioning and composition
composition technique reduces
reduces the combinatorics
partitioning
problem-solver attempts to solve sub-problems
sub-problems and coordinate
coordinate solutions
solutions
2. The problem-solver
multiple levels
levels of
of abstraction. For
For example, PROTEAN
PROTEAN operates
operates at two
at multiple
levels of
of abstraction. At
At the “Solid”
"Solid" level, itit positions
positions elements of
of the
levels
At the
protein's
secondary structure: alpha-helices, beta-sheets,
beta-sheets, and coils. At
protein’
s secondary
"Atom" level, itit positions the protein’
protein's
individual atoms. Partial
Partial solutrons
solutions at
“Atom”
s individual
reduce the combinatorics
combinatorics of
of search
search at the lower level.
the solid level reduce
level introduce
introduce
Conversely, tightly
tightly constrained partial
partial solutions at the lower levei
Conversely,
new constraints on solid level solutions.
3. The problem-solver
problem-solver preserves
preserves the “family”
"family" of
of solutions
solutions consistent with
with all
3.
constraints applied thus far. For example,
positioning a helix
example, in positioning
helix within
within a
partial solution, PROTEAN
partial
PROTEAN does
does not attempt
attempt to identify
identify a unique spatial
position
position for
for the helix. Instead,
Instead, it
it identifies
identifies the entire
entire spatial volume within
within
which the helix might
might lie, given the constraints applied thus far. Preserving
the family
family of
of legal
legal solutions accommodates
accommodates problems with
with incomplete
incomplete
constraints; the solution
It also
also
solution is constrained only
only as
as the data indicate. It
permitting disjunctive
accommodates
accommodates incompatible
incompatible constraints by permitting
disjunctive subfamilies,
families, which may be
be necessary
necessaryfor
for flexible
flexible proteins.
4. The problem-solver
problem-solver applies constraints one at a time, successively
successively restricting
restricting
sub-problems.
the family
family of
of solutions hypothesized
hypothesized for
for different
different
PROTEAN
PROTEAN successively
successively applies
applies constraints on the positions of
of protein
protein
structures,
structures, restricting
restricting spatial volumes
volumes within
within which they may lie. This
This allows
the different
different kinds of
of constraints to be
be applied by integrating
integrating their effects
on a family
family of
of solutions.
5.
5. The problem-solver
probiem-solver tolerates
tolerates overlapping
overlapping solutions for
for different
different subproblems.
problems. For example,
example, in identifying
identifying the volume wIthin
within which structure-a
structure-a
might
might lie in partial solution I,
1, PROTEAN
PROTEAN may include
include part of
of the volume
identified
identified for
for structure-b. Overlapping volumes for
for two structures indicate
either: (a) that the
the two structures
structures actually occupy disjoint
disjoint sub-volumes that
cannot be
be distinguished within
within the larger, overlapping
overlapping volumes identified
identified for
for
them
them because
because the constraints are
are incomplete; or (b) that the two structures
are
are mobile and
and alternately occupy
occupy the
the shared
shared volume.
6.
6. The problem-solver
problem-solver reasons
reasons explicitly
explicitly about control
control of
of its own problemproblemsolving actions:
actions: which sub-problems it
it will
will attack, which partial
partial solutions it
will
Control reasoning
reasoning guides
guides
will expand,
expand, and
and which constraints it
it will
will apply. Control
the
the problem-solver
problem-solver to perform
perform actions
actions that minimize
minimize computation,
computation. while
It
maximIzmg
progress toward
toward a complete solution.
solution.
It also
also provides a:I
maximizing progress
foundation
foundation for
for the
the problem-solver's
problem-soiver’s explanatIOn
explanatron of
of problem-solving
problem-solving
l39
139
E.
E. H. Shonliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
ONCOCIN Project
ONCOCIN
for its learning
learning of
of new
actlvitles and intermediate
intermediate partial
partial solutions
solutions and for
activities
control heuristics.
control
in PROTEAN
PROTEAN allow
allow several
several sets
sets of
of knowledge to be used.
used. A
A
Multiple blackboards in
Multiple
information about proteins
proteins and secondary structures,
biochemical knowledge base
base stores
stores information
biochemical
amino acids,
acids, and atoms. A
A concept blackboard describes
describes a concept hierarchy
hierarchy of
of natural
natural
amino
types, object types,
types, role types,
types, contexts, constraint
constraint types,
types, and problem
problem solving
solving methods.
types,
The ACCORD
ACCORD language
language blackboard explicitly
explicitly represents
represents the actions that
that can be taken in
in
The
language for
for arrangement assembly
assembly problems. The problem
problem blackboard describes
describes the
the language
for the molecule. Finally,
Finally, the
protein to be solved and all
all experimental data observed
observed for
protein
of the protein
protein structure is built
built on a third
third solution
solution blackboard.
evolving solution
solution of
evolving
PROTEAN determines the structure of
of a protein
protein by assembling the protein
protein from
from
PROTEAN
several levels of
of detail. Initially,
Initially, the major
major secondary
secondary structures of
of the
components at several
as solid
solid structures,
protein are positioned relative
relative to each
each other by considering
considering them as
protein
of the amino acids
acids and representing constraints with
with respect to
ignoring the side chains of
ignoring
This solid
of the protein
protein backbone.
backbone.
solid Zevef
level approximation
approximation is sufficient
sufficient to
atoms of
of the molecule, but
but leaves
leaves details of
of the structure
shape of
determine the overall shape
atomic level
level representation of
of the protein
protein including
including side chains
indistinct. Second,
indistinct.
Second, an atomic
used with
with more precise
precise distance,
distance, bond length, and bond angle
angle constraints
constraints to remove
is used
The atomic
atomic level
chemically infeasible
infeasible structures generated
generated at the solid level.
chemically
of the structure, at the cost of
of larger
description allows a more detailed description
description of
description
of components to consider and increased
increased computation
computation time.
numbers of
component of
of PROTEAN
PROTEAN includes domain
domain and control
control knowledge
The reasoning component
for the assembly
assembly of
of a protein.
protein. Each domain knowledge
knowledge source
source directs a small
sources for
sources
These knowledge
knowledge sources
sources develop partial
partial
portion
portion of
of the construction
construction of
of the molecule. These
solutions that position
position alpha helices,
helices, beta
beta strands,
strands, and coils at the solid
solid level
level and refine
refine
Control knowledge
the resulting state
state families
families using
usin,0 all available distance constraints. Control
sources
sources determine which of
of the possible
possible assembly
assembly actions is the best
best to perform
perform at each
each
stage
stage of
of the problem solving.
We have
have built
built a first
first extension to PROTEAN
PROTEAN that assembles
assembles a protein
protein at the level of
of
the atomic
atomic backbone.
backbone. The facilities
facilities available include programs to manipulate
manipulate protein
protein
data bank files and generate
generate test
test data
data automatically,
automatically, use
use atomic
atomic level constraints
constraints to
prune solid level solutions, generate
generate example instances
instances of
of the protein
protein backbone from
from
generate candidate structures for
for unstructured coil
coil
the solid level structures,
structures, and generate
segments
segments of
of a protein. Work
Work is in progress
progress to combine
combine the atomic
atomic level of
of assembly
assembly
with
with the solid level to provide additional
additional constraints at the more abstract level of
ot
assembly.
assembly.
The PROTEAN
PROTEAN system
system has
has been
been used
used to construct a complete solution
solution at the solid level
level
of
of detail for
for the Lac-repressor headpiece,
headpiece, a protein
protein with
with fifty-one
fifty-one amino
amino acids
acids
consisting of
of four
four coil sections
sections and
and three
three alpha helices.
helices. In this work, the constraints
were
were determined experimentally
experimentally from
from NMR
NMR studies.
studies.
we have
have applied PROTEAN
PROTEAN to
In addition
addition to the Lac-repressor headpiece
headpiece protein, we
sperm whale
whale myoglobin,
myoglobin, T4 lysozyme,
lysozyme, and cytochrome B.
B. Each
Each of
of these
these latter
latter proteins
each case,
case, we
we extracted
extracted features
features of
of the protem
protein
has
has a known crystal structure. In each
structure and
and distance
distance constraints from
from the crystal structure to build
build data
data sets
sets for
for
PROTEAN.
We then
PROTEAN.
then applied the
the PROTEAN
PROTEAN system
system to the resulting data
data sets
sets to
determine the
the behavior of
of the system
system with different
different kinds of input.
To determine the correctness
correctness and
and capabilities of
of the
the PROTEAN
PROTEAN method, we
we have
have
applied PROTEAN
PROTEAN to sperm
sperm whale
whale myoglobin, a molecule whose
whose crystal structure isis
known. In this test,
test, we
we used
used distance
distance constraints that would be
be measured
measured as
as NOEs,
NOES,
overall size
size information,
information, and
and the
the interaction between
between the heme
heme group and the amino
acids.
acids. We also
also sys[ematically
systematically explored
explored the
the dependence
dependence of the
the precision and accuracy
accuracy Jf
a,I‘
E.
E. H.
H. Shortliffe
Shortliffe
140
140
ONCOCIN
ONCOCIN Project
Project
SP41-RR0078S-14
5P41-RR00785-14
quality of
of the input
input data available. In
In all
all cases,
cases, the solutions
solutions
solutions on the quality
the solutions
from PROTEAN
PROTEAN enclose
enclose the actual structure of
of the molecule, with
with the best
obtained from
obtained
coming from
from data that
that includes many short
short range constraints.
results coming
for structures such
such as
as the heme group in
in
defined representations for
We have also defined
myoglobin and other
other cofactors
cofactors that
that can be used
used in constraint
constraint satisfaction
satisfaction operations
operations to
myoglobin
further restrict
restrict the positions
positions of
of the secondary
secondary structures in
in the protein.
protein.
further
PROTEAN system
system takes
takes the secondary
secondary structure as
as input.
input. For
For molecules in
in
The PROTEAN
of the helical, sheet,
sheet, and unstructured coil
coil segments
segments of
of a protein
protein is
sol ution, the extent
extent of
solution,
from NMR
NMR data between
between backbone and side chain hydrogen atoms. We
largely from
derived largely
uses heuristic
heuristic knowledge and
knowledge-based system
system called ABC that uses
have developed a knowledge-based
NMR data to automate this important
important step in
in protein
protein structure determination.
determination. ABC
ABC is
NMR
In
implemented using the BBl
BBl blackboard architecture.
In addition
addition to solving
solving the
implemented
structure classification
classification problem,
problem, ABC
ABC provides a flexible
flexible and extensible
secondary structure
framework for
for experimenting
experimenting with
with identification
identification methods
methods for
for secondary
secondary structures as
as
framework
as for
for data interpretation
interpretation and pattern recognition
recognition techniques.
techniques.
well as
Work is proceeding on several
several aspects
aspects of
of the protein
protein structure problem, including
including
Work
of several
several partial
partial arrangements
arrangements and integration
integration of
of these
these pieces
pieces of
of solution
solution into
into
assembly of
assembly
atomic level volume exclusion of
of atoms and information
information on
larger structures, using atomic
precise atomic level solutions, and developing more
sidechain packing to produce more precise
appropriate representations for
for unstructured coil
coil sections
sections of
of proteins.
appropriate
D. Relevant
Relevant Publications
Publications
D.
1.
for the determination
1. Altman,
Altman, R. and Jardetzky, 0.:
0.: New strategies
strategies for
determination of
of
macromolecular
macromolecular structures
structures in solution.
solution. Journal of
of Biochemistry
Biochemistry (Tokyo),
(Tokyo),
Vol.
No.6,
Vol. 100,
100, No.
6, p. 1403-1423,
1403-1423, 1986.
1986.
2.
Partial Compilation
Knowledge.
Compilation of
of Control
Control Knowledge.
2. Altman,
Altman, R. and Buchanan,
Buchanan, B.G.:
B.G.: Partial
To appear
appear in Proceedings
Proceedings of
of the AAAI
AAAI 1987.
1987.
3.
3. Brinkley,
Brinkley, J.,
J., Cornelius.
Cornelius, c..
C., Altman,
Altman, R., Hayes-Roth, B.
B. Lichtarge, 0.,
O., Duncan,
B.,
Application of
B., Buchanan,
Buchanan, B.G.,
B.G., Jardetzky, 0.:
0.: Application
of Constraint
Constraint Satisfaction
Satisfaction
Techniques
Determination of
Protein Tertiary
Techniques to the Determination
of Protein
Tertiary Structure.
Structure. Report
Report
KSL-86-28,
KSL-86-28, Department
Department of
of Computer Science,
Science, 1986.
1986.
4.
4. Brinkley,
Brinkley, James
James F., Buchanan,
Buchanan, Bruce
Bruce G., Altman,
Altman, Russ
Russ B.,
B., Duncan, Bruce
Bruce S.,
S.,
Cornelius.
for Spatial
Heuristic Refinement
Refinement Method
Method for
Spatial Constraint
Constraint
Cornelius, Craig W.: A Heuristic
Satisfaction
Satisfaction Problems.
Problems. Report KSL 87-0S,
87-05, Department of
of Computer Science.
Science.
S.
O., Altman,
Altman, A., Brinkley,
Brinkley, J.,
J.,
5. Buchanan,
Buchanan, B.G., Hayes-Roth, B.,
B., Lichtarge, 0.,
Hewett, M., Cornelius, C.,
C., Duncan, B.,
B., Jardetzky,
Jardetzky, O.:The Heuristic
Heuristic Refinement
Refinement
Method
for Deriving
Method for
Deriving Solution
Solution Structures
Structures of
of Proteins.
Proteins. Report KSL-8S-41.
KSL-85-41.
October 1985.
1985.
6.
6. Garvey, Alan, Cornelius, Craig, and Hayes-Roth, Barbara:
Barbara: Computational
Computational
Costs versus Benefits
Benefits of
of’ Control
Control Reasoning. Report KSL 87-11, Department
Department
of Computer
Computer Science.
Science.
7.
for
Blackboard Architecture:
Architecture: A General Framework
Framework for
7. Hayes-Roth, B.:
B.: The Blackboard
Problem
of Computer Science.
Science,
Problem Solving?
Solving ? Report HPP-83-30, Department of
Stanford University,
University, 1983.
1983.
Environment for
Building Blackboard
Blackboard Systems
Svstems
8.
An Environment
for Building
8. Hayes-Roth, B.:
B.: BBl:
BBI:
that
Learn about their
their own
own Behavior.
Behavior. Report
that Control,
Control, Explain,
Explain, and Learn
HPP-84-16,
HPP-84-16, Department
Department of Computer Science,
Science, Stanford University,
University, 1904.
1984.
141
141
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
ONCOCIN
ONCOCIN Project
9. Hayes-Roth,
Blackboard
Hayes-Roth, B.: A Blackboard
Intelligence 26:251-321,
26:251-321, 1985.
1985.
Intelligence
Architecture
Architecture
for
for
Control.
Control.
Artificial
Artificial
Learning Control
Heuristics in BBI.
BBl. Report
10.
Control Heuristics
Report
10. Hayes-Roth,
Hayes-Roth, B. and Hewett, M.: Learning
HPP-85-2,
HPP-85-2, Department
Department of
of Computer
Computer Science,
Science, 1985.
1985.
11.
O., Hewett, M., Altman,
Altman, R.,
11. Hayes-Roth,
Hayes-Roth, B.,
B., Buchanan, B.G., Lichtarge, 0.,
PROTEAN:
PROTEAN:
Brinkley,
Brinkley, 1.,
J., Cornelius, C., Duncan, B.,
B., and Jardetzky, 0.:
0.:
from constraints.
Deriving protein
protein structure
Deriving
structure from
constraints. Proceedings
Proceedings of
of the AAAI,
AAAI, 1986,
1986,
p. 904-909.
Method for
for the Definition
Definition of
12.
12. Jardetzky, 0.:
0.: A Method
of the Solution
Solution Structure
Structure of
of
Proteins from
from NMR
NMR and Other
Physical Measurements:
Measurements: The LAC-Repressor
LAC-Repressor
Proteins
Other Physical
Headpiece. Proceedings
Headpiece.
Proceedings of
of the International
International Conference on the Frontiers
Frontiers of
of
Biochemistry
Biochemistry and Molecular
Molecular Biology, Alma
Alma Alta,
Alta, June 17-24, 1984,
1984, October,
1984.
proteins in solution
NMR.
13.
13. Lichtarge,
Lichtarge, Olivier:
Olivier: Structure
Structure determination
determination of
of proteins
solution by NMR.
Ph.D. Thesis, Stanford
Stanford University,
University, November, 1986.
1986.
14.
14. Lichtarge,
Lichtarge, Olivier,
Olivier, Cornelius,
Cornelius, Craig W., Buchanan,
Buchanan, Bruce G., Jardetzky,
Jardetzky, Oleg:
Oleg:
Validation
First Step of
Heuristic Refinement
Refinement Method
Method for
for the
Validation of
of the First
of the Heuristic
Proteins from
from NMR
NMR Data.,
Data., April
Derivation of
April 1987.
1987.
Derivation
of Solution
Solution Structures
Structures of
of Proteins
Submitted to Proteins: Structure, Function,
Function, and Genetics.
Genetics.
E. Funding
Funding Support
Support
Title:
NMR Data from
Title: Interpretation
Interpretation of
of NMR
from Proteins Using AI
AI Methods
PI's:
PI’s: Oleg Jardetzky and Bruce G. Buchanan
Buchanan
Agency: National
National Science
Science Foundation
Foundation
Grant
Grant identification
identification number: DMB-8402348
DMB-8402348
2/l/87 - 9/30/89
9/30/89
2/1/87
Total
Total Award
Award Period and Amount:
Amount:
(includes direct
direct and indirect
indirect costs)
costs)
$120,000
$120,000
2/l/87 - 9/30/89
9/30/89
2/1/87
Current
Current award period and amount:
(includes direct
direct and indirect
indirect costs)
costs)
$120,000
$120,000
following grants and contracts each
each provide
funding for
for
The following
provide partial
partial funding
PROTEAN personnel.
PROTEAN
Title:
Title: Modeling
.Modeling Exper
Exper Control
Control
PI: Bruce G. Buchanan
Buchanan
Office of
of Naval Research
Research
Agency: Office
Grant
Number: ONR
N00014-86-K -0652
Grant Identification
Identification Number:
ONR N00014-86-K-0652
Total
Total award period and amount:
(direct
(direct and indirect)
indirect)
Shortliffe
E. H. Shortliffe
6/1/85
6/l/85 - 5/31/85,
5/31/85, $96,879
$96,879
142
ONCOCIN
ONCOCIN Project
Project
5P41-RR00785-14
5P41-RR00785-14
Current award period
period and amount:
amount: 6/l/85
6/1/85 - 5/31/85,
5/31/85, $96,879
Current
(direct and indirect)
indirect)
(direct
PROTEAN component
component is $48,440 (direct
(direct &
& indirect)
indirect) or
or 50% of
of grant
grant
PROTEAN
Title: Research on Blackboard
Blackboard Problem-Solving
Problem-Solving Systems
Systems
Title:
PI's:
Edward A.
A. Feigenbaum
Feigenbaum and Bruce G. Buchanan
PI’
s: Edward
Computer Services Corporation
Corporation
Boeing Computer
Agency: Boeing
Grant identification
identification number:
number: W-271799
W-271799
Grant
Total award period
period and amount:
amount:
Total
indirect)
(direct and indirect)
(direct
8/l/86
8/1/86 - 7/31/87,
7/31/87, $245,432
$245,432
Current award period
period and amount:
amount: 8/1/86
7/31/87, $245,432
$245,432
Current
8/l/86 - 7/31/87,
indirect)
(direct and indirect)
(direct
PROTEAN component
component is $12,730
$12,730 (direct
(direct & indirect)
indirect) or 5% of
of grant
grant
PROTEAN
Title: Knowledge-Based
Knowledge-Based Systems
Systems Research
Research
Title:
PI: Edward
Edward A. Feigenbaum
Feigenbaum
Research Agency
Agency: Defense Advanced Projects Research
Grant
N00039-86-0033
Grant identification
identification number:
number: N00039-86-0033
Total
Total award period
period and amount:
amount:
(direct
(direct and indirect)
indirect)
10/l/85 - 9/30/88
9/30/88 $4,130,230
$4,130,230 (in
(in negotiation)
negotiation)
10/1/85
Current
Current award period
period and amount:
amount: 10/1/86
10/l/86 - 9130/87
9/30/87 $1,549,539
$1549,539
(direct
(direct and indirect)
indirect)
1.9 % of
of grant
grant total
total
PROTEAN
PROTEAN component
component is $29031,
$29031, or 1.9
II.
WITH
RESOURCE
II. INTERACTIONS
INTERACTIONS
WITH THE
THE SUMEX-AIM
SUMEX-AIM
RESOURCE
A. Medical
Medical Collaborations
Collaborations
Several
Several members of
of Prof.
Prof. Jardetzky's
Jardetzky’s research
research group
group are involved
involved in this research.
B. Interactions
EX-AIM projects
projects
Interactions with
with other
other SUM
SUMEX-AIM
We are occasionally
occasionally in contact with
with researchers
researchers at Robert
Robert Langridge's
Langridge’s laboratory
laboratory at the
Un
i versi ty of
University
of San
San Francisco.
anagernent
C. Critique
Critique of
of Resource
Resource M
Management
The SUMEX
SUMEX staff
staff has
has continued
continued to be most cooperative 10
in supporting
supporting PROTEA~
PROTEAN
research.
research. The SUMEX
SUMEX computer
computer facility
facility is well maintained
maintarned and managed
managed for
for effective
effective
support
support of
of our work. The computer
computer network
network and Lisp workstations
workstations are supported very
effectively
effectively by the SUMEX
SUMEX staff.
143
143
E. H. Shorttiffe
Shortliffe
E.
5P4l-RR00785-l4
5P41-RR00785-14
ONCOCIN ,Project
ONCOCIN
Project
III. RESEARCH
RESEARCH PLANS
PLANS
III.
Plans
A. Goals
Goals & Plans
Our long-range
long-range goal is to build
build an automatic
automatic interpretation
interpretation system
system similar
similar to
CRYSALIS
CRYSALIS (which
(which worked with
with x-ray
x-ray crystallography
crystallography data). In
In the shorter term, we are
building
NMR data on small
building interactive
interactive programs that
that aid in the interpretation
interpretation of
of NMR
proteins. The current
current version of
of PROTEAN
PROTEAN has
has domain
domain and control
control knowledge sources
sources
that implement
implement the reasoning techniques
techniques described
described above
above to build
build a solution
solution using a
These knowledge sources
sources develop partial
partial solutions
solutions
dynamically
dynamically created
created strategic plan. These
that position
position multiple
multiple alpha helices,
helices, coils, and beta structures at the Solid level and
refine
refine those
those helices
helices using distance, surface,
surface, and volume constraints.
PROTEAN
PROTEAN also
also includes programs that use
use atomic
atomic level representations of
of the amino
amino
acid backbone and side chains. These
These routines
routines use
use more precise
precise atomic
atomic level distance
constraints
constraints to prune the solutions
solutions obtained by the more abstract solid level geometry
computations.
computations. Programs are also available to find
find acceptable
acceptable backbone segments
segments for
for
unstructured coil
coil segments
segments between
between alpha helices
helices and beta structures.
structures.
The proposed research
research would expand PROTEAN
PROTEAN to include
include knowledge
knowledge sources
sources that:
1.
1. merge highly
highly constrained partial
partial solutions at the Solid level.
2.
2. propagate
propagate emergent constraints
constraints at the atomic
atomic level back up to the solid level
to further
further restrict
restrict the relative
relative positions
positions of
of superordinate helices,
helices, beta sheets,
sheets,
and coils.
3.
3. further
further restrict
restrict the relative
relative locations of
of atoms relative
relative to one another.
another
4. select
select instances
instances of
of structures to be used
used as
as starting
starting points for
for other kinds of
of
refinement
refinement procedures,
procedures, such
such as
as the solution
solution of
of the Bloch equations,
equations, which
define
NMR spectrum that can possibly arise from
define the NMR
from a given structure.
These
These equations provide
provide a very strong test
test of
of the correctness
correctness of
of our method,
proposed structures.
as
as well as
as providing
providing an additional
additional constraint
constraint on proposed
5. develop efficient
efficient and effective
effective control
control strategies
strategies for
for
5.
intermediate
intermediate and large molecules.
molecules.
solution
the solution
of
of
6.
6. reason
reason about mobility
mobility of
of structures when
when the data indicate
indicate that mobility
mobility is
possible.
possible.
We have
have built
built an effective
effective strategy for
for automatically
automatically determining
determining the families
families of
of solid
level solutions
solutions for
for small proteins, such
such as
as the Lac-repressor headpiece.
headpiece. We will
will extend
the current
current work
work to develop control
control strategies
strategies to guide PROTEAN's
PROTEAN’s constraint
constraint
satisfaction
protein
satisfaction in medium
medium and large protein
protein to identify
identify the family
family of
of legal
!egal protein
conformations
conformations as
as efficiently
efficiently as
as possible.
possible.
Justification for
for continued
B. Justification
continued SUMEX
SUMEX use
We will
will continue
continue to use
use SUMEX
SUMEX for
for developing parts of
of the program before integrating
integrating
them with
with the whole system.
system. We are using Interlisp
Interlisp to implement
implement PROTEAN
PROTEAN within
within
addition, the local area
area network
network that
the Blackboard model flexibly
flexibly and quickly.
quickly. In addition,
SUMEX maintains
maintains is crucial
crucial to the communications
communications between
between our reasoning system
system in
SUMEX
BBl,
BBl, running
running on Xerox Lisp
Lisp machines,
machines, and our geometry programs and display systems,
systems,
running
running on the IRIS 3020
3020 workstation.
workstation.
Shortliffe
E. H. Shortliffe
144
144
ONCOCIN
ONCOCIN Project
Project
5P41-RR00785-14
5P41-RR00785-14
c. Need
Need for
for other
other computing
computing resources
resources
C.
At this
this time
time our
our computational
computational resources
resources are almost
almost adequate.
adequate. However, access
access to
to Lisp
Lisp
At
for program
program development
development is often
often a limiting
limiting factor
factor in
in our
our ability
ability to
to continue
continue
machines for
research. In
In addition,
addition, faster
faster computation
computation of
of the operations of
of the GS would
would be
the research.
facilitated by
by a special-purpose
special-purpose array
array processor
processor or
or an additional
additional workstation
workstation for
for
facilitated
computing.
computing.
145
145
E. H.
H. Shortliffe
Shortliffe
E.
SP41-RR0078S-14
5P41-RR00785-14
RADIX
RADIX Project
IV.A.5.
IV.A.5. RADIX
RADIX Project
Project
The RADIX
RADIX Project:
Project: Deriving
Deriving Medical
Medical Knowledge from
from
Time-Oriented
Time-Oriented Clinical
Clinical Databases
Databases
Robert
Robert L. Blum,
Blum, M.D.,
M.D., Ph.D.
Ph.D.
Department
Department of
of Computer
Computer Science
Stanford
Stanford University
University
Gio
Gio C. M.
M. Wiederhold,
Wiederhold, Ph.D.
Ph.D.
Departments
Departments of
of Computer
Computer Science and Medicine
Medicine
Stanford
Stanford University
University
I.
I.
SUMMARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
Introduction
A. Technical
Technical Goals - Introduction
Medical and Computer
of the RADIX
RADIX project are
are 1)
Computer Science
Science Goals --- The objectives of
Discovery: to provide
provide knowledgeable assistance
assistance to a research
research investigator
investigator in studying
process of
medical hypothese:;
hypotheses on large databases,
databases, and to automate the process
of hypothesis
hypothesis
generation and exploratory
exploratory confirmation,
confirmation, 2) Summarization:
Summarization: to develop
develop a program and
set of
patient records,
of techniques for
for automated summarization
summarization of
of patient
records, and 3) Peer
Peer Review:
Review:
to develop a program to assist
assist physician reviewers examine case
case databases
databases for
for medical
peer
For system
system development we
we have
have used
used a subset
subset of
of
peer review and quality
quality assurance.
assurance. For
the ARAMIS
ARAMIS database.
database. We will
will first
first describe our work
work on discovery, followed
followed by
summarization
summarization and peer review.
RADIX
RADIX Discovery
Discovery Module
Module
Computerized
been under
Computerized clinical
clinical databases
databases and automated medical records
records systems
systems have
have been
development throughout
throughout the world
world for
for at least a decade.
decade. Among the earliest of
of these
these
endeavors
endeavors was
was the ARAMIS
ARAMIS Project, (American
(American Rheumatism Association Medical
Information
Information System)
System) under development since 1969
1969 in the Stanford
Stanford Department
Department of
of
Medicine.
ARAMIS
ARAMIS contains records
records of
of over 17,000
17,000 patients with
with a variety of
of
Medicine.
rheumatologic
patient visits have
been recorded,
rheumatologic diagnoses.
diagnoses. Over 62,000
62,000 patient
have been
recorded, accounting for
for
50,000
been generalized
50,000 patient-years
patient-years of
of observation. The ARAMIS
ARAMIS Project has
has now been
generalized to
include
include databases
databases for
for many chronic
chronic diseases
diseasesother than arthritis.
arthritis.
The fundamental
fundamental objective
objective of
of the ARAMIS
ARAMIS Project and many other clinical
clinical database
database
projects is to use
use the data that have
have been
been gathered
gathered by clinical
clinical observation in order to
study the evolution
Unfortunately, the
evolution and medical management
management of
of chronic
chronic diseases.
diseases. Unfortunately,
process
process of
of reliably
reliably deriving
deriving knowledge has
has proven to be exceedingly
exceedingly difficult.
difficult.
Numerous problems arise stemming from
from the complexity
complexity of
of disease,
disease, therapy, and
outcome definitions,
definitions, from
from the complexity
complexity of
of causal
causal relationships, from
from errors
A major
major objective
introduced
introduced by bias, and from
from frequently
frequently missing and outlying
outlying data.
data. A
of
of the RADIX
RADIX Project is to explore the utility
utility of
of symbolic
symbolic computational
computational methods
methods and
knowledge-based
knowledge-based techniques
techniques at solving
solving some
some of
of these
these problems.
The RADIX
RADIX computer
computer program is designed
designed to examine a time-oriented
time-oriented clinical
clinical database
database
such
such as
as ARAMIS
ARAMIS and to produce a set
set of
of (possibly) causal
causal relationships. The algorithm
algorithm
exploits
exploits three properties
properties of
of causal
causal relationships: time
time precedence,
precedence, correlation,
correlation, and
parametric correlations
nonspuriousness.
First, a Discovery Module
Module uses lagged,
lagged, non
nonparametric
correlations to
nonspuriousness. First,
generate
Second, a Study Module uses
uses a
generate an
an ordered .lis·t
list of
of tentative relationships. Second,
E. H. Shortliffe
Shortliffe
146
146
RADIX Project
RADIX
5P41-RR00785-14
5P41-RR00785-14
of medicine
medicine and statistics
statistics to try
try to establish nonspuriousness
nonspuriousness by
base (KB)
(KB) of
knowledge base
controlling for
for known
known confounders.
confounders.
controlling
principal innovations
innovations of
of RADIX
RADIX are the Study Module
Module and the KB. The Study
The principal
Module takes
takes a causal
causal hypothesis obtained
obtained from
from the Discovery
Discovery Module
Module and produces
produces a
Module
from the KB.
KB. The study design is then
comprehensive study design, using knowledge from
executed by an on-line
on-line statistical
statistical package,
package, and the results are automatically
automatically incorporated
incorporated
executed
into the KB. Each new causal
causal relationship
relationship is incorporated
incorporated as
as a machine-readable
machine-readable record
into
distribution across
across patients, functional
functional form,
form, clinical
clinical setting,
specifying its intensity,
intensity, distribution
specifying
validity, and evidence.
evidence. In
In determining
determining the confounders
confounders of
of a new hypothesis the Study
validity,
Module uses
uses previously
previously “learned”
"learned" causal
causal relationships.
relationships.
Module
In creating a study design the Study Module
Module follows
follows accepted
accepted principles
principles of
of
In
epidemiological research.
research.
feasibility and study design:
design: crossItIt determines study feasibility
epidemiological
uses the KB
KB to determine
determine the confounders
confounders of
of a given
versus longitudinal.
longitudinal. ItIt uses
sectional versus
hypothesis, and itit selects
selects methods for
for controlling
controlling their
their influence:
influence: elimination
elimination of
of patient
patient
hypothesis,
elimination of
of confounding
confounding time
time intervals,
intervals, or statistical
statistical control.
control. The Study
records, elimination
records,
appropriate statistical
statistical method, using knowledge stored as
as
Module then determines an appropriate
Module
production rules.
rules. Most
Most studies have used
used a longitudinal
longitudinal design
design involving
involving a multiple
multiple
production
regression model applied
applied to individual
individual patient
patient records.
records. Results across
across patients are
regression
based on the precision of
of the estimated regression
regression coefficient
coefficient
combined using weights based
for each
each patient.
for
More recently, we have undertaken a new component
component to the RADIX
RADIX program: a
More
of the knowledge-based
knowledge-based discovery module
knowledge-based
knowledgebased discovery module. The goal of
of the limitations
limitations of
of the original,
original, statistics-based, RX
RX discovery
is to overcome some of
notions of
of
module. In creating disease
disease hypotheses,
hypotheses, researchers
researchers make extensive use
use of
of notions
causation, mechanism of
of action,
action, tempo, :ld
:jd quantitative
quantitative sufficiency,
sufficiency, as
as well as
as detailed
knowledge of
process of
of pathophysiology. We are :lESS
AGNESS or are modeled
modeied ~lfter
Shortliffe
E. H. Shortliffe
174
174
5P41-RR00785-14
5P41-RR00785-14
SOL VER Project
Project
SOLVER
computation network
network rather
rather than
than aa production
production rule
rule base
base and
and supports
supports values
values of
of any
any
computation
well-defined data
data type,
type, the
the Merit
Merit questioning
questioning scheme,
scheme, an
an explanation
explanation facility,
facility, and
and
well-defined
inference methods.
methods. The
The first
first major
major application
application of
of AGNESS
AGNESS was
was to
to
expert-defined inference
expert-defined
implement the
the clinical
clinical expert
expert system
system ETA
ETA (Exercise
(Exercise Test
Test Analyzer).
Analyzer). The
The cases
cases studied
studied
implement
from the
the Program
Program on
on the
the Surgical
Surgical Control
Control of
of the
the Hyperlipidemas
Hyperlipidemas (POSCH),
(POSCH), aa
came from
came
of
the
effect
of
reduced
cholesterol
on
heart
attack
victims.
study
study of the effect of reduced cholesterol on heart attack victims.
Kent Spackman,
Spackman, M.D.
M.D. was
was aa post-doctoral
post-doctoral fellow
fellow in
in medical
medical informatics
informatics at
at the
the
Kent
of Minnesota
Minnesota who
who is
is completing
completing aa Ph.D.
Ph.D. thesis
thesis in
in Artificial
Artificial Intelligence
Intelligence at
at
University of
University
the University
University of
of Illinois.
Illinois. During
During his
his residency
residency at
at the
the University
University of
of Minnesota
Minnesota
the
Hospitals, Dr.
Dr. Spackman
Spackman collaborated
collaborated with
with the
the SOLVER
SOLVER project.
project. Dr.
Dr. Spackman’
Spackman'ss
Hospitals,
for
medical
expert
systems.
research
addressed
issues
in
automated
knowledge
acquisition
research addressed issues in
knowledge acquisition for medical expert systems.
C. Highlights
Highlights of
of Research
Research Progress
Progress
C.
of This
This Past
Past Year
Year
Accomplishments of
Accomplishments
Dr. Connelly
Connelly has continued
continued supervising
supervising the development
development of
of an expert
expert system, ESPRE, to
to
Dr.
monitoring requests
requests for
for platelet
platelet transfusions.
transfusions. The
The prototype
prototype knowledge
knowledge base
base
be used
used in
in monitoring
was refined
refined and extended, communications
communications protocols
protocols to communicate
communicate with
with laboratory
laboratory
was
computer systems
systems have been improved,
improved, a standin,
standing0 order
order feature
feature has been implemented,
implemented,
computer
the inference
inference engine has
has been modified,
modified, and a preliminary
preliminary evaluation
evaluation has
has been
been
completed. In
In the evaluation,
evaluation, 68 transfusion
transfusion requests
requests were processed
processed by the system.
system. In
In
of the cases,
cases, the expert system
system agreed
agreed with
with the blood
blood bank decision to
more than 80% of
of the remaining
remaining cases,
cases, the expert
expert system
system declined to propose
In six of
transfuse platelets. In
because there was
was no recent platelet
platelet count
count available to it.
it. In
In four
four cases,
cases,
a decision because
additional clinical
clinical factors
factors known
known to the blood
blood bank physician were brought
brought to bear, and
additional
criteria had not
not been
been
even though usual
usual transfusion
transfusion criteria
the transfusions were authorized even
placed in parallel operation
operation in
in the blood
blood bank to be
met. The expert system
system is being placed
used as
as a consultation
consultation tool.
used
In addition,
with detection of
of deviations
project dealing with
addition, Dr. Connelly
Connelly is supervising
supervisin,0 the project
in time series
implementation of
of a
This project
project involves the implementation
series by the human observer. This
number of
reader. During
During the
of small expert systems
systems used
used in modeling
modeling the human graph reader.
past
differences in
past year
year the work
work has
has been
been extended
extended by examining
examining individual
individual observer differences
performance
and
approach
to
graph
reading.
Time
deviation
detection
deviation
performance
Time trend graphs
graphs
representing monthly
monthly monitoring
monitoring of
of serum
serum carcinoembryonic
carcinoembryonic antigen (CEA)
(CEA) levels
levels in
simulated patients with
with surgically-removed
surgically-removed breast cancer
cancer were
were presented
presented to twelve
clinical
clinical laboratory
laboratory observers
observers and
and a time series
series analysis
analysis (TSA)
(TSA) routine
routine which is based
based on
The
observers
described
their
rationale
in
assigning
a
a homeostatic model.
model.
observers described their
assigning level of
of
suspicion regarding the presence
presence of an
an important
important deviation
deviation as
as the observation points
were
were serially revealed
revealed to them. The verbalization
verbalization reports were
were analyzed
analyzed to develop
develop rules
rules
that described
Strategies were
were compared
compared for
for
described each
each reader's
reader’s graph reading strategy.
strategy. Strategies
commonality
commonality and
and difference.
difference. Rules
Rules obtained from
from the
the top two observers
observers were
were merged
merged
into aa common rule base
base and
and an
an expert system
system implemented. A second
second expert system
system
was
was constructed
constructed by merging consistent rules
rules from
from all observers.
observers. The deviation
deviation detection
performance of
of all
all three
three approaches
approaches (TSA,
(TSA, observers,
observers, and
and both expert systems)
systems) are
are being
being
compared.
compared. The
The analysis
analysis isis currently
currently in
in progress.
progress.
Dr.
Dr. lohnson's
Johnson’s research
research group
group has
has developed
developed an
an expert
expert system
system inference engine
engine called
called
"Cleric."
“Cleric.” Cleric
Cleric isis aa rule
rule based
based language
language written
written in
in Common
Common Lisp
Lisp which
which resembles
resembles aa
forward
forward chaining
chaining production
production system.
system. Cleric
Cleric has
has been
been written
written to
to investigate
investigate diagnostic
diagnostic
from
simple
production
systems
problem
solving
tasks.
Cleric
differs
problem solving tasks. Cleric differs from simple production systems because
because itit can
can
dynamically
dynamically create
create new
new specialized
specialized forms
forms of
of existing
existing rules
rules for
for later
later execution.
execution. In
In
addition,
de Kleer's
Kleer’s assumption
assumption based
based truth
truth maintenance
maintenance system
system
addition, Cleric
Cleric uses
uses aa subset
subset of
of de
(ATMS).
(ATMS). A
A computer
computer hardware
hardware diagnosis
diagnosis expert
expert system
system called
called "Vesalius"
“Vesalius” has
has been
been
implemented
implemented in
in Cleric.
Cleric.
175
175
E.
E. H.
H. Shortliffe
Shortliffe
SOLVER Project
Project
SOLVER
5P4l-RR00785-l4
5P41-RR0078514
The ETA
ETA (Exercise
(Exercise Test
Test Analyzer)
Analyzer) expert
expert system
system has been
been implemented
implemented and
and tested.
tested. The
The
The
of the
the patient’
patient's
heart, as measured
measured by
by treadmill
treadmill ECG
ECG tests, between
between
change in
in the
the health
health of
change
s heart,
any two
two tests was rated
rated on
on a seven-point
seven-point scale; each subject
subject was rated
rated on
on several
several
any
for sub-area
sub-area ratings
ratings were
were built
built from
from the
the verbal
verbal protocols
protocols of
of
features and
and overall.
overall. Rules
Rules for
features
POSCH cardiologist,
cardiologist, and
and then
then weighrings
weightings for
for combining
combining sub-area
sub-area ratings
ratings into
into an
a POSCH
on 100 cases from
from the
the POSCH
POSCH study
study
overall rating
rating were
were determined.
determined. ETA
ETA was tested on
overall
and outperformed
outperformed both
both the
the average POSCH
POSCH cardiologist
cardiologist and
and a previously
previously developed
developed
and
multiple regression
regression model.
model.
multiple
of Serial
Serial Coronary
Coronary Angiograms”)
Angiograms")
In the
the past
past year,
year, the
the expert
expert system
system ESCA
ESCA (“Evaluator
("Evaluator of
In
developed with
with domain
domain knowledge
knowledge organized
organized in
in an inference
inference network
network modeled
modeled
has been developed
after that
that of
of AGNESS.
AGNESS. The
The domain
domain knowledge
knowledge was gathered
gathered from
from verbal
verbal protocols
protocols of
of a
after
inferring changes
changes in
in atherosclerotic
atherosclerotic disease from
from changes in
in the
the flow
flow
POSCH member
member inferring
POSCH
of blood
blood as revealed
revealed in
in angiograms
angiograms taken
taken at different
different times.
times. In
In some
some cases,
cases, the
the POSCH
POSCH
of
first asked to
to determine
determine the
the change
change solely
solely from
from a form
form recording
recording the
the
member was first
member
consensus of
of a two-member
two-member sub-panel,
then was shown
more detailed
detailed and
and less
consensus
sub-panel, and then
shown a more
and allowed
allowed to
to modify
modify his conclusion.
conclusion. AA sub-panel
sub-panel working
working from
from the
the
stylized
diagram and
stylized diagram
films was also observed
observed so the
the influence
influence of
of the perceptual
perceptual component
component could
could be judged.
judged.
films
of ESCA’
ESCA's
factoring the
the domain
domain into
into a perceptual
perceptual
Indeed, much
much of
Indeed,
s success is due to factoring
component followed
followed by
by an expert
expert system component.
component. Its
Its success
success thus
thus dispels
dispels doubts
doubts
component
about the applicability
applicability of
of expert
expert system technology
technology to
to domains
domains with
with significant
significant
about
of clinicians
clinicians
perceptual components.
components. ESCA
ESCA performed
performed slightly
better than
than the
the sub-panel
sub-panel of
perceptual
slightly better
for
the cases
cases examined.
examined.
Using ESCA
ESCA for
for subjective
clinical evaluation,
evaluation, and
and one
one
Using
subjective clinical
for the
i.o screen the conclusions,
conclusions, POSCH
POSCH can now
now evaluate
evaluate films
films faster,
faster, more
more
cardiologist io
cardiologist
consistently, and
and with
with less cost.
consistently,
Research in
Progress
Research
in Progress
The
continuation of
of projects
projects that
that
The research
research in
in progress
progress for
for the current
current year will
will be a continuation
have
have been underway
underway for
for some time.
time. The
The main
main areas will
will be ---
I.
Inference engine
1. Inference
engine mechanisms
mechanisms in
in diagnostic
diagnostic reasoning.
reasoning.
This will
will be a
This
continuation
of
project
The
Cleric language
language will
will be
The Cleric
continuation
of the Cleric/Vesalius
Cleric/Vesalius
project.
compare and
used to model
model different
different diagnostic
diagnostic strategies
strategies --- path-following,
path-following,
compare
conquer,
conquer, and stateless analysis.
analysis.
2. Merit
Merit system
for question
AGNESS
being used in
in developing
developing
system for
question selection.
seleclion.
AGNESS is being
in cystic
cystic fibrosis
fibrosis (CF)
(CF)
an expert
expert system
system for
for early
early detection
detection of
of clinical
clinical trends
trends in
patients.
patients. In
In addition,
addition, the ESC
ESCAA expert
expert system will
will be extended
extended to consider
consider
multiple
multiple lines
lines of
of reasoning
reasoning and to make
make use
use of
of the Dempster-Shafer
Dempster-Shafer method.
method.
3. Detection
Detection of
of deviations
deviations in time
time series
series by the human
human observer.
observer. Surveillance
Surveillance
and early
early detection
detection of
of deviation
deviation from
from a homeostatic
homeostatic state are goals common
common
to health
health care programs
programs for
for the apparently
apparently healthy
healthy as well
well as for
for groups
groups of
of
patients
patients known
known to have or
or have had specific
specific diseases.
diseases. Automated
Automated approaches
approaches
to detecting
detecting deviations
deviations have the advantage
advantage of
of being
being reliably
reliably applied,
applied,
traceable,
traceable, consistent
consistent in
in outcome,
outcome, and conserving
conserving of
of professional
professional resources.
Rule
Rule based expert
expert systems based upon
upon analysis
analysis of
of human
human graph
graph reading
reading
strategies
strategies are being
being evaluated.
evaluated.
4. Knowledge
for improving
practice. The
Knowledge based
based system
sy’stem for
improving transfusion
rransfusion practice.
The ESPRE
ESPRE
expert
expert system has undergone
undergone preliminary
preliminary evaluation
evaluation and is now
now being
being used
used
in parallel
parallel with
with traditional
traditional decision
decision processes in transfusion
transfusion therapy.
therapy.
E. H. Shortliffe
Shortliffe
176
5P41-RR00785-14
5P41-RR00785-14
SOLVER Project
Project
SOLVER
D. List
List of
of Relevant
Relevant Publications
Publications
D.
1. Bailey,
Bailey, A.
A. et
et al.:
al.: Auditing,
Auditing, artificial
artificial intelligence,
intelligence,
DECISION
SUPPORT
SYSTEMS:
THEORY
AND
DECISION SUPPORT SYSTEMS: THEORY
AND
Whinston (ed.),
(ed.), North-Holland
North-Holland Publications,
Publications, 1985.
Whinston
and expert
expert systems,
systems,
and
APPLICATION,
A.B.
APPLICATION,
A.B.
2. Connelly,
Connelly, D.P.
D.P. and
and Rich,
Rich, S.S.:
S.S.: Detection
Detection of
of deviations
deviations in
in time
time series
series by
by the
the
human observer.
observer. Proc.
Proc. IMIA
IMIA Workshop
Workshop on
on Maintaining
Maintaining a Healthy
Healthy State
State
human
within the
the Individual.
Individual. (in
(in press)
within
Connelly, D.
D. and
and Johnson,
Johnson, P.E.: Medical
Medical probfem
problem solving.
solving. Human
Human Pathology,
Pathology,
3. Connelly,
11(5):412-419, 1980.
11(5):412-419,
Elstein, A., Gorry,
Gorry, A.,
A., Johnson,
Johnson, P. and Kassirer,
Kassirer, J.: Proposed
Proposed Research
Research
4. Elstein,
Efforts. In
In D.C.
D.C. Connelly,
Connelly, E. Benson and
and D.
D. Burke
Burke (Eds.),
(Eds.), CLINICAL
CLINICAL
Efforts.
DECISION MAKING
MAKING AND
AND LABORATORY
LABORATORY USE.
USE. University
University of
of Minnesota
Minnesota
DECISION
Press, 1982, pp. 327-334.
327-334.
Press,
5. Feltovich,
Feltovich, P.J.:
PJ.: Knowledge
Knowledge based
based components of
0/ expertise
expertise in
in medical
medical
diagnosis. Learning
Learning Research and Development
Development Center
Center Technical
Report
diagnosis.
Technical Report
PDS-2, University
University of
of Pittsburgh,
Pittsburgh, September,
September, 1981.
PDS-2,
Feltovich, P.J.,
PJ., Johnson,
Johnson, P.E., Moller,
Moller, J.H. and
and Swanson, D.B.: The Role
Role and
and
6. Feltovich,
Development of
of Medical
Medical Knowledge
Knowledge in
in Diagnostic
Diagnostic Expertise.
Expertise. ln
In W.
W. Clancey
Clancey
Development
and E.H. Shortliffe
Shortliffe (Eds.),
(Eds.), READINGS
READINGS IN
IN MEDICAL
MED[CAL Al,
AI, Addison-Wesley,
Addison-Wesley,
1984, pp. 275-319.
275-319.
7. Johnson,
P.E. and Zualkernan,
I.: Building
Building expert
systems for
for
Johnson,
Zualkernan,
expert
troubleshooting
from
knowledge
of
the design
process,
lASTED
troubleshooting
from
knowledge
of
design
process,
lASTED
International
International Conference
Conference on Expert
Expert Systems, Geneva,
Geneva, Switzerland,
Switzerland, June, 1987.
8. Johnson,
0/
Johnson, P.E., Zualkernan,
Zualkernan, I., and Garber,S.:
Garber,S.: Specification
Specification
of expertise,
expertise,
International
International Journal
Journal of
of Man-Machine
Man-Machine Studies (in
(in press).
9. Johnson,
Johnson, P.E. Cognitive
Cognitive models
models 0/
of expertise,
expertise, Proceedings
Proceedings of
of USC Symposium
Symposium
on Expert
Expert Systems and Auditor
Auditor Judgment,
Judgment, February,
February, 1986.
OF
10. Johnson,
In ENCYCLOPEDIA
ENCYCLOPEDIA
OF SCIENCE
SCIENCE AND
AND
Johnson, P.E.: Problem
Problem Solving.
Solving. In
TECHNOLOGY,
McGraw-Hill,
1985.
TECHNOLOGY,
McGraw-Hill,
11. Johnson,
Johnson,
Medical
Medical
EXPERT
EXPERT
P.E., Moen,
Moen, 1.B.,
J.B., and Thompson,
Thompson, W.B.: Garden
Garden Path
Path Errors
Errors in
Diagnosis.
In Bolc,
Diagnosis.
Bolt, L. and Coombs,
Coombs, M.J.
?vl.J. (Eds.), COMPUTER
COMPUTER
SYSTEMS,
SYSTEMS, Springer-Verlag
Springer-Verlag (in
(in press).
press).
12.
12. Johnson, P.E.:
P.E.: Cognitive
Cognitive Models
Models of
of Medical
Medicai Problem
Problem Solvers.
Soivers. [n
ln D.C.
Connelly,
DECISION
Connelly, E. Benson, D. Burke
Burke (Eds.), CLINICAL
CLINTCAL
DECTSION MAKING
MAKING
AND
AND LABORATORY
LABORATORY USE. University
University of
of Minnesota
Minnesota Press,
Press, 1982,
1982, pp. 39-51.
39-5’1.
13.
13. Johnson, P.E.:
P.E.: What
What kind
kind of
of expert
expert should
shouid a(z system be?
be? Journal
Journal of
of Medicllle
Medic.lne
and Philosophy,
Philosophy, 8:77-97,
8:77-97, 1983.
1983.
14.
Chailenge for
for the
the Information
Information
14. Johnson, P.E.:
P.E.: The
The Expen
Expert Mind:
Mind: A New Challenge
Scientist
In Th.
SYSTEM
Scientist
Th. M. A. Bemelmans
Bemelmans (Ed.),
(Ed.), INFORMATION
INFORMATION
SYSTEM
DEVELOPMENT
FOR ORGANIZATIONAL
EFFECTIVENESS,
Elsevier
DEVELOPMENT
ORGANIZATIONAL
EFFECTIVENESS,
Elsevier
1984.
Science
Science Publishers
Publishers B.
B. V. (North-Holland),
(North-Holland),
1984.
15.
15. Johnson,
Johnson, P.E.,
P.E., Severance,
Severance, D.G.
D.G. and
and Feltovich,
Feltovich, PJ.:
P.J.: Design
Design 0/
of decision
decision support
support
systems
Rationaie and
und principies
principies from
j”rotn (he
the analysis
analysis or
:,i
systems in
in medicine:
medicine: Rationale
177
177
E.
E. H.
H. Shoreliffe
Shorcliffe
5P41-RR00785-14
5P41-RR00785-14
SOL
VER Project
SOLVER
Project
physician expertise.
Hawaii
Conference
Twelfth
Hawaii International
International
Conference on
physician
expertise. Proc. Twelfth
System Science, Western
Western Periodicals
Periodicals Co. 3:105-118,
3:105-118, 1979.
16. Johnson,
Johnson, P.E., Duran,
Duran, A., Hassebrock,
Hassebrock, F., Moller,
Moller, J., Prietula,
Prietula, M., Feltovich,
Feltovich,
P. and Swanson, D.: Expertise
Expertise and
and error
error in diagnostic
diagnostic reasoning.
reasoning. Cognitive
Cognitive
Science 5:235-283,
5:235-283, 1981.
Models
17. Johnson,
Johnson, P.E. and Hassebrock,
Hassebrock, F.: Validating
Validating Computer
Computer Simulation
Simulation Models
of
Expert Reasoning.
Reasoning. In
AND
of Expert
In R. Trappl
Trappl (Ed.),
(Ed.), CYBERNETICS
CYBERNETICS
AND SYSTEMS
SYSTEMS
RESEARCH.
North-Holland Publishing
RESEARCH. North-Holland
Publishing Co., 1982.
down the garden
garden path:
path:
18. Johnson,
Strolling
Johnson, P.E. and Thompson,
Thompson, W.B.: Strolling
Detection and recovery
from error
problem solving.
Detection
recovery from
error in expert
expert problem
solving. Proc.
Proc. Seventh
Seventh
IJCAI,
IJCAI, Vancouver,
Vancouver, B.C., August,
August, 1981, pp. 214-217.
214-217.
Multimethod study
19. Johnson,
Johnson, P.E., Hassebrock,
Hassebrock, F. and Moller,
Moller, J.H.: Muttimethod
study of
of clinical
clinical
judgement. Organizational
Behavior
judgement.
Organizational
Behavior and Human
Human Performance
Performance 30:201-230,
30:201-230,
1982.
20. Moller,
Moller, J.H., Bass,
Bass, G.M.,
G.M., Jr. and Johnson,
Johnson, P.E.: New techniques
techniques in the
construction
patient management
problems. Medical
construction of
of patient
management problems.
Medical Education
Education 15:150-153,
15:150-153,
1981.
Knowledge-based
21. Sielaff,
Knowledge-based
Sielaff, B.H
B.H.,.. Galen.
Galen, G
G.,.. Scott, E. and Connelly.
Connelly, D.P.:
In AAMSI
AAMSI Congress 87 (in
(in
system for
for improving
practice.
In
improving transfusion
transfusion practice.
press).
22. Simpson,
Simpson, D.E
D.E.,.• Rich,
Rich, E., Dalgaard.
Dalgaard, K., Gjerdingen,
Gjerdingen, K
K.,.. Crowson,
Crowson, T., O'Brien.
O’Brien,
D., Johnson,
process in
primary care:
Johnson, P.E.: The diagnostic
diagnostic process
in primary
care: A comparison
comparison of
of
SOCIAL SCIENCE
SCIENCE
AND
general internists
and
family physicians.
physicians.
SOCIAL
AND
general
internists
and family
MEDICINE
MEDICINE (in
(in press).
23. Slagle,
“Expert systems in
in medical
medical studies
studies --- A
A new twist."
twist,”
Slagie, J. et al.: "Expert
Proceedings of
Applications of
Artificial fntelligence,
Intelligence.
Proceedings
of the Conference
Conference on Applications
of Artificial
SPIE, 1986.
system for
for a resource
24. Slagle, J.: An expert
expert
resource
Communications
Communications of
of the ACM,
ACM, September,
September, 1985.
allocation
allocation
problem,
problem,
25. Slagle, 1.:
Alpha Beta
Beta Pruning,
Pruning, ENCYCLOPEDIA
OF
ENCYCLOPEDIA
OF ARTIFICIAL
ARTTFICIAL
J.: Alpha
INTELLIGENCE,
S.
New York,
INTELLIGENCE,
S. Shapiro
Shapiro (ed.).
(ed.), John
John Wiley
Wiley and Sons, Inc., New
York,
1985.
26. Slagle.
control
for computer
Slagle, J. and Gaynor,
Gaynor, M.: An intelligem
intelligent
control strategy
strategy for
computer
consultation.
consultation, IEEE
IEEE Trans.
Trans. on Pattern
Pattern Analysis
Analysis and Machine
Machine Intelligence,
Tnteiligence,
March.
.*
March, 1984.
Knowledge-based systems in
27. Spackman, K.A.
K.A. and Connelly.
Connelly, D.P.: Knowledge-based
in laboratory
laboratory
Archives of
of
medicine
pathology: A review
field. Archives
medicine and pathology:
review and survey
survey of
of the field.
Laboratory Medicine
Medicine and Pathology
Pathology 111:116-119,
111:116-119, 1987.
Laboratory
Swanson, D.B., Feltovich,
Feltovich, PJ.
P.J. and Johnson,
Johnson, P.E.: Psychological
of
28. Swanson.
Psychological Anafysis
Analysis OJ
Physician Expertise:
Expertise: Implications
Implications for
for The Design
Design of
Decision Support
Physician
of- Decision
Support
Systems. In
North-Holland
In D.B. Shires
Shires and H. Wold
Wold (Eds.),
(Eds.), MEDINF077.
MEDINF077,
North-Holland
Publishing
Publishing Co
Co.,.. Amsterdam,
Amsterdam, 1977, pp. 161-164.
161-164.
29. Thompson,
Recognition-based diagnostic
Thompson, W.B., Johnson.
Johnson, P.E. and Moen.
Moen, J.B.: Recognition-based
dia,pnosttc
reasoning. Proc. Eighth
Eighth IJCAI.
IJCAL, Karlsruhe.
Karisruhe, West Germany,
Germany, August.
Auyusr, 1983.
j-983.
reasoning.
Shortliffe
E. H. Shortliffe
178
SOLVER
SOL VER Project
Project
5P41-RR00785-14
5P41-RR00785-14
E. Funding
Funding and
and Support
Support
E.
Work on
on the
the SOLVER
SOLVER project
project is currently
currently supported
supported by
by grants
grants from
from the
the Control
Control Data
Data
Work
Corporation ($95,000;
($95,000; 1986-88)
1986-88) and
and IBM
IBM ($81,000; 1987)
1987) to
to Paul
Paul Johnson
Johnson ($95,000;
($95,000;
Corporation
1986-88) and by a grant
grant from
from the Microelectronics
Microelectronics and Information
Information Sciences Center
Center
1986-88)
(MEIS) at the University
University of
of Minnesota
Minnesota to
to Paul
Paul Johnson,
Johnson, William
William Thompson,
Thompson, James
(MEIS)
1986-7).
Slagle ($300,000; 1986-7).
in medical
medical informatics
informatics is supported,
supported, in
in part,
part, by
by a
Research in
National Library
Library of
of Medicine,
Medicine, LM-00160,
LM-00160, in
in the
the amount
amount of
of
National
1984-1989. Dr.
Dr. Connelly
Connelly and Prof.
Prof. Johnson
Johnson are participants
participants
1984-1989.
doctoral fellowship
fellowship of
of Dr.
Dr. Spackman
Spackman was funded
funded by
by this
this grant.
grant.
doctoral
training grant
grant from
from the
training
$712,573 for
for the period
period
in this
this grant.
grant. The
The postpostin
WITH THE
THE SUMEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
II. INTERACTIONS
INTERACTIONS WITH
II.
Medical Collaborations
Collaborations and Program
Program Dissemination
Dissemination via
via SUMEX
SUMEX
A. Medical
Work in
in medical
medical diagnosis
diagnosis is carried
carried out
out with
with the cooperation
coope{ation of
of faculty
faculty and students
studen ts
Work
in the University
University of
of Minnesota
Minnesota Medical
Medical School and St. Paul Ramsey Medical
Medical Center.
Center.
in
The Galen
Galen system is available
available on SUMEX
SUMEX from
from the University
University of
of Minnesota
Minnesota as
as an
The
for the study
study of
of recognition
recognition based reasoning
reasoning systems.
unsupported research tool
tool for
unsupported
B. Sharing
Sharing and
and Interactions
Interactions with
with Other
Other SCIMEX-AIM
SUMEX-AIM Projects
Projects
II.
The SOLVER
SOLVER project
project has not
not been engaged
engaged in
in any
any formal
formal sharing
sharing with
with other
other projects
projects in
The
the last year. The
has continued
The SUMEX
SUMEX resource has
continued to serve as a communications
communications vehicle
vehicle
Dr. Johnson
Johnson conducted
conducted informal
informal
for
contacts
Dr.
for informal
informal
contacts with
with other
other researchers.
conferences
conferences during
during the year with
with Drs.
Drs. Bruce Buchanan
Buchanan and William
William Clancey.
Clancey.
Resource Management
Management
C. Critique
Critique of
of Resource
None.
None.
III.
III. RESEARCH
RESEARCH PLANS
PLANS
Project Goals
Plans
A. Project
Goals and
and Plans
An
An overall
overall goal of
of the project
project is to describe
describe methods
methods for
for the specification
specification of
of
Our
Our objective
objective is to construct
construct an artifact
artifact (for
(for example,
example, an expert
expert system)
system) that
that
a class
class of
of problems
problems which
which is currently
currently solved by an expert.
expert. To
To construct
construct this
this
specification
specification of
of the requirements
requirements is needed
needed which
which outlines
outlines what
what needs
needs to be
to solve the problem.
problem.
expertise.
expertise.
can solve
artifact
artifact a
computed
computed
A
A number
number of
of artifacts
artifacts may achieve the same performance
performance in
in a variety
variety of
of ways. The
The
expert's
expert’s method
method works
works because
because it
it is adapted to the capabilities
capabilities of
of the human
human
information
information processing
processing system and the demands of
of the problem-solving
problem-solving task. Since we
may implement
implement our
our specification
specification on various
various kinds
kinds of
of processors, we seek a description
description
that
that does not
not depend on a particular
particular processing
processing architecture.
architecture. The
The purpose
purpose of
of knowledge
knowledge
acquisition
acquisition is not
not to learn
learn how to solve a problem,
problem, but
but rather
rather to discover
discover what is
required
required to solve a problem.
problem.
Our
actIvIty
Our goal is to use
use protocol
protocol records of
of problem-solving
problem-solving
activity
specification
specification of
of the requirements
requirements for
for any artifact
artifact that
that would
would attempt
attempt to
problem.
problem. Given
Given a class
class of
of problems,
problems, such as
as medical
medical diagnostic
diagnostic tasks,
tasks,
record from
from experts solving
solving these
these problems,
problems, the task is to determine
determine
transforming
transforming the protocol
protocol into
into a specification
specification of
of expertise.
expertise.
179
to develop
develop a
solve the same
and a protocol
protocol
a method
method for
for
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
SOL VER Project
Project
SOLVER
Our goal
goal is to
to investigate
investigate the
the following
following framework
framework for
for specification
specification of
of expertise:
expertise:
Our
processor that
that has the
the capability
capability of
of producing
producing
The expert
expert can be viewed
viewed as a processor
1. The
certain problem-solving
problem-solving behavior
behavior using
using expertise.
expertise. The
The task
task of
of knowledge
knowledge
certain
acquisition is to
to determine
determine this
this expertise.
expertise.
acquisition
The expert
expert has developed
developed a set of
of actions
actions and
and abilities
abilities that
that are necessary to
to
2. The
realize this
this expertise.
expertise.
realize
Although we cannot
cannot observe
observe the
the expertise
expertise directly,
directly, we can observe
observe the
the
3. Although
invocation of
of the
the expert’
expert's
actions and abilities
abilities in
in a record
record of
of problemprobleminvocation
s actions
solving behavior.
behavior.
solving
Since we can observe
observe the
the invocation
invocation of
of actions
actions and abilities
abilities by
by the expert,
expert,
4. Since
develop some representation
representation of
of the expertise.
expertise.
we can develop
A statement
statement of
of the expertise
expertise required
required to
to perform
perform a task serves as a
5. A
specification of
of the requirements
requirements for
for a computer
computer program
program that
that is designed
designed to
specification
perform the task.
perform
of a specific
specific methodology
methodology for
for collecting
collecting and analyzing
analyzing protocol
protocol data
data
The development
development of
The
arrive at a forma1
formal specification
specification of
of expertise.
expertise.
to arrive
Justification and
and Requirements
Requirements for
for Continued
Continued SUMEX
SUMEX Use
B. Justification
Our current
current model
model development
development takes advantage of
of the sophisticated
sophisticated Lisp
Lisp programming
programming
Our
environments on SUMEX
SUM EX and local
local facilities.
facilities. Although
Although much
much current
current work
work with
with Galen
Galen
environments
is done
Y 111780,
11/780, we continue
continue to benefit
benefit from
from
done using
using a version
version running
running on a local'
local Y '(
the interaction
interaction with
with other
other researchers facilitak.
facilitat- : by the SUMEX
SUMEX system. We
We expect
expect to
We also plan
plan to
use
EX to allow
We
use SUM
SUMEX
allow other
other groups
groups access
access to the Galen
Galen program.
program.
continue
continue use
use of
of the knowledge
knowledge engineering
engineering tools available
available on SUMEX.
SUMEX.
We have completed
completed a CommonLisp
CommonLisp implementation
implementation of
of the Galen
Galen system and expect
expect to
rely
projects.
rely heavily
heavily on CommonLisp
CommonLisp for
for future
future projects.
C. Needs
Needs and Plans
Plans for
for Other
Other Computing
Computing Resources Beyond SUMEX-AIM
SUMEX-AIM
Our
Our current
current research support
support has permitted
permitted us to purchase Sun workstations
workstations for
for our
our
Artificial
Intelligence
The
Artificial
Intelligence laboratory.
laboratory.
The availability
availability of
of CommonLisp
CommonLisp on these machines
machines
is one reason why
why we expect
expect to make use
use of
of that
that language in the future.
future.
SUMEX
SUMEX will
will continue
continue to be used
used for
for collaborative
collaborative activities
activities and for
for
development
development requiring
requiring tools
tools not
not available
available locally.
locally.
program
program
D. Recommendations
for Future
Recommendations for
Future Community
Community and Resource Development
Development
As a remote
remote site, we particularly
particularly appreciate
appreciate the communications
communications that
that the SUMEX
SUMEX
facility
We, too, are
facility provides
provides our
our researchers with
with other
other members of
of the community.
community.
moving
development
but
moving toward
toward a workstation-based
workstation-based
development environment,
environment,
but we hope
hope that
that
SUMEX
In
SUMEX will
will continue
continue to serve as
as a focal
focal point
point for
for the medical
medical AI
AI community.
community.
In
addition
addition to communication
communication and sharing
sharing of
of programs,
programs, we are interested
interested in development
development
of
continued existence
existence of
of the
of CommonLisp
CommonLisp based
based knowledge
knowledge engineering
engineering tools. The continued
SUMEX
SUMEX resource is very important
important to us.
us.
E. H. Shortliffe
Shortliffe
180
ATTENDING Project
Project
ATTENDING
5P41-RR00785-14
SP41-RR00785-14
IV.B.5. ATTENDING
ATTENDING Project
Project
IV.B.5.
ATfENDING Project--Expert
Project--Expert Critiquing
Critiquing Systems
ATTENDING
Perry L.
L. Miller,
Miller, M.D.
M.D. Ph.D.
Ph.D.
Perry
Department of
of Anesthesiology
Anesthesiology
Department
Yale University
University School
School of
of Medicine
Medicine
Yale
New Haven,
Haven, CT
CT 06510
06510
New
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
I.I. SUMMARY
A. Project
Project rationale
rationale
A.
Our project
project is exploring
exploring the “critiquing”
"critiquing" approach
approach to bringing
bringing computer-based
computer-based advice
advice to
Our
practicing physician.
physician.
the practicing
the
Critiquing is a different
different approach
approach to
to the design of
of artificial
artificial intelligence
intelligence based expert
expert
Critiquing
systems. Most
Most medical
medical expert
expert systems attempt
attempt to simulate
simulate a physician’
physician's
decIsion-making
s decision-making
As a result,
result, they
they have the clinical
clinical effect
effect of
of trying
trying to tell
tell a physician
physician what
what to
process. As
critiquing system first
first asks
asks the physician
physician
how to practice
practice medicine.
medicine. In
In contrast,
contrast, a critiquing
do: how
how he contemplates
contemplates approaching
approaching his patient’
patient's
critiques that
that plan.
plan. In the
how
s care, and then critiques
any risks
risks or
or benefits
benefits of
of the proposed
proposed approach,
approach, and of
of any
any
critique, the system discusses any
critique,
other
It
It is anticipated
anticipated that
that the critiquing
critiquing
other approaches
approaches which
which might
might be preferred.
preferred.
approach
particularly well
ted for
ike medicine,
medicine, where decisions
decisions
approach may
may be particularly
well sui
suited
for domains,
domains, Ilike
involve
subjective judgment.
judgment.
involve a great
great deal of
of subjective
To
To date, several prototype
prototype critiquing
critiquing systems have been developed
developed in different
different medical
medical
domains:
domains:
1. ATTENDING,
the first
ATTENDING,
first system
critiques
critiques anesthetic
anesthetic management.
management.
to implement
impiement the critiquing
critiquing approach,
approach,
2. HT-A
TTENDING critiques
HT-ATTENDING
critiques the pharmacologic
pharmacologic management
management of
of essential
hypertension.
hypertension.
3.
critiques
3. VQ-ATTENDING
VQ-ATTENDING
critiques aspects
aspects of
of ventilator
ventilator management.
management.
4. PHEO-ATTENDING
critiques
PHEO-ATTENDING
critiques [he
the laboratory
laboratory and radioiogic
radioiogic workup
workup of
of a
patient
patient for
for a suspected pheichromocytoma.
pheichromocytoma.
5.
system, ESSENTIAL-ATTENDING,
has
5. In addition,
addition, aa domain-independent
domain-independent
ESSENTIAL-ATTENDING,
has
of
been
been developed
developed to facilitate
facilitate the implementation
implementation
of critiquing
critiquing systems in
other
other domains.
domains.
C. Highlights
Highlights of
of Research
Research Progress
Progress
Current
Current projects
projects include
include the following:
following:
HT-ATIENDING
The
has
of HT-ATTENDING
HT-ATTENDING
has been
been
HT-AITENDING
The original
original prototype
prototype version of
converted
format,
converted to the ESSENTIAL-ATTENDING
ESSENTIAL-ATTENDING
format, and
and updated to reflect
reflect current
current
thinking
major priority
prioricy isis to subject
subject this
thinking in
in the field
field of
of hypertension
hypertension management.
managemenr. A major
system
and to explore
explore how best
best to disseminate
disseminate the
the
system to validation
validation and
and clinical
clinical evaluation,
evaluation, and
system
system as
as aa practical
practical consultation
consultation tool.
tool.
181
181
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
ATTENDING Project
Project
ATTENDING
DxCON:
Critiquing Radiologic
Radiologic Workup
Workup DxCON
DxCON extends
extends the design
design developed
developed in
in
Critiquing
DxCON:
PHEO-ATTENDING to
to critique
critique the
the radiologic
radiologic workup
workup of
of suspected obstructive
obstructive
PHEO-ATTENDING
jaundice.
Workup is an area in
in which
which we will
will aggressively
aggressively pursue
pursue the
the critiquing
critiquing
Workup
jaundice.
approach for
for two
two reasons. 1) Since
Since many
many areas of
of workup
workup are quite
quite constrained,
constrained, itit may
may
approach
prove possible
possible to
to develop
develop and
and test
test complete
complete systems in
in a reasonably
reasonably short
short time-frame.
time-frame.
prove
2) Since
Since workup
workup is expensive,
expensive, and
and very
very wasteful
wasteful of
of resources ifif performed
performed improperly,
improperly, a
2)
computer system
system which
which helps
helps to
to optimize
optimize a physician’
physician's
workup plans
plans could
could have
computer
s workup
significant economic
economic benefits.
benefits. The
The present
present national
national emphasis
emphasis on
on controlling
controlling health
health
significant
makes this
this project
project very
very topical.
topical. We
We are also using
using this
this domain
domain to explore
explore issues
costs makes
of knowledge
knowledge acquisition
acquisition and verification.
verification.
of
ICON: Critiquing
Critiquing Radiological
Radiological Differential
Differential Diagnosis
Diagnosis Most
Most existing
existing diagnostic
diagnostic computer
computer
ICON:
systems produce
produce a ranked
ranked differential
differential diagnosis
diagnosis as their
their output.
output. In
In this
this process, the rich
rich
systems
structure of
of the knowledge
knowledge that
that went
went into
into developing
developing the diagnoses may
may be lost
lost to the
structure
ICON explores
explores a different
different approach
approach to diagnostic
diagnostic advice
advice in
in the domain
domain of
of
ICON
user.
radiology. To
To use
use ICON,
ICON, a radiologist
radiologist describes
describes a set of
of findings
findings seen
seen on chest x-ray,
x-ray,
radiology.
together with
with a proposed
proposed diagnosis.
diagnosis. [CON
produces a detailed
detailed analysis
analysis of
of why the
[CON then produces
together
observed findings
findings serve to support
support or
or to
to rule
rule out
out the diagnosis.
diagnosis. ItIt may also suggest
suggest
observed
further findings
findings that
that might
might help
help refine
refine the diagnosis,
diagnosis, again explaining
explaining why
why the findings
findings
further
important.
are important.
D. Publicarions
Publications
D.
Miller, P.L.:
Expert Critiquing
Critiquing Systems:
Practice-Based
1. Miller,
Expert
Practice-Based
Consultation by Computer.
Computer. New
New York:
York: Springer-Verlag,
Springer-Verlag, 1986.
Consultation
Medical
Medical
2. Miller,
Intelligence.
Miller, P.L. (Ed.):
(Ed.): Selected Topics
Topics in
in Medical
Medical Artificial
Artificial
Intelligence.
York:
York: Springer-Verlag
Springer-Verlag (in
(in press).
New
New
3. Miller,
Critiquing
Miller, P.L., Shaw, C., Rose, l.R.,
J.R., Swett, H.A.:
H.A.:
Critiquing the process of
of
radiologic
diagnosis.
Computer
radiologic differential
differential
diagnosis.
Computer Methods
Methods and Programs
Programs in
in
Biomedicine
Biomedicine 22:12-25,
22:12-25, 1986.
4. Miller,
The evaluation
evaluation of
of artificial
artificial intelligence
intelligence systems in
in medicine.
medicine.
Miller, P.L.: The
Computer
Computer Methods
Methods and Programs
Programs in
in Biomedicine
Biomedicine 22:5-11,
22:5-11, 1986.
5. Rennels,
E.H.,
Choice
Choice of
of explanation
explanation in
Rennels, G.D.,
G.D., Shortliffe,
Shortliffe,
E.H., Miller,
Miller, P.L.:
medical
A
model
intelligence
A multi-attribute
multi-attribute
model of
of artificial
artificial
intelligence
medical management:
management:
approaches.
Medical Decision
Decision Making
Making 7:22-31,
7:22-31, 1987.
approaches. Medical
6. Rennels,
intelligence
Rennels, G.D.,
G.D., Miller,
Miller, P.L.: Artificial
Artificial
intelligence research in anesthesia and
intensive
intensive care. Anesthesiology
Anesthesiology (submitted).
(submitted).
7. Miller.
Prose generation
Miller. P.L., Rennels,
Rennels, G.D.:
G.D.:
generation from
from expen
expert systems:
applied
applied computational
computational linguistics
linguistics approach
approach (submitted).
(submitted).
An
An
8. Mars,
Mars, N.J.I.,
N.J.I., Miller,
Miller, P.L.: Knowledge
Knowledge acquisition
acquisition and verification
verification tools for
for
medical
medical expert
expert systems. Medical
Medical Decision
Decision Making
Making 7:6-11,
7:6-11, 1987.
9. Miller,
SJ.,
Miller, P.L., Blumenfrucht,
Blumenfrucht,
S.J., Rose,
Rose, J.R., Rothschild,
Rothschild, M., Swett, H.A.,
H.A.,
Weltin,
A knowledge
Weltin, G., Mars, N.J.I.: HYDRA:
HYDRA:
knowledge acquisition
acquisition tool for
for expen
expert
systems which
which critique
critique medical
medical workup.
workup. Medical
!Medical Decision
Decision Making
Making 7:12-21,
7:12-21,
1987.
10.
10. Swett, H.A.,
H.X., Miller,
Miller, P.L.: [CON:
ICON: A computer-based
computer-based approach
approach [0
co differential
differential
diagnosis
diagnosis in radiology.
radiology. Radiology
Radiology (in
(in press).
E.
E. H. Shonliffe
Shortliffe
182
182
ATTENDING Project
Project
ATTENDING
SP41-RR0078S-14
5P41-RR00785-14
Rennels, G.D.,
G.D., Shortliffe,
Shortliffe, E.H.,
E.H., Stockdale,
Stockdale, F.E.,
F.E., Miller,
Miller, P.L.:
11. Rennels,
AA
of reasoning
reasoning from
from the
the clinical
clinical literature.
literature. Computer
Computer
computational model
model of
computational
and Programs
Programs in
in Biomedicine
Biomedicine (in
(in press).
Methods and
Methods
Rennels, G.D.,
G.D., Shortliffe,
Shortliffe, E.H.,
E.H., Stockdale,
Stockdale, F.E.,
F.E., Miller,
Miller, P.L.:
P.L.: AA structured
structured
12. Rennels,
of the clinical
clinical literature
literature and
and its
its use
use in
in a medical
medical management
management
representation of
representation
advice system.
system. Bulletin
Bulletin du Cancer
Cancer (in
(in press).
advice
13. Miller,
Miller, P.L.,
P.L., Barwick,
Barwick, K.W.,
K.W., Morrow,
Morrow, J.S.,
J.S., Powsner,
Powsner, S.M.,
S.M., Riely,
Riely, CA.:
C.A.:
Semantic relationships
relationships and
and medical
medical bibliographic
bibliographic retrieval:
retrieval: AA preliminary
preliminary
Semantic
(submitted).
assessment (submitted).
Miller, P.L.:
P.L.: Exploring
Exploring the
the critiquing
critiquing approach:
approach: Clinical
Clinical practice-based
practice-based
14. Miller,
Informatics and Control
Control
feedback by
by computer.
computer. Biomedical
Biomedical Measurement,
Measurement, Informatics
feedback
(submitted).
(submitted).
IS. Rennels,
Rennels, G.D.,
G.D., Shortliffe,
Shortliffe, E.H.,
E.H., Stockdale,
Stockdale, F.E., Miller,
Miller, P.L.:
A
15.
A
cOIl)putational model
model of
of reasoning
reasoning from
from the clinical
clinical literature.
literature. The
The AI
AI
computational
Magazine (accepted
(accepted pending
pending revision).
revision).
Magazine
16. Miller,
Miller, P.L.: Exploring
Exploring the critiquing
critiquing approach:
approach: Sophisticated
Sophisticated practice-based
practice-based
feedback by computer.
computer.
Proceedings of
of the Fifth
Fifth World
World Conference
Conference on
Proceedings
feedback
Medical Informatics
Informatics MEDINFO-86,
MEDINFO-86, Washington,
Washington, D.C.,
D.C., October
October 1986, pp 2-6.
2-6.
Medical
0U .r., Miller,
Miller, P.L.: Tools
Tools for
for knowledge
knowledge acquisition
acquisition and verification
verification
17. Mars,
Mars, N.J.I.,
of the
the. Tenth
Tenth Symposium
Symposium on Computer
Computer
in medicine.
medicine.
Proceedings of
Proceedings
Applications in
in Medical
Medical Care, Washington,
Washington, D.C.,
D.C., October
October 1986, pp. 36-42.
36-42.
]4npplications
18. Miller,
Miller, P.L., Blumenfrucht,
Blumenfrucht, SJ.,
S.J., Rose, J.R., Rothschild,
Rothschild, M., Weltin,
Weltin, G., Swett,
H.A.,
N.J.I.: Expert
Expert system knowledge
knowledge acquisition
acquisition for
for domains
domains of
of
H.A., Mars,
Mars, N.J.I.:
medical
An
An augmented
augmented transition
transition network
network model.
model. Proceedings
Proceedings of
of
medical workup:
workup:
the Tenth
Applications
in
Tenth Symposium
Symposium on Computer
Computer
Applications
in Medical
Medical Care,
3S.
Washington,
Washington, D.C.,
D.C., October
October 1986, pp. 3030-35.
19. Rennels,
Rennels, G.D.,
G.D., Shortliffe,
Shortliffe, E.H.,
E.H., Stockdale,
Stockdale, F.E., Miller,
Miller, P.L.: Reasoning
Reasoning from
from
the clinical
The
Proceedings of
of the Fifth
Fifth
clinical literature:
literature:
The Roundsman
Roundsman system. Proceedings
World
World Conference
Conference on Medical
Medical Informatics
Informatics MEDINFO-86,
MEDINFO-86, Washington,
Washington, D.C.,
October
October 1986, pp. 771-77S.
771-775.
20. Rennels,
Rennels, G.D.,
G.D., Shortliffe,
Shortliffe, E.H.,
E.H., Stockdale,
Stockdale, F.E., Miller,
Miller, P.L.: Updating
Updating an
expert
Examples
Examples from
from
expert knowledge
knowledge base
base as
as medical
medical knowledge
knowledge evolves:
oncology
oncology management.
management. Proceedings
Proceedings of
of the American
American Association
Association of
of Medical
Medical
Systems and Informatics
Informatics Congress-87,
Congress-87, San Francisco,
Francisco, May
May 1987, pp. 238-231.
238-231.
21. Fisher,
A
A script-based
script-based representation
representation of
of
Fisher, P.R., Miller,
Miller, P.L., Swett, H.A.:
H.A.:
medical
Proceedings
Proceedings of
of the
medical knowledge
knowledge involving
involving multiple
multiple perspectives.
perspectives.
American
American Association
Association of
of Medical
Medical Systems and Informatics
Informatics Congress-87,
Congress-87, San
Francisco,
Francisco, May
May 1987, pp. 233-237.
233-237.
22. Miller,
A
Expert consultation
consultation systems in
in medicine:
medicine:
A complex
complex and
Miller, P.L.: Expert
fascinating
Proceedings
fascinating domain.
domain.
Proceedings of
of the Annual
Annual Meeting
Meeting of
of the IEEE
IEEE
(Electro-87),
(Electra-87), New
New York,
York, April
April 1987, pp. 112:1-4
l/2:1-4 (invited
(invited paper).
paper).
23. Miller,
Miller, P.L., Fisher,
Fisher, P.R.: Causal models
models in medical
medical artificial
artificial intelligence.
intelligence.
Proceedings
in
Proceedings of
of the Eleventh
Eleventh Symposium
Symposium on Computer
Computer Applications
Applications
Medical
Medical Care, Washington,
Washington, D.C., November
November 1987 (submitted).
(submitted).
183
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
ATTENDING
Project
ATTENDING
Project
Powsner, S.M.,
S.M., Barwick,
Barwick, K.W.,
K.W., Morrow,
Morrow, J.S., Riely,
Riely, C.A.,
C.A., Miller,
Miller, P.L.:
P.L.: Coding
Coding
24. Powsner,
semantic relationships
relationships for
for medical
medical bibliographic
bibliographic retrieval:
retrieval: AA preliminary
preliminary
semantic
study. Proceedings
Proceedings of
of the
the Eleventh
Eleventh Symposium
Symposium on
on Computer
Computer Applications
Applications
study.
in Medical
Medical Care,
Care, Washington,
Washington, D.C.,
D.C., November
November 1987
1987 (submitted).
(submitted).
in
E. Funding
Funding Support
Support
E.
EXPERT COMPUTER
COMPUTER SYSTEMS
SYSTEMS WHICH
WHICH CRITIQUE
CRITIQUE PHYSICIAN
PHYSICIAN PLANS
PLANS
EXPERT
NIH Grant
Grant ROl
R01 LM04336
LM04336
NIH
Principal Investigator:
Investigator: Perry
Perry L.
L. Miller,
Miller, M.D.,
M.D., Ph.D.
Ph.D.
Principal
Annual Direct
Direct Costs: approximately
approximately $100,000
$100,000
Annual
Period of
of Support:
Support: 9/l/85-8/31/87
9/1/85-8/31/87
Period
This two-year
two-year grant
grant supports
supports the exploration
exploration of
of the critiquing
critiquing
This
approach to
to bringing
bringing computer-based
computer-based advice
advice to the physician,
physician,
approach
focusing primarily
primarily on
on the underlying
underlying system design
design issues.
focusing
SUPPORT OF
OF THE
THE UNIFIED
UNIFIED MEDICAL
MEDICAL LANGUAGE
LANGUAGE PROGRAM
PROGRAM
SUPPORT
NLM Contract
Contract NOl-LM-6-3524
N01-LM-6-3524
NLM
Principal Investigator:
Investigator: Perry
Perry L.
L. Miller,
Miller, M.D.,
M.D., Ph.D.
Ph.D.
Principal
Annual Direct
Direct Costs: approximately
approximately $100,000
Annual
Period of
of Support:
Support: 8/22/86-8/21/88
8/22/86-8121/88
Period
This two-year
two-year research contract
contract is part
part of
of the NLM
NLM Unified
Unified
This
Medical
program. We
Medical Language
Language (UML)
(UML) program.
We are defining
defining a set of
of
semantic
semantic relationships
relationships which
which could
could be used to augment
augment the UML,
UML,
to facilitate
facilitate such functions
functions as medical
medical bibliographic
bibliographic retrieval.
retrieval.
SUPPORT
AND
INTELLIGENCE
SUPPORT FOR
FOR MEDICAL
MEDICAL INFORMATICS
INFORMATICS
AND ARTIFICIAL
ARTIFICIAL
INTELLIGENCE
Ira
Ira DeCamp
DeCamp Foundation
Foundation
Co-Principal
Co-Principal Investigators:
Investigators: Henry
Henry A.
A. Swett,
Swett, M.D.
M.D.
Perry
Perry L.
L. Miller,
Miller, M.D.,
M.D., Ph.D.
Ph.D.
Annual
Annual Costs: $75,000
Period
Period of
of Support:
Support: 7/1/86-6/30/90
7/l/86-6/30/90
This
This grant
grant supports
supports our
our present
present Medical
Medical Informatics
Informatics program
program
and is currently
currently being
being used
used primarily
primarily to support
support Medical
Medical
Informatics
Informatics research training.
training. If
If the present
present training
training
application
application is funded,
funded, the Ira
Ira DeCamp
DeCamp support
support could
could be used
used
for
for other
other activities
activities in
in support
support of
of the training
training such as
as for
for
a program
program secretary
secretary and for
for computing
computing programming
programming support.
support.
MEDICAL
RESEARCH
AT
MEDICAL INFORMATICS
INFORMATICS
RESEARCH TRAINING
TRAINING
AT YALE
YALE
Principal
Principal Investigator:
Investigator: Perry
Perry L. Miller,
Miller, M.D.,
M.D., Ph.D.
Ph.D.
We have been informed
informed that
that we will
will receive
receive a five-year
five-year
training
training grant
grant starting
starting July
July 1, 1987.
E. H. Shortliffe
Shortliffe
184
5P41-RR00785-14
5P41-RR00785-14
ATTENDING Project
Project
ATTENDING
Pending Support
Support
Pending
EXPERT COMPUTER
COMPUTER SYSTEMS
SYSTEMS WHICH
WHICH CRITIQUE
CRITIQUE PHYSICIAN
PHYSICIAN PLANS
PLANS
EXPERT
Principal Investigator:
Investigator: Perry
Perry L.
L. Miller,
Miller, M.D.,
M.D., Ph.D.
Ph.D.
Principal
Annual Direct
Direct Costs: approximately
approximately $100,000
$100,000
Annual
Period of
of Support:
Support: 9/l/87-8/31/90
9/1/87-8/31/90
Period
This grant
grant requests continuation
continuation of
of our
our currently
currently funded
funded grant
grant
This
which is exploring
exploring the
the critiquing
critiquing approach
approach to
to bringing
bringing
which
computer-based advice
advice to
to the
the practicing
practicing physician.
physician. This
This
computer-based
continuation grant
grant application
application focuses
focuses especially
especially on
on refining
refining
continuation
and evaluating
evaluating the
the HT-ATTENDING
HT-ATTENDING system which
which critiques
critiques
and
hypertension management.
management.
hypertension
II. INTERACTIONS
INTERACTIONS WITH
WITH THE
THE SUMEX-,4IM
SUMEX-AIM RESOURCE
RESOURCE
IT.
Until recently
recently we have been using
using the RUTGERS-ATM
RUTGERS-AIM Resource. We
We used that
that facility
facility
Until
to
implement
all
of
our
early
critiquing
systems.
We
are
currently
in
the
early
We
currently in
early stages
to implement all of our early critiquing
of moving
moving part
part of
of our
our critiquin,
critiquing0 research to
to the
the SUMEX-AIM
SUMEX -AIM facility.
facility. Our
Our main
main uses
of
of SUMEX-AIM
SUMEX-AIM will
will be the following:
following:
of
1. We
We will
will use
use SUMEX-AIM
SUMEX-AIM to demonstrate
demonstrate two
two of
of our
our systems, ATTENDING
ATTENDING
HT-ATTENDING.
and HT-ATTENDlNG.
We will
will use SUMEX-AIM
SUMEX-AIM for
for the continued
continued refinement
refinement of
of HTHT2. We
ATTENDING,
and for
planned cmtrolled
ATTENDING,
for a planned
c?ntroIled clinical
clinical experiment
experiment to measure
HT-ATTENDING's~vice
patient care.
This
the effect
This will
will be
:!vice on patient
effect of
of HT-ATTENDING’
s
performed in
New Havel.
flospital Primary
Primary Care Center,
Center, and is
performed
in the Yale
Yale New
Haven :-Iospital
planned to commence
planned
commence this
this coming
coming year.
3. We will
use SUMEX-AIM
for
will use
SUMEX-AIM
for communication
communication access
access to the national
national AIM
AIM
community.
community.
facility
We have found
found our
our use of
of the RUTGERS-AIM
RUTGERS-AIM
facility to be extremely
extremely valuable.
valuable. [t
It
provided
provided us
us the resources needed to initiate
initiate our
our research :md
2nd to continue
continue several
projects
projects which
which are still
still active.
active. It
It provided
provided a natural
natural vehicle
vehicle to allow
allow us
us to demonstrate
demonstrate
the various
various systems easily, both
both in the United
United States and
and in Europe.
Europe. Also,
Also, it
it enabled
enabled us
us
to collaborate
collaborate very closely
closely with
with Dr.
Dr. Glenn
Glenn Rennels
Rennels in his Stanford
Stanford Medical
Medical Information
Information
Science thesis project
and RUTGERSproject on the Roundsman
Roundsman system. Via
Via SUMEX-AIM
SUMEX-AIM
RUTGERSAIM,
AIM, Dr. Rennels and Dr. Miller
Miller maintained
maintained very close contact,
contact, typically
typically with
with multiple
multiple
messages
messages each
each week,
week, and sometimes
sometimes within
within a single
single day.
1I1.
III. FUTURE
FUTURE PLANS
PLANS
We plan to continue
continue our
our crltlquing
critiquing research as
as outlined
outlined above. One of
of our
our highest
highest
priorities
priorities will
will be the controlled
controlled experimental
experimental evaluation
evaluation of
of the HT-ATTENDING
HT-ATTENDING
system, which
We will
which will
will be done using SUMEX-AIM.
SUMEX-AIM.
will also continue
continue to utilize
utilize
SUMEX-AIM
as
SUMEX-AIM
as outlined
outlined above. Although
Although we are increasingly
increasingly moving
moving a great deal
deal of
of
our
our work
work onto
onto internal
internal workstations,
workstations, we
we nevertheless olan
plan to continue
continue our
our use
use of
of
SUMEX-AIM,
SUMEX-AIM, especially
especially in
in the further
further refinement
refinement and
and evaluation
evaluation of
of HT-ATTENDING.
HT-ATTENDING.
185
185
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Pilot Stanford
Stanford Projects
Projects
Pilot
IV.C. Pilot
Pilot Stanford
Stanford Projects
Projects
1V.C.
Following are descriptions
descriptions of
of the
the informal
informal pilot
pilot projects
projects currently
currently using
using the Stanford
Stanford
Following
portion of
of the SUMEX-AIM
SUM EX - AIM resource,
resource, pending
pending funding,
funding, full
full review,
review, and
and authorization.
authorization.
portion
In addition
addition to
to the progress reports
reports presented
presented here, abstracts
abstracts for
for each project
project are
In
submitted on
on a separate Scientific
Scientific Subproject
Subproject Form.
Form.
submitted
E. H. Shortliffe
Shortliffe
186
Pilot Stanford
Stanford Projects
Projects
Pilot
5P41-RR00785-14
5P41-RR00785-14
IV.C.l. REFEREE
REFEREE Project
Project
IV.C.l.
REFEREE
REFEREE Project
Project
Bruce G. Buchanan,
Buchanan, Ph.D.,
Ph.D., Principal
Principal Investigator
Investigator
Bruce
Computer
Computer Science Department
Department
Stanford
Stanford University
University
Byron
Byron W.
W. Brown,
Brown, Ph.D., Co-Principal
Co-Principal Investigator
Investigator
Department
Department of
of Medicine
Medicine
Stanford
Stanford University
University
Daniel
Daniel E. Feldman,
Feldman, Ph.D., M.D.,
M.D., Associate
Associate Investigator
Investigator
Department
Department of
of Medicine
Medicine
Stanford
Stanford University
University
I. SUMMARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
I.
Project Rationale
Rationale
A. Project
The
The goals of
of this
this project
project are related both
both to medical
medical science and artificial
artificial intelligence:
intelligence:
(a) use
use AI
AI methods
methods to allow
allow the informed
informed but
but non-expert
non-expert reader of
of the medical
medical
literature
literature to evaluate
evaluate a randomized
randomized clinical
clinical trial,
trial, and (b)
(b) use
use the interpretation
interpretation of
of the
medical
medical literature
literature as
as a test problem
problem for
for studies
studies of
of knowledge
knowledge acquisition
acquisition and fusion
fusion of
of
REFEREE and REVIEWER,
REVIEWER, a planned
planned extension,
extension,
information
information from
from disparate
disparate sources.
sources. REFEREE
will
will be used
used to evaluate
evaluate the medical
medical literature
literature of
of clinical
clinical trials
trials to determine
determine the quality
quality
of a clinical
clinical trial,
trial, make
make judgements
efficacy of
of the treatment
treatment proposed,
proposed, and
and
of
judgements on the efficacy
The research is an initial
initial step toward
toward a more
more
synthesize
synthesize rules
rules of
of clinical
clinical practice.
practice. The
general goal - building
building computer
computer systems to help
help the clinician
clinician and medical
medical scientist
scientist
read the medical
medical literature
literature more
more critically
critically and more
more rapidly.
rapidly.
Medical Relevance
Relevance
B. Medical
The
The explosive
explosive growth
growth of
of the medical
medical literature
literature has created a severe information
information gap for
for
the busy clinician.
Most
Most physicians
physicians can afford
afford neither
neither the time
time required
required to study
study all
all
clinician.
journal articles
the pertinent
pertinent journal
articles in their
their field,
field, nor
nor the risk
risk of
of ignoring
ignoring potentially
potentially
significant
significant discoveries.
discoveries. The
The majority
majority of
of clinicians,
clinicians, in fact,
fact, have little
little sophistication
sophistication in
in
epidemiology
epidemiology and statistics;
statistics: they must
must nonetheless
nonetheless base
base their
their pragmatic
pragmatic decisions
decisions on a
The clinician's
clinician’s
combination
of
experience
The
combination
of clinical
clinical
experience and published
published literature.
literature.
computerized
computerized assistant
assistant must
must ferret
ferret out
out useful
useful maxims
maxims of
of clinical
clinical practice
practice from
from the
the
medical
judgment on the quality
medical literature,
literature, pass
pass judgment
quality of
of medical
medical reports,
reports, evaluate
evaluate the efficacy
efficacy
of
of
and even
of proposed
proposed treatments,
treatments, and adjudicate
adjudicate the interpretation
interpretation
of conflicting
conflicting
contradictory studies.
contradictory
C. Highlights
Highlights of
Progress
of Progress
REFEREE,
REFEREE, a rule-based
rule-based system built
built upon the EMYCIN
EMYCIN framework,
framework,
the epidemiological
knowledge
epidemiological
knowledge of
of two
two highly
highly regarded experts
experts
biostatistician (Dr.
(Dr. Bill
Bill Brown)
Brown) and a clinician
clinician (Dr.
(Dr. Dan
Dan Feldman).
Feldman).
biostatistician
in particular,
allows the informed
informed but
but non-expert
non-expert reader
reader
system, in
particular, allows
literature
literature to evaluate
evaluate the believability
believability of
of a randomized
randomized clinical
clinical trial.
trial.
partially
partially encodes
at Stanford,
Stanford, a
The REFEREE
REFEREE
The
of
of the medical
medical
In
In the future,
future, REFEREE
REFEREE and its extensions
extensions will
will alleviate
alleviate the knowledge-acquisition
knowledge-acquisition
187
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Pilot Stanford
Stanford Projects
Projects
Pilot
for an automated
automated medical
medical decision-maker:
decision-maker: the
the program
program will
will evaluate
evaluate the
the
bottleneck for
bottleneck
of a clinical
clinical trial,
trial, judge
judge the
the efficacy
efficacy of
of the
the treatment
treatment proposed
proposed therein,
therein, and
and
quality of
quality
of clinical
clinical practice.
practice. For
For the present,
present, however,
however, the
the fusion
fusion of
of knowledge
knowledge
synthesize rules
rules of
synthesize
from disparate
disparate sources remains
remains a problem
problem in
in pure
pure AI.
AI. The
The efforts
efforts of
of the
the REFEREE
REFEREE
from
team have instead
instead focused
focused their
their efforts
efforts on
on the
the refinement
refinement and
and deepening
deepening of
of
team
REFEREE's
biostatistical knowledge
knowledge by
by applying
applying effective
effective knowledge
knowledge acquisition
acquisition and
REFEREE’
s biostatistical
knowledge engineering
engineering techniques.
techniques. Dr.
Dr. Diana
Diana Forsythe
Forsythe and Dr.
Dr. Harold
Harold Lehman
Lehman are
knowledge
from Dr.
Dr. Brown,
Brown, and
developing and using
using interview
interview methods
methods to
to acquire
acquire this
this knowledge
knowledge from
developing
Martin Chavez
Chavez is implementing
implementing this
this in
in the prototype
prototype REFEREE
REFEREE expert
expert system.
R. Martin
The REFEREE
REFEREE prototype
prototype is a consultant
consultant that
that evaluates the
the design
design and reporting
reporting of
of a
The
from randomized
randomized control
control trial
trial for
for its
its believability.
believability. ItIt contains,
contains, in
in
single conclusion
conclusion from
single
REFEREE
preliminary form,
form, Professor
Professor Brown’
Brown's
expert knowledge
knowledge of
of biostatistics.
biostatistics.
REFEREE
s expert
preliminary
evaluates each statistical
statistical procedure
procedure described
described by
by the authors
authors of
of the
the paper.
paper.
The
The
evaluates
for the problem
problem at
at
automated consultant
consultant then
then determines
determines the most
most appropriate
appropriate method
method for
automated
hand, based on the design of
of the trial
trial and the hypotheses to be tested. REFEREE
REFEREE
hand,
critical assumptions,
assumptions, looks
looks for
for possible
possible statistical
statistical abuses,
abuses, verifies
verifies adjustments,
adjustments,
checks critical
that employs
employs the Cox
Cox
re-computes the statistics.
statistics.
In a beta-blocker
beta-blocker study
study that
In
and re-computes
proportional-hazards model,
model, for
for instance,
instance, REFEREE
REFEREE will
will analyze
analyze the Kaplan-Meier
Kaplan-Meier
proportional-hazards
survival curve
curve and verify
verify or
or reject
reject the presence of
of a significant
significant treatment
treatment effect.
effect.
survival
of a statistical
statistical test,
The Knowledge
Knowledge Base: In
In order
order to
to evaluate
evaluate the paper’
paper'ss presentation
presentation of
The
REFEREE must
must apply
apply three
three kinds
kinds of
of knowledge:
knowledge:
REFEREE
l.
statistical techniques
techniques that
that are relevant
relevant to the kinds
kinds of
of data likely
likely to be
1. the statistical
found in
in a randomized
randomized clinical
clinical trial.
trial.
found
2. the methods
methods to perform
perform statistical
statistical tests to verify
verify the paper's
paper’s results.
results.
3. the techniques
techniques to test hypotheses, to determine
determine if
if the data in
in a paper
paper support
support
the conclusions
conclusions of
of that
that paper.
Randomized
Randomized controlled
controlled trials
trials are used
used to test hypotheses regarding
regarding the effectiveness
effectiveness of
of
various
Dr.
various kinds
kinds of
of medical
medical interventions.
interventions.
Dr. Brown
Brown classifies
classifies studies
studies on the basis of
of
three
three major
major attributes:
attributes: the type of
of intervention
intervention tested (e.g. drug,
drug, surgery,
surgery, health
health process
was tested (e.g.
change, etc.); the type of
of endpoint
endpoint against
against which
which that
that intervention
intervention
mortality,
mortality, objective
objective morbidity,
morbidity, subjective
subjective morbidity,
morbidity, etc.); and the type of
of conclusion
conclusion
on the basis of
drawn
drawn by
by the investigator/author
investigator/author
of the research (e.g. that
that different
different
treatments
treatments do or
or do not
not produce
produce different
different outcomes,
outcomes, that
that a particular
particular treatment
treatment is or
or is
not
Foilowing this
this classificatory
classificatory scheme, we decided
decided to begin
begin by
not cost-effective,
cost-effective, etc.). Following
producing
producing a prototype
prototype REFEREE
REFEREE system that
that would
would help
help the reader to evaluate
evaluate a single
single
published
published conclusion
conclusion concerning
concerning the effect
effect of
of a given
given drug treatment
treatment on mortality.
mortality.
Knowledge Acquisition:
Having defined
defined the scope of
of the initial
initial knowledge
knowledge base,
base, we
Acquisition: Having
turned
turned to the problem
problem of
of collecting
collecting the information
information from
from Dr.
Dr. Brown
Brown for
for inclusion
inciusion in
the system, i.e. knowledge
This
knowledge acquisition.
acquisition.
This task generally
generally involves
involves a relatively
relatively longIongterm
term process of
of face-to-face
face-to-face information
information gathering
gathering during
during sessions between the expert
expert
and one or
or more
more knowledge
knowledge engineers. Dr.
Dr. Diana
Diana Forsythe
Forsythe has noted
noted a parallel
parallel between
the communicative
communicative and analytical
analytical tasks involved
involved in
in knowledge
knowledge acquisition
acquisition and those
undertaken
undertaken in
in ethnographic
ethnographic research. For
For this reason, we included
included an anthropologist
anthropologist in
the research team and make use of
of ethnographic
ethnographic techniques
techniques in order
order to maximize
maximize the
efficiency
efficiency and quality
quality of
of the data collection
collection process.
process.
Dr..
Lehmann and Dr.
Dr.,Lehmann
Dr. Forsythe
Forsythe have carried
carried out
out several months
months of
of systematic
systematic interviews
interviews
with
with Dr.
Dr. Brown
Brown in
in order
order to begin the process
process of
of constructing
constructing and
and refining
refining the
knowledge
knowledge base
base for
for the current
current REFEREE
REFEREE prototype.
prototype. We have combined
combined a case-based
approach
as
approach that allows
allows us
us actively
actively to observe Dr. Brown
&Own
as he reads papers, with
with semisemi-
E. H. Shortliffe
Shortliffe
188
Pilot Stanford
Stanford Projects
Projects
Pilot
5P41-RR00785-14
5P41-RR00785-14
directed interviewing
interviewing oriented
oriented toward
toward understanding
understanding his
his terminology
terminology and
and category
category
directed
system. We
We find
find that
that these
these techniques
techniques work
work very
very well:
well: Dr.
Dr. Brown’
Brown's
interest in
in the
the
s interest
system.
knowledge acquisition
acquisition process
process has
has been
been sustained,
sustained, and
and indeed
indeed has
has increased
increased over
over time
time as
as
knowledge
the
system
based
on
his
expertise
has
evolved.
He
is
clearly
comfortable
with
this
He
is
clearly
comfortable
with
this
the system based on his expertise has evolved.
approach, and
and notes
notes that
that itit has actually
actually afforded
afforded him
him additional
additional insight
insight into
into the
the way
way he
approach,
interprets the
the literature.
literature.
interprets
In order
order to
to codify
codify the
the information
information gathered
gathered from
from Dr.
Dr. Brown,
Brown, Dr.
Dr. Lehmann
Lehmann chose a
In
model
based
on
the
influence
diagrams
used
in
decision
analysis,
in
which the
the expert
expert
model
on the influence diagrams
in decision analysis, in which
indicates which
which factors
factors or
or parameters
parameters he finds
finds crucial
crucial in
in making
making his
his judgement
judgement about
about
indicates
the quality
quality of
of the
the paper.
paper.
on information
information from
from our
our expert,
expert, we have taken
taken
Based on
the
"believabili ty" as the
the primary
primary parameter
parameter of
of the
the present
present system, defined
defined operationally
operationally by
by
“believability”
Dr.
Brown
as
"the
odds
I
am
willing
to
give
that
the
conclusions
of
the
paper
would
Dr. Brown
“the odds I am willing to give that the conclusions of
paper would be
replicated in
in an experiment
experiment based on
on the
the methods
methods reported
reported in
in the
the paper
paper but
but without
without
replicated
any of
of the
the flaws”.
flaws". Within
Within the
the influence
influence diagram,
diagram, parameters
parameters are connected
connected to each
any
other in
in a structure
structure indicating
indicating the
the information
information considered
considered by Dr.
Dr. Brown
Brown in
in making
making
other
particular
judgments.
In
assessing
believability,
for
instance,
he
considers
In assessing believability,
for instance,
considers the
particular judgments.
acceptability of
of the randomization,
randomization, the quality
quality of
of the blinding,
blinding, other
other sources .of
.of bias,
acceptability
and how
how well
well the results
results substantiate
substantiate the conclusion.
conclusion. Our
Our use
use of
of influence
influence diagrams
diagrams has
and
numerous advantages: the approach
approach is acceptable
acceptable to
to DT.
Dr. Brown,
Brown, itit is flexible,
flexible, itit can
numerous
represent several aspects of
of the structure
structure of
of the
the knowledge
knowledge used
used by the expert,
expert, and the
represent
resultant data can be entered
entered easily
easily into
into the computer.
computer.
resultant
Once entered
entered into
into the machine,
machine, the influence
influence diagram
diagram is converted
converted into
into rules
rules such
such as
as
c)nce
following:
the following:
If
If
of
randomization
is
and
: The
The quality
quality
of the
the
randomization
is high
high
and
The
of
blinding is
poor and
The quality
quality
of the
the
blinding
is poor
and
The
sources
of
bias are
unknown and
The other
other
sources
of bias
are
unknown
and
The
substantiate
the
conclusion,
The results
results
substantiate
the
conclusion,
Then:
is
evidence
(0.7)
that
the
believability of
Then
: There
There
is suggestive
suggestive
evidence
(0.7)
that
the
believability
of the
the
clinical
trial
is
clinical
trial
is high.
high.
The
The number
number (0.7) captures
captures
conclusion
conclusion from
from the specific
specific
The
The mathematics
mathematics of
of certainty
certainty
the uncertainty
uncertainty
antecedents; this
this
factors
factors has been
of
of the expert
expert in drawing
drawing a specific
specific
number
factor.
number is known
known as
as a certainlY
certainty facror.
widely
widely discussed
discussed in
in the literature.
literature.
Inference
built
an AI
Inference in REFEREE:
REFEREE: REFEREE
REFEREE was
was originally
originally
buiit within
within EMYCIN,
EMYCIN,
AI
environment
from
In 1986 Chavez introduced
introduced some
environment developed
developed
from MYCIN
MYCIN at Stanford.
Stanford.
fundamental
improvements
fundamental
improvements to the REFEREE
REFEREE program;
program; among other
other things,
things, these
these
changes greatly
with
greatly improved
improved communication
communication
with the user
user lsee
(see "The
“The User Interface",
Interface”,
below).
below).
The system is programmed
programmed to act as
as aa problem
problem solver, following
following the rules in the
knowledge
For instance, the machine
machine has
has the
knowledge base
base in aa backwards
backwards chaining
chaining path. For
determination
determination of
of the paper's
paper’s believability
believability as
as its goal.·
goal.. At
At the outset it
it finds
finds aa rule that
that
reasons
It then examines each
each
reasons about
about the paper's
paper’s believability
believability (the
[the above example).
example).
antecedent
antecedent of
of that rule in turn
turn and looks for
for rules that draw aa conclusion
conclusion on that
that
parameter,
parameter, recursively,
recursively, until
until an
an antecedent
antecedent isis found
found that has
has no rules. REFEREE
REFEREE then
queries
instance, from
from the
the rule "If
“If the method
method of
of
queries the user
user about
about that antecedent.
antecedent. For instance,
randomization
randomization was
was reported
reported and
and the design
design of
of the randomization
randomization was
was good
good and
and the
implementation
implementation of
of the randomization
randomization was
was poor -- Then
Then there is
is suggestive
suggestive evidence
evidence (.6)
(.6)
that quality
quality of
of the randomization
randomization method
method was
was acceptable",
acceptable”, the machine
machine would
would find
find that
there are
are no rules that conclude
conclude that the method
method of
of randomization
randomization was
was reported.
reported. It
It
would
would then
then ask
ask the
the user,
user, "Was
“Was the
the method
method of
of randomization
randomization reported?"
reported?” If
If the
the answer
answer isis
"No",
the
but saves
saves [he
the response
response for
for
“No”, [hen
then the
the machine
machine abandons
abandons
the rule
rule :n
:n question.
question, but
189
189
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Pilot Stanford
Stanford Projects
Projects
Pilot
other rules.
rules. Note
Note how
how this
this differs
differs from
from aa traditional
traditional paper-andpaper-andpossible use
use with
with other
possible
pencil checklist,
checklist, for
for instance,
instance, where
where the
the user
user is
is confronted
confronted with
with each
each question
question regardless
regardless
pencil
of its
its relevance.
relevance.
of
The User
User Interface:
Interface: The
The first
first versions
versions of
of REFEREE
REFEREE were
were written
written to
to be used with
with a
The
terminal connected
connected to
to a large
large mainframe
mainframe computer.
computer. In
In the
the past
past year
year Chavez
Chavez has
terminal
transformed the
the program
program so as to
to function
function at
at a stand-alone
stand-alone workstation.
workstation. His
His first
first new
new
transformed
in an
an commercial
commercial expert
expert system
system shell
shell (KEE)
(KEE) which
which rested
rested on
on an
version was written
written in
version
INTERLISP base; however,
however, we then
then re-wrote
re-wrote the
the program
program for
for the
the Texas
Texas Instrument
Instrument
INTERLISP
Explorer in
in CommonLisp.
Common Lisp.
Explorer
The program
program code is
is now
now entirely
entirely independent
independent of
of the
the knowledge
knowledge required
required for
for reading
reading
The
REFEREE
has
a
new
interface
that
is
intuitive
and
consistent.
There
papers.
papers. REFEREE
new interface that is intuitive and consistent. There is an
innovative consultation
consultation mode
mode in
in which
which questions
questions are presented
presented in
in free-format
free-format menus.
menus.
innovative
The dialogues
dialogues are
are mixed-initiative
mixed-initiative and
and of
of mixed
mixed levels,
levels, allowing
allowing the
the user such options
options
The
more detailed
detailed questions
questions or
or cutting
cutting off
off apparently
apparently fruitless
fruitless lines
lines of
of
as requesting
requesting more
questioning.
With
the
new
REFEREE
prototype,
the
user
interacts
with
the
machine
With the new REFEREE prototype, the
interacts with
machine
questioning.
mouse-pointing device,
device, as with
with the
the Macintosh.
Macintosh. All
All questions
questions are asked in
in a
using a mouse-pointing
using
similar format.
format. Finally,
Finally, the
the screen enables
enables the
the user to orient
orient himself
himself at all
all times,
times,
similar
obviating the
the need for
for special
special commands
commands to help
help the user “navigate”
"navigate" through
through the
the
obviating
Our expert
expert recently
recently provided
provided the best indication
indication of
of the
the useability
useability of
of
knowledge base. Our
knowledge
this new
new system. After
After only
only a brief
brief introduction
introduction to
to the new machine
machine and interface,
interface, he
this
for the
the first
first time
time - to run
run an entire
entire consultation
consultation by himself.
himself.
able - for
was able
Current Status:
Status: At
At this
this point,
point, REFEREE
REFEREE is a stable
stable prototype
prototype that
that enables the clinician
clinician
Current
REFEREE represents
represents only
only the first
first step
to read clinical
clinical trials
trials more
more critically.
critically. As
As such, REFEREE
to
in a larger
larger research plan,
plan, the automation
automation of
of knowledge
knowledge acquisition
acquisition (see section
section on
in
Research Plans,
below). Current
Plans, below).
Current work
work in
in the restricted
restricted domain
domain of
of clinical
clinical trials
trials will,
will, we
hope, illustrate
principles in
illustrate general
general principles
in the design of
of decision
decision makers
makers that
that gather
gather expertise
expertise
from
from written
written text
text and multiple
multiple knowledge
knowledge sources.
Relevant Publications
Publications
D. Relevant
prototypes for
for assessing the quality
Haggerty,
Haggerty, J.: REFEREE
REFEREE and RULECRITIC:
RULECRITIC: Two prototypes
quality of
of
a medical
paper. REPORT
Master's
medical paper.
REPORT KSL-84-49.
KSL-84-49.
Master’s Thesis,
Thesis, Stanford
Stanford University,
University, May
May 1984.
E. Funding
Funding Support
Support
REFEREE
REFEREE currently
currently receives only
only a small
small amount
amount of
of funding.
funding. Most
Most of
of the research is
performed
performed in
in time
time contributed
contributed by the researchers to this
this project.
project.
Title:
Title: Knowledge-Based
Knowledge-Based Systems Research
PI: Edward
Edward A. Feigenbaum
Feigenbaum
Agency:
Agency: Defense
Defense Advanced
Advanced Projects
Projects Research Agency
Agency
Grant
number:
Grant identification
identification
number: N00039-86-0033
N00039-86-0033
Total
Total award period
period and amount:
amount:
and indirect)
indirect)
10/1/85
10/l/85 - 9/30/88
9/30/88 $4,130,230 (in
(in negotiation)
negotiation) (direct
(direct
Current
Current award
award period
period and
and amount:
amount: 10/1/86
10/l/86 -- 9130/87
9/30/87 $1,549,539
$1,549,539 (direct
(direct and
and indirect)
indirect)
REFEREE
REFEREE component
component is $29,296,
$29,296, or 1.9
1.9 %
% of
of grant
grant total.
total.
E.
E. H.
H. Shortliffe
Shortliffe
190
190
Pilot
Pilot Stanford
Stanford Projects
Projects
5P41-RR00785-14
5P41-RR00785-14
II. INTERACTIONS
INTERACTIONS WITH
WITH THE
THE SUMEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
II.
A. Medical
Medical Collaborations
Collaborations
A.
Dr. Brown
Brown and Dr.
Dr. Feldman
Feldman of
of the
the Stanford
Stanford University
University School
School of
of Medicine
Medicine are actively
actively
Dr.
involved in
in the
the REFEREE
REFEREE project
project and
and are the
the primary
primary domain
domain experts
experts for
for this
this project.
project.
involved
Critique of
of Resource
Resource Management
Management
C. Critique
The SUMEX
SUMEX computer
computer resource
resource and Lisp
Lisp workstations
workstations have been very
very important
important for
for the
The
work to
to date, and the SUMEX
SUMEX staff
staff has continued
continued to be very
very cooperative
cooperative with
with the
work
REFEREE project.
project.
REFEREE
III. RESEARCH
RESEARCH PLANS
PLANS
III.
& Plans
Plans
A. Goals
Goals &
The overall
overall objective
objective of
of the REFEREE
REFEREE project
project is to use
use recent
recent Artificial
Artificial Intelligence
Intelligence
The
techniques to build
build a system that
that helps the informed
informed but
but statistically
statistically non-expert
non-expert reader
reader
techniques
evaluate critically
critically the medical
medical literature
literature on
on randomized
randomized controlled
controlled trials
trials (RCT’
(RCT's).
to evaluate
s).
This system will
will contain
contain and be able
able to apply
apply dynamically
dynamically the detailed
detailed specialized
specialized
This
knowledge of
of Dr.
Dr. Byron
Byron W.
W. Brown,
Brown, a biostatistician
biostatistic ian expert
expert in
in the design and evaluation
evaluation
knowledge
of randomized
randomized controlled
controlled trials.
trials. We
We have divided
divided our
our overall
overall objective
objective into
into two
two goals:
of
.• Goal
Goal 1 is the construction
construction of
of an expert
expert system to
to help
help readers (e.g. medical
medical
students, medical
medical researchers, clinicians,
clinicians, journal
journal editors,
editors, or
or editorial
editorial
students,
assistants) assess
assess the credibility
credibility of
of a single
single conclusion
conclusion drawn
drawn from
from a single
single
assistants)
already made
journal report
We have already
journal
report of
of a randomized
randomized controlled
controlled trial.
trial.
substantial
substantial progress toward
toward this
this goal with
with the development
development of
of the prototype
prototype
REFEREE
REFEREE system.
Goal 2 is the expansion
expansion of
of REFEREE
REFEREE to an expert
expert system that
that can be used
used
•. Goal
by a similar
similar range of
of readers to facilitate
facilitate the evaluation
evaluation of
of multiple
multiple reports
reports
based
This expanded
expanded system, to be known
known
based on randomized
randomized controlled
controlled trials.
trials. This
as the REVIEWER,
REVIEWER, will
will thus perform
perform meta-analysis.
meta-analysis.
The
The task of
of extending
extending and refining
refining the prototype
prototype REFEREE
REFEREE system in
in order
order to achieve
achieve
these goals can be characterized
characterized in
in terms
terms of
of three
three dimensions:
dimensions:
Making the system more
more accessible to a variety
variety of
of people
people by improving
improving the
•. Making
user interface,
interface, validating
validating the system's
system’s performance
performance with
with different
different types of
of
users,
users, Clnd
2nd providing
providing Cln
an explanatory
explanatory capability
capability
•. Expanding
Expanding the knowledge
knowledge base
base by continuing
continuing the knowledge
knowledge acquisition
acquisition
process to cover
cover additional
additional types of
of RCT's
RCT’s
Improving the inference
inference engine
engine to ensure consistency
consistency of
of the knowledge
knowledge base
base
•. Improving
and to focus
focus the consultation
consultation process on questions
questions relevant
relevant to the situation
situation
and the individual
individual user.
The
The specific
specific steps that
that are planned
planned for
for the enhancement
enhancement of
of the REFEREE
REFEREE system
include
include the following:
following:
1. Critique
Critique individual
individual clinical
clinical trials
trials according
according to the methodological
methodological quality
quality of
of
the trial;
trial:
2.
2. Measure the efficacy
zcficacy of
of treatment
treatment as
as demonstrated
demonstrated in a randomized
randomized control
control
trial;
rrlal;
191
E. H. Shortliffe
Shortliffe
E.
SP41-RR0078S-14
5P41-RR00785-14
Pilot
Pilot Stanford
Stanford Projects
Projects
Compare and contrast
contrast the credibility
credibility and efficacy
efficacy of
of treatment
treatment reported
reported by
3. Compare
multiple journal
articles; and
multiple
journal articles;
4. Combine
the qualitative
techniques
reasoning
Combine
qualitative
techniques of
of heuristic
heuristic
reasoning and the
quantitative
methods
quantitative
methods of
of statistical
statistical meta-analysis
meta-analysis to extract
extract a consensus
consensus
opinion
opinion from
from multiple
multiple knowledge
knowledge sources.
In
In addition,
addition, plans
plans for
for Goal
Goal 2, the
the REVIEWER
REVIEWER system to analyze
analyze multiple
multiple RCT's
RCT’s and
form
judgment, include:
form a consensus judgment,
include:
1. Complete
Complete a review
review of
of the available
available literature
literature on meta-analysis
meta-analysis and augment
augment
the REFEREE
prototype
REFEREE
prototype to produce
produce estimators
estimators for
for meta-analysis
meta-analysis and
incorporate expert
expert knowledge
knowledge on
on the appropriateness
appropriateness ~f
of these methods.
methods.
incorporate
Add explicit
explicit and heuristic
heuristic knowledge
knowledge needed for
for the calculation
calculation of
of robust,
robust,
2. Add
non-parametric
non-parametric estimators
estimators of
of effect
effect size.
3. Construct
Construct a prototype
prototype of
of a system that
that builds
builds categorical
categorical models
models in
in the
domain
in
domain of
of meta-analysis,
meta-analysis, to perform
perform autonomous
autonomous investigations
investigations
in the
The REVIEWER
REVIEWER
will utilize
utilize expert
expert
domain
The
will
domain of
of statistical
statistical model-building.
model-building.
knowledge in
in biostatistics
biostatistics to guide
guide its
its search for
for meaningful
meaningful models.
models.
knowledge
4. Build
Build a prototype
prototype of
of a system that
that can explore
explore the domain
domain of
of regression
regression
models
models for
for multiple
multiple RCT's
RCT’s that
that will
will use
use expert
expert knowledge
knowledge in
in its selection
selection of
of
predictor
predictor variables.
variables.
S.
5. Package the REVIEWER
REVIEWER in
in a form
form suitable
suitable for
for use
use by physicians
physicians and their
their
assistan
ts.
assistants.
6. Verify
Verify the expertise
expertise of
of the REVIEWER
REVIEWER system on a suite
suite of
of papers drawn
drawn
from clinical
clinical trials,
trials, similar
similar to the validation
validation of
of REFEREE
REFEREE above.
from
Justification for
for continued
B. Justification
continued SUMEX
SUMEX use
The local
local area network
network maintained
maintained by the SUMEX
SUMEX staff
staff is essential
essential to the effective
effective
The
development and use
use of
of the REFEREE
REFEREE system on Lisp
Lisp workstations.
workstations.
The availability
availability
development
The
of
makes possible
of the Xerox
Xerox workstations
workstations
possible the evaluation
evaluation of
of prototypes
prototypes in
in that
that
environment,
facilitates the development
development of
of good user interfaces.
interfaces.
The
environment,
and also facilitates
The
connections through
through the 2060 to local
local and national
national computer
computer networks
networks such as
as
connections
ARPAnet are important
important for
for sharing
sharing ideas and results
results with
with other
other medical
medical researchers.
ARPAnet
other computing
computing resources
resources
C. Need
Need for
for other
The REFEREE
REFEREE project
project needs access
access to an additional
additional high
high performance
performance Lisp
Lisp workstation
workstation
The
in the development
development and execution
execution of
of the REFEREE
REFEREE programs.
programs.
to assist in
Such a
machine is important
important to explore
explore user interface
interface issues,
issues, in
in addition
addition to building
building the
machine
knowledge base
base for
for current
current and planned
planned development.
development. In
In addition,
addition, we intend
intend to explore
explore
knowledge
implementation
of REFEREE
REFEREE on
on less expensive
expensive personal
personal computers
computers such as
as the
the implementation
of
Macintosh
Macintosh II
II and other
other high
high performance
performance machines.
machines. We
We anticipate
anticipate the need for
for at least
two
two of
of these machines
machines for
for transporting
transporting our
our system and developing
developing new modes of
of
interaction with
with both
both naive
naive and experienced
experienced users.
interaction
E. H. Shortliffe
Shortliffe
192
Pilot
Pilot AIM
AIM Projects
Projects
5P41-RR00785-14
5P41-RR00785-14
IV.D. Pilot
Pilot AIM
AIM Projects
Projects
1V.D.
Following is
is a description
description of
of the
the informal
informal pilot
pilot project
project currently
currently using
using the
the AIM
AIM portion
portion
Following
of the
the SUMEX-AIM
SUMEX -AIM resource,
resource, pending
pending funding,
funding, full
full review,
review, and
and authorization.
authorization.
of
In addition
addition to
to the
the progress
progress report
report presented
presented here,
here, an abstract
abstract is submitted
submitted on
on a separate
separate
In
Form.
Scientific Subproject
Subproject Form.
Scientific
193
193
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
PATHFINDER Project
Project
PATHFINDER
IV.D.I. PATHFINDER
PATHFINDER Project
Project
IV.D.1.
PATHFINDER Project
Project
PATHFINDER
Bharat Nathwani,
Nathwani, M.D.
M.D.
Bharat
of Pathology
Pathology
Department of
Department
of Southern
Southern California
California
University of
University
M. Fagan, M.D.,
M.D., Ph.D.
Ph.D.
Lawrence M.
Lawrence
of Medicine
Medicine
Department of
Department
Stanford University
University
Stanford
I.
I.
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
SUMMARY
Project Rationale
Rationale
A. Project
Our project
project addresses
addresses difficulties
difficulties in
in the diagnosis
diagnosis of
of lymph
lymph node
node pathology.
pathology. Five
Five studies
studies
Our
from cooperative
cooperative oncology
oncology groups
groups have documented
documented that,
that, while
while experts
experts show
show agreement
agreement
from
with one another,
another, the diagnosis
diagnosis made by practicing
practicing pathologists
pathologists may
may have to
to be changed
changed
with
expert hematopathologists
hematopathologists in
in as many
many as 50% of
of the cases.
cases. Precise
Precise diagnoses are
by expert
crucial for
for the determination
determination of
of optimal
optimal treatment.
treatment.
To make the knowledge
knowledge and
and
To
crucial
diagnostic reasoning
reasoning capabilities
capabilities of
of experts
experts available
available to the practicing
practicing pathologist,
pathologist, we
diagnostic
have developed
developed a pilot
pilot computer-based
computer-based diagnostic
diagnostic program
program called
called PATHFTNDER.
PATHFINDER. The
The
project is a collaborative
and the
project
collaborative effort
effort of
of the University
University of
of Southern
Southern California
California
Stanford
pilot version
program
Stanford University
University Medical
Medical Computer
Computer Science Group.
Group. A
A pilot
version of
of the program
provides diagnostic
benign and malignant
provides
diagnostic advice
advice on 72 common
common benign
malignant diseases
diseases of
of the lymph
lymph
node based
based on 110 histologic
plans are to develop
Our research plans
develop a full-scale
full-scale
histologic features.
features. Our
version
program by substantially
version of
of the computer
computer program
substantially increasing
increasing the quantity
quantity and quality
quality of
of
knowledge
knowledge and to develop
develop techniques
techniques for
for knowledge
knowledge representation
representation and manipulation
manipulation
appropriate
program has
The design of
of the program
has been strongly
strongly
appropriate to this
this application
application area. The
influenced
ICADUCEUS program
program developed
on
influenced by the INTERNIST
INTERNIST/CADUCEUS
developed
on the
the SUMEX
SUME.Y
resource.
PATHFINDER
computer
PATHFINDER
computer science research is focused
focused on the exploration
exploration and extension
extension
of
of formal
formal techniques
techniques for
for decision
decision making
making under
under uncertainty.
uncertainty. Research foci
foci include
include (1)
(1)
the assessment
probabilistic
dependencies
assessment and representation
representation of
of important
important
probabilistic
dependencies among
among
morphologic
morphologic features
features and diseases,
diseases, (2) the representation
representation of
of knowledge
knowledge about
about the
progression
progression of
of disease
disease over
over time,
time, (3) the acquisition
acquisition and use
use of
of independent
independent expert
expert
knowledge
knowledge bases,
bases, (4) the customization
customization of
of the system's
system’s reasoning
reasoning and
and explanation
explanation
behaviors
behaviors to reflect
reflect the expertise
expertise of
of the user.
user, and, (5) the explanation
expianation of
of complex
complex
formal
formal reasoning
reasoning techniques.
techniques.
Toward
Toward the pragmatic
pragmatic goal of
of constructing
constructing a useful
useful pathology
pathology teaching
teaching and decision
decision
support
THFINDER investigators
are attempting
to use intelligen
support system, PA
PATHFINDER
investigators
attempting
intelligent t
computation
computation to substantially
substantially increase the quantity
quantity and quality
quality of
of pathology
pathology knowledge
knowledge
available
Important
available to pathologists.
pathologists.
Important areas
areas of
of this
this knowledge
knowledge integration
integration task involve
involve
ongoing
ongoing research on the crisp
crisp definition
definition important
important morphologic
morphologic features
features and feature
feature
severities,
severities, the synthesis
synthesis of
of information
information from
from multiple
multiple experts, the translation
translation among
among
multiple
of
multiple pathology
pathology classification
classification schemes,
schemes, and the incorporation
incorporation
of knowledge
knowledge about
about
advances in immunology,
immunology, cytogenetics,
cytogenetics, cell kinetics,
kinetics, and immunogenetics.
immunogenetics.
A
A group
group of
of expert
expert pathologists
pathologists from
from several centers in the U.S. have showed interest
interest in
the program
program and helped to provide
provide the structure
structure of
of the knowledge
knowledge base
base for
for the
PATHFINDER
PATHFINDER system.
E.
E. H. Shortliffe
Shortliffe
194
PATHFINDER Project
Project
PATHFINDER
5P41-RR00785-14
5P41-RR00785-14
B. Medical
Medical Relevance
Relevance and
and Collaboration
Collaboration
B.
One of
of the
the most
most difficult
difficult areas in
in surgical
surgical pathology
pathology is the
the microscopic
microscopic interpretation
interpretation
One
of lymph
lymph node
node biopsies.
biopsies. Most
Most pathologists
pathologists have
have difficulty
difficulty in
in accurately
accurately classifying
classifying
of
that while
while
lymphomas. Several cooperative
cooperative oncology
oncology group
group studies
studies have documented
documented that
lymphomas.
experts show
show agreement
agreement with
with one
one another,
another, the
the diagnosis
diagnosis rendered
rendered by
by a “local”
"local"
experts
pathologist may
may have to
to be changed by
by expert
expert lymph
lymph node
node pathologists
pathologists (expert
(expert
pathologist
many as 50% of
of the
the cases.
cases.
hematopathologists) in
in as many
hematopathologists)
The National
National Cancer
Cancer Institute
Institute recognized
recognized this
this problem
problem in
in 1968 and created
created the
The
Lymphoma Task
Task Force
Force which
which is now
now identified
identified as the
the Repository
Repository Center
Center and
and the
Lymphoma
Pathology Panel
Panel for
for Lymphoma
Lymphoma Clinical
Clinical Studies. The
The main
main function
function of
of this
this expert
expert
Pathology
of pathologists
pathologists is to
to confirm
confirm the
the diagnosis
diagnosis of
of the
the “local”
"local" pathologists
pathologists and
and to
to
panel of
panel
ensure that
that the pathologic
pathologic diagnosis
diagnosis is made uniform
uniform from
from one
one center
center to
to another
another so
that the comparative
comparative results
results of
of clinical
clinical therapeutic
therapeutic trials
trials on
on lymphoma
lymphoma patients
patients are
that
The panel
panel is
valid. An
An expert
expert panel
panel approach
approach is only
only a partial
partial answer to this
this problem.
problem. The
valid.
useful in
in only
only a small
small percentage (3%)
(3%) of
of cases;
cases; the Pathology
Pathology Panel annually
annually reviews
reviews
useful
of lymphomas
lymphomas are reported
reported each
only 1,000 cases
cases whereas more
more than
than 30,000 new cases
cases of
only
cannot
A panel approach
approach to diagnosis
diagnosis is not
not practical
practical and lymph
lymph node
node pathology
pathology cannot
year. A
routinely practiced
practiced in
in this
this manner.
manner.
be routinely
We believe
believe that
that practicing
practicing pathologists
pathologists do not
not see
see enough
enough case
case material
material to maintain
maintain a
We
high level
level of
of diagnostic
diagnostic accuracy. The
The disparity
disparity between the experience
experience of
of expert
expert
high
hematopathology teams and those in
in community
community hospitals
hospitals is striking.
striking. An
An experienced
experienced
hematopathology
of cases
cases per year.
In contrast,
contrast, in
in a
hematopathology team may review
review thousands
thousands of
In
hematopathology
of only
only ten new cases
cases of
of malignant
malignant lymphomas
lymphomas are
community hospital,
hospital, an average of
community
diagnosed each year.
Even in
in a university
university hospital,
hospital, only
only approximately
approximately 100 new
Even
diagnosed
patients
patients are diagnosed every year.
Because
Because of
of the limited
limited numbers
numbers of
of cases
cases seen,
seen, pathologists
pathologists may
may not
not be conversant
conversant with
with
the differential
differential diagnoses consistent
consistent with
with each of
of the histologic
histologic features
features of
of the lymph
lymph
node; they may
may lack
lack familiarity
familiarity with
with the complete
complete spectrum
spectrum of
of the histologic
histologic findings
findings
associated with
In addition,
addition, pathologists
pathologists may
may be unable
unable to
with a wide
wide range of
of diseases.
diseases. In
of
fully
comprehend
concepts
fully
comprehend the conflicting
conflicting
concepts and terminology
terminology
of the different
different
classifications
of
lymphomas,
not be cognizant
cognizant of
of the
classifications
of non-Hodgkin's
non-Hodgkin’s
lymphomas, and may not
data
significance
significance of
of the immunologic,
immunologic, cell
cell kinetic,
kinetic, cytogenetic,
cytogenetic, and immunogenetic
immunogenetic
associated with
with each of
of the subtypes
subtypes of
of the non-Hodgkin's
non-Hodgkin’s lymphomas.
lymphomas.
In
In order
order to promote
promote the accuracy of
of the knowledge
knowledge base development
development we will
will have
participants
participants for
for multiple
multiple institutions
institutions collaborating
collaborating on the project.
project. Dr.
Dr. Nathwani
Nathwani will
will be
joined
joined by experts from
from Stanford
Stanford (Dr.
(Dr. Dorfman),
Dorfman), St. Jude's
Jude’s Children's
Children’s Research Center
Center
--- Memphis
Memphis (Dr.
(Dr. Berard)
Berard) and City
City of
of Hope
Hope (Dr.
(Dr. Burke).
Burke).
C. Highlights
Highlights of
of Research Progress
Progress
C.l
C.i Previous
Previous Accomplishments
Accomplishments
Since the project's
project’s inception
inception in
in September,
September, 1983, we have constructed
constructed several versions
versions of
of
PATHFINDER.
The
PATHFINDER.
The first
first several versions
versions of
of the program
program were rule-based
rule-based systems like
like
MYCIN
MYCIN and ONCOCIN
ONCOCIN which
which were developed
developed earlier
earlier by the Stanford
Stanford group.
group. We
We soon
discovered,
discovered, however, that
that the large number
number of
of overlapping
overlapping features
features in
in diseases
diseases of
of the
lymph
We next
next
lymph node would
would make a rule-based
rule-based system cumbersome
cumbersome to implement.
implement.
considered
considered the construction
construction of
of a hybrid
hybrid system, consisting
consisting of
of a rule-based
rule-based algorithm
algorithm
scoring
that
that would
would pass
pass control
control to an INTERNIST-like
INTERNIST-like
scoring algorithm
algorithm if
if it
it could
could not
not
confirm
confirm the existence of
of classical sets
sets of
of features. We finally
finally decided that
that a modified
modified
form
form of
of the INTERNIST
INTERNIST program
program would
would be most
most appropriate.
appropriate. The
The original
original version
version of
of
PATHFINDER
is written
PATHFINDER
written in the computer
computer language Maclisp
Maclisp and runs on the SUMEX
SUMEX
DEC-20.
DEC-20. This
This was
was transferred
transferred to Portable
Portable Standard Lisp
Lisp (PSL)
(PSL) on the DEC-20,
DEC-20, and
195
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
PA THFINDER Project
Project
PATHFINDER
later transferred
transferred to
to PSL
PSL on
on the
the HP
HP 9836 workstations.
workstations. Two
Two graduate
graduate students,
students, David
David
later
Heckerman and
and Eric
Eric Horvitz,
Horvitz, designed and
and implemented
implemented the
the program
program and
and are
Heckerman
continuing to
to lead
lead research
research on
on the
the project.
project.
continuing
The prototype
prototype knowledge
knowledge base was constructed
constructed by
by Dr.
Dr. Nathwani.
Nathwani. During
During the
the early
early part
part
The
of 1984, we organized
organized two
two meetings
meetings of
of the entire
entire team, including
including the
the pathology
pathology experts,
experts,
of
to define
define the
the selection
selection of
of diseases
diseases to
to be included
included in
in the system, and
and the
the choice
choice of
of
to
features to
to be used in
in the scoring
scoring process.
features
During the
the last
last two
two years, we have focused
focused on
on methodologies
methodologies for
for more
more accurately
accurately
During
representing expert
expert beliefs.
beliefs. In
In particular,
particular, we have used influence
influence diagrams
diagrams to
to represent
represent
representing
dependencies among
among features
features in
in the
the PATHFINDER
PATHFINDER knowledge
knowledge base. A
A great
great deal of
of
dependencies
effort has
has been devoted
devoted to
to assessing
assessing and
and representing
representing the intricate
intricate relationships
relationships among
among
effort
features that
that exist
exist in
in the domain.
domain. We
We believe
believe that
that this
this process will
will help
help to
to overcome
overcome
features
some of
of the limitations
limitations of
of medical
medical diagnostic
diagnostic systems.
some
on the problem
problem of
of complex
complex information-theoretic
information-theoretic inference.
inference. The
The
We have also focused
focused on
We
explanation of
of a systems diagnostic
diagnostic behavior
behavior has been found
found to be of
of extreme
extreme
explanation
importance to physicians.
physicians. Unfortunately,
Unfortunately, itit is often
often difficult
difficult to explain
explain reasoning
reasoning based
importance
optimal models
models of
of inference.
inference. We
We have worked
worked on the use of
of a set of
of alternative
alternative
on optimal
abstraction hierarchies
hierarchies to
to control
control inference.
inference. Our
Our current
current techniques
techniques enable
enable us to
to trade
trade
abstraction
off optimality
optimality for
for the transparency
transparency of
of reasoning.
reasoning. We
We are now
now studying
studying the control
control of
of
off
this tradeoff
tradeoff to optimize
optimize inference.
inference.
this
C.l The PATHFINDER
PATHFINDER knowledge
knowledge base
C.1
The basic
basic building
building block
block of
of the
the PATHFINDER
PATHFINDER knowledge
knowledge base is the disease profile
profile or
or
The
frame. Each
features useful
frame.
Each disease frame
frame consists
consists of
of features
useful for
for diagnosis
diagnosis of
of lymph
lymph node
diseases.
Currently
both
diseases.
Currently these features
features include
include histopathologic
histopathologic findings
findings seen
seen in
in both
lowEach feature
low- and high-power
high-power magnifications.
magnifications.
feature is associated with
with a list
list of
of
exhaustive
pseudofollicularity
exhaustive and mutually
mutually exclusive
exclusive values.
values. For
For example,
example, the feature
feature pseudofollicularity
can take on anyone
slight, moderate,
prominent. These lists
any one of
of the values absent, slight,
moderate, or
or prominent.
lists of
of
values give
program access
severity information.
In
In addition,
addition, these lists
lists
give the program
access to severity
information.
eliminate
eliminate obvious
obvious interdependencies
interdependencies among
among the values for
for a given
given feature.
feature. For
For example,
example,
if
pseudofollicularity
is
moderate,
it
cannot
also
be
absent.
if pseudofollicularity
moderate, it cannot
Qualitative
Qualitative dependencies
dependencies among
among features
features for
for each disease
disease are represented
represented using
using the
influence
influence diagram
diagram methodology
methodology mentioned
mentioned above. An
An influence
influence diagram
diagram contains
contains nodes
and arcs.
arcs. Nodes
Nodes represent
represent features
features and arcs represent
represent dependencies
dependencies among
among features.
features. In
In
particular,
particular, an arc is drawn
drawn from
from one feature
feature to another
another when an expert
expert believes
believes that
that
knowing
knowing one feature
feature can change his beliefs
beliefs that
that another
another feature
feature will
will take on its
its possible
possible
values even when the diagnosis
Probabilities
diagnosis is known.
known.
Probabilities are used
used to quantitate
quantitate the
beliefs
beliefs asserted by the expert.
expert.
C.2 Hewlett-Packard
Workstation
Hewlett-Packard
Workstation
Through
Information
Through the USC-affiliated
USC-affiliated
Information Sciences
Sciences Institute,
Institute, Dr.
Dr. Nathwani
Nathwani has obtained
obtained
a Hewlett-Packard
Hewlett-Packard Workstation
Workstation that
that is similar
similar to the 9836. The
The Pathfinder
Pathfinder program
program has
been brought
brought up on this
this machine.
machine. This
This means that
that the program
program now
now exists
exists on three
three
different
different machines,
machines, in
in three separate locations,
locations, using one standard
standard language (Portable
(Portable
Standard
and communications
Standard Lisp).
Lisp). Thus,
Thus, the need for
for support
support of
of networked
networked machines
machines.and
communications
has
has increased
increased during
during this last year. Current
Current plans are to move
move the system onto
onto the
Macintosh
Macintosh II
II system.
E.
E. H. Shortliffe
Shortliffe
196
196
PATHFINDER
Project
PATHFINDER
Project
5P41-RR00785-14
5P41-RR00785-14
Publications Since
January 1984
D. Publications
Since January
1. Horvitz,
Nathwani, B.N.
Diagnostic
Horvitz, E.J., Heckerman,
Heckerman, D.E.,
D.E., Nathwani,
B.N. and Fagan, L.M.:
L.M.: Diagnostic
Strategies
in
Hypothesis-directed PATHFINDER
PATHFINDER System, Node
Node
Strategies
in the Hypothesis-directed
Pathology. HPP
Proceedings
Pathology.
HPP Memo
Memo 84-l3.
84-13.
Proceedings of
of the First
First Conference
Conference on
Artificial
Artificial Intelligence
Intelligence Applications,
Applications, Denver,
Denver, Colorado,
Colorado, Dec., 1984.
2. Heckerman,
Heckerman, D. E., and Horvitz,
Horvitz, E. J., "The
“The Myth
Myth of
of Modularity
Modularity in
in RuleRulebased Systems,"
Intelligence,
in Uncertainty
Uncertainty in
in Artificial
Artificial
Intelligence, Vol.
Vol. 2, 1.
J. Lemmer,
Lemmer,
Systems,” in
L.
North Holland,
New York,
L. Kanal,
Kanal, ed., North
Holland, New
York, 1987.
3. Horvitz,
Nathwani, B.N.
Horvitz, EJ.,
E.J., Heckerman,
Heckerman, D.E.,
D.E., Nathwani,
B.N. and Fagan, L.M.:
L.M.: The Use of
of
a Heuristic
Heuristic Problem-solving
Problem-solving Hierarchy
Hierarchy to Facilitate
Facilitate the Explanation
Explanation of
of
Hypothesis-directed Reasoning.
Reasoning. KSL
Hypothesis-directed
KSL Memo
Memo 86-2.
86-2. Proceedings
Proceedings of
of MedInfo,
MedInfo,
Washington
Washington D.C., October,
October, 1986.
4. Horvitz,
Horvitz, E. J., "Toward
“Toward a Science of
of Expert
Expert Systems,"
Systems,” Invited
Invited Paper,
Computer
Computer Science and Statistics:
Statistics: Proceedings
Proceedings of
of the 18th
18th Symposium
Symposium on the
Interface,
Interface, American
American Statistical
Statistical Association,
Association, March,
March, 1986, pgs.
pgs. 45-52.
45-52.
5. Heckerman.
Framework
Heckerman, D.E.,
D.E., "An
“An Axiomatic
Axiomatic
Framework for
for Belief
Belief Updates,"
Updates,” in
in
Uncertainty
in
Intelligence,
Uncertainty
in Artificial
Artificial
Intelligence, Vol.
Vol. 2, J. Lemmer,
Lemmer, L. Kanal,
Kanal, ed.,
North Holland,
New York,
North
Holland, New
York, 1987.
Funding Support
E. Funding
Support
Research Grant
National Institutes
Grant submitted
submitted to National
Institutes of
of Health
Health
Grant
Node Diseases"
Grant Title:
Title: "Computer-aided
“Computer-aided Diagnosis
Diagnosis of
of t-.falignant
hjalignant Lymph
Lymph Node
Diseases”
Principal
Nathwani
Principal Investigator:
Investigator: Bharat
Bharat Nathwani
Funding
National Library
Funding for
for three
three years from
from the National
Library of
of Medicine
Medicine
1 ROI
ROl LM
LM 04529
$766,053 (direct
(direct and indirect)
indirect)
Professional
Professional Staff
Staff Association,
Association, Los Angeles
Angeles County
County Hospital,
Hospital, $10,000.
University
University of
of Southern
Southern California,
California, Comprehensive
Comprehensive Cancer
Cancer Center,
Center, $30,000.
Project
Project Socrates, Univ.
Univ. of
of Southern
Southern Calif.,
Calif., Gift
Gift from
from IBM
IBM of
of IBM
IBM PC/XT.
PC/XT.
II.
II.
INTERACTIONS
WITH
RESOURCE
INTERACTIONS
WITH THE
THE SUMEX-AIM
SUMEX-AIM
RESOURCE
Medical Collaborations
Program Dissemination
Dissemination via
A. Medical
Collaborations and Program
via SUMEX
SUMEX
Because
Because our
our team of
of experts
experts are in
in different
different parts of
of the country
country and the computer
computer
scientists
scientists are not
not located
located at the USC, we envision
envision a tremendous
tremendous use
use of
of SUMEX
SUMEX for
for
communication,
communication, demonstration
demonstration of
of programs,
programs, and remote
remote modification
modification of
of the knowledge
knowledge
base.
base. The
The proposal
proposal mentioned
mentioned above was developed
developed using the communication
communication faCilities
facilities
of
of SUMEX.
SUMEX.
Interaction with
Projects
B. Sharing
Sharing and
and Interaction
with Other
Other SUMEX-AIM
SUMEX-AIM
Projects
Our
project
depends
heavily
on
the
techniques
developed
Our
project
heavily
techniques
developed
INTERNIST
/CADUCEUS project.
lNTERNIST/CADUCEUS
project. We have been in
in electronic
electronic contact
contact and
with
project,
with members
members of
of the INTERNIST/CADUCEUS
INTERNIST/CADUCEUS
project, as
as well
well as
as been able
information
and experience
program
information
experience with
with the INTERNIST
INTERNIST
program gathered over
over
Our
through
through Lhe
the AIM
:\IM conferences
conferences and on-line
on-!ine interaction.
interaction.
Our expenence
experience
197
by
the
have met
met
to utilize
utilize
the years
with
with the
r,he
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
PATHFINDER Project
Project
PATHFINDER
extensive development
development of
of the
the pathology
pathology knowledge
knowledge base utilizing
utilizing multiple
multiple experts
experts should
should
extensive
for intense
intense and
and helpful
helpful discussions
discussions between
between our
our two
two projects.
projects.
provide for
provide
The SUMEX
SUMEX pilot
pilot project,
project, RXDX,
RXDX, designed
designed to
to assist in
in the
the diagnosis
diagnosis of
of psychiatric
psychiatric
The
of the
the PATHFINDER
PATHFINDER program
program on
on the
the DEC-20
DEC-20
disorders, is currently
currently using
using a version
version of
disorders,
for the
the development
development of
of early
early prototypes
prototypes of
of future
future systems.
for
Critique of
of Resource
Resource Management
Management
C. Critique
The SUMEX
SUMEX resource
resource has provided
provided an excellent
excellent basis for
for the development
development of
of a pilot
pilot
The
project. The
The availability
availability of
of a pre-existing
pre-existing facility
facility with
with appropriate
appropriate computer
computer languages,
project.
communication facilities
facilities (especially
(especially the TYMNET
TYMNET network),
network), and document
document preparation
preparation
communication
facilities allowed
allowed us to
to make
make good
good progress
progress in
in a short
short period
period of
of time.
time. The
The management
management
facilities
our needs during
during the
the start
start of
of this
this project.
project.
very useful
useful in
in assisting
assisting with
with our
has been very
III. RESEARCH
RESEARCH PLANS
PLANS
III.
A. Project
Project Goals
Goals and
and Plans
Plans
Collection and refinement
refinement of
of knowledge
knowledge about
about lymph
lymph node pathology
pathology
Collection
The knowledge
knowledge base of
of the
the program
program is about
about to undergo
undergo revision
revision by the experts,
experts, and
The
next step would
would be to extend
extend the program
program to
to
then will
will be extensively
extensively tested. A
A logical
logical next
then
as well
well as possible
possible extensions
extensions of
of the knowledge
knowledge base.
base.
clinical settings,
settings, as
clinical
Other possible
possible extensions
extensions include:
include: developing
developing techniques
techniques for
for simplifying
simplifying the acquisition
acquisition
Other
and verification
of
verification
of knowledge
knowledge from
from experts,
experts, and creating
creating mapping
mapping schemes that
that will
will
facilitate
facilitate the understanding
understanding of
of the many
many classifications
classifications of
of non-Hodgkin's
non-Hodgkin’s lymphomas.
lymphomas.
We
We will
will also attempt
attempt to represent
represent knowledge
knowledge about
about special
special diagnostic
diagnostic entities,
entities, such as
as
multiple
multiple discordant
discordant histologies
histologies and atypical
atypical proliferations,
proliferations, which
which do not
not fit
fit into
into the
classification
classification methods
methods we have utilized.
utilized.
Representation
Representation Research
We hope to enhance
model
enhance the INTERNIST-1
INTERNIST-l
model by structuring
structuring features
features so that
that
overlapping
weighted
overlapping features
features are not
not incorrectly
incorrectly
weighted in
in the decision
decision making
making process,
implementing
implementing new methods
methods for
for scoring
scoring hypotheses,
hypotheses, and creating
creating appropriate
appropriate explanation
explanation
Ii ties.
capabi
capabilities.
B. Requirements
for Continued
Requirements for
Continued SUMEX
SUMEX Use
We are currently
currently dependent
dependent on the SUMEX
SUMEX computer
computer for
for the use
use of
of the program
program by
remote
remote lIsers.
users, and for
for project
project coordination.
coordination. We
We have transferred
transferred the program
program over
over to
Portable
Portable Standard
Standard Lisp
Lisp which
which is used
used by several users
users on the SUMEX
SUMEX system. While
While
the switch
switch to workstations
workstations has lessened our
our requirements
requirements for
for computer
computer time
time for
for the
development
development of
of the algorithms,
algorithms, we will
will continue
continue to need the SUMEX
SUMEX facility
facility for
for the
interaction
interaction with
with each of
of the research locations
locations specified
specified in
in our
our NIH
NIH proposal.
proposal. The
The HP
equipment
equipment is currently
currently unable
unable to allow
allow remote
remote access,
access, and thus the program
program will
will have to
be maintained
maintained on the 2060 for
for use
use by all
all non-Stanford
non-Stanford lIsers.
users.
C. Requirements
for Additional
Requirements for
Additional Computing
Computing Resources
Most
Most of
of our
our computing
computing resources will
will be met
met by the 2060 plus the use
use of
of the Macintosh
Macintosh
II
We will
11 workstations.
workstations.
will need additional
additional file
file space
space on the 2060 as
as we
we quadruple
quadruple the
size of
of our
our knowledge
knowledge base
base through
through the construction
construction of
of multiple
multiple knowledge
knowledge bases.
bases. We
will
will continue
continue to require
require access
access to the 2060 for
for communication
communication purposes, access
access to other
other
programs,
programs, and for
for file
file storage and archiving.
archiving.
E.
E. H. Shortliffe
Shortliffe
198
PA THFINDER Project
Project
PATHFINDER
5P41-RR00785-14
5P41-RR0078514
D. Recommendations
Recommendations for
for Future
Future Community
Community and
and Resource
Resource Development
Development
D.
We encourage
encourage the
the continued
continued exploration
exploration by
by SUMEX
SUM EX of
of the
the interconnection
interconnection of
of
We
workstations within
within the
the mainframe
mainframe computer
computer setting.
setting. We
We will
will need to
to be able
able to
to
workstations
quickly move
move a program
program from
from workstation
workstation to
to workstation,
workstation, or
or from
from workstation
workstation back
back
quickly
and forth
forth to
to the mainframe.
mainframe. Software
Software tools
tools that
that would
would help
help the transfer
transfer of
of programs
programs
and
from one
one type
type of
of workstation
workstation to
to another
another would
would also be quite
quite useful.
useful. Until
Until the type
type of
of
from
workstations that
that we are using
using in
in this
this research becomes inexpensive,
inexpensive, we will
will continue
continue to
to
workstations
machine like
like SUMEX
SUMEX to
to provide
provide others
others with
with a chance
chance to
to experiment
experiment with
with our
our
need a machine
software.
software.
199
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
RXDX Project
Project
RXDX
IV.D.2. RXDX
RXDX Project
Project
IV.D.2.
RXDX Project
Project
RXDX
Robert Lindsay,
Lindsay, Ph.D.
Ph.D.
Robert
Michael Feinberg,
Feinberg, M.D..,
M.D., Ph.D.
Ph.D.
Michael
University of
of Michigan
Michigan
University
Ann
Arbor,
Michigan
Ann Arbor, Michigan
I. SUMMARY
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
1.
A. Project
Project Rationale
Rationale
A.
We are developing
developing a prototype
prototype expert
expert system that
that could
could act
act as a consultant
consultant in
in the
We
and management
management of
of depression.
depression. Health
Health professionals
professionals will
will interact
interact with
with the
diagnosis and
diagnosis
program as they
they might
might with
with a human
human consultant,
consultant, describing
describing the
the patient,
patient, receiving
receiving advice,
advice,
program
and asking
asking the consultant
consultant about
about the
the rationale
rationale for
for each recommendation.
recommendation. The
The program
program
and
uses a knowledge
knowledge base constructed
constructed by
by encoding
encoding the clinical
clinical expertise
expertise of
of a skilled
skilled
uses
psychiatrist in
in a set of
of rules
rules and
and other
other knowledge
knowledge structures.
structures. ItIt will
will use this
this knowledge
knowledge
psychiatrist
to decide
decide on the most
most likely
likely diagnosis
diagnosis (endogenous
(endogenous or
or nonendogenous
non endogenous depression),
depression),
base to
for hospitalization,
hospitalization, and recommend
recommend specific
specific somatic
somatic treatments
treatments when
assess the need for
assess
this
The
The treatment
treatment recommendation
recommendation will
will
this is indicated
indicated (e.g.,
(e.g., tricyclic
tricyclic antidepressants).
antidepressants).
patient's
take into
concurrent illnesses,
illnesses, and concurrent
concurrent
into account
account the patient’
s diagnosis,
diagnosis, age,
age, concurrent
treatments
treatments (drug
(drug interactions).
interactions).
Medical Relevance
Relevance and
B. Medical
and Collaboration
Collaboration
There
There is a documented
documented shortage
shortage of
of psychiatrists
psychiatrists in
in the US (GMENAC,
(GMENAC, 1980), and the
estimates
estimates of
of the prevalence
prevalence of
of psychiatric
psychiatric illness
illness used
used to develop
develop that
that report
report were lower
lower
than
Further, most
most
than the figures
figures in
in recent
recent population
population surveys (Myers
(Myers et aI.,
al., 1984). Further,
prescriptions
prescriptions for
for antidepressants
antidepressants are written
written by non-psychiatrists
non-psychiatrists (Johnson,
(Johnson, 1974; Kline,
Kline,
1974) and the great
great majority
majority of
of depressed patients
patients seen by a sample
sample of
of primary
primary care
(Weissman
physicians
physicians were treated
treated inappropriately
inappropriately
(Weissman et aI.,
al., 1981). These data highlight
highlight
the need for
for improving
improving the treatment
treatment provided
provided to the majority
majority of
of mentally
mentally ill
ill patients.
patients.
We believe that
that computers
computers can act as
as consultants
consultants to non-psychiatrist
non-psychiatrist clinicians,
clinicians, resulting
resulting
in
in improved
improved patient
patient care.
The
making relatively
relatively skilled
skilled psychiatric
psychiatric
The potential
potential benefits
benefits to psychiatry
psychiatry include:
include: making
consultation
widely
available
in
underserved
areas,
including
some
public
consultation wideiy available
underserved areas, including
public mental
mental health
health
facilities
facilities where patients
patients are seen
seen by non-psychiatrists
non-psychiatrists and
and have relatively
relatively little
little direct
direct
patient-physician
contact;
non-psychiatrically
trained
physicians
patient-physician
contact; providing
providing
non-psychiatrically
trained
physicians with
with
additional
about
additionai information
information
about psychiatric
psychiatric diagnosis
diagnosis and treatment;
treatment; avoiding
avoiding errors
errors of
of
oversight
oversight caused
caused by inaccessible
inaccessible patient
patient data; and increased productivity
productivity in patient
patient care.
Like
Like any good consultant,
consultant, the program
program will
will be able to teach the interested
interested user,
user, and can
function
function as
as aa teaching
teaching tool
tool independent
independent of
of direct
direct clinical
clinical application.
application.
C.
C. Highlights
Highlights of
of Research
Research Progress
Progress
Our
Our major
major project
project during
during the past year has
has been
been an
an expert
expert system
system for
for the somatic
somatic
treatment
treatment of
of (endogenous)
(endogenous) depression,
depression, where somatic
somatic treatment
treatment includes
includes antidepressant
antidepressant
drugs,
We are
are writing
writing this
this system
system using KEE,
KEE, an
an expert
expert
drugs, electroshock,
electroshock, and
and lithium.
lithium.
system
system shell
shell generously
generously donated
donated by
by lmellicorp,
Incellicorp, running
running on
on aa Xerox
Xerox n08
1108 workstation.
workstation.
We
EX, either
We have
have been1ble
been able to
to incorporate
incorporate the
the work
work we
we did
did earlier
earlier (JT1
<)II SUM
SUMEX,
either direcrly
direc:ly
E.
E. H.
H. Shortliffe
Shortliffe
200
200
RXDX
RXDX Project
Project
5P41-RR00785-14
5P41-RR00785-14
by transporting
transporting the
the rules
rules or
or indirectly
indirectly by
by using
using what
what we learned
learned about
about building
building expert
expert
by
in general.
general. The
The knowledge
knowledge base includes
includes information
information about
about the
the side
side effects
effects of
of
systems in
of the
the drugs
drugs and
and about
about the
the physiological
physiological mechanisms
mechanisms of
of these side
side effects.
effects. This
This
each of
information allows
allows us to
to predict
predict drug
drug interactions
interactions and
and the
the likelihood
likelihood of
of occurrence
occurrence of
of
information
various side
side effects
effects in
in a given
given patient,
patient, and
and to
to base explanations
explanations on
on knowledge
knowledge of
of the
various
The knowledge
knowledge base also includes
includes specific
specific information
information about
about
underlying physiology.
physiology. The
underlying
drug regimens,
regimens, about
about preventing
preventing and
and treating
treating side effects,
effects, and
and about
about how
how to
to take all
all of
of
drug
this into
into account
account in
in selecting
selecting a drug
drug and dosage regimen
regimen for
for the
the individual
individual patient.
patient.
this
D. List
List of
of Relevant
Relevant Publications
Publications
D.
Feinberg, M.
M. and
and Lindsay,
Lindsay, R. K.:
K.: Expert
Expert systems
systems in
in Psychiatry
Psychiatry
1. Feinberg,
Psychopharmacology. Psychopharmacol.
Psychopharmacol. Bull.,
Bull., 22, 1986, 311-316.
311-316.
Psychopharmacology.
Lindsay, R. K.:
K.: Expert
Expert Systems in
in Psychiatric
Psychiatric Diagnosis:
Diagnosis:
2. Lindsay,
Medlnf086, Washington,
Washington, D.C.
D.C.
Systems. Presented at MedInfo86,
Feinberg, M.:
M.:
What Psychiatrists
Psychiatrists Can’
Can't
Do.
What
t Do.
3. Feinberg,
Washington, D.
D. C.
Washington,
and
and
Rule-Based
Rule-Based
Presented at Medinfo86,
Medinf086,
Funding Support
Support
E. Funding
None.
None.
INTERACTIONS WITH
WITH THE
THE SUMEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
II. INTERACTTONS
II.
Medical Collaboration
and Program
Program Dissemination
Dissemination via
via SUMEX
SUMEX
A. Medical
Collaboration and
We
We have established
established via
via SUMEX
SUMEX a community
community of
of researchers who
who are interested
interested in
in AI
AI
applications
applications in
in psychiatry.
psychiatry. We
We also have used the message
message system to communicate
communicate with
with
other
other AI
AI scientists
scientists at SUMEX
SUMEX and elsewhere.
EX-AIM Projects
Projects
B. Sharing
Sharing and Collaboration
Collaboration with
with other
other SUM
SUMEX-AIM
During
During this
this past year we have had no occasion
occasion to engage
engage in
in collaboration
collaboration with
with other
other
SUMEX
- AIM Projects.
SUMEX-AIM
Projects.
C. Critique
Resource Management
Management
Critique of
of Resource
Our
This
Our sole use
use of
of the system this
this year has
has been for
for communication.
communication.
This has
has been very
useful,
useful, but
but hampered
hampered by difficulties
difficulties in matching
matching the characteristics
characteristics of
of various
various networks
networks
and terminals.
of SUMEX,
SUMEX, even for
for mail,
mail, awkward.
awkward. It
It would
would be
terminals. This
This has made use of
helpful
helpful to have some assistance with
with these problems.
problems.
III.
III. RESEARCH PLAN
PLAN
A. Project
Project Goals
Goals and Plans
Plans
Our
patients
Our immediate
immediate objective
objective is to 'develop
.develop expert
expert systems that
that can differentiate
differentiate
patients
with
with the various
various subtypes
subtypes of
of depressive
depressive disorder,
disorder, and prescribe
prescribe appropriate
appropriate treatment.
treatment.
This
psychiatrist,
This system .should
.should perform
perform at about
about the level of
of a board-certified
board-certified
psychiatrist, i.e.
better
better than an average resident
resident but
but not
not as
as well as
as a human
human expert
expert in depression.
depression.
Eventually,
Eventually, we plan
plan to enlarge
enlarge the knowledge
knowledge base
base so that
that the expert
expert system can
diagnose and prescribe
th
prescribe for
for a wider
wider range of
of psychiatric
psychiatric patients,
patients, particularly
particularly those wi
with
illnesses that
that are likely
likely to respond
respond to psychopharmacological
psychopharmacological agents. We will
will design the
system so that
that it
it could
could be used by non-medical
non-medical clinicians
clinicians or by non-psychiatrist
non-psychiatrist ~r.D.'s
M.D.‘s
problems
as
as an
an adjunct
adjunct to consultation
consultation with
with a human
human expert.
expert. We plan also to focus on problems
of
of [he
;he user
user interface
interface and the integration
integration of
of this system with
with other
other databases.
databases.
201
201
E. H. Shortliffe
Shorrliffe
E.
5P41-RR00785-14
SP41-RR0078S-14
RXDX Project
Project
RXDX
B. Justification
Justification and
and Requirements
Requirements for
for Continued
Continued SUMEX
SUMEX use
B.
The access
access to
to SUMEX
SUMEX resources is essentially
essentially our
our sole
sole means of
of maintaining
maintaining contact
contact
The
with the
the community
community of
of researchers working
working on
on applications
applications of
of AI
AI in
in medicine.
medicine.
with
moved our
our system to
to local
local workstations,
workstations, the
the communications
communications
Although we have moved
Although
capability of
of SUMEX
SUMEX will
will continue
continue to
to be important.
important.
capability
We anticipate
anticipate that
that our
our requirements
requirements for
for computing
computing time
time and file
file space will
will continue
continue at
We
about the
the same low
low level
level for
for the next
next year.
about
e. Needs
Needs and
and Plans
Plans for
for Other
Other Computing
Computing Resources
Resources
C.
We anticipate
anticipate that
that the
the need for
for additional
additional computing
computing power
power will
will continue
continue to
to be met
met by
by
We
local workstations.
workstations.
local
Recommendations for
for Future
Future Community
Community and
and Resource
Resource Development
Development
D. Recommendations
Valuable as the present
present SUMEX
SUMEX facilities
facilities are to us,
us, they
they are in many
many ways limited
limited and
Valuable
awkward to use.
use. The
The major
major limitation
limitation we feel
feel is the difficulty
difficulty and sometimes
sometimes the
awkward
impossibility of
of making
making contact
contact with
with everyone
everyone who
who could
could be of
of value
value to us.
us. We
We hope
impossibility
important not
not only
only to
that greater
greater emphasis
emphasis will
will be put
put on internetwork
internetwork gateways. It
that
Tt is important
of these, but
but to
to develop
develop consistent
consistent and convenient
convenient standards
standards for
for
establish more
more of
establish
electronic mail,
mail, electronic
electronic file
file transfers,
transfers, graphic
graphic information
information transfer,
transfer, national
national archives
archives
electronic
filing and retrieval
retrieval (categorization)
(categorization) systems. The
The present
present
bases, and personal
personal filing
and data bases,
of the art
art feels quite
quite limiting,
limiting, now
now that
that the basic concepts
concepts of
of computer
computer networking
networking
state of
available and have proved
proved their
their potential.
potential.
have become available
We expect
expect that
that the role
role of
of the SUMEX-AIM
SUMEX-AIM resource
resource will
will continue
continue to
to evolve
evolve in
in the
We
direction
direction of
of increased importance
importance of
of communication,
communication, including
including graphical
graphical information,
information,
The need for
for
electronic
electronic dissemination
dissemination of
of preprints,
preprints, and database and program
program access.
access. The
We hope to have continued
continued access
access
computer
computer cycles on a large
large mainframe
mainframe will
will diminish.
diminish. We
to the system for
for communication,
communication, but
but do not
not anticipate
anticipate continued
continued use
use of
of it
it as
as a Lisp
Lisp
computation
computation server.
If
If fees for
for using SUMEX
SUMEX resources were imposed,
imposed, this
this would
would have a drastically
drastically limiting
limiting
effect
if we had a budget
budget to purchase such
effect on the value of
of the system to us.
us. Even if
services, the inhibiting
inhibiting effect
effect of
of having
having a meter
meter running
running would
would cause us
us to make
make less
less
use
use of
of it
it that
that we should.
should. We have been conscious
conscious of
of the costs of
of the system and feel
feel
that
that we have not
not used
used it
it imprudently,
imprudently, even though
though we have not
not directly
directly borne
borne its costs.
E. H. Shortliffe
Shortliffe
202
Dynamic Systems Project
Project
Dynamic
5P41-RR00785-14
5P41-RR0078514
IV.D.3. Dynamic
Dynamic Systems
Systems Project
Project
IV.D.3.
for Time-Varying
Time-Varying Clinical
Clinical Problems
Decision Support for
Lawrence Widman,
Widman, M.D.,
M.D., Ph.D.
of Cardiology
Cardiology
Division of
Division
Case Western Reserve
Reserve University
University
Case
Adelbert Road
2065 Adelbert
Cleveland, OH
OH 44106
Cleveland,
(216) 844-3153
844-3153
(216)
I. SUMMARY
SUMMARY OF RESEARCH
RESEARCH PROGRAM
PROGRAM
PROJECT RATIONALE
RATIONALE
A. PROJECT
Time-varying systems, which
which include
include many
many areas of
of medicine,
medicine, science, economics,
economics, and
Time-varying
mathematically by differential
differential equations.
equations. They
They are distinct
distinct
described mathematically
business, can be described
from the pattern-matching
pattern-matching and logic-based
logic-based domains
domains dealt
dealt with
with so successfully
successfully by
from
expert system methods,
methods, because
because they
they can include
include feedback
feedback relationships.
relationships. ItIt is
existing expert
existing
generally felt
felt that
that they
they are best
hest approached
approached by enhancement
enhancement of
of existing
existing methods
methods for
for
generally
model-based reasoning.
reasoning.
deep model-based
The goal of
of this
this project
project is to develop
develop AI
AI methods
methods for
for capturing
capturing and using
using knowledge
knowledge
The
about
problems in
about time-varying
time-varying systems. The
The strategy
strategy is to
to address general
general problems
in model-based
model-based
knowledge
The
knowledge representation
representation and reasoning.
reasoning.
The intermediate
intermediate objective
objective is to develop
develop
powerful enough
methods
methods which
which are powerful
enough to work
work in
in selected realistic
realistic situations
situations yet are
unrelated knowledge
general enough to be transportable
transportable to other,
other, unrelated
knowledge domains.
domains.
The
problems
The tactical
tactical approach
approach is to work
work on well-defined
well-defined yet complex
complex and interesting
interesting problems
in
in the medical
medical domain.
domain. We have, therefore,
therefore, selected the human
human cardiovascular
cardiovascular system
as
prototype of
as our
our prototype
of a time-varying
time-varying system, and are developing
developing methods
methods for
for representing
representing
and reasoning
reasoning about
about its mechanical
mechanical and electrical
electrical activities
activities in
in the normal
normal and diseased
states.
states.
A.I Technical
A.1
Technical Goals
Goals
This
This project
project presently
presently has
has two distinct
distinct tracks:
arrhythmia
arrhythmia interpretation.
interpretation.
hemodynamic
hemodynamic modeling
modeling and cardiac
cardiac
1. Hemodyn:lmic
Hemodynnmic Modeling
Modeling
The
The goals of
of this
this subproject
subproject are to develop:
develop:
method
(a) a knowledge-representation
knowledge-representation
method using symbolic
symbolic modeling
modeling which
which captures
captures the
qualitative
qualitative and, when possible, the quantitative
quantitative behavior
behavior of
of systems with
with feedback
feedback
relationships.
Preferably,
relationships.
Preferably, the symbolic
symbolic model
model should
should be translatable
translatable into
into the
differential
differential equations
equations which
which describe
describe the behavior
behavior of
of the system being
being modeled.
modeled.
(b)
(b) a reasoning
reasoning method
method based
based on the symbolic
symbolic modeling
modeling tool
tool created in subgoal (a)
which
which permits
permits the inference
inference of
of differential
differential diagnoses (a set of
of hypothesized
hypothesized diagnoses)
from
from incomplete
incomplete data.
(c) a reasoning
reasonin, 0 method
method based
based on subgoals (a) and (b)
(b) which
which permits
permits inference
inference of
of the
state of
of the model for
for each
each hypothesized
hypothesized diagnosis.
diagnosis. This
This subgoal would
would be satisfied
satisfied by
an algorithm
algorithm which
which specifies
specifies a self-consistent
self-consistent set of
of values for
for all variables
variables in the
203
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Dynamic Systems Project
Project
Dynamic
model, for
for a given
given hypothesis
hypothesis based on
on a given
given set of
of data. Such sets of
of data
data would
would
model,
constitute initial
initial conditions
conditions for
for differential
differential equations
equations derived
derived from
from the model.
model.
constitute
(d) a simulation
simulation method,
method, based on
on the model
model and
and its
its equivalent
equivalent differential
differential equations
equations
(d)
together with
with the
the initial
initial conditions
conditions derived
derived from
from the
the differential
differential diagnosis
diagnosis (steps a-c
a-c
together
above), for
for predicting
predicting the
the expected
expected time
time course of
of the system being
being modeled
modeled for
for each
above),
This method
method could
could also be used to
to predict
predict the
the effects
effects of
of
hypothesized diagnosis.
diagnosis. This
hypothesized
treatments being
being considered
considered for
for recommendation
recommendation by
by the
the program.
program.
treatments
reasoning method,
method, based on
on
(e) a reasoning
shrinking and/or
and/or expanding
expanding the
the
shrinking
configuration to
to account
account for
for normal
normal
configuration
domain-independent properties
properties of
of the model,
model, for
for
domain-independent
model automatically
automatically to
to use a minimal
minimal model
model
model
abnormal data.
and abnormal
(f) an explanation
explanation facility
facility for
for examining
examining the
the model,
model, the given
given data, the inferred
inferred
(f)
hypothesized diagnoses, predicted
predicted behaviors,
behaviors, and modifications
modifications of
of the model,
model, to answer
answer
hypothesized
user queries
queries and to
to teach fundamental
fundamental concepts.
Cardiac Arrhythmia
Arrhythmia Recognition
Recognition
2. Cardiac
The goals of
of this
this subproject
subproject are to
to develop:
develop:
The
of the electrical
electrical system of
of the human
human heart,
heart, including
including pertinent
pertinent
(a) a symbolic
symbolic model
model of
(a)
anatomic and electrophysiologic
electrophysiologic features
features of
of the normal
normal and diseased heart.
heart.
The
anatomic
The
electrophysiologic features
features would
would include
include deterministic
deterministic characteristics
characteristics (e.g., conduction
conduction
electrophysiologic
velocities, refractory
refractory periods),
periods), stochastic
stochastic features
features (e.g.,
(e.g., behavior
behavior of
of automatic
automatic foci),
foci), and
velocities,
temporal interactions
interactions (e.g., competing
competing pacemakers).
temporal
(b) a symbolic/numeric
symbolic/numeric representation
representation of
of the observable
observable features
features of
of the electrical
electrical
(b)
activity
both surface
a~':' intracardiac
activity of
of the heart,
heart, both
surface EKG
EKG a:-:!
intracardiac recordings,
recordings, including
including noise.
This
This representation
representation would
would be intended
intended to all ;J;I a feature
feature extraction
extraction module
module working
working on
actual
patient data to communicate
actual patient
communicate with
with a symbolic
symbolic reasoning
reasoning module,
module, and would
would be
translatable
translatable directly
directly into
into waveform
waveform display
display format.
format.
(c)
(c) a reasoning
reasoning method
method for
for extracting
extracting features
features from
from raw, digitized
digitized signal
signal data. This
This
method
method would
would augment
augment established
established signal
signal processing techniques
techniques by using knowledgeknowledgebased algorithms
algorithms to improve
improve detection
detection of
of P and
and T-U
T-U waves and to improve
improve rejection
rejection of
of
noise. It
It should
should be noted
noted that
that this
this is itself
itself a major
major research undertaking
undertaking in
in the signal
signal
processing domain.
processing
domain.
(d)
(d) a reasoning
reasoning method
method for
for inferring
inferring the cardiac
cardiac rhythms
rhythms consistent
consistent with
with a given
given disease
state in
in the model,
model, similar
similar to the prediction
prediction of
of consequences of
of the hemodynamic
hemodynamic
model
The
model in the first
first subproject.
subproject.
The output
output of
of this
this method
method would
would be in
In the
symbolic/numerical
representation
symbolic/numerical
representation of
of subgoal (b).
(b).
(e) a reasoning
reasoning method
method for
for inferring
inferring possible disease
disease states in
in the model
model
feature-extracted
recording
feature-extracted
recording of
of (he
the electrical
electrical activity
activity of
of the heart.
constitutes
constitutes cardiac
cardiac arrhythmia
arrhythmia interpretation,
interpretation, and is itself
itself a major
major research
from
from a given
given
This
This subgoai
subgoal
project.
project.
(f)
(f) a categorization
categorization method
method for
for inferring
inferring hierarchies
hierarchies of
of diagnoses from
from elementary
elementary
up to 30 minutes
abnormalities.
For
abnormalities.
For example,
example, "periods
“periods of
of atrial
atria1 fibrillation
fibrillation
minutes at up to
150 beats/min,
beats/min, supraventricular
supraventricular tachycardia
tachycardia of
of up to 10 beats length
length at a rate of
of 130
beats/min,
beats/min, and sinus bradycardia
bradycardia with
with a minimum
minimum rate of
of 45, all consistent
consistent with
with the
sick
sick sinus ("tachy-brady")
(“tachy-brady”) syndrome"
syndrome” and "two
“two QRS morphologies
morphologies are present: they are
narrow
narrow at rates less
less than 120 and are wide at rates above 120, consistent
consistent with
with a rateratedependent
dependent bundle
bundle branch
branch block".
biock”.
(g) an explanation
for
explanation facility
facility
for examining
examining the model,
model, the input
input data, and
interpretations
interpretations to answer user queries and to teach fundamental
fundamental concepts.
E. H. Shortliffe
Shortliffe
204
the
Dynamic Systems Project
Project
Dynamic
5P41-RR00785-14
5P41-RR00785-14
B. MEDICAL
MEDICAL RELEVANCE
RELEVANCE AND
AND COLLABORATIONS
COLLABORATIONS
B.
The two
two subprojects
subprojects have related
related but
but separate medical
medical goals:
The
1. Hemodynamic
Hemodynamic Modeling.
Modeling.
There are three
three subgoals
subgoals in
in this
this subproject:
subproject: model-based
model-based sensor integration,
integration, modelmodelThere
caregiver assistance, and
and model-based
model-based experiment
experiment interpretation.
interpretation.
based caregiver
Model-based Sensor Integration.
Integration.
a. Model-based
The long-range
long-range application
application of
of this
this subproject
subproject is the integration
integration of
of patient-related
patient-related data
The
in the
the intensive
intensive care environment.
environment. ModelModel- based real-time
real-time systems would
would allow
allow the
in
global understanding
understanding of
of the patient’
patient's
condition with
with the human
human
to share a global
system to
s condition
caregivers. Thus,
Thus, itit could
could interpret
interpret significant
significant trends
trends in
in key parameters
parameters and could
could draw
draw
caregivers.
attention to
to relationships
relationships which
which might
might otherwise
otherwise escape
escape attention
attention in
in the
the constant
constant flood
flood
attention
of data common
common to these environments.
environments.
of
Model-based Caregiver
Caregiver Assistance.
Assistance.
b. Model-based
could also serve as
as an assistant
assistant to the caregiver.
caregiver. In
In this
this mode, the human
human caregiver
caregiver
ItIt could
merits of
of proposed
proposed diagnostic
diagnostic and therapeutic
therapeutic measures in
in light
light of
of
could evaluate
evaluate the merits
could
available data on
on the patient’
patient's
condition.
available
s condition.
Practical application
application of
of these concepts
concepts requires
requires further
further development
development of
of the model
model and
Practical
reasoning algorithms,
algorithms, and
and extensive
extensive testing
testing against
against real clinical
clinical scenarios.
scenarios.
the reasoning
Refinement and quality
quality control
control are presently
presently the responsibility
responsibility of
of the principal
principal
Refinement
investigator, who
who is a board-certified
board-certified internist
internist with
with subspecialty
subspecialty training
training in
in invasive
invasive
investigator,
cardiology.
cardiology.
Practical
~:cceptance of
;;cceptance
of standardized
standardized hospital
hospital data buses
Practical application
application also awaits
awaits gen
for
parameters now
primarily on paper or
for automatic
automatic acquisition
acquisition of
of importa
importa..,.. " parameters
now stored primarily
or on
computers
as fluid
fluid inputs
inputs and outputs,
outputs,
computers outside
outside the intensive
intensive care setting,
setting, such as
medications,
medications, and results
results of
of invasive
invasive and non-invasive
non-invasive tests.
tests. Further,
Further, improved
improved user
better graphics
part of
interfaces
interfaces will
will require
require better
graphics and
and increased computer
computer literacy
literacy on the part
of
caregi
verso
caregivers.
c. Model-based
Model-based Experiment
Experiment Interpretation.
Interpretation.
An
An intriguing
intriguing third
third application
application of
of this
this subproject
subproject is in
in the area of
of interpretation
interpretation of
of
biomedical
The
biomedical experiments.
experiments.
The symbolic
symbolic model
model concept,
concept, which
which enforces
enforces objectivity,
objectivity, can
of
assist investigators
by allowing
them to compare
investigators
allowing
compare alternate
alternate interpretations
interpretations
of
experimental
experimental data. In
In this
this application,
application, several alternate
alternate models would
would be proposed
proposed by
the experimenter
experimenter to explain
explain a given
given experimental
experimental outcome.
outcome. The
The consequences of
of each
model
model given
given different
different experimental
experimental parameters
parameters could
could then be evaluated
evaluated and compared
compared
with
with real data [0
to confirm
confirm or
or refute
refute competing
competing proposed models.
The
-consistent
The advantage of
of using a computer
computer in
in this
this manner
manner is the guarantee of
of self
self-consistent
and objective
objective exploration
exploration of
of each possibility.
possibility. The
The advantage of
of using a symbolic
symbolic model,
model,
rather
model
rather than
than a numerical
numerical model
model such as
as the Guyton-Coleman
Guyton-Coleman
model or
or a simpler
simpler
derivative,
derivative, is that
that the underlying
underlying cause
cause and effect
effect relationships
relationships are explicit
explicit and can be
The AI
easily
easily modified
modified by the experimenter.
experimenter.
AI interest
interest in this
this subgoal would
would be the
refinement
refinement of
of the symbolic
symbolic model
model through
through application
application to real experiments.
experiments. Unlike
Unlike in
in
the MOLGEN
MOLGEN project
project at Stanford,
Stanford, automatic
automatic hypothesis
hypothesis formation
formation would
would not
not be an
objecti
ve in this
objective
this subgoal.
A
A new collaboration
collaboration to explore
explore this
this application
application is being explored
explored with
with Dr.
Dr. E. Merrill
Merrill
Adams,
Adams, an experimental
experimental physiologist
physiologist in the Department
Department of
of Surgery at Case
Case Western
Western
Re2ierve
Reserve University
C:nivzrsity School of
of Medicine.
Medicine. Dr. Adams
Adams approached
approached us
us because
because of
af his longlongstanding
standing interest
interest in applying
applying AI
AI techniques
techniques to·
to‘ his experiments
experiments on ~i1e
;he interactIOns
interactions of
of
205
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Dynamic Systems Project
Project
Dynamic
the cardiovascular
cardiovascular and
and pulmonary
pulmonary systems. AA discussion
discussion group
group is being
being organized,
organized, and
the
we hope
hope to
to continue
continue despite
despite the
the move
move of
of the
the principal
principal investigator
investigator to
to Texas
Texas this
this
summer.
summer.
Cardiac Arrhythmia
Arrhythmia Recognition.
Recognition.
2. Cardiac
The long-range
long-range application
application of
of this
this sub-project
sub-project is in
in clinical
clinical devices such as intensiveintensiveThe
arrhythmia monitors,
monitors, portable
portable Holter
Holter monitors,
monitors, and
and implantable
implantable cardiovertercardiovertercare arrhythmia
care
defibrillators.
There are two
two subgoals:
subgoals: recognition
recognition of
of surface
surface electrocardiographic
electrocardiographic
There
defibrillators.
(EKG) recordings
recordings and
and recognition
recognition of
of intracardiac
intracardiac recordings.
recordings.
(EKG)
Recognition of
of surface
surface electrocardiographic
electrocardiographic recordings.
recordings.
a. Recognition
and well-recognized
well-recognized obstacles
obstacles in
in signal
signal processing
processing will
will likely
likely prevent
prevent non-AI
non-AI
Substantial and
Substantial
algorithms from
from advancing
advancing beyond
beyond the
the current
current state of
of the art
art of
of interpretation
interpretation of
of
algorithms
surface EKG
EKG recordings.
recordings.
obstacles are primarily
primarily the problems
problems of
of reliable
reliable
These obstacles
surface
of P and
and T-U
T-U waves, and rejection
rejection of
of noise.
noise. We hope that
that AI
AI techniques
techniques will
will
detection of
detection
helpful with
with these problems,
problems, as is suggested by
by the work
work of
of Muldrow
Muldrow et al.
be helpful
Cardiology, 1986, in
in press).
(Computers and Cardiology,
(Computers
further that,
that, by
by mimicking
mimicking the behavior
behavior of
of expert
expert human
human cardiologists,
cardiologists, these
hope further
We hope
they cannot
cannot be overcome.
overcome. We
We have enlisted
enlisted as consultants
consultants
obstacles can be bypassed ifif they
obstacles
of MIT.,
M.I.T., who
who supervised
supervised Muldrow
Muldrow in
in the paper
paper cited
cited above, and Dr.
Dr.
Dr. William
William Long
Long of
Dr.
Benjamin Kuipers
Kuipers of
of the University
University of
of Texas at Austin,
Austin, who
who is interested
interested in
in Al
AI
Benjamin
techniques for
for physiological
physiological modeling.
modeling.
techniqties
of intracardiac
intracardiac recordings.
recordings.
Recognition of
b. Recognition
Intracardiac
Intracardiac recordings,
recordings, which
which are taken from
from wires
wires placed in
in the heart
heart by percutaneous
percutaneous
venous
venous puncture
puncture or
or around
around the heart
heart by surgery,
surgery, are relatively
relatively free
free of
of P wave ambiguity
ambiguity
and of
of noise. They
They are representative
representative of
of the quality
quality of
of signals
signals available
available to implantable
implantable
cardioverter-defibrillators.
cardioverter-defibrillators.
Cardioverter-defibrillators
are devices like
Cardioverter-defibrillators
like pacemakers
pacemakers in
in that
that they monitor
monitor the heart
heart
rhythm
rhythm in a patient
patient to determine
determine if
if an abnormality
abnormality exists. They
They are capable
capable of
of taking
taking
Unlike
Unlike
action
action (electrical
(electrical countershock)
countershock) if
if an appropriate
appropriate abnormality
abnormality is detected.
ordinary
ordinary pacemakers,
pacemakers, these devices detect
detect abnormalities
abnormalities characterized
characterized by rapid
rapid rates of
of
They have been shown
shown to reduce oneoneheart
heart activity,
activity, rather
rather than
than excessively
excessively slow
slow rates. They
year mortality
mortality in
in high-risk
high-risk patients
patients from
from 30% to 2%,
2% and they are expected to play
play an
increasingly
increasingly large role
role in
in treatment
treatment of
of such patients.
patients.
These relatively
new devices currently
use
to detect
relatively
currently
use quite
quite simple
simple algorithms
algorithms
The
abnormalities.
abnormalities.
The action
action they take consists
consists of
of applying
applying an electrical
electrical shock directly
directly to
the heart. This
This shock
shock is frequently
frequently unpleasant
unpleasant to the patient.
patient. The
The problem
problem is that [he
rhe
algorithms
algorithms sometimes
sometimes confuse
confuse innocent
innocent rapid
rapid heart
heart rates, such as
as from
from exercise or atrial
atria1
This has
has proved
proved troublesome
troublesome
This
tachyarrhythmias,
tachyarrhythmias, with
with lethal
lethal ventricular
ventricular arrhythmias.
arrhythmias.
literature
enough
enough to prompt
prompt repeated calls
calls in
in the electrophysiology
electrophysiology
literature for
for improved
improved
algorithms
algorithms for
for arrhythmia
arrhythmia recognition
recognition in
in these devices.
The
The algorithms
algorithms developed
developed in this
this subproject
subproject would
would be suitable
suitable for
for this
this
Because these devices
the computer
Because
computer power
power in the devices improves.
improves.
energy sources to perform
perform repeated shocks over
over their
their lifetimes
lifetimes of
of 2-3
drain
drain of
of more
more sophisticated
sophisticated computer
computer chips
chips is less
less important
important than
ordi
nary pacemakers.
ordinary
E. H. Shortliffe
Shortliffe
206
application
application when
require powerful
powerful
require
years,
years, the power
it
it would
would be in
Dynamic
Dynamic Systems Project
Project
5P41-RR00785-14
5P41-RR00785-14
C. Highlights
Highlights of
of Research
Research Progress
Progress
Hemodynamic Modeling
Modeling
1. Hemodynamic
Subgoals (a)
(a) through
through (d)
(d) have been accomplished
accomplished in
in prototype
prototype form.
form. The
The approach
approach
Subgoals
relies on
on a semi-quantitative
semi-quantitative representation
representation [subgoal
[subgoal (a)]
(a)] which
which assigns values by
by
relies
default ifif the
the user does not
not specify
specify more
more detailed
detailed information.
information. The
The second phase of
of
default
this project
project yielded
yielded subgoal
subgoal (d),
(d), the
the simulation
simulation of
of a given
given model.
model. This
This phase was
this
accomplished by
by translating
translating the
the model
model into
into a set of
of dynamical
dynamical systems equations,
equations, which
which
accomplished
then integrated
integrated in
in the standard
standard manner.
manner.
were then
More recently,
recently, subgoals
subgoals (b)
(b) and
and (c)
(c) have been accomplished
accomplished in
in prototype
prototype form.
form.
More
Constraint propagation
propagation using
using a dynamically
dynamically generated
generated semi-quantitative
semi-quantitative quantity
quantity space
Constraint
performed by
by interpreting
interpreting the model
model as a set of
of constraint
constraint equations.
equations. DomainDomainis performed
independent heuristics
heuristics which
which recognize
recognize morphological
morphological features
features of
of the
the model
model are used
used to
to
independent
further constrain
constrain the propagation
propagation of
of constraints
constraints and to generate hypotheses
hypotheses when
further
ambiguities arise.
arise. These heuristics
heuristics generate
generate a set of
of self-consistent
self-consistent hypotheses, each of
of
ambiguities
which is a hypothesized
hypothesized diagnosis
diagnosis (subgoal
(subgoal b). Dr.
Dr. Yong-Bok
Yong-Bok Lee of
of the Case Western
Western
which
University Department
Department of
of Electrical
Electrical Engineering
Engineering and Applied
Applied Physics
Reserve University
participated in
in this
this subgoal
subgoal for
for his doctoral
doctoral dissertation.
dissertation. The
The doctoral
doctoral dissertation,
dissertation,
participated
in August,
August, 1986, was co-supervised
co-supervised by Professor
Professor Yoh-Han
Yoh-Han Pao of
of that
that
awarded in
awarded
Department and by
by Dr.
Dr. Widman.
Widman.
Department
hypothesized diagnosis
diagnosis is then
then refined
refined by mathematical
mathematical relaxation,
relaxation, in
in which
which the
Each hypothesized
propagated values are treated
treated as initial
initial guesses,
guesses, and the values are refined
refined iteratively,
iteratively,
propagated
interpreting the model
model as a set of
of constraint
constraint equations
equations (subgoal
(subgoal c). In the
again by interpreting
again
scenarios which
which have been examined,
examined, the value
value assignments
assignments achieved
achieved by
several scenarios
several
hypothesis
hypothesis and iterative
iterative refinement
refinement have achieved
achieved correlation
correlation coefficients
coefficients up to 0.90
with
with the values obtained
obtained by
by simulation
simulation of
of the same model.
model.
We
We do not
not anticipate
anticipate beginning
beginning work
work on the remaining
remaining subgoals until
until the above
prototype
prototype methods
methods have been further
further refined
refined and tested.
2.
2. Cardiac
Cardiac Arrhythmia
Arrhythmia Recognition
Recognition
built a prototype
prototype symbolic
symbolic model
model of
of the
This
just beginning.
We have built
This subproject
subproject is just
beginning.
The
The
electrical
electrical conduction
conduction system of
of the heart
heart and have reproduced
reproduced simple
simple rhythms.
rhythms.
important
important issues
issues of
of stochastic
stochastic variation
variation and of
of noise
noise have not
not been addressed.
addressed. We are
hopeful
insights
hopeful that
that important
important
insights will
will obtained
obtained from
from newly
newly developing
developing literature
literature on
Evidential Reasoning Using Stochastic
Stochastic Simulation
Simulation
stochastic
stochastic simulation
simulation (e.g.,
(e.g.. Pearl,
Pearl, J.:
J.: Evidential
of
Intelligence.
of Causal
Causal Models.
Models. Artificial
Artificial
Intelligence. 1987;32:245-257).
1987;32:245-257).
Following
Following the move
move of
of the
the hemodynamic
hemodynamic modeling
modeling
will
will be supported
supported in
in part
part
Texas Affiliate.
Affiliate.
principal
principal investigator
investigator to Texas, this
this subproject
subproject will
will replace
subproject
subproject as
as the major
major research focus. This
This researrh
research effort
effort
by a Grant-in-Aid
from
Grant-in-Aid
from the American
American Heart
Heart Association,
Association,
The
The principal
principal investigator
investigator will
will have access
access to intracardiac
intracardiac signals from
from a variety
variety of
of
This should
should
appropriate
This
appropriate patients
patients on the clinical
clinical service
service at his hospital
hospital complex.
complex.
facilitate
facilitate the development
development of
of practical
practical algorithms.
algorithms.
We have also begun discussions
discussions with
with a major
major pacemaker
pacemaker manufacturer
manufacturer with
with the goal of
of
establishing
establishing a working
working relationship.
relationship. The
The purpose
purpose of
of the relationship
relationship would
would be to enable
practical
practical pacemaker
pacemaker manufacturing
manufacturing constraints
constraints to be taken into
into account
account from
from an early
So far,
far, the discussions
discussions have demonstrated
demonstrated
stage
stage in the development
development of
of this
this subproject.
subproject. So
interest
interest on both
both sides, but
but will
will require
require further
further algorithm
algorithm development
development in order
order to
proceed.
207
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Dynamic
Dynamic Systems Project
Project
D. List
of Relevant
List of
Relevant Publications
Publications
Reasoning about
about Diagnosis
Diagnosis and Treatment
Treatment in
in a Causal
1. Widman,
Reasoning
Widman, L.E.
L.E.
Inference.
Medical
Model
using
Simulation
and Inference.
Medical
Model
using Semi-Quantitative
Semi-Quantitative
Simulation
Workshop
Intelligence
National Conference
Workshop on Artificial
Artificial
Intelligence in
in Medicine,
Medicine, National
Conference on
Artificial
-87, Seattle.
Artificial Intelligence
Intelligence AAAI
AAAI-87,
2. Widman,
Widman, L.E.,
L.E., Lee, Y.-B.,
Y.-B., and Y.-H.
Y.-H. Pao. Diagnosis
Diagnosis of
of Causal Models
Models by
Semi-Quantitative
Semi-Quantitative Reasoning.
Reasoning. (submitted
(submitted to SCAMC
SCAMC 1987).
3. Widman,
Widman, L.E.,
L.E., Lee, Y.-B.,
Y.-B., and Y.-H.
Y.-H. Pao. Diagnosis
Diagnosis of
of Causal Medical
Medical
In: Miller,
Miller, P.L. (ed.). Topics
Topics in
in
Reasoning.
Models
Models by Semi-Quantitative
Semi-Quantitative
Reasoning. In:
Medical
Medical Artificial
Artificial Intelligence,
Intelligence, Springer-Verlag
Springer-Verlag (in
(in preparation).
preparation).
4. Lee, Y.-B.
Reasoning
Y.-B. and L.E.
L.E. Widman.
Widman.
Reasoning about
about Diagnosis
Diagnosis and Treatment
Treatment in
in
Domain
Simulation
a Causal Time-varying
Time-varying
Domain using Semi-Quantitative
Semi-Quantitative
Simulation and
Inference.
Workshop
Intelligence
National
Workshop on Artificial
Artificial
Intelligence and Simulation,
Simulation, National
Inference.
Conference
Intelligence
Conference on Artificial
Artificial
Intelligence AAAI-86,
AAAI-86, Philadelphia.
Philadelphia.
5. Widman,
Widman, L.E.
L.E. Representation
Representation Method
Method for
for Dynamic
Dynamic Causal
Causal Knowledge
Knowledge Using
Using
Fifth
World
Conference
Medical
Semi-Quantitative
Simulation.
Fifth
World
Conference
on Medical
Semi-Quantitative
Simulation.
Informatics.
1986: 180-184.
Informatics.
180-184.
Funding Support
E. Funding
Support
1. American
American Heart
Heart Association,
Association, Texas Affiliate
Affiliate
Grant-in-Aid
Award.
Grant-in-Aid
Award.
Knowledge-Based
Knowledge-Based Computer
Computer Algorithms
Algorithms for
for Arrhythmia
Arrhythmia Analysis.
Analysis.
Principal Investigator:
Investigator: Lawrence
Lawrence E. Widman.
Widman.
Principal
Award
Award period:
period: July,
July, 1987 - June, 1988.
Level: $24,850 direct
direct costs.
Level:
II.
II. INTERACTIONS
INTERACTIONS WITH
WITH THE
THE SUMEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
A. Sharing
Sharing and
and interactions
interactions with
with other
other SUMEX-AIM
SUMEX-AIM
projects
projects
The
EX-AIM have been (1)
The major
major interactions
interactions with
with SUM
SUMEX-AIM
(1) computational
computational support
support and (2)
(2)
communication
communication with
with members
members of
of the AIM
AIM community.
community.
(1)
- AIM is the major
(1) SUMEX
SUMEX-AIM
major source of
of computing
computing power
power at this
this time.
time. Dr.
Dr. Widman
Widman
expects that
that a LISP
LISP workstation
workstation will
will be available
available after
after he relocates
relocates to Texas this
this
will then be needed primarily
primarily
for
summer.
SUMEX-AIM
computing
power
for
summer.
SUMEX-AIM
computing
power will
demonstrations
demonstrations at meetings
meetings and as
as backup
backup during
during workstation
workstation down-time.
down-time.
central location
location allows
allows ready
(2)
is the current
Its central
(2) SUMEX-AIM
SUMEX-AIM
current electronic
electronic mailbox.
mailbox.
This access
access has proved
proved invaluable
invaluable
Email
Email access
access by users
users of
of Arpanet,
Arpanet, Bitnet
Bitnet and Csnet. This
to Dr.
rapidly
Dr. Widman
Widman in
in communicating
communicating
rapidly and effectively
effectively with
with co-workers
co-workers at other
other
The value
value of
of this
this type of
of communication
communication
demonstrated several
institutions.
The
has been demonstrated
institutions.
times
times during
during the past year, when
when he had to make
make major
major career, equipment
equipment negotiation,
negotiation,
and manuscript
manuscript revision
revision decisions,
decisions, without
without local
local expertise,
expertise, within
within short
short periods
periods of
of time.
time.
Review
Review of
of the longer
longer term
term history
history of
of this
this project
project shows that
that it
it would
would not
not exist
exist had
SUMEX-AIM
not
SUMEX-AIM
not provided
provided telecommunication
telecommunication support
support for
for the initial
initial feasibility
feasibility project
project
in
in 1984-1985,
1984-1985, which
which was carried
carried out
out on the computers
computers of
of the MIT
MIT Laboratory
Laboratory for
for
Computer
Computer Science, Clinical
Clinical Decision
Decision Making
Making Group.
Group.
Shortliffe
E. H. Shortliffe
208
5P41-RR00785-14
5P41-RR00785-14
Dynamic Systems
Systems Project
Project
Dynamic
C. Critique
Critique of
of Resource
Resource Management
Management
C.
The service
service provided
provided by
by SUMEX-AIM
SUMEX-AIM has
has been
been exemplary,
exemplary, largely
largely because
because of
of prompt
prompt
The
and effective
effective response
response to
to difficulties
difficulties as
as they
they arise.
arise. There
There has
has been
been aa clear
clear effort
effort to
to
and
that telecommunication
telecommunication access remained
remained reliable
reliable during
during changes in
in commercial
commercial
assure that
vendors, and
and the
the staff
staff have
have responded
responded to
to several
several technical
technical questions
questions promptly
promptly and
and
vendors,
accurately.
Down-time
has
been
minimal
compared
to
that
of
other
systems
we
have
accurately. Down-time
been minimal compared to that of other systems
have
and is almost
almost always
always scheduled
scheduled several
several days in
in advance.
advance.
used, and
The reason we sought
sought contact
contact with
with the
the AIM
AIM community
community was that
that itit seemed the
the natural
natural
The
niche for
for our
our research
research interests.
interests. There
There is no
no short-term
short-term prospect
prospect that
that this
this project
project will
will
niche
reach
commercial
maturity
or
that
it
will
lose
sight
of
fundamental
AI
issues,
and
so
we
reach commercial maturity or that it will lose sight of fundamental AI
and
feel that
that itit still
still belongs
belongs in
in the
the scientific
scientific AIM
AIM framework.
framework.
feel
As noted
noted in
in the
the previous
previous section,
section, the
the communication
communication with
with other
other members
members of
of the AIM
AIM
As
community has proved
proved invaluable
invaluable in
in the,advancement
the advancement of
of this
this project.
project.
community
III. RESEARCH
RESEARCH PLANS
PLANS
III.
Project Goals
Goals and
and Plans
Plans
Project
The long
long range goals of
of this
this project
project are to
to develop
develop intelligent
intelligent comprehensive
comprehensive
The
monitoring/alarm systems for
for intensive
intensive care unit
unit settings;
settings; and intelligent
intelligent arrhythmia
arrhythmia
monitoring/alarm
recognition systems for
for monitors,
monitors, Holter
Holter recorders,
recorders, and
and implantable
implantable cardioverterj
c:udioverterl
recognition
defibrillators. The
The short
short term
term strategies
strategies for
for achieving
achieving these goals are discussed above.
defibrillators.
The next
next phase of
of this
this research will
will be conducted
conducted at the University
University of
of Texas Health
Health
The
Science Center
Dr.
joining the faculty
faculty of
Medicine
Center at San Antonio.
Antonio.
Dr. Widman
Widman will
will be joining
of Medicine
there
His
His clinical
clinical duties
duties will
will include
include
there on July
July 1, 1987, in
in the Division
Division of
of Cardiology.
Cardiology.
invasive
patients. Substantial
invasive hemodynamic
hemodynamic and electrophysiological
electrophysiological studies
studies on selected patients.
Substantial
time
project will
time is committed
committed to research, and this
this project
will constitute
constitute his
his major
major research
emphasis.
B. Justification
for continued
EX use
Justification and
and requirements
requirements for
continued SUM
SUMEX
The
justification for
The justification
for this
this project
project is its potential
potential for
for advancing
advancing the state of
of the art
art of
of
expert
expert system technology
technology in
in the area of
of temporal
temporal reasoning
reasoning and deep causal modeling,
modeling,
and for
for demonstrating
demonstrating practical
practical use
use of
of expert
expert symbolic
symbolic computing
computing in
in potentially
potentially lifelifesaving,
saving, knowledge-intensive
knowledge-intensive environments.
environments.
The
use
The requirements
requirements for
for continued
continued SUMEX-AIM
SUMEX-AIM
use should
should be the same as
as currently:
currently:
telecommunications
telecommunications support,
support, Arpanet
Arpanet access,
access, about
about 3 megabytes of
of disc space,
space, and a
reasonable amount
amount of
of CPU
CPU time.
time. When
When the Lisp
Lisp workstation
workstation becomes available
available (see
(see
below),
below), the requirement
requirement for
for telecommunication
telecommunication support
support and CPU
CPU time
time should
should decrease.
decrease.
C. Needs and plans
plans for
for Other
Other Computing
Computing Resources Beyond SUMEX-AfM
SUMEX-AIM
The symbolic
symbolic computing
computing needs
needs for
for the hemodynamic
hemodynamic modeling
modeling subproject
subproject are being met
by SUMEX.
SUMEX. Once embarked
embarked on the arrhythmia
arrhythmia recognition
recognition subproject,
subproject, there will
will be aa
strong
need
for
high-resolution
graphics,
and
processing
of
tens
of
megabytes
of
strong need for high-resolution
graphics,
processing of
of
of data.
To meet these
these needs,
needs, aa Lisp
Lisp workstation
workstation will
will be provided
provided by the University
University of
of Texas.
Data
Data acquisition
acquisition in
in real time
time and
and initial
initial signal
signal processing will
will be
be done
done with
with an
an IBM
TBM AT
AT
class
multichannel
class microcomputer
microcomputer equipped
equipped with
with aa standard
standard third-party
third-party
multichannel analog-toanalog-todigital
digital converter.
converter. Communication
Communication between the machines
machines will
will be
be by RS232
RS232 or
or the local
local
Ethernet
Once
will
Ethernet LAN.
LAN.
Once these
these machines
machines are
are in
in place, SUMEX-AIM
SUMEX-AIM
will be
be needed
needed
primarily
primarily for
for communication
communication and
and demonstration
demonstration projects,
projects, as
as noted above.
above.
209
209
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR0078514
Dynamic Systems Project
Project
Dynamic
·D.
Recommendations for
for Future
Future Community
Community and
and Resource
Resource Development
Development
D. Recommendations
Our strong
strong recommendation
recommendation is that
that SUMEX-AIM
SUMEX - AIM be maintained
maintained as a national
national AIM
AIM
Our
for communication,
communication, development
development of
of software
software useful
useful to
to the
the AIM
AIM community,
community,
resource for
resource
sharing of
of demonstration
demonstration projects.
projects. SUMEX-AIM
SUMEX-AIM could
could also
also serve as a central
central
and sharing
of advice
advice for
for new
new workstation
workstation users who
who may
may be geographically
geographically isolated
isolated from
from
source of
experienced workstation
workstation users.
experienced
would strongly
strongly support
support retention
retention of
of the current
current telecommunication
telecommunication
Additionally, we would
Additionally,
support and enough
enough computing
computing power
power to
to support
support promising
promising young
young investigators
investigators who
who
support
would otherwise
otherwise not
not have access
access to symbolic
symbolic computing
computing power.
power.
would
E.
E. H. Shortliffe
Shortliffe
210
5P41-RR00785-14
5P41-RR00785-14
Knowledge Engineering
Engineering for
for Radiation
Radiation Therapy
Therapy
Knowledge
IV.D.4. Knowledge Engineering
Engineering for
for Radiation
Radiation Therapy
Therapy
IV.D.4.
KNOWLEDGE ENGINEERING
ENGINEERING FOR
FOR RADIATION
RADIATION THERAPY
THERAPY
KNOWLEDGE
Ira J. Kalet,
Kalet, Ph.D.
Ph.D.
Ira
Witold Paluszinski
Witold
of Washington
Washington
University of
University
Seattle, Washington
Washington
T. Summary of
of Research
Research Program
Program
1.
Project Rationale
Rationale
A. Project
We are developing
developing an expert
expert system for
for planning
planning of
of radiation
radiation therapy
therapy for
fGr head and
and
We
neck cancers. The
The project
project will
will ultimately
ultimately combine
combine knowledge-based
knowledge-based planning
planning with
with
neck
numerical simulation
simulation of
of the radiation
radiation treatments.
treatments. The
The numerical
numerical simulation
simulation is needed
numerical
in order
order to determine
determine ifif the proposed
proposed treatment
treatment will
will conform
conform to
to the goals of
of the plan
plan
in
(required tumor
tumor dose, limiting
limiting dose to
to critical
critical organs).
organs). The
The space of
of possible
possible radiation
radiation
(required
treatments is numerically
numerically very
very large, makin,
making0 traditional
traditional search techniques
techniques impractical.
impractical.
treatments
Yet, with
with modern
modern radiation
radiation therapy
therapy equipment,
equipment, the design of
of treatment
treatment plans
plans might
might be
Yet,
significantly aided by
by automatically
automatically generating
generating plans
plans that
that meet
meet the treatment
treatment constraints.
constraints.
significantly
The project
project will
will result
result in
in systematization
systematization of
of knowledge
knowledge about
about radiation
radiation treatment
treatment design,
design,
The
and will
provide an example
problems with
will also provide
example of
of how
how to represent
represent and solve
solve design problems
with a
knowledge
based system.
.
knowledge based
Medical Relevance and Collaborations
B. Medical
Collaborations
Radiation
Radiation therapy
therapy has shown
shown dramatic
dramatic improvement
improvement in
in the cure
cure rate for
for many
many tumor
tumor
sites in
in the last two
two decades.
decades. Much
Much of
of this
this can be attributed
attributed to the improved
improved
penetration
-ray machines.
penetration capability
capability of
of modern
modern megavoltage
megavoltage X
X-ray
machines. These high
high energy
energy beams
can deliver
deliver high
high tumor
tumor doses
doses without
without overdosing
overdosing surrounding
surrounding tissue in many
many cases.
cases.
However,
However, they are typically
typically used
used in
in very
very limited
limited ways,
ways, because
because of
of the lack
lack of
of suitable
suitable
simulation
simulation systems to compute
compute the dose distribution
distribution for
for any but
but a few
few narrow
narrow choices
choices
of
of treatment
treatment geometry.
geometry. In
In the last few
few years these simulation
simulation systems have been
extended to the full
full range of
of geometric
geometric treatment
treatment arrangement
arrangement that
that any therapy
therapy machine
machine
is capable of.
of. Thus it
it would
would be valuable
valuable to be able to generalize
generalize our
our knowledge
knowledge of
of
treatment
In addition,
addition, even
treatment technique
technique by exploring
exploring these
these expanded possibilities.
possibilities.
treatments
treatments with
with standard
standard geometries
geometries can be very complex,
complex, and it
it is tedious
tedious to explore
explore
A
all of
A knowledge-based
knowledge-based system can generate a few
few "best"
“best” plans
of them individually.
individually.
which
which satisfy
satisfy the constraints
constraints and allow
allow more time
time for
for the physician
physician La
co evaluate the
[he
options,
options, or make minor
minor adjustments
adjustments for
for optimization.
optimization.
Since cancer treatment
approach
surgery and
treatment is a multi-disciplinary
multi-disciplinary
approach involving
involving
chemotherapy
chemotherapy as
as well as
as radiation,
radiation, it
it is important
important to coordinate
coordinate this
this work
work with
with
knowledge-based
knowledge-based program
program projects
projects in those areas.
areas. Most
Most significant
significant is the ONCOCIN
ONCOCIN
project,
project, which
which addresses
addresses management
management of
of patients
patients on
on chemotherapy
chemotherapy protocols.
protocols.
This
This project
project has
has some
some relevance to computer
computer science
science as
as well, in
in that our
our approach,
approach, if
if
successful,
successful, may contribute
contribute to aa better understanding
understanding of
of design
design problem
problem solving
solving with
with
knowledgebased systems.
knowledge-based
systems.
211
211
E.
E. H.
H. Shortliffe
Shortliffe
Knowledge Engineering
Engineering for
for Radiation
Radiation Therapy
Therapy
Knowledge
5P41-RR00785-14
5P41-RR00785-14
Highlights of
of Research
Research Progress
Progress
C. Highlights
In the
the past
past year, we have
have made
made significant
significant additions
additions to
to the
the rule
rule database for
for details
details of
of
In
and neck
neck cancer
cancer treatment.
treatment. We
We have devised
devised a representation
representation of
of parameters
parameters for
for
head and
radiation treatment
treatment fields
fields and
and created
created a set of
of prototype
prototype treatment
treatment field
field arrangements.
arrangements.
radiation
The prototypes
prototypes are used as building
building blocks
blocks for
for constructing
constructing complex
complex treatment
treatment plans.
plans.
The
addition we have examined
examined the
the issues of
of control
control strategy
strategy associated with
with using
using
In addition
In
prototypes in
in planning.
planning.
prototypes
Our expert
expert system now
now has about
about two
two hundred
hundred rules,
rules, a two-level
two-level (agenda-based)
(agenda-based) control
control
Our
strategy, and
and about
about ten
ten prototypes
prototypes for
for plan
plan construction.
construction. ItIt is written
written in
in Interlisp
Interlisp on
on a
strategy,
running the VMS
VMS operating
operating system. This
This environment
environment was chosen because itit is
V AX running
VAX
the environment
environment used for
for a graphic
graphic simulation
simulation system that
that does radiation
radiation dose
also the
calculations for
for arbitrary
arbitrary treatment
treatment plans. The
The dose calculation
calculation is needed to
to determine
determine
calculations
whether a plan
plan meets the
the treatment
treatment goals set by
by the
the system in
in its
its early
early phases of
of
whether
planning.
planning.
D. List
List of
of Relevant
Relevant Publications
Publications
D.
1. I.
Kalet and W.
W. Paluszynski:
Paluszynski: A
A Production
Production Expert
Expert System for
for Radiation
Radiation
I. Kalet
Therapy Planning.
Planning. Proceedings
Proceedings of
of the AAMSI
AAMSI Congress 1985, May
May 20-22,
20-22,
Therapy
California.
Edited by
by Allan
Allan H. Levy
Levy and Ben
1985, San Francisco,
Francisco, California.
Edited
American Association
Association for
for Medical
Medical Systems and Informatics,
Informatics,
T. Williams.
Williams. American
T.
Washington, D.C., 1985.
Washington,
Kalet: Radiation
Radiation Therapy
Therapy Planning:
Planning: A
A Design
Design Oriented
Oriented
2. W.
W. Paluszynski
Paluszynski and I. Kalet:
Expert System.
WESTEX-87 (Western
(Western Conference
Conference on
on Expert
Expert Systems),
WESTEX-87
Expert
Anaheim,
Anaheim, California,
California, June 2-4,
2-4, 1987.
3. I. Kalet
Kalet and 1.
J. Jacky: Knowledge-based
Knowledge-based Computer
Computer Simulation
Simulation for
for Radiation
Radiation
Therapy
Proceedings
Ninth International
Proceedings of
of the Ninth
International Conference
Conference on
Therapy Planning.
Planning.
the use
Netherlands, June
use of
of Computers
Computers in
in Radiotherapy,
Radiotherapy, Scheveningen,
Scheveningen, the Netherlands,
1987. North
North Holland,
Holland, 1987.
II.
- AIM Resource
II. Interactions
Interactions with
with the SUMEX
SUMEX-AIM
Resource
Our
resource has been as a means to be in
Our main
main use
use of
of the SUMEX-AIM
SUMEX-AIM
in contact
contact with
with
other
projects. The
The existence
existence of
of a mailbox
mailbox at SUMEXSUMEXother researchers working
working on AIM
AIM projects.
AIM
AIM has made it
it much
much easier for
for colleagues at other
other institutions
institutions to communicate
communicate with
with
us,
us, and has been valuable
valuable in
in assisting
assisting us
us with
with organizing
organizing the AIM
AIM Workshop
Workshop for
for 1987.
We
We have had a great deal of
of contact
contact with
with members
members of
of the ONCOCIN
ONCOCIN project
project and other
other
groups.
This has
has been valuable
valuable to us
us in stimulating
stimulating creative
creative approaches [0
co our
our projecL
project.
groups. This
III.
III. Research
Research Plan
A. Project
Project Goals
Goals and Plans
Plans
We
We plan
plan to continue
continue to acquire
acquire rules and develop
develop our
our current
current expert
expert system. This
This
includes
includes solving
solving problems
problems of
of use
use of
of prototypes,
prototypes, satisfaction
satisfaction of
of constraints
constraints by some kind
kind
of
of backtracking
backtracking search, and incorporating
incorporating evaluation
evaluation of
of plans by using the results
results of
of the
This
dose computation.
This last idea involves
involves coupling
coupling the expert
expert system with
with the dose
dose
computation.
computation
Our long-term
long- term
computation system (written
(written in
in PASCAL)
PASCAL) in suitably
suitably efficient
efficient ways.
ways. Our
goal is to shape the user interface
interface and improve
improve the system performance
performance to where it
it can
provide
provide assistance to clinicians
clinicians in treatment
treatment design for
for patients
patients in the normal
normal course of
of
treatment.
treatment.
E. H. Shortliffe
Shortliffe
212
5P41-RR00785-14
5P41-RR00785-14
Knowledge
Radiation Therapy
Knowledge Engineering
Engineering for.
for .Radiation
Therapy.
Justification and
Requirements for
for Continued
B. Justification
and Requirements
Continued SUMEX
SUMEX use
We
We foresee
foresee continued
continued need to be in
in touch
touch with
with other
other members
members of
of the AIM
AIM community,
community,
particularly
particularly projects
projects centered
centered at SUMEX.
SUMEX. While
While we do not
not expect
expect to use
use the computing
computing
resources
resources of
of SUMEX
SUMEX directly,
directly, some more
more extensive
extensive communication
communication and involvement
involvement is
likely to be useful.
useful.
likely
Other Computing
Computing Resources
Plans For
For Other
C. Plans
The
The main
main computing
computing resources for
for our
our project
project will
will continue
continue to be local.
local. We will
will be
rewriting the expert
expert system code in
in VAX
VAX Lisp,
Lisp, an implementation
implementation of
of Common
Common Lisp
Lisp on
on
rewriting
DEC VAXstation.
VAXstation.
We expect
expect delivery
delivery of
of a VAXstation
VAXstation II/GPX
II/GPX in
in the near future.
future.
the DEC
We
This
This appears to
to be a good choice
choice to satisfy
satisfy our
our need for
for high
high performance
performance graphic
graphic
simulation
and a reasonable
However,
simulation
reasonable Lisp
Lisp system.
However, the resources for
for the dose
computation
computation may
may not
not be adequate as
as we incorporate
incorporate more
more sophisticated
sophisticated computation
computation
models. As this
this develops,
develops, we hope to experiment
experiment with
with distributed
distributed systems, in
in which
which the
models.
dose computation
computation may
may run
run on a remote
remote resource, which
which mayor
may or may not
not be at SUMEX.
SUMEX.
D. Recommendations
Community and Resource Development
Recommendations for
for Future
Future Community
Development
communication
Two
communication
Two areas will
will be of
of increasing
increasing importance
importance to us
us in
in the future:
future:
capabilities
capabilities (electronic
(electronic mail
mail and file
file transfer)
transfer) and centralized
centralized databases.
databases. By
Ey centralized
centralized
databases, we refer
refer to the need for
for better
better maintenance
maintenance of
of mailing
mailing lists,
lists, information
information
about
Dr. Kalet's
Kalet’s experience
experience in
in organizing
organizing the
about projects,
projects, and possibly
possibly on-line
on-line reports.
reports. Dr.
AIM Workshop
Workshop for
for 1987 demonstrated
demonstrated that
that electronic
electronic communication
communication is invaluable,
invaluable,
AIM
even in
in its present
present state, but
but in
in order
order to create a list
list to send announcements
announcements to, we
expended
expended many
many hours
hours of
of manually
manually cutting
cutting and pasting
pasting messages
messages containing
containing past lists
lists
and searching
searching for
for up-to-date
up-to-date electronic
electronic mail
mail addresses.
addresses.
If
If fees for
for use of
of SUMEX
SUMEX resources were imposed,
imposed, the main
main impact
impact on our
our project
project
would be one of
of increased
increased isolation,
isolation, unless we could
could find
find grant
grant support
support for
for the fees.
would
213
213
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Pathophysiologic Diagnosis
Diagnosis Project
Project
Pathophysiologic
IV.D.S.
IV.D.5. Pathophysiologic
Pathophysiologic Diagnosis
Diagnosis Project
Project
COMPUTER-BASED
COMPUTER-BASED EXERCISES
EXERCISES IN
IN PATHOPHYSIOLOGIC
PATHOPHYSIOLOGIC DIAGNOSIS
DIAGNOSIS
Robert Beck, M.D.
M.D.
J. Robert
Dartmouth College School of
of Medicine
Medicine
Dartmouth
2 Maynard
Maynard St.
Hanover, N.H.
N.H. 07355
SUMMARY OF
OF RESEARCH
RESEARCH PROGRAM
PROGRAM
I. SUMMARY
A. Project
rationale
Project rationale
Research in
in artificial
artificial intelligence
intelligence at Dartmouth
Dartmouth Medical
Medical School
School focuses on three main
main
areas: 1) knowledge-based
knowledge-based systems applied
applied to laboratory
laboratory medicine
medicine and pathology,
pathology, 2)
using
knowledge
knowledge acquisition
acquisition
using machine
machine learning
learning techniques,
techniques, and 3) computer-based
computer-based
instruction
intelligence
instruction using
using artificial
artificial
intelligence techniques
techniques to critique
critique students'
students’ workup
workup plans.
These projects
projects have in
in common
common the fundamental
fundamental research questions
questions of
of how
how knowledge
knowledge
should
should be represented
represented and used
used in
in a classification
classification approach
approach to problem-solving
problem-solving related
related
to the use
use of
of laboratory
laboratory data.
Knowledge-based
laboratory medicine:
medicine:
Knowledge-based systems in laboratory
We
We are investigating
investigating the use
use of
of knowledge-based
knowledge-based systems to review
review requests for
for blood
blood
products.
being developed
developed to advise pathologists
pathologists and pathology
pathology residents
residents
products. A system is being
about the appropriateness
appropriateness of
of transfusion
transfusion requests.
about
The
The system will
will have both
both diagnostic
diagnostic and therapeutic
therapeutic objectives.
objectives. The
The diagnostic
diagnostic part
part of
of
the system will
available
will be used
used to evaluate
evaluate information
information
available in
in machine-readable
machine-readable form,
form,
Based on the available
available information,
information, the
and then ask the user a few
few relevant
relevant questions.
questions. Based
system will
will determine
determine possible
possible diagnoses relevant
relevant to transfusion
transfusion medicine.
medicine. The
The current
current
prototype
The
prototype is focusing
focusing on coagulopathies
coagulopathies and bleeding
bleeding disorders.
disorders.
The objective
objective is to
have a system that
that can quickly
quickly provide
provide a summary
summary of
of relevant
relevant laboratory
laboratory information
information
to the pathologist
of
pathologist charged
charged with
with the responsibility
responsibility
of evaluating
evaluating appropriateness
appropriateness of
of
The therapeutic
therapeutic recommendations
recommendations of
of the system will
will be focused
focused
transfusion
transfusion requests. The
on determining
determining appropriate
appropriate choices
choices and quantities
quantities of
of blood
blood products
products or
or substitutes.
substitutes.
One
One of
of the purposes
purposes of
of this
this investigation
investigation is to determine
determine whether
whether a knowledge-based
knowledge-based
system can eventually
eventually reduce inappropriate
inappropriate use
use of
of blood
blood products.
products. The
The purpose of
of the
but
tool
tool is not
not to usurp decision-making,
decision-making,
but to pre-process
pre-process large volumes
volumes of
of transfusion
transfusion
requests and large volumes
volumes of
of data on each request, in
in order
order to focus
focus the pathologist's
pathologist’s
attention
manner
The
attention in
in a time-efficient
time-efficient
manner on the most
most relevant
relevant information.
information.
The system is
The initial
initial prototype
prototype is being
being built
built using
in
in the early
early knowledge
knowledge acquisition
acquisition stage.
stage. The
IBM's
IBM’s Expert
Expert System Environment
Environment tool.
tool.
Knowledge acquisition
for knowledge-based
Knowledge
acquisition for
knowledge-based systems:
systems:
The
The purpose
purpose of
of this
this project
project is to develop
develop machine
machine learning
learning tools
tools that
that can be used
used for
for
The focus
focus is on deriving
deriving classification
classification rules in
in
knowledge
knowledge acquisition
acquisition from
from databases. The
the form
form of
of criteria
criteria tables. The
The criteria
criteria table
table format
format has been used
used for
for many
many years in
in
medicine,
medicine, and is still
still in
in use
use particularly
particularly in
in the area of
of rheumatic
rheumatic diseases
diseases (for
[for example,
example,
the ARA
ARA criteria
criteria for
for systemic
systemic lupus
lupus erythematosus).
erythematosus). Other
Other diseases
diseases for
for which
which diagnostic
diagnostic
criteria
criteria tables have been developed
developed include
include polycythemia
polycythemia vera.
Vera. multiple
multiple myeloma
myeloma ~lIld
and
E. H. Shortliffe
Shortliffe
214
5P41-RR00785-14
5P41-RR00785-14
Pathophysiologic Diagnosis
Diagnosis Project
Project
Pathophysiologic
primary biliary
biliary cirrhosis,
cirrhosis. In
In addition,
addition, criteria
criteria tables
tables have
have been
been found
found useful
useful as
as aa
primary
knowledge representation
representation for
for expert
expert systems.
knowledge
CRiteria Learning
Learning System
System (CRLS)
(CRLS) which
which is
is
We have
have developed
developed aa program,
program, called
called the
the CRiteria
We
capable
of
automatically
generating
criteria
tables
from
a
database
of
positive
and
capable of automatically generating criteria tables from
of positive and
negative examples.
examples. CRLS
CRLS is
is implemented
implemented in
in Common
Common LISP
LISP on
on a SUN-3
SUN-3 workstation.
workstation.
negative
utilizes not
not only
only the
the raw
raw data
data but
but also
also some
some background
background knowledge
knowledge supplied
supplied by
by the
the
ItIt utilizes
user about
about the
the concepts
concepts to
to be learned,
learned, the
the features
features of
of the
the problem,
problem, and
and the
the type
type of
of
diagnostic
performance
the
user
wishes
to
optimize
(i.e.
sensitivity,
specificity,
the
wishes to optimize
(i.e. sensitivity, specificity,
diagnostic performance
that are more
more comprehensible
comprehensible than
than the
the
efficiency, etc.).
etc.). CRLS
CRLS learns
learns decision
decision rules
rules that
efficiency,
rules generated
generated by
by other
other machine
machine learning
learning programs.
programs. Tests of
of the system have also
rules
that itit is capable
capable of
of handling
handling large
large databases containing
containing as many
many as 1500
1500 cases
cases
shown that
shown
variables each.
with 50 variables
with
Teaching medical
medical pathophysiology
pathophysiology using
using computer-based
computer-based tools:
tools:
Teaching
The project
project “Computer-based
"Computer-based Exercises
Exercises in
in Pathophysiologic
Pathophysiologic Diagnosis”
Diagnosis" is funded
funded
The
National Library
Library of
of Medicine’
Medicine's
Medical Informatics
Informatics research initiative.
initiative. ItIt
through the National
through
s Medical
has four
four specific
specific aims:
aims:
To develop
develop two
two computer-assisted
computer-assisted laboratory
laboratory exercises for
for baste
basic content
content
1. To
(anemia and
and coronary
coronary artery
artery disease) in
in second-year
second-year medical
medical education,
education,
areas (anemia
oriented toward
toward the processes of
of diagnosis
diagnosis and evaluation,
evaluation, utilizing
utilizing
oriented
of medical
medical decision
decision science, critiquing,
critiquing, and software
software engineering
engineering
techniques of
techniques
(the PLAN-ALYZER
PLAN-AL YZER system);
system);
(the
To utilize
utilize the computerized
computerized teaching
teaching modules
modules to test
test· the hypothesis
hypothesis that
that
2. To
students
process-oriented educational
students with
with access
access to process-oriented
educational tools
tools can integrate
integrate their
their
didactic
non-process
didactic knowledge
knowledge more
more effectively
effectively than
than with
with access
access only
only to non-process
oriented
education,
oriented traditional
traditional
education, including
including lecture
lecture notes, texts, and nonnonintelligent
intelligent audiovisual
audiovisual aids;
aids:
3. To
practical application
versions
YZER
To develop
develop practical
application
versions of
of the two
two PLAN-AL
PLAN-ALYZER
models
models that
that can be used
used as
as diagnostic
diagnostic tools with
with more
more senior
senior medical
medical
students,
students, residents,
residents, and physicians
physicians in
in the clinic,
clinic, providing
providing them with
with :J.a
decision
decision analysis
analysis tool
tool and expert
expert critiques
critiques of
of their
their evaluations
evaluations of
of real
patients;
patients;
4. To
To utilize
utilize the originally
originally proposed
proposed and the advanced systems to explore
explore the
process of
of how the effective
effective physician
physician solves clinical
clinical problems,
problems, a process
process
which
which has been found
found to be different
different from
from traditional
traditional problem
problem solving.
solving.
PLAN-AL
YZER prototyping
is being
PLAN-ALYZER
prototyping
being accomplished
accomplished on the Macintosh
Macintosh Plus and
Macintosh
Novel
Macintosh II
II workstations,
workstations, using the Macintosh
Macintosh Programmer's
Programmer’s Workshop.
Workshop.
Novel AI
Al
features
YZERs include
features of
of the PLAN-AL
PLAN-ALYZERs
include a scoring
scoring metric
metric based
based on unate boolean
functions,
functions, to compare
compare students'
students’ decision
decision trees with
with gold standard
standard trees,
trees, a mechanism
mechanism by
which
which augmented
augmented transition
transition network
network critiques
critiques can be developed for
for decision
decision models, and
the encoding
encoding of
of the domain
domain experts'
experts’ instructional
instructional styles as
as well as
as content
content into
into the
models.
An
team of
An interdisciplinary
interdisciplinary
of computer
computer scientists,
scientists, physicians,
physicians, and
and educators
educators is working
working
puter- based Exercises project.
on
on the Com
Computer-based
project. A prototype
prototype system
system is
is nearing completion,
completion,
with
with formative
formative evaluation
evaluation scheduled
scheduled for
for Fall,
Fall, 1987.
1987.
215
215
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR0078514
5P41-RR00785-14
Pathophysiologic Diagnosis
Diagnosis Project
Project
Pathophysiologic
D. Relevant
Relevant Publications
Publications
D.
Beck, J.R., Prietula,
Prietula, M.J.,
M.J., Russo, E.A.:
E.A.: AA role
role for
for intelligent
intelligent systems in
in teaching
teaching medical
medical
Beck,
Jorgenson, M.
M. (eds). Proc.
Proc. Fifth
Fifth Conf.
Conf.
pathophysiology. In:
In: Salamon,
Salamon, R.,
R., Blum,
Blum, B., Jorgenson,
pathophysiology.
Med. Inform.
Inform. (MEDINFO
(MEDINFO ‘'86),
Elsevier-North Holland,
Holland, Amsterdam,
Amsterdam, 1986, 936-938.
936-938.
Med.
86), Elsevier-North
Artificial intelligence:
intelligence: AA topic
topic for
for Medical
Medical Decision
Decision Making?
Making? (edit.)
(edit.) Med.
Med.
Beck, J.R.: Artificial
Beck,
Decis. Making
Making 1987; 7:4.
Decis.
II. INTERACTIONS
INTERACfIONS WITH
WITH THE
THE SUMEX-AIM
SUMEX-AIM RESOURCE
RESOURCE
II.
The Dartmouth
Dartmouth group
group is pleased to
to be a new
new addition
addition to
to the SUMEX
SUMEX research resource.
resource.
The
of our
our projects
projects take
take place
place on
on the Dartmouth
Dartmouth campus,
campus, but
but we require
require access
access to
to the
Most of
Most
national AI
AI community
community in
in order
order to
to share ideas, disseminate
disseminate research results,
results, and grant
grant
national
our trainees
trainees and
and junior
junior faculty
faculty access
access to the developments
developments of
of others.
others. Also,
Also, inasmuch
inasmuch as
our
our research in
in medical
medical educational
educational applications
applications of
of computer
computer science and decision
decision
our
significant potential
potential for
for dissemination,
dissemination, the SUMEX
community of
of scholars
scholars
making has significant
making
SUMEX community
forms a natural
natural .group
for focusing
focusing and broadening
broadening our
our research ideas.
forms
group for
E. H. Shortliffe
Shortliffe
216
SP41-RR0078"S-14
5P41-RR00785-14
AIM Management
Management Committee
Committee Membership
Membership
AIM
Appendix AA
Appendix
AIM Management
Management Committee
Committee Membership
Membership
AIM
Following are the
the current
current membership
membership lists
lists of
of the
the various
various SUMEX-AIM
SUMEX-AIM management
management
Following
committees:
committees:
AIM Executive
Executive Committee:
Committee:
AIM
(Chairman)
SHORTLIFFE, Edward
Edward H.,
H., M.D.,
M.D., Ph.D.
Ph.D.
(Chairman)
SHORTLIFFE,
Principal Investigator
Investigator -- SUMEX
SUMEX
Principal
Medical School
School Office
Office Building,
Building, Rm.
Rm. X271
X271
Medical
Stanford University
University Medical
Medical Center
Center
Stanford
Stanford, California
California 94305
9430S
Stanford,
(415) 723-6970
723-6970
(415)
FEIGENBAUM, Edward
Edward A.,
A., Ph.D.
Ph.D.
FEIGENBAUM,
Co-Principal Investigator
Investigator - SUMEX
SUMEX
Co-Principal
Programming Project
Project
Heuristic Programming
Heuristic
Department of
of Computer
Computer Science
Department
Welch Road,
Road, Building
Building C
701 Welch
Stanford
University
Stanford University
California 94305
Stanford, California
Stanford,
(41S)
(415) 723-4879
723-4879
KULIKOWSKI,
Casimir,
KULIKOWSKI,
Casimir, Ph.D.
Ph.D.
Department
Department of
of Computer
Computer Science
Rutgers
Rutgers University
University
New
New Brunswick,
Brunswick, New
New Jersey 08903
(201)
(201) 932-2006
932-2006
LEDERBERG,
LEDERBERG, Joshua, Ph.D.
Ph.D.
President
President
The
The Rockefeller
Rockefeller University
University
1230 York
York Avenue
Avenue
New
New York,
York, New
New York
York 10021
(212) 5708080, 570-8000
570-8080,
570-8000
Adv Grp
Grp Chrmn)
Chrmn)
LINDBERG,
(Past Adv
LINDBERG, Donald
Donald A.B., M.D.
M.D.
Director,
Director, National
National Library
Library of
of Medicine
Medicine
8600 Rockville
Rockville Pike
Pike
Bethesda, Maryland
Maryland 20814
(301)496-6221
(301)496-6221
MYERS,
MYERS, Jack
Jack D.,
D., M.D.
M.D.
School
School of
of Medicine
Medicine
Scaife
Scaife Hall,
Hall, 1291
1291
University
University of
of Pittsburgh
Pittsburgh
Pittsburgh,
Pittsburgh, Pennsylvania
Pennsylvania 15261
15261
(412)
(412) 648-9933
648-9933
217
217
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
AI"M
AIM Management
Management Committee
Committee Membership
Membership
AIM Advisory
Advisory Group:
AIM
Group:
(Chairman)
MYERS,
(Chairman)
MYERS, Jack D., M.D.
M.D.
School of
of Medicine
Medicine
School
Scaife Hall,
Hall, 1291
Scaife
University
University of
of Pittsburgh
Pittsburgh
Pittsburgh, Pennsylvania
Pennsylvania 15261
Pittsburgh,
(412)
(412) 648-9933
648-9933
AMAREL,
AMAREL, Saul, Ph.D.
Ph.D.
Department
Department of
of Computer
Computer Science
Rutgers University
University
Rutgers
New Brunswick,
New Jersey 08903
New
Brunswick, New
(201)
(201) 932-3546
932-3546
(Exec. Secretary)
Secretary)
COULTER,
(Exec.
COULTER, Charles
Charles L., Ph.D.
Ph.D.
Bldg
Bldg 31, Room
Room 5B41
Biomedical
Biomedical Research Technology
Technology Program
Program
National
Institutes of
of Health
Health
National Institutes
Rockville Pike
Pike
9000 Rockville
Maryland 20892
Bethesda, Maryland
(301)
(301) 496-5411
496-5411
FEIGENBAUM,
Edward
FEIGENBAUM,
Edward A., Ph.D.
Ph.D. (Ex-ofncio)
(Ex-officio)
Co-Principal
Co-Principal Investigator
Investigator - SUMEX
SUMEX
Heuristic
Heuristic Programming
Programming Project
Project
Department
Department of
of Computer
Computer Science
701 Welch
Welch Road, Building
Building C
Stanford
Stanford University
University
Alto, ,California
California 94305
Palo Alto,
(415)
(415) 723-4879
723-4879
KULIKOWSKI,
Casimir,
KULIKOWSKI,
Casimir, Ph.D.
Ph.D.
Department of
of Com
Computer
Department
puter Science
Hill
Hill Center
Center Busch Campus
Campus
Rutgers
Rutgers University
University
New Brunswick,
New Jersey 08903
New
Brunswick, New
(201)
(201) 932-2006
932-2006
LEDERBERG,
LEDERBERG, Joshua, Ph.D.
Ph.D.
President
President
The
The Rockefeller
Rockefeller University
University
York Avenue
Avenue
1230 York
New York,
New York
New
York, New
York 10021
(212)
(212) 570-8080,
570-8080, 570-8000
570-8000
LINDBERG,
LINDBERG, Donald
Donald A.B.,
A-B., M.D.
M.D.
Director, National
Library of
of Medicine
Medicine
Director,
National Library
Building
Buiiding 38, Rm.
Rm. 2E-17B
2E-17B
Rockville Pike
Pike
8600 Rockville
Maryland 20814
Bethesda, Maryland
(301)
(301) 496-6221
496-6221
Shortliffe
E. H. Shortliffe
218
5P41-RR00785-14
5P41-RR00785-14
Committee Membership
Membership
AIM Management
Management Committee
AIM
MINSKY, Marvin,
Marvin, Ph.D.
Ph.D.
MINSKY,
Artificial Intelligence
Intelligence Laboratory
Laboratory
Artificial
of Technology
Technology
Massachusetts Institute
Institute of
Massachusetts
545
Technology
Square
545 Technology Square
Cambridge, Massachusetts
Massachusetts 02139
02139
Cambridge,
(617) 253-5864
253-5864
(617)
MOHLER, William
William C.,
C., M.D.
M.D.
MOHLER,
Associate
Director
Associate Director
of Computer
Computer Research
Research and
and Technology
Technology
Division of
Division
of Health
Health
National Institutes
Institutes of
National
Building 12A,
12A, Room
Room 3033
3033
Building
9000 Rockville
Rockville Pike
Pike
9000
Bethesda. Maryland
Maryland 20892
20892
Bethesda,
(301) 496-1168
496-1168
(301)
M.D.
PAUKER. Stephen
Stephen G.,
G .. M.D.
PAUKER,
Department of
of Medicine
Medicine -- Cardiology
Cardiology
Department
Tufts New
New England
England Medical
Medical Center
Center Hospital
Hospital
Tufts
171 Harrison
Harrison Avenue
Avenue
Boston. Massachusetts 02111
02111
Boston,
(617) 956-5910
956-5910
(617)
SHORTLIFFE, Edward
Edward H.,
H.. M.D.,
M.D .• Ph.D.
(Ex-officio)
Ph.D.
(Ex-officio)
SHORTLIFFE,
Principal
Investigator
SUMEX
Principal Investigator
SUMEX
Medical School
School Office
Office Building,
Building, Rm.
Rm. X271
X271
Medical
Stanford University
University Medical
Medical Center
Center
Stanford
California 94305
Stanford, California
Stanford,
(415)
723-6979
(415) 723-6979
SIMON,
SIMON, Herbert
Herbert A., Ph.D.
Ph.D.
Department
Department of
of Psychology
Psychology
Baker Hall.
Hall, 339
Carnegie-Mellon
Carnegie-Mellon University
University
Schenley Park
Park
Pittsburgh.
Pittsburgh, Pennsylvania
Pennsylvania 15213
(412) 578-2787.
578-2787, 578-2000
578-2000
219
219
E. H.
H. Shortliffe
Shortiiffe
E.
5P41-RR00785-14
5P41- RR00785-14
AIM Management
Management Committee
Committee Membership
Membership
AIM
Stanford Community
Community Advisory
Advisory Committee:
Committee:
Stanford
FEIGENBAUM, Edward
Edward A.,
A., Ph.D.
Ph.D. (Chairman)
(Chairman)
FEIGENBAUM,
Heuristic Programming
Programming Project
Project
Heuristic
of Computer
Computer Science
Science
Department of
Department
Margaret Jacks Hall
Hall
Margaret
Stanford University
University
Stanford
Stanford, California
California 94305
94305
Stanford,
(415) 723-4879
723-4879
(415)
LEVINTHAL, Elliott
Elliott C., Ph.D.
Ph.D.
LEVINTHAL,
Departments of
of Mechanical
Mechanical and
and Electrical
Electrical Engineering
Engineering
Departments
Building 530
Building
Stanford University
University
Stanford
Stanford, California
California 94305
Stanford,
(415) 723-9037
723-9037
(415)
M.D., Ph.D.
Ph.D.
SHORTLIFFE, Edward
Edward H., M.D.,
SHORTLIFFE,
Principal Investigator
Investigator - SUMEX
SUMEX
Principal
Medical School
School Office
Office Building,
Building, Rm.
Rm. X271
X271
Medical
Stanford University
University Medical
Medical Center
Center
Stanford
Stanford, California
California 94305
Stanford,
(415) 723-6979
723-6979
(415)
E.
E. H.
H. Shortliffe
Shortliffe
220
220
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
5P41-RR00785-14
5P41-RR00785-14
Appendix BB
Appendix
Subproject Abstracts
Abstracts
Scientific Subproject
Scientific
The following
following are brief
brief abstracts
abstracts of
of our
our collaboratjve
collaborative research projects.
projects.
The
221
221
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
Stanford Project:
Project:
Stanford
GUIDON/NEOMYCIN
GUIDON/NEOMYCIN --KNOWLEDGE
KNOWLEDGE ENGINEERING
ENGINEERING
FOR
TEACHING MEDICAL
MEDICAL DIAGNOSIS
DIAGNOSIS
FOR TEACHING
Principal Investigators:
Investigators:
Principal
William J. Clancey,
Clancey, Ph.D.
Ph.D.
William
Welch Road
Road
701 Welch
Department of
of Computer
Computer Science
Department
Stanford University
University
Stanford
California 94304
Palo Alto,
Alto, California
Palo
(415) 723-1997
723-1997 (CLANCEY@SUMEX-AIM)
(CLANCEY@SUMEX-AIM)
(415)
Bruce G.
G. Buchanan,
Buchanan, Ph.D.
Ph.D.
Bruce
Computer Science Department
Department
Computer
701
701 Welch
Welch Road
Road
Stanford
University
Stanford University
Palo
California 94304
Palo Alto,
Alto, California
(415) 723-0935
723-0935 (BUCHANANGSUMEX-AIM)
(BUCHANAN@SUMEX-AIM)
(415)
ARE AVAILABLE
A VAILABLE ON
ON SUMEX
SUMEX
SOFTW
SOFTWARE
GUIDON--A system developed
developed for
for intelligent
intelligent computer-aided
computer-aided instruction.
instruction. Although
Although itit
GUIDON--A
developed in
in the context
context of
of MYCIN’
MYCIN's
infectious disease knowledge
knowledge base,
base, the tutorial
tutorial
was developed
s infectious
EMYCIN knowledge
knowledge base.
base.
rules will
will operate
operate upon
upon any
any EMYCIN
rules
NEOMYCIN--A consultation
consultation system derived
derived from
from MYCIN,
MYCIN, with
with the knowledge
knowledge base
NEOMYCIN--A
In
for use
use in
in teaching.
teaching.
In contrast
contrast with
with MYCIN,
MYCIN,
greatly extended
extended and reconfigured
reconfigured for
greatly
diagnostic
diagnostic procedures,
procedures, common
common sense
sense facts,
facts, and disease
disease hierarchies
hierarchies are factored
factored out
out of
of
The diagnostic
diagnostic procedures
procedures are abstract
abstract (not
(not
the basic
associations.
The
basic finding/disease
finding/disease
associations,
specific
specific to any problem
problem domain)
domain) and model
model human
human reasoning,
reasoning, unlike
unlike the exhaustive,
exhaustive,
top-down
top-down approach
approach implicit
implicit in
in MYCIN's
MYCIN’s medical
medical rules. This
This knowledge
knowledge base
base will
will be
used in
family
programs,
in the GUIDON2
GUIDON2
family of
of instructional
instructional
programs, being
being developed
developed on DDmachines.
machines.
REFERENCES
REFERENCES
Clancey,
Clancey, W.J.: Knowledge-Based
Knowledge-Based Tutoring:
Tutoring: The GUIDON
GUIDON Program,
Cambridge:
Cambridge: The
The MIT
MIT Press,
Press, 1987.
for building
Clancey,
Clancey, W.J.: Methodology
Methodology for
building an intelligent
intelligent tutoring
tutoring system.
In Kintsch,
Kintsch, Polson,
Polson, and Miller,
Miller, (Eds.),
(Eds.), METHODS
METHODS AND
AND TACTICS
TACTICS IN
IN COGNITIVE
COGNITIVE
SCIENCE.
SCIENCE. L. Erlbaum
Erlbaum Assoc., Hillsdale,
Hillsdale, NJ. 1984. (Also
(Also STAN-CS-81-894,
STAN-CS-81-894,
HPP
HPP 81-18)
81-18)
Clancey,
Clancey, W.J.: Acquiring,
Acquiring, representing,
representing, and evaluating
evaluating a competence
model
mode! of
of diagnosiS.
diagnosis. In Chi,
Chi, Glaser,
Glaser, and Farr
Farr (Eds.),
(Eds.), THE
THE NATURE
NATURE
OF
In preparation.
OF EXPERTISE.
EXPERTISE.
preparation. HPP-84-2.
HPP-84-2.
E. H. Shortliffe
Shortliffe
222
5P41-RR00785-14
5P41-RR00785-14
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
Stanford Project:
Project:
Stanford
MOLGEN ---- AN
AN EXPERIMENT
EXPERIMENT PLANNING
PLANNING SYSTEM
SYSTEM
MOLGEN
FOR MOLECULAR
MOLECULAR GENETICS
GENETICS
FOR
Principal Investigators:
Investigators:
Principal
Edward A.
A. Feigenbaum,
Feigenbaum, Ph.D.
Ph.D.
Edward
Department of
of Computer
Computer Science
Science
Department
Stanford University
University
Stanford
Charles Yanofsky,
Yanofsky, Ph.D.
Ph.D. (YANOFSKY@SUMEX-AIM)
(YANOFSKY@SUMEX-AIM)
Charles
Department
of
Biology
Department of Biology
Stanford University
University
Stanford
Stanford, California
California 94305
94305
Stanford,
(415) 725-3815
725-3815
(415)
FRIEDLAND@SUMEX-AIM
Contact: Dr.
Dr. Peter
Peter FRIEDLAND@SUMEX-AIM
Contact:
(415) 723-3728
723-3728
(415)
The MOLGEN
MOLGEN project
project has focused
focused on
on research
research into
into the
the applications
applications of
of symbolic
symbolic
The
computation and
and inference
inference to
to the
the field
field of
of molecular
molecular biology.
biology. This
This has taken
taken rhe
the
computation
form of
of systems which
which provide
provide assistance
assistance to
to the
the experimental
experimental scientist
scientist in
in
specific form
specific
most
important
of
which
have
been
the
design
of
complex
experiment
various
tasks,
the
various
the most important of which
the
of complex experiment
plans and
and the
the analysis
analysis of
of nucleic
nucleic acid
acid sequences. Our
Our current
current research concentrates
concentrates on
on
plans
scientific discovery
discovery within
within the
the subdomain
subdomain of
of regulatory
regulatory genetics.
genetics. We
We desire
desire to
to explore
explore
scientific
to modify,
modify, extend,
extend, and test theories
theories of
of genetic
genetic
the methodologies
methodologies scientists
scientists use to
the
emulate that
that process within
within a computational
computational system.
regulation, and then
then emulate
regulation,
or model
model formation
formation is a fundamental
fundamental part
part of
of scientific
scientific research. Scientists
Scientists both
both
Theory or
Theory
form such models
models dynamically.
dynamically. They
They are used to predict
predict results
results (and
(and therefore
therefore
use and form
experiments to
to test the model)
model) and also to
to explain
explain experimental
experimental results.
results.
to suggest experiments
to
revised both
both as
as a result
result of
of logical
logical conclusions
conclusions from
from existing
existing
Models are extended
extended and revised
Models
of new experimental
experimental evidence.
evidence.
premises and as a result
result of
premises
Theory
cognitive
in which
which there
there is substantial
Theory formation
formation is a difficult
difficult
cognitive task, and one in
substantial
toward building
building a system
scope for
Our research is toward
for intelligent
intelligent computational
computational assistance. Our
which
interact with
with a scientist
scientist
which can form
form theories
theories to explain
explain experimental
experimental evidence,
evidence, can interact
to help to suggest experiments
experiments to discriminate
discriminate among
among competing
competing hypotheses, and
and can
then revise and extend
extend the growing
growing model
model based
based upon
upon the results of
of the experiments.
experiments.
The
The MOLGEN
MOLGEN project
project has continuing
continuing computer
computer science goals of
of exploring
exploring issues
issues of
of
knowledge
representation,
problem-solving,
discovery,
and
planning
within
a
real
knowledge representation, problem-solving,
discovery,
planning within
reai and
xnd
complex
The
The project
project operates in a framework
framework of
of collaboration
collaborarion between the
complex domain.
domain.
Heuristic
Heuristic Programming
Programming Project
Project (HPP)
(HPP) in the Computer
Computer Science Department
Department and various
various
domain
It
domain experts in the
[he departments
departments of
of Biochemistry,
Biochemistry, Medicine,
Medicine, and Biology.
Biology.
It draws
from
which deal with
wirh applications
applications
from the experience
experience of
of several other
other projects
projects in the HPP which
of
of artificial
artificial intelligence
intelligence to medicine,
medicine, organic
organic chemistry,
chemistry, and engineering.
engineering.
SOFTWARE
ON SUMEX
SOFTWARE AVAILABLE
AVAILABLE
SUMEX
SPEX
SPEX system
system for
for experiment
experiment design.
design.
UNITS
UNITS system
system for
for knowledge
knowledge representation
representation and
and acquisition.
acquisition.
for
nucleotide
sequence
analysis.
SEQ
system
SEQ system for nucleotide sequence analysis.
REFERENCES
REFERENCES
1.
1. Friedland,
Friedland, P.E.:
P.E.: Knowledge-based
Knowledge-based experiment
experiment design
design in
in molecular
molecular genetics,
genetics,
(Ph.D.
(Ph.D. thesis).
thesis). Stanford
Stanford Computer
Computer Science
Science Report,
Report, STAN-CS-79-771.
STAN-CS-79-771.
223
223
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Scientific
Scientific Subproject
Subproject Abstracts
Abstracts
2. Friedland,
skeletal
Friedland, P.E. and Iwasaki,
Iwasaki, Y.: The concept and
and implementation
implementation of
of skeletal
plans, Journal
plans,
Journal of
of Automated
Automated Reasoning,
Reasoning, 1(2):161-208,
1(2):161-208, 1985.
3. Friedland,
P.E. and Kedes, L.:
Discovering the secrets
secrets of
Friedland,
L.: Discovering
of
Communications
November, 1985.
Communications of
of the
the ACM,
ACM, 28(11):1164-1186,
28(11):1164-1186, November,
DNA,
DNA,
4. Stefik,
frame-structured representation
system,
Stefik, MJ.:
M.J.: An examination
examination of
of a frame-structured
representation system,
Proc.
Proc. Sixth
Sixth IJCAI,
IJCAI, Tokyo,
Tokyo, August,
August, 1979, pp. 845-852.
845-852.
5. Stefik,
Planning with
Stefik, MJ.:
M.J.: Planning
with constraints,
constraints, (Ph.D.
(Ph.D. thesis).
thesis).
Science Report,
March,
Report, STAN-CS-80-784,
STAN-CS-80-784,
March, 1980.
Stanford
Stanford Computer
Computer
6. Karp,
Analysis of
Deep/Shallow Distinction
Distinction for
for
Karp, P., and D. Wilkins:
Wilkins: An Analysis
of the Deep/Shallow
Expert Systems.
Expert
Systems. Stanford
Stanford University
University Knowledge
Knowledge Systems Laboratory
Laboratory Report
Report
KSL-86-32,
KSL-86-32, 1986.
7. Karp,
the Use of
and
Karp, P., and P. Friedland:
Friedland: Coordinating
Coordinating
of Qualitative
Qualitative
and
Quantitative
Knowledge in
Declarative Device
Device Modeling.
Modeling. Stanford
Quantitative Knowledge
in Declarative
Stanford University
University
Knowledge
Knowledge Systems Laboratory
Laboratory Report
Report KSL-87-09,
KSL-87-09, 1987.
8. Round,
Environment for
for the Qualitative
&SOPS: A Workbench
Workbench
Environment
Qualitative
Round, A.: QSOPS:
Simulation
Physical Processes.
Processes. Stanford
Simulation of
of Physical
Stanford University
University Knowledge
Knowledge Systems
Laboratory
Laboratory Report
Report KSL-87-37,
KSL-87-37, 1987.
E. H. Shortl
iffe
Shortliffe
224
Scientific
Scientific Subproject
Subproject Abstracts
Abstracts
5P41-RR00785-14
5P41-RR00785-14
Stanford Project:
Project:
Stanford
ONCOCIN --- KNOWLEDGE
KNOWLEDGE ENGINEERING
ENGINEERING FOR
FOR
ONCOCIN
ONCOLOGY CHEMOTHERAPY
CHEMOTHERAPY CONSULTATION
CONSULTATION
ONCOLOGY
Principal Investigator:
Investigator:
Principal
Edward H.
H. Shortliffe,
Shortliffe, M.D.,
M.D., Ph.D.
Ph.D.
Edward
Departments of
of Medicine
Medicine and
and Computer
Computer Science
Science
Departments
Stanford University
University Medical
Medical Center
Center
Stanford
Medical School
School Office
Office Building
Building
Medical
Stanford, California
California 94305
Stanford,
(SHORTLIFFE@SUMEX-AIM)
(415) 723-6979
723-6979 (SHORTLIFFE@SUMEX-AIM)
(415)
Project Director:
Director:
Project
Dr. Lawrence
Lawrence M.
M. Fagan (FAGANGSUMEX-AIM)
(FAGAN@SUMEX-AIM)
Dr.
The ONCOCIN
ONCOCIN Project
Project is overseen by
by a collaborative
collaborative group
group of
of physicians
physicians and
and computer
computer
The
scientists who
who are developing
developing an intelligent
int~lligent system that
that uses
uses the
the techniques
techniques of
of knowledge
knowledge
scientists
in the management
management of
of patients
patients receiving
receiving cancer
cancer
engineering to
to advise
advise oncologists
oncologists in
engineering
The general
general research foci
foci of
of the group
group members
members include
include knowledge
knowledge
chemotherapy. The
chemotherapy.
acquisition, inexact
inexact reasoning,
reasoning, explanation,
explanation, and the representation
representation of
of time
time and of
of expert
expert
acquisition,
thinking patterns.
patterns. Much
Much of
of the work
work developed
developed from
from research in
in the 1970's
1970’s on the
thinking
efforts that
that helped
helped define
define the group’
group's
MYCIN and EMYCIN
EMYCIN programs,
programs, early
early efforts
MYCIN
s research
directions for
for the coming
coming decade. MYCIN
MYCIN and EMYCIN
EMYCIN are still
still available
available on SUMEX
SUMEX
directions
for demonstration
demonstration purposes.
for
The prototype
prototype ONCOCIN
ONCOCIN system is in
in limited
limited experimental
experimental use
use by oncologists
oncologists in
in the
The
Stanford Oncology
Oncology Clinic.
Clinic. Thus,
Thus. much
much of
of the emphasis
emphasis of
of this
this research has been on
Stanford
human engineering
engineering so that
that the physicians
physicians will
will accept
accept the program
program as a useful
useful adjunct
adjunct to
human
ONCOCIN has generally
generally been well-accepted
well-accepted since
since its
its
their patient
patient care activities.
activities. ONCOCIN
their
introduction,
and we are now
now testing
testing a version
version of
of the program
program which
which runs
runs on
introduction,
professional
professional workstations
workstations (rather
(rather than
than the central
central SUMEX
SUMEX computer)
computer) so that
that it
it can be
implemented
implemented and evaluated
evaluated at sites away from
from the University.
University.
SOFTW
ARE A
VAILABLE ON
SOFTWARE
AVAILABLE
ON SUMEX
SUMEX
MYCIN-MYCIN--
A
A consultation
consultation system designed to assist physicians
physicians with
with the selection
selection
of
It
of antimicrobial
antimicrobial therapy
therapy for
for severe infections.
infections.
It has achieved
achieved expert
expert
level
level performance
performance in
in formal
formal evaluations
evaluations of
of its ability
ability to select
seiect
therapy
therapy for
for bacteremia
bacteremia and meningitis.
meningitis. Although
Although MYCIN
MYCIN is no
no longer
longer
the subject
subject of
of an active
active research program,
program, the system continues
continues to be
available
available on SUMEX
SUMEX for
for demonstration
demonstration purposes and as
as a testing
testing
environment
environment for
for other
other research projects.
projects.
EMYCIN-EMYCIN--
The
The "essential
“essential MYCIN"
MYCIN” system is a generalization
generalization of
of
It is
knowledge
It
knowledge representation
representation and control
control structure.
structure.
facilitate
facilitate the development
development of
of new expert
expert consultation
consultation
both
both clinical
clinical and non-medical
non-medical domains.
domains.
ONCOCIN-ONCOCIN--
This
This system is in
in clinical
clinical use
use but
but requires
requires Lisp
Lisp machines
machines to be run.
run.
Much
Much of
of the knowledge
knowledge in the domain
domain of
of cancer chemotherapy
chemotherapy is
already
in
already welI-specified
well-specified
in protocol
protocol documents,
documents, but
but expert
expert judgments
judgments
also need to be understood
understood and modeled.
modeled.
225
22S
the MYCIN
MYCIN
designed to
systems for
for
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
REFERENCES
REFERENCES
Shortliffe, E.H.,
E.H., Scott, A.C.,
A.C., Bischoff,
Bischoff, M.B.,
M.B., Campbell,
Campbell, A.B., van Melle,
Melle,
1. Shortliffe,
W. and Jacobs,
Jacobs, C.D.: ONCOCIN:
ONCOCIN: An
An expert
expert system
system for
Jor oncology
oncology protocol
protocol
W.
management. Proc. Seventh IJCAI,
IlCAI, pp. 876-881, Vancouver,
Vancouver, B.C., August,
management.
1981.
2. Duda, R.O. and
Shortliffe, E.H.: Expert
Expert
Shortliffe,
systems
systems
research.
research.
Science
220:261-268, 1983.
220:261-268,
Langlotz, C.P. and Shortliffe,
Shortliffe, E.H.: Adapting
Adapting aa consultation
consultation system
system to
3. Langlotz,
critique user
user plans.
plans. Int.
Int. J. Man-Machine
Man-Machine Studies 19:479-496, 1983.
critique
Bischoff, M.B., Shortliffe,
ShOI:tliffe, E.H., Scott, AC.,
A.C., Carlson, R.W.
R.W. and Jacobs,
Jacobs, C.D.:
4. Bischoff,
Integration of
oj aa computer-based
computer-based consultant
consultant into
into the
the clinical
clinical setting.
setting.
Integration
Proceedings 7th
7th Annual
Annual Symposium
Symposium on Computer
Computer Applications
Applications in Medical
Medica!
Proceedings
Baltimore, Maryland,
Maryland, October
October 1983.
Care, pp. 149-152, Baltimore,
Hickam, D.H., Shortliffe,
Shortliffe, E.H., Bischoff,
Bischoff, M.B., Scott, A.C., Jacobs,
Jacobs, C.D.:
CD.: A
A
5. Hickam,
of the treatment
treatment advice of
of a computer-based
computer-based cancer chemotherapy
study of
Annals of
oj Internal
Internal Medicine
Medicine 103(6 pt
pt
protocol advisor (Memo
(Memo KSL-85-21).
KSL-85-21). Annals
protocol
1):928-936 (1985).
Kent; D.L., Shortliffe,
Bischoff, M.B., Jacobs,
Jacobs, C.D.:
6. Kent;
Shortliffe, E.H., Carlson, R.W., Bischoff,
use of
of a computer-based
Improvements in data collection
collection through physician use
Improvements
Journal of
oj Clinical
treatment consultant
consultant (Memo
(Memo KSL-85-22).
KSL-85-22). Journal
chemotherapy treatment
Clinical
Oncology
Oncology 3:1409-1417, 1985.
1985.
7. Tsuji,
Tsuji. S.
S. and Shortliffe,
Shortliffe, E.H.: Graphical
Graphical access
access to a medical expert system:
system:
I. Design of
Meth.
of a knowledge engineer's
engineer’s interface
interface (Memo
(Memo KSL-85-11).
KSL-85-11).
Inr.
Inf. Med., 25:62-70,
25:62-70. 1986.
1986.
8. Lane.
Lane, C.D.,
CD., Differding,
Differding, J.c.,
J.C., Shortliffe,
Shortliffe, E.H.: Graphical
Graphical access
access to a medical
expert system:
system: II. Design of
of an interface
interface for
for physicians (Memo
(Memo KSL-85-15).
KSL-85-15).
Meth.
Meth Inr.
Inf Med.,
Med 25:143-150,
25:143-150. 1986.
1986.
ALL)
for physicians.
9.
Medical expert
expert systems:
systems: Knowledge tools for
9. Shortliffe,
Shortliffe, E.H. Medical
Memo KSL-86-52.
Special issue
issue on Medical Informatics,
lnformatics, West. L
J. Med.
KSL-86-52.
145:830-839,
145:830-839, 1986.
1986.
E.
E. H.
H. Shortliffe
Shortliffe
226
226
Scientific
Scientific Subproject Abstracts
5P41-RR00785-14
Stanford Project:
Stanford
PROTEAN Project
PROTEAN
Principal Investigators:
Principal
Oleg Jardetzky
(JARDETZKY@SUMEX-AIM.STANFORD.EDU)
(JARDETZKY@SUMEX-AIM.STANFORD.EDU)
Nuclear Magnetic Resonance
Resonance Lab, School of
of Medicine
Medicine
Stanford
Stanford University
University Medical Center
Stanford, California
California 94305
94305
Bruce G. Buchanan,
Buchanan, Ph.D.
(BUCHANAN@SUMEX-AIM.STANFORD.EDU)
(BUCHANAN@SUMEX-AIM.STANFORD.EDU)
Science D~partment
Department
Computer Science
Stanford University
University
Stanford
California 94305
94305
Stanford, California
Person:
Con tact Person:
Buchanan
Bruce G. Buchanan
The goals
goals of
of this project are related both to biochemistry
biochemistry and artificial
artificial intelligence:
intelligence: (a)
use
use existing AI
AI methods
methods to aid in the determination
determination of
of the 3-dimensional
3-dimensional structure of
of
proteins in solution
protein
solution (not
(not from
from x-ray
x-ray crystallography proteins),
proteins), and (b) use
use protein
problem-solving
structure determination
determination as
as a test
test problem for
for experiments with
with the AI
AI problem-solving
Empirical data from
from nuclear magnetic
structure known as
as the Blackboard Model. Empirical
resonance
resonance (NMR)
(NMR) and other sources
sources may provide enough
enough constraints
constraints on structural
structural
descriptions to allow protein
protein chemists to bypass
bypass the laborious
laborious methods of
of crystallizing
crystallizing a
This problem
problem
protein
-ray crystallography to determine its structure.
This
protein and using
using X
X-ray
exhibits
exhibits considerable
considerable complexity,
complexity, yet there is reason
reason to believe that
that AI
AI programs can be
A prototype
written
A
prototype
written that reason
reason much as
as experts do to resolve
resolve these
these difficulties.
difficulties.
knowledge-based
knowledge-based system
system assembles
assembles major secondary
secondary structures of
of a protein
protein into
into families
families
of
of structures compatible with
with a given set
set of
of distance constraints
constraints under the control
control of
of an
explicit
explicit assembly
assembly strategy.
strategy. Structures can
can also
also be refined
refined at the atomic
atomic level of
of detail
using constraints within
within secondary
secondary structures
structures and between
between amino
amino acid side chains to
further
further restrict
restrict the 3-dimensional
3-dimensional structure found. By generalizing this approach to the
assembly
assembly of
of arrangements
arrangements of
of objects
objects subject to constraints, we have
have developed
developed a
language
language for
for specifying actions and control
control for
for problem solving
solving in similar
similar problem
problem
domains.
REFERENCES
REFERENCES
strategies for
determination of
o-f
1.
for the determination
1. Altman,
Altman, R. and Jardetzky,
Jardetzky, 0.: New strategies
of Biochemistry
Biochemistry (Tokyo),
(Tokyo),
macromolecular
macromolecular structures
structures in solution.
soiution. Journal of
Vol. 100,
100, No.6,
No. 6, p. 1403-1423,
1403-1423, 1986.
1986.
Compilation of
of Control
Control Knowledge.
2.
Partial Compilation
Knowledge.
2. Altman,
Altman, R. and Buchanan,
Buchanan, B.G.: Partial
appear in: Proceedings
Proceedings of
of the AAAI
AAAI 1987.
1987.
To appear
3.
3. Brinkley,
Brinkley, J.,
J., Cornelius, c.,
C., Altman,
Altman, R., Hayes-Roth, B.
B. Lichtarge, 0.,
0.. Duncan,
B., Buchanan,
Application of
Buchanan, B.G.,
B.G., Jardetzky,
Jardetzky, 0.:
0.: Application
of Constraint
Constraint Satisfaction
Satisfaction
of Protein
Tertiary Structure.
Structure. Report
Techniques to the Determination
Determination of
Protein Tertiary
KSL-8628, Department of
KSL-86-28,
of Computer Science,
Science, 1986.
1986.
4. Brinkley,
Brinkley, James
James F., Buchanan,
Buchanan, Bruce
Bruce G., Altman,
Altman, Russ
Russ B.,
B., Duncan, Bruce S.,
S.,
4.
Cornelius, Craig W.: A Heuristic
Heuristic Refinement
Refinement Method
Method for
for Spatial
Spatial Constraint
Constraint
Satisfaction Problems.
I(SL 87-05,
S7-05, Department
Department of
of Computer
Computer Science.
Science.
Satisfaction
Problems. Report KSL
227
E. H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Scientific Subproject Abstracts
Scientific
Hayes-Roth, B., Lichtarge,
Lichtarge, O.,
0., Altman,
Altman, A., Brinkley,
Brinkley, J.,
1.,
5. Buchanan, B.G., Hayes-Roth,
Cornelius, C., Duncan, B., Jardetzky, O.:The Heuristic
Heuristic Refinement
Refinement
Hewett, M., Cornelius,
Method for
for Deriving
Deriving Solution
Solution Structures
Structures of
of Proteins.
Proteins. Report
Report KSL-85-41.
KSL-85-41.
Method
1985.
October 1985.
6. Garvey, Alan,
Alan, Cornelius, Craig, and Hayes-Roth,
Hayes-Roth, Barbara: Computational
Computational
Benefits of
of Control
Control Reasoning. Report
Report KSL
KSL 87-11, Department
Department
Costs versus Benefits
of Computer
Computer Science.
Science.
of
7. Hayes-Roth, B.:
Blackboard Architecture:
Architecture: AA General
General Framework
Framework for
for
The Blackboard
Problem Solving?
Solving? Report
Report HPP-83-30,
HPP-83-30, Department
Department of
of Computer
Computer Science,
Science,
Problem
Stanford University,
University, 1983.
1983.
Stanford
BBI:
8. Hayes-Roth, B.: BBI:
that Control,
Control,
that
Explain,
Explain,
An Environment
Environment for
for Building
Building Blackboard
Blackboard Systems
Systems
An
and Learn
Learn about their
their own Behavior.
Behavior. Report
Report
and
HPP-84-16, Department
Department of
of Computer Science,
Science, Stanford
Stanford University,
University, 1984.
HPP-84-16,
1984.
9. Hayes-Roth, B.:
B.: A Blackboard
Blackboard
9.
Intelligence 26:251-321,
26:251- 321, 1985.
Intelligence
1985.
Architecture
Architecture
for
for
Control.
Control.
Artificial
Artificial
Learning Control
Control Heuristics
Heuristics in
in BBI.
BBl. Report
Report
10.
10. Hayes-Roth, B. and Hewett, M.: Learning
HPP-85-2, Department
Department of
of Computer
Computer Science,
Science, 1985.
HPP-85-2,
1985.
11.
B., Buchanan,
Buchanan, B.G., Lichtarge, O.,
0., Hewett, M., Altman,
Altman, R.,
11. Hayes-Roth, B.,
Brinkley, 1.,
c., Duncan, B., and Jardetzky, 0.: PROTEAN:
PROTEAN:
Brinkley,
J.. Cornelius, C.,
Deriving protein
protein structure
structure from
from constraints.
constraints. Proceedings
Proceedings of
of the AAAI,
AAAI, 1986,
Deriving
1986,
p. 904-909.
Method for
for the Definition
Definition of
12.
12. Jardetzky, 0.:
0.: A Method
of the Solution
Solution Structure
Structure of
of
Proteins from
from NMR
NMR and Other
Physical Measurements:
Measurements: The LAC-Repressor
LAC-Repressor
Proteins
Other Physical
Headpiece. Proceedings
Headpiece.
Proceedings of
of the International
International Conference on the Frontiers
Frontiers of
of
Biochemistry
Biochemistry and Molecular
Molecular Biology, Alma
Alma Alta,
Alta, June 17-24, 1984,
1984, October,
1984.
1984.
13.
proteins in solution
solution by NMR.
NM R.
13. Lichtarge, Olivier:
Olivier: Structure
Structure determination
determination of
of proteins
Ph.D. Thesis,
Thesis, Stanford
Stanford University,
University, November, 1986.
1986.
14.
Buchanan, Bruce G., Jardetzky, Oleg:
Oleg:
14. Lichtarge, Olivier,
Olivier, Cornelius, Craig W., Buchanan,
Validation
for
Validation of
of the First
First Step of
of the Heuristic
Heuristic Refinement
Refinement Method
Method /‘
or the
Derivation
from N
M R Data.,
April 1987.
1987.
Derivation of
of Solution
Solution Structures
Structures of
of Proteins
Proteins from
NMR
Data., April
Submitted to Proteins: Structure, Function,
Function, and Genetics.
Genetics.
E.
E. H.
H. Shortliffe
Shortliffe
228
228
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
5P41-RR00785-14
5P41-RR00785-14
Stanford Project:
Project:
Stanford
RADIX --- DERIVING
DERIVING KNOWLEDGE
KNOWLEDGE FROM
FROM
RADIX
TIME-ORIENTED CLINICAL
CLINICAL DATABASES
DATABASES
TIME-ORIENTED
Principal Investigators:
Investigators:
Principal
Robert L.
L. Blum,
Blum, M.D.
M.D.
Robert
of Medicine
Medicine
Departments of
Departments
Computer Science
Science
and Computer
Stanford University
University
Stanford
California 94305
94305
Stanford, California
Stanford,
(415) 497-9421 (BLUMGSUMEX-AIM)
(BLUM@SUMEX-AIM)
Gio C.M. Wiederhold,
Wiederhold, Ph.D.
Gio
Department of
of Computer
Computer Science
Science
Department
Stanford University
University
Stanford
California 94305
94305
Stanford, California
Stanford,
(WIEDERHOLD@SUMEX-AIM)
(415) 497-0685 (WIEDERHOLD@SUMEX-AIM)
objective of
of clinical
clinical database
database (DB)
(DB) systems
systems is to derive medical knowledge from
from the
The objective
patient observations.
process of
of reliably
reliably deriving
deriving causal
causal
stored patient
However, the process
difficult because
because of
of the complexity
complexity of
of disease
disease states
states
relationships has
has proven to be quite difficult
relationships
of bias, and problems of
of missing and outlying
outlying
and time
time relationships,
relationships, strong sources
sources of
data.
first goal of
of the RADIX
RADIX Project is to explore the usefulness
usefulness of
of knowledge-based
knowledge-based
The first
computational techniques in
in solvin,
solving0 this problem of
of accurate
accurate knowledge
knowledge inference
inference from
from
computational
non-randomized,
patient records.
base
RADIX is a knowledge base
non-randomized, non-protocol
non-protocol patient
records. Central to RADIX
(KB)
with
(KB) of
of medicine
medicine and statistics, organized as
as a taxonomic tree consisting of
of frames with
attached
procedures. The KB
used to retrieve time-intervals
KB is used
time-intervals of
of interest
interest
attached data and procedures.
from
from the DB and to assist
assist with
with the statistical analysis.
analysis. Derived knowledge is
incorporated
incorporated automatically
automatically into
into the KB. The American Rheumatism Association DB
containing
used.
containing records of
of 1700
1700 patients is used.
The second
second goal of
of the project is to develop
develop a program and set
set of
of techniques
techniques for
for
automated summarization
summarization of
of patient records.
records. The summarization program is designed
designed to
automatically
automatically create
create patient
patient summaries
summaries of
of arbitrary
arbitrary and appropriate complexity
complexity as
as an
aid for
for tasks
tasks such
such as
as clinical
clinical decision making, real-time
real-time patient monitoring,
monitoring, surveillance
of
of quality
quality of
of care,
care, and eventually automated discovery.
discovery. Two prototype summarization
summarization
modules
modules have
have been
been implemented in KEE
KEE on the
the Xerox n08
1108 workstation.
SOFTWARE
SOFTWARE AVAILABLE
AVAILABLE ON SUMEX
SUMEX
RADIX--(excluding
RADTX--(excluding the knowledge
knowledge base
base and
and clinical
clinical database)
database) consists
consists of approximately
approximately
400
400 INTERLISP
INTERLISP functions.
functions. The following
following groups
groups of functions
functions may
may be
be of interest apart
from
from the RADIX
RADIX environment:
environment:
SPSS Interface
Interface Package
Package --- Functions which create
create SPSS
SPSS source
source decks
decks and
and read
read
SPSS
SPSS listings from
from within
within INTERLISP.
INTERLISP.
Statistical
Tests
Statistical
Tests in
in INTERLISP
INTERLISP
-- Translations of the
the
approximations
approximations for
for the
the T, F,
F, and
and Chi-square ,ests
:ests into LISP.
LISP.
Piezer-Pratt
Piezer-Pratt
Time-Oriented
Time-Oriented Data
Data Base and GraphiCS
Graphics Package --- Autonomous package
package for
for
maintaining
maintaining aa time-oriented
time-oriented database
databaseand
and displaying labelled
labelled time-intervals.
time-intervals.
229
229
E.
E. H.
H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Scientific Subproject
Subproject Abstracts
AbstraCts
Scientific
REFERENCES
REFERENCES
Monograph
Monograph
Blum, R-L.:
R.L.: Discovery
Discovery and
and representation
representation of
of causal
causal relationships
relationships
Blum,
from a large
large time-oriented
time-oriented clinical
clinical database:
database: The RX
RX project.
project.
from
in D.A.B.
D.A.B. Lindberg
Lindberg and P.L. Reichertz
Reichertz (Eds.), LECTURE
LECTURE NOTES
NOTES IN
IN
in
MEDICAL INFORMATTCS,
INFORMATICS, Vol.
Vol. 19, Springer-Verlag,
Springer-Verlag, New York,
York, 1982.
1982.
MEDICAL
Journal Articles
Articles
Journal
Blum, R.L.: Computer-Assisted
Computer-Assisted Design
Design of
of Studies
Studies using
using Routine
Routine Clinical
Clinical
Blum,
Data: Analyzing
Analyzing the Association
Association of
of Prednisone
Prednisone and
and Cholesterol.
Cholesterol.
Data:
Annals of
of Internal
Internal Medicine
Medicine 104(6):858-868, June, 1986.
Annals
1986.
Discovery, confirmation,
confirmation, and incorporation
incorporation of
of causal
causal
Blum, R.L.: Discovery,
Blum,
relationships from
from a large
large time-oriented
time-oriented clinical
clinical database:
database:
relationships
RX Project.
Project. Computers and Biomedical
Biomedical Research
Research 15(2):164-187,
The RX
April, 1982.
April,
1982.
How Feasible
Feasible is Automated
Automated Discovery?
Discovery? IEEE
IEEE Expert
Expert
Walker, M.G.
M.O. How
Walker,
2(1):70-82, Spring, 1987.
1987.
Conference
Proceedings
Conference Proceedings
DeZegher-Oeets,
DeZegher-Geets, I.M.,
I-M., Freeman, A.O.,
A.G., Walker,
Walker, M.O.,
M.G., Blum, R.L.,
R-L., and
Wiederhold,
Wiederhold, O.C.M.
G.C.M. Computer-aided
Computer-aided Summarization
Summarization of
of a Time-oriented
Time-oriented
Medical Data
Data Base. In Proceedings
Medical
Proceedings of
of the Third
Third Annual
Annual Conference on
Institute for
for Medical Record Economics,
Computerization
Computerization of
of Medical
Medical Records.
Records. Institute
1987.
1987.
Automated Summarization
Downs, S.,
S., Walker,
Walker, M.O.,
M.G., and Blum, R.L. Automated
Summarization of
of
On-line
Medical Records.
Records. In Proceedings
On-line Medical
Proceedings of
of the Fifth
Fifth World
World Congress
Congress on
Medical
Medical Informatics
Informatics (Medinfo),
(Medinfo), pages
pages 800-804. Elsevier Science
Science Publishers,
Publishers,
1986.
1986.
from Clinical
Walker,
Towards Automated
Automated Discovery
Discovery from
Clinical
Walker, M.O.,
M.G., and Blum, R.L. Towards
Databases:
RADIX Project.
Project. In Proceedings
Proceedings of
of the Fifth
Fifth World
World Congress
Congress
Databases: the RADIX
on Medical Informatics
Informatics (Medinfo),
(Medinfo), pages
pages 32-36. Elsevier Science
Science Publishers,
Publishers,
1986.
1986.
E.
E. H. Shortliffe
Shortliffe
230
230
5P41-RR00785-14
5P41-RR00785-14
National AIM
AIM Project:
National
Scientific Subproject AbstractsAbstracts
Scientific
CADUCEUS
CADUCEUS
(INTER NIST - I)
(INTERNIST-I)
QMR
QMR
(Quick Medical
Medical Reference)
(Quick
Principal Investigators:
Principal
CADUCEUS PROJECT:
CADUCEUS
Harry E. Pople,
Pop Ie, Ph.D. (POPLE@SUMEX-AIM)
(POPLE@SUMEX-AIM)
Harry
M.D. (MYERS@SUMEX-AIM)
(MYERS@SUMEX-AIM)
Jack D. Myers, M.D.
QMR PROJECT:
QMR
Randolph A. Miller,
Miller, M.D.(RMILLER@SUMEX-AIM)
M.D.(RMILLER@SUMEX-AIM)
Randolph
(MASARIE@SUMEX-AIM)
Fred E. Masarie, Jr. M.D. (MASARIE@SUMEX-AIM)
(MYERS@SUMEX-AIM)
Jack D. Myers, M.D. (MYERS@SUMEX-AIM)
University of
of Pittsburgh
Pittsburgh
University
Pittsburgh, Pennsylvania 15261
Pittsburgh,
15261
Dr.
Dr.
Dr.
Dr.
Dr.
Dr.
Dr.
Pople: (412) 624-3490
Myers: (412) 648-9933
Miller: (412) 648-3190
Miller:
Masarie: (412) 648-3190
The major
Projects is to
major goal of
of both the CADUCEUS
CADUCEUS and INTERNIST-IIQMR
INTERNIST-I/QMR
produce a reliable
reliable and adequately
adequately complete diagnostic consultative
consultative program in the field
field
of
primarily to aid skilled
of internal
internal medicine. Although
Although this program is intended primarily
skilled
internists
internists in complicated
complicated medical problems, the program may have
have spin-offs
spin-offs as
as a
physicians' assistants,
rural health clinics,
diagnostic and triage aid to physicians’
assistants, rural
clinics, military
military medicine
and space
have attempted to
space travel. In the design
design of
of INTERNIST-I
INTERNIST-I and QMR, we have
model the creative, problem-formulation
problem-formulation aspect
process. The
aspect of
of the clinical
clinical reasoning
reasoning process.
program employs a novel heuristic
heuristic procedure that composes
composes differential
differential diagnoses,
diagnoses,
dynamically,
basis of
dynamically, on the basis
of clinical
clinical evidence.
evidence. During
During the course
course of
of a INTERNIST-1
INTERNIST-l
consultation,
consultation, it
it is not uncommon for
for a number of
of such
such conjectured problem foci
foci to be
be
proposed
proposed and investigated.
investigated, with
with occasional
occasional major shifts
shifts taking
taking place
place in the program's
program’s
conceptualization
conceptualization of
of the task
task at hand.
hand. QMR
QMR is broader in scope
scope than TNTERNIST-I
INTERNIST-I or
CADUCEUS,
CADUCEUS, in that it
it provides quick and efficient
efficient access
access to [he
the INTERNIST-TlQMR
lNTERNIST-l/QhlR
knowledge
knowledge base
base to provide low and intermediate
intermediate level informational
informational support for
for
physicIans'
physicians’ decision-making,
decision-making, in addition
addition to providing
providing consultative
consultative advice.
advice.
SOFTW
ARE A
V AILABLE ON SUMEX
SOFTWARE
AVAILABLE
SUMEX
Versions of
of INTERNIST-I
INTERNIST-I are
are available for
for experimental use,
use, but the project continues
to be
be oriented
oriented primarily
primarily towards
towards research
research and development; hence.
hence, a stable
stable production
production
version of
of the
the system
system is not yet available for
for general
general use.
use. QMR has
has been
been shared
shared on a
restricted basis
basis with
with a limited
limited number of
of academic
academic colleagues,
colleagues, who have
have agreed
agreed to give
[he
the QMR development team
team feedback
feedback on the
the program's
program’s strengths
strengths and
and weaknesses.
weaknesses.
231
231
E.
E. H. Shortliffe
Shortiiffe
5P41-RR00785-14
5P41-RR00785-14
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
National AIM
AIM Project:
National
CLIPR --- HIERARCHICAL
HIERARCHICAL MODELS
MODELS
CLIPR
OF HUMAN
HUMAN COGNITION
COGNITION
OF
Principal Investigators:
Investigators:
Principal
Walter Kintsch,
Kintsch, Ph.D. (KINTSCH@SUMEX-AIM)
(KINTSCH@SUMEX-AIM)
Waiter
(POLSON@SUMEX-AIM)
Peter G. Polson, Ph.D. (POLSON@SUMEX-AIM)
Computer Laboratory
Laboratory for
for Instruction
Instruction
Computer
in Psychological Research
Research (CLIPR)
(CLIPR)
in
Campus Box 345
Department of
of Psychology
Department
University of
of Colorado
Colorado
University
Boulder, Colorado
Colorado 80309
80309
Boulder,
(303) 492-6991
Contact: Dr.
Dr. Peter G. Polson (Polson@SUMEX-AIM)
(Polson@SUMEX-AIM)
Contact:
CLIPR Project
Project is concerned with
with the modelin,
modeling0 of
of complex psychological processes.
processes.
The CLIPR
It
of two research
research groups. The
The prose
prose comprehension group has
has completed
It is comprised of
that carries out the text analysis described
described by van Dijk
Dijk & Kintsch
Kintsch (1983)
(1983),
project that
a project
yielding predictions
predictions of
of the recall and readability
readability of
of that text by human subjects.
subjects. The
yielding
human-computer interaction
interaction group is developing
developing a quantitative
quantitative theory of
of that predicts
human-computer
performance for
for a wide range
range of
of computer-tasks, e.g.
e.g. text editing,
editing,
learning, transfer,
transfer, and performance
learning,
Kieras & Polson (1985).
SOFTWARE AVAILABLE
AVAILABLE ON
ON SUMEX
SUMEX
SOFTWARE
of programs has
has been
been developed to perform
perform the microstructure
microstructure text
AA set of
described
described in
in van Dijk
Dijk & Kintsch
Kintsch (1983) and Kintsch
Kintsch & Greeno
Green0 (1985). The
propositionalized text as
produces indices that can
accepts
accepts a propositionalized
as input,
input, and produces
can be
estimate the text's
text’s recall and readability.
readability.
analysis
program
used to
used
REFERENCES
REFERENCES
1.
solving word arithmetic
arithmetic problems: A
A
1. Fletcher, R.C.:
R.C.: Understanding
Understanding and solving
computer
computer simulation.
simulation. Technical
Technical Report No. 135,
135, Institute
Institute of
of Cognitive
Cognitive
Science,
Science, Colorado,
Colorado, 1984.
1984.
2.
formal analysis
2. Kieras, D.E. and Polson, P.G.:
P.G.: The formal
analysis of
of user complexity.
complexity. Int.
J.
J. Man-Machine
Man-Machine Studies,
Studies, 22,
22, 365-394, 1985.
1985.
3.
3. Kintsch,
Kintsch, W. and van Dijk,
Dijk, T.A.: Toward
Toward a model
model of
of text
text comprehension
comprehension and
production. Psychological Rev.
Rev. 85:363-394,
85:363-394, 1978.
1978.
production.
4.
4. Kintsch,
Kintsch, W. and Greeno, J.G.:
J.G.: Understanding
Understanding and solving
solving word arithmetic
arithmetic
problems. Psychological Review, 1985,
probfems.
1985, 92,
92, 109-129.
5.
formal description
5. Polson, P.G. and Kieras, D.E.: A formal
description of
of users'
users’ knowledge
knowledge of
of
how to operate a device
Behavior Research
device and user complexity.
complexity.
Research Methods,
Instrumentation,
Instrumentation, & Computers, 1984,
1984, 16,
16, 249-255.
6.
6. Polson,
Polson, P.G.
P.G. and Kieras, D.E.: A quantitative
quantitative model of
of the learning
learning and
performance
performance of
of text
text editing
editing knowledge.
knowledge. In Borman, L. and
and Curtis, B.
B. (Eds.)
(Eds.)
Proceedings
Proceedings of
of the CHI
CHI 1985
1985 Conference on Human Factors in Computing.
New York:
York: Association for
for Computing
Computin g Machinery. pp.
pp. 207-212, 1985.
i985.
E.
E. H. Shortliffe
Shortliffe
232
232
Scien tific Subproject
Subproject Abstracts
Abstracts
Scientific
5P41-RR00785-14
5P41-RR00785-14
Jeffries, R.: Instruction
Instruction in
in general
general problem
problem solving
solving skills:
skills:
7. Polson, P.G. and Jeffries,
An analysis
analysis of
of four
four approaches.
approaches. In
In (Eds.) Siegel,
Siegel, J., Chipman,
Chipman, S.,
S., and Glaser,
An
Thinking and learning
learning skills:
skills: Relating
Relating instructions
instructions to basic research:
research:
R. Thinking
YoU. Hillsdale,
Hillsdale, N.J.: Lawrence Erlbaum
Erlbaum Associates,
Associates, pp. 414-455.
Vol.1.
Muncher, E., and Engelbeck, G.: Test of
of a common elements
elements
8. Polson, P.G., Muncher,
theory of
of transfer.
transfer. In
In Mantei,
Mantei, M.
M. and Orbeton,
Orbeton, P. (Eds.) Proceedings
Proceedings of
of the
theory
CHI 1986 Conference on Human
Human Factors in
in Computing.
Computing. New York:
York:
CHI
for Computing
Computing Machinery.
Machinery. pp. 78-83, 1986.
1986.
Association for
Dijk, T.A.
T.A. and Kintsch,
Kintsch, W.: STRATEGIES
STRATEGIES
Dijk,
COMPREHENSION. Academic Press,
Press, New York,
York, 1983.
1983.
COMPREHENSION.
9. van
OF
OF
DISCOURSE
DISCOURSE
S.: A
A theory and simulation
simulation of
of macrostructure. Technical
Technical Report
Report No.
10. Young, S.:
Institute of
of Cognitive
Cognitive Science,
Science, Colorado,
Colorado, 1984.
1984.
134, Institute
11. Walker,
Walker, H.W. & Kintsch,
Kintsch, W.: Automatic
Automatic and strategic aspects
aspects of
of knowledge
retrieval. Cognitive
Cognitive Science,
Science, 1985,
retrieval.
1985, 9, 261-283.
233
233
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Scientific Subproject Abstracts
Abstracts
Scientific
National AIM
AIM Project:
National
MENTOR --- MEDICAL
MEDICAL EVALUATION
EVALUATION OF
OF
MENTOR
THERAPEUTIC ORDERS
ORDERS
THERAPEUTIC
Principal Investigators:
Principal
Speed ie, Ph.D. (SPEEDIE@SUMEX-AIM)
(SPEEDIE@SUMEX-AIM)
Stuart Speedie,
of Pharmacy
School of
of Maryland
Maryland
University of
University
20 N.
N. Pine
Pine Street
Baltimore,
Baltimore, Maryland
Maryland 21201
21201
(301) 528-7650
Terrence F. Blaschke,
Blaschke, M.D.
M.D. (BLASCHKE@SUMEX-AIM)
(BLASCHKE@SUMEX-AIM)
Department of
of Medicine
Medicine
Department
of Clinical
Clinical Pharmacology
Division of
Division
Stanford University
University Medical
Medical Center
Stanford
Stanford.
Stanford, California
California 94305
Contact: either PI
The goal of
of the MENTOR
MENTOR project
project is to implement
implement and begin evaluation
evaluation of
of a computercomputerbased methodology for
for reducing therapeutic
therapeutic misadventures. The project
project uses
uses an on-line
on-line
based
system to continuously
continuously monitor
monitor the drug therapy of
of individual
individual patients and
expert system
of potential
potential and/or
and/or actual unintended effects of
of therapy.
generate specific
specific warnings of
generate
The appropriate
appropriate patient
patient information
information is automatically
automatically acquired through interfaces to a
information system.
system. This
This data is monitored
monitored by a system
system that
that is capable of
of
hospital information
employing complex chains of
of reasoning to evaluate
evaluate therapeutic decisions and arrive
arrive at
employing
valid
valid conclusions in the context
context of
of all information
information available on the patient. The results
reached
reached by the system
system are fed back to the responsible physicians to assist
assist future
future
decision making.
Specific objectives of
of this project
project include:
1.
1. Implement
Implement a prototype
prototype computer-based expert system
system to continuously
continuously
monitor
monitor in-patient
in-patient drug therapy that
that uses
uses a modular
modular medical knowledge
knowledge base
base
and a separate
separate inference
inference engine to apply the knowledge
knowledge to specific
specific situations.
situations.
2.
2. Select
Select a small number of
of important
important and frequently
frequently occurring
occurring drug therapy
problems that can
can lead
lead to therapeutic misadventures
misadventures and construct
construct a
comprehensive
comprehensive knowledge
knowledge base
base necessary
necessary to detect these
these situations.
3.
implementation of
of an
an evaluation of
of the prototype
prototype
3. Design
Design and
and begin implementation
MENTOR
MENTOR system
system with
with respect
respect to its impact on the on the physicians'
physicians’
therapeutic decision making as
as well as
as its effects on the patient
patient in terms of
of
specific mortality
mortality and morbidity
morbidity measures.
measures.
The work in this project builds
builds on the extensive
extensive previous work in drug monitoring
monitoring
done by these
these investigators in the Division
Division of
of Clinical
Clinical Pharmacology at Stanford
Stanford and
the University
University of
of Maryland
Maryland School of
of Pharmacy.
Pharmacy.
E.
E. H. Shortliffe
Shortliffe
234
234
Sci
en tific Subproject Abstracts
Scientific
5P41-RR00785-14
National
AIM Project:
National AIM
SOLVER -PROBLEM SOLVING
SOLVING EXPERTISE
EXPERTISE
-- PROBLEM
Principal
Principal Investigators:
Paul
Paul E. Johnson,
Johnson, Ph.D.
School
School of
of Management and
Center for
for Research
Research in Human Learning
Learning
205 Elliott
Elliott Hall
Hall
205
University of
of Minnesota
University
Minneapolis,
Minneapolis, Minnesota
Minnesota 55455
55455
(612) 376-2530 (PJOHNSON@SUMEX-AIM)
(PJOHNSON@SUMEX-AIM)
James
James R. Slagle,
Slagle, Ph.D.
Department of
of Computer Science
Science
Department
136 Lind
Lind Hall
Hall
136
University
University of
of Minnesota
Minnesota
Minneapolis,
Minneapolis, Minnesota 55455
55455
(612) 373-0132
373-0132 (SLAGLE@SUMEX-AIM)
(SLAGLEGSUMEX-AIM)
William
William B.
6. Thompson, Ph.D.
Department
Department of
of Computer
Computer Science
Science
136
136 Lind
Lind Hall
Hall
University
University of
of Minnesota
Minnesota
Minneapolis,
Minneapolis, Minnesota
Minnesota 55455
55455
(612) 373-0132 (THOMPSON@SUMEX-AIM)
(THOMPSON@SUMEX-AIM)
The Minnesota SOLVER
SOLVER project focuses
focuses upon the development of
of strategies
strategies for
for
discovering and representing the knowledge
problem solvers.
knowledge and skill
skill of
of expert problem
solvers.
Although
Although in the last fifteen
fifteen years
years considerable progress
progress has
has been
been made
made in synthesizing
the expertise
expertise required for
for solving complex problems, most expert systems
systems embody only
only a
limited
limited amount of
of expertise.
expertise. What is still
still lacking is a theoretical framework
framework capable
capable of
of
reducing dependence
dependence upon the expert's
expert’s intuition
intuition or on the near exhaustive
exhaustive testing of
of
extensive use
use of
of verbal
possible
possible organizations. Our methodology consists
consists of:
of: (1) extensive
thinking
thinking aloud protocols as
as a source
source of
of information
information from
from which to make inferences
about underlying
underlying knowledge
knowledge structures
structures and processes;
processes; (2) development of
of computer
models
protocol studies;
models as
as a means
means of
of testing the adequacy
adequacy of
of inferences derived from
from protocol
studies;
(3) testing and refinement
refinement of
of the cognitive
cognitive models
models based
based upon the study of
of human and
Currently, we are investigating
investigating problemmodel performance in experimental settings.
problemsettings. Currently,
solving expertise
expertise in domains of
of medicine, computer hardware
hardware diagnosis,
diagnosis, offline
offline quality
quality
control,
control, financial
financial auditing, management,
management, and law.
law.
SOFTW
ARE A
V AILABLE ON
SOFTWARE
AVAILABLE
ON SUMEX
SUMEX
A redesigned
redesigned version of
of the Diagnoser simulation
simulation model, named
named Gaien, has
has been
been
implemented
implemented on SUMEX.
SUMEX. Galen is an
an expert system
system which uses
uses recognition-based
recognition-based
pediatric cardiology.
reasoning in pediatric
REFERENCES
REFERENCES
1.
Path Errors
Errors in
Garden Path
1. Johnson,
Johnson, P.E.,
P.E., Moen, 1.B.,
J.B., and Thompson, W.B.:
W.B.: Garden
Medical Diagnosis.
Diagnosis. . In Bolc,
Bolt, L. and Coombs,
Coombs, MJ.
M.J. (Eds.),
(Eds.), COMPUTER
COMPUTER
Medical
EXPERT
EXPERT SYSTEMS,
SYSTEMS, Springer-Verlag
Springer-Verlag (in
(in press).
press).
2.
Expertise in trial
2. Johnson,
Johnson, P.E..
P.E., Johnson,
Johnson, M.G., and Little,
Little, R.K.: Expertise
triaf advocacy:
advocacy:
Some
for inquiry
Some considerations
considerations for
inquiry into
into its
its nature
nature and development, Campbell
Law Review,
Review, (in
(in press).
press).
235
Shortliffe
E. H. Shortliffe
5P41-RR00785-14
5P41-RR00785-14
Scientific
Scientific Subproject Abstracts
3.
Expert Mind:
Mind: A New Challenge
for the Information
Information
3. Johnson,
Johnson, P.E.,
P.E., The Expert
Challenge for
scientist,
in BEYOND
INFORMATION
SYSTEM
BEYOND PRODUCTIVITY:
PRODUCTIVITY:
INFORMATION
SYSTEM
scientist,
DEVELOPMENT
EFFECTIVENESS,
DEVELOPMENT FOR ORGANIZATIONAL
ORGANIZATIONAL
EFFECTIVENESS, Th. M. A.
Bemelmans
Bemelmans (editor),
(editor), Elsevier Science
Science Publishers B.V. (North-Holland),
(North-Holland), 1984.
1984.
4. Johnson,
Johnson, P.E.:
P.E.: What
What kind
kind of
of expert
expert should
should a system be?
be? Journal of
of Medicine
Medicine
and Philosophy, 8:77-97,
8:77-97, 1983.
1983.
5.
5. Johnson,
Johnson, P.E.,
P.E., Duran, A., Hassebrock,
Hassebrock, F., Moller,
Moller, 1.,
J., Prietula,
Prietuia. M., Feltovich,
Feltovich,
P. and Swanson,
Expertise and error
error in diagnostic
diagnostic reasoning.
reasoning. Cognitive
Cognitive
Swanson, D.: Expertise
Science
Science 5:235-283,
5:235-283, 1981.
1981.
6.
6. Simpson, D.E., Rich, E., Dalgaard,
Dalgaard, K., Gjerdingen, K., Crowson, T., O'Brien,
O’Brien,
D., Johnson,
process in primary
primary care:
Johnson, P.E.:
P.E.: The diagnostic
diagnostic process
care: A comparison
comparison of
of
SOCIAL SCIENCE
SCIENCE AND
AND
general internists
family physicians.
physicians.
SOCIAL
general
internists and family
MEDICINE
MEDICINE (in
(in press).
press).
7.
Knowledge-based systems in laboratory
7. Spackman,
Spackman, K.A.
K.A. and Connelly, D.P.:
D.P.: Knowledge-based
laboratory
of
medicine
pathology: A review
field. Archives of
medicine and pathology:
review and survey
survey of
of the field.
Laboratory
Laboratory Medicine
Medicine and Pathology 111:116-119,
111:116-119, 1987.
1987.
8.
Recognition-based diagnostic
8. Thompson, W.B., Johnson,
Johnson, P.E.
P.E. and Moen, 1.B.:
J.B.: Recognition-based
diagnostic
reasoning. Proc.
Proc. Eighth
Eighth DCAI,
IJCAI, Karlsruhe, West Germany, August, 1983.
1983.
Shortliffe
E. H. Shon/tffe
236
236
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
5P41-RR00785-14
5P41-RR00785-14
National AIM
AIM Project:
National
ATTENDING Project:
ATTENDING
Critiquing Approach
Approach to
to
AA Critiquing
Expert Computer
Computer Advice
Advice
Expert
Principal Investigator:
Investigator:
Principal
Perry L.
L. Miller,
Miller, M.D.,
M.D., Ph.D.
Perry
Department of
of Anesthesiology
Department
Yale University
University School of
of Medicine
Medicine
Yale
CT 06510
New Haven, CT
(203) 785-2802
(203)
Our project
project is exploring
exploring the “critiquing”
"critiquing" approach to bringing
bringing computer-based advice to
Our
practicing physician.
the practicing
Critiquing is a different
different approach to the design
design of
of artificial
artificial intelligence
intelligence based
based expert
Critiquing
systems. Most
Most medical expert systems
systems attempt
attempt to simulate a physician’
physician'ss decision-making
decision-making
systems.
process. As a result, they have
have the clinical
clinical effect
effect of
of trying
trying to tell a physician what to
process.
In contrast, a critiquing
critiquing system
system first
first asks
asks the physician
do: how to practice medicine. In
patient'ss care,
care, and then critiques
critiques that
that plan. In the
how he contemplates approaching his patient’
of the proposed
proposed approach, and of
of any
critique, the system
system discusses
discusses any risks or benefits of
critique,
other approaches
approaches which might
might be preferred.
critiquing
ItIt is anticipated that the critiquing
other
particularly well suited for
for domains, like
like medicine, where decisions
approach may be particularly
involve a great deal of
of subjective
subjective judgment.
judgment.
involve
To
prototype critiquing
have been
been developed
developed in
in different
different medical
To date,
date, several
several prototype
critiquing systems
systems have
domains:
1. ATTENDING,
the first
ATTENDING,
first system
system to implement
implement the critiquing
critiquing approach,
approach,
critiques
anesthetic
management.
critiques
management.
2.
critiques the pharmacologic management
2. HT-ATTENDING
HT-ATTENDING
management of
of essential
essential
hypertension.
3.
3. VQ-ATTENDING
VQ-ATTENDING critiques aspects
aspects of
of ventilator
ventilator management.
management.
4.
ATTENDING critiques the laboratory
4. PHEOPHEO-ATTENDING
laboratory and
and radiologic
radiologic workup of
of a
patient for
for a suspected
suspected pheichromocytoma.
5.
has
5. Tn
In addition,
addition, a domain-independent
domain-independent system,
system, ESSENTIAL-ATTENDING,
ESSENTIAL-ATTENDING,
has
been
been developed
developed to facilitate
facilitate the
the implementation
implementation of
of critiquing
critiquing systems
systems in
other domains.
PUBLICA
TIONS
PUBLICATIONS
1.
Practice-Based Medical Consultation
Consultation
1. Miller,
Miller, P.L.:
P.L.: Expert Critiquing
Critiquing Systems:
Systems: Practice-Based
by Computer. New York:
York: Springer-Verlag,
Springer-Verlag, 1986.
1986.
2.
2. Miller,
Miller, P.L. (Ed.): Selected
Selected Topics in Medical Artificial
Artificial Intelligence.
York: Springer-Verlag
Springer-Verlag (in
(in press).
press).
New
3.
3. Miller,
Miller, P.L.,
P.L., Shaw,
Shaw, c.,
C., Rose,
Rose, J.R.,
J.R., Swett,
Swett, H.A.: Critiquing
Critiquing the
the process
process of
of
radiologic
Computer Methods
Methods and
and Programs
Programs in
radiologic differential
differential diagnosis.
diagnosis. Computer
Biomedicine
Biomedicine 22:12-25,
22:12-25, 1986.
1986.
4.
4. Miller.
Miller, P.L.:
P.L.: The
The evaluation
evaluation of
of artificial
artificial intelligence
intelligence systems
systems in
in medicine.
medicine
Computer
Computer Methods
Methods and
and Programs
Programs in
in Biomedicine
Biomedicine 22:5-11,
22:5-11, 1986.
1986.
237
237
E.
E. H.
H. Shortliffe
Shortliffe
SP41-RR0078S-14
5P41-RR00785-14
Scientific
Scientific Subproject Abstracts
S.
5. Rennels,
Rennels, G.D., Shortliffe,
Shortliffe, E.H., Miller;
Miller, P.L.: Choice of
of explanation in
medical management:
A multi-attribute
multi-attribute model of
of artificial
artificial intelligence
intelligence
management: A
approaches.
approaches. Medical
Medical Decision
Decision Making
Making 7:22-31, 1987.
1987.
6.
N.H., Miller,
acquisition and verification
verification tools for
for
6. Mars, N.J.I.,
Miller, P.L.: Knowledge acquisition
medical expert systems.
systems. Medical Decision Making
Making 7:6-11, 1987.
1987.
7. Miller,
Miller, PL,
P.L., Blumenfrucht,
Blumenfrucht, S.1.,
S.J., Rose,
Rose, 1.R.,
J.R., Rothschild,
Rothschild, M., Swett,
Swett, H.A.,
N.H.: HYDRA:
Weltin,
HYDRA: A
A knowledge
knowledge acquisition
acquisition tool for
for expert
Weltin, G., Mars, N.J.I.:
systems
systems which
which critique
critique medical workup. Medical Decision Making
Making 7:12-21,
1987.
1987.
8.
8. Swett,
Swett, H.A., Miller,
Miller, P.L.: ICON:
ICON: A
A computer-based approach to differential
differential
diagnosis in radiology.
press).
radiology. Radiology
Radiology (in
(in press).
A
9.
9. Rennels,
Rennels, G.D., Shortliffe,
Shortliffe, E.H., Stockdale,
Stockdale, F.E., Miller,
Miller, P.L.:
P.L.:
A
computational
computational model of
of reasoning from
from the clinical
clinical literature.
literature. Computer
Computer
Methods and Programs in Biomedicine
press).
Biomedicine (in
(in press).
10.
10. Rennels,
Rennels, G.D., Shortliffe,
Shortliffe, E.H., Stockdale,
Stockdale, F.E., Miller,
Miller, P.L.: A
A structured
representation of
of the clinical
clinical literature
literature and its use
use in a medical management
management
advice system.
press).
system. Bulletin
Bulletin du Cancer (in
(in press).
11.
11. Miller,
Miller, P.L.: Exploring
Exploring the critiquing
critiquing approach:
approach: Sophisticated practice-based
Proceedings
feedback by computer.
Proceedings of
of the Fifth
Fifth World
World Conference on
Medical Informatics:
Informatics: MEDINFO-86,
MEDINFO-86, Washington, D.C., October 1986,
1986, pp 2-6.
N.H., Miller,
12.
12. Mars, N.J.I.,
Miller, P.L.: Tools for
for knowledge
knowledge acquisition
acquisition and verification
verification
in medicine.
Proceedings
Computer
Proceedings of
of the Tenth
Tenth Symposium on Computer
Applications
]4npplications in Medical Care, Washington, D.C., October 1986,
1986, pp. 36-42.
l3.
13. Miller,
Miller, P.L., Blumenfrucht,
Blumenfrucht, S.1.,
S.J., Rose,
Rose, 1.R.,
J.R., Rothschild,
Rothschild, M., Weltin,
Weltin, G., Swett,
Swett,
N.H.: Expert
H.A.,
Expert system
system knowledge acquisition
acquisition for
for domains of
of
H-A., Mars, N.J.I.:
medical workup: An
An augmented
augmented transition
transition network
network model. Proceedings
Proceedings of
of
the Tenth
Tenth Symposium on Computer
Computer Applications
Applications in Medical Care,
Care,
Washington, D.C., October 1986,
1986, pp. 30-35.
14.
14. Rennels,
Rennels, G.D., Shortliffe,
Shortliffe, E.H.,
E-H., Stockdale,
Stockdale, F.E., Miller,
Miller, P.L.:
P.L.: Reasoning
Reasoning from
from
the clinical
clinical literature:
literature: The Roundsman system.
system. Proceedings
Proceedings of
of the Fifth
Fifth
World
World Conference on Medical Informatics:
Informatics: MEDINFO-86,
MEDINFO-86, Washington, D.C.,
D.C.,
October 1986,
1986, pp. 771-775.
15.
15. Rennels,
Rennels, G.D., Shortliffe,
Shortliffe, E.H., Stockdale,
Stockdale, F.E., Miller,
Miller, P.L.:
P.L.: Updating
Updating an
an
expert knowledge base
base as
as medical knowledge
knowledge evolves:
evolves: Examples from
from
oncology management.
management. Proceedings
Proceedings of
of the American
American Association of
of Medical
Systems
Systems and Informatics
Informatics Congress
Congress 87,
87, San
San Francisco, May 1987,
1987, pp. 228-23l.
228-231.
16.
16. Fisher, P.R.,
P.R., Miller,
Miller, P.L., Swett,
Swett, H.A.: A
A script-based representation of
of
medical knowledge involving
perspectives.
Proceedings
involving multiple
multiple perspectives.
Proceedings of
of the
American
American Association
Association of
of Medical Systems
Systems and Informatics
Informatics Congress-87,
Congress-87, San
San
Francisco, May 1987,
1987, pp. 233-237.
17.
17. Miller,
Miller, P.L.: Expert
Expert consultation
consultation systems
systems in medicine: A
A complex and
fascinating
Proceedings
fascinating domain.
domain.
Proceedings of
of the Annual
Annual Meeting of
of the IEEE
IEEE
(Electro-87),
paper).
(Electra-87), New York,
York, April
April 1987,
1987, pp. 112:1-4
l/2:1-4 (invited
(invited paper).
E. H. Shortliffe
Shortliffe
238
238
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
5P41-RR00785-14
5P41-RR00785-14
Stanford Project:
Project:
Stanford
REFEREE Project
Project
REFEREE
Principal Investigators:
G. Buchanan, Principal
Principal Investigator
Investigator
Bruce G.
Principal
Computer Science Department
Department
Computer
Stanford University
University
Stanford
Stanford, California
California 94305
Stanford,
Byron W.
W. Brown,
Brown, Co-Principal
Co-Principal Investigator
Investigator
Byron
Department of
of Medicine
Medicine
Department
Stanford University
University Medical
Medical Center
Stanford
California 94305
Stanford, California
Stanford,
Daniel E. Feldman, Associate Investigator
Investigator
Daniel
Department of
of Medicine
Medicine
Department
Stanford University
University Medical
Medical Center
Stanford
Stanford, California
California 94305
Stanford,
of this project
project are related both to medical science
science and Artificial
Artificial Intelligence:
The goals of
informed but
but non-expert
non-expert reader
reader of
of the medical
(a) use
use AI
AI methods to allow
allow the informed
literature to evaluate
evaluate a randomized
randomized clinical
clinical trial,
trial, and (b) use
use the interpretation
interpretation of
of the
literature
for studies
studies of
of knowledge acquisition
acquisition and fusion
fusion of
of
literature as
as a test problem
problem for
medical literature
information from
from disparate sources.
sources. REFEREE
REFEREE and REVIEWER,
REVIEWER, a planned extension,
information
will
will be used
used to evaluate the medical literature
literature of
of clinical
clinical trials
trials to determine the quality
quality
of
proposed, and
of a clinical
clinical trial,
trial, make judgements on the efficacy
efficacy of
of the treatment proposed,
synthesize
research is an initial
initial step
step toward a more
synthesize rules of
of clinical
clinical practice. The reseq.fch
general
building computer
computer systems
systems to help the clinician
clinician and medical scientist
scientist
general goal - building
read
read the medical literature
literature more critically
critically and more rapidly.
rapidly.
239
239
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
SP41-RR0078S-14
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
National AIM
AIM Project:
National
Com puter- Aided Diagnosis of
of
Computer-Aided
Lymph Node
Node Pathology (PATHFINDER)
(PATHFINDER)
Lymph
Principal Investigator:
Investigator:
Principal
Bharat Nathwani,
Nathwani, M.D.
M.D.
Bharat
Department of
of Pathology
Department
HMR 204
HMR
Zonal Avenue
2025 Zonal
University
University of
of Southern California
California
of Medicine
Medicine
School of
California 90033
90033
Los Angeles, California
(NATHWANI@SUMEX-AIM)
(213) 226-7064 (NATHWANIGSUMEX-AIM)
Fagan, M.D., Ph.D.
Lawrence M. Fagan,
Medical Computer
Computer Science
Science Group
Group
Medical
Department of
of Medicine
Medicine
Department
Office Building
Building
Medical School Office
Stanford, California
California 94305
94305
Stanford,
(FAGAN@SUMEX-AIM)
(415) 723-6979 (FAGANGSUMEX-AIM)
PATHFINDER Project
Project is centered on the construction
construction of
of an expert system
system for
for
The PATHFINDER
with the diagnosis of
of tissue
tissue pathology. PATHFINDER
PATHFINDER research
research is
assisting pathologists with
The project is based
domain of
of lymph
lymph node pathology.
based at the
focused on the domain
University of
of Southern California
California in collaboration
collaboration with
with the Stanford
Stanford University
University Medical
University
Computer Science
Science Group.
Group. Ongoing
Ongoing AIM
AIM research
research has
has been
been addressing
addressing fundamental
fundamental
Computer
of knowledge representation,
representation, reasoning strategies,
strategies, user
user modeling, explanation,
explanation,
problems of
and user
user acceptance.
acceptance. A
A pragmatic
pragmatic goal of
of the project is to provide
provide a valuable diagnostic
and educational tool for
for pathologists with
with different
different levels of
of training
training and experience
experience by
It is hoped that
integrating
It
integrating diverse knowledge about lymph
lymph node pathology.
PATHFINDER
PATHFINDER basic
basic research
research on representation and inference
inference in combination
combination with
with the
pragmatic goals
goals of
of constructing
constructing a clinically-relevant
clinically-relevant diagnostic aid will
will lead
lead to useful
advances
advances in medical computing.
computing.
A
A pilot
pilot version of
of the program provides diagnostic advice on eighty common benign
and malignant
malignant diseases
diseases of
of the lymph
lymph nodes
nodes based
based on 150
150 histologic
histologic features.
features. Our
research
research plans are to develop a full-scale
full-scale version of
of the computer
computer program by
substantially
substantially increasing the quantity
quantity and quality
quality of
of knowledge
knowledge and to develop techniques
techniques
for
for knowledge
knowledge representation and manipulation
manipulation appropriate
appropriate to this application
application area.
area. The
design
design of
of the program has
has been
been strongly
strongly influenced
influenced by the INTERNIST/CADUCEUS
lNTERNIST/CADUCEUS
program developed
developed on the SUMEX
SUMEX resource.
resource.
SOFTWARE
SOFTWARE AVAILABLE
AVAILABLE ON
ON SUMEX
SUMEX
PATHFINDER-program is available for
PATHFINDER
for
PATHFINDER-- A
A version of
of the PATHFINDER
experimentation
experimentation on the DEC 2060
2060 computer. This version is a pilot
pilot
pletely tested.
version of
of the program, and therefore has
has not been
been com
completely
tested.
E.
E. H.
H. Shortliffe
ShoTtliffe
240
240
Scientific
Scientific Subproject Abstracts
5P41-RR00785-14
5P41-RR00785-14
AIM Pilot
Pilot Project:
AIM
RXDX
RXDX Project
Project
Principal Investigators:
Principal
Robert Lindsay, Ph.D. (313) 764-4227
Robert
Michael Feinberg, M.D., Ph.D. (215) 842-4208
Michael
University of
of Michigan
Michigan
University
Ann Arbor,
Arbor, Michigan
Michigan
Ann
prototype expert system
system that
that could act as
as a consultant
consultant in
in the
We are developing a prototype
of depression.
depression. Health
Health professionals will
will interact
interact with
with the
diagnosis and management of
as they might
might with
with a human consultant,
consultant, describing
describing the patient,
patient, receiving
receiving advice,
program as
consultant about the rationale
rationale for
for each
each recommendation.
recommendation. The program
and asking the consultant
uses a knowledge base
base constructed by encoding the clinical
clinical expertise of
of a skilled
skilled
uses
psychiatrist in
in a set
set of
of rules and other
other knowledge structures. It
will use
use this knowledge
psychiatrist
It will
base to decide on the most likely
likely diagnosis (endogenous
(endogenous or
or nonendogenous
nonendogenous depression),
base
assess the need
need for
for hospitalization,
hospitalization, and recommend specific
specific somatic treatments when
assess
indicated (e.g.,
(e.g., tricyclic
tricyclic antidepressants).
antidepressants). The treatment
treatment recommendation will
will
this is indicated
into account the patient’
patient'ss diagnosis,
diagnosis, age,
age, concurrent
concurrent illnesses,
illnesses, and concurrent
concurrent
take into
treatments (drug interactions).
interactions).
potential benefits to psychiatry include: making
making relatively
relatively skilled
skilled psychiatric
psychiatric
The potential
consultation widely
widely available in underserved
underserved areas,
areas, including
including some public
public mental health
consultation
facilities where patients are seen
seen by non-psychiatrists
non-psychiatrists and have
have relatively
relatively little
little direct
direct
facilities
patient-physician
trained
patient-physician contact; providing
providing non-psychiatrically
non-psychiatrically
trained physicians with
with
additional
additional information
information about psychiatric
psychiatric diagnosis and treatment; avoiding
avoiding errors of
of
oversight
patient care.
oversight caused
caused by inaccessible
inaccessible patient
patient data; and increased
increased productivity
productivity in patient
care.
Like
Like any good consultant, the program will
will be able to teach
teach the interested user,
user, and can
function
function as
as a teaching tool independent of
of direct
direct clinical
clinical application.
application.
PUBLICA
TIONS
PUBLICATIONS
1.
Expert systems in Psychiatry
Psychiatry
1. Feinberg, M. and Lindsay, R. K.:
K.: Expert
Psychopharmacology. Psychopharmacol.
Psychopharmacology.
Psychopharmacol. Bull., 22,
22, 1986,311-316.
1986, 311-316.
and
2.
2. Lindsay, R. K. Expert
Expert Systems
Systems in Psychiatric
Psychiatric Diagnosis: Rule-Based
Rule-Based Systems.
Systems.
Presented
Presented at MedInf086,
MedInfo86, Washington, D.C.
DC.
3.
3. Feinberg,
M. What
What
Washington, D. C.
Psychiatrists
Can't
Can’t Do.
241
Presented at
Presented
Medinfo86,
Medinfo86,
E. H.
H. Shortliffe
Shortliffe
E.
5P41-RR00785-14
5P41-RR00785-14
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
National AIM
AIM Project:
Project:
National
DECISION SUPPORT
SUPPORT FOR
FOR
DECISION
TIME-VARYING CLINICAL
CLINICAL PROBLEMS
PROBLEMS
TIME-VARYING
Principal Investigator:
Investigator:
Lawrence Widman,
Widman, M.D.,
M.D., Ph.D.
Ph.D.
Lawrence
Principal
Division of
of Cardiology
Cardiology
Division
Case Western Reserve University
University
Case
Adelbert Road
2065 Adelbert
Cleveland, OH
OH 44106
Cleveland,
(216)
844-3153
(216)
Time-varying systems,
systems, which
which include
include many
many areas
areas of
of medicine,
medicine, science,
science, economics, and
Time-varying
business, can be described mathematically
mathematically by
by differential
differential equations. They
They are distinct
distinct
business,
from the pattern-matching
pattern-matching and logic-based domains
domains dealt with
with so successfully by
from
existing expert
expert system
system methods, because
because they can include
include feedback relationships.
relationships. ItIt is
existing
generally felt
felt that
that they are best approached
approached by enhancement of
of existing
existing methods for
for
generally
deep model-based reasoning.
The goal of
of this
this project
project is to develop AI
AI methods for
for capturing
capturing and using knowledge
The
about time-varying
time-varying systems.
systems. The strategy is to address
address general problems in
in model-based
about
knowledge representation
representation and reasoning.
reasoning.
The intermediate
intermediate objective
objective is to develop
which are powerful
powerful enough to work
work in selected
selected realistic
realistic situations
situations yet are
methods which
transportable to other, unrelated knowledge domains.
general enough to be transportable
tactical approach is to work
work on well-defined
well-defined yet complex and interesting
interesting problems
The tactical
in
have, therefore, selected
system
in the medical domain. We have,
selected the human cardiovascular system
as
prototype of
as our
our prototype
of a time-varying
time-varying system,
system, and are developing methods for
for representing
and reasoning about its mechanical and electrical
normal and diseased
electrical activities
activities in the normal
diseased
states.
states.
REFERENCES
REFERENCES
1.
1. Widman,
Widman, L.E. Rea~oning
Reasoning about Diagnosis and Treatment
Treatment in a Causal
Causal
Medical
Simulation
Medical Model USlOg
using Semi-Quantitative
Semi-Quantitative
Simulation and Inference.
Workshop
Workshop on Artificial
Artificial Intelligence
Intelligence in Medicine,
Medicine, National
National Conference on
Artificial
Artificial Intelligence,
Intelligence, AAAI-87,
AAAI-87. Seattle.
Seattle.
2.
2. Widman,
Widman, L.E., Lee,
Lee, Y.-B., and
and Y.-H.
Y.-H. Pao.
Pao. Diagnosis of
of Causal
Causal Models by
Semi-Quantitative
Semi-Quantitative Reasoning
Reasoning (submitted
(submitted to SCAMC 1987).
1987).
J.
3. Widman,
Widman, L.E., Lee,
Lee, Y.-B., and
and Y.-H.
Y.-H. Pao.
Pao.
Models by Semi-Quantitative
Semi-Quantitative Reasoning.
Reasoning. In:
Medical Artificial
Artificial Intelligence, Springer-Verlag
Springer-Verlag
Diagnosis of
of Causal
Causal Medical
Miller,
Miller, P.L. (ed.)
(ed.) Topics in
(in
(in preparation).
preparation).
4.
4. Lee,
Lee, Y.-B.
Y.-B. and
and L.E. Widman.
Widman. Reasoning
Reasoning about Diagnosis and
and Treatment in
aa Causal
Causal Time-varying
Time-varying Domain
Domain using
using Semi-Quantitative
Semi-Quantitative Simulation
Simulation and
and
Inference. Workshop on Artificial
Artificial Intelligence
Intelligence and
and Simulation,
Simulation, National
National
Conference on Artificial
Artificial Intelligence, AAAI-86,
AAAI-86, Philadelphia.
5.
5. Widman,
Widman, L.E. Representation
Representation Method for
for Dynamic
Dynamic Causal
Causal Knowledge
Knowledge Using
Semi-Quantitative
Semi-Quantitative Simulation.
Simulation. Fifth
Fifth World
World Conference on Medical
Informatics.
Informatics. 1986:
1986: 180-184.
180-184.
E.
E. H.
H. Shortliffe
Shortliffe
242
242
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
5P41-RR00785-14
5P41-RR00785-14
National AIM
AIM Project:
Project:
National
KNOWLEDGE ENGINEERING
ENGINEERING FOR
FOR
KNOWLEDGE
RADIATION THERAPY
THERAPY
RADIATION
Principal Investigator:
Investigator:
Ira J.
1. Kale&
Kalet, Ph.D.
Ph.D.
Principal
Ira
of Medicine
Medicine
School of
School
University of
of Washington
Washington at
at Seattle
Seattle
University
Seattle, Washington
Washington 98195
98195
Seattle,
(206) 548-4107
548-4107
(206)
We are
are developing
developing an
an expert
expert system
system for
for planning
planning of
of radiation
radiation therapy
therapy for
for head
head and
and
We
neck cancers.
cancers. The
The project
project will
will ultimately
ultimately combine
combine knowledge-based
knowledge-based planning
planning with
with
neck
numerical simulation
simulation of
of the
the radiation
radiation treatments.
treatments. The
The numerical
numerical simulation
simulation is needed
needed
numerical
order to
to determine
determine ifif the
the proposed
proposed treatment
treatment will
will conform
conform to
to the
the goals of
of the
the plan
plan
in order
in
(required turn
tumor
dose, limiting
limiting dose
dose to critical
critical organs).
organs).' The
The space
space of
of possible radiation
radiation
(required
or dose,
treatments is numerically
numerically very
very large, making
making traditional
traditional search
search techniques impractical.
impractical.
treatments
Yet, with
with modern radiation
radiation therapy equipment, the design of
of treatment
treatment plans might
might be
Yet,
significantly aided by automatically
automatically generating plans that
that meet the treatment
treatment constraints.
constraints.
significantly
The project
project will
will result in
in systematization of
of knowledge
knowledge about radiation
radiation treatment
treatment design,
and will
will also provide
provide an example of
of how to represent and solve design
design problems
problems with
with a
based system.
system.
knowledge based
This project
project has
has some relevance
relevance to computer science
science as
as well, in
in that our approach, ifif
This
successful, may contribute
contribute to a better understanding
understanding of
of design
design problem solving
solving with
with
successful,
knowledge-based
systems.
knowledge-based systems.
REFERENCES
REFERENCES
1.
Kalet and W. Paluszynski: A
A Production
Production Expert
Expert System
System for
for Radiation
Radiation Therapy
Therapy
I. Kalet
Planning.
Planning. Proceedings
Proceedings of
of the AAMSI
AAMSI Congress
Congress 1985,
1985, May 20-22, 1985,
1985, San
San Francisco,
California.
California. Edited by Allan
Allan H. Levy and Ben
Ben T. Williams.
Williams. American
American Association
Association for
for
Medical
Medical Systems
Systems and Informatics,
Informatics, Washington, D.C.,
D.C., 1985.
1985.
W. Paluszynski
Paluszynski and I. Kalet: Radiation
Radiation Therapy Planning: A
A Design
Design Oriented Expert
Expert
System.
WESTEX-87 (Western Conference on Expert
Expert Systems),
Systems), Anaheim, California,
California,
System. WESTEX-87
June 2-4, 1987.
1987.
I. Kalet
Kalet and J.J. Jacky:
Jacky: Knowledge-based
Knowledge-based Computer Simulation
Simulation for
for Radiation
Radiation Therapy
Planning. Proceedings
Proceedings of
of the Ninth
Ninth International
International Conference on the use
use of Computers
in Radiotherapy, Scheveningen,
Scheveningen, the
the Netherlands,
Netherlands, June
June 1987.
1987. North
North Holland, 1987.
1987.
243
243
E.
E. H.
H. Shortliffe
Shortliffe
5P41- RR00785-14
5P41-RR00785-14
Scientific Subproject
Subproject Abstracts
Abstracts
Scientific
AIM Pilot
Pilot Project:
Project:
AIM
COMPUTER-BASED EXERCISES
EXERCISES IN
IN
COMPUTER-BASED
PATHOPHYSIOLOGIC DIAGNOSIS
DIAGNOSIS
PATHOPHYSIOLOGIC
Principal Investigator:
Investigator:
Principal
Robert Beck, M.D.
M.D.
J. Robert
MIS Butler
Butler II
MIS
Maynard St.
2 Maynard
Dartmouth College
Dartmouth
of Medicine
Medicine
School of
NH 03755
Hanover, NH
(603) 646-7171
646-7171
(603)
Dartmouth Medical
Medical School focuses
focuses on three main
main
Research in
in artificial
artificial intelligence
intelligence at Dartmouth
Research
areas: 1) knowledge-based
knowledge-based systems
systems applied
applied to laboratory
laboratory medicine
medicine and pathology, 2)
areas:
acquisition using machine learning
learning techniques,
techniques, and 3) computer-based
knowledge acquisition
instruction using artificial
artificial intelligence
intelligence techniques
techniques to critique
critique students’
students' workup
workup plans.
instruction
of how knowledge
in common the fundamental
fundamental research
research questions of
These projects have in
represented and used
used in
in a classification
classification approach to problem-solving
problem-solving related
should be represented
use of
of laboratory
laboratory data.
data.
to the use
An interdisciplinary
interdisciplinary team
team of
of computer
computer scientists, physicians, and educators
educators is working
working
An
Exercises project. A
A prototype
prototype system
system is nearing completion,
completion,
on the Computer-based Exercises
for Fall, 1987.
with formative
formative evaluation scheduled
scheduled for
with
1987.
REFERENCES
REFERENCES
Beck, lR.,
J.R., Prietula,
Prietula, MJ.,
M.J., Russo,
Russo, E.A.: A
A role for
for intelligent
intelligent systems
systems in teaching medical
pathophysiology. In: Salamon,
Salamon, R., Blum, B., Jorgenson,
Jorgenson, M. (eds).
(eds). Proc. Fifth
Fifth Conf.
Conf.
Med. Inform.
Inform. (MEDINFO
(MEDINFO '86),
‘86), Elsevier-North
Elsevier-North Holland,
Holland, Amsterdam, 1986,
1986, 936-938.
Beck, lR.:
Artificial intelligence: A
A topic
topic for
for Medical Decision Making? (edit.)
(edit.) Med.
J.R.: Artificial
Decis. Making
Making 1987;
1987; 7:4.
7:4.
E.
E. H.
H. Shortliffe
Shortliffe
244
244
References
References
5P41-RR00785-14
5P41-RR00785-14
References
Barr, Avron,
Avron, Paul R. Cohen and Edward A. Feigenbaum. The Handbook
Handbook of
of
1. Barr,
Artificial
Intelligence,
Volumes
I,
II,
and
III.
William
Kaufmann,
Inc.,
Los
Altos, CA,
Altos,
Artificial
Intelligence, Volumes I, II, and III. William Kaufmann,
1981
and
1982.
1981
1982.
Research Service, Library
Library of
of Congress.
Congress. The Impact
Impact of
of Information
Information
2. Congressional Research
Technology on Science.
Science. Science
Science Policy
Policy Study Background Report
Report Number
Number 5, Task Force
Technology
Science Policy,
Policy, Committee
Committee on Science
Science and Technology, U.S. House
House of
of
on Science
Representatives, September, 1986.
Representatives,
"Research Instrument
Instrument Sharing.”
Sharing." Science
Science 202,
201, 4354 (1978).
Coulter, C. L. “Research
3. Coulter,
Harmon, Paul and King,
King, David.
David. Expert
Expert Systems
Systems - Artificial
Artificial Intelligence
Intelligence in
in
4. Harmon,
Business. John Wiley
Wiley &
& Sons,
Sons, Inc., New York,
York, NY,
NY, 1985.
Business.
1985.
5. Hayes-Roth,
Hayes-Roth, F., Waterman, D. A. &
& Lenat, D. B. (Eds.). Building
Building Expert
Expert Systems.
Addison-Wesley,
Reading,
MA,
1983.
Addison-Wesley, Reading, MA, 1983.
Horvitz, E.J.,
EJ., Heckerman, D.E., and Langlotz,
Langlotz, C.P.
c.P. A
A Framework for
for Comparing
Comparing
6. Horvitz,
Alternative Formalisms
Formalisms for
for Plausible Reasoning.
Reasoning. Proceedings
Proceedings of
of AAAI-86,
AAAI-86, Fifth
Fifth
Alternative
National Conference on Artificial
Artificial Intelligence,
Intelligence, American
American Association for
for Artificial
Artificial
National
Intelligence, Menlo
Menlo Park, CA,
CA, August, 1986,
Intelligence,
1986, pp. 210-214.
Information Technology
Technology R&D
R&D Project Staff.
Staff. Information
Information Technology R&D:
R&D: Critical
Critical
7. Information
Trends and Issues
Issues (Summary). Report
Report OTA-CIT-269,
OTA-CIT-269, Congressional
Congressional Office
Office of
of
Technology Assessment,
Assessment, October, 1985.
1985.
Reference Manual.
Manual.
8. Keith
Keith A. Lantz
Lantz et. al. V - System
System 4.1
4.2 Reference
Computer
Computer Systems
Systems Laboratory,
Laboratory, Stanford
Stanford University.
University.
December
December 1, 1983.
1983.
9. Keith
Nowicki. Third
Keith A. Lantz,
Lantz, David
David R. Cheriton,
Cheriton, and William
William I. Nowicki.
Third Generation
Distributed
Systems.
Computer
Systems
Laboratory.
Stanford University,
Graphics for
for Distributed Systems.
Systems
University,
December 21, 1982
1982
10.
“Digital Communications
Communications and the Conduct of
of Science:
Science: The New
10. Lederberg,
Lederberg, J.
J. "Digital
Literacy."
Proceedings of
of the IEEE
IEEE 66, 11
11 (1978).
Literacy.” Proceedings
11.
Intelligence.
Principles of
of Artificial
Artificial
Intelligence. Tioga, Palo Alto,
Alto, CA.
CA, 1980.
1980
11. Nilsson. N.. Principles
12.
12. NLM
NLM Planning
Planning Panel
Panel 4.
4. NLM
NLM Long Range
Range Plan: Medical Informatics.
Informatics. Shortliffe,
Shortliffe,
E.H.,
E-H., Panel
Panel 4 Chairman,
Chairman , National
National Library
Library of
of Medicine, January,
January, 1987.
1987.
13.
Intelligence.
13. Rich, E
E.... Artificial
Artificial
Intelligence. McGraw-Hill,
McGraw-Hill, New York,
York, 1983.
1983.
14.
14. Paul S.
S. Rosenbloom, John E. Laird.
Laird. Mapping
Mapping Explanalion-Based
Explanation-Based Generalization
Generalization
onto SOAR. Proceedings
Proceedings of
of the Fifth
Fifth National
National Conference on Artificial
Artificial Intelligence,
American
American Association for
for Artificial
Artificial Intelligence, Philadelphia, Penn.,
Penn., August
11-15, 1986,
1986, pp. 561-567. Also KSL-86-46
KSL-86-46 and STAN-CS-86-1111
STAN-CS-86-1111
15.
15. Scheifler, R.W. and
and Gettys, J.J. The X
X Window
Window System.
System. Draft
Draft October 1986.
1986. To
appear
appear in a special
special issue
issue of
of the
the ACM
ACM Transactions
Transactions on GraphiCS
Graphics --- user
user interface
software
245
245
E.
E. H. Shortliffe
Shortliffe
5P41-RR00785-14
5P41-RR0078514
Guy L., Jr..
JI.. Common
Common Lisp
Lisp - The Language.
Language. Digital
Digital Press,
Press, Burlington,
Burlington, MA,
MA,
Steele, Guy
16. Steele,
1984.
NeWS Preliminary
Preliminary Technical
Technical Overview.
Overview.
17. NeWS
1986.
Sun Microsystems, Inc., 1986.
Waterman, Donald
Donald A..
A .. AA Guide
Guide to Expert
Expert Systems.
Systems. Addison-Wesley
Addison-Wesley Publishing
Publishing
18. Waterman,
Company, Menlo
Menlo Park, CA,
CA, 1986.
Company,
David C. Wilkins
Wilkins and Bruce G. Buchanan. On Debugging Rule Sets
Sets When
19. David
Under Uncertainty.
Uncertainty. Tech. Rept. KSL
KSL 86-30, Stanford
Stanford University,
University, Knowledge
Reasoning Under
Systems Laboratory,
Laboratory, July, 1986. Earlier
Earlier version appeared
appeared in
in Proceedings
Proceedings of
of the Fifth
Fifth
Systems
National Conference
Conference on Artificial
Artificial Intelligence,
Intelligence, August 1986
National
David C. Wilkins,
Wilkins, Willian
Willian J.
20. David
Differential
Modeling
Evaluating
Evaluating Differential Modeling
Systems. Tech. Rept. KSL
KSL 86-62,
Systems.
October, 1986.
1986.
Buchanan. Using and
Clancey and Bruce G. Buchanan.
in Intelligent
Intelligent Tutoring
Tutoring and Apprentice
Apprentice Learning
Learning
Stanford University,
University, Knowledge Systems
Systems Laboratory,
Laboratory,
Stanford
David C. Wilkins.
Wilkins. Knowledge Base
Base Debugging Using Apprenticeship
Apprenticeship Learning
Learning
21. David
KSL 86-63, Stanford
Stanford University,
University, Knowledge Systems
Systems
Techniques. Tech. Rept. KSL
Laboratory, December, 1986.
Laboratory,
1986.
David C. Wilkins,
Wilkins, William
William J. Clancey and Bruce G. Buchanan.
Buchanan. Knowledge Base
Base
22. David
Refinement Using
Using Abstract
Abstract Control
Control Knowledge. Tech. Rept. KSL
KSL 87-01, Stanford
Stanford
Refinement
University,
University, Knowledge Systems
Systems Laboratory,
Laboratory, January, 1987.
1987.
23. Winston,
Artificial Intelligence,
Intelligence, 2nd ed. Addison-Wesley,
Winston, PP.... Artificial
Addison-Wesley, Reading,
Reading, MA,
MA, 1984.
1984.
E.
E. H. Shortliffe
Shortliffe
246
246
Source Exif Data:
File Type : PDF File Type Extension : pdf MIME Type : application/pdf PDF Version : 1.6 Linearized : No XMP Toolkit : Adobe XMP Core 4.2.1-c041 52.342996, 2008/05/07-21:37:19 Format : application/pdf Creator : StreetI Description : Title : Create Date : 2001:06:16 09:59:26Z Creator Tool : Acrobat Capture Server 2.01 Modify Date : 2018:03:20 09:09:35-07:00 Metadata Date : 2018:03:20 09:09:35-07:00 Keywords : Producer : Adobe Acrobat 9.0 Paper Capture Plug-in Document ID : uuid:9aaf7e16-b88d-6347-96f1-c670a5a852dd Instance ID : uuid:efb63085-79ee-db44-b967-991935183c3e Page Layout : SinglePage Page Count : 247 Author : StreetI Subject :EXIF Metadata provided by EXIF.tools