Presented at the 1999 APIII Conference Return to 1999 Abstract Index
EXPLORATION, MANIPULATION AND PROCESSING OF VERY LARGE DATASETS USING FILTERS
Johns
Hopkins Medical School
Department of Pathology
University of Maryland, College Park
Department of Computer Science
Baltimore, Maryland
Joel
Saltz MD, PhD
Mike Beynon, Joel Saltz MD,PhD, Mustafa Uysal PhD, Alan Sussman, PhD
We are developing a software suite that supports the decomposition of retrieval and processing programs into a set of interconnected filters coupled by streams. Interconnected networks of filters will be able to carry out patterns of processing and retrieval needed to support a wide range of applications. Filters have the following attributes:
- ability to run on any one of a variety of distributed data or computer servers,
- can be configured to make use of bounded quantities of memory and disk space, and
- coupled by data streams.
We are
particularly interested in applications that access, manipulate
and process extremely large distributed datasets. In the
medical domain, these datasets can arise from acquiring
entire pathology specimens at high power or from radiology
imaging modalities such as spiral CT and functional MRI.
In non-medical domains, motivating applications include
oil reservoir simulation, groundwater modeling, exploration,
visualization and analysis of very large datasets arising
from scientific simulations, and analysis of satellite imagery.
We have developed and will discuss a prototype of our filter-based
environment that is able to support the interactive exploration
of large microscopy datasets obtained using a Bliss microscope
with a robotically controlled stage. We will also briefly
describe related projects that target processing and exploration
of large datasets (Active Data Repository for disk based
data and DataCutter for data collections maintained in tertiary
storage).
