David M. Parsons, James E. Cabral, Jr., Yongmin Kim, Gregory L. Lipski, and Mark S. Frank*
Image Computing Systems Laboratory, Departments of Electrical Engineering and Radiology*, University of Washington Seattle, WA 98195
Building on the success of the MediaStation 5000 (MS5000) multimedia system which was developed in our laboratory in 1994, we have developed a prototype telemedicine workstation which is programmable and supports high-bandwidth telecommunications links to connect together many medical treatment facilities. The system can support various telemedicine and consultation functions to collaboratively transfer, manipulate, and view radiological images, image sequences, audio, and video. The requirements for a telemedicine workstation include high performance, flexibility, and upgradability. The unique components of our workstation include an advanced parallel processor, highly-integrated multimedia support circuitry, high-speed network interface, and a graphical user interface. Each of these is a key ingredient to a successful telemedicine system.
In order to increase the access and quality of medical care, especially in remote areas, we have begun work on a flexible, cost-effective telemedicine workstation. The purpose of telemedicine is to link hospitals, clinics, and remote locations to enable health care providers to exercise their expertise at the location of the patient or other collaborating care providers using a combination of video, audio, and externally acquired images. The telemedicine system should give remote clinics better access to speciality care and help manage limited health care resources more effectively. Possible application areas in teleconsultation and telediagnosis include ultrasound examinations, radiology, pathology, endoscopy, dermatology, and psychiatry. In addition, the system can be used for educational purposes to support bidirectional video/audio communications for grand round lectures, classes, and case conferences. The ultimate goal of the project is not only to demonstrate the feasibility of a MS5000-based telemedicine workstation, but to make it a clinically useful system.
In order to gain hands-on experience on telemedicine and its requirements, we have developed a prototype telemedicine workstation. Because of its flexibility and large processing power, the MS5000, developed at the Image Computing Systems Laboratory at the University of Washington, is being used as the centerpiece of the workstation1. The MS5000 is a single board multimedia system capable of digitizing audio and video, displaying up to 1280 x 1024 pixels, and performing 2 billion operations per second using the Texas Instruments TMS320C80 MVP (Multimedia Video Processor). Coupled with a network card such as an asynchronous transfer mode (ATM) interface adapter, the system can be equipped to transmit and receive video, audio, and medical images. For high-bandwidth applications, the MS5000 can encode video using the Motion Picture Experts Group (MPEG) standard and for low-bandwidth links, the International Telecommunication Union (ITU) H.320 standard for video conferencing can be used. Since the MVP is programmable, it can also perform other tasks in addition to compression including image display, image processing, and graphics functions such as window and level, unsharp masking, and 3-D reconstruction.
We have developed software to tailor the MediaStation 5000 for use as an integrated teleradiology/telemedicine workstation prototype. Using ATM network adapters in the MediaStation 5000s, a high-speed link up to OC-3 at 155 Mbits/s (Mbps) can be established between health care providers at different sites. For example, using the ATM switching fabric over the LATA (Local Access Transport Area) Integrated Optical Network (LION) fiber-ring in the Puget Sound area, high-speed connections can be made among academic, military, and Veterans Administration (VA) hospitals and clinics in the region. All of the compression and image processing features are implemented in software rather than using a dedicated hardware approach. This allows the MediaStation 5000 to adapt to changing processing needs, e.g., evolving compression algorithms with wavelets and vector quantization. In this paper, we present the requirements for a successful telemedicine system as well as the architecture, organization, and our experience with the prototype telemedicine workstation that we developed.
In order for a telemedicine system to be clinically useful, it must have several features including programmability, high-performance, flexibility, and upgradability. It must provide programmable handling and compression of video, audio, and images to support applications ranging from typical video teleconferencing to diagnostic-quality consultations. Programmable handling of data will allow the system to improve and adapt to changing requirements from continued research in telemedicine. Programmable compression is necessary for the system to adapt to new compression algorithms and standards (both international and de facto). For high-resolution images or full motion video, high-performance is required to handle the large amounts of incoming data. Depending on the medical specialty supported by the system, it must also be able to interface to various medical imaging modality equipment ranging from a high-resolution digital still camera for dermatology and X-ray, CT and MR images for teleradiology to an ultrasound machine for obstetrics and an electronic stethoscope. Desirable features include the ability to store consultation sessions so that the remote expert is not required to be present simultaneous to the examination. In order to maximize the utilization of the available transmission medium (from land-based fiber optic cable to satellite link) while providing the best quality video and audio, the system should adapt to a wide variety of bandwidths from 56 kbps to over 45 Mbps. The more challenging and difficult the remote consultation and diagnosis, the higher bandwidth the clinical application will require to provide better quality services.
With regard to distance, there are two types of telemedicine systems, remote and regional. Where remote telemedicine provides medical care to distant, under-served locations2, regional telemedicine occurs between medical treatment facilities located in the same metropolitan area. Each facility may not be able to justify a full-time specialist, but they could use a telemedicine system to share one full-time specialist position. Instead of having the physicians or patients travel between the facilities, a telemedicine system could be used to transport the patient information. However, when the medical facilities are located only an hour or two apart, physicians must perceive the system as an acceptable alternative to physically referring the patient to another hospital. To meet the clinical requirements of a regional telemedicine system, it must be fast, reliable, easy to use, and provide excellent image quality. Otherwise, a physician will choose to have the patient travel to the other medical facility bypassing the telemedicine system. In our prototype telemedicine system, we have addressed these problems by using a high-bandwidth ATM connection between the sites for fast, high-quality image transfer, an established operating system for reliability, and a graphical user interface for ease of use.
The telemedicine workstation we have prototyped is a medical imaging workstation with added multimedia capabilities for teleconferencing. Medical images can be displayed at a spatial resolution of 1280 x 1024. Various image manipulation capabilities are available through the graphical user interface such as rotate, flip, pan, window/level, zooming, and unsharp masking. There are also additional capabilities for image sequences such as cine display of the studies from cross-sectional imaging modalities and synchronized slice scrolling for MRI studies where the same anatomical slices are acquired in different pulse sequences. For example, T1, T2, and proton density images are displayed simultaneously and scrolled together. As shown in Fig. 1, images can be acquired through the built-in video digitizer and decoder of the MediaStation 5000 or through add-on hardware such as a digital still camera or laser film digitizer connected through the SCSI chain or a plug-in adapter card.
Figure 1. Telemedicine workstation
A telemedicine session is initiated by selecting a remote destination from which consultation is being sought. Once a connection is established, video, audio, or still images can be sent or received. The expert at the remote site can either view the images or video as they are being transmitted or have them stored in the expert's telemedicine workstation for later review and report. When still images are sent, they are displayed as linked images. This means that changes that one user makes to the images, such as panning an image or window/level, are reflected on the image display at the other end of the link. An additional feature is the ability to locate areas of interest on the image with the cursor. This is displayed in real time to the other user. Whe n the remote expert is ready to make a diagnosis, he or she can send it back simply by talking through the audio channel or by sending a text message.
The workstation is based on a 486 PC running Windows NT from Microsoft (Redmond, WA). The main components of the system are the MediaStation 5000 card and an ATM adapter card. We developed two of these systems, one installed at the University of Washington in Seattle and the other at Madigan Army Medical Center (MAMC) at Fort Lewis, Washington, a distance of approximately 50 miles.
The key component of the MS5000 system is the Texas Instruments MVP3. The MVP can perform both video and audio compression with a single chip. The MVP contains multiple processors on a single chip connected to shared on-chip SRAMs through a crossbar network. The MVP architecture can handle different numbers of advanced digital signal processing (DSP) cores depending on the performance requirements of the system. For the MediaStation 5000 system, we use the MVP with four DSP cores to perform MPEG4 video and audio encoding and various image processing tasks. The crossbar is capable of switching on a cycle-by-cycle basis with a maximum rate of 2.4 Gbytes/s for data in addition to 1.8 Gbytes/s for instructions, for a total of 4.2 Gbytes/s. The transfer controller (TC) is responsible for all data movement between off-chip memory and internal SRAM. The MVP provides two video controllers, which are used to control image input or output devices, such as video cameras and display monitors.
The MVP has a general-purpose RISC processor with an IEEE 754-compatible floating-point unit (FPU). The RISC processor has 32-bit instructions and can load and store 64 bits of data at a time. It has an efficient FPU pipeline and contains features that simplify the parsing of Huffman-coded bitstreams, including left-most/right-most one detection, and left-most/right-most bit-change detection. This processor also acts as a supervisor to the advanced DSP cores.
The advanced DSP cores are capable of maintaining a very high rate of operation, and are especially optimized for performing DCT, FFT, motion estimation, and other image and video processing routines. This high processing rate is accomplished through a combination of advanced features.
The MVP's video controllers provide the necessary timing signals and VRAM serial port control to support video output. This significantly simplifies the external logic. The programmable nature of the video controllers allows a wide variety of video resolutions to be supported. In the MediaStation 5000 system, the video controllers are used along with the Brooktree Bt885 videocache DAC to control the merging of the video buffer and frame buffer serial output streams.
The transfer controller (TC) is in charge of interfacing with the external memory system. It accepts, prioritizes, and executes data transfers requested by the various internal processors. It also performs memory refresh and VRAM serial register control. The TC has a flexible interface that supports a wide variety of memory including DRAM, VRAM, SRAM, and user-defined memory interfaces.
Fig. 2 is a block diagram of the MediaStation 5000 system. A 64-bit DRAM memory bank, consisting of two 32-bit wide single in-line memory modules (SIMMs), is connected directly to the MVP's data lines to minimize buffer delays. Using the DRAM's page mode allows us to achieve one memory access (64 bits) every 2 clock cycles. Since most of the memory accesses are targeted to DRAM, the system is able to maintain high bus throughput.
Figure 2. MediaStation 5000 block diagram.
Video input decoding is performed by a single-chip video decoder. This chip accepts NTSC, PAL, or S-Video as input, digitizes the incoming video data, and outputs the pixels in CCIR 601 4:2:2 (Y-Cr-Cb) format. This 16-bit video data is written alternately to two 64 x 16-bit FIFOs expanding the data to 32 bits to match the width of the video bus. The MediaStation 5000 system uses an audio codec for audio input and output. The codec is a single chip that contains 16-bit stereo A/D and D/A converters and supports sampling frequencies up to 48 kHz. Data can be manipulated in mono or stereo, 8 or 16 bits per sample in linear or companded (u-law or A-law) formats. Audio data is buffered bidirectionally through two 4k x 8-bit FIFOs.
The network architecture of the system is shown in Fig. 3. Because of its flexibility and ability to accommodate the large amount of data required for telemedicine, we decided to use the ATM standard5 for networking the workstations. One ESA-200PC ATM adapter card from Fore Systems (Pittsburgh, PA) was installed in each workstation. Using a fiber optic cable, the adapter card was then connected to one of Fore's ASX-200 ATM switches located at each site. The switch was then connected to a high-speed line provided by US West, i.e., a DS-3 connection operating at 45 Mbps between the two sites.
Figure 3. Network Architecture.
Software for the telemedicine workstation was developed in three major areas, user interface, networking, and image processing. The first two were implemented on the host PC using Visual C++ 2.0 from Microsoft, while the image processing functions were implemented on the MVP using tools available from Texas Instruments and other tools developed in our laboratory.
The graphical user interface was developed from the Microsoft Foundation Classes, an object-oriented class library included with Visual C++. When video or images are displayed, they are represented with movable, resizable windows. Subtle abnormalities in an image can be made clearer by using operations such as window/level and zooming which are immediately available to the user through selecting menu items or pushing "toolbar buttons." Menu selections are used to connect to another telemedicine workstation and to send images, video, audio, or text messages. Cursor sharing is active while the two systems are connected. This allows the user to see where the other user is pointing to on an image.
At the time we began designing the networking software, an ATM application programming interface (API) was not available for Windows NT. Additionally, for us to be able to use non-ATM networks in the future, we used the WinSock API provided with Visual C++ to access TCP/IP services over ATM.
A different logical channel is created for each type of data, including video, audio, images, and system messages. Data transfers are coordinated through the system message channel. Whenever one workstation needs to start sending video, audio, or image data, a system message is sent. Information is also sent on the system channel that ensures the two workstations are in sync, including cursor location and image manipulation commands.
The MVP on the MediaStation 5000 handles MPEG encoding and decoding of video, audio capture and playback, and various image processing functions. To support basic medical image manipulation the following image processing function were implemented:
All of these operations are supported by software. For example, the window/level operation calculates for each pixel in the image a new gray level through multiplication, addition, and clipping. Each of the four advanced DSPs in the MVP work on one-forth of the image calculating a new pixel value every three cycles. On a 512 x 512 image, the window/level operation is performed in 8 ms, zoom in 9 ms, shrink in 2 ms, 90 degree rotate in 5 ms, and flip in 4 ms. Initially, zoom and shrink operate only in increments of two. In other words, the image size can only be doubled or halved, although this can be done multiple times.
From our experience with the prototype system, we have determined which additional features are necessary for a successful telemedicine workstation. First, the system must be designed for high data throughput. We found the PC's EISA/VL-bus to be a bottleneck. Using the high-performance PCI standard for the bus may help, but ideally the MediaStation 5000 and ATM adapter would be integrated together via a daughter card implementing a dedicated, high-speed communication channel. This integration could be achieved with a direct connection or an auxiliary bus. An important software feature is the utilization of existing standards such as the Digital Imaging and Communications in Medicine (DICOM) format for medical images and H.320 for teleconferencing. The workstation will then support interoperability, being able to interface with other equipment and systems supporting H.320.
By using the truly programmable nature and processing performance of the MediaStation 5000, we have built a prototypical workstation to be used in a regional telemedicine system which requires high-performance, flexible and upgradable workstations with several key telemedicine functions tightly integrated. The workstation provides functions for fast medical image display and manipulation as well as more telemedicine specific features such as collaborative viewing, multimedia data (image, video, audio, and text) transmission and cursor sharing. With its graphical user interface and high performance, the system is both easy and interactive. Along with advances in telecommunications and data storage and archiving, highly-integrated multimedia workstations such as the MediaStation 5000 will increase the number of useful applications of telemedicine in the future.