Engineering Real-Time DICOM Imaging for Browser-Based Radiology Workflows

The Radiology Workflow Problem

Radiology has uniquely demanding requirements for image viewing software. It needs the highest fidelity of any medical discipline (subtle differences in tissue density are diagnostic), handles the largest data volumes in healthcare (a single CT study can exceed 1GB), and requires sub-second interaction responsiveness for window/level adjustments, slice scrolling, and measurement tools. For decades, this meant radiologists were tethered to expensive dedicated workstations running proprietary PACS viewing software.

The case for moving to the browser is straightforward: eliminate hardware lock-in, enable remote reading from any device, and reduce refresh-cycle costs. The case against is also straightforward: browsers historically could not handle the rendering performance or data throughput that radiology demands. Previous browser-based viewers were too slow for clinical use, limited to basic 2D viewing, or required Java/ActiveX plugins that negated the browser advantage.

We believed the gap had closed with modern WebGL, WebAssembly, and HTTP streaming protocols. This post describes what it took to validate that belief and where the gap still exists.

DICOM Parsing in the Browser

DICOM (Digital Imaging and Communications in Medicine) is a format designed by committee in the 1990s and it shows. A single DICOM file contains both pixel data and hundreds of metadata tags describing everything from patient demographics to acquisition geometry and display calibration. The format supports multiple "transfer syntaxes" (endianness and compression combinations), has both implicit and explicit value representation modes, and encodes sequences of nested objects using a tag-length-value scheme that predates XML and JSON. Parsing it is not conceptually difficult, but the specification has enough edge cases and vendor-specific extensions to keep you busy.

Why WebAssembly for the DICOM Decoder

We wrote the DICOM parser in Rust and compiled it to WebAssembly. The alternative was a pure JavaScript parser, and several good open-source options exist (dicomParser.js, for instance). We benchmarked both approaches. The WASM parser handles a typical CT DICOM file in about 2ms versus 18ms for the JS implementation. For a single file that difference is irrelevant, but for a 500-slice CT study the difference is between 1 second and 9 seconds of total parse time, and 9 seconds is not acceptable for clinical use.

The parser uses lazy tag reading: it extracts only the tags needed for initial display (pixel data offset, image dimensions, window/level defaults, photometric interpretation) and defers full metadata parsing until requested. This reduces memory allocation during initial load, which matters for studies that embed large structured reports or private tags in the DICOM headers.

Handling Compressed Transfer Syntaxes

This is where browser-based DICOM viewing gets genuinely hard. Medical imaging uses compression codecs that have no native browser support. JPEG 2000 is used for digital mammography and some CT. JPEG-LS is the most common lossless format for CT and MR. RLE appears in legacy systems. None of these have browser-native decoders. You have to bring your own, which means compiling C/C++ codec libraries to WebAssembly.

We compiled OpenJPEG for JPEG 2000 decoding and CharLS for JPEG-LS. The WASM compilation itself is not the hard part; the hard part is memory management. These codecs allocate and free memory in patterns that the WASM linear memory model handles differently than native execution. We wrote a custom memory allocator that pools and reuses decode buffers, which eliminated the garbage collection pauses (200ms+) that caused visible stuttering during rapid scrolling.

• JPEG 2000: ~180MB/s decode throughput via OpenJPEG WASM. Sufficient for real-time scrolling through mammography studies, though mammography's 4096x4096 images push this to the limit.
• JPEG-LS: ~240MB/s throughput via CharLS WASM. Handles the most common lossless CT/MR compression with mathematically lossless reconstruction.
• RLE: ~320MB/s throughput. Simple codec, fast decode, still encountered in legacy PACS systems.
• Memory management: Custom WASM allocator with buffer pooling reduced GC pauses from 200ms+ to under 5ms during rapid scrolling through a study.

WebGL Rendering Pipeline for Medical Imaging

Displaying medical images is fundamentally different from rendering photographs or video. Medical images are typically 12-bit or 16-bit grayscale (up to 65,536 intensity values), while displays output 8-bit (256 levels). The window/level operation, which maps a clinically relevant portion of the full dynamic range to the display's 8-bit output, must execute in real time as radiologists drag to adjust contrast and brightness. This operation happens on every frame, on every pixel.

GPU-Accelerated Window/Level

We implemented window/level as a WebGL fragment shader. The shader reads from a 16-bit texture (WebGL2's R16UI format), takes window center and width as uniforms, and outputs the mapped 8-bit pixel. This runs entirely on the GPU and achieves consistent 60fps even for 4096x4096 digital mammography images. For comparison, a CPU-based Canvas 2D implementation of the same operation managed 8-12fps on the same hardware. The difference is that window/level is embarrassingly parallel (every pixel is independent), and GPUs have thousands of cores to exploit that parallelism.

We extended the shader pipeline to support lookup tables (LUTs) for specialized display modes. Radiologists use inverted grayscale, hot body, and other pseudocolor LUTs for different clinical scenarios. LUT data is uploaded as a 1D texture and applied in the fragment shader, so switching LUTs is instantaneous with zero CPU involvement.

Multi-Planar Reconstruction

Multi-planar reconstruction (MPR) generates sagittal and coronal views from axially-acquired CT data. It is one of the most demanding rendering tasks in radiology. Our implementation loads the CT volume into GPU memory as a 3D texture and performs trilinear interpolation in the shader to extract arbitrary plane orientations. A standard 512x512x500 CT volume consumes about 500MB of GPU memory, which fits comfortably on modern integrated GPUs (Intel UHD 770 and above). Dedicated GPUs are not required for MPR, which is important because we are targeting standard laptops, not workstations.

• 2D rendering: Sub-16ms frame time (60fps) for all operations including window/level, pan, zoom, and measurement overlays.
• MPR: 45fps for real-time multi-planar reconstruction with trilinear interpolation on integrated Intel UHD 770 graphics. Usable, though not as smooth as a dedicated GPU workstation.
• Volume rendering: Ray-casting at 24fps in the browser. This is the area with the largest gap versus native applications, which achieve 60fps+ with the same data. Sufficient for clinical review but not for primary 3D diagnostic work.
• Measurement tools: Length, angle, area (ellipse and freehand), and Hounsfield unit sampling rendered as WebGL overlays with sub-pixel accuracy.

PACS Integration and WADO-RS Architecture

A real-world deployment means integrating with existing PACS systems, and most organizations run more than one due to mergers and departmental purchases. Rather than writing point integrations for each vendor, we built a DICOMweb gateway that normalizes all PACS communication to the WADO-RS (Web Access to DICOM Objects by RESTful Services) standard. PACS systems that already support DICOMweb natively get a pass-through proxy with caching. Legacy systems that only speak DIMSE (the traditional DICOM network protocol) get real-time protocol translation.

The Gateway Abstraction

The gateway handles three DICOMweb operations: QIDO-RS for querying studies, WADO-RS for retrieving images, and STOW-RS for storing annotations. For each operation, the gateway translates to the appropriate vendor-specific protocol. The value of this abstraction was validated concretely when one PACS vendor was replaced during the first year of operation with zero changes to the viewer code. The new PACS got a new gateway adapter; everything upstream was untouched.

We added a prefetching layer that analyzes the radiologist's worklist and begins streaming studies to an edge cache before they are opened. By the time a radiologist clicks on a study, most of the image data is already cached locally. This prefetching is the single largest contributor to perceived performance improvement.

Progressive Streaming

Traditional PACS viewers download an entire study before displaying anything. Our viewer uses progressive streaming: it fetches a low-resolution key image first (displayed in under 200ms), then the full-resolution version of the currently viewed slice, then adjacent slices in both directions. The radiologist sees a usable image quickly and can begin scrolling while remaining slices stream in the background. This changes the experience from "wait for the study to load" to "start reading immediately."

• Initial display: Under 2 seconds from click to first diagnostic-quality image for most studies. Under 1 second when the study is in the edge cache from prefetching.
• Full study availability: Around 8 seconds for a 500-slice CT to be fully scrollable, with progressive quality improvement during load.
• Cache hit rate: About 73% of studies served from edge cache due to worklist-based prefetching.
• Bandwidth strategy: JPEG 2000 progressive encoding delivers quality in layers, starting at a low bit rate and refining to mathematically lossless. This is a natural fit for progressive streaming.

AI-Assisted Airway and Anatomical Analysis

We embedded an AI-assisted analysis feature that runs inference directly in the browser. The initial model performs airway segmentation and measurement on CT studies, a task that takes 15-20 minutes manually (trace the airway boundaries, measure diameters, calculate cross-sectional areas). Our model completes it in under 30 seconds. Running in the browser means no data leaves the local environment for inference, which simplifies the PHI compliance picture.

Browser-Based Inference with ONNX Runtime Web

We chose ONNX Runtime Web because it uses WebGL for GPU acceleration without browser plugins. The model is a modified 3D U-Net with 12M parameters, quantized from FP32 to FP16 with negligible accuracy loss (Dice score dropped from 0.943 to 0.939). The quantized model is 24MB, small enough to cache in the browser's Cache API for instant subsequent loads. Inference runs on overlapping patches of 128x128x64 voxels, achieving about 18 patches per second of segmentation throughput.

There was a trade-off here between running inference on a server (faster, more powerful GPU) versus in the browser (no PHI transmission, simpler architecture). For a model this size, browser inference is fast enough to be useful. For larger models (say, a 100M+ parameter segmentation model for full-body CT), server-side inference would be necessary. The architecture supports both paths, but for the initial deployment, browser-side inference was the simpler option.

Validation

We validated against 500 expert-annotated CT studies. The model achieved a Dice coefficient of 0.939 for airway segmentation with a mean surface distance error of 0.8mm. Automated tracheal diameter measurements correlated with manual expert measurements at r=0.97, bronchial angles at r=0.94. These numbers were reviewed by the radiology AI governance committee and the model was approved for clinical use as a decision-support tool, meaning radiologists use it to accelerate their work but sign off on the final measurements.

• Airway segmentation: Dice 0.939, mean surface distance 0.8mm, 28 seconds processing time per study.
• Automated measurements: Tracheal diameter within 0.4mm of expert measurement (95% CI), bronchial angles within 2.1 degrees.
• Incidental finding detection: A secondary model flags potential pulmonary nodules >4mm as a safety net. Sensitivity is good (91%), specificity acceptable (87%). Not a replacement for the radiologist's read, but catches things that might be overlooked in a busy worklist.
• Time savings: Airway analysis time went from ~17 minutes (manual tracing and measurement) to ~2 minutes (AI-assisted review and confirmation).

Performance Benchmarks and Clinical Validation

We benchmarked the browser viewer against the existing desktop PACS across 12 clinical scenarios on three hardware configurations: a dedicated radiology workstation (the baseline to beat), a standard business laptop (the target for remote reading), and an iPad Pro (mobile reading).

Where the Browser Matches Native

On the standard laptop, the browser viewer hit about 92% of the desktop workstation's rendering performance for 2D operations (measured as average frame time across a standardized interaction sequence). For the workflows that represent 95% of daily radiology work (axial scrolling, window/level, measurements, comparison with priors), the difference was imperceptible in blinded testing. Radiologists could not reliably tell which viewer they were using.

On the iPad Pro, 2D rendering maintained 60fps for everything except digital mammography (which dropped to 30fps due to the 4096x4096 image size). Touch-based measurement tools worked well. The iPad was validated for on-call consultation and preliminary interpretation but not primary diagnostic reads.

Where Native Still Wins

Volume rendering is the gap. The browser achieved 67% of desktop performance for ray-casting 3D visualization. At 24fps, it is usable for review but not for the smooth interaction that radiologists expect when rotating a volume. This gap will narrow as WebGPU matures and replaces WebGL for compute-heavy tasks, but today, if your primary workflow is 3D reconstruction, you still want a native application.

Read Time Impact

After three months of production use across 142,000 studies, average read time decreased by about 14% for CT studies (driven by faster load times and AI-assisted analysis) and about 22% for complex multi-series studies (driven by improved comparison tools and hanging protocol automation). Simple studies like chest X-rays showed no change, which makes sense: they load instantly on any platform and the read is limited by clinical assessment time, not tooling.

Deployment Outcomes and Radiologist Feedback

The viewer went into production across all campuses over a 4-month phased rollout. Here is what we observed after 12 months.

• Hardware costs: Eliminated the dedicated PACS workstation refresh cycle. Standard hospital laptops replaced 120 workstations, which is a meaningful capital expense reduction.
• Remote reading: All radiologists can now read from home, which improved overnight and weekend coverage from 2 on-site radiologists to 8 reading remotely. This was the most operationally impactful change.
• Study load time: P95 initial display time of about 2 seconds, down from 14+ seconds. The tail latency improvement is what radiologists notice most.
• Vendor flexibility: The DICOMweb abstraction layer proved its value when one PACS system was replaced during the year with zero changes to the viewer.

The two features radiologists cited most in feedback were progressive loading (eliminating "waiting for study" dead time) and the AI-assisted airway analysis (turning a tedious manual task into a rapid review). Several noted that starting a read on their office laptop and continuing at home without workflow disruption was something they had wanted for years.

Browser-based medical imaging has crossed the performance threshold for clinical viability in the majority of radiology workflows. WebGL and WebAssembly closed the gap with native applications for 2D work. The remaining gap in 3D rendering is real but narrowing. WebGPU, once widely available, should close it for most use cases. For now, the practical conclusion is: browser-based for remote reading, on-call, and daily 2D workflows; native workstation for primary 3D and volume rendering work.