Publications

2012
Visualization in Connectomics
Pfister H, Kaynig V, Botha CP, Bruckner S, Dercksen VJ, Hege H-C, Roerdink JBTM. Visualization in Connectomics. In: arXiv.org. 2012 Paper
Multi-Video Browsing and Summarization
Dale K, Shechtman E, Avidan S, Pfister H. Multi-Video Browsing and Summarization. International Workshop on Large-Scale Video Search and Mining 2012;Abstract
We propose a method for browsing multiple videos with a common theme, such as the result of a search query on a video sharing website, or videos of an event covered by multiple cameras. Given the collection of videos we first align each video with all others. This pairwise video alignment forms the basis of a novel browsing interface, termed the Browsing Companion. It is used to play a primary video and, in addition as thumbnails, other video clips that are temporally synchronized with it. The user can, at any time, click on one of the thumbnails to make it the primary. We also show that video alignment can be used for other applications such as automatic highlight detection and multivideo summarization.
Paper
2011
Detection of Neuron Membranes in Electron Microscopy Images using Multi-scale Context and Radon-like Features
Seyedhosseini M, Kumar R, Jurrus E, Giuly R, Ellisman M, Pfister H, Tasdizen T. Detection of Neuron Membranes in Electron Microscopy Images using Multi-scale Context and Radon-like Features. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2011;6891:670-677.Abstract
Automated neural circuit reconstruction through electron microscopy (EM) images is a challenging problem. In this paper, we present a novel method that exploits multi-scale contextual information together with Radon-like features (RLF) to learn a series of discrimi- native models. The main idea is to build a framework which is capable of extracting information about cell membranes from a large contextual area of an EM image in a computationally efficient way. Toward this goal, we extract RLF that can be computed efficiently from the input image and generate a scale-space representation of the context images that are obtained at the output of each discriminative model in the series. Com- pared to a single-scale model, the use of a multi-scale representation of the context image gives the subsequent classifiers access to a larger con- textual area in an effective way. Our strategy is general and independent of the classifier and has the potential to be used in any context based framework. We demonstrate that our method outperforms the state-of- the-art algorithms in detection of neuron membranes in EM images.
Paper
Display-aware Image Editing
Jeong WK, Johnson MK, Yu I, Kautz J, Pfister H, Paris S. Display-aware Image Editing. In: International Conference on Computational Photography (ICCP). 2011Abstract
We describe a set of image editing and viewing tools that explicitly take into account the resolution of the display on which the image is viewed. Our approach is twofolds. First, we design editing tools that process only the visible data, which is useful for images larger than the display. This encompasses cases such as multi-image panoramas and high-resolution medical data. Second, we propose an adaptive way to set viewing parameters such brightness and contrast. Because we deal with very large images, different locations and scales often require different viewing parameters. We let users set these parameters at a few places and interpolate satisfying values everywhere else. We demonstrate the efficiency of our approach on different display and image sizes. Since the computational complexity to render a view depends on the display resolution and not the actual input image resolution, we achieve interactive image editing even on a 16 gigapixel image.
Paper Video
Evaluation of Artery Visualizations for Heart Disease Diagnosis
Borkin MA, Gajos KZ, Peters A, Mitsouras D, Melchionna S, Rybicki FJ, Feldman CL, Pfister H. Evaluation of Artery Visualizations for Heart Disease Diagnosis. IEEE Transactions on Visualization and Computer Graphics 2011;17(12):2479.Abstract
Heart disease is the number one killer in the United States, and finding indicators of the disease at an early stage is critical for treatment and prevention. In this paper we evaluate visualization techniques that enable the diagnosis of coronary artery disease. A key physical quantity of medical interest is endothelial shear stress (ESS). Low ESS has been associated with sites of lesion formation and rapid progression of disease in the coronary arteries. Having effective visualizations of a patient’s ESS data is vital for the quick and thorough non-invasive evaluation by a cardiologist. We present a task taxonomy for hemodynamics based on a formative user study with domain experts. Based on the results of this study we developed HemoVis, an interactive visualization application for heart disease diagnosis that uses a novel 2D tree diagram representation of coronary artery trees. We present the results of a formal quantitative user study with domain experts that evaluates the effect of 2D versus 3D artery representations and of color maps on identifying regions of low ESS. We show statistically significant results demonstrating that our 2D visualizations are more accurate and efficient than 3D representations, and that a perceptually appropriate color map leads to fewer diagnostic mistakes than a rainbow color map.
Paper Slides
Maximizing All Margins: Pushing Face Recognition with Kernel Plurality
Kumar R, Banerjee A, Vemuri B, Pfister H. Maximizing All Margins: Pushing Face Recognition with Kernel Plurality. In: International Conference on Computer Vision (ICCV). 2011 p. 2375-2382. Paper Video
Neural Process Reconstruction from Sparse User Scribbles
Roberts M, Jeong W-K, Vazquez-Reina A, Unger M, Bischof H, Lichtman J, Pfister H. Neural Process Reconstruction from Sparse User Scribbles. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2011;6891:621-628. Paper Slides
Segmentation Fusion for Connectomics
Vazquez-Reina A, Huang D, Gelbart M, Lichtman J, Miller E, Pfister H. Segmentation Fusion for Connectomics. In: International Conference on Computer Vision (ICCV). Barcelona, Spain: IEEE; 2011 p. 177-184. Paper Video
Video Face Replacement
Dale K, Sunkavalli K, Johnson MK, Vlasic D, Matusik W, Pfister H. Video Face Replacement. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 2011;30 Paper Video
2010
Design and Fabrication of Materials with Desired Deformation Behavior
Bickel B, Bacher M, Otaduy MA, Lee HR, Pfister H, Gross M, Matusik W. Design and Fabrication of Materials with Desired Deformation Behavior. ACM Transactions on Graphics (Proc. ACM SIGGRAPH) 2010;29(3):63:1-63:10. Paper Video
Interactive Histology of Large-Scale Biomedical Image Stacks
Jeong W-K, Schneider J, Turney SG, Faulkner-Jones BE, Meyer D, Westermann R, Reid CR, Lichtman J, Pfister H. Interactive Histology of Large-Scale Biomedical Image Stacks. IEEE Transactions on Visualization and Computer Graphics 2010;16(6):1386-1395.Abstract
Interactive Histology of Large-Scale Biomedical Image Stacks Authors Won-Ki Jeong; Jens Schneider; Stephen G. Turney; Beverly E. Faulkner-Jones; Dominik Meyer; Ruediger Westermann; R. Clay Reid; Jeff Lichtman; Hanspeter Pfister Abstract Histology is the study of the structure of biological tissue using microscopy techniques. As digital imaging technology advances, high resolution microscopy of large tissue volumes is becoming feasible; however, new interactive tools are needed to explore and analyze the enormous datasets. In this paper we present a visualization framework that specifically targets interactive examination of arbitrarily large image stacks. Our framework is built upon two core techniques: display-aware processing and GPUaccelerated texture compression. With display-aware processing, only the currently visible image tiles are fetched and aligned on-the-fly, reducing memory bandwidth and minimizing the need for time-consuming global pre-processing. Our novel texture compression scheme for GPUs is tailored for quick browsing of image stacks. We evaluate the usability of our viewer for two histology applications: digital pathology and visualization of neural structure at nanoscale-resolution in serial electron micrographs.
Paper Video Images
Multi-scale Image Harmonization
Sunkavalli K, Johnson MK, Matusik W, Pfister H. Multi-scale Image Harmonization. ACM Transactions on Graphics (Proc. ACM SIGGRAPH) 2010;29(4)Abstract
Multi-scale Image Harmonization Authors Kalyan Sunkavalli; Micah K. Johnson; Wojciech Matusik; Hanspeter Pfister Abstract Traditional image compositing techniques, such as alpha matting and gradient domain compositing, are used to create composites that have plausible boundaries. But when applied to images taken from different sources or shot under different conditions, these techniques can produce unrealistic results. In this work, we present a framework that explicitly matches the visual appearance of images through a process we call image harmonization, before blending them. At the heart of this framework is a multi-scale technique that allows us to transfer the appearance of one image to another. We show that by carefully manipulating the scales of a pyramid decomposition of an image, we can match contrast, texture, noise, and blur, while avoiding image artifacts. The output composite can then be reconstructed from the modified pyramid coefficients while enforcing both alpha-based and seamless boundary constraints. We show how the proposed framework can be used to produce realistic composites with minimal user interaction in a number of different scenarios.
Paper Images Slides
Radon-Like Features and their Application to Connectomics
Kumar R, Reina AV, Pfister H. Radon-Like Features and their Application to Connectomics. Computer Vision and Pattern Recognition Workshops (CVPRW) 2010;:186-193.Abstract
Radon-Like Features and their Application to Connectomics Authors Ritwik Kumar; Amelio V. Reina; Hanspeter Pfister Abstract In this paper we present a novel class of so-called Radon-Like features, which allow for aggregation of spatially distributed image statistics into compact feature descriptors. Radon-Like features, which can be efficiently computed, lend themselves for use with both supervised and unsupervised learning methods. Here we describe various instantiations of these features and demonstrate there usefulness in context of neural connectivity analysis, i.e. Connectomics, in electron micrographs. Through various experiments on simulated as well as real data we establish the efficacy of the proposed features in various tasks like cell membrane enhancement, mitochondria segmentation, cell background segmentation, and vesicle cluster detection as compared to various other state-of-the-art techniques.
Paper Images
SSECRETT and NeuroTrace: Interactive Visualization and Analysis Tools for Large-Scale Neuroscience Datasets
Jeong W-K, Beyer J, Hadwiger M, Blue R, Law C, Vazquez A, Reid C, Lichtman J, Pfister H. SSECRETT and NeuroTrace: Interactive Visualization and Analysis Tools for Large-Scale Neuroscience Datasets. IEEE Computer Graphics and Applications 2010;30:58-70.Abstract
SSECRETT and NeuroTrace: Interactive Visualization and Analysis Tools for Large-Scale Neuroscience Datasets Authors Won-Ki Jeong; Johanna Beyer; Markus Hadwiger; Rusty Blue; Charles Law; Amelio Vazquez; Clay Reid; Jeff Lichtman; Hanspeter Pfister Abstract Recent advances in optical and electron microscopy allow scientists to acquire extremely high-resolution images for neuroscience research. Datasets imaged with modern electron microscopes can range between tens of terabytes to about one petabyte in size. These large data sizes and the high complexity of the underlying neural structures make it very challenging to handle the data at reasonably interactive rates. To provide neuroscientists flexible and interactive tools for their scientific work we introduce SSECRETT and NeuroTrace, two systems that were designed for interactive exploration and analysis of large-scale optical and electron microscope images to reconstruct complex neural circuits of the mammalian nervous system.
Paper
CG2Real: Improving the Realism of Computer Generated Images using a Large Collection of Photographs
Johnson MK, Dale K, Avidan S, Pfister H, Freeman WT, Matusik W. CG2Real: Improving the Realism of Computer Generated Images using a Large Collection of Photographs. IEEE Transactions on Visualization and Computer Graphics 2010;17(9):1273-1285. Paper
Enabling a High Throughput Real Time Data Pipeline for a Large Radio Telescope with GPUs
Edgar RG, Clark MA, Dale K, Mitchell DA, Ord SM, Wayth RB, Pfister H, Greenhill LJ. Enabling a High Throughput Real Time Data Pipeline for a Large Radio Telescope with GPUs. Computer Physics Communications 2010;181(10):1707-1714.Abstract
Enabling a High Throughput Real Time Data Pipeline for a Large Radio Telescope with GPUs Authors Richard G. Edgar; Mike A. Clark; Kevin Dale; Daniel A. Mitchell; Stephen M. Ord; Randall B. Wayth; Hanspeter Pfister; Lincoln J. Greenhill Abstract The Murchison Widefield Array (MWA) is a next-generation radio telescope currently under construction in the remote Western Australia Outback. Raw data will be generated continuously at 5 GiB/s, grouped into 8 s cadences. This high throughput motivates the development of on-site, real time processing and reduction in preference to archiving, transport and off-line processing. Each batch of 8 s data must be completely reduced before the next batch arrives. Maintaining real time operation will require a sustained performance of around 2:5 TFLOP/s (including convolutions, FFTs, interpolations and matrix multiplications). We describe a scalable heterogeneous computing pipeline implementation, exploiting both the high computing density and FLOP-per-Watt ratio of modern GPUs. The architecture is highly parallel within and across nodes, with all major processing elements performed by GPUs. Necessary scatter-gather operations along the pipeline are loosely synchronized between the nodes hosting the GPUs. The MWA will be a frontier scientific instrument and a pathfinder for planned peta- and exascale facilities.
Paper
Fast and Automatic Object Pose Estimation for Range Images on the GPU
Park IK, Germann M, Breitenstein MD, Pfister H. Fast and Automatic Object Pose Estimation for Range Images on the GPU. Machine Vision and Applications 2010;21:749-766.Abstract
Fast and Automatic Object Pose Estimation for Range Images on the GPU Authors In Kyu Park; Marcel Germann; Michael D. Breitenstein; Hanspeter Pfister Abstract We present a pose estimation method for rigid objects from single range images. Using 3D models of the objects, many pose hypotheses are compared in a data-parallel version of the downhill simplex algorithm with an imagebased error function. The pose hypothesis with the lowest error value yields the pose estimation (location and orientation), which is refined using ICP. The algorithm is designed especially for implementation on the GPU. It is completely automatic, fast, robust to occlusion and cluttered scenes, and scales with the number of different object types. We apply the system to bin picking, and evaluate it on cluttered scenes. Comprehensive experiments on challenging synthetic and real-world data demonstrate the effectiveness of our method.
Paper Images
MulteeSum: A Tool for Comparative Spatial and Temporal Gene Expression Data
Meyer M, Munzner T, DePace A, Pfister H. MulteeSum: A Tool for Comparative Spatial and Temporal Gene Expression Data. IEEE Transactions on Visualization and Computer Graphics 2010;Abstract
MulteeSum: A Tool for Comparative Spatial and Temporal Gene Expression Data Authors Miriah Meyer; Tamara Munzner; Angela DePace; Hanspeter Pfister Abstract Cells in an organism share the same genetic information in their DNA, but have very different forms and behavior because of the selective expression of subsets of their genes. The widely used approach of measuring gene expression over time from a tissue sample using techniques such as microarrays or sequencing do not provide information about the spatial position within the tissue where these genes are expressed. In contrast, we are working with biologists who use techniques that measure gene expression in every individual cell of entire fruitfly embryos over an hour of their development, and do so for multiple closely-related subspecies of Drosophila. These scientists are faced with the challenge of integrating temporal gene expression data with the spatial location of cells and, moreover, comparing this data across multiple related species. We have worked with these biologists over the past two years to develop MulteeSum, a visualization system that supports inspection and curation of data sets showing gene expression over time, in conjunction with the spatial location of the cells where the genes are expressed — it is the first tool to support comparisons across multiple such data sets. MulteeSum is part of a general and flexible framework we developed with our collaborators that is built around multiple summaries for each cell, allowing the biologists to explore the results of computations that mix spatial information, gene expression measurements over time, and data from multiple related species or organisms. We justify our design decisions based on specific descriptions of the analysis needs of our collaborators, and provide anecdotal evidence of the efficacy of MulteeSum through a series of case studies.
Paper Images
Multiple Hypothesis Video Segmentation from Superpixel Flows
Vazquez-Reina A, Avidan S, Pfister H, Miller E. Multiple Hypothesis Video Segmentation from Superpixel Flows. European Conference on Computer Vision (ECCV) 2010;:268-281.Abstract
Multiple Hypothesis Video Segmentation from Superpixel Flows Authors Amelio Vazquez-Reina; Shai Avidan; Hanspeter Pfister; Eric Miller Abstract Multiple Hypothesis Video Segmentation (MHVS) is a method for the unsupervised photometric segmentation of video sequences. MHVS segments arbitrarily long video streams by considering only a few frames at a time, and handles the automatic creation, continuation and termination of labels with no user initialization or supervision. The process begins by generating several pre-segmentations per frame and enumerating multiple possible trajectories of pixel regions within a short time window. After assigning each trajectory a score, we let the trajectories compete with each other to segment the sequence. We determine the solution of this segmentation problem as the MAP labeling of a higher-order random field. This framework allows MHVS to achieve spatial and temporal long-range label consistency while operating in an on-line manner. We test MHVS on several videos of natural scenes with arbitrary camera and object motion.
Paper Images
Pathline: A Tool for Comparative Functional Genomics
Meyer M, Wong B, Styczynski M, Munzner T, Pfister H. Pathline: A Tool for Comparative Functional Genomics. Computer Graphics Forum (Proc. of EuroVis) 2010;29Abstract
Pathline: A Tool for Comparative Functional Genomics Authors Miriah Meyer; Bang Wong; Mark Styczynski; Tamara Munzner; Hanspeter Pfister Abstract Biologists pioneering the new field of comparative functional genomics attempt to infer the mechanisms of gene regulation by looking for similarities and differences of gene activity over time across multiple species. They use three kinds of data: functional data such as gene activity measurements, pathway data that represent a series of reactions within a cellular process, and phylogenetic relationship data that describe the relatedness of species. No existing visualization tool can visually encode the biologically interesting relationships between multiple path- ways, multiple genes, and multiple species. We tackle the challenge of visualizing all aspects of this comparative functional genomics dataset with a new interactive tool called Pathline. In addition to the overall characterization of the problem and design of Pathline, our contributions include two new visual encoding techniques. One is a new method for linearizing metabolic pathways that provides appropriate topological information and supports the comparison of quantitative data along the pathway. The second is the curvemap view, a depiction of time series data for comparison of gene activity and metabolite levels across multiple species. Pathline was developed in close collaboration with a team of genomic scientists. We validate our approach with case studies of the biologists’ use of Pathline and report on how they use the tool to confirm existing findings and to discover new scientific insights.
Paper Slides

Pages