Connectomics

We are interested in the segmentation, visualization, and analysis of brain scans in electron and optical microscopy. Our group collaborates with the Center for Brain Science on the segmentation and visualization and analysis of such images. Some of the challenges our group deals with involve the processing of multi-terabyte and multi-spectral image datasets.

The wiring of brains is staggeringly complex. Our own brains have tens of billions of neurons connected through perhaps one hundred trillion synapses. This circuitry is the result of our development and experience; the neural activity that courses through and alters it, somehow accounts for our thoughts, our behavior, our memories. One hundred years from now, this brain circuitry will be known; today, for the first time, we can contemplate mapping it in detail. New forms of laser-scanning light microscopy and semi-automated electron microscopy allow high-resolution imaging of 'connectomes'—that is, complete neural wiring diagrams.

Software tools for connectome research are available on our GitHub.

Funding: Sponsored by IARPA, NSF, NIH and private foundations.

Information, Biomedical and Scientific Visualization

Information Visualization

We investigate the use of data visualizations for presentation and communication of data, reexamining the role of visualization beyond data analysis and exploration. Most visualization systems to date focus on supporting analytical tasks for domain experts, such as exploring data, confirming hypotheses, and discovering insights. However, these systems typically have complex designs that are unintuitive and cumbersome for non-expert users. On the other hand, visualizations are more and more being used to communicate data and messages to a general audience. But visualization tools for communication are still in their infancy and do not address the cognitive aspects of storytelling. To this aim, we conduct cognitive studies to understand what makes a visualization effective for communication, develop techniques for creating expressive, engaging, and communication-minded visualizations, and finally build tools for telling compelling data-driven stories using visualizations.

Biomedical Visualization

We are working to apply novel visualization tools and techniques to leading edge biomedical research. Our current focus is on the 4D Nucleome program which is a collaboration of doctors, biologists, and computational scientists working together to understand the principles underlying nuclear organization in space and time, the role nuclear organization plays in gene expression and cellular function, and how changes in nuclear organization affect normal development as well as various diseases. We are developing visualization methods and tools for exploring and analyzing the large amounts of datasets generated throughout the program.

Visualization for Connectomics
We are developing scalable systems for interactive visual exploration and analysis of large (petavoxel) volumes resulting from high-throughput electron microscopy data streams. This project focuses on the interactive analysis of biomedical 3D volumes and their corresponding segmentation volumes. We research efficient techniques to combine methods from volume visualization with information visualization to allow neuroscientists to interactively explore and analyze their data.

Scientific Visualization
We are researching GPU-based volume rendering techniques for large-scale neuroscience and medical data, with emphasis on visualization of large and multimodal volumes. The tremendous resolution and the high complexity of these volumes present big challenges to storage, processing, visualization and visual analysis at interactive rates. We are currently researching multi-resolution and visualization-driven designs which restrict most computations to a small subset of the data.

Funding: This work was partially supported by NIH Common Fund U01 CA200059, NSF grant OIA-1125087, and King Abdullah University of Science and Technology.

Computer Vision

Image Processing
We are interested in computationally restoring corrupted images to be visually more pleasing, as well as in feature-based exploration of large-scale images.

More and more photos are being taken by hand-held cameras, especially smart phones. Degradation is often inevitable in the capture process and greatly affects the visual experience. As most capture moments are ephemeral, these moments usually get lost when severe degradations happen. We develop algorithms and tools to reduce degradations such as image blur and noise. Our algorithms can well handle images of different categories, including natural, text, face, and low-illumination images.

Enabled by technologies such as robot-assisted gigapixel photography, the resolution of acquired images is increasing while the resolution of display devices lag behind. Part of our work is in bridging this gap by providing users with tools to 1) compute fast yet accurate image operations to any coarse target resolution, and 2) guide users in feature-based exploration of high-resolution images.

Video Processing
Digital video editing and computer graphics have allowed unbridled creativity for professional film makers. However, the software tools and techniques used to accomplish these tasks require expert knowledge and are often slow. We aim to improve existing video processing tools which currently slow down existing users and block new users from the creative process.

One particularly difficult task in advanced video processing is seamlessly changing the appearance of objects, or changing the lighting in scenes. Typically this requires painstaking manual painting, which is laborious, or it requires significant compute resources, taking many hours for just a few seconds of footage. We develop new techniques which provide fast interactive tools for video editing. Our approach greatly quickens the creative process, and so opens the door to more sophisticated video content editing of appearance and illumination.

Funding: Sponsored by NSF, and private foundations.

SportsXR

Our research focuses on developing novel visualization techniques using augmented and virtual reality for sports analytics. Immersive analytics presents promising opportunities to advance sports data analytics to non-data experts in sports domains through the benefits of large displays, natural interactions, and spatial immersion.

Our team is dedicated to exploring innovative ways to enhance the data experience and generate insights for real-world sports tasks. We collaborate with a diverse range of sports domain users, including athletes, coaches, and sports fans, to address user-driven problems and improve sports performance and user experiences using immersive analytics.

In the world of sports, every movement, every decision, and every play matters. From the way a player positions themselves on the field to the trajectory of a ball, every detail can make the difference between victory and defeat. With SportsXR, we believe that the future of sports analytics lies in the ability to visualize and interact with data in real-time, in ways that were previously impossible. With our cutting-edge visualization tools and technologies, we aim to empower athletes, coaches, and fans alike to gain deeper insights into the game, and ultimately, elevate the way we experience and appreciate sports.

SportsBuddy
This initiative targets bringing immersive visualizations to real-world game viewing experiences. Our collaboration with the Harvard men’s basketball team aims to make advanced analytics and visual insights accessible to all levels of sports, from professional to recreational leagues. Link