The BigDataViewer Ecosystem

My Favourite Bioimage Analysis Tools, by NEUBIAS Members

  • Fig. 1: BigDataViewer window. The overview in the top left shows the position of the displayed pixels (grey square) in the context of the whole volume (wireframe cube). Image  from Fiji’s sample data, accessible via File>Open Samples>First-instar brain (6.3MB).Fig. 1: BigDataViewer window. The overview in the top left shows the position of the displayed pixels (grey square) in the context of the whole volume (wireframe cube). Image from Fiji’s sample data, accessible via File>Open Samples>First-instar brain (6.3MB).
  • Fig. 1: BigDataViewer window. The overview in the top left shows the position of the displayed pixels (grey square) in the context of the whole volume (wireframe cube). Image  from Fiji’s sample data, accessible via File>Open Samples>First-instar brain (6.3MB).
  • Martin Jones Undergraduate MPhys degree in Physics with Electronics and Optoelectronics at Sussex University, UK (1997-2001) MSc in Evolutionary and Adaptive Systems at Sussex University, UK (2001-2002) DPhil in Experimental atomic physics at Sussex University, UK (2002-2008) Postdoctoral research in Experimental Quantum Information, University of Leeds, UK (2007-2010) Postdoctoral research in Vascular Biology Lab, Cancer Research UK (2010-2014)  Senior scientific officer, Electron microscopy unit, Cancer Research UK (2014-2016) Deputy Head of Microscopy Prototyping, Electron microscopy science technology platform, The Francis Crick Institute, UK (2016-present)
  • Fig. 2: Visualization of EM data, showing (left) a face on view of an SBF SEM benchmark HeLa cell dataset containing 8192x8192x518 voxels at 5x5x50 nm. (DOI: 10.6019/EMPIAR-10106) (middle) A zoomed-in view showing high resolution ultrastructure visualisation, note the very small field-of-view denoted by the grey square in the overview inset. (right) An oblique view where the data has been rotated approximately 45º around the y-axis to show the out-of-plane extended structure of the labelled mitochondrion (yellow arrow).
One of the defining characteristics of many types of modern microscopy is that we have well and truly entered the era of Big Data, thanks to ever-increasing spatial and temporal resolution. Once the size of a dataset gets beyond the local RAM capacity, visualizing images becomes very tricky and BigDataViewer [1,2] is an open-source Fiji plugin built to cope with this problem.
 
BigDataViewer was originally designed with selective plane illumination microscopy (SPIM) in mind, a modality which makes use of the new breed of super fast scientific CMOS cameras that acquire multi-megapixel images at rates up to 100 frames per second. SPIM allows high temporal resolution imaging of the development of embryos of a variety of model organisms in a way that causes minimal photodamage over periods extending into many hours or days.
Other developments that have vastly increased biomedical image data size include volume electron microscopy methods [3], such as serial block face SEM (SBF SEM) and focused ion beam SEM (FIB SEM). These methods work by iteratively imaging the surface of the sample and physically removing a tiny sliver, down to a few nanometres in thickness per slice. This can result in thousands of z-planes per sample, with single frames typically in the 10-100 megapixel range.
 
Handling Increased Data Footprint
The price we pay for these wonderful advances in the spatial and temporal resolution of our imaging is, of course, a massively increased data footprint. Both SPIM and volume EM routinely produce datasets in the terabyte regime. Once the size of a dataset gets beyond the available RAM on a computer, it becomes very tricky to deal with – even for simple tasks like visualization. Before we even get to analyzing the data, we often want to give it a quick sense-check, or perhaps scan quickly for interesting phenotypes or features. With a normal tiff-stack, this is rarely a trivial operation for large images. Even a modest volume can quickly overwhelm the system resources and leave the user cursing at the jerky scrolling or frequent crashes. Features like loading data as virtual stacks (i.e.

not loading every slice into RAM, but only loading the chosen slice on-demand when browsed to) allow navigation of huge datasets on limited resources, but at the cost of the file reading time delay between each slice – definitely not so useful to quickly scan through a dataset.

What BigDataViewer does is to remove the structural constraint of 3D volumes being composed of sequences of complete 2D slices and instead make use of the industry standard Big Data file format, HDF5. This allows storage in small “chunks” that can be accessed randomly from large monolithic files.
By storing the data in this way, arbitrary pixels can be read from the data without having to load the whole volume, or even a whole 2D plane, into memory. Pixels that intersect the viewing plane and are within the current field of view of the screen can be identified on-the-fly to be displayed, so that only a minimal amount of data needs to be handled to fill the display. Figure 1 shows a fluorescence image viewed in BigDataViewer. The inset at the top left shows how the viewed region (grey square) is oriented within the whole volume (wireframe cube).
On top of this more fine-grained addressing, smart caching methods make sure that data is only moved around in memory when necessary. To further improve multiscale navigation, BigDataViewer uses image pyramids, or mipmaps, storing a series of downscaled versions of the image. While this might sound like it uses a lot of memory, scaling in X and Y this way only leads to a theoretical maximum of 1.333 times the storage footprint of the raw data (even less than this if you scale in Z as well), which can often be easily clawed back with the lossless compression that can be applied via the plugin. Using image pyramids allows easy zooming in and out over enormous ranges of scales, just like Google Maps.
 
Fast and Responsive Exploration of Data
For example, finding a small structure of interest in a large 2D EM image can span a similar range of scales to the task of finding the borough of London in which I work by looking at a map of the world. The problem becomes even harder when working in 3D or 4D. Being able to zoom through different scales to identify appropriate landmarks at each scale helps enormously in BigDataViewer, just as it does in Google Maps. All of this combines to give fast and responsive exploration of the data since quick movements through the volume can be shown by rapidly grabbing the lower resolution information, only settling to the appropriate higher resolution when user interaction has slowed down or stopped.
As well as efficient visualisation of the data, the chunk-wise reading of the file also allows an image plane to be retrieved at any orientation from within the volume. This means that any oblique plane can be chosen, and the view rotated arbitrarily just as simply as zooming in or out. This is all done without changing the raw pixel values, unlike the alternative method of calculating and applying an affine transformation to the data and rewriting the whole stack in the new coordinates, which is both time-consuming and difficult to tweak with small corrections. Being able to do this with terabyte scale datasets is just mindblowing and I must admit it felt like some kind of magic when I first saw it. The pixel-serving is so slick that one of the first demonstrations I saw was Pavel Tomancak tethering his laptop to a mobile phone data connection and seamlessly viewing a terabyte dataset streamed from a server in a different country.
When viewing oblique planes of data containing non-isotropic voxels, a simple interpolation can be applied (again, only to the minimal number of pixels being presented on the screen) to smooth out any reslicing striping artifacts. Figure 2 shows a useful application of viewing non-orthogonal planes to elucidate out-of-plane extended structures.
Another application that the rapid reslicing is useful for is to perform manual “fly-through” checks of data, particularly useful for elongated structures such as blood vessels or neurons. It is often very informative to scroll through the data while tweaking the orientation of the slicing plane to follow a central axis, making it easier to spot interesting features than in the original acquisition orientation. I’d highly recommend learning the keyboard shortcuts for this!
 
Using BigDataViewer Framework for CLEM Data
One of the areas that our lab is most interested in is correlative multimodal imaging, in particular correlative light and electron microscopy (CLEM). Many CLEM experiments begin with live imaging in, say, a confocal microscope to track dynamic processes. At the timepoint of interest, the sample is fixed and prepared for electron microscopy. The sample preparation inevitably causes disruption to the sample, often in the form of differential shrinkage, and EM sample mounting is rarely in precisely the same orientation as in the fluorescence microscope. This gives a two-fold problem: first of all, there is a 6-axis alignment problem to position the two datasets into the same coordinates, so they overlap to enable correlative studies. Beyond this, the deformations from the EM preparation must be corrected to produce accurate results at small scales. This functionality is provided by BigWarp [4, 5, 6], which uses the BigDataViewer framework as an interface to manually place landmarks in the LM and EM images. By using a thin plate spline method (analogous to the bending of a sheet of metal), the LM data can by non-linearly warped onto the EM data to correct for the artifacts introduced during EM sample preparation. Since the BigDataViewer functionality means the transform can be applied without rewriting pixel values into a new rigid grid structure, sharing and improving the registration is simple as the only ingredients needed are the raw data and a small text file containing the landmarks. The landmarks can be tweaked, and the transform immediately recomputed without having to laboriously rewrite whole datasets – very useful for avoiding multiple copies of slightly differing almost-raw data during collaborations. A word of warning, however – with great power comes great responsibility! It’s important to make sure you don’t inadvertently introduce subjective bias into your correlations by incorrectly warping the two datasets together.
 
Summary
The ability to view and navigate through larger-than-RAM datasets has seeded a growing ecosystem of tools that use the BigDataViewer framework, for example BigStitcher [7], BigWarp [4,5] and Mamut [8]. It’s a very active field of development as more and more people find themselves acquiring images that are too big to handle with conventional means and I think that this type of approach will be increasingly necessary in the future. For quickly eyeballing a dataset, and flexible viewing from multiple angles, BigDataViewer and its ecosystem has proved to be extremely useful for our work with large scale data from different imaging modalities and I’d highly recommend people to give it a go for their own Big Data needs!

 

Author
Martin Jones
 

Contact
Martin Jones
Electron Microscopy STP
The Francis Crick Institute
London, UK
martin.jones@crick.ac.uk

References

[1] Pietzsch, T.; Saalfeld, S. & Preibisch, S. et al. (2015), "BigDataViewer: visualization and processing for large image data sets", Nature Methods 12(6): 481-483 DOI: 10.1038/nmeth.3392
[2] https://imagej.net/BigDataViewer [last visited on: 18.06.19]
[3] Peddie, CJ and Collinson, LM (2014) Exploring the third dimension: volume electron microscopy comes of age Micron. Jun;61:9-19. doi: 10.1016/j.micron.2014.01.009
[4] Bogovic, J. A.; Hanslovsky, P. & Wong, A. et al. (2016), "Robust registration of calcium images by learned contrast synthesis", ISBI: 1123-1126, doi:10.1109/ISBI.2016.7493463
[5] https://imagej.net/BigWarp [last visited on: 18.06.19]
[6] Russell, MRG et al. (2017) 3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy - J Cell Sci 2017 130: 278-291; doi: 10.1242/jcs.188433
[7] https://imagej.net/BigStitcher [last visited on: 18.06.19]
[8] https://imagej.net/MaMuT [last visited on: 18.06.19]

Contact

The Francis Crick Institute


Register now!

The latest information directly via newsletter.

To prevent automated spam submissions leave this field empty.