Brain Tumor Segmentation using Fully Convolutional Tiramisu Deep Learning Architecture

Introduction

Since 1990s first methods for automated segmentation of brain tumors have been published, but all of them were validated on small, private datasets, until 2012, when MICCAI organized a Multimodal Brain Tumor Image Segmentation Challenge (BraTS)1 and releasing publicly available dataset, consisting 4 modalities of MRI images: T1, T1 with contrast (gadolinium), T2 and FLAIR with manually segmented tumors. Recent improvements in neural networks training performance by taking advantage of high-performance CPUs, GPUs and ReLU activation function have led to the usage of convolutional neural networks for semantic segmentation.

BRATS logo

Problem statement

The aim of the work was to implement, train and evaluate the quality of automated multi-label brain tumor segmentation technique for Magnetic Resonance Imaging based on Tiramisu deep learning architecture, which would allow in the future medical professionals to effortlessly and quickly create precise tumor segmentation needed for both diagnosis and treatment of patients.

The provided method should allow segmentation of all 4 labels of tumor labels (Whole Tumor, Enhancing Tumor, Tumor Core) at once without a need to run a single model for each class and allow interference on Intel processors.

Background

The examples of successfully used architectures are 2015 U-Net2 and 2016 100-layer Tiramisu DenseNet3. Especially, Tiramisu has shown great performance on semantic segmentation of urban scene benchmarks. It contains up-paths and up-paths, but also Dense blocks with skip-paths include Concatenation of feature maps from the output of Convolutional layer along with its input. Each hidden layer uses ReLU activation and Batch Normalization and L2 regularization. Different variants of Tiramisu consist of a various number of layers per Dense block (usually 4 ÷ 5, up to 12) and growth factor which defines how a number of feature layers growths after each Concatenation (usually k = 12 ÷ 15). The rise of these parameters causes a rapid increase of trainable parameters.

Tiramisu DenseNet scheme[3]

Figure. 1 Tiramisu Densenet Scheme3.

Hardware and software used in the project

The neural network has been trained with Intel® Xeon® processor server with Nvidia® gtx1080 GPU and 48GB of ram and Anaconda Python® 3.6 distribution. Following libraries has been used:

  • Scipy - 3D matrix image operations,
  • Numpy® - matrix operations,
  • TensorFlow® 1.4 and Keras - Neural Network implementation and training,
  • Nibabel - NIFTI files IO,
  • h5py - a saving of models and datasets.

Implementation

We have used a variant of Tiramisu Densenet with k=8 and number of layers per Dense Block = [4,5,7,10,12,15] with 9.3 mln of trainable parameters. Training dataset has been prepared by cropping each slice to 192x160 pixels, to eliminate redundant background reducing the amount of needed memory and standardized by Z-Score. The network has been trained with categorical cross-entropy loss and RMSprop optimizer with learning rate 1e-3. Training on Nvidia GTX1080 took approximately 2 days.

Table 1. Model layers description.

Model layer description
noBuilding blockConnected withOutput shape
 Input layer 192x160x4
1Conv2D (3x3) 192x160x48
2DB (4 layers) + TD 96x80x80
3DB (5 layers) + TD 48x40x120
4DB (7 layers) + TD 24x20x176
5DB (10 layers) + TD 12x10x256
6DB (12 layers) + TD 6x5x352
7DB (15 layers) 6x5x472
8TU + DB (12 layers)612x10x568
9TU + DB (10 layers)524x20x432
10TU + DB (7 layers)448x40x412
11TU + DB (5 layers)396x80x308
12TU + DB (4 layers)2192x160x252
13Conv2D (1x1) 194x160x4(prob)

Results

Example segmentation has been prepared for to compare with target (expert's) segmentation.

Figure. 2 Segmentation created with the proposed model.

The proposed model has been also tested on whole BraTS validation dataset (with organizer's evaluation website) both before and after additional postprocessing procedure. It is based on morphological closing, to remove small voxels, that don't belong to biggest object - main tumor region.

Presented boxplots are showing the distribution of dice and sensitivity metrics for each sample of validation dataset both before and after postprocessing.

Figure. 3 Boxplot of validation set segmentation evaluation.

Table. 2 Method's mean performance on validation (blind) dataset.

 ETWTTC
Dice0.660.880.67
Sensitivity0.780.890.69

Summary

The proposed method allows for single-model multi-label glioma tumor segmentation with great results. Comparing with U-Net segmentation Dice score is 1% higher and also other tumor tissue segmentations have been created.

Segmentation time has been measured with Intel Xeon processors using the Intel® AI DevCloud. Mean time of segmentation pipeline consisting of: data loading, preprocessing, segmentation, postprocessing and saving NIFTI segmentation is 53.76 secs per patient.

References:

1. Menze BH., et al,  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS), IEEE Transactions on Medical Imaging 34(10), 1993-2024 (2015) DOI: 10.1109/TMI.2014.2377694

2. Ronneberger O., Fischer P., Brox T., U-Net: Convolutional Networks for Biomedical Image Segmentation, Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS, Vol.9351: 234-241, 2015

3. Jégou S., Drozdzal M., Vazquez D., Romero A., Bengio Y., The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation, IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017

有关编译器优化的更完整信息,请参阅优化通知