Since 1990s first methods for automated segmentation of brain tumors have been published, but all of them were validated on small, private datasets, until 2012, when MICCAI organized a Multimodal Brain Tumor Image Segmentation Challenge (BraTS)1 and releasing publicly available dataset, consisting 4 modalities of MRI images: T1, T1 with contrast (gadolinium), T2 and FLAIR with manually segmented tumors. Recent improvements in neural networks training performance by taking advantage of high-performance CPUs, GPUs and ReLU activation function have led to the usage of convolutional neural networks for semantic segmentation.
The aim of the work was to implement, train and evaluate the quality of automated multi-label brain tumor segmentation technique for Magnetic Resonance Imaging based on Tiramisu deep learning architecture, which would allow in the future medical professionals to effortlessly and quickly create precise tumor segmentation needed for both diagnosis and treatment of patients.
The provided method should allow segmentation of all 4 labels of tumor labels (Whole Tumor, Enhancing Tumor, Tumor Core) at once without a need to run a single model for each class and allow interference on Intel processors.
The examples of successfully used architectures are 2015 U-Net2 and 2016 100-layer Tiramisu DenseNet3. Especially, Tiramisu has shown great performance on semantic segmentation of urban scene benchmarks. It contains up-paths and up-paths, but also Dense blocks with skip-paths include Concatenation of feature maps from the output of Convolutional layer along with its input. Each hidden layer uses ReLU activation and Batch Normalization and L2 regularization. Different variants of Tiramisu consist of a various number of layers per Dense block (usually 4 ÷ 5, up to 12) and growth factor which defines how a number of feature layers growths after each Concatenation (usually k = 12 ÷ 15). The rise of these parameters causes a rapid increase of trainable parameters.
Figure. 1 Tiramisu Densenet Scheme3.
Hardware and software used in the project
The neural network has been trained with Intel® Xeon® processor server with Nvidia® gtx1080 GPU and 48GB of ram and Anaconda Python® 3.6 distribution. Following libraries has been used:
- Scipy - 3D matrix image operations,
- Numpy® - matrix operations,
- TensorFlow® 1.4 and Keras - Neural Network implementation and training,
- Nibabel - NIFTI files IO,
- h5py - a saving of models and datasets.
We have used a variant of Tiramisu Densenet with k=8 and number of layers per Dense Block = [4,5,7,10,12,15] with 9.3 mln of trainable parameters. Training dataset has been prepared by cropping each slice to 192x160 pixels, to eliminate redundant background reducing the amount of needed memory and standardized by Z-Score. The network has been trained with categorical cross-entropy loss and RMSprop optimizer with learning rate 1e-3. Training on Nvidia GTX1080 took approximately 2 days.
Table 1. Model layers description.
|no||Building block||Connected with||Output shape|
|2||DB (4 layers) + TD||96x80x80|
|3||DB (5 layers) + TD||48x40x120|
|4||DB (7 layers) + TD||24x20x176|
|5||DB (10 layers) + TD||12x10x256|
|6||DB (12 layers) + TD||6x5x352|
|7||DB (15 layers)||6x5x472|
|8||TU + DB (12 layers)||6||12x10x568|
|9||TU + DB (10 layers)||5||24x20x432|
|10||TU + DB (7 layers)||4||48x40x412|
|11||TU + DB (5 layers)||3||96x80x308|
|12||TU + DB (4 layers)||2||192x160x252|
Example segmentation has been prepared for to compare with target (expert's) segmentation.
Figure. 2 Segmentation created with the proposed model.
The proposed model has been also tested on whole BraTS validation dataset (with organizer's evaluation website) both before and after additional postprocessing procedure. It is based on morphological closing, to remove small voxels, that don't belong to biggest object - main tumor region.
Presented boxplots are showing the distribution of dice and sensitivity metrics for each sample of validation dataset both before and after postprocessing.
Figure. 3 Boxplot of validation set segmentation evaluation.
Table. 2 Method's mean performance on validation (blind) dataset.
The proposed method allows for single-model multi-label glioma tumor segmentation with great results. Comparing with U-Net segmentation Dice score is 1% higher and also other tumor tissue segmentations have been created.
Segmentation time has been measured with Intel Xeon processors using the Intel® AI DevCloud. Mean time of segmentation pipeline consisting of: data loading, preprocessing, segmentation, postprocessing and saving NIFTI segmentation is 53.76 secs per patient.
1. Menze BH., et al, The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS), IEEE Transactions on Medical Imaging 34(10), 1993-2024 (2015) DOI: 10.1109/TMI.2014.2377694
2. Ronneberger O., Fischer P., Brox T., U-Net: Convolutional Networks for Biomedical Image Segmentation, Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS, Vol.9351: 234-241, 2015
3. Jégou S., Drozdzal M., Vazquez D., Romero A., Bengio Y., The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation, IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017