Experiences on generating synthetic medical data with GAN models

Medical data is privacy-sensitive and protected by national legislation and GDPR making data sharing between hospitals and research organizations difficult. In addition, the amount of data for a specific medical condition and imaging modality can be relatively small. Being able to generate synthetic medical data via AI in large quantities would thus be very valuable. GAN algorithms perform well in 2D, e.g. in generating human facial images, but there are very only few publications on 3D case. In this presentation, we will discuss characteristic GAN challenges to 3D medical imaging, including large memory footprint, training with limited data and mode collapse due to low variation. Furthermore, we apply the state-of-the-art methods to a publicly available magnetic resonance (MRI) dataset consisting of brain images from 1 112 healthy human subjects.

First, we employed ProgressiveGAN3D, an open-source toolkit, which implements NVIDIA's progressive growing of GANs algorithm in 3D. Although the algorithm seemingly worked well and produced visually good results, in closer inspection we noticed that algorithm suffers from mode collapse i.e. the variance in the generated images is very low. More recently, an updated version of the progressive GAN algorithm was published as part of the Nobrainer framework. Applying this, we found that the synthetic MRIs exhibit larger variation and the original mode collapse issues were at least partially resolved. We also studied the effect of augmentation on the generated results. A medical expert reviewed the generated synthetic MRIs and saw them anatomically correct. However, some details in the synthetic MRI quality allowed medical expert to distinguish them from real MRIs.

We plan to expand the approach by including another channel, such as CT volume or segmentation of target of interest, to the synthetic MRIs. Also the volume quality issues need to be solved as well as performance with smaller datasets e.g. with hundreds of subjects.