Designing GANs for 3D Data Synthesis

3D-GANWu, Jiajun, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. "Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling." Advances in neural information processing systems 29 (2016). was designed to generate a 3D point cloud of certain types of objects. The design and training process of 3D-GAN is very similar to the vanilla GAN, except that the input and output tensors of the 3D-GAN are five-dimensional rather than four-dimensional.

Generators and discriminators in 3D-GAN

The architecture of the generator network of 3D-GAN is as follows:

The generator network consists of five transposed convolution layers (nn.ConvTranspose3d), in which the first four layers are followed by the batch normalization layer (nn.BatchNorm3d) and ReLU activation function, and the last layer is followed by a sigmoid activation function. The kernel size, stride size, and padding size are set to 4, 2, and 1 in all the transposed convolution layers, respectively. Here, the input latent vector can be gradually expanded to a $1 \times 32 \times 32 \times 32$ cube, which can be considered as a 1-channel 3D image. In this 3D image, the pixel value is actually the possibility of whether a point exists at these $32 \times 32\times 32$ grid locations. Normally, we reserve all the ...