Sony has paid attention to the network topology.
Hong Kong scientists have previously reported that were able to reduce the training time of the neural network ResNet-50 to 6.6 minutes. Their record did not last long – the Japanese index has improved to 224 seconds.
For researchers and engineers, the training time of the neural network is an important metric, especially in the case of large-scale projects. Possibilities of optimization of this process are looking for teams of experts in different parts of the world. In Sony found a way that makes learning by 43.4% more efficient, says Synced.
In the experiment, the researchers worked with the algorithm of ResNet-50, which was studied image recognition at ImageNet database. To improve efficiency, the Japanese engineers have optimized the data transfer by combining small packets (so did from colleagues in Hong Kong). In addition, Sony drew attention to the network topology.
They created a grid 2D-Torus (two-dimensional torus) consisting of horizontal and vertical networks of the type "ring". The work of the network topology significantly increase the speed of learning.
First, the researchers used 2176 Tesla GPUs V100 for ResNet training-50 and achieved accuracy in 75.03%. Then thanks to the scalability of the resulting system, they were able to increase the accuracy of 91,62% when using 918 Tesla processors V100.