Victor Eijkhout: I see several problems with the state of parallel programming. For starters, we have too many different programming models, such as threading, message passing, and SIMD or SIMT ...

This is a schematic showing data parallelism vs. model parallelism, as they relate to neural network training. Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases ...