Hang Qi, Evan R. Sparks, and Ameet Talwalkar.
Paleo: A Performance Model for Deep Neural Networks.
1 The current live demo only supports data parallelism and a predefined set of models and devices. Features including customization and model parallelism will be available in later releases.
2 The current live demo does not deal with for multiple GPUs on the same host. More flexibilities will be added in later releases.
3 Cost is calculated based on $0.9/hour per GPU.
Speedup in throughput (images/sec) relative to one worker.
Total time of forward pass, backward pass, and weights update for the given epochs.
Cost for running a fixed number of epochs. Only support AWS EC2 P2 instances for now.3