Deep Learning and Cloud Platforms are transforming the field of Machine Learning from theory to practice. However, implementation differences across frameworks and inference engines make the comparison of benchmark results difficult. SPEC and TPCC benchmarks are not accurate due to the complex interactions between implementation choices such as batch size, hyperparameters, or numerical precision. To address this complexity requires systematic benchmarking that is both representative of real-world use cases and valid across different software/hardware platforms.
This talk will present the best Machine Learning benchmarking tools to use with OpenStack and Kubernetes. We will show how MLPerf and Thoth help data scientists to improve their system performance and fully benefit from their CPUs, GPUs, or FPGAs. We will share insights and lessons learned over the journey of key Machine Learning training and inference use cases selection.
We will share insights and lessons learned over the journey of the selection of key machine learning training and inference workloads representative of important production use cases for CPUs, GPUs and FPGAs
