I originally built my deep learning rig in mid-2018 while I was in the middle of making a career change to data science and machine/deep learning. I built the rig to teach myself the basics and run small deep learning projects at night and on the weekends. When I spec’d the machine, I made sure to have the flexibility for adding a second GPU in the future. I recently decided to add the second GPU to enable the training of larger models with realistic training times. Since the current GPU market makes getting a brand-new model at a reasonable price nearly impossible, I opted to buy a used GPU.
My current GPU is an ASUS ROG Strix 1080Ti OC, so I decided to try to find a used one so that they would match. While this decision was mostly driven by aesthetics, I’ve also really enjoyed my current GPU and haven’t had any issues with it. Additionally, for 32-bit floating point applications, two 1080Ti GPUs have comparable performance to a V100 32GB GPU at a fraction of the cost. I was able to find a used 1080Ti that matched my current GPU on Amazon through their “Renewed” program which provides certain guarantees and a pretty generous return policy. I ended up having to return the first GPU I received but the second unit I got seemed to not have any issues. One thing I wanted to test is if there is truly any performance degradation on used GPUs. Since I don’t know the history of the used GPU I purchased, there is a strong possibility that it was used for crypto mining. To test the performance of the used GPU, I ran the 32-bit floating point PyTorch deep learning benchmarks maintained by Lambda Labs.
With my current setup, I can run a few different comparisons. First, I run the benchmark on my original GPU which I would describe as “like-new” since it has had fairly light use and been used exclusively for deep learning. I can then run the benchmarks on the “renewed” GPU as well as running both GPUs at the same time. I care mostly about throughput so that is all I report here. To normalize the benchmarks, we use the benchmark results presented by Lambda Labs for the V100 32GB. The results of the benchmarks are summarized in the charts below, including the benchmark results provided by Lambda Labs for the 1080Ti.
It can be seen from the results above that the “renewed” card performs pretty much identically to my current “like-new” card. Additionally, using the two cards in parallel for distributed training achieves about 84% of the performance of a V100 32GB at a fraction of the cost. One important note is that not all of the benchmarks ran successfully, resulting in a throughput of 0. The failed benchmarks are not included in the average metric. Not all of the benchmarks worked immediately and required some tweaks to the configuration files (the remaining ones I was unable to get running). The differences between my 1080Ti results and those reported by Lambda Labs may be attributed to the differences in the 1080Ti model, the rest of the system configuration, or changes to the underlying benchmarks since they have been run. The difference in system specs are shown below.
Lambda Labs Configuration:
CPU: Intel(R) Core(TM) i9-7920X CPU @ 2.90GHz
CPU Memory: 62G
GPU: GeForce_GTX_1080_Ti
GPU Memory: 11175_MiB
NVIDIA driver: 440.44
CUDA Version: V10.2.89
CUDNN Version: 7.6.5
Motherboard: ASUSTeK COMPUTER INC. WS X299 PRO_SE Rev 1.xx
OS: Ubuntu 18.04.3 LTS
PyTorch Version: 1.4.0a0+a5b4d78
My Configuration:
CPU: Intel(R) Core(TM) i5-7600K CPU @ 3.80GHz
CPU Memory: 31G
GPU: NVIDIA_GeForce_GTX_1080_Ti
GPU Memory: 11178_MiB
NVIDIA driver: 495.44
CUDA Version: V11.1.74
CUDNN Version: ..
Motherboard: ASUSTeK COMPUTER INC. PRIME Z270-A Rev 1.xx
OS: Ubuntu 18.04.5 LTS
PyTorch Version: 1.7.0a0+7036e91
In summary, I have had a pretty good experience so far with purchasing a used GPU. The GPU is functional and does not seem to have any performance degradation. There is always some risk buying used and I am happy I chose to use Amazon since I did end up having to return the first unit; the return process went very smoothly. Happy computing!