SPEC is running certain versions of Intel compilers due to complaints that apparently targeted optimizations for certain workloads (via ServeTheHome and Phoronix) are essentially cheating. It has stated that it will no longer publish SPEC CPU 2017 results for Intel CPUs. A note was added to over 2,600 benchmark results published for the compiler in question, effectively invalidating results primarily from machines running 4th generation Xeon Sapphire Rapids CPUs.
SPEC CPU 2017 is a benchmark primarily used in high-end servers, data centers, and workstations/PCs that tests the performance of different workloads in a standardized way so that different computers can be compared to each other. Masu. The superior performance of SPEC CPU 2017 depends not only on hardware but also on software. One of the key elements in software-side optimization is the compiler. A compiler is basically a program that takes written code and reformats it in a way that the processor can perform optimally.
A disclaimer attached to the 2,600+ SPEC CPU 2017 results says, “The compiler used for this result uses prior knowledge to specifically improve performance on the 523.xalancbmk_r / 623.xalancbmk_s benchmarks. Compilation was in progress.” This means that the compiler (in this case Intel's oneAPI DPC++/C++ compiler) was not optimized for the workload type of the two SPEC CPU 2017 benchmarks tested in question, but specifically This means that the two benchmarks themselves are not optimized.
Higher performance is clearly better, so it is expected that the compiler should optimize, but benchmark-specific optimizations are controversial and frowned upon. SPEC wants its benchmarks to reflect the actual performance of the hardware and provide a standardized way to compare different processors. But if a compiler optimization only improves performance for a particular benchmark, but not in real-world scenarios, it clearly does not reflect the real world, and only for that particular benchmark.
According to Phoronix, optimizations can improve SPECint's overall performance by 9%. The publication also states that Intel oneAPI compiler versions 2022.0 through 2023.0 are affected. This means that most of the currently disabled results were run in his 2022 primarily on Sapphire Rapids CPUs. The results for the 5th generation Xeon Emerald Rapids CPUs indicate that Emerald Rapids appeared after a good version of the compiler was available, so it is unlikely that you are running a version of the compiler with prohibited optimizations. Very low.
Benchmark-specific optimization has been a hot topic for many years. Back in 2003, Nvidia was accused of performing driver-side optimizations in his 3DMark 2003 to improve the performance of his GPU. In 2010 he claimed that Nvidia itself was cheating by AMD not enabling certain settings on the driver side that had a significant impact on actual games. Improved visual quality at the expense of performance. Although recent accusations have not been as intense, SPEC has certainly humiliated Intel in this case.
