CHEF-KOCH's Microblog ✨

Against bias in Hardware Benchmark Reviews

Bias in Benchmark Reviews

There are multiple problems with lots of popular Benchmark reviews and people who are doing or posting them.

  • They get paid or directly their products from the corpo they review. Sponsoring, advertisement, marketing, ...
  • There is an NDA that needs to be signed.
  • There is a code of conduct which disallows saying X.
  • There is a code of conduct disallowing them to post things on platform X.

Results can be misinterpreted

Does it apply in the real-world or is it only a synthetic Benchmark. Most people do not give much about theoretical scores.

What should a Benchmark review minimum include

I make it quick and list specific categories that needs to be included to get the whole picture.

  • Objective title
  • Introduction
  • Architecture analysis, are there new changes compared to the old one
  • Core analysis
  • Performance
  • Efficiency
  • Performance per Watt analysts
  • OS power profiles and the influence
  • Synthetic benchmarks
  • Real-world in-game benchmarks
  • Application performance with real-world apps
  • Compression and decompression
  • Turbo, boost and maximum clock
  • RAM configuration as well as impact of RAM
  • OS tests across different platforms, Windows and Linux
  • Bottleneck benchmarks
  • Security changes and the impact on performance
  • Comparison to previous generation
  • Conclusion if it is worth to update when you posses generation x on hardware already
  • Overclocking aspect
  • Usability like drivers Drivers
  • Maintenance cost as well as platform switching in consideration
  • Price
  • Conclusion

Draft

This is currently a draft I am working on and not the final version, please do not link it.

#benchmark #bias

- 2 toasts