See the previous 1.2 changelog.
With this iteration of the vacuum test bench, we've completely overhauled our floor performance tests to more clearly illustrate debris pickup performance. We've listed the changes made to our test methodology below.
Test Group | Change |
Hard Floor Pick-Up |
|
High-Pile Carpet Pick-Up |
|
Low-Pile Carpet Pick-Up |
|
Pet-Hair Pick-Up |
|
Here, we'll examine these new changes and how they can help you find the right vacuum for your needs.
Our previous method of testing performance on hard floors was done by measuring separate amounts of cereal and rice; we'd only test performance with one debris type at a time. After making three back-and-forth passes across the testing surface, we'd measure the difference in weight between the test surface before and after cleaning. This relatively constrained testing scenario had little to do with real-world vacuum usage, where you're far more likely to encounter a mix of debris types in larger amounts. The relatively small amount of debris also overly exaggerated differences between similarly-performing vacuums.
For our new testing methodology, we measure out a larger quantity of debris, comprised of cereal, sand, and rice, and spread it onto the testing surface as a mix. Two back-and-forth passes are then made. Unlike the previous testing method, we keep the vacuum head on the ground instead of lifting it. This allows us to make a closer assessment of vacuum head performance, rather than motor performance, for which we already have airflow and suction performances; several vacuums, such as the Miele Complete C3, come with heads that have inconveniently placed bristles that push around larger amounts of debris, instead of allowing the vacuum to pass over it. Instead of determining the vacuum's performance score based on the delta in weight between its empty and filled dirt compartment, we've opted to look at the surface before and after it's been cleaned and assign a subjectively determined score.
We've also eliminated the 'Performance Over Time' aspect of the performance test; in the vast majority of cases, we noticed little in the way of performance hit as a vacuum's dirt compartment or dirtbag was filled with debris, so we deemed this aspect of the test unnecessary.
Instead of including a bar graph showing an estimated percentage of debris picked up, we've replaced that with a video of the performance test being conducted and a photo of the surface after cleaning it. This new, more robust testing methodology, combined with the clearer, more intuitive manner of displaying performance results, is intended to make it easier to determine what vacuum best suits your needs.
Robot vacuums are tested slightly differently. We set up the test rig as usual, but the vacuum is given as much time as it needs to clean the testing area until it stops. This test is conducted in this way to evaluate raw debris pickup performance independently from a vacuum's ability to provide adequate area coverage.
Our pet hair pickup performance testing methodology has also been similarly redesigned to provide a clearer, more real-world relevant example of performance. We now test for pet hair pickup on low-pile carpeting, including a video of the surface being cleaned and a photo of the surface after we've made the requisite single pass.
Previous test bench iterations allowed for multiple passes over the same area. However, since most people in day-to-day use will want to spend a lot of time cleaning the same small area repeatedly, we've chosen to evaluate single-pass performance.
As always, our test methodology is a work in progress, and we depend on your feedback to improve it. Our methodology can always benefit from more meaningful tests. If you have comments, questions, or suggestions about this or any future updates, reach out to us in the forums.
We have retested popular models. The test results for the following models have been converted to the new testing methodology. However, the text might be inconsistent with the new results.