Test Bench 1.4  
Changelog

 0
Updated 

See the previous 1.3 changelog.

What's Changed?

Vacuums can have a significant impact on air quality: they can blow dust out of their exhaust, scattering dust all over the room you're trying to clean, and cause you to breathe it in. Later, when you're emptying a vacuum, you can also create a dust cloud. Now, we're measuring the dust. This test bench features our new air quality tests, which measure air quality while the vacuum is running and during emptying. Since we can now measure these impacts, we incorporated these results into new ratings and performance usages and revised existing ones. We also consolidated, improved, and simplified existing performance usages.

We'll go into more detail about the main reason for this test bench update, our new Air Quality tests.

New Tests

Particle Filtration

We use a particle sensor mounted at head height to measure the dust concentration released by a vacuum during operation. This is dust you could breathe in when you're vacuuming, and which will settle down all over the floors you thought you just vacuumed. To make sure we're comparing vacuums on an equal basis, we vacuum up the same weight of standardized filter test powder for each vacuum we test. This test powder has a very consistent mix of particle sizes. Since this powder is hazardous when inhaled due to the small particle sizes, we conduct these tests in a sealed acrylic test chamber, and our testers wear respirators for maximum protection.

A large transparent test chamber, nearly the height of a person. The person operates the vacuum inside the chamber using arm-length gloves built into the chamber.
Early testing of the test chamber with the Miele Complete C3 Cat & Dog before the addition of a respirator.

You can see the particle concentration live in a small display at the top of the chamber on the right. We have one of our best air purifiers in the chamber to clean the powder out of the chamber air before and after a test.

We also record whether the vacuum has a HEPA-rated filter, as before.

Air Quality During Emptying

If you empty a bagless vacuum, you've noticed that you release a cloud of dust when you empty it. You might take steps to mitigate this dust by emptying the vacuum while it's inside a garbage bag or emptying it outdoors. Are bagged vacuums better? Now we show you how much dust you'll release and how hard it is to keep it at bay, even if you follow the manufacturer's recommendations. This test consists of a video we take while we empty the vacuum as the manufacturer recommends, or as you normally would. We empty the vacuum after the Particle Filtration test.

Emptying the Dyson V15 Detect.

We capture this video against a dark background with a bright light underneath the vacuum. You can see every dust particle.

We decided not to measure particle concentrations in this test because the visuals make the results very clear. Large visible dust clouds indicate a failure of this test, while only a few dust particles indicate good performance. Early testing with a sensor made it clear that there are no "in between" results of this test.

New & Revised Usages

We introduced several new usages, revised others, and consolidated some to better reflect how you use a vacuum.

Before After
Usage scores showing Bare Floor, Low-Pile Carpet, High-Pile Carpet, Pets, Stairs, Cars, and Workshop usages.
The previous usage scores.
Usage scores showing House, Small Apartment, Stairs, Cars, Bare Floor Pickup, Carpet Pickup, Pet Hair Pickup, and Air Quality usages.
The usage scores for Test Bench 1.4.

House

There's nothing worse than having to stop your cleaning session partway through because your vacuum battery died. That's why a vacuum's Runtime is especially important for our House verdict. Other key considerations include pickup performance across a variety of surfaces and Air Quality during operation and while emptying.

Scoring box reading 35% Runtime, 15% Bare Floor Pickup, 15% Pet Hair Pickup, 15% Build Quality, 7.5% Air Quality, 7.5% Carpet Pickup and 5% Vacuum Type.
The score components for the new House usage for Test Bench 1.4.

Small Apartment

In our new Small Apartment usage score, we emphasize our new Air Quality tests, because a vacuum that scores poorly for air quality will have a greater impact in a smaller space. It's also important that you can store this vacuum in a compact space, and we take pickup tests on surfaces you'll typically encounter in a small apartment into consideration as well.

Scoring box reading 30% Air Quality, 28% Storing, 15% Pet Hair Pickup, 11% Vacuum Type, 8% Carpet Pickup and 8% Bare Floor Pickup.
The score components for the new Small Apartment usage for Test Bench 1.4.

Stairs

In previous Test Benches, the Stairs usage was a complicated mix of scores. We simplified and revised this usage to emphasize Weight, since lugging a heavy vacuum up and down stairs will be very unpleasant, and a light vacuum is much better suited to this usage. We also incorporate pickup tests on the surfaces you're likely to encounter on stair treads, Air Quality tests, since some stairwells can be quite confined, as well as Range, because you need enough cord to reach the nearest outlet in tall staircases.

Before After
Complicated score components box, incorporating 12 tests.
The previous Stairs score components.
Scoring box reading 40% Total Weight, 20% Bare Floor Pickup, 15% Range, 10% Air Quality, 10% Pet Hair Pickup, and 5% Carpet Pickup.
The new Stairs score components.

Cars

We revised the score components for this usage, incorporating our new consolidated Carpet Pickup usage, which now carries a higher weight in the scoring. We place the greatest emphasis on Portability because it's the most important factor in cleaning a car. For some vacuum types, like upright vacuums, we don't assign a score at all because they aren't suitable for cleaning your car.

Before After
Complicated score components box, incorporating 13 tests.
The previous Cars score components.
Score box reading 33.8% Portability, 17.4% Carpet Pickup, 16.4% Suction, 8.9% Pet Hair Furniture Performance, 5.4% Range, 5% Build Quality, 4% Storing, 3.1% User Maintenance, 2% Turbo Brush, 2% Crevice Tool, and 2% Extension Wand.
The new Cars score components.

Bare Floor Pickup

We introduced a very simple usage that scores a vacuum on how well it cleans bare floors. This usage includes only the Hard Floor Pickup test and the Cracks test, since these are the two tasks you have to tackle when vacuuming bare floors.

Score box reading 70% Hard Floor Pickup and 30% Cracks.
The score components for the new Bare Floor Pickup usage for Test Bench 1.4.

Carpet Pickup

This is another new, simple usage that incorporates the test scores on the two types of carpet we test with the High-Pile Carpet Pickup test and the Low-Pile Carpet Pickup test. This score shows how a vacuum would perform on these typical carpet types.

Score box reading 50% High-Pile Carpet Pickup and 50% Low-Pile Carpet Pickup.
The score components for the new Carpet Pickup usage for Test Bench 1.4.

Pet Hair Pickup

This new usage is for pet owners. It shows how well a vacuum picks up pet hair from low-pile carpet using its floorhead (the Pet Hair Pickup test) and furniture using its turbo brush or upholstery tool (the Pet Hair Furniture Performance test).

Score box reading 65% Pet Hair Pickup and 35% Pet Hair Furniture Performance.
The score components for the new Pet Hair Pickup usage for Test Bench 1.4.

Air Quality

This is the new usage score showing the results of our new Air Quality tests. It reflects the vacuum's impact on the air quality around you, what you breathe in, and whether it'll deposit dust throughout your home. This score incorporates not only air quality during vacuuming but also during emptying, since it includes both the new Particle Filtration and Air Quality During Emptying tests.

Score box reading 70% Particle Filtration and 30% Air Quality During Emptying.
The score components for the new Air Quality usage for Test Bench 1.4.

Deleted or Replaced Usages and Tests

We also deleted or revised old ratings, performance usages, or tests to improve readability, clarity, and conciseness.

Deleted Rating / Performance Usage / Test Reason
Bare Floor Replaced by Bare Floor Pickup performance usage
Low-Pile Carpet Replaced by Carpet Pickup performance usage
High-Pile Carpet Replaced by Carpet Pickup performance usage
Pets Replaced by Pet Hair Pickup performance usage
Workshop Deleted since most household vacuums can't be used in a workshop
Tools and Brushes Covered elsewhere in the review
Alternative Configuration Covered elsewhere in the review
Air Quality Replaced by the much more detailed new Air Quality test groups

Let Us Know What You Think!

As always, our test methodology is a work in progress, and we depend on your feedback to improve it. Our methodology can always benefit from more meaningful tests. If you have comments, questions, or suggestions about this or any future updates, reach out to us in the comments.

15 Vacuums Updated So Far

We are retesting popular models first. So far, the test results for the following models have been converted to the new testing methodology. However, the text might be inconsistent with the new results.

25 Vacuums Planned To Be Updated

We are also planning to retest the following products over the course of the next few weeks: