Recently, Consumer Reports reviewed pellet grills and published their Best Pellet Grills of 2020. While we respect CR, they missed the mark on this review. Consumer Reports tested 12 pellet grills as a category, along with their tests of gas grills and Komodos, as part of a larger review of the Best Grills of 2020. We want to focus, of course, on their 2020 pellet grill reviews. In summary, they unfortunately missed the mark completely, and their ratings are therefore utterly useless. We base that on the following:
Problem One: What they Rated
Consumer Reports reviewed the grills they tested across 5 categories. Their description of these categories, quoted from their review criteria, is below:
- Indirect cooking: This indicates grill temperature at the smoke or lowest setting, to represent a cooking task where the food requires slow cooking.
- Evenness performance: Evenness of heating over the grill’s surface at the highest and lowest setting using thermocouples.
- Cleaning: How easy it is to clean the ash out of the grill and to remove pellets from the hopper. Most pellet grills require using a shop vacuum to clean the ash out of the grill.
- Convenience: The convenience score is based on the main and secondary cooking areas, grate type, if it has wheels or casters, etc..
- Temperature range: Reflects how wide a temperature span the pellet grill offers by comparing temperatures at the highest and lowest settings possible.
What They Got Right
Evenness performance is a great thing to test. They tested how uniform the temperatures were across the entire grate surface, using data logging software and industrial thermocouples. We also test this, using thermocouples. They used TONS of them, which isn’t all that necessary, but this is Consumer Reports labs, and we applaud the thoroughness. This is an important thing to review and compare because it greatly affects cooking results and is comparable across the range of pellet smoker feature packages, prices, and sizes.
What They Got Wrong
Indirect cooking is a ridiculous category title to test in pellet grills. ALL pellet grills use indirect cooking. What they were instead testing was “grill temperature at the smoke or lowest setting.” We assume that the lower a grill could go, the better the rating. In reality, as long as a grill gets down to around 200, it’s going to do great at smoking food. This was a weird way to rank grills, in our opinion.
What they should have done: A better measurement would be how accurate the temperatures were, as consistent known temperatures is a key to good smoking.
Cleaning is a viable topic, but doesn’t rank in our top 10, much less as a main factor in ranking a grill. All pellet grills will require emptying ash. And not everyone needs to empty pellets from a hopper. Using this as a category skewed the rankings, as any grill with an ash cup or a pellet dump would get a boost in their scores. That happened. Let’s put it this way: you are going to eat food off your grill far more frequently than you are going to have to empty pellets (I never have to, by the way.) So why would you want a review of your grill to be 20% about cleaning convenience?
What they should have done: CR would have been better off talking about the ownership experience–finding recipes, getting customer support, warranty coverage. Cleaning is just a tiny fraction of ownership.
Convenience seems like a great ranking criteria for a grill. How easy is it to use? Except CR got this wrong. Their score was “based on the main and secondary cooking areas, grate type, if it has wheels or casters, etc.” That doesn’t seem too bad, but they used grills of all different sizes, from the flagship large Weber Smokefire EX6 to the Camp Chef Woodwind. The Weber is literally twice the size of the Camp Chef. Guess which one got dinged for cooking area? And guess which one has the beefier (and more expensive) grates?
What they should have done: CR was on the right track with convenience scores, but they rated grills wildly, comparing base models to top-of-the-line models, and medium-sized grills to large grills. Convenience should have focused on the grill’s controllers, on their wifi app, and on startup and shutdown procedures. In other words, the things that affect usability on a daily basis.
Temperature range is a strange way to rate the grills, especially the way CR scored it: “reflects how wide a temperature span the pellet grill offers by comparing temperatures at the highest and lowest settings possible.” This is odd because they put 20% of their rankings on the lowest setting with their Indirect cooking category. A large, full-featured, great performing grill that only hits 480 degrees is going to get dinged here compared to a smaller grill that can hit 575. Heck, our test of the GMG Davey Crockett had it hit 700 degrees, when it was supposed to be at 450 degrees. It would have earned high marks for this in the CR test.
What they should have done: Instead of high and low temperatures, how about temperature swings? When you set a grill to 225, does it swing between 175 and 250, or does it sit between 220 and 230. To a brisket, the difference is enormous. Come to think of it, they should have just done all temperature ratings in one category and touched on something else as their 5th category. Perhaps material quality? I would have expected Consumer Reports to care about thickness of metal, durability of wheels, stainless steel used, corrosion resistance, etc.
Problem Two: Who They Rated
| Pellet Grill Reviewed | Price | CR Rating |
|---|---|---|
| Monument Grills 87578 | $500 | 61 |
| Monument Grills 89679 | $400 | 51 |
| Oklahoma Joe’s Rider DLX 20202106 | $650 | 51 |
| Traeger Ironwood 650 TFB65BLE | $1,200 | 83 |
| Camp Chef Woodwind PG24WWSS | $900 | 76 |
| Weber SmokeFire EX6 23510001 | $1,200 | 75 |
| Z Grills Master Pellet ZPG-700D | $690 | 74 |
| Landmann Pellet Kettle 470106 | $500 | 74 |
| Broil King Pellet 500 Pro 495051 | $1,200 | 74 |
| Louisiana Grills LG900 | $900 | 74 |
| Cuisinart Woodcreek 4-in-1 Pellet Grill | $500 | 73 |
| Pit Boss 820FB | $600 | 63 |
The Grills Span Several Price Categories
Would you put a BMW alongside a Nissan? That’s what Consumer Reports did here. Unless budget is no consideration in your purchase, the rankings are useless. There are $1,200 grills (3 of them) alongside grills 1/3 their cost. This is ludicrous. Their recommended grills–the Traeger Ironwood 650 and Camp Chef Woodwind–are actually similar grills, if they had chosen the comparable Traeger Pro 575 instead. The features they rated for are very price-dependent. Just like a Nissan Sentra doesn’t quite corner like a BMW 3-Series, a $500 pellet grill isn’t going to have the same performance as a $1,200 model.
They Used a Wide Range of Pellet Grill Sizes
If you were to rate 12 cars, would you rate compact cars along with full-size sedans?Would you rate two-door cars alongside 4-door cars, and be critical of backseat space in the 2-doors? Essentially, that’s what CR has done by including grills in a wide variety of sizes, and then using cooking area as one of the review categories. That Weber is 3 times the size of the Woodwind.
They Chose Some Odd Brands
One-sixth of the reviewed grills are a Traeger/Z Grills generic clone that most people have never heard of–the two Monument Grills. And the Z Grills is basically the same grill, so that’s now 1/4th of the review group. The Cuisinart model they reviewed is a pellet/griddle/charcoal/ceramic briquette hybrid that doesn’t even belong in this category. And they included a pellet kettle by Landmann. Yeah, we had to look it up, too. Notable brands excluded from the list, which happen to be two of the three most popular brands:
- Rec Tec (now re-branded as recteq)
- Green Mountain Grills
Instead, they included Landmann, Monument Grills, and Broil King. We haven’t reviewed these brands yet, but they are obscure brands. Were we to review them, one of our biggest concerns would be the service and support and, let’s be honest here, whether they will even still exist in 5 years. To go back to the car analogy, CR decided to test some cars. They included only Ford, Mitsubishi, Saab, Peugeot, and Suzuki and then ranked the best among them as their overall best car recommendation.
They included only Ford, Mitsubishi, Saab, Peugeot, & Suzuki and then ranked the best among them as the recommendation for ‘Best Car.’
Missed Opportunity
We respect CR Labs. They have a unique ability to offer valuable advice about safety, value, and consumer satisfaction because of their superior equipment, staff, and vast experience. Those traits would have been welcome in a review of pellet grills. We couldn’t hold a candle to the weight that a Consumer Reports review normally carries. And we applaud their appearance among pellet grill reviews, as it shows just how high they’ve risen in general consumer awareness. In this case, however, we are disappointed. They seem to have chosen a random group of pellet grills, and applied a random set of criteria on which to judge them. We sort of hope they never figure out where they went wrong reviewing pellet grills. But we still value their car, dishwasher, and baby stroller reviews.
