Computers are smarter than people

Dr. Giganto

Sports Moderator

So, I remember reading a bunch of stuff early in theyear about how baseball prospectus forecasting program, PECOTA, predicted the White Sox to win 73 games this year. A bunch of people flipped out saying "WHAT??? They won the WS two years ago! No way they finish the year 8 games under .500!!!" Well, sure enough, the W. Sox are having an awful year. Which got me thinking...How did the prgram do for all the other teams, this is what I found:

Amazingly, the computer picked 10 teams (1/3rd of the league) to within 1 game of their current win percentage. It nailed the Orioles on the head, and the Phillies, White Sox, Blue Jays, A's, Reds, Padres, Cubs, Indians, and Cardinals were one game off (through 125-128 games)

The computer was within two games of the Brewers, Braves, Yankees, and Tigers

Within 3 games of the Diamondbacks and Rockies

Within 4 games of the Dodgers

Within 5 games of the Red Sox

So, for 18 of the 30 teams, it was within 5 games. That's pretty impressive.

The computer missed by 10 games or more for only 3 teams: The Twins (10 games worse), Devil Rays (16 games worse), and Mariners (20 games better)

Some of the amazing things it predicted correctly:

The Diamondbacks leading the NL West

The White Sox fall to below .500

The Cardinals being a roughly .500 team

The Cubs and Brewers being the cream of a mediocre crop in the NL Central

The A's not being able to squeeze another second half surge this year (take that everyone who thinks Billy Beane invented computers to make himself look good!)

I gotta say, despite being totally wrong on a few teams like the Mariners, and Twins, this was pretty impressive, on the whole.

I'm trying to find a human-generated set of predictions to compare it to, but I keep finding things that just give predicted division winners, not total win-loss records. I'll keep looking though...

Felix

Maintenance Worker

That is pretty impressive, however... how long has this program been in use / tracking statistics to come up with these figures? Is this the first year of it's use, and how much data had to be put into the computer for it to come to these conclusions.

Either way though, your statement about computers being smarter than people would be incorrect. A person had to create the program, and it's calculations, for it to give a answer. The computer in fact may have been able to do it in a matter of a few minutes, but none the less, a human programed it to think that way... A thought to ponder ;)

Suntan Superman

WC's Resident Samoan

Still amazing that it was THAT accurate

Dr. Giganto

Sports Moderator

The system was developed in 2003, but the statistics it uses go back decades.

Basically, the methodology they use is this:

There have been thousands upon thousands of baseball players in the 100+ years baseball has been around. Players who exhibit certain characteristics tend to put up similar numbers, or follow similar career paths.

And, based on a player's past performance, you can kinda figure out what their career path is going to be (you can even do it for minor leaguers, to estimate how they'll do in the pros). Like, most players tend to improve until their early 30's and then start a decline, which can be either gradual or steep. Certain players have certain characteristics that make them more likely to have a gradual decline (or gradual incline early in their career), while others have certain characteristics that make them more prone to have their performance fall off a cliff (or become a break-out superstar). It takes into account many many many things which dont include just statistics, but also height, weight, left-handed/right-handedness, the ballpark where the player plays his home games, etc.

When all is said and done, they have a set of projections for each player's performance, not only in the current year, but several years into the future. Actually, they don't just have one set of projections...they have 7 sets of projections which range from their average projection (which they call the 50th percentile), to an extremely optimistic projection (which they call 90th percentile), to an extremely pessimistic projection (which they call 10th percentile). The reason for having a range is that weird shit does happen, like guys having breakout years at a late age, or guys just mysteriously losing it, or a good player just having a down year.

For instance, their projections for Derek Jeter this year ranged from an AVG/OBP/SLG of .288/.351/.381 (10th percentile) to .347/.417/.504 (90th percentile). The 50th percentile was .321/.389/.450. Jeter's current numbers are .325/.393/.442...very close to the 50th percentile

By comparison, Alex Rodriguez is absolutely crushing his 90th percentile projections, which had him hitting "only" 42 home runs this year (he already has 42, with 5 weeks left to play)

So, any way, to come up with the projected team records, they just take the PECOTA projections for each player on each team, come up with an estimate of how many runs they'll score and how many they'll allow, and then make an estimate of win-loss record from that.

And yes, I know computers arent actually smarter than people lol. It was just the first thing that came into my head when I was making the post.

I'm still amazed that the program was so accurate

Dr. Giganto

Sports Moderator

Here are the final results:

The program nailed 1 team exactly, the Cubs

It was 1 game off for the Yankees, Reds and White Sox

It was 2 games off for the Dogers, Phillies, Mets, Braves, Diamondbacks and Brewers

It was 3 games off for the Cardinals, Royals, Tigers, Padres, and Blue Jays

It was 4 games off for the Red Sox and A's

It was 5 games off for Texas

It was 6 games off for the Indians and Orioles

It was 7 games off for the Nationals and Astros

It was 8 games off for the Giants, Marlins, Pirates, and Angels

It was 10 games off for the Rockies

It was 12 games off for the Twins and Devil Rays

It was 15 games off for the Mariners

On average, it was 5 games off. But the median (the midpoint of all the differences when placed in numerical order) was 3.5 and the mode (the most common number) was 2

If you take positive/negative into account, the average was -.07, the median was 1.5, and the mode was 2

It correctly predicted 3 of the 4 AL playoff teams, and they had the 4th (the Indians) in 2nd place by 1 game...so they were very close there

If the Padres win the Wild Card, it will have correctly predicted all 4 NL playoff teams

On the whole, I'm impressed

Blitzkrieg

Administrator

That is very impressive. Rockies/Padres game is a tough call tonight. Rockies are home and on FIRE, but they'll be facing the NL Cy Young in Jake Peavy.

Dr. Giganto

Sports Moderator

Yeah...I'm actually really looking forward to this game. It's sure to be intense.

I wonder how the Rockies fans will be. It's a chance at their first playoff berth since 1995...the crowd should be pretty lively

Blitzkrieg

Administrator

I think it will be a hot crowd. Not sure if anything can match that Phillies crowd yesterday. They were LOUD.. waving the towels, etc..

Dr. Giganto

Sports Moderator

I was at my friend's bar yesterday, and he's a big Mets fan. He had a stuffed doll of the Philly Phanatic being hung from a noose on the ceiling of the bar haha

Blitzkrieg

Administrator

Haha. I don't know, I might hate the typical Philly fan more then I hate the typical New York fan..it's close. Actually, I'm not big on Patriot fans either. The Red Sox fans, I love.. it's weird. I realize they are the same people most likely.. I just get the feeling like 10 years ago nobody gave a shit about the Pats and it irks me to see all of these guys coming out of the woodwork.

I have to admit though, I love the Rollins/Utley/Howard trio.