
                     Our Approach to Rating Players
                     ------------------------------


Our philosophy
--------------

The most important point we can make about our approach to rating players is
that we rate players based on performance, not reputation.  This philosophy
influences the meaning of our ratings and how we come up with them in the 
first place.

Our outfielder throwing rating, for example, measures the fielder's ability
to prevent runners from taking extra bases and to throw them out when they
try to advance.  A fielder earns a good rating by positioning himself well, 
getting to the ball quickly, making a quick release, throwing with power,
throwing with accuracy, and throwing to the right base.  Someone with a 
powerful arm might still get a subpar rating if he doesn't get into throwing 
position quickly, throws wildly, or throws to the wrong base.            

Similarly, our running rating is not just a measure of raw speed, but also 
the ability to read the ball off the bat, get a good jump, and make good 
decisions about when to go for the extra base.  Speed does matter, and 
you'll find plenty of speedsters with top ratings for baserunning.  But
there are other players who run the bases well because their instincts are
so good, and some swift runners who don't make the most of their chances.

And our rating for defensive range measures the ability to make plays.  An 
infielder earns a good rating through positioning, quickness, soft hands, 
and effective throwing (quick release, arm strength, accuracy).  It's not
always the flashiest player who makes the most of the balls that are hit
his way.

In other words, the things you can see when you watch them play, and the
things the scouts often talk about, don't always translate into results.
Sometimes the things you cannot see -- the mental aspects of the game in 
particular -- are as important or more important in how the player actually
performs in game action.  Our results-oriented analysis often uncovers 
players with good instincts who accomplish a lot more than the scouts might
have predicted based on their raw tools.


The role of analysis
--------------------

Because our ratings measure overall ability to succeed in a certain phase of
the game, we look for ways to evaluate performance by analyzing play-by-play
data.  This approach is not a radical one.  Baseball people have been doing
this for over a century to measure batting and pitching performances.  We 
don't, after all, give the batting title to the guy with the prettiest swing, 
we give it to the player who hit for the highest average.  We don't give the 
Cy Young to the pitcher with the best mechanics, we give it to the one who 
was most effective.  Using statistics to evaluate performance is part of the 
tradition of the game.  

But this tradition extends only to hitting and pitching.  You never hear a
television or radio analyst talk about meaningful measures of baserunning, 
throwing or defense.  Instead, they talk about their impressions of the 
player -- how fast he looks, his quickness, strength and athleticism.  If
we applied the same standard to hitters, we'd never talk about on-base
percentage or slugging average.  If we did the same with pitchers, we'd 
never talk about walk-strikeout ratios and hits per nine innings.

Our approach is to apply the time-honored tradition of using well-crafted 
statistics to evaluate baseball performances.  The difference is that we 
don't stop at hitting and pitching.  We design ways to measure results in
all phases of the game.

We sometimes find players whose performance is better or worse than you 
would guess by watching them a few times a year.  And our ratings are
occasionally at odds with the opinions being expressed by some of 
baseball's most famous writers and TV personalities.  But we sincerely 
believe that doing original research into player skills is an important 
part of producing an accurate baseball simulation.

Suppose a player has a reputation for great defense but our analysis doesn't
show a superior performance.  If we gave in to public opinion and rate him
higher than his performance justifies, we'd have these options:

* reduce the range rating of one of his teammates so the team's defense 
  isn't overrated

* reduce the effectiveness of the team's pitchers to compensate for the
  extra plays this player will now make in the simulation
  
* disregard these side effects and allow the player, the team, and its 
  pitchers to produce better results than they should

We don't think it's fair to downgrade teammates so we can give a popular
player a better rating than he deserves.  And we don't think you'd want us
to disregard the side effects and publish a season disk with players and
teams who will overperform.  So we do our best to rate players based on
performance, even if that means we might take a little heat for a few of
our ratings. 


Bunting Ratings
---------------

Assigning bunt ratings would be easy if all we had to do was look at the
leaders in sacrifice bunts each year.  But it's not that simple, because
a player's batting stats don't tell us how many times he failed to get a
bunt down when called upon.  And we have separate ratings for sacrificing
and bunting for a hit, and if you're limited to the official batting stats,
there's no way to tell how many bunt singles a player legged out this year.

So we wrote a program to compile bunting information from the play-by-play 
and pitch-by-pitch data we work with each year.  This program looks at 
three situations -- bunting for a hit, sacrifices, and squeeze bunts -- and
counts the number of opportunities to bunt, the number of bunt attempts and
the results of those attempts.  The results include separate categories for
bunt hits, reaching on an error, being put out at first with other runners 
advancing, popping out, grounding into a fielders choice in which a lead 
runner is retired, grounding or popping into a double play, and fouling off 
one or more bunt attempts before swinging away.   

The number of bunts fouled off is quite important.  In 2001, for example, 
in ordinary sacrifice situations on grass and with nobody out, batters 
were successful in advancing one or more runners 87% of the time.  Provided
they got the ball in play, that is.  But 32% of the time, they bunted foul 
once or twice before swinging away.  Counting this type of failure drops 
the success rate to 59%.  In other words, there are more bunting "failures" 
due to fouled bunts than to poorly-placed bunts, and we would be missing a 
lot of important information if we tried to measure bunting ability without
considering fouls.

So our bunt ratings take all of these factors into consideration.  And it
sometimes means that a player with loads of successful sacrifices is given
a mediocre bunt rating because he also failed on a lot of attempts.


Running and throwing
--------------------

There are no official statistics for baserunning, though you may occasionally 
see some data on how often runners go from first to third on a single.  Our 
studies examine how often runners took extra bases on hits and fly balls, 
taking into account where the ball was hit (it's much easier to go to third 
on a ball hit to right), the number of outs (getting the jump with two outs 
makes a big difference), and whether the playing surface is grass or 
artificial turf (ground balls reach the outfielder more quickly on turf, so 
it's harder to take the extra base on many singles.)

Before wrapping up our running ratings, we also review a report showing the
number of times players were used as or replaced by a pinch runner.  If a 
manager frequently uses player A to run for player B, it's a good indication 
that the manager feels A is a much better runner.  It doesn't mean that A is
a great runner or that B is a lousy runner.  It could be an Av runner 
replacing a Pr one, or it could be an Ex runner replacing an Av one.  That's
why we look at who's replacing whom, not just the raw count of pinch-running
appearances or replacements. 

Outfield assist totals give us some insight into outfielder throwing, but 
they can also be misleading.  Some great throwing outfielders have their 
assist totals cut down by their opponent's unwillingness to run on them in 
the first place.  And not all assists are created equal.  Some assists
result when a throw nails a lead runner, while others occur when the lead
runners score but the batter is out at second advancing on the throw.  So
we've written a program that allows us to see how many quality assists
each outfielder earned during the season.

Our throwing ratings focus on the ability to prevent the lead runners from 
advancing (and to throw out those who try) using a combination of skills:  
getting to the ball quickly, getting the throw off quickly, and throwing 
with strength and accuracy.  Some players with strong arms are not very 
accurate.  Some players with weak arms compensate by getting into throwing 
position more quickly.  As with all of our ratings, we measure results, not 
just how often a player looks good making a throw.


Measuring Defensive Range
-------------------------

Defensive range -- the ability to reach batted balls and turn them into 
outs -- is one of the hardest baseball skills to evaluate.

Tradional fielding statistics -- putouts, assists, and errors -- tell us
how many plays a fielder was involved in, but they don't tell us how many
times he had an opportunity to make a play.  A sure-handed shortstop may
have very few errors but let his team down by failing to reach ground balls
that other shortstops turn into outs.  You can't figure that out just by
looking at traditional fielding statistics.

Some publications rank players based on range factors (putouts plus assists
per game) or fielding runs (a metric developed by Pete Palmer for the Total
Baseball encyclopedia).  But these statistics are based on the traditional
statistics and therefore suffer from the same limitations.  Specifically,
rankings based on range factors assume that each of those players had the 
same number of opportunities to make plays.  But they don't for several 
reasons:

- some pitching staffs strike out more hitters and therefore allow fewer
batted balls to enter the field

- the mix of ground balls and fly balls can vary quite substantially from
one pitching staff to another

- the mix of left- and right-handed pitchers varies from team to team, 
meaning that those teams face different percentages of left-handed hitters,
which in turn affects how many ground balls are hit to each side of the 
infield

- some pitchers are better than others, and weaker pitchers present fielders
with a higher percentage of difficult chances

- some ballparks require that fielders position themselves differently, 
affecting how many batted balls can be reached with normal effort

For all of these reasons, it's much better to count the number of 
opportunities presented to each fielder and the number of times each fielder
was able to turn those opportunities into outs.  To that end, we have 
developed a series of analytical programs that use play-by-play data to
assess fielding ability.  Our methods, which are described in detail in an
article ("Evaluating Defense") that you can find on our web site, take all
of the above-mentioned factors into account.

We believe this analytical approach is much better than relying on the 
subjective opinions of baseball writers and broadcasters.  And it is much
better than relying on the Gold Glove voters to identify the best fielders.
Much too often, these observers focus almost all of their attention on one
aspect of defensive performance -- errors -- and neglect the ability to
reach batted balls in the first place.

In previous years, these season disk notes have included our comments on the
Gold Glove winners and other fielders whose ratings differ somewhat from 
past years or from their reputations.  This year, we have decided to publish
those comments in a separate note instead.


Wrapping up
-----------

We put a lot of effort into our player ratings each year.  It would be 
easier, we suppose, if we just carried forward a player's ratings from
year to year or if we based our ratings on what we hear in the media.
But we don't believe we'd be doing our job if we did that.  

Everybody knows that batting and pitching performances go up and down over 
the course of a player's career.  We believe the same is true of other
skills such as defense and baserunning.  Using sophisticated analytical
techniques and the best information available helps us spot changes in
performance when they happen, not later when the media picks up on them.

But judgment and common sense are essential, too.  Many players do not get 
enough playing time in a single season to enable us to reach conclusions 
based on numbers alone.  Sometimes the way a player is used can produce a 
biased view of his abilities.  So we always look at the player's results in 
the context of his team, career (including minor league records for younger 
players) and an overall understanding of how baseball works.  
