After 17 years I have decided cease publishing the Prep Performance Ratings for Tennessee high school football. I had thought about doing this for the last couple of years, but decided the time is now. I have had many enjoyable experiences doing the ratings, but times change and the emergence of the Internet has lessened the need to achieve the original purpose of the ratings – and that was to give people across the state an idea of who the best teams were in each classification.
By doing the ratings I have been fortunate to meet many people across the state, have stories written about them and heard them discussed on numerous radio shows. It has also allowed me to build a database that contains every Tennessee football score for over 33,000 high school games since 1993.
But times change and the original purpose for having the ratings is now met by the Internet, numerous other rating systems, and by other instant communication methodology that is available. People can now copy/paste scores into many of the hundreds of rating systems that are available and generate their own state-wide ratings. My main goal was to get information out about Tennessee high school football and now that is possible by many different venues.
I am still involved heavily in high school sports through my work at TSSAA and have the challenge of writing the computer code to generate the new football selection criteria. This effort superseded the desire and need to continue to do my computer ratings.
Earl Nall
Earlnall@gmail.com
The Early Years – In 1993 there was no Internet and the only way to collect scores was to get all papers from across the state, make phone calls, and request missing scores from the TSSAA. None of this was easy. On Saturday morning I would get the Knoxville News Sentinel and get some scores. I would then drive to Crossville, eat breakfast at the Bean Pot Restaurant and buy the Nashville Tennessean for more scores. Later in the day I would drive to the Shell gas station in Kingston and get a copy of the Chattanooga Times, as they were published later and had scores the other two papers did not have. On Monday I would go to the library and find a copy of the Sunday Memphis Commercial Appeal and get west Tennessee scores. Even with all of this, I would have to contact TSSAA and ask if they had other scores (back then TSSAA wrote every score on a big board in assistant director's Bob Baldridge's office). All this work and would still be missing 10-15 scores a week and had to call schools to get results.
By Thursday, I would be ready to do the ratings. Since there was no Internet back then I faxed them to various media outlets across the state. A number of newspapers ran the ratings in the paper. The only other ranking system I was aware of was the Associated Press.
This mode of collecting scores went on for 4-5 years and then the Internet started creeping in, and by 1998 email and websites began to make score collection easier and by 1997 I had the ratings on the Internet. By 2000 it wasn't unusual to get all but of couple of scores by Saturday.
The Accuracy – To this day, I am still amazed at the accuracy of the ratings. I had no idea what I was doing when I developed my algorithm. In the 17 years of the ratings there have been 90 football championships. In only two cases has the state champion not been ranked in the top 8 schools of the final ratings of their class. This occurred with Maryville (ranked #11 in final regular season ratings) in 2000 where they lost their first 4 games and won the state and Livingston Academy (ranked #16 in final regular season ratings) in 2005 when they entered the playoffs with a 5-5 record and won the state championship.
The ratings would start the year with an accuracy rate of about 70%, but since the ratings learned about a schools' strength each week and by the 10th week they would always get over 90% of the games correctly. It was always interesting to me that the ratings were more accurate with the lower division teams. Class A accuracy was better than AA, AA better than AAA. This of course makes perfect sense as the ratings decrease in accuracy the more evenly matched the teams are. This is why I feel certain that if I took my rating system and tried to do the NFL or major college sports, the accuracy would decrease as the completion was more balanced.
Each year I would take the final week 10 ratings of all the other rating systems that did Tennessee High School football and compare to my week 10 ratings. I did this by seeing how the final top teams in each rating system did against mine based on how teams performed in the playoffs. I measured this by using a standard weighting system of more point s for a team the further they advanced. Of all the comparisons that were made the only system to do as good as mine were the Kenneth Massey ratings. This isn't bad since Kenneth is a Phd Professor at Carson-Newman and is part of the BCS and the main guy for HighSchoolSports.net. But, even against Kenneth, the yearly comparisons were neck and neck.
The Formula – My rating system was written in Perl, a platform independent programming language that is still used today to run websites and analyze data. Perl does a lot of nifty things that programmers embrace, e.g. implied looping, reading a file of data with one line of code, and being able to embed Unix regular expression metacharacters (grep) in your code. This enables the programmer to verify a valid email address or date or almost anything with just a few symbols.
The base idea for my ratings is really nothing new to people who develop ratings programs. I used a method called "split difference" but modified it to add some statistical methods that I learned in graduate school.
The split difference method is basically this. Say two teams are playing. Team A has a rating of 41.0 and Team B has a rating of 30.0. If Team A is the home team you would expect them to win by 14 points (41.0-30.0) + 3.0 for the home field advantage. (year after year the home field statistically is worth 3 points in Tennessee high school football). If Team A wins by 8 points, then they didn't win by their expected difference of 14 points. So you have to take the difference between their expected win and actual win and you would see that number is 6 (14-8).
Now here is where it gets a little tricky. You have to take the actual difference of the expected difference minus the actual score difference, in this case (6) and multiply it by a factor. This factor is larger earlier in the season and decreases each week. So let's say that in week 1 the factor is 0.2. You would multiply .20 times the actual difference which in this case would be 6 times .2 so the actual ratings difference in this case would be 1.2. In this case, Team A won by 8 when they were expected to win by 14 so their rating would drop from 41.0 to 39.8, while the losing Team B would actually see their rating increase from 30.0 to 31.2 (30.0 + 1.2).
The above logic is then applied to data each week and the season is played over and over again – this is how the ratings program "learns" about how strong each team is and improves with each week.
Let's take this example. Suppose Team A starts the season with a high computer rating (starting rating based on historical results) and they are playing Team B that has a low rating. Team B soundly defeats Team A in what appears to be a big upset. But the computer doesn't know that Team A graduated all starters and Team B has a veteran team with a new coach. So if you knew the teams well it would not be an upset in your eyes because you knew what was happening. But, the computer didn't know this and has to learn. Team A continues to lose each week and by the say, third week their initial high rating has dropped. What the computer does is take a team's new rating and then goes back to week 1 of the season and plugs that rating in as the team's starting rating and re-runs itself with that new data. The rating program runs itself over and over, each time calculating a new power rating and starts over again – the program may re-run itself many hundreds of times each week. It will do so until the rating differences become asymptotically close to a delta value that is stored in the ratings program. Then the ratings know it is time to stop. So as the season progresses the season is actually played over and over again many hundreds of thousands of times until it is satisfied that it has the best model it can achieve. By the fifth week of the season the computer has learned that Team B is better than Team A and that difference is reflected in the new ratings.
The delta difference that is used and the weekly factor (in first example) were calculated using some statistical models that optimized the values. Every year I rerun theses statistical models and refine the values. That is how my system is different than a basic split-difference model.
The cutoff value for win margin was set at 33 points in my program. If you beat someone by 80 points you got the same ratings points as if the team had won by 33.
The Inspiration – My college mathematics professor was Herman Matthews. Matthews was one of the original contributors to the BCS, but dropped out when they quit using point differential as a factor in computer ratings. Matthews developed a rating system to rate our intramural teams in 1965 and when I saw that I was hooked. He and I discussed the philosophy of ratings. Matthews passed away couple of years ago and I felt a great loss. He and I talked many times over the years, shared emails, and told ratings stories. I also never knew his rating algorithm but I think logarithms were involved. A remarkable man.
The Highlights – Many highlights from doing the ratings. Met many unforgettable people, made a lot of people mad and got praised by many.
Few years back Gary Parrish, now a writer for CBSSportline.com, was the high school sports guy in Memphis for the Commercial Appeal. He challenged the ratings and wrote a column each week about how he was "killing the mathematician". The whole thing was fun. He did in fact jump to a big lead, but as the computer learned the Memphis schools they began to close the gap. On the ninth week of the season the computer had closed within a single game of Parrish by picking correctly all 25 games in Memphis that week. But Gary wasn't going to lose – so in week 10 he looked at the ratings and picked the same teams the computer picked – he couldn't lose – and didn't. He won by one game. Good times.
Was written up on front page of Nashville Tennessean on a story about computer ratings. Good article that talked about how different computer rating systems worked and how people follow them. Article well researched. Kudos to sports editor Larry Taft for having the vision to have such an article written.
The rating system allowed me to converse (through email) with some of the BCS contributors. Had a few emails about ratings philosophy with Jeff Sagrin and he made a statement that says the single most important statistic in doing computer ratings is margin of victory. He said he had run his programs using historical data from college and professional sports to arrive at that calculation. But, alas he had to change his college football ratings since the BCS eliminated the margin of victory.
This wasn't a highlight, but was a depressing notification. The week after 9/11 happened I didn't do the ratings. A number of threatening emails were received demanding that I put the ratings up. It was then I realized that some people used them for gambling. A little unnerving.
A huge highlight though was my meeting Stan Crawley. Then the high school sports editor for the Chattanooga Times-Free Press. He referenced my ratings in his weekly column and for the 1997 state championships he mentioned that I could probably get a press pass to the Clinic Bowl state championship games because I was known across the state for media coverage. He suggested I contact the TSSAA media person – then the legendary Edgar Allen and he provided me with a press pass. While at the games I met then TSSAA Executive Director Ronnie Carter and he and I started discussing technology. One thing led to another and he wanted me to do the TSSAA website for them. Thus, started a job and lasting friendship with Mr. Carter that exists today. I worked part time for a number of years doing contract work until I retired from the Nuclear Facilities in Oak Ridge and them begin to work at the TSSAA as technology director. Probably none of this would have happened if Stan Crawley had not brought up me getting that press pass.
Maybe the best highlight was my daughter Sara's interest in high school football. Over the years she has helped me collect scores and tweak the ratings. By the time she was ten years old she knew every school in the state, their nickname and team colors. She doesn't miss any state championship games. She helped me when she was a cheerleader at Kingston high school, when she was in college at Tennessee Tech and now while she teaches 8th grade in Kingston. We still have a copy of the Columbia Daily Herald where she and I were pictured on the front page of Sports when she was 12 years old. She is on my facebook profile photo with me at the state football championships (thanks to Murphy Fair for the photo)
The Best Teams – Two Riverdale teams in the early 2000s had ratings of over 100 pts. The computer clearly thought they stood out. I had to check my programming logic when I saw that. However, no team dominated a class like the early 1990's Marion County team. They had an unheard of rating of 25 ratings points higher than the next best team in their class. As good and dominating as Alcoa has been these past few years, they come nowhere close to being that far ahead of teams in their own class.