Now I have figured out how the lower minors work, how they lead to the upper levels and what it takes to make it to the majors. Now, I had enough data and knowledge to find what makes a major leaguer successful. I can now tell you how rare that "#3 starter" actually is, along with a lot more information.
I used data from 1961-2012 for my analysis at the big league level. I used the same theory as I did with minor leaguers, only I included everyone in the calculations. Oddly enough, the basic numbers are pretty similar.
There are 27 different categories for pitchers. This is the list of them and how frequently they are found in the majors.
Some categories have a wider base of players but many of these categories are very specific. Below is short synopsis of what type of player fits in each of category and some basic info about each group.
To sum this up, less than 1% of MLB pitchers would be ace level or #1 starters. About 17% are #2 and #3 starters. So when someone says that they could be a #3 starter, they are predicting that they will be in the top 18% of MLB pitchers.
I hope that using these labels going forward help to delineate these titles more than just randomly assigning them.