Plumlee attended high school at Christ School in Arden, North Carolina, where he led the school to a 63–6 record as a junior and senior. During his high school career he was a two-time NCISAA 3A all-state team selection, an Asheville Citizen-Times All-Western North Carolina first team pick as a junior and senior. He also lettered in track as a senior and posted a school record high jump of 6’9’ showcasing his outstanding athletic abilities that he would become known for during his college basketball career.
Plumlee had originally committed to play at Stanford University for the 2008–09 season, but after Stanford coach Trent Johnson left for LSU and brother Mason had committed to play at Duke the following year, he chose to become a Blue Devil. After serving a limited role his freshman season, Plumlee earned the starting center position over senior Brian Zoubek at the beginning of the 2009–10 season before returning to a bench role later in the season. Plumlee finished the championship winning season averaging 16.4 minutes per game, 4.9 rebounds and 5.2 points per game. During the 2009–10 season Plumlee and brother Mason were often the first two players off the bench usually being brought in to games to replace Zoubek and Lance Thomas.
Miles played a similar role in his junior campaign to that of his sophomore, contributing solid offense and defense for the team in limited playing time. He finished the season averaging 17 minutes per game, 4.9 rebounds and 4.8 points. In his final season in 2011–12, Plumlee would again play the majority of the season off the bench while starting games every now and then. He went on to have his best season from a statistical view, grabbing the second most rebounds per game for the team with 7.1, behind brother Mason, as well as leading the team in field goal percentage, connecting on 61% of his shot attempts. In all Plumlee completed his four seasons at Duke having played 135 games for 654 rebounds and 650 points.
Plumlee was drafted by the Indiana Pacers with the 26th pick in the 2012 NBA Draft. During the 2012–13 season, he had multiple assignments to the Fort Wayne Mad Ants, the Pacers' D-League affiliate team.
Miles has three siblings, brothers Mason and Marshall and sister Madeleine. All three brothers have been involved in Duke's basketball program; Miles and younger brother Mason were the sixth pair of brothers to play at Duke at the same time. His sister Madeleine has committed to play volleyball at the University of Notre Dame.
His parents are Perky (a former Tennessee Tech basketball player) and Leslie (née Schultz) Plumlee (a former Purdue women's basketball player who set the school single-game rebound record with 25). The two met at a basketball camp during the summer of 1979. His grandfather Albert "Bud" Schultz played basketball at Michigan Tech (1944), his uncle William Schultz played basketball at Wisconsin-Eau Claire (1971–72), and another uncle, Chad Schultz played basketball at Wisconsin-Oshkosh (1983–86).