The court ruled 5-4 in the case of Joseph Frederick, who unfurled his handiwork at a school-sanctioned event in 2002, triggering his suspension and leading to a lengthy court battle.
In a concurrence, Justices Samuel Alito and Anthony Kennedy said the court's opinion "goes no further" than speech interpreted as dealing with illegal drug use.
"It provides no support" for any restriction that goes to political or social issues, they said.
In dissent, Justice John Paul Stevens said the ruling "does serious violence to the First Amendment."
Students in public schools don't have the same rights as adults, but neither do they leave their constitutional protections at the schoolhouse gate, the court said in a landmark speech-rights ruling from Vietnam era.
The court has limited what students can do in subsequent cases, saying they may not be disruptive or lewd or interfere with a school's basic educational mission.
Frederick said his banner was a nonsensical message that he first saw on a snowboard. He intended it to proclaim his right to say anything at all.
Frederick displayed his handiwork on a winter morning as the Olympic torch made its way through Juneau, Alaska, en route to the Winter Olympics in Salt Lake City.
School principal Deborah Morse said the phrase was a pro-drug message. Frederick denied that he was advocating for drug use and brought a federal civil rights lawsuit.
Former independent counsel Ken Starr, whose law firm represented the school principal, called it a narrow ruling that "should not be read more broadly."
Taking issue with that, Steven R. Shapiro, national legal director of the American Civil Liberties Union, said, "It is difficult to know what its impact will be in other cases involving unpopular speech."
The Students for Sensible Drug Policy said it was sad that the court thought there should be a drug exception to the First Amendment.
In their concurrence Alito and Kennedy said that the decision "goes no further than to hold that a public school may restrict speech that a reasonable observer would interpret as advocating illegal drug use."
Nor does it address political or social issues such as the wisdom of the war on drugs or of legalizing marijuana for medicinal use, Alito and Kennedy said, embracing language from Stevens' strong dissent.
Stevens said the First Amendment protects student speech if the message itself neither violates a permissible rule nor expressly advocates conduct that is illegal and harmful to students.
"This nonsense banner does neither," Stevens said.
Justice Stephen Breyer said the court should not have decided the First Amendment issue, but should have simply held that Frederick's claim for monetary damages because school officials have qualified immunity in carrying out their duties.
Frederick, now 23, said he later had to drop out of college after his father lost his job. The elder Frederick, who worked for the company that insures the Juneau schools, was fired in connection with his son's legal fight, the son said. A jury recently awarded Frank Frederick $200,000 in a lawsuit he filed over his firing.
Joseph Frederick, who has been teaching and studying in China, pleaded guilty in 2004 to a misdemeanor charge of selling marijuana at Stephen F. Austin State University in Nacogdoches, Texas, according to court records.
The case is Morse v. Frederick, 06-278.