Richard asks if Bill Clinton narrates books on tape because he finds his voice to be nice. Edward Herrmann, the actor who plays Richard, is known for his narration of documentaries as well as books on tape.
Before singing to Rory, Emily, played by Kelly Bishop, says, "Please excuse us. We're not singers." However, in real life, Kelly Bishop is an accomplished singer having originated the role of Sheila in A Chorus Line, for which she won a Tony Award for Best Performance by a Featured Actress in a Musical.
When Richard and Emily sing to Rory at the pre-graduation party, the pianist in the background is none other than Brad Ellis who is most recognizable as the pianist in the TV show Glee.
The title of this episode is (part of) a quote from Shakespeare's "Henry V" (Act III), and is spoken by Paris Geller just before the girls walk onto the stage at graduation. It is from the first line of the king's speech to his troops, before staging another attack on the French city they are besieging. The speech ends with the famous line "Cry 'God for Harry, England, and Saint George!'"
When Luke goes to Liz's garage for a present for Rory we see the huge wall of drawers that are from Luke's apartment.