Three weeks into fatherhood and two weeks into football season, Tom Brady has a personal problem to tackle: balancing his schedule between playing with the New England Patriots and seeing his infant son, who is home in Los Angeles with his mom, Bridget Moynahan.

“I kind of cuddled him like a football,” the 6′4″ triple Super-Bowl-winning quarterback, 30, tells the new issue of Sports Illustrated about John Edward Thomas Moynahan, who was born Aug. 22.

“I’d love to be out there all the time, year-round, but it’s hard to make that a reality,” the Boston-based Brady says about L.A., where the baby’s mom, and Brady’s ex-girlfriend, lives and works.

“I live here,” explains the athlete. “But I’ll start lobbying for off days throughout the year.”

And while SI columnist Rick Reilly harps on Brady’s hunky good looks that make him such a babe magnet who just happens to be dating the $33-million-a-year Brazilian supermodel Gisele Bündchen, the quarterback does confess to the magazine, “I was the little sports nerd in high school who hardly had a girlfriend!”