Weak population gains and higher government spending could hamper growth over the next 30 years, the CBO said Thursday.