There could be a few problems here and possibly more :
1) You have a lot of collections and it takes a long time to run through them
2) You don't hard commit very often so you have a lot of transaction logs to replay
3) There is the new SolrSuggester. If your logs look like:
2015-04-02 14:13:38,906 INFO [searcherExecutor-6-thread-1] SolrSuggester.java (line 139) build()
2015-04-02 14:14:33,766 INFO [ScheduledThreadPoolExecutorUtil-1] ....
Note the timing here: 14:13:38 -> 14:14:33. In this case almost a minute and it can be worse. So if you're not using it you should remove it and possibly mark it to not build on startup (there is a param to do that:
There is a Solr JIRA here about removing it from the default solrconfig.xml: