This bug has been live on holodeck since S4 went live. And yes it does happen from SFA to ESD, earlier tonight i actually got kicked from the game because it took more than 15 minutes to load ESD...and that was going from SFA (on tribble).
Truth to tell, I've only just started logging back on to Holodeck to get the DeltaFlyer before it's a C$tore exclusive. But prior to October, when I was still using Holodeck regularly, I wasn't noticing lag like this loading ESD. Also true though is that my Holodeck character had/has little reason to keep visiting ESD so a little lag could have gone unnoticed. However today is the first time I've ever gone idle while loading into an instance.