Potential Office365 (Exchange Online) Throttling Issue
-
-
@bigbear said in Potential Office365 (Exchange Online) Throttling Issue:
You are going to want to do a network upload of your PST files using AzCopy to free Azure storage space MS offers for migrations. I have done hundreds of larger PST files like the one you are stuck on and I can tell your current import won't complete ever.
Just google "365 pst network upload".
MS365 has become more and more useless with the shift to CSP and "partner first" initiatives. Do everything you can to get escalated back to the U.S. if you ever need real help.
I gave him the link to that 4 hours ago.
he didn't want to do that. -
@dashrender said in Potential Office365 (Exchange Online) Throttling Issue:
@bigbear said in Potential Office365 (Exchange Online) Throttling Issue:
You are going to want to do a network upload of your PST files using AzCopy to free Azure storage space MS offers for migrations. I have done hundreds of larger PST files like the one you are stuck on and I can tell your current import won't complete ever.
Just google "365 pst network upload".
MS365 has become more and more useless with the shift to CSP and "partner first" initiatives. Do everything you can to get escalated back to the U.S. if you ever need real help.
I gave him the link to that 4 hours ago.
he didn't want to do that.Did not see anything in this thread about it. The method he is using has never worked for larger files for me.
-
@bigbear yes @Dashrender did give me the link offline a while back.
However, I figured that would be just as much of a CF as the method I am using.
Part of the issue is that we do not have PST files. The MDaemon Outlook Connector software we are using has its own format.
So, I am assuming this is what I would want to do...
- Disable user access to their mailbox
- Set up a new profile with IMAP and download all the messages to this.
- Export this new profile's e-mail to a PST file
- Run the PST import method as you (and @Dashrender) suggested
How long would you say a PST import of, say 20GB would take using this method?
-
@bigbear said in
I can tell your current import won't complete ever.
Oh boy I hope that is not true!
That would leave this particular user in disarray.
They keep telling me they can turn this throttling off temporarily, but then the next sentence is always "but this is normal behavior" so who knows what to believe.
The second guy has also left for the day, and won't be back for 2 days. So in the morning I am going to call and get this hopefully escalated to US tech support.
This is the first really large mailbox I have worked with. I had no idea there were such limits. To be honest, it is something I never would have thought of.
-
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
They keep telling me they can turn this throttling off temporarily, but then the next sentence is always "but this is normal behavior" so who knows what to believe.
Those aren't conflicting statements in any way.
-
@scottalanmiller said in Potential Office365 (Exchange Online) Throttling Issue:
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
They keep telling me they can turn this throttling off temporarily, but then the next sentence is always "but this is normal behavior" so who knows what to believe.
Those aren't conflicting statements in any way.
Well, that is true.
I guess what I meant was ... I get the feeling that is a preface to "hey what we did didn't work, but too bad"
-
I might also try IMAP migration, which a couple people have suggested. IMAP migration has a throttle as well, but the expected throughput is 14-20GB an hour, which would be perfect for me. (The current scenario is .5 GB per hour.)
Honestly, the way I was doing it was working so well, I didn't see a need to do anything else.
-
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
I used IMAP to download the messages to a new profile with no issue.
What file format did you download to? You can still create a non-exchange profile and import to a PST file, then upload that to Azure using AzCopy.
Also for $10 or $12 you can use IMAP to 365 with BitTitan Migrationwiz which you may find much simpler, since its only this single mailbox.
I will admit MigrationWiz is much simpler and it does multiple passes. Just gets expensive when you have hundreds of mailboxes.
-
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
I might also try IMAP migration
The IMAP tool I have found leaves out data, the MigrationWiz IMAP migration is probably worth the few dollars for the single large mailbox. Just to save you another headache. The Microsoft tools has missed large date ranges of data for me in the past. MIgrationwiz has always come through for me.
-
@bigbear said in Potential Office365 (Exchange Online) Throttling Issue:
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
I might also try IMAP migration
The IMAP tool I have found leaves out data, the MigrationWiz IMAP migration is probably worth the few dollars for the single large mailbox. Just to save you another headache. The Microsoft tools has missed large date ranges of data for me in the past. MIgrationwiz has always come through for me.
That was my fear.
My users have all these weird folder structures and whatnot. Was really afraid the MS IMAP migration tool would louse things up. And TBH, it would mean I would have to go through folder by folder to ensure everything is there, both the folder itself and the number of messages in it.
I will give the other one a try.
-
@bigbear said in Potential Office365 (Exchange Online) Throttling Issue:
What file format did you download to? You can still create a non-exchange profile and import to a PST file, then upload that to Azure using AzCopy.
Since I had issues with the MDaemon connector tool, I set up a VM, and set up a new Outlook profile. I then add in the existing IMAP account, and download everything. (THe new versions of Outlook download the entire mailbox.)
Once that is done, I add the O365 account, and just drag things over.
-
If they would just open the pipe up for a few days for me, this would be so simple.
They said they can, but I am not confident they know what they are talking about since they both said they did it to no avail.
-
@bigbear said
BitTitan Migrationwiz which you may find much simpler, since its only this single mailbox.
I wonder what "class" they consider MigrationWiz.
Because "third party Exchange migrations" (and they list BitTitan in this group) are also crippled at .5GB an hour if you notice.
-
We used to just use a Ruby script for IMAP transfers, worked really well.
-
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
If they would just open the pipe up for a few days for me, this would be so simple.
BTW are you using Outlook 2016? Just curious because of PST file limits in older versions.
I have always just setup the new user, run a migrationwiz pass for the last 2 weeks, contacts, calendar etc, let the user into their new 365 mailbox then started the full pass migration. This is what MS refers to as a "dial tone" migration. Then I run another pass a couple weeks later before shutting down the old server or services.
I would imagine the upload limits to be DoS Security best practices of some sort.
-
@bigbear said in Potential Office365 (Exchange Online) Throttling Issue:
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
If they would just open the pipe up for a few days for me, this would be so simple.
BTW are you using Outlook 2016? Just curious because of PST file limits in older versions.
I have always just setup the new user, run a migrationwiz pass for the last 2 weeks, contacts, calendar etc, let the user into their new 365 mailbox then started the full pass migration. This is what MS refers to as a "dial tone" migration. Then I run another pass a couple weeks later before shutting down the old server or services.
I would imagine the upload limits to be DoS Security best practices of some sort.
How long does the actual migration take? The actual file copy, I mean.
In theory, if what I am doing works, it's not the end of the world if I do it on their actual machine.
It's the fact I staged it that is causing the problem.
-
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
How long does the actual migration take?
If you are talking about Outlook import I have never seen one complete over 15GB. Left one out there for 2 weeks.
If you are talking about Migrationwiz its all server to server and doesn't affect the user. You could manually import the last few weeks and all contact, setup their new Outlook and let the import run overnight. Outlook should only be caching 90 days or so of email anyway.
Are you going to configure an archiving policy?
-
@bigbear said in Potential Office365 (Exchange Online) Throttling Issue:
If you are talking about Outlook import I have never seen one complete over 15GB. Left one out there for 2 weeks.
Shhhhhhhh! It's working its way through nicely. Just slooooowwwwww.
If you are talking about Migrationwiz its all server to server and doesn't affect the user. You could manually import the last few weeks and all contact, setup their new Outlook and let the import run overnight. Outlook should only be caching 90 days or so of email anyway.
Right, but according to that link, MS throttles 3rd party migrations to .1 to .3 GB per hour. So it would take forever, not overnight. You've seen large mailboxes import using Migrationwiz overnight?
Are you going to configure an archiving policy?
You mean to filter messages into the in-place archive?
No.
-
@brrabill said in Potential Office365 (Exchange Online) Throttling Issue:
Right, but according to that link, MS throttles 3rd party migrations to .1 to .3 GB per hour.
We always open a ticket to request a temp lift on EWSLimits. I believe this has already been lifted for you per your comment above, but per my comment I don't believe the support guys know what you are talking about. And the EWS limits being increased has never made an Outlook import for a large mailbox go faster or finish for me. But will be interesting to see if it does for you. I just wouldnt bet my lunch on it.
