Large file sharing to, from and within mainland China
-
@wrx7m said in Large file sharing to, from and within mainland China:
@coliver said in Large file sharing to, from and within mainland China:
@wrx7m said in Large file sharing to, from and within mainland China:
So it looks like there are essentially 2 speeds when dealing with China- The inside China speed and the International speed. Their inside speed is 100Mbps but their average international speed (they refer to it as the International Corridor) is 1Mbps. Yeah... 1!
I have yet to find an optimal way of doing this.
I have asked them to find out how to increase their international bandwidth. I was doing some Googling over the weekend and found that larger companies have this capability but I don't know how they are doing it and how much it costs (probably more than we can spend).
Also, if I can work with a Chinese host for a nextcloud server, setting up some public-facing websites, which nextcloud may or may not be classified as, may or may not require some sort of license from the Chinese government, which could take a month or more or get denied. If I can get that setup, then we are still dealing with an issue where we would need to replicate the data across to a second nextcloud server here. From what I was reading on the nextcloud forums, there is no native way of doing this yet and even if we could, we are still constrained by the 1Mbps average international speed.
The user proposed syncing two Synology NAS devices (1 in China and 1 here), but that introduces all sorts of other headaches on how to get the data shared out to our users here and backed up consistently. As we all know, syncing does not equal a backup.
Would something like RDS or VDI work in this instance?
I have a server, here, running RDS via RDGateway but they are still needing to transfer the large files to, from and within China.
What about a more manual process? Like a dedicated machine inside China that has a NextCloud client? Then they manually manage files inside of China, rather than having them be automatically handled elsewhere? I don't know the end user use case, but if there aren't a lot of users, this is likely pretty simple.
-
@scottalanmiller The problems with that, is that we still need to give and receive the large files from them and also get them backed up.
-
WAN Optimization can help, but with users not in a central location in China, it doesn't help much. There are some serious WAN optimization products out there, but this isn't a great use case for them.
Is there anything that you can do to compress these large files? How big are they, how many and what is the change rate on them? And how much is changed in China vs, in the US?
-
@wrx7m said in Large file sharing to, from and within mainland China:
@scottalanmiller The problems with that, is that we still need to give and receive the large files from them and also get them backed up.
I know and NextCloud will do that.
-
So here is my thought.... You have one machine IN China that syncs to the NextCloud server in the US (or vice versa, but fewer hassles this way.) The Chinese staff grab the file from there and copy it around however they need. They always save changes back to that location, though. So the latest copy is always on the machine syncing to NextCloud. This machine will start sending the updated copy to the US the moment that it is saved, but the Chinese staff get to keep working.
Then the NextCloud server just has to be backed up as usual in the US.
-
@scottalanmiller I had suggested that we can upload the files to a nextcloud server here and send our Chinese users the links so that they can share the data with the factories/vendors but he said that would only work for some of the situations.
EDIT- That would cut out the downloading twice and place the burden on the factories. The reason this isn't ideal is because they sometimes manipulate/edit the files before sending them onto the factories/vendors
The large files are compressed in a zip format.
We also have video, docs and other files that are used for QC of our products that need to be accessed and stored for our R&D and Execs here .
-
@wrx7m said in Large file sharing to, from and within mainland China:
@scottalanmiller I had suggested that we can upload the files to a nextcloud server here and send our Chinese users the links so that they can share the data with the factories/vendors but he said that would only work for some of the situations.
EDIT- That would cut out the downloading twice and place the burden on the factories. The reason this isn't ideal is because they sometimes manipulate/edit the files before sending them onto the factories/vendors
The large files are compressed in a zip format.
We also have video, docs and other files that are used for QC of our products that need to be accessed and stored for our R&D and Execs here .
Right, my suggestion sounds like it would work fine then since it solves all of those problems. It's your thought that you could share links from Nextcloud that isn't going to work. Take that out and re-read mine again. I think you will find that it will work.
Also, don't use zip it's VERY inefficient. There are much more powerful compression mechanisms that might shave a lot of time off of your transfers.
-
@wrx7m said in Large file sharing to, from and within mainland China:
We also have video, docs and other files that are used for QC of our products that need to be accessed and stored for our R&D and Execs here .
Where do they get modified?
-
@scottalanmiller said in Large file sharing to, from and within mainland China:
So here is my thought.... You have one machine IN China that syncs to the NextCloud server in the US (or vice versa, but fewer hassles this way.) The Chinese staff grab the file from there and copy it around however they need. They always save changes back to that location, though. So the latest copy is always on the machine syncing to NextCloud. This machine will start sending the updated copy to the US the moment that it is saved, but the Chinese staff get to keep working.
Then the NextCloud server just has to be backed up as usual in the US.
What system would I put there? A Windows desktop OS box?
-
@scottalanmiller said in Large file sharing to, from and within mainland China:
@wrx7m said in Large file sharing to, from and within mainland China:
@scottalanmiller I had suggested that we can upload the files to a nextcloud server here and send our Chinese users the links so that they can share the data with the factories/vendors but he said that would only work for some of the situations.
EDIT- That would cut out the downloading twice and place the burden on the factories. The reason this isn't ideal is because they sometimes manipulate/edit the files before sending them onto the factories/vendors
The large files are compressed in a zip format.
We also have video, docs and other files that are used for QC of our products that need to be accessed and stored for our R&D and Execs here .
Right, my suggestion sounds like it would work fine then since it solves all of those problems. It's your thought that you could share links from Nextcloud that isn't going to work. Take that out and re-read mine again. I think you will find that it will work.
Also, don't use zip it's VERY inefficient. There are much more powerful compression mechanisms that might shave a lot of time off of your transfers.
We use 7zip and RAR files a lot, but zip is the most ubiquitous and universal when dealing with the factories and vendors.
-
@wrx7m said in Large file sharing to, from and within mainland China:
@scottalanmiller said in Large file sharing to, from and within mainland China:
So here is my thought.... You have one machine IN China that syncs to the NextCloud server in the US (or vice versa, but fewer hassles this way.) The Chinese staff grab the file from there and copy it around however they need. They always save changes back to that location, though. So the latest copy is always on the machine syncing to NextCloud. This machine will start sending the updated copy to the US the moment that it is saved, but the Chinese staff get to keep working.
Then the NextCloud server just has to be backed up as usual in the US.
What system would I put there? A Windows desktop OS box?
Any system. Raspberry Pi, Linux desktop, Windows desktop, Windows server.. whatever you like.
-
@wrx7m said in Large file sharing to, from and within mainland China:
We use 7zip and RAR files a lot, but zip is the most ubiquitous and universal when dealing with the factories and vendors.
You aren't dealing with factories and vendors, you are dealing with internal staff going between US and China. 7Zip cranked to Ultimate would make a lot more sense here.
-
@scottalanmiller said in Large file sharing to, from and within mainland China:
@wrx7m said in Large file sharing to, from and within mainland China:
We use 7zip and RAR files a lot, but zip is the most ubiquitous and universal when dealing with the factories and vendors.
You aren't dealing with factories and vendors, you are dealing with internal staff going between US and China. 7Zip cranked to Ultimate would make a lot more sense here.
We are dealing with internal staff but they would also need to be able to get these files to the factories via some sort of electronic means. How do my China users get the info to the Chinese factories/vendors (within China). It seems that I am still missing a piece of the puzzle.
-
@wrx7m said in Large file sharing to, from and within mainland China:
@scottalanmiller said in Large file sharing to, from and within mainland China:
@wrx7m said in Large file sharing to, from and within mainland China:
We use 7zip and RAR files a lot, but zip is the most ubiquitous and universal when dealing with the factories and vendors.
You aren't dealing with factories and vendors, you are dealing with internal staff going between US and China. 7Zip cranked to Ultimate would make a lot more sense here.
We are dealing with internal staff but they would also need to be able to get these files to the factories via some sort of electronic means. How do my China users get the info to the Chinese factories/vendors (within China). It seems that I am still missing a piece of the puzzle.
They can do anything that they want. Send them in email, provide them from a web server. Deliver then on USB stick. Access them via FTP. Doesn't matter, that's all "after this point." We have to solve getting the data to China first. They can get it around China in numerous ways once it is there.
-
@scottalanmiller Unfortunately, I must also provide them with the means to do that. Maybe they can use one of the Chinese government approved large file transfer services.
-
@wrx7m said in Large file sharing to, from and within mainland China:
@scottalanmiller Unfortunately, I must also provide them with the means to do that. Maybe they can use one of the Chinese government approved large file transfer services.
I understand that you need to do it, I'm just saying that it happens after the NextCloud stage. That's to get the data between the two places. Getting it around in China is likely very easy.
-
@scottalanmiller True but my OP was also for that part
-
@wrx7m said in Large file sharing to, from and within mainland China:
@scottalanmiller True but my OP was also for that part
You can do anything you want once past the NextCloud step. The file is inside China. So the sky is the limit - anything you are allowed and/or willing to do. You have bandwidth at that point or can just walk the files around.
-
@wrx7m said in Large file sharing to, from and within mainland China:
@msff-amman-Itofficer said in Large file sharing to, from and within mainland China:
IF they are not so tech users, dont let them use the desktop client. Just let them use the web based app.
The desktop client maybe faster, but I have NextCloud in Prod with Onlyoffice integration and the desktop app keeps asking them question that they fail to understand especially when a folder is cleared.
Also cause it tries to keep copy offline on end user machine it can create headache for users working on same files, since you said few clients just let them work directly using the web.
Thanks for the information. I don't know how many people would be working on the same files at the same time but I could see how that would be an issue.
Also, I wonder how much of an issue any language barrier would be. Does nextcloud have a Mandarin translation option?
You have CAD and video. So office integration would be pointless.
As for the prompts that @msff-amman-Itofficer mentioned, a little pre thought in folder structure will prevent that. The only time the client prompts about something is when you completely empty a root folder. So if you plan the folder structure with that in mind you will not encounter it.
-
True, but it depends on the users, right ? and i am blessed with my users who cant even bypass HTTPS SSL warning for internal sites.
