XenServer Export Performance Seems Poor
-
@scottalanmiller said:
@Dashrender said:
As it was explained to me, there aren't user permissions in the system that allow full reading without also allowing for some level of writing.
So this is a database that the vendor does not control? How does such a weird situation arise?
I don't know what you mean?
We bought a product called Clinician. It was bought/sold 3 times while we were using it. The last company bought it and killed it, of course in the hopes that we (and the rest) would just jump onto the new owner's main product - which we did not do.
-
@Dashrender said:
@scottalanmiller said:
@Dashrender said:
As it was explained to me, there aren't user permissions in the system that allow full reading without also allowing for some level of writing.
So this is a database that the vendor does not control? How does such a weird situation arise?
I don't know what you mean?
We bought a product called Clinician. It was bought/sold 3 times while we were using it. The last company bought it and killed it, of course in the hopes that we (and the rest) would just jump onto the new owner's main product - which we did not do.
I mean, if you don't want the system to be writeable... that's up to you, not up to the product, right? So if it is read only, you can make it so. That the product requires the ability to write seems to be inconsequential, how would it do that? You can make the system either refuse writes or just make it roll back on reboot. In either case, you have no needs to treat it any way other than RO.
-
@scottalanmiller said:
If you have it running on a VM, surely you can make it read only from a highly level so that there is no need to write?
I don't understand this either.
The VM presents an IIS .Net website that only works in IE with a back end of SQL 2008 Server.
Are you proposing that I could either
- make the VM read only
- make the SQL DB read only
- make the IIS site read only
In all of these cases I have to assume that making it read only would break the way it works - I don't know this, but I'm assuming this.
-
@Dashrender said:
Are you proposing that I could either
- make the VM read only
- make the SQL DB read only
- make the IIS site read only
In all of these cases I have to assume that making it read only would break the way it works - I don't know this, but I'm assuming this.
All, lock it down. If the data is read only, you don't want it changing, ever, right? So stop it from changing. IIS should be read only anyway, why would that not be stateless? The DB is probably doing something, but what do you care? You want it read only. So just roll it back with every reboot or forbid it to write at all.
-
@scottalanmiller said:
@Dashrender said:
@scottalanmiller said:
@Dashrender said:
As it was explained to me, there aren't user permissions in the system that allow full reading without also allowing for some level of writing.
So this is a database that the vendor does not control? How does such a weird situation arise?
I don't know what you mean?
We bought a product called Clinician. It was bought/sold 3 times while we were using it. The last company bought it and killed it, of course in the hopes that we (and the rest) would just jump onto the new owner's main product - which we did not do.
I mean, if you don't want the system to be writeable... that's up to you, not up to the product, right? So if it is read only, you can make it so. That the product requires the ability to write seems to be inconsequential, how would it do that? You can make the system either refuse writes or just make it roll back on reboot. In either case, you have no needs to treat it any way other than RO.
Oh well sure - I can definitely make it role back on reboots - though, now that I think about it.. I can't do that either.. because while I want the data itself to be read only - I need access logs. Those logs are part of the system itself. Now those logs are in the SQL DB, and I could just backup the SQL DB or find a way to export them and only backup that part... then worry less about the rest...
-
I think what @scottalanmiller is saying is similar to what I was asking.
Why not make the VM "static" pushing all of the files off to something else. This way you're protecting the VM while still providing a good way to recover should something go sideways.
-
@Dashrender said:
Oh well sure - I can definitely make it role back on reboots - though, now that I think about it.. I can't do that either.. because while I want the data itself to be read only - I need access logs. Those logs are part of the system itself. Now those logs are in the SQL DB, and I could just backup the SQL DB or find a way to export them and only backup that part... then worry less about the rest...
So it is NOT read only, hence the problem.
-
@scottalanmiller said:
@Dashrender said:
Oh well sure - I can definitely make it role back on reboots - though, now that I think about it.. I can't do that either.. because while I want the data itself to be read only - I need access logs. Those logs are part of the system itself. Now those logs are in the SQL DB, and I could just backup the SQL DB or find a way to export them and only backup that part... then worry less about the rest...
So it is NOT read only, hence the problem.
Continuing discussion has brought this to the surface.. I wasn't intentionally just not mentioning it earlier.. so yeah... that part at least is not read only.
-
@DustinB3403 said:
Why not make the VM "static" pushing all of the files off to something else. This way you're protecting the VM while still providing a good way to recover should something go sideways.
But that amount of data is still going to take a while to recover, regardless of where it is.
Granted, you can get the applciation VM back up and start restoring the important pieces of the data. Is that what you mean?
-
@Dashrender said:
@scottalanmiller said:
@Dashrender said:
Oh well sure - I can definitely make it role back on reboots - though, now that I think about it.. I can't do that either.. because while I want the data itself to be read only - I need access logs. Those logs are part of the system itself. Now those logs are in the SQL DB, and I could just backup the SQL DB or find a way to export them and only backup that part... then worry less about the rest...
So it is NOT read only, hence the problem.
Continuing discussion has brought this to the surface.. I wasn't intentionally just not mentioning it earlier.. so yeah... that part at least is not read only.
Can the logs just go elsewhere? ELK for example?
-
@scottalanmiller said:
@Dashrender said:
@scottalanmiller said:
@Dashrender said:
Oh well sure - I can definitely make it role back on reboots - though, now that I think about it.. I can't do that either.. because while I want the data itself to be read only - I need access logs. Those logs are part of the system itself. Now those logs are in the SQL DB, and I could just backup the SQL DB or find a way to export them and only backup that part... then worry less about the rest...
So it is NOT read only, hence the problem.
Continuing discussion has brought this to the surface.. I wasn't intentionally just not mentioning it earlier.. so yeah... that part at least is not read only.
Can the logs just go elsewhere? ELK for example?
If I pay a developer to learn how it works - sure it could.
-
Where are the logs going now?
-
If I recall correctly @Dashrender said the files on this server aren't the critical point for it, IE they are used when they are created and then put away into the storage on the VM.
Which is that's the case, why not limit the size of the VM, allowing for a faster recovery of the VM, and then piece meal restore the data as it's needed from something like a Synology NAS.
My point is, the VM as describe is only 700GB because it was allowed to grow to this size, but it could be a meager 150GB.
-
@scottalanmiller said:
Where are the logs going now?
Into the SQL DB on the server.
same place where the EHR data lives. -
@DustinB3403 said:
If I recall correctly @Dashrender said the files on this server aren't the critical point for it, IE they are used when they are created and then put away into the storage on the VM.
Which is that's the case, why not limit the size of the VM, allowing for a faster recovery of the VM, and then piece meal restore the data as it's needed from something like a Synology NAS.
My point is, the VM as describe is only 700GB because it was allowed to grow to this size, but it could be a meager 150GB.
This is not correct - I guess there was a misunderstanding somewhere.
-
@Dashrender said:
@scottalanmiller said:
Where are the logs going now?
Into the SQL DB on the server.
same place where the EHR data lives.A developer could very quickly make a little component that takes those logs and outputs to a text file. I mean, realistically, you could do this with a one line script - just one SQL query going out to file. ELK will grab the file and boom, all done.
-
This server has a 60 GB SQL db, 500+ GB of TIF (scanned in paper documents) and another 100+ of application and other files associated with the old EHR.
At this point in time, the only thing changing on this system should be the access logs - who's logging in, who they are searching for, etc. The data in the DB and the TIF files, etc should all remain static.
The system (other than the log growth) should not be growing. It has around 50 GB of free space currently. This should be a lifetime of space since the main data isn't growing anymore.
-
So @Dashrender do you need the static data on the VM including everything that makes up the 700GB to function?
Or can all of the extra stuff get pushed off to something else?
If the goal is to ensure the VM boots, and the database is accessible, then you should reduce the size of the VM as much as possible.
Anything that is static and that can get moved out of it, I would imagine should be, so you could recover from a faulty OS update that much more quickly.
-
@scottalanmiller said:
@Dashrender said:
@scottalanmiller said:
Where are the logs going now?
Into the SQL DB on the server.
same place where the EHR data lives.A developer could very quickly make a little component that takes those logs and outputs to a text file. I mean, realistically, you could do this with a one line script - just one SQL query going out to file. ELK will grab the file and boom, all done.
I'm guessing that you're assuming that all of the logs are in a single table - and assuming that's true, then I agree with you.
-
@DustinB3403 said:
So @Dashrender do you need the static data on the VM including everything that makes up the 700GB to function?
yes - if anything on there is removed (or not mapped into it) the whole thing doesn't function as it should.
