Tech Forum

We are running MLS 3.3.1. Something I've noticed ever since the CUBS system was implemented back in Oct 2010 was the size of the backup file has been increasing week after week. Last year, prior to CUBS, our backup files were consistently just over 2 Megabytes in size, while today's backup size is just over 9 Megabytes. After CUBS, they have been growing consistently week after week..

Not sure if this is due to extra information being maintained within the Finance portion of MLS, or if they is related to additional debug information that was added. Is anyone able to provide some insight into the file growth rate, and should this be a concern?

wpierson wrote:We are running MLS 3.3.1. Something I've noticed ever since the CUBS system was implemented back in Oct 2010 was the size of the backup file has been increasing week after week. Last year, prior to CUBS, our backup files were consistently just over 2 Megabytes in size, while today's backup size is just over 9 Megabytes. After CUBS, they have been growing consistently week after week..

Not sure if this is due to extra information being maintained within the Finance portion of MLS, or if they is related to additional debug information that was added. Is anyone able to provide some insight into the file growth rate, and should this be a concern?

All Financial Statements are now included in the backup. That means that if you don't ever clean out the financial statements, that factor alone will lead to ever-increasing backup sizes. If you've printed out the statements, you can delete them and reduce the size of your backups.

Backup size will also increase throughout the year as transactions are created (both manually created in MLS and automatically downloaded from CUBS). The room taken by regular transactions will immediately shrink by about 25% in January when you close the fiscal year, and then start gradually growing again. But I think the biggest factor in backup size is the financial statements, which you can control by doing some purging of older financial statements.

What aebrown says it accurate - I recently ran into this issue, as I hadn't ever deleted an old statement and had all of them back to Jan 2007. My backup file was 12GB and backups were very slow... and getting larger/slower each month.

I since removed all but my last 2 or 3 statements and my backup file is now about 2.5GB.

Given we can request and get past statements it seems like a bug to me that MLS includes statements in the backup.

ckmcdonald wrote: My backup file was 12GB and backups were very slow... and getting larger/slower each month.

I since removed all but my last 2 or 3 statements and my backup file is now about 2.5GB.

I'm assuming you actually mean megabytes, not gigabytes.

Storing this much data is not as much a problem as inefficient writing of data to the filesystem, which unfortunately IS a current bug within MLS. For some reason the backup is not buffering the writes and as a result each byte is written out, one by one, which means for my 19+MB backup (with just this year's financial statements) I wait three solid minutes as the computer does nineteen MILLION write operations. Buffering would easily improve this as I can write the same amount of data in less-than one second using other utilities.

I have reported this bug to MLS Support but so far am being told that it's just me and not something they can fix, which probably means I've reached the frontline support person. Feel free to e-mail your own situations so they don't think it's a one-off, environment-specific issue. I explained all of the above in my e-mail just the last couple of days after proving the problem by watching the filesystem operations.

dajoker wrote:I have reported this bug to MLS Support but so far am being told that it's just me and not something they can fix, which probably means I've reached the frontline support person. Feel free to e-mail your own situations so they don't think it's a one-off, environment-specific issue. I explained all of the above in my e-mail just the last couple of days after proving the problem by watching the filesystem operations.

I haven't taken the time to watch the file system operations, but I can confirm that the backup takes an outrageously long time. For our stake backup, which is around 8MB right now, it takes over two minutes to write. Unbuffered writes could well explain that.

aebrown wrote:I haven't taken the time to watch the file system operations, but I can confirm that the backup takes an outrageously long time. For our stake backup, which is around 8MB right now, it takes over two minutes to write. Unbuffered writes could well explain that.

I've noticed this too. As a result, when we make our off-site backups, I will let MLS make the first, and then simply copy and paste the file to the USB drives. It saves us around three minutes per backup.

allenjpl wrote:I've noticed this too. As a result, when we make our off-site backups, I will let MLS make the first, and then simply copy and paste the file to the USB drives. It saves us around three minutes per backup.

I think what you're saying is let MLS make the backup to the local hard drive and then copy it to the external.

Since the internal drive usually has write caching and external drives do not, I was just about to suggest that.

RussellHltn wrote:Since the internal drive usually has write caching and external drives do not, I was just about to suggest that.

That helps a bit, but even backing up to an internal hard drive takes far longer than it should. It might take the 3 minute backup time down to 2 minutes, but it should really only take a few seconds if it were done efficiently.

Agreed. This has nothing to do with the media and everything to do with the way the writes are happening. When first getting trained the old clerk was doing the same thing... backing up to each USB drive (and the hard drive) directly from MLS. The first time I showed him backing up once to the hard drive or USB drive and then copying to the other media (no matter which order I did it) he didn't even think it'd worked because of how fast it was. The fact of the matter is that MLS is broken in how it writes out backups and the fix is literally a one-liner. After disassembling (probably the wrong term in Java land, but oh well) the classes last night I'm pretty sure, given the source code, I could fix it in two minutes, and I sent the examples on how to do that to MLS Support after doing so. Unfortunately the reply I received was "your slowness is actually normalness, and therefore there must not be a problem."

Hopefully they're forwarding on my pestering e-mails to developers who can understand the issue.

A snippet from the e-mail I sent last night for others to verify and see, without a doubt, that if nothing else the backup can be faster. How much faster will depend on exactly how the backup gets (not writes) its data.... for example, if the read is also unbuffered.

<quote>
More information to make this as painless as possible for the programmers.

http://www.java2s.com/Code/Java/File-In ... rmance.htm

I was able to compile the code on my system and modify it to match the size of my backup from MLS and duplicate the symptom seen very closely as far as the time to write data to a file. Results from the code above (with 19,900,000 as the number of bytes matching the size of my backups currently) returned the following output proving the performance benefit attained through proper coding:

ab@mybox00:~/Desktop> java BufferDiff
Write file unbuffered: 33,766ms
Write file buffered: 456ms
</quote>