Data log taking up too much memory

Discuss issues and ideas you have to configuring displays with PowerVision
drphil69
Posts: 139
Joined: Wed Mar 02, 2011 5:59 pm

Data log taking up too much memory

Post by drphil69 » Tue Oct 03, 2017 8:56 am

Hello,

A few months back I asked about calculating the data log size, as my customer wanted to know how much/how long he could store date in a PV450.

It appears to take A LOT more memory than I calculated. Here is the explanation of how to estimate that Boyce provided -

"Re: Is there a way to calculate data logger file size in advance?
Postby boyce » Mon Jul 24, 2017 11:10 am

According to the developer, the Data Logger application stores the value of the variables in memory in a bit format. So, the size of each value will depend on the number of digits in the value including the number of decimal digits. Each digit is stored in 4 bits and values with fewer digits will use less space. It is an effective way to compress the log data in memory and makes it easy to convert into the CSV format when storing out to USB. But it makes it hard to calculate the size required because it is dependent on the data. Big numbers and numbers with a lot of decimal digits will use more space in the log file. For the best estimate a sample of the typical data could be recorded and then calculate an average sample size per time interval."


I did some testing by logging data at 5 second intervals for 30, 60 and 90 seconds. There were 33 digits recorded per entry, which should take 132 bits, or 17 bytes. But the actual log size shows it is taking 34 kB per entry and over 1 kB per digit! This is off by a factor of 250! I extrapolated the data which showed a mere 73 minutes of data logging capability @ 30 MB log size.

I have attached the 90 second log file and a spreadsheet of file size extrapolated to 30 MB for your review.

Am I doing something wrong, and how can I improve this without eliminating data or reducing the sample time? Or is the log file size shown on the display wrong? The LogFile is much smaller that what was shown on the display (display - 847 kB; file - 1 kB). Is the file size in bytes maybe?

Thanks,
Phil
Attachments
LogFile.csv
(1.14 KiB) Downloaded 8 times
Data Log Calcs 2.xlsx
(14.01 KiB) Downloaded 8 times
drphil69
Posts: 139
Joined: Wed Mar 02, 2011 5:59 pm

Re: Data log taking up too much memory

Post by drphil69 » Tue Oct 03, 2017 9:15 am

Looking at the variable DataLogger.LogSize - is this in kB or in Bytes? I think that might be my issue!
drphil69
Posts: 139
Joined: Wed Mar 02, 2011 5:59 pm

Re: Data log taking up too much memory

Post by drphil69 » Tue Oct 03, 2017 4:02 pm

OK, after testing for over 4 hours I believe the DataLogger.FileSize is in Bytes.

Thanks,
Phil
boyce
Enovation Controls Development
Enovation Controls Development
Posts: 322
Joined: Wed Sep 08, 2010 5:09 pm

Re: Data log taking up too much memory

Post by boyce » Wed Oct 04, 2017 1:55 pm

The DataLogger.FileSize is in bytes.
Boyce Schrack
Enovation Controls