Splitting or truncating-Converting big psdata file to mat

Forum for discussing PicoScope version 6 (non-automotive version)
Post Reply
Bmda
Newbie
Posts: 0
Joined: Fri Oct 27, 2017 4:28 pm

Splitting or truncating-Converting big psdata file to mat

Post by Bmda » Fri Oct 27, 2017 4:57 pm

Hi,
We are having trouble converting some large psdata files to mat format. The file properties are as follows:
picoscope: 6: 6.12.5.2458
No samples: 125,000,004
sample interval: 15 bits

When trying to load the mat file from matlab, we got some error saying "error in using load". The reason we need to access the binary file is that we need to do some post-data-collection analysis such as curve fitting.

Is there a way to handle the conversion of such large files in picoscope 6 --- Such as, when the sample number is over the max number allowed by the Mat format, picoscope 6 can offer some options such as splitting the raw file to two smaller files, and the user can stitch them back together using matlab or other software? Or even just truncating the data when saving as matlab files.

We would greatly appreciate your help.
Thanks.

Hitesh
Site Admin
Site Admin
Posts: 2793
Joined: Tue May 31, 2011 3:43 pm
Location: St. Neots, Cambridgeshire

Re: Splitting or truncating-Converting big psdata file to ma

Post by Hitesh » Mon Oct 30, 2017 12:49 pm

Hi bmda,

The issue is that MATLAB is unable to handle arrays from .mat v4 format files with more than 100 million elements.

Could you please e-mail support@picotech.com and we will look to assist you further.

Regards,
Hitesh

Technical Support Engineer

Bmda
Newbie
Posts: 0
Joined: Fri Oct 27, 2017 4:28 pm

Re: Splitting or truncating-Converting big psdata file to ma

Post by Bmda » Mon Oct 30, 2017 3:10 pm

Hi Hitech,

Thanks for your reply. I solved this by first converting the mat4 data from picoscope to mat 5 format using Python. Then I can load the data in matlab with no problem. However, given the number of traces we need to collect per experiment (>100 or so), this process is tedious and time consuming. I wonder if 'saving as Mat 5 file' can be included in the next update of the picoscope 6.

Thanks,
Bmda

bennog
Advanced User
Advanced User
Posts: 106
Joined: Mon Nov 26, 2012 9:16 am
Location: Netherlands

Re: Splitting or truncating-Converting big psdata file to ma

Post by bennog » Tue Oct 31, 2017 6:54 am

If i remember correctly scilab 64 bit can read the v4 files without problem.
It should also be possible to save them again in v5 format (I think)

Benno

Bmda
Newbie
Posts: 0
Joined: Fri Oct 27, 2017 4:28 pm

Re: Splitting or truncating-Converting big psdata file to ma

Post by Bmda » Tue Oct 31, 2017 3:44 pm

Thanks Bento. I haven't used Scilab before, but I'll check it out. Bmda

lab!fyi
Active User
Active User
Posts: 38
Joined: Mon Dec 12, 2016 6:58 pm
Contact:

Re: Splitting or truncating-Converting big psdata file to ma

Post by lab!fyi » Wed Nov 01, 2017 1:42 pm

I had similar issues. Just as proof of concept did download GNU Octave and it can open Matlab v4 files and save as v7 for example:
>> load -v4 "somefile.mat"
>> save -v7 "somefile_v7.mat"
Think this could be done from command line also by executing script. Currently no time to investigate further.

Post Reply