Test and Measurement Forum

Splitting or truncating-Converting big psdata file to mat

Forum for discussing PicoScope version 6 (non-automotive version)

Splitting or truncating-Converting big psdata file to mat

Postby Bmda » Fri Oct 27, 2017 4:57 pm

Hi,
We are having trouble converting some large psdata files to mat format. The file properties are as follows:
picoscope: 6: 6.12.5.2458
No samples: 125,000,004
sample interval: 15 bits

When trying to load the mat file from matlab, we got some error saying "error in using load". The reason we need to access the binary file is that we need to do some post-data-collection analysis such as curve fitting.

Is there a way to handle the conversion of such large files in picoscope 6 --- Such as, when the sample number is over the max number allowed by the Mat format, picoscope 6 can offer some options such as splitting the raw file to two smaller files, and the user can stitch them back together using matlab or other software? Or even just truncating the data when saving as matlab files.

We would greatly appreciate your help.
Thanks.
Bmda
Newbie
 
Posts: 0
Joined: Fri Oct 27, 2017 4:28 pm

Re: Splitting or truncating-Converting big psdata file to ma

Postby Hitesh » Mon Oct 30, 2017 12:49 pm

Hi bmda,

The issue is that MATLAB is unable to handle arrays from .mat v4 format files with more than 100 million elements.

Could you please e-mail support@picotech.com and we will look to assist you further.

Regards,
Hitesh

Technical Support Engineer
Hitesh
Site Admin
Site Admin
 
Posts: 2498
Joined: Tue May 31, 2011 3:43 pm
Location: St. Neots, Cambridgeshire

Re: Splitting or truncating-Converting big psdata file to ma

Postby Bmda » Mon Oct 30, 2017 3:10 pm

Hi Hitech,

Thanks for your reply. I solved this by first converting the mat4 data from picoscope to mat 5 format using Python. Then I can load the data in matlab with no problem. However, given the number of traces we need to collect per experiment (>100 or so), this process is tedious and time consuming. I wonder if 'saving as Mat 5 file' can be included in the next update of the picoscope 6.

Thanks,
Bmda
Bmda
Newbie
 
Posts: 0
Joined: Fri Oct 27, 2017 4:28 pm

Re: Splitting or truncating-Converting big psdata file to ma

Postby bennog » Tue Oct 31, 2017 6:54 am

If i remember correctly scilab 64 bit can read the v4 files without problem.
It should also be possible to save them again in v5 format (I think)

Benno
bennog
Advanced User
Advanced User
 
Posts: 91
Joined: Mon Nov 26, 2012 9:16 am
Location: Netherlands

Re: Splitting or truncating-Converting big psdata file to ma

Postby Bmda » Tue Oct 31, 2017 3:44 pm

Thanks Bento. I haven't used Scilab before, but I'll check it out. Bmda
Bmda
Newbie
 
Posts: 0
Joined: Fri Oct 27, 2017 4:28 pm

Re: Splitting or truncating-Converting big psdata file to ma

Postby lab!fyi » Wed Nov 01, 2017 1:42 pm

I had similar issues. Just as proof of concept did download GNU Octave and it can open Matlab v4 files and save as v7 for example:
>> load -v4 "somefile.mat"
>> save -v7 "somefile_v7.mat"
Think this could be done from command line also by executing script. Currently no time to investigate further.
lab!fyi
Active User
Active User
 
Posts: 21
Joined: Mon Dec 12, 2016 6:58 pm


Return to PicoScope 6 for Windows

Who is online

Users browsing this forum: No registered users and 0 guests