How to export PS6000's big traces (2Gs)

Forum for discussing PicoScope version 6 (non-automotive version)
Post Reply
francois
Newbie
Posts: 0
Joined: Fri Aug 05, 2016 2:29 pm

How to export PS6000's big traces (2Gs)

Post by francois »

In order to apply signal processings to my acquisitions I need to export files. The .mat format is convenient however I end up with .mat files exceeding 1GB. My usual tools fail to import .mat files of such size. They report an issue "dimension too large". I'm working with Scilab 6b2 but I also tried Octave 4. So I tried to export a part of the signal in order to be able to process the file. Unfortunately the csv export resample the signal which is not compatible with my application. I'm able to store 2G samples but I'm still looking for a way to use this feature. Can you help to solve that issue?
Thanks,
François

bennog
Advanced User
Advanced User
Posts: 206
Joined: Mon Nov 26, 2012 9:16 am
Location: Netherlands

Re: How to export PS6000's big traces (2Gs)

Post by bennog »

For this kind of big data you need to use A 64bit os at least 8GB ram and a 64Bit version of Octave or any other program to handle this needs to be 64Bit.
quote from http://www.tatsuromatsuoka.com/octave/Eng/Win/
(working on 64 bit but it does not support large arrays.
Scilab 5.5.2 seems to have a 64Bit version so I would go for this 64bit version
http://www.scilab.org/download/latest

Benno

Hitesh

Re: How to export PS6000's big traces (2Gs)

Post by Hitesh »

Hi francois,

The PicoScope 6 software currently exports to MATLAB v4 format.

Unfortunately, MATLAB has an issue whereby it cannot read from arrays containing more than 100 million elements when saved with this format. I am not sure if Octave or SciLab have this issue but you can try it as bennog suggested.

If you encounter any further problems, please e-mail support@picotech.com and we may be able to provide some further assistance with this process.

Regards,

francois
Newbie
Posts: 0
Joined: Fri Aug 05, 2016 2:29 pm

Re: How to export PS6000's big traces (2Gs)

Post by francois »

I'm working with Win7 64Bits and tried with Scilab6 b2 64bits and Octave "i686-w64-mingw32". Both platforms understand the structure but tell me that the array dimention is out of range. Maybe the only solution is that PicoScope6 allows to split the export into several parts or allows to export a selection without resampling.
Regards,
François

bennog
Advanced User
Advanced User
Posts: 206
Joined: Mon Nov 26, 2012 9:16 am
Location: Netherlands

Re: How to export PS6000's big traces (2Gs)

Post by bennog »

Can you post a .mat export from picoscope ? (Dropbox share or web link) file is too big for this forum software
See if I can fix something for you.

Regards,
Benno

francois
Newbie
Posts: 0
Joined: Fri Aug 05, 2016 2:29 pm

Re: How to export PS6000's big traces (2Gs)

Post by francois »

Yes. One example file is here:
https://www.dropbox.com/s/8ct917fg0bukz ... e.mat?dl=0
Regards,
François

bennog
Advanced User
Advanced User
Posts: 206
Joined: Mon Nov 26, 2012 9:16 am
Location: Netherlands

Re: How to export PS6000's big traces (2Gs)

Post by bennog »

I got it to work with 'scilab-6.0.0-beta-2 (64-bit)' needs at least 8 or 16 GB of ram (I used 16GB), don't know if 8GB is sufficient. It has 312 million rows of data.
Memory usage Win10 64bit
Memory usage Win10 64bit
memory.png (3.9 KiB) Viewed 9754 times
variable list from the editor
variable list from the editor
variables.png (5.05 KiB) Viewed 9754 times
used commands in SciLab
used commands in SciLab
commands.png (4.29 KiB) Viewed 9754 times
I exported it to CSV because the mat V5 V6 and V7 export did not work (app crashes etc.)

I saved the CSV so you can download it (it is a winrar compressed CSV file)
http://www.ans-net.nl/picotech/Example-to-CSV.rar

Regards,
Benno

francois
Newbie
Posts: 0
Joined: Fri Aug 05, 2016 2:29 pm

Re: How to export PS6000's big traces (2Gs)

Post by francois »

Thank you bennog,
Characteristics of the signal seems to be accurately retreived by Scilab. It seems to be a solution if enough RAM is available on the host PC.In order to avoid increasing RAM on all our PC it would be good to have a solution from Picoscope with an option to export signals by parts or selection of parts. This without signal resampling as I have seen it. Why this behavior ? When such an option could be available from Picoscope 6?
Thanks,
François

Hitesh

Re: How to export PS6000's big traces (2Gs)

Post by Hitesh »

Hi François,

I have added your comments to a feature request in order to be able to save to split .mat files.

There was a feature that allowed saving of zoomed regions in some previous releases but this was found to have some issues and so is not present in PicoScope 6.11 or the current PicoScope 6.12 Beta. I have asked the relevant persons in the Development Team if this would be reintroduced but it would also depend on the priority of other projects.

Regards,

Post Reply