Test and Measurement Forum

How to export PS6000's big traces (2Gs)

Forum for discussing PicoScope version 6 (non-automotive version)

How to export PS6000's big traces (2Gs)

Postby francois » Fri Aug 05, 2016 3:54 pm

In order to apply signal processings to my acquisitions I need to export files. The .mat format is convenient however I end up with .mat files exceeding 1GB. My usual tools fail to import .mat files of such size. They report an issue "dimension too large". I'm working with Scilab 6b2 but I also tried Octave 4. So I tried to export a part of the signal in order to be able to process the file. Unfortunately the csv export resample the signal which is not compatible with my application. I'm able to store 2G samples but I'm still looking for a way to use this feature. Can you help to solve that issue?
Thanks,
François
francois
Newbie
 
Posts: 0
Joined: Fri Aug 05, 2016 2:29 pm

Re: How to export PS6000's big traces (2Gs)

Postby bennog » Fri Aug 05, 2016 5:12 pm

For this kind of big data you need to use A 64bit os at least 8GB ram and a 64Bit version of Octave or any other program to handle this needs to be 64Bit.
quote from http://www.tatsuromatsuoka.com/octave/Eng/Win/
(working on 64 bit but it does not support large arrays.


Scilab 5.5.2 seems to have a 64Bit version so I would go for this 64bit version
http://www.scilab.org/download/latest

Benno
bennog
Guru
Guru
 
Posts: 79
Joined: Mon Nov 26, 2012 9:16 am

Re: How to export PS6000's big traces (2Gs)

Postby Hitesh » Mon Aug 08, 2016 8:47 am

Hi francois,

The PicoScope 6 software currently exports to MATLAB v4 format.

Unfortunately, MATLAB has an issue whereby it cannot read from arrays containing more than 100 million elements when saved with this format. I am not sure if Octave or SciLab have this issue but you can try it as bennog suggested.

If you encounter any further problems, please e-mail support@picotech.com and we may be able to provide some further assistance with this process.

Regards,
Hitesh

Technical Specialist
Pico Technology
Hitesh
Site Admin
Site Admin
 
Posts: 1997
Joined: Tue May 31, 2011 3:43 pm
Location: St. Neots, Cambridgeshire

Re: How to export PS6000's big traces (2Gs)

Postby francois » Mon Aug 08, 2016 10:00 am

I'm working with Win7 64Bits and tried with Scilab6 b2 64bits and Octave "i686-w64-mingw32". Both platforms understand the structure but tell me that the array dimention is out of range. Maybe the only solution is that PicoScope6 allows to split the export into several parts or allows to export a selection without resampling.
Regards,
François
francois
Newbie
 
Posts: 0
Joined: Fri Aug 05, 2016 2:29 pm

Re: How to export PS6000's big traces (2Gs)

Postby bennog » Mon Aug 08, 2016 11:14 am

Can you post a .mat export from picoscope ? (Dropbox share or web link) file is too big for this forum software
See if I can fix something for you.

Regards,
Benno
bennog
Guru
Guru
 
Posts: 79
Joined: Mon Nov 26, 2012 9:16 am

Re: How to export PS6000's big traces (2Gs)

Postby francois » Mon Aug 08, 2016 12:49 pm

Yes. One example file is here:
https://www.dropbox.com/s/8ct917fg0bukz ... e.mat?dl=0
Regards,
François
francois
Newbie
 
Posts: 0
Joined: Fri Aug 05, 2016 2:29 pm

Re: How to export PS6000's big traces (2Gs)

Postby bennog » Mon Aug 08, 2016 4:00 pm

I got it to work with 'scilab-6.0.0-beta-2 (64-bit)' needs at least 8 or 16 GB of ram (I used 16GB), don't know if 8GB is sufficient. It has 312 million rows of data.
memory.png
Memory usage Win10 64bit
memory.png (3.9 KiB) Viewed 225 times

variables.png
variable list from the editor
variables.png (5.05 KiB) Viewed 225 times

commands.png
used commands in SciLab
commands.png (4.29 KiB) Viewed 225 times


I exported it to CSV because the mat V5 V6 and V7 export did not work (app crashes etc.)

I saved the CSV so you can download it (it is a winrar compressed CSV file)
http://www.ans-net.nl/picotech/Example-to-CSV.rar

Regards,
Benno
bennog
Guru
Guru
 
Posts: 79
Joined: Mon Nov 26, 2012 9:16 am

Re: How to export PS6000's big traces (2Gs)

Postby francois » Tue Aug 09, 2016 9:10 am

Thank you bennog,
Characteristics of the signal seems to be accurately retreived by Scilab. It seems to be a solution if enough RAM is available on the host PC.In order to avoid increasing RAM on all our PC it would be good to have a solution from Picoscope with an option to export signals by parts or selection of parts. This without signal resampling as I have seen it. Why this behavior ? When such an option could be available from Picoscope 6?
Thanks,
François
francois
Newbie
 
Posts: 0
Joined: Fri Aug 05, 2016 2:29 pm

Re: How to export PS6000's big traces (2Gs)

Postby Hitesh » Wed Aug 10, 2016 9:20 am

Hi François,

I have added your comments to a feature request in order to be able to save to split .mat files.

There was a feature that allowed saving of zoomed regions in some previous releases but this was found to have some issues and so is not present in PicoScope 6.11 or the current PicoScope 6.12 Beta. I have asked the relevant persons in the Development Team if this would be reintroduced but it would also depend on the priority of other projects.

Regards,
Hitesh

Technical Specialist
Pico Technology
Hitesh
Site Admin
Site Admin
 
Posts: 1997
Joined: Tue May 31, 2011 3:43 pm
Location: St. Neots, Cambridgeshire


Return to PicoScope 6 for Windows

Who is online

Users browsing this forum: No registered users and 0 guests