Data loss in between waveforms

Having problems ? let us know the details here
Post Reply
Stephanie
User
User
Posts: 3
Joined: Thu Sep 22, 2022 6:17 pm

Data loss in between waveforms

Post by Stephanie »

Problem: Data is lost in the transition from one waveform to the next. The amount of loss is variable.
Setup: Pulsing LED plus photodiode, Picoscope 2204A, Pico 7 on mac & Pico 6 on Windows
Observations:
- The sinusoids do not connect from one waveform to the next when we click through them after the recording
- When we save the data to either .txt files or as .pngs it is very obvious that pulse sequences are missing and the intervals between them are irregular while the intervals between the physical stimuli were constant.
- The problem occurs independent of sampling rate and waveform size. Within a waveform everything looks fine.
Our Goal:
- Find a way in which we can save continuous recordings lasting a few minutes to .txt or .csv without data loss.

bennog
Advanced User
Advanced User
Posts: 206
Joined: Mon Nov 26, 2012 9:16 am
Location: Netherlands

Re: Data loss in between waveforms

Post by bennog »

I suppose you don't use triggers.
There is time needed to transfer data from the scope to the PC (even if you do triggers) except rapid trigger.
But the 2204A has only 8k samples of memory probably not enough for wat you want.

Best solution for you is use streaming mode over a period of a few minutes at the (20 s/div or 50s/div)
And after the data is collected save to a .csv where you will have all the data of the 2-3 minutes as a single csv without gaps.
You can also zoom in on the signal in Pico 7 software and make you .png files.

In timebase set sampling at sample-rate and target sampling rate at 60MS/s
2022-09-23_08-00-27.png
2022-09-23_08-00-27.png (24.8 KiB) Viewed 2246 times
The real sample rate will be at the green arrow, in my case for a 3403D MSO scope that is 625kS/sec (2 ch) or 1250kS/sec (1 ch)
With the 2204A you will have significantly less samples per second because (usb2 and far less onboard memory)

P.S. stop the capture before it is at the end of the screen

Benno

Stephanie
User
User
Posts: 3
Joined: Thu Sep 22, 2022 6:17 pm

Re: Data loss in between waveforms

Post by Stephanie »

Thanks a lot for the fast reply! Dankjewel!
Yes, we do not use triggers but want to constantly stream, save the data, and analyze it in R/matlab.
I think you probably already said it in your postscript but just to confirm: all the samples need to be in one waveform/buffer/window? It is to be expected that we loose samples when it switches from one waveform/buffer/window to the next?

These are .pngs of two consecutive waveforms one can see quite well that the ends do not match.
20220923-0003_2.png
20220923-0003_3.png

bennog
Advanced User
Advanced User
Posts: 206
Joined: Mon Nov 26, 2012 9:16 am
Location: Netherlands

Re: Data loss in between waveforms

Post by bennog »

Yes between captures you lose time because of USB transfer and restart of the capture.
So streaming mode is the way to go. with the SDK you can get higher streaming rates than in the picoscope software.

What are the desired number of samples / second you need for your application.
Seen from your screen shouts you should have enough wen capturing at 100S/sec
2022-09-25_10-36-46.png
Above the settings for 200 Samples/second for a time period of 100sec/div (totaling more than 16 min of sample data)
Also set the progressive mode at 200 ms/div (the default setting is some absurd high value)
Then you can see your data live on the screen and even do some crude zooming on the data.
After you stopped the capturing you can do good and detailed zooming on the wave data.

Benno

Stephanie
User
User
Posts: 3
Joined: Thu Sep 22, 2022 6:17 pm

Re: Data loss in between waveforms

Post by Stephanie »

Thanks so much, Benno!
Our desired sampling rate depends a little bit on what we want to check (a) whether the LED pulse frequency matches our expectations, (b) whether the LED pulses are temporally aligned with sounds we measure in Channel B, and (c) whether the time interval between pulse sequences matches our expectations. My take would now be that for (a) and (b) we record only for one or two minutes with a higher sampling rate and for (c) we record with a lower sampling rate for ten to fifteen minutes to have enough stimuli to judge the timing.

I've got another follow-up question though: you mentioned we could get higher streaming rates with the SDK. Do you mean when we write our own application (or modify ones of those on GitHub) we'll get more samples into one capture/waveform/window or will lose less samples between captures/waveforms/windows? If so, do you have any recommendations regarding an uploaded application or interface / language that works especially well?

Post Reply