PicoScope 7 Software
Available on Windows, Mac and Linux
Streaming data between two buffers
Streaming data between two buffers
Hello everyone
I recently bought a picoscope 5444D and I am planning to use it to store data while performing some tratment simultanously, everything using python. I would like the device to continously store the data between two buffers so that, when buffer 1 fills, the picoscope can start storing on buffer 2 while a different thread treats data in buffer 1, everything in a loop between the buffers. I only have experience using the Streaming Mode, so I don't know whether there are different modes which already do something similar. Can you help me with that?
Thank you in advance and have a nice day!
NPianta
I recently bought a picoscope 5444D and I am planning to use it to store data while performing some tratment simultanously, everything using python. I would like the device to continously store the data between two buffers so that, when buffer 1 fills, the picoscope can start storing on buffer 2 while a different thread treats data in buffer 1, everything in a loop between the buffers. I only have experience using the Streaming Mode, so I don't know whether there are different modes which already do something similar. Can you help me with that?
Thank you in advance and have a nice day!
NPianta
Re: Streaming data between two buffers
What sample rate are you planning on sampling (no of samples per second)
I usually use a single ring buffer with a head and tail index.
Scope is wring incomming samples at the head index. and increases the head index after the write (loop around at the end)
And 2nd thread reading the data from the ringbuffer from the tail index. (until it catches up with the head index then wait a couple of ms and start catching up to the head index again)
Benno
I usually use a single ring buffer with a head and tail index.
Scope is wring incomming samples at the head index. and increases the head index after the write (loop around at the end)
And 2nd thread reading the data from the ringbuffer from the tail index. (until it catches up with the head index then wait a couple of ms and start catching up to the head index again)
Benno
Re: Streaming data between two buffers
Hello!
The sampling frequency is variable but my worst case will be 10 MHz (so 10 millions of samples per second). My aim is to perform an FFT algorithm on a chunk of 10 MSamples, then store the values at some precise frequencies and then cancel my dataset so not to fill the RAM (while maintaining something around 50 samples per second via undersampling the bigger dataset). I made some tests and my PC should be able to handle the data treatment (so FFT + data storage ad different frequencies) in less than a second if performed on a data chunk of 10 MSa. I also need the data storage flow never to stop, so that I can perform other FFT algorithms on the undersampled datasets I'm planning to store.
NPianta
The sampling frequency is variable but my worst case will be 10 MHz (so 10 millions of samples per second). My aim is to perform an FFT algorithm on a chunk of 10 MSamples, then store the values at some precise frequencies and then cancel my dataset so not to fill the RAM (while maintaining something around 50 samples per second via undersampling the bigger dataset). I made some tests and my PC should be able to handle the data treatment (so FFT + data storage ad different frequencies) in less than a second if performed on a data chunk of 10 MSa. I also need the data storage flow never to stop, so that I can perform other FFT algorithms on the undersampled datasets I'm planning to store.
NPianta
Re: Streaming data between two buffers
ok then I would use a ringbuffer of about 25 - 30 Msamples and loop it in streaming mode as in previous post.
Then you can take the last 10M samples at any moment in time by taking the most recent 10Msamples from the head index down (and loop to the end at 0)
In that case if your FFT is fast you can do updates in less than 1 second (or even multiple updates per second)
And if it gets too slow you don't lag behind on the buffers and still have the fastest FFT update rate as possible with the most recent data from the scope. At slower sampling speeds you will doing a FFT on a moving window on the data in this case, and keep a fast update rate of your FFT.
You also can use block mode with blocks of 10Msamples. Then you won't be sampling while getting the block of samples from the scope to the PC. And if your FFT is slower than 1 update / second then the blocks will lag behind and you are doing FFT's on old data (up to 1 second if you are using 2 buffers) and up to 4 seconds if you are using 5 buffers. At slower sampling speeds this will slow down the update rate of the FFT significantly.
That is my opinion about how to approach this challenge.
Benno
Then you can take the last 10M samples at any moment in time by taking the most recent 10Msamples from the head index down (and loop to the end at 0)
In that case if your FFT is fast you can do updates in less than 1 second (or even multiple updates per second)
And if it gets too slow you don't lag behind on the buffers and still have the fastest FFT update rate as possible with the most recent data from the scope. At slower sampling speeds you will doing a FFT on a moving window on the data in this case, and keep a fast update rate of your FFT.
You also can use block mode with blocks of 10Msamples. Then you won't be sampling while getting the block of samples from the scope to the PC. And if your FFT is slower than 1 update / second then the blocks will lag behind and you are doing FFT's on old data (up to 1 second if you are using 2 buffers) and up to 4 seconds if you are using 5 buffers. At slower sampling speeds this will slow down the update rate of the FFT significantly.
That is my opinion about how to approach this challenge.
Benno
Re: Streaming data between two buffers
Thank you very much for the suggestion!
Do you know whether there is a function in picosdk that can be used to access the head index?
Sorry for all the questions but my experience in programming is quite limited (I'm actually a chemist and I barely started using picos).
Thank you again and have a nice day!
NPianta
Do you know whether there is a function in picosdk that can be used to access the head index?
Sorry for all the questions but my experience in programming is quite limited (I'm actually a chemist and I barely started using picos).
Thank you again and have a nice day!
NPianta
Re: Streaming data between two buffers
the SDK streams the data in chunks of various size depending on the pc speed and cpu load.
So you have to make the ringbuffer yourselves (simple buffer of 30 MSamples (60 MB)) and an integer that increases every chunk of data from the SDK with the size of the data from the SDK.
Benno
So you have to make the ringbuffer yourselves (simple buffer of 30 MSamples (60 MB)) and an integer that increases every chunk of data from the SDK with the size of the data from the SDK.
Benno