5444D - Increasing delay in streaming

Post discussions on projects you are working on
Post Reply
potizakus
Newbie
Posts: 0
Joined: Sat May 30, 2020 9:48 am

5444D - Increasing delay in streaming

Post by potizakus »

Hi,

I have developed a program in Python derived from the ps5000aStreamingExample.py (available online) to acquire streaming data from Picoscope 5444D and process them in an infinite loop.

The issue is that despite I take the data every 200ms (sizeofonebuffer = 500, numberofbufferstocapture=20, sampleinterval = 20us) and therefore I should get 5 packages per second, I only get two.

What's more, even though I take the data every 200ms and in the meantime, I do the processing (that takes only 14ms) I notice a delay in the data:
a) if I start the program and immediately change the input signal, I immediately see the change in the obtained data
b) if I start the program, wait 10 minutes and change the input signal, I see the change in the measured data with a delay of approximately 30 seconds
c) if I start the program, wait 10 hours (or so), the program starts showing weird data that does not correspond with the input signal, and the processing therefore fails.

Could you please give me an insight into what could be wrong or what am I missing?

Thanks very much in advance
Potizakus

Martyn
Site Admin
Site Admin
Posts: 4499
Joined: Fri Jun 10, 2011 8:15 am
Location: St. Neots

Re: 5444D - Increasing delay in streaming

Post by Martyn »

The PicoScope 5444D can stream at up to 125 MS/s (8-bit) or 62.5 MS/s (12 to 16 bit modes) shared between active channels so I don't believe there is an issue with the device.

The problem is likely to be with the way you are handling the data when reading it from the driver and into your application. When you start streaming two circular buffers are created in the driver, one will be of size preTriggerSamples plus postTriggerSamples and will hold data for reading after the scope has stopped collecting if you are using the auto stop function as the PicoScope 6 software does, and is not relevant to you. For your case the other buffer is the important one, it is the one that you created with GetDataBuffer(s) and you need to be reading from this buffer using GetStreamingLatestValues at a sufficient rate that it does not fill up. For your particular application I would be copying data from this buffer into a linked list of buffers of the exact size you require for each chunk of data and when full they can be processed by your application, before being added back to the pool to use again.

I will leave further follow up for the email ticket you created in our Helpdesk.
Martyn
Technical Support Manager

potizakus
Newbie
Posts: 0
Joined: Sat May 30, 2020 9:48 am

Re: 5444D - Increasing delay in streaming

Post by potizakus »

Hi Martyn,

thank you for your reply! :) I have tried to disable and comment out all the processing code leaving only a very simple loop consisting of ps.ps5000aGetStreamingLatestValues(chandle, cFuncPtr, None) and printing the current time.

Yet the samples came in these intervals: (they should come every 200ms)
2020_06_04-07_14_38.569000
2020_06_04-07_14_38.864000
2020_06_04-07_14_39.335000
2020_06_04-07_14_39.708000
2020_06_04-07_14_40.282000
2020_06_04-07_14_40.602000
2020_06_04-07_14_41.058000
2020_06_04-07_14_41.432000
2020_06_04-07_14_41.857000
2020_06_04-07_14_42.187000
2020_06_04-07_14_42.419000
2020_06_04-07_14_42.868000
2020_06_04-07_14_43.224000
2020_06_04-07_14_43.660000

Is not there some settings that I forgot to tune? For now I am using
sampleInterval = ctypes.c_int32(20)
sampleUnits = ps.PS5000A_TIME_UNITS['PS5000A_US']

sizeOfOneBuffer = 500
numBuffersToCapture = 20

totalSamples = sizeOfOneBuffer * numBuffersToCapture

Martyn
Site Admin
Site Admin
Posts: 4499
Joined: Fri Jun 10, 2011 8:15 am
Location: St. Neots

Re: 5444D - Increasing delay in streaming

Post by Martyn »

You can't guarantee to receive blocks of samples at fixed intervals, however you can guarantee that all samples will have been collected at the correct sampling interval on the device.

This is due to the buffering mechanism on the device and the transfer process from the device over the USB bus to the driver and then on to the application. Control of the USB bus is managed by the operating system so the time available to transfer data from the device to the driver will vary. You can be assured that no data samples will be lost, but the data transfer will consist of bursts but as the sampling rate on the device is significantly lower than the data transfer rate over the USB there should be no data loss. It will be the responsibility of the application to read data from the driver at a suitable rate and then package this into it's own buffers for processing. It may take more than one call to the driver to collect sufficient data to fill an application buffer, but that will depend upon how well the main loop of the program is structured and how efficient any data processing is.
Martyn
Technical Support Manager

potizakus
Newbie
Posts: 0
Joined: Sat May 30, 2020 9:48 am

Re: 5444D - Increasing delay in streaming

Post by potizakus »

Hi Martyn,

I understand that I have to read at a suitable rate, yet I don't know how to ensure it. In my main loop is now just

Code: Select all

	while(True):
        # Fetch data from the driver in a loop, copying it out of the registered buffers and into our complete 
	# one.
        while nextSample < totalSamples and not autoStopOuter:
            wasCalledBack = False
            status["getStreamingLastestValues"] = ps.ps5000aGetStreamingLatestValues(chandle, cFuncPtr, None)
            if not wasCalledBack:
                # If we weren't called back by the driver, this means no data is ready. Sleep for a short while 		 
                # before trying again.
                time.sleep(0.01)
                
        # Print time
	now = datetime.now()
	print(now.strftime("%Y_%m_%d-%H_%M_%S.%f")) 
But as you say, the data is still read too slow.


Can you please advise me, is there some method to speed up the reading or to ensure minimum delay?

Thank you very much sir!

Martyn
Site Admin
Site Admin
Posts: 4499
Joined: Fri Jun 10, 2011 8:15 am
Location: St. Neots

Re: 5444D - Increasing delay in streaming

Post by Martyn »

Really need to see the full code to be able to comment
Martyn
Technical Support Manager

potizakus
Newbie
Posts: 0
Joined: Sat May 30, 2020 9:48 am

Re: 5444D - Increasing delay in streaming

Post by potizakus »

Hi,

in the following block I attach the full simplified code:

Code: Select all

# -*- coding: cp1250 -*-

# Copyright (C) 2018-2019 Pico Technology Ltd. See LICENSE file for terms.
# PS5000 Series (A API) STREAMING MODE EXAMPLE
# This example demonstrates how to call the ps5000a driver API functions in order to open a device, setup 2 channels and collects streamed data (1 buffer).
# This data is then plotted as mV against time in ns.

import ctypes
import numpy as np
from picosdk.ps5000a import ps5000a as ps
import matplotlib.pyplot as plt
from picosdk.functions import adc2mV, assert_pico_ok
import time
from datetime import datetime

if __name__ == '__main__':
    # Create chandle and status ready for use
    chandle = ctypes.c_int16()
    status = {}

    # Open PicoScope 5000 Series device
    # Resolution set to 12 Bit
    resolution = ps.PS5000A_DEVICE_RESOLUTION["PS5000A_DR_15BIT"]
    # Returns handle to chandle for use in future API functions
    status["openunit"] = ps.ps5000aOpenUnit(ctypes.byref(chandle), None, resolution)

    try:
        assert_pico_ok(status["openunit"])
    except: # PicoNotOkError:

        powerStatus = status["openunit"]

        if powerStatus == 286:
            status["changePowerSource"] = ps.ps5000aChangePowerSource(chandle, powerStatus)
        elif powerStatus == 282:
            status["changePowerSource"] = ps.ps5000aChangePowerSource(chandle, powerStatus)
        else:
            raise

        assert_pico_ok(status["changePowerSource"])


    enabled = 1
    disabled = 0
    analogue_offset = 0.0

    # Set up channel A
    # handle = chandle
    # channel = PS5000A_CHANNEL_A = 0
    # enabled = 1
    # coupling type = PS5000A_DC = 1
    # range = PS5000A_2V = 7
    # analogue offset = 0 V
    channel_range = ps.PS5000A_RANGE['PS5000A_10V']
    status["setChA"] = ps.ps5000aSetChannel(chandle,
                                            ps.PS5000A_CHANNEL['PS5000A_CHANNEL_A'],
                                            enabled,
                                            ps.PS5000A_COUPLING['PS5000A_DC'],
                                            channel_range,
                                            analogue_offset)
    assert_pico_ok(status["setChA"])

    # Set up channel B
    # handle = chandle
    # channel = PS5000A_CHANNEL_B = 1
    # enabled = 1
    # coupling type = PS5000A_DC = 1
    # range = PS5000A_2V = 7
    # analogue offset = 0 V
    channel_range = ps.PS5000A_RANGE['PS5000A_50MV']
    status["setChB"] = ps.ps5000aSetChannel(chandle,
                                            ps.PS5000A_CHANNEL['PS5000A_CHANNEL_B'],
                                            enabled,
                                            ps.PS5000A_COUPLING['PS5000A_DC'],
                                            channel_range,
                                            analogue_offset)
    assert_pico_ok(status["setChB"])

    # Size of capture
    sizeOfOneBuffer = 500
    numBuffersToCapture = 20

    totalSamples = sizeOfOneBuffer * numBuffersToCapture

    # Create buffers ready for assigning pointers for data collection
    bufferAMax = np.zeros(shape=sizeOfOneBuffer, dtype=np.int16)
    bufferBMax = np.zeros(shape=sizeOfOneBuffer, dtype=np.int16)

    memory_segment = 0

    # Set data buffer location for data collection from channel A
    # handle = chandle
    # source = PS5000A_CHANNEL_A = 0
    # pointer to buffer max = ctypes.byref(bufferAMax)
    # pointer to buffer min = ctypes.byref(bufferAMin)
    # buffer length = maxSamples
    # segment index = 0
    # ratio mode = PS5000A_RATIO_MODE_NONE = 0
    status["setDataBuffersA"] = ps.ps5000aSetDataBuffers(chandle,
                                                         ps.PS5000A_CHANNEL['PS5000A_CHANNEL_A'],
                                                         bufferAMax.ctypes.data_as(ctypes.POINTER(ctypes.c_int16)),
                                                         None,
                                                         sizeOfOneBuffer,
                                                         memory_segment,
                                                         ps.PS5000A_RATIO_MODE['PS5000A_RATIO_MODE_NONE'])
    assert_pico_ok(status["setDataBuffersA"])

    # Set data buffer location for data collection from channel B
    # handle = chandle
    # source = PS5000A_CHANNEL_B = 1
    # pointer to buffer max = ctypes.byref(bufferBMax)
    # pointer to buffer min = ctypes.byref(bufferBMin)
    # buffer length = maxSamples
    # segment index = 0
    # ratio mode = PS5000A_RATIO_MODE_NONE = 0
    status["setDataBuffersB"] = ps.ps5000aSetDataBuffers(chandle,
                                                         ps.PS5000A_CHANNEL['PS5000A_CHANNEL_B'],
                                                         bufferBMax.ctypes.data_as(ctypes.POINTER(ctypes.c_int16)),
                                                         None,
                                                         sizeOfOneBuffer,
                                                         memory_segment,
                                                         ps.PS5000A_RATIO_MODE['PS5000A_RATIO_MODE_NONE'])
    assert_pico_ok(status["setDataBuffersB"])

    # Begin streaming mode:
    sampleInterval = ctypes.c_int32(20) # do budoucna 20
    sampleUnits = ps.PS5000A_TIME_UNITS['PS5000A_US']
    # We are not triggering:
    maxPreTriggerSamples = 0
    autoStopOn = 1
    autoStopOff = 0
    # No downsampling:
    downsampleRatio = 1
    status["runStreaming"] = ps.ps5000aRunStreaming(chandle,
                                                    ctypes.byref(sampleInterval),
                                                    sampleUnits,
                                                    maxPreTriggerSamples,
                                                    totalSamples,
                                                    autoStopOff,
                                                    downsampleRatio,
                                                    ps.PS5000A_RATIO_MODE['PS5000A_RATIO_MODE_NONE'],
                                                    sizeOfOneBuffer)
    assert_pico_ok(status["runStreaming"])

    actualSampleInterval = sampleInterval.value
    actualSampleIntervalNs = actualSampleInterval * 1000

    print("Capturing at sample interval %s ns" % actualSampleIntervalNs)

    # We need a big buffer, not registered with the driver, to keep our complete capture in.
    bufferCompleteA = np.zeros(shape=totalSamples, dtype=np.int16)
    bufferCompleteB = np.zeros(shape=totalSamples, dtype=np.int16)
    nextSample = 0
    autoStopOuter = False
    wasCalledBack = False

    def streaming_callback(handle, noOfSamples, startIndex, overflow, triggerAt, triggered, autoStop, param):
        global nextSample, autoStopOuter, wasCalledBack
        wasCalledBack = True
        destEnd = nextSample + noOfSamples
        sourceEnd = startIndex + noOfSamples
        # print(nextSample)
        bufferCompleteA[nextSample:destEnd] = bufferAMax[startIndex:sourceEnd]
        bufferCompleteB[nextSample:destEnd] = bufferBMax[startIndex:sourceEnd]
        nextSample = nextSample + noOfSamples
        if autoStop:
            autoStopOuter = True


    # Convert the python function into a C function pointer.
    cFuncPtr = ps.StreamingReadyType(streaming_callback)

    bufferCompleteA_total = []
    bufferCompleteB_total = []

    i = 0 # detekce prvniho prubehu
    
    while(True): # neni potreba uzavirat, jenom pohlidat odsazeni

        nextSample = 0
        # Fetch data from the driver in a loop, copying it out of the registered buffers and into our complete one.
        while nextSample < totalSamples and not autoStopOuter:
            wasCalledBack = False
            status["getStreamingLastestValues"] = ps.ps5000aGetStreamingLatestValues(chandle, cFuncPtr, None)
            if not wasCalledBack:
                # If we weren't called back by the driver, this means no data is ready. Sleep for a short while before trying
                # again.
                time.sleep(0.01)
        
        # Find maximum ADC count value
        # handle = chandle
        # pointer to value = ctypes.byref(maxADC)
        maxADC = ctypes.c_int16()
        status["maximumValue"] = ps.ps5000aMaximumValue(chandle, ctypes.byref(maxADC))
        assert_pico_ok(status["maximumValue"])

        # Print time
	now = datetime.now()
	print(now.strftime("%Y_%m_%d-%H_%M_%S.%f")) 
        
    # Stop the scope
    # handle = chandle
    status["stop"] = ps.ps5000aStop(chandle)
    assert_pico_ok(status["stop"])

    # Disconnect the scope
    # handle = chandle
    status["close"] = ps.ps5000aCloseUnit(chandle)
    assert_pico_ok(status["close"])

    # Display status returns
    # print(status)
    print("Done.")


Post Reply