PicoLog CM3 to live graphing dashboard

Post general discussions on using our drivers to write your own software here
Post Reply
daln
Newbie
Posts: 0
Joined: Thu Jan 19, 2017 10:09 pm

PicoLog CM3 to live graphing dashboard

Post by daln » Fri Jan 20, 2017 12:03 am

I am interested in enabling a PicoLog CM3 to live replicate data to a graphing dashboard [Grafana] via records stored in a time series database.

For now I have extracted data samples via Wireshark whilst having the Picolog recorder application running.

How do I decode these samples?
I have made a spreadsheet of my analysis that I'm attaching as an image

00 20 11 aa ad 01 20 11 a8 36 02 20 11 a8 b0 03 20 11 a8 3e
04 20 19 67 7b 05 20 19 68 76 06 20 19 68 0d 07 20 19 67 ba
08 20 1a 26 82 09 20 1a 27 16 0a 20 1a 2c 0f 0b 20 1a 28 9c
0(CH) PAD 1() ()D DD 0(CH+1) PAD 1() ()D D6 0(CH+2) PAD 1() ()D DD 0(CH+3) PAD 1() ()D DD
Attachments
Decode CM3.png
Image of spreadsheet analysing sample data from a CM3 over a network

Hitesh
Site Admin
Site Admin
Posts: 2829
Joined: Tue May 31, 2011 3:43 pm
Location: St. Neots, Cambridgeshire

Re: PicoLog CM3 to live graphing dashboard

Post by Hitesh » Fri Jan 20, 2017 12:19 pm

Hi daln,

The ethernet protocol for the PicoLog CM3 isn't currently documented but I have added your comments to a request to make this information available.

It should be similar to that used for the USB PT-104 which should give you a starting point.

If you require further assistance, please post back or e-mail support@picotech.com and we will look to assist you further.

Regards,
Hitesh

Technical Support Engineer

daln
Newbie
Posts: 0
Joined: Thu Jan 19, 2017 10:09 pm

Re: PicoLog CM3 to live graphing dashboard

Post by daln » Wed Feb 01, 2017 12:17 am

Based on the USB PT-104 programming guide.
I have programmed parts of a Ethernet client and the function described in "6.5 To calculate a resistance"
decode.py: https://github.com/DavidLutton/PicoTech ... /decode.py
PicoTechEthernet.py: https://github.com/DavidLutton/PicoTech ... thernet.py

I would like to validate my function with some example data, if you can provide.

Can I have some example data to decode?
Is the channel calibration an integer or float?
Are the measurements integers or floats?

Channel calibration in LSB and in integer/float
At least one sample in hexadecimal with each measurement: zero, one, two, three decoded.
The result of the calculation for each sample.

If all this is possible can I also have details on how the "To calculate a resistance" function varies, if at all, for the CM3 unit.

Regards,

Hitesh
Site Admin
Site Admin
Posts: 2829
Joined: Tue May 31, 2011 3:43 pm
Location: St. Neots, Cambridgeshire

Re: PicoLog CM3 to live graphing dashboard

Post by Hitesh » Thu Feb 02, 2017 7:30 am

Hi daln,

Thank you for the links - you may wish to post information about this in our Applications section or submit it via our PicoApps page.

I have referred this to the relevant persons in our Development Team and will provide an update once I have further information.

Regards,
Hitesh

Technical Support Engineer

Hitesh
Site Admin
Site Admin
Posts: 2829
Joined: Tue May 31, 2011 3:43 pm
Location: St. Neots, Cambridgeshire

Re: PicoLog CM3 to live graphing dashboard

Post by Hitesh » Fri Feb 03, 2017 2:25 pm

Hi daln,

The calibration information is not required in the case of the PicoLog CM3.

I've run some test C# code and you will need to undertake the following:

1. Take an average of the 4 readings returned for each channel
2. To convert the reading into a voltage use the following formula:

Code: Select all

voltage = 2.5 * reading / 2^28
Below are extracts from the code (the readings are converted to millivolts in this case:

Code: Select all

    /// <summary>
    /// Parses a <see cref="byte"/> array into an <see cref="Int32"/>. 
    /// </summary>
    /// <remarks>We cannot use BitConverter here because it has the wrong endian.</remarks>
    /// <returns>the measurement</returns>
    int ParseMeasure(byte[] bytes, uint index)
    {
        return (bytes[index + 1] << 16) | (bytes[index + 2] << 8) | (bytes[index + 3]);
    }

    private void ReceiveData(byte[] receiveBytes)
    {
        lock (_lockObject)
        {
            // The data returned will be in the following format
            //
            // 00XXXX01XXXX02XXXX03XXXX  data from channel 1
            // 04XXXX05XXXX06XXXX07XXXX  data from channel 2
            // 08XXXX09XXXX0aXXXX0bXXXX  data from channel 3
            //
            //(data format is: Measurement 0, 1, 2 for respective channels)

            // Channel 1
            if (receiveBytes[0] == 0x00 && receiveBytes[5] == 0x01 && receiveBytes[10] == 0x02 && receiveBytes[15] == 0x03)
            {
                int measure0 = ParseMeasure(receiveBytes, 1);
                int measure1 = ParseMeasure(receiveBytes, 6);
                int measure2 = ParseMeasure(receiveBytes, 11);
                int measure3 = ParseMeasure(receiveBytes, 16);

                // Find average of readings - see programmers guide for other measurement types
                Ch1 = (measure0 + measure1 + measure2 + measure3) / 4;

                // Convert to millivolts
                Ch1Millivolts = (2.5 * Ch1 * 1000) / Math.Pow(2, 28);

                if (NewData != null)
                {
                    NewData(this, EventArgs.Empty);
                }
            }

            // Channel 2
            if (receiveBytes[0] == 0x04 && receiveBytes[5] == 0x05 && receiveBytes[10] == 0x06 && receiveBytes[15] == 0x07)
            {
                int measure4 = ParseMeasure(receiveBytes, 1);
                int measure5 = ParseMeasure(receiveBytes, 6);
                int measure6 = ParseMeasure(receiveBytes, 11);
                int measure7 = ParseMeasure(receiveBytes, 16);

                // Find average of readings - see programmers guide for other measurement types
                Ch2 = (measure4 + measure5 + measure6 + measure7) / 4;

                // Convert to millivolts
                Ch2Millivolts = (2.5 * Ch2 * 1000) / Math.Pow(2, 28);

                if (NewData != null)
                {
                    NewData(this, EventArgs.Empty);
                }
            }
        }

    }
In the measurements, the MSB can be ignored as is shown in the ParseMeasure() function above.

To give an example from my test setup, I received the following packet:

Integer: 0 32 0 89 18 1 32 0 91 139 2 32 0 92 227 3 32 0 91 151
Hex: 00 20 00 59 12 01 20 00 5b 8b 02 20 00 5c e3 03 20 00 5b 97

The reading number is highlighted in bold, passing values are shown in red and the 3 bytes of data for each reading (MSB to LSB order).

When the measurements are passed into the ParseMeasure function in the code, this gives:

measure0 = 22802
measure1 = 23435
measure2 = 23779
measure3 = 23447

with average of 23365.75.

When this is converted to millivolts:

millivolts = (2.5 * 23365.75 * 1000) / 2^28 = ~ 0.218 mV

With a scaling of 1 mV = 1 A, this gives approximately 0.218 A.

I hope this helps.

If you would like to test the example application, please e-mail support@picotech.com

Regards,
Hitesh

Technical Support Engineer

daln
Newbie
Posts: 0
Joined: Thu Jan 19, 2017 10:09 pm

Re: PicoLog CM3 to live graphing dashboard

Post by daln » Sun Dec 24, 2017 6:40 pm

This is a client for a CM3 over ethernet where probably still need initial configuration over USB
This is written in Python https://github.com/DavidLutton/PicoTechEthernet, change IP address as needed

If you comment out the lines in client.py `lines 3, 23` to do with requests, you don't have to install any python modules to collect data. You will just be shown the channel number and the measured value

If anyone wanted to, I could make a simple client that saves values measured & time to a CSV file

Code: Select all

YYYYMMDD_HHMMSS, CH0Value, CH1Value, CH2Value
I use InfluxDB to record the time series data https://www.influxdata.com/ and Grafana https://grafana.com/ to display the data

I've had my implementation running for just over 6 months continuously now
Only two phases shown, We don't have three phase at this site, just 2 100A supplies

I usually set the time line to show the last 3 hours and update every 10 seconds
Attachments
Screenshot from 2017-12-21 13-25-01.png
25 minute span with heaters turning on and off
Screenshot from 2017-12-21 13-25-36.png
6 Months of continuous logging

Martyn
Site Admin
Site Admin
Posts: 3333
Joined: Fri Jun 10, 2011 8:15 am
Location: St. Neots

Re: PicoLog CM3 to live graphing dashboard

Post by Martyn » Wed Dec 27, 2017 4:15 pm

Looks interesting :)

Will take a closer look when back in the office in the New Year
Martyn
Technical Support Manager

Post Reply