Decoding RS232/UART data as packets

Hi guys, this is my first day using the PicoScope... We've purchased a 3406D MSO and I'm attempting to find the cause of a few comms problems we're having.

All the screenshots on the PicoScope website show decoded data arranged as packets. I've since discovered this is only the case for protocols where the format is described in the standard - which doesn't help us at all.

So, I'm looking through a couple hundred Kb of packets which has become an extremely tedious process every time I take a new sample.

Is there a method whereby I can specify what defines a packet and have the data as strings across the decode window? The most obvious would be a fixed sequence indicating the start of a new packet, but I suspect the easiest (and most useful to me) would be simply to specify a minimum idle period between packets.

I'm not interested in voltages or timings (at the moment) as I'm using all the digital channels anyway - so losing all that data doesn't make a difference.

I've read that there's a beta version with a new serial decoder being tested but I can't find any information as to whether this would be of any use to me.

Is there any easier way to do this please?

Thanks
Darcy

Martyn wrote:...all feedback on serial decoding is welcome.

Serial decoding of data that is already packetized (CAN, Flex-ray, etc.) is comparatively easy. Handling arbitrary byte streams, and partitioning them properly into packets is harder, but is almost always significantly more useful than the raw byte stream. I've commented on this before, in other threads, but will go into a bit more detail.

There are 5 (or 6) generally used methods for achieving this:

1) the Idle-time threshold that Fullboar refers to. Bytes that are "close together" are collected into an aggregated packet, and split at points that exceed a user-defined idle-time threshold.

2a) Fixed length. Packets always contain the next N bytes.

2b) Embedded length. The length info is contained at a specific offset within the header section of the packet. Many ISO protocols implement this mechanism.

3) Start delimiter. Every packet beginning is marked by some specific pattern. This may be a full-byte value (or values), or a specific nibble in a byte, so a masked comparison may be required.

4) End delimiter. Similar to the Start delimiter.

5) Bit-flagged. Sometimes the start/end flagging is indicated by the presence of a high-order bit, when the remaining contents doesn't need to use the entire byte. This is a variation of a bit-masked Type 3, so perhaps shouldn't be listed separately.

Sometimes, for the sake of redundancy (to improve robustness), methods get combined. (Proprietary protocols I've defined in the past have combined (3) and (2b), for example, to enable Resync to be obtained after a corrupted packet header.)

The key thing to keep in mind when shifting to a packet-structured List Mode is that inter-byte stats (on each byte) will be lost. Though it would be possible to report min/max stats (at EOP) to replace those. However, in almost all situations, the value of seeing an entire packet in context greatly outweighs any info losses on the individual byte level.

~~

Next in line, right behind this, is the ability to see a two-way "conversation", where Transmit/Receive interactions (from 2 separate sample Channels) are visible in the same context (List), without having to flip back & forth to another window or Tab.

In almost every full-duplex comms-bus environment I've worked in (UARTs=Tx+Rx, SPI=MISO+MOSI, etc.) this is a really fundamental capability. It's difficult to overestimate the increase in productivity that such a capability can make. (Or how much it's absence negatively impacts the same, while contributing to operator frustration.) Almost all stand-alone desktop scopes I'm aware of get this right (each decoder has enough channels to handle the correlated comms-channel pairs).

- Mark

Right, using my work account now that it's sorted out... Yes, we use a combination of 3 and 2b as well - and this would certainly be the preferred implementation. I was hoping that the idle period would be the fastest to implement on the PicoScope side of things.

Absolutely agree about the two-ways comms needing to be seen together.

After purchasing this scope I was actually very annoyed at the "false advertising" of sorts... Not deliberate, but very misleading. The website talks about viewing data in frames and the protocols it supports, but does not say anything about not being able to configure this yourself, or it not working at all for RS232 comms. We nearly sent the scope back to the supplier upon finding that out... but it was only because our previous scope was so rubbish, and we were pressed for time, that we kept it.

https://www.picotech.com/library/oscilloscopes/serial-bus-decoding-protocol-analysis
This page is very misleading for anyone hoping that the PicoScope software will decode UART/RS232 frames

DarcyW wrote:I was hoping that the idle period would be the fastest to implement on the PicoScope side of things.

I'm sure it would be, but Pico has a history of implementing more complete and flexible solutions than what is minimally required. So having a broader set of design criteria should be helpful, both to them and their customers.

[Some day, these things being complained about now will get implemented, and then Pico will be bragging about the new capabilities. And rightly so, because they will make their analytical tools vastly more useful, and save large amounts of engineer time.]

After purchasing this scope I was actually very annoyed at the "false advertising" of sorts... Not deliberate, but very misleading. The website talks about viewing data in frames and the protocols it supports, but does not say anything about not being able to configure this yourself, or it not working at all for RS232 comms.

I take your point. It's easy to make assumptions, when most things about Pico are very powerful, that one example implies things about other examples. In this case, not so much.

But there is a major and fundamental difference between protocols over comms channels that are byte-stream based, vs. packet-based. So the fact that the former might not be handled as well as the later isn't really a complete shock.

This page is very misleading for anyone hoping that the PicoScope software will decode UART/RS232 frames

That page does have a link at the bottom, to a video showing exactly how UART/RS232 is handled... a single column, 1-byte wide. So it's not like they were hiding anything. But I agree that not everyone is going to take the time to look at EVERY protocol. They will pick one or two, and make assumptions about the rest. And it's not surprising at all that Pico would choose to focus on examples that highlighted their best capabilities, for the more complicated protocols.

They redid this page fairly recently, and I was quite impressed that they shifted from their previous I2C examples to a really detailed look at CAN. Somebody at Pico spent some quality time on that, and I thought it was well done and very helpful.

[But it was a bit embarrassing to see their examples displaying impossible CAN ID values. An ExtendedID field is a maximum of 29 bits, which means the high-byte can never be greater than 0x1F. Yet many places in their capture show values of 0xAn, and 0xCn, etc. So one wonders where those extra 3-bits came from.]

- Mark