CM3 Ethernet Connection Dropping

Post any questions you may have about our current range of USB data loggers
Post Reply
knucklehead
Newbie
Posts: 0
Joined: Tue May 28, 2013 10:14 pm

CM3 Ethernet Connection Dropping

Post by knucklehead »

We are running CM3s' over our LAN via command line scripts and they are recording very well. However, we are seeing one of these devices losing its connection too frequently, sometimes 2 or 3 times a minute. I have the auto reconnect set to 5 seconds and it is reconnecting but would like to understand what could be causing this. Has anyone seen this issue before and could give me some advice about what to look for?

Thanks for taking the time to respond in advance.

Very best, Knucklehead

Martyn
Site Admin
Site Admin
Posts: 4491
Joined: Fri Jun 10, 2011 8:15 am
Location: St. Neots

Re: CM3 Ethernet Connection Dropping

Post by Martyn »

How is the device powered ?
Whereabouts in the network is it (switches/hubs etc) and is there any heavy network use ?
Can you ping the device continuously without packets dropping ?
Martyn
Technical Support Manager

knucklehead
Newbie
Posts: 0
Joined: Tue May 28, 2013 10:14 pm

Re: CM3 Ethernet Connection Dropping

Post by knucklehead »

The device is being powered with POE through an injector.

We do not have a heavily trafficked network but the location is interesting. We have both fiber and Ethernet with business class switches making up the LAN. The CM3 is attached to a wireless bridge client which is turn synced up to a wireless access point about 180 feet away, straight line of site. These are Hawking HOWABN1 devices. WAP has Ethernet to a cisco/netgear gigabit switch which routes the packets over fiber to another gigabit switch and Ethernet to a smaller switch and Ethernet to my computer.

Our IT guy has been trying to sort this out with me and it is pretty clear we have gremlin in our network, possibly a bad switch. It is entirely possible that the problem is on our network but I am also trying to understand if there could be other requirements on the CM3 side that are not being met.

The disconnected notification window seems to be appearing with intermittent frequencies of a few seconds to 15 minutes or more.

Martyn
Site Admin
Site Admin
Posts: 4491
Joined: Fri Jun 10, 2011 8:15 am
Location: St. Neots

Re: CM3 Ethernet Connection Dropping

Post by Martyn »

The CM3 is a relatively simple device in terms of the communication between the driver on your PC and the device over ethernet. Once communication is established, and the channels set up, the CM3 will transmit data back in a UDP packet as it converts the channels. Additionally every 3 seconds the driver will check that it has communication with the device by asking if it is alive and expecting a response. My guess would be either the request, or the response, is going missing.

I would suggest putting a packet sniffer, such as Wireshark, on the PC and look at the data flow to/from the CM3.
Martyn
Technical Support Manager

knucklehead
Newbie
Posts: 0
Joined: Tue May 28, 2013 10:14 pm

Re: CM3 Ethernet Connection Dropping

Post by knucklehead »

Thanks for the suggestion. I installed the WireShark and captured a typical stream from my system here. Now you have to understand that I do not know what to look for but this is what I have observed. I see packets coming from the CM3 followed by packets going back to the CM3 from my computer. There seem to be something wrong with the packets from my computer going back to the CM3.

Can you give me some clues about working with WireShark and what to look for.

Best regards

Martyn
Site Admin
Site Admin
Posts: 4491
Joined: Fri Jun 10, 2011 8:15 am
Location: St. Neots

Re: CM3 Ethernet Connection Dropping

Post by Martyn »

On my system every 3 seconds my PC sends a packet to the CM3, the next packet the CM3 sends to my PC has the data "Alive" in it. Additionally every 0.8s approximately the CM3 is sending a packet containing 20 data bytes representing the channel readings.

You could post your wireshark capture of a few seconds worth of data and I can take a look at it.
Martyn
Technical Support Manager

knucklehead
Newbie
Posts: 0
Joined: Tue May 28, 2013 10:14 pm

Re: CM3 Ethernet Connection Dropping

Post by knucklehead »

I have a 100 packet capture attached.
The CM3 is at 10.0.0.79, my system is at 10.0.0.39

Thanks for looking it over.
Very best,
Knucklhead

Martyn
Site Admin
Site Admin
Posts: 4491
Joined: Fri Jun 10, 2011 8:15 am
Location: St. Neots

Re: CM3 Ethernet Connection Dropping

Post by Martyn »

I have looked at the file, and so deleted it from the forum, the CM3 is sending out a data packet every 0.66secs which is fine.

At packets 46 and 97 the driver sends out a keep alive packet with a response in packets 47 and 98 from the CM3 saying yes I am alive. You need to trace this for an extended period and see if these keep alive packet requests/replies disappear when the CM3 connection drops.

To help cut down the displayed data you can use a Filter: in wireshark of

(ip.addr eq 10.0.0.79 and ip.addr eq 10.0.0.39)
Martyn
Technical Support Manager

knucklehead
Newbie
Posts: 0
Joined: Tue May 28, 2013 10:14 pm

Re: CM3 Ethernet Connection Dropping

Post by knucklehead »

Problem Solved!
I realize this post reply is a little stale but it might be useful knowledge for others.

Ultimately, I could not devote enough time to learning how to do IP filtering with Wireshark but we were able to track the problem down with extended ping diagnostics. i.e. ping -n [count] [target]

By pinging from the workstation to the CM3 and then pinging back from the server we were able to isolate the device that was dropping packets by winnowing out the devices on the LAN between the CM3 and the workstation running the logging software, PLW.exe. It was especially useful to be able to ping from both sides of each device, from the server side and from the workstation side. Eventually, we found that we had a small wireless access point with a switch in it that was failing and dropping packets. We were able to replace this device with a new 8 port switch and the problem went away immediately.

It was not until we started using the ping command with extended counts that we were able to see the problems occurring. So, this was a bit lower tech diagnostic solution than using Wireshark but it works.

Martyn, thanks for your help.

Cheers, Knucklehead

Post Reply