Math tools and 12 bit resolution

VincenzoV · Post by **VincenzoV** » Wed Sep 16, 2020 9:07 pm

Hi all
I compute a simple formula: V*V/R with a Picoscope 3000. I'm using last Picoscope software on Windows 10.
It work fine setting 8 bit resolution, but with 12 bit resolution the result is completely wrong, about an order of magnitude.

bennog · Post by **bennog** » Thu Sep 17, 2020 11:14 am

Do you have a .psdata file.

It looks like black and blue are both math channels.

Benno

VincenzoV · Post by **VincenzoV** » Thu Sep 17, 2020 4:21 pm

> Do you have a .psdata file
No. I will create a new one

> It looks like black and blue are both math channels.
Yes, both.
Black: B/27000 (27000 is a resistor value)
Blue: B*B/27000

Thanks

Vv

VincenzoV · Post by **VincenzoV** » Thu Sep 17, 2020 5:33 pm

Updated...
This problem born if you set 12 bit resolution AND X10 probe
No problems using 12 bit resolution and X1 probe
No problems using 8 bit resolution and X10 probe

Red and blue: measure
Black: math (B/27000)
Cyano: math (B*B/27000)
Lime: math (A*B/27000)

Thanks

Vv

Test & Measurement Forum

Math tools and 12 bit resolution

Math tools and 12 bit resolution

Re: Math tools and 12 bit resolution

Re: Math tools and 12 bit resolution

Re: Math tools and 12 bit resolution