whatever SNR value you want without changing the signal

Having problems ? let us know the details here
Post Reply
Math
User
User
Posts: 2
Joined: Wed Aug 21, 2019 5:44 pm

whatever SNR value you want without changing the signal

Post by Math » Wed Aug 21, 2019 6:59 pm

Hi everybody,
I'd like to share some strange observations I made today. I wanted to measure some noise with my new 4444 and quickly found out that the built-in SNR measurement did not provide reasonable results. So I searched the forum and came across Gerrys post of Mon Aug 11, 2014, where he tried to give an explanation and proposed a correction factor based on the system gain. I was not convinced, because
1. Why should I want to apply a bin-number-based correction factor in a scope measurement
2. Why should the user have to correct measurement values when the spectrum analyzer could do that automatically by itself?
So I made some measurements myself. I started a spectrum view, connected my signal generator and inserted a SNR measurement. I repeated the measurement 14 times, only changing the number of bins, and corrected the values with the formula proposed by Gerry. Expecting to always get nearly the same (corrected) SNR value, I was upset by the fact that I could get nearly every desired *result* between 27 and 113 dBc by simply clicking the appropriate number of bins. I could imagine some situations in which this would be very convenient ;-), but how can I find out which is the correct no-of-bins? Do I miss anything?
I was even more confused when I noticed that I could also manipulate the SNR value from 38 to 118 dBc by only changing the "spektrum peak span". Isn't that the area where Picoscope looks for the peak, e.g. the level of the signal? The bigger I made the span the higher my SNR value became. If at all, I would have expected the opposite. Is there anybody who can help me out of my crossness?
I try to include the .psdata file, the Excel file with the value tables and the corresponding diagrams. It's my first post, so please be nice if it doesn't work at once.
Have a nice day Mathias.
Ooops, the .psadata is way too big, so a pdf must do. As i said...
Attachments
corrected autom measurements.pdf
(55.71 KiB) Downloaded 21 times
corrected autom measurements.xls
(13 KiB) Downloaded 17 times
corrected autom measurements.ods
(26.54 KiB) Downloaded 20 times
corrected autom measurements.xls
(13 KiB) Downloaded 19 times
corrected autom measurements.ods
(26.54 KiB) Downloaded 19 times

Gerry
PICO STAFF
PICO STAFF
Posts: 494
Joined: Mon Aug 11, 2014 11:14 am

Re: whatever SNR value you want without changing the signal

Post by Gerry » Thu Aug 22, 2019 11:01 am

Hi Math,

First of all, the Forum post that you refer to was an answer to why our older version of the software had an error in it (which you will see if you scroll back up to the top of the post). The Forum user was referring to a very old video on Youtube that showed SNR measurement in very old software.

I've opened your pdf and I can see exactly what I would expect to see in your results. The SNR measurement in our current software (which is correct and has been for a long time) shows that the measurement becomes more consistent (and therefore more accurate) as you increase the number of bins. This is because a bin-width is actually the smallest unit of measure, so it is the precision with which you can measure the noise. So, if you are measuring the amount of noise voltage present in a bin and the bin width is large (i.e. precision is low) then more of the signal voltage will be included in the noise measurement (because you can't discriminate between the signal and the noise within the bin itself) which means that the SNR value will fall, which you will notice in your pdf that it does from 512 bins down to 128 bins.

What you refer to as the 'corrected values' in your pdf are wrong, because with our current software (and software in use since the time of the old Youtube video I mentioned), the correction for Process Gain is already included in the automatic SNR measurement, so you are applying it twice.

So to answer your questions:

1. You should apply a correction factor for Process Gain if (a) you are not making automatic measurements from our Measurements menu, and are instead making manual measurements using the measurement rulers and (b) those measurements include noise. This is because we apply Process Gain to the spectrum values plotted to ensure that the plotted Amplitudes of the original signal are at the correct level (for a clear explanation of why you need to do this go here: https://community.plm.automation.siemens.com/t5/Testing-Knowledge-Base/Window-Correction-Factors/ta-p/431775 ). However, this correction factor is correct for signals, but not correct for noise (for a clear explanation of why, go to the following link [note that in this link the Spectrum Plots of Voltage are referred to as Autopower plots] : https://community.plm.automation.siemens.com/t5/Testing-Knowledge-Base/What-is-a-Power-Spectral-Density-PSD/ta-p/360969 ).

2. First of all, I assume that by 'Spectrum Analyzer', you mean 'Spectrum Mode' plot of our Software (as it is not a Spectrum Analyzer in the traditional sense, because that is an instrument that sweeps a filter across an analog signal to plot frequency values, so it doesn't need to apply Process Gain, while our Spectrum Mode plotting calculates frequency values by performing a Fast Fourier Transform on the Digitized values of the signal so it does need Process Gain). To answer your question, our Spectrum Mode plot does correct the measurement, as mentioned when discussing your pdf.
However, if you perform an FFT on a signal then you must choose between either compensating for voltage or compensating for power/energy in your plotted values (as described in the 2nd link I gave you in the answer to your 1st question). So, as we chose to compensate for voltage this means that any noise measurements that you make manually, using the rulers will be wrong, and that is when you should make the correction.

Finally, regarding corrections for Process gain, this is to some extent specific to the type of Window being applied (but our software applies the correct adjustment for all of the window functions we provide). You would need to use the appropriate correction factor for the specific window function for manual measurements (but for the more common ones used, e.g. Blackman, Blackman-Harris, Hamming, & Han, the example given is correct).

Regarding your comment about Spectrum peak span, without using the rulers the software just finds the highest peak, assumes that is the fundamental frequency or carrier frequency (i.e. the reference for dBc), and then calculates the SNR using that. The 'Spectrum Peak Span' allows you take more control over the SNR measurement using the measurement ruler to define a different fundamental/carrier reference, allowing you to define the number of bins either side of the ruler that the software will look for the reference. So, if your reference is more than the specified number of bins away from the measurement ruler, then the software will just pick the highest signal within the specified bin range, (which, for instance, could even be just the highest noise spike). This is why your first value of -4.426 is way off.
Also, as you increase the number of bins to look for the reference, you decrease the number of bins left to calculate noise with, so the SNR gets artificially raised in value as you can start to see happening when go past a 1000 bin search range.

I hope this "helps you out of your crossness" :D

As to the reason why the measurement of SNR may have not given reasonable results in the first place, we would need to see how you have captured the data, performed the measurement, and interpreted the results (for instance if you were expecting less noise have you taken the input noise of the PicoScope into account).
Could you post an example of what you get, and tell us why you believe it is wrong, using a shorter capture time for a smaller psdata file? Alternatively could you put the psdata file you have on a cloud storage site such as google drive or drop box and post a download link to it?

Regards,

Gerry
Gerry
Technical Specialist

Math
User
User
Posts: 2
Joined: Wed Aug 21, 2019 5:44 pm

Re: whatever SNR value you want without changing the signal

Post by Math » Sat Aug 24, 2019 3:23 pm

Hi Gerry,
Nice to meet you. Yes, you are of course perfectly right, I use some well known and reliable instruments in conjunction with this test, a PeakTech 4125 arbitrary waveform generator, a Rohde&Schwarz SME Signal Generator, a Tektronix TDS2014b digital storage oscilloskope, a LeCroy LA314 analog Oscillocope with a significantly lower noise level, a R&S FS300 spectrum analyzer (you would call it a genuine one ;-)), an R&S UPGR itu-r 468 conforming noise analyzer, some 400kHz bandwidth digital voltmeters, and a Spectran NF 5020 20MHz low frequency spectrum analyzer. The last one is a bit noisy and may be not applied this time.

The device under test is an Asus Xonar U7 MkII external USB soundcard. It is a 24bit/192kHz device, which provides an SNR of 114dB. Together with appropriate software - I prefer REW or Visual Analyzer for this job - it forms a low noise signal generator, which pushes even the most sophisticated test gear to its limits (e.g. the R&S UPV which I had on the bench lately for a few hours). In other words: With the Xonar as a generator the devices measure mainly their own noise. I do not claim to reach this 114dB SNR under normal conditions (that is outside a shielded room or our "atomic-war-proof" special shielded test box), but 100dB are possible, if you keep the cables short (the RG58 e.g. normally has a shield effectiveness of only 60dB). You can of course also use the Xonar as a low distortion fft spectrum analyzer. In loopback mode, when you directly connect the inputs to the outputs, you then get an impression of the overall perfomance of
the system: noise floor below -100dB, THD below 0,002%.
Okay, lets start. I set up the Xonar in loop back mode for a short test: 98,4 dB SNR at 1750 Hz (bandwidth 96kHz). Fine, just as usual. Let's test it with the UPGR: 0,023mVrms noise unweighted, that means 92,7 dB. With 2m of cable in between. Okay for now, I'm too lazy to carry the UPGR around besides the fact that the bench is already crowded enough. Now I connect it to the 4444 using the shortest cable (RG180 which I prefer for audio measurements because oft its lower capacity of only 50pF/m, around 30cm long).
Picos internal SNR routine says: 47,29 dBc at a bandwidth of 100 kHz (20190824-0001-method1 internal SNR
routine.psdata)
Ooops. Something must be wrong.
To my mind there are two possible reasons for this:
a - the SNR routine is a bit off target, or
b - the SNR of the 4444 is much worse than has to be expected from the data sheet. So, what do we have to expect: "<180 µV eff für ±10 mV-Bereich" in 14bit mode, cited from the "4444 detailed technical data" sheet. The data sheet does not provide any figures for the other ranges, and that is not necessary. This means that this is the worst case, which is plausible because the SNR is always the worst when the gain of the input amplifiers is the highest. Our signal level is 1 Vrms, the SNR to expect therefore is 74,89dBc. Minimum. Or maybe a bit higher because I use the +/- 2V range and hopefully haven't got a 4444 made on a monday ;-).
Maybe we come close to the 90dB tag? That would be really great.
Le's try to find out where the error comes from: I generally use up to four types of SNR measurements:
1. The direct method with the built in noise measurement routines
2. The spectrum method: With a ruler I try to find the peak noise level (in peak hold mode or in "Magnitude" mode, never in average mode of course. And never place the ruler in the middle of the noise floor, because there is the baseline, U_noise=0.)
3. A simple switch method, that means measuring the rms voltage with the signal switched on and the signal switched off, calculating the noise quotient from these two measurements with a pocket calculator. This is of course only applicable
if the output of the DUT is not completely interrupted in the Off state (as with the PeakTech for example). I call it "UPGR style".
4. With the 4444 we are lucky and can use the "between rulers" method: we pick a part of the sign curve where it is virtually flat and make the interval between the rulers as short as possible. Then we calculate the dB value of the quotient of "AC rms whole trace" and "ACrms between rulers".

We have already done method 1.

Method 2: There are some peaks in the spectrum view which eventually do not come from the generator (at 12 and 24 kHz e.g.). We can see that when we switch it off (20190824-0001- method 2 generator switched off.psdata). The nice feature of this method is that we could mask them just by ignoring them, but we must not do that here, because then this measurement would not be comparable to the others any more. The result therefore is SNR=73dB. Much better! (20190824-0001- method 2 generator on.psdata). Supposed that these spikes only appear in the Picoscope cabling and not in loopback and UPGR measurements, we push the ruler down to the dense "hedge" of the noise floor: 92dB. Bingo! That's exactly the result of the UPGR mode and relatively close to loopback mode.

Okay, method 3: with signal: Urms=1,009V average (20190824-0001- with signal.psdata), without: 614,2uV (sigma 4,152uV, 1000 captures), SNR therefore 64,2dB (20190824-0001- without signal.psdata). That's not what I hoped, but better than the automatic measurement.

Method 4: To achieve a sigma smaller than 10% I have to place the second ruler at least 3,3uS apart. Signal voltage: 1,009Vrms, noise between rulers 1,621mVrms. SNR therefore: 55,79dB. Sad. Close to the automatic measurement before. (20190824-0001 method 3 scope mode between rulers.psdata)

Okay, let's look at the 18bit mode. I haven't tried that out yet. Let's use some white noise for this, so that we get a frequency response curve. The result is here: 20190823-0001 noise 0-20kHz 4MS 65536bins.psdata.
Good gracious! Thats a comb filter with a pole spacing of 173Hz, overlaid with some e^-x amplitude decrease! H (the level span) is 40dB! Spontantaneously I can't imagine whatever I could use that for. And where is the 18bit resolution hidden? Ah, I see, the noise level is lower. But the nonlinearity of the amplitude cripples it all. Couldn't even use it for THD measurements. Hmmm, wait, maybe I could make a math channel filter to compensate for the decrease. Oh no, I just remember that math channels in general seem not to work in spectrum mode. Bugger!
Nevertheless, let's measure the SNR (automatically, of course, how else?):52,44 dBc. No change, astonishingly. Should be a much higher value, we are in 18 bit mode! Maybe we could iron out the frequency resonse ripples be reducing the no of bins (which is an eyewash of course, but nevertheless...). The result is to be found here: 20190823-0001 SNR 0-20kHz 40kSamp 256bins.psdata. Everthing nice 'n easy, only the steep decline of more than 44dB between 0 and 6 kHz remains,which goes on, 6dB follow down to 20 kHz. The SNR is 50,38dBc, even lower than before!!
You see what I mean? You get lots of values from your Picoscope, and many of them seem to be wrong. You never know which one to trust.

And in the end the Pico really starts kidding me: When I extend the bandwidth from 100 kHz to 1 MHz (fsignal=1kHz), the SNR gets * MUCH BETTER*: 83,88%!!! And when I further extend it to 8 MHz, it gets worse again: 64 dBC. You say, more bins make for a better measurement? When I REDUCE the bins form 16384 to 512, the SNR RISES from 57 to 76 dBc. But that is all nonsense. It is due to the steep amplitude decline, which makes the signal appear much bigger in comparison to the noise, hence the high SNR values.
You see what I mean?

Abstract: The picoscope 4444 hardware with the corresponding latest version of the Picoscope 6 software was intensely tested by measuring noise levels and SNR values. It was especially tested for congruence of the delivered values using four different measurement procedures.
1. The built in SNR routine in spectrum mode delivers an SNR of 47,29dB
2. A measurement with rulers in spectrum mode delivers 73dB (respectively 92dB)
3. A simple "Signal On / Signal off" comparison in scope mode delivers 64,2dB
4. A measurement with vertical rulers in scope mode delivers 55,79dB

The picoscope was also tested for congruence with the measurement results of rivalling systems from Asus (Software REW) and Rohde & Schwarz:
1. The measurement with the Rohde&Schwarz UPGR delivers an SNR of 92dB
2. A loopback test with the rivalling hardware alone delivers an SNR of 98,4dB

Results: The Picoscope delivers varying, partly contradictory results. Especially the results of the automatic SNR measurements seem to be wrong. Only in the spectrum-ruler-measurement (no 2), the result was anything near the congruent output of the rivalling systems.

Have a nice day! Mathias.

Psdata and image on:
https://www.dropbox.com/sh/zkbwcpopqwo7p2x/AAATp8id1wJU0RwA4gE0VFANa?dl=0

Gerry
PICO STAFF
PICO STAFF
Posts: 494
Joined: Mon Aug 11, 2014 11:14 am

Re: whatever SNR value you want without changing the signal

Post by Gerry » Wed Aug 28, 2019 11:15 am

Hi Mathius,

Just for clarification our specification of noise for the PicoScope 4444, where we quote <180uVrms, on the ±10mV range, this represents a noise ratio to the full scale range of 180*10^(-6)/7.07*10^(-3)= 0.02546, or 31.88dBc. So for a signal at say -1dBc that would be an SNR of 30.88dBc, which is certainly not a low noise figure (but this is the worst case for the smallest signals, and we have software tools that may enable you to significantly reduce this). As you correctly mentioned, the smallest input ranges do represent the worst case noise, but as you can see from these calculations your measurement of 47.29dBc (and you are only using a signal at half the level of the input range, so you are also loosing 3dB of signal versus noise in your measurement setup) is not worse than what we specify for the noise. So nothing is wrong, and the SNR routine is not off target.

It appears that you have some good equipment, but even measuring noise with the best equipment available doesn't necessarily mean that you have the correct measurement, because it is essential to use an appropriate method, especially as noise measurements vary dramatically depending upon the method used and the environment in which they are performed. Even using short length RG58U could mean picking up significant interference in the PicoScope itself if it is too close to a source which your signal generator may well be (in your case it might actually be better to use a longer cable as the electric field strength of any interference falls off in proportion to the square of the distance). So, the valid method that we use in Test and Measurement, to characterize the noise, is to terminate the input with a 50 ohm terminator, move the PicoScope as far away as possible from potential sources of interference, and then measure the AC signal (after a period long enough for the thermal variations within the equipment to have stabilized) which will be the residual input noise. The 50 ohm termination represents the typical impedance of the connected cable without having a length of cable acting as an antenna (although the PicoScope 4444 also has a short length of cable for the BNC to D9 converter, but this very short).
As you can see from the attached psdata file, when using the correct method the RMS value of the input noise is 1.666mV, on the 2V input scale, used for your measurement. If we factor in a signal from a Signal Generator (i.e. with noise, but much lower than 1.666mV) at 1V as you used, then the signal to noise ratio would be 0.001666 which is -55.57 dBc. That's a difference of -8.32dB to your measurement, which could be explained by your description of your setup, i.e. you said that you had your UPGR on a 2m cable length a fair distance from the other equipment on your bench (so it certainly would not be picking up any interference). However, the PicoScope only has a 30cm cable length, on a bench crowded with equipment (and I have the same issue, so I have to move my Picoscope far away from the equipement to get a satisfactory result).
Reviewing your other methods:

Method 2
This doesn't compensate for Process Gain. We've already discussed this in some detail, (so I'm confused as to why you are repeating the same error here - if you still don't believe that you need to compensate then look at this explanation from Audio Precision, who's business is based around high accuracy and precision audio measurements: https://www.ap.com/technical-library/fft-scaling-for-noise/, so they have to ensure that they do it correctly). When you do compensate for process gain you need to subtract 45dB for 65536 bins used.
You also are using the absolute peak of the noise (i.e. not waiting for the peak hold to settle to replace the 'dense hedge' that you refer to with a much smoother line, and placing the ruler at the top of the peaks). You should be using average and waiting to get a smooth point to measure (as shown on page 2 of this document by Analog Devices, a well known and respected manufacturer of Integrated Circuits that implement the technology that we are discussing: https://www.analog.com/media/en/training-seminars/tutorials/MT-003.pdf who place the ruler in the middle of the noise) as noise is a stochastic process and needs to be measured in a statistical manner, e.g. taking the average.
Finally you haven't placed the other measurement ruler at the top of the Signal in your example. If you make all of these corrections then you might get the consitency that you are looking for.

Method 3
If I have understood this correctly, I'm assuming that you're measurement goal is to measure signal + noise and then measure just noise to subtract it from signal + noise giving you the 2 values to use for noise divided by signal. But, if my assumption is correct then this is seriously flawed, because the input ranges you would need to use are too far apart in level, so the signal value is just not accurate at all below millivolts and the noise value is not accurate at all above 100's of microvolts.

Method 4
This method although very approximate (because the value can vary depending upon where you position the rulers) is at least useful for giving you feel for whether or not your original measurement is way off, but won't typically be very accurate with the automatic measurement, as demonstrated in the values you got.

Regarding the 18-bit mode, what you are using is something we call Resolution Enhancement, commonly used on Digital oscilloscopes to (a) reduce noise, and (b) increase resolution. It implements a moving average filter, which is very effective at removing noise for signals much lower in bandwidth. It uses the statistical nature of noise to insert new signal levels to interpolate from, based upon the density of the noise around a signal (a bit like the concentration of dots in a newspaper creating an image) which gives the extra resolution. It Is not meant to be applied to noise, so you wouldn't use it for measuring noise, but it is meant for measuring signals and is very useful in that respect (and it is a filter so you should expect filtered data, e.g. your comb filter).
Resolution enhancement is a software technique applied to the data once it has been captured to aid in the analysis of signals buried in noise. In order to work it needs a signal, and in order to measure Signal to Noise Ratio, you obviously also need a signal, which the example you created doesn't have (so the software will just pick out a noise peak, or distortion peak to use as a signal). So, the automatic measurement of SNR should not be much higher in 18-bit mode if the peak is moving with the noise.
Trying to make SNR measurements on pure noise is pointless, as is commenting on the rest of your 18-bit discussion, which goes on to discuss other SNR measurements on noise..

Unfortunately, you are not using the functions of the software correctly so you are finding faults that just aren't there (it's a bit like saying "why do we need brakes on cars because they just stop me from going where I want to go?", without realizing that their purpose is to stop you from going where you don't want to go!).

Regarding your Abstract, only one of your 'intense tests' appears to be completely valid and one other a rough guide.

Your statement about a test for congruence flawed because the equipment you mentioned is not rivalling the PicoScope 4444, and we don't claim that it is. Your equipment, as mentioned by you has very low input noise. As previously mentioned the worst case input noise of the PicoScope 4444 is not low....for some reason you have taken it upon yourself to challenge the notion that it is a low input noise scope but we don't market the device as a low input noise scope. If you need low input noise then you should be looking at our PicoScope 4262 which has 1/20th of the noise of the PicoScope 4444 (see here: https://www.picotech.com/oscilloscope/4262/picoscope-4262-specifications), so that is our low input noise scope. However, again, the noise won't be as low as the figures you mention for your equipment.

So, finally, I hope you can now see the errors in you tests and results, and that our PicoScope 4444 and PicoScope 6 software in fact do not deliver varying contradictory results.

Regards,

Gerry
Gerry
Technical Specialist

Post Reply