Final score (penalty shoot-out)
Yaesu FT-817 15
Flex 1500 10
The Yaesu took an early lead and the Flex 1500 never caught up. In fact, the Flex kept up the same slower pace all the way through and just kept falling further and further behind the FT-817.
The referee ended the contest as there was a clear leader - the Yaesu FT-817 - at every stage.
I have to say straight away that this is not the sort of test you would do in everyday amateur life. It is an extreme test under weak signal conditions. However, it is using the sort of conditions I meet every day.
The Flex is a fine rig for most purposes. But I wanted to know if I was right to think that it was not performing as well as I need in very low signal situations.
I know I am paranoid, but I felt sure that the Flex was not performing as well as I had expected. Just a feeling based on non-decoded meteor scatter signals.
If you read endless earlier posts, you will know that I found that my ME4-T transverter and the 817 produced good results on weak signals. Without giving it a lot more thought, I replaced the 817 with the Flex 1500, reasoning that it was better suited to transverter driving. However, I then found that something was not quite right. The Flex/ME4T combination seemed to decode too few meter scatter signals. I then found that the Flex AGC-T control was maxing out the VAC with noise at high settings. Even after resetting AGC-T, it SEEMED less successful than I expected.
IDEA BEHIND TEST.
Produce two weak identical signals and feed one of them into the Flex and another into some other receiver and prove there was nothing wrong with the Flex.
I was not likly to be a fault with the transverter, though if the results were identical it might be. So I decided to put a "tee piece" into the transverter output and feed identical signals to two receivers. The initial plan was for a comparison between the Flex and my Fun Cube Dongle. However, I had a problem with the Fun Cube dongle and decided to use the FT-817 instead. Not that the Flex would have a big problem seeing off the 817, but worth a try ??????
The plan was to point the antenna at the GB3RAL beacon on 70.050 MHz. At 480km, RAL is marginal copy on JT65 from here, somewhere between no signal and -15dB in QSB measured on the WSJT software. That should be a good test for weak signal reception. Just set it up with two instances of WSJT software copying the signal, one from the Flex and one from the 817, leave it for several hours, and then compare.
REASONS WHY THE FLEX SHOULD WIN EASILY
1) The Flex is state of the art SDR and the Yaesu is a 20 year+ older analogue design (and a stripped down version of the FT-857 too)
2) The Flex is specifically designed for use as a transverter driver
3) For convenience, the 817 antenna connection was via the front BNC socket, which introduces noise from the nearby display
4) The Flex was set to the ideal 2.0kHz filter width, the 817 was just using the (optional) 2.7kHz Collins SSB filter - this difference should also improve the signal to noise figures produced by the software for the Flex.
5) The Flex keeps the signal in digital format all the way to the computer and the WSJT software, whereas the 817s signal is processed in analogue and converted to audio and then fed into the computers sound card (a Soundblaster) for conversion back to digital - lots of losses possible on the Yaesu side
6) While the digital signal passes directly from the Flex to the PC, the analogue audio signal passes via another stage, a ZLP Digimaster MiniPro data unit, for isolation purposes. This introduces another stage into the process, plus more noisy plugs, sockets and cables.
The diagram shows the set up-
After the test I went through all the results. I was looking for differences between signal to noise (S/N) figures - I was not expecting to find that the 817 decoded 50% more signals than the Flex, but that is what happened.
There were differences in S/N too. Of ten occasions when both rigs decoded a signal, S/N was the same in 5 occasions. On 3 occasions the Flex was 1dB lower, on 2 occasions the Flex was 2dB lower. As I reported earlier, I would have expected the Flex to be better on all occasions due to all the factors around using an SDR with better filters and lower losses due to digital signal route all the way.
So it was surprising to find the 817 doing better on S/N. What I had DEFINITELY NOT expected was to find 5 occasions when the Flex failed entirely to decode a signal heard on the 817. This was despite two of these being -22dB on the 817 (a level where the Flex did capture results 3 times) and the 817 also captured results at -24dB (twice) and -25dB, when nothing showed on the Flex at all.
OK, so it was marginal conditions and there will always be variations. And of course the S/N figures will be subject to variation due to other factors and all that sort of careful fiddling about. But nevertheless - the 817 did better than the Flex on EVERY count - it never showed lower S/N, it never failed to decode every signal the Flex did, but it also decoded 50% more.
Here's the thing. The waterfalls looked similar, but the 817 "sounded" brighter on the loudspeakers. I know that is just my opinion, but it is what started all this comparison in the first place.
I was just looking at the actual decodes on the WSJT screen. WSJT can also do "average" decodes based on processing - this can show up decodes based on partial reception of the JT65 signal. Of course I was not using that for this test, though the contribution to the average was also higher on the 817 - 25/50 compared to 16/50 on the Flex. this just confirms the other results. The Flex threw up an "average" decode for the RAL within a couple of cycles, whereas the Flex did not until it got a full decode.
This test has me back to thinking again. Should I run it for longer? Is there something about my Flex set-up which is causing a problem?
Was I right all along that the Flex is under performing? THEREFORE, is my paranoia correct?
Conclusions? Eh? More confusions.