I have had to re-write this posting. I initially spent quite some time dealing with issues around the possibility of false QSOs - indeed the possibility of JT65 QSOs carried out using JT65 not being QSOs at all. The statistics of this are clear to me - JT65 with or without "Deep Search" is unlikely ever to produce any more false QSOs than may happen anyway using other modes. Sometimes mistakes are made, sometimes I get QSL requests for SSB QSOs I never made. It happens.
However, then I began to look again at SM2CEW's note and I realised that a central plank in his argument is not that DS produces doubtful QSOs but that:-
"There is, therefore, a great risk that over time this will devalue the achievement of making QSO’s . Furthermore, it creates an imbalance when comparing achievements for awards, toplists or in contests." I think that maybe he doesn't like the idea that JT65 used with Deep Search challenges the efforts of those who did EME before JT65 and Deep Search came along. You will see where this changed the tack of my article.
Sorry this is so long but it is both complex and complicated.
================================
SUMMARY: Deep Search only generates callsign suggestions, it does not generate QSOs. These callsign suggestions are occasionally wrong (false) but that does not, in my view call the entire process into question. Good operating still makes the difference.
Earth-Moon-Earth
communication, "EME" (or "Moonbounce" as it is often rather inaccurately
called), is bound to be on the edge of radio amateur capabilities. The
path length is about three-quarters of a million kilometres, and the
path loss on 144MHz is around 250dB. 99.9% of your signal is never going
to reach the Moon and only about 7% of that is going to be "bounced"
(scattered) back. The Moon is not a flat surface, and it moves around in
space. Despite what you hear about it always having one face turned
towards the Earth it wobbles, and for a lot of the time it is below the
horizon. Add to that the fact that its elliptical orbit moves it closer
and farther away from the Earth over time and you have a pretty tricky
object to use to scatter your radio signals back to Earth.
The
fact remains that EME can be done, and therefore many of us set it as a
target to be achieved. For me it is not an end in itself but a level of
technical efficiency to be reached, whereas for others it is an
obsession to throw themselves into. Just like any aspect of amateur
radio it has the potential to take over your life. This was especially
so in the days before specially designed data modes, which basically
means before the advent of home computers with sound cards. Before data
modes EME was the preserve of the technically brilliant and the very
determined amateur (i.e. not me). After data modes it became possible
for modest stations to achieve EME contacts, though possibly only by
taking advantage of the "super stations" with good set-ups at the other end of the QSO.
 |
| Watching a super station at GM4FVM provides hours of innocent fun (note the failed call ...) |
By "modest" station I mean one like mine. I have a
reasonable radio, 300W available, an antenna with a 3m boom and a
rotator capable of pointing the antenna accurately at the Moon. That
might be "modest" in the world of terrestrial stations, but it is
positively tiny in EME terms. My station is just about the simplest
setup capable of sustaining communication via the Moon, and even then I
depend on a super station being at the far end. The super station would
have multiple antennas, switchable polarisation, at least 1000W of
power, very low loss co-ax, low noise masthead pre-amp, and antennas
capable of being tilted upwards to track the Moon's elevation. I am not
in that league so I am just chipping off a few contacts to find out how
it can be done, and hopefully get better over time.
The
difference between me making a QSO and not making one is down to
fractions of a decibel. I obviously want to have all the advantages but
there is no point trying if it is not genuine. I would not want to do
it if it was not valid.
This rather long ramble is
meant to set the scene for a discussion of "Deep Search". Deep Search
(DS) is a setting in the WSJT-X suite allowing the use of algorithms
which compare partial received data with data already stored in the
receiving station's computer. For example it would allow partial
callsign information to be compared with a callsign stored in the
computer, and then the software will make a judgement as to whether it
has detected enough of the data to confirm the callsign. This has been
compared to a CW operator hearing "GM4F?M", not being sure of the ?
symbol and concluding that this is likely to be GM4FVM, rather than
GM4FDM. I would say it is a pretty common occurrence for operators to do
this, or to conclude that what sounds like a greeting of 72 is really
73, or that what sounds like a signal report of 2299 is really 599 with a
dot missing.
That representation of DS, of the CW
operator correcting obvious missed characters, is often quoted but it is
not quite accurate. The inner workings of the decoding process for JT65
are fiendishly complex. It is not as simple as correcting a character.
And here it gets a bit like self-driving cars. People are alarmed to
think that a computer might make a decision which could result in a
crash or loss of life. At the same time, these decisions are taken by
all sorts of people every day, and many of those are not folks I would
trust with my life.
I remember staring up at someone
from my position prone in the gutter, having been thrown down the road
after his car drove headlong into my motorbike, and him saying "I didn't
see you". As the ambulance staff scraped me of off the road I might
have thought that a computer controlled car might possibly not have
driven into me. The point is, computers might be able to make better
decisions than a human, but do we trust them? Why should I trust humans
more when they keep driving into me?
I blundered into this sea of doubt during my posting here on 30 March,
here.
As I said, I followed W7GJ's settings, save that I used less aggressive
values to save some processing power. W7GJ certainly advocates the full
settings, including turning on "averaging" (building up data across
listening periods) and reducing "synch" which is likely to show up as
some spurious responses as well as correct ones.
Gav, GM0WDD, posed me this interesting question ...
What are your thoughts on deep search? Peter, SM2CEW, has written a thought provoking article on it.
http://www.sm2cew.com/Digital%20communications%20using%20minimal%20transfer.pdf
A
good question Gav and it deserves a good answer. Sadly I am not very
experienced in the world of EME, but not knowing much never stopped me
pontificating before.
Far be it for me to disagree with
SM2CEW, who has been around EME for much longer than I have. I dare say
he is right and fundamentally I agree with him if indeed the limited
information which is transmitted in DS has an injurious outcome. I just
have not seen any evidence that his doom-laden outcome has come to
pass.
I think that SM2CEW is over-reacting.
Or
rather
was over-reacting, as I first read Peter's comments some years
ago. He seemed to be making good points at the time. Now I have actually
tried EME and
set this argument against the practicalities of using the Moon, it all
looks entirely
different. Things have moved on in my thinking.
The issue started for me with the question of
whether JT65, Deep Search and the whole thing is a con trick? This is not what Peter said, but it was what immediately mattered to me.
If
indeed deep search was liable to cause lots of false contacts, using
callsigns generated in the WSJT software, then presumably active EME
stations would be deluged with requests for QSL cards for those
contacts. If they were false contacts based on DS-generated data then
the QSOs would not be in their log books and they would have to reject
them. I have not heard of this deluge happening.
In fact, why isn't it happening? What is the factor stopping us all being fooled into meaningless non-QSOs with callsigns generated in our computers?
Could it be that both stations are colluding by denying the weaknesses of JT65 and DS, and confirming QSOs they know are not genuine? Maybe, but I am not so easily taken in and all it takes is a few of us to blow the whistle on something like that and down it comes.
Are we all being hoodwinked by DS? I cannot see how we could be, but I wanted to test it out.
As I understand it, only the callsigns are included in deep search. The
shorthand responses are not. I am not familiar with how the shorthand
responses are verified. For this purpose when I write DS I tend to mean deep search operating as part of the wider JT65 software, as I think that most people would regard any contacts in which any part has been subject to deep search as affecting the whole QSO.
Yes, DS can produce false results:-
 |
| A false decode at GM4FVM while using deep search and averaging. |
This decode is obviously false. It features a
callsign of a station I was trying to work the previous day on FT8. The
frequency (86) is silly as EME stations tend to be evenly spaced 1kHz
apart close to the 1200 mark on the waterfall. A message in that format
is a reply to my CQ call and I was not calling CQ at the time. If he was
calling 1210Hz lower than my tx frequency then he was pretty crazy, or
more likely he does not exist. That call is simply an artifact and it is
easy to see it as such.
In a funny sort of way that type of false decode is good news. It shows how unsophisticated DS's guesses really are.
At this point in my first draft I outlined what I see as a strong argument that after seeing a false posting the chances of a false QSO are tiny. Believe it or not this is a shortened version.
At random, the probability of hitting the correct sequence of responses for a QSO is one right sequence in 823,542 tried outcomes. A JT65 QSO may not look like a random situation but it is - only in the one case where the outcome is correct is there a "right" answer to a decode. All my answers to a false decode would be "wrong" and I might as well send anything, if, indeed, DS is conjuring the whole thing up in my hard drive.
If this statistic seems rather far-fetched then feel reassured that it is the same principle which stops the lottery from being easy to win, or your bank card being cloned. I doubt if this was done deliberately, but the structure around the amateur QSO sets up a format which is very difficult to fake - and just generating random replies does not work very well. So, even if DS could generate callsigns which seemed plausible, the chances of anything randomly generated hitting on the right QSO format and fooling you into thinking you had a real station are tiny.
My analysis is the sort of thing you get when an economist becomes a radio amateur.
So if random responses would not work, could DS accidentally be programmed into JT65 with the result that it produces false QSOs? In other words, can it progress down the QSO structure or does it just fire off once any time it sees something plausible? I put this to various tests. In particular I spent two days waiting for a false decode and when I got one I tried answering it to see if I could generate a false result.
Of course I did not actually transmit my reply. There is no need to as DS had generated the false call out of nowhere (actually a birdie), then could it generate a QSO? However, I still put my reply into the WSJT-X message box and pressed "Enable TX", only turning off the transmit signal path further downstream. The receive path stayed as normal, and the birdie which produced the false decode remained.
The false decode I did get is again obviously wrong, not a CQ, not a reply to my CQ call (which I did not make), so I was being a bit silly replying to it. However, in the interests of science ...
 |
| What happened when I replied to a false decode --- nothing. |
20 minutes later and all my replies had gone without any response. The false QSO had no sign of arriving. It was never going to anyway because no response on the receive side is not something I had figured into my statistics. I have seen the same false decode repeat in the past and I expected that to happen again. No response (or a repeat) means that there is FAR less of a chance of a false QSO being generated than random figures would suggest. At this stage I accepted that the chances of a DS originated false QSO is likely to be no worse than human error and JT65 and DS is probably more reliable that a human.
Anyway, I am not going to do that exercise 823,542 more times only to find that a false QSO is, indeed, less likely than that. Replying to a false decode, or not replying, seems to make no difference, so there is no sign that a false QSO could ever result other than by a hugely improbable set of circumstances.
I also looked at the possibility of there being a real station calling CQ and being answered by somebody else. If I happened to call them at exactly the right time they might reply to someone else and my DS substituted my callsign for the other calling station. Once again probabilities come into this but lets just say that a similar thing could happen in any mode - I have seen it happen often in DXpeditions and contest operations where callsigns are not repeatedly used. The use of shortcuts in EME makes this type of mistake more probable, but the shortcuts came into use before JT65 was developed. This is where I find that validation from the chat room comes into its own, rather than being only a drawback as Peter seems to suggest.
If there really is no practical chance of JT65 with DS actually fooling us, beyond the odd obvious howler, then apart from making it easier for us to complete EME contacts, what is the problem?
What point is Peter trying to make in his piece? He says:-
"We appreciate that all of the historical achievements, like tropo distance records or DXCC
awards that were made using CW/SSB are indeed impossible to compare to the ones made
using digital modes, such as JT65. The fact that Deep Search QSO’s presently have the same
status as traditional modes for DXCC is something that I find totally ridiculous. The amount
of DXCC awards issued on 144 MHz have ”exploded” since the launching of Deep Search.
This is of course nice for the operators who have been wanting such an award, but they cannot
be compared to earlier achievements, using traditional modes."
Hold on. What is a "traditional mode"?
Amateurs previously used CW for EME, in most cases anyway. When it came to working CW they departed from the conventional QSO structure by dispensing with the exchange of callsigns at each over of transmission after the first exchange and adopted the shorthand exchanges "RO", "RRR" and "73". Those were sent without confirmatory callsigns and therefore you could have been receiving the replies from anybody. Are CW exchanges made with that shortcut real QSOs? To what extent is RO a signal report anyway? Does an isolated CW shortcut without a callsign attached confirm anything?
By avoiding the difficulty of having to decode CW reports like 559 the CW EME operators opened the door to challenges that their own minimising information was making EME "too easy". Therefore there is not much scope for them to blame the rest of us when things move on along the same path.
Have we gone too far with DS in further minimising the exchange of information to constitute a QSO? Surely that is a matter of opinion. In my opinion, no, we have not gone too far with DS, simply because I reckon that it use it is supported by the evidence.
I cannot see how JT65, DS, averaging and all the rest are not just further steps in making EME achievable. I am not trying to compare my achievements with people who did EME using CW. In any case, CW EME operators used shortcuts when it suited them, and the minimal information they chose as acceptable could have been said to be insufficient at the time.
This brings my mind back to the comment made by the editor of Practical Wireless who said that he hoped that the arrival FT8 did not make 6m DX so easy that it challenged his DXCC standing. Perhaps he forgets that yagi antennas made DX "easy" for him to work his DXCC entities. Or that SSB made phone contacts "easy". The whole process of developing new methods, circuits, antennas and understanding of propagation is about making contacts "easy".
Sure QSOs set up in chat rooms can open the door to too much information being exchanged over the Internet, which reduces the standing of the QSO. Could the same not have been said about skeds arranged over the phone, or via HF links, or even by post card in the good ole' days?
Over time lots of things have made awards and contests easier or harder. After World War 2 there were three entities (East Germany, West Germany and Saarland) which are now part of one entity - Germany. Did that make it harder to achieve DXCC then, but then before we also had Yugoslavia and now we have Slovenia, Croatia, Bosnia and Herzegovina, Montenegro, Macedonia, Serbia and now also Kosovo. Overall, do these things make DXCC easier or harder over time? You cannot blame deep search for all that, and frankly you cannot blame deep search for people exploiting it as a technical possibility which did not exist before either.
In my view amateur radio is for all of us, and not for the few. Perhaps these few who worked hard and gathered their status in difficult circumstances deserve to be specially recognised. I do not believe that flinging mud at JT65 is the way to increase their standing.
It goes further than that for me. By making EME possible for those with "modest" stations (albeit by piggy-backing on the lavish expenditure of the super stations), JT65 and DS is part of widening and democratising amateur radio. What else should we do? Un-invent JT65 and DS so that ancient old dozers can talk about how they had it so tough in the old days and therefore we should too?
SM2CEW has listed several issues with DS. However, he does not seem to be opposing it. He seems to be asking for the achievement of other operators to be recognised. Fair enough.
He then says:-
"But more important is to consider a better definition of what constitutes a digital QSO!
Increasingly, it appears to be be NOT about an RF radio contact, which in my view a bad
thing for our hobby. I like radio communication and I feel that our QSO’s should be made via
the radio waves. If we continue to allow the computer to decide how little information is
received and guesses what is missing, doing away with the operator, we are heading the
wrong way."
This goes too far for me. It isn't the computer which decides, it is the software which decides what to suggest. I have no problem with the software deciding that the signal level is so low that it will make a suggestion based on probabilities. As you can see, I reckon I am able to distinguish between good and bad suggestions. This is not doing away with the operator in my view.
Peter's statement ignores the rest of the QSO apart from the callsigns, it ignores that part played by QSL cards since the dawn of amateur radio to validate contacts, it ignores the shortcuts and assumptions operators were already making. It is something to hide behind if you want to avoid progress.
"If we continue to do xxx, then something will happen" is often said. But when we did continue, and this did not happen, then can we eventually accept that what we are doing is reasonable? In a particular instance, EME, we are comparing a representation of part
of a callsign with a stored version of a callsign. It seems to work.
Why not do it? I do not see this as part of an inevitable descent into making us Skype operators.
I agree with Peter about this:-
"It should be about skill, judgement and personal integrity."
I trust Joe Taylor on those counts. I hope I show some too.
As I often say about data modes, if you don't like them, don't do them. They are not compulsory.
It is perfectly possible to work EME with JT65 but without DS. You can turn it off.
The challenge of trying EME has prompted me to use a lot of new skills and learn new things. It has prompted me to improve my station, and not just for EME use either. If in the end I have to content myself with a lower standing than the great and the good in this hobby, then so be it. Times change.
=========================================
When Professor Higgs predicted the existence of the famous boson which is named after him I guess he thought he would never see one. Nobody has. The Higgs boson has been "proved to exist", but only by asserting that the probability of certain events occurring for any other reason was very low. Often all we have to go on is probabilities.
It seems to me that SM2CEW's fears are not justified. He is entitled to his views, and so am I. He and others are free to place their earlier achievements on a higher plane than mine. That was then and this is now. I am not in this hobby to preserve my exalted place (just as well?) so it matters naught to me.
There are issues around the exchange of real-time supporting information on chat rooms. The same comment could apply to any mode and KST. Nothing that I have seen on EME chat rooms so far devalues the contacts claimed. In fact the chat rooms seem only to confirm that the contacts reported are genuine.
Does
using JT65 with DS (and averaging) robustly produce results which tally
with real-time activity reported via the Internet and supported by QSL
cards? I believe so.
Do the extremes of improbable QSOs
suggested by using an unreliable mode actually happen with DS and JT65? For example, if DS is
really predicting callsigns out of random noise and scraps of birdies, I could imagine false
QSOs occurring when the Moon was below the horizon? Not in my limited
experience. You might expect a false decode, but not a false QSO.
Has it fooled me too? Like the Apollo moon missions, which I now realise were faked in a gravel pit in Essex.
Ask me later to see if I am still satisfied.
So far it seems to be going fine.
73
Jim
GM4FVM
