Scientific explanations for how Sporadic E happens, and how amateur literature differs (long).

I have not been posting much lately. [What have you been doing Jim?] Well, at this time of year there is not much on the radio so I have been watching Ski Sunday. Isn't  Clément Noël amazing? Why is the Ski Sunday season so much shorter than the ski season?

And now Paris-Nice and the start of the cycling season. Very distracting. Remco Evenepoel, Cian Uijtdebroeks, and all those people.

I just wrote a VERY long piece quoting all the research papers about Sporadic E which I have also been reading. I have been reading research papers because most amateur books and websites say things about Es which are either just plain wrong or full of imaginings about thunderstorms, jet streams, upwards pointing plasma bursts and other fanciful tales.

Mike GM3PPE sent me a good scientific paper and then I turned up another one which together seem to explain the process. Further reading amongst amateur tales made me turn up a third one. So now, rather than using all that material in detail, I will try to set out what I understand. What I understood before was wrong. I can see that now.

I learn that the ionised layer does appear to come from material ionised during entry into the Earth's atmosphere. The E-layer ions last for a remarkably long time before recombining, thanks to being organised by the magnetic field and due their mass being greater than gaseous ions in other layers. The daily pattern of daytime Es during the Summer is driven by solar influenced wind shear. The Es layers descend towards the Earth on a twice-daily air current. There is a wealth of new information in these papers, plus some useful maps.

Here is one of my diagrams. It is an attempt to precis what comes later in the boring text.

Probable Sporadic Es process. Definitely not to scale. (GM4FVM after Arras)

This is a diagram you will need to click on to enlarge, if you want to see the detail. It covers a slice across the ionosphere between about 150km and 80km above ground. The magnetic field would run vertically into the page from above.

What follows replaces the VERY LONG piece with a LONG piece. Sorry, but it isn't easy to summarise.

I will put in the details of the papers at the end. Anything I have taken up wrongly in this piece is my fault.

For this purpose I will try to explain a timeline through the process.

Firstly the iron which will make up the metallic component of the E-layer arrives in the Earth's atmosphere. This is material from meteorites and similar bodies. Most comes from other parts of our solar system but a small amount may come from cosmic dust.

We amateurs are familiar with some of this stuff through meteor scatter propagation. However, many amateurs concentrate their activity on certain meteor showers during the year. These showers allow 5 or 6 days activity per annum. This tends to deflect attention from the fact that the majority of this material arrives at all other times, 365 days per year, 24 hours per day. These are the "random" meteors which hardly souls like GM4FVM and OZ1JXY used to make over 130 contacts entirely outside the meteor shower season.

There have been suggestions that if meteors are involved then the Es season should be influenced by the meteor shower timetable. This only affects Es to a limited extent. The showers produce only a short lived peak in the material arriving and anyway the peak period for arrival of meteors does not coincide with the shower season. Meteor showers are just the short but energetic ones we see and the vast majority are much smaller. In fact these small meteors, called micrometeors, are about the size of a grain of sand. 

Estimates vary but but overall about 25 million objects arrive on Earth each year weighing in total about 15,000,000 kg. The larger of these break up in to small grains. Amounts vary during the year, with a minimum in February and a maximum during Autumn. The amount and energy levels of the arriving particles varies during the day due to the orientation of the observer to the plane of the ecliptic as the Earth circles around the Sun. Amateurs can exploit these variations in meteor scatter propagation but none of it aligns with Es peaks.

These meteors arrive travelling at huge speeds relative to Earth. As they get closer the atmosphere becomes dense enough for friction and chemical action cause them to be heated to huge temperatures. Some are ionised. The ones we are interested in are made up of, or contain, metallic elements - principally iron. In addition to iron, some meteors contain smaller amounts of magnesium, sodium and calcium which also may become ionised. The density of the atmosphere is about right at around 100 to 150km above the Earth for meteors to be heated to the required temperatures. This region thus become a relatively dense layer of ions and we call it the E-layer.

We now have the iron ions (Fe+) and electrons in the right area. What you would expect from something as heavy as a metal is that it would fall to Earth. Even these tiny specks of matter will descend under gravity, though perhaps slowly as the density of the atmosphere increases. 

You would also expect that they would recombine with electrons to convert from Fe+ to Fe. We are familiar with the gaseous ions in the D- and F-layers recombining once the influence of the Sun is deminished at night or during the winter. However, in the E-layer they remain as ions and electrons for long periods. The Earth's magnetic field will exert an influence and tend to organise the two sets of + and - charges along the magnetic field lines. The Fe+ ions are a lot heavier than the gaseous plasma ions in the D- and F-layers and are therefore less likely to wizz around and find electrons. The combination of these processes give enough time for this mass of ioinised matter to come under further influences. The papers reckon it takes between 27 hours and 3 days for the Fe+ ions to recombine.

One of the surprising things to me about this is that the papers suggest that this ionisation takes place by the process of micrometeors arriving on Earth and not from the Sun's energy as in the other layers. I say suggest, as they do not mention solar energy at all as part of the ionisation process. I was wedded to the idea that ionisation in all three layers would be caused by the energy from the Sun. This would explain why Es occurs during the day and mostly during the summer. But apparently not. It matters little though, because the Sun still has a major effect as we shall see shortly. I cling to the idea that the Sun may play a part in keeping the energy level high enough for this story to continue.

The ionised material descends in a particular pattern. What is called "diurnal tides" result in a twice a day air current bringing the layer downwards at a specific rate. Amateurs report a twice a day peak in Es, presumably as the layer passes through the right region for propagation. Other patterns are superimposed on this, including daily- and trice-daily tides, so the picture is not simple.

Next thing in the story is "wind shear". As I understand it, wind shear is a type of turbulence created when a mass of air meets a more or less static object. As a callow youth I looked out the window of my lofty office in Dundonald House, a strangely curved multi-story edifice, and saw the snow falling upwards past the window. Everywhere else it did what you would expect, but here right beside the building it rose and swirled in eddys as if trying to polish the window glass. Not that the air is as dense in the E-layer as it was inside or outside the DHSS office in which I toiled.

The theory here is that the wind shear effect, when present, helps to shape and compress the metallic ion layer into a thin (about 1.3km thick) shaped mass of relatively dense molecules, atoms and electrons. Key to all this is that the wind shear effect is driven by the Sun - which makes Es a largely daytime and summer event. Crucial to it all is that at certain regions the mass of ions becomes sandwiched between eastwards (below) and westwards (above) winds. Although similar conditions occur in both hemispheres, the wind directions in the Southern Hemisphere are reversed in relation to the Northern Hemisphere.

So, many variables are coming into this. Although the quantity and mass of meteors is large it will vary to some extent, the ions are descending and will need to pass through the height we need them to be, they need to stay ionised long enough to create the layer, the diurnal tides need to draw them to the right place and the wind shear needs to be in the right aspect (the papers suggest generally eastwards winds in the Northern Hemisphere, and with an upwards component) to create the layer we need, and reversed winds above. It is surprising it happens at all.

None of this explains the effect we amateurs know as "Winter Es". This is not mentioned in any of the papers I have seen. It may be caused by one of the alternative Es mechanism which create other minor effects. Or it could be explained by the "spillover" theory. Possibly it could just be an aspect of the effect, mentioned in the papers, that Es tends to trail the annual seasonal cycle to some degree. Whatever the explanation, I think that it is important to keep in mind that I find Winter Es is responsible for about 1% of my activity and none of my DX whereas regular Es accounts for the other 99% of the activity and all of the interesting contacts.Winter Es is an odd quirk, but also a very small quirk.

How does the research method used differ from the methods we as amateur use?

The method used in the scientific papers was GPS Radio Occultation (GPS RO or just RO). This involves observing scintillations in GPS radio signals passing through the Es cloud. GPS satellites orbit at about 20200km. For this purpose the GPS signals are not received directly on Earth but are detected by low Earth orbiting (LEO) satellites which orbit at about 2000km above Earth.

Both transmitting (GPS) and receiving (LEO) satellites are outside the E-layer and are, in effect, watching the E-layer from "above" in relation to ground level. Actually they are on either side of the E-layer. As they travel at different speeds to maintain different orbits there are regular windows of about 5 minutes when the GPS signal can be scanned for the presence of an ionised E-layer between them. Any phase changes or multipath interference can be attributed to the E-layers This is not simple though, as other variables in other layers have to be eliminated from the results. However this appears to have been possible.

By using the RO method the researchers are able to observe the Sporadic E layers themselves. We as radio amateurs are only able to observe the results of Es in our log book. Beyond that we can use sites like DX Maps and PSK Reporter (PSKr) to look for patterns and interesting events.

PSKr works by receiving reports from amateurs over the internet and presenting them as lines on maps and other formats. It therefore shows only contacts made, not paths available. It mainly shows data contacts and rarely records beacon reception. Whilst some amateurs can click the box on their software to pass reports on to PSKr, not all do. Many amateurs use directional antennas which limit the possible paths. The results are therefore partial.

DX Maps and similar sites collect data from the DX Cluster and present it in the form of maps and various tables. This information is limited to anything reported to the Cluster, though it includes more reports of CW and SSB contacts plus selected reception reports. DX Cluster has a very helpful map showing the locator squares in which Es layers must be located plus the MUF at that point. This uses a mid point between the path between the reported two stations. This is useful but it a rough estimate, it is historic and it relies on reports - it does not plot the layers themselves.

I use both PSKr and DX Maps extensively. They help me chase DX, but they do not give any clue as to how the E-layer is made up, nor any predictions about how or when the propagation might appear next.

In my view the methods used by amateurs tends to give a distorted view of Es, as of course we rely on other stations being around to receive our transmissions - and for example there are not many in Africa and vanishingly few in the middle of the Atlantic Ocean. The Es map in DXMaps then estimates that the refraction happens mid-way between the stations and shows this as one locator square. This is very helpful but not really scientific.

How does the scientific literature differ from what we read in amateur literature?

The scientific view is that the presence of Fe+ ions and wind shear are the crucial factors in forming a dense thin layer which makes Sporadic E propagation possible. Solar driven winds in the E-layer and the effects of the Earth's magnetic field are also crucial.

Reputable amateur publications give details of the scientific data but steer clear of going further.

Other amateurs have often taken the results from DX Maps, PSKr and other sites and laid them onto weather maps. From this they have come up with the idea that somehow weather systems close to ground play a part in Es formation. As there seems to be no direct relationship between our weather and conditions 100km or so up in the atmosphere they often postulate all sorts of mechanisms without any scientific basis. They then publish this on the Internet as is their right, just as I am doing now. Sometimes their ideas appear in books, society publications and magazines.

The amateur approach creates huge variables because of the indirect and variable nature of the data collection. This is in addition to their often cranky ideas about the influence of Earth weather on the upper atmosphere.

Many amateurs have looked for "triggers" for Es. Unlike the scientists they cannot see the E-layer except where it creates reported propagation for amateurs. I doubt very much if there is a "trigger" involved. The ham literature I have read says that the authors are looking for something they think needs to cause the ionisation in some direction or at a certain time. The scientific papers say that tons of ionised particles arrive every day without any other input and variables in the ionsphere explain the variability. If ionisation is already present, why look for a trigger?

One particular internet piece written by an amateur seems to go right off the path beaten by the scientists. He claims that tools like DX Maps and PSKr give amateurs a reliable way of spotting and tracking Es formation. In my view they only provide a method of tracking the contacts. He reckons that storms and lightning from Earth weather trigger Es. Having looked at the scientific evidence I feel that he is mistaken. All that DX Maps and PSK reporter do is to record some of the outcome of Es filtered through a skewed fabric of random influences.

PSKr often shows just single receptions due to transient effects which are not Es related. DX Maps also shows contacts after they have happened, and only the ones reported to the DX cluster. They cannot show paths which are not reported for all sorts of reasons. They never show potential paths which have not been exploited nor ones that don't exist. Only scientific approaches which, like the ones summarised here, can actually image the existence or not of Sporadic E layers and their location. This is the best way to try to find an explanation for what is happening. This research can show the process involved in creating the propagation in real time as scanned by the satellites.

When well meaning amateurs take charts from PSK Reporter or DX Maps and use them to try to plot Es activity it is almost inevitable that they are using unreliable data. When they go further and try to align this with weather, storm or lightning maps, they are likely to find doubtful patterns. If they go further again and try to predict future Es based on already unreliable information then the outcome is likely to be pretty wide of the mark.

Finally.

Right. Where have we got after all this waffle? There is plenty of scientific evidence as to what causes Es. We as amateurs can use the excellent tools available to us (including PSKr and DXMaps) to exploit this and have lots of interesting fun. By understanding the process by which Es is formed we can develop our learning. As for trying to out-think the professionals - you can try but you might be wasting your time. 

A basic human freedom is that everybody is authorised to waste their time as they think fit. Even me.

Sources.

These are the working links as I write this. Contact me if they do not work and I will try to help if I can.

1) A Global Survey of Sporadic E Layers based on GPS Radio Occultations by CHAMP, GRACE
and FORMOSAT–3 / COSMIC. Christina Arras. 2010 GroForschungsZentrum Helmholtz-Zentrum Postdam. Scientific Technical Report STR10/09

https://gfzpublic.gfz-potsdam.de/pubman/faces/ViewItemOverviewPage.jsp?itemId=item_23022

You can download the entire paper by clicking "full text".

This is a very full (~100 pages) paper, one of the first using the RO method. There is some very good information on how RO works.

2) Examining the Wind Shear Theory of Sporadic E With ICON/MIGHTI Winds and COSMIC-2 Radio Occultation Data. Y. Yamazaki. 2011 Geophysical Research Letters Vol 49 Issue 1.

https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2021GL096202

 As the wind shear theory was not well proven at this stage this paper concentrated on that matter. These are some interesting maps and diagrams.

3) Morphology of sporadic E layer retrieved from COSMIC GPS radio occultation measurements: Wind
shear theory examination. Y H Chu et al. 2014 Journal of Geophysical Research: Space Physics. Research Article 10.1001/2013JA019437.

https://pdfs.semanticscholar.org › c9a0 › 3c2ab977b9507b019e661904c5a4ebdf6fa2.pdf

A further verification of the wind shear theory. This paper includes many more E-layer maps and diagrams which will be of interest to the amateur.