WRTC2014 Site Evaluation Methodology and Results

by Rich Assarabowski K1CC

This paper describes the evaluation methodology used by the WRTC2014 organizing committee to identify suitable operating sites.

The WRTC2014 Site Selection Committee made use of an analytical tool to evaluate sites as part of the search process to find 65 suitable sites. The need for such a tool became very obvious after a few early attempts at deciding whether a site would be acceptable or not. First of all, there is no ideal level ground in New England (except over water!). Second, every location will have varying terrain in different directions. It quickly became apparent that intuition does not work in comparing sites to each other.

A valuable tool for such an analysis is to use N6BV’s High Frequency Terrain Assessment (HFTA) model (Ref. 1). This ray-tracing program uses the terrain profile in a given direction to generate an antenna pattern over the actual topography and then compares it against a statistical distribution of arrival angles for a given target area (e.g. Europe, US, etc.) and calculates a figure of merit (FOM) in dBi. This figure of merit is a weighted average of antenna gain over the range of expected arrival angles for that particular target area. Typical use of HFTA is to optimize antenna heights at a particular location. In our situation, the antennas were fixed and we used the model to evaluate station locations.

The essence of the WRTC2014 terrain analysis methodology is to perform the following for each site:

1. Calculate the FOM for a given direction and band over the actual terrain profile.
2. Calculate the FOM for the same direction and band over flat ground.
3. The difference between (1) and (2) yields the FOM for that direction and band.

An ideal WRTC site would have an FOM of 0.0 in each direction for each band. Since this is highly unlikely except on water, we defined two critical directions for WRTC; Europe (45°) and the US. Because the US covers a wide azimuthal range, we calculated the US FOM as the average of the FOMs at 210° (US Southeast) and 270° (US West Coast).

Our sites often showed significant variation in FOM between bands and directions. Some bands and directions are clearly more important than others. The solution was to weigh the contribution of each band/direction to the final score.

What about other directions? During our site search, we checked to make sure there were no significant hills or slopes in other directions but did not model them. It would simply have taken too much time and would require additional assumptions regarding the contribution of these other directions to the final score which essentially yield only multipliers.

In 2012, in order to estimate a detailed breakdown of a winning WRTC2014 score, we surveyed Committee members and came up with a set of band-weighting factors. A comparison of the predicted winning score from 2012 and the actual winning score of N6MJ and KL9A shows good correlation (see http://konecc.wix.com/HFTA). The bottom line is that we predicted that approximately 80 percent of the score (not number of QSOs) would derive from the European direction and approximately 20 percent from the US direction.

The composite (or overall) figure of merit (FOM) is then calculated as a weighted average of the FOM toward Europe and the United States for each band. These calculations were done for each candidate site and for various antenna locations as they evolved. The various shades on the table made it easy to spot outliers. A perfect site would be close to 0.0 (white), with darker shading corresponding to a higher FOM.

Figure 2 offers an example of FOM results for 15U (Myles Standish) where the K1A team operated in 2014. Note in this example that even though the US FOM is 0.9, it only added 0.1 to the overall FOM. The US direction is just not that important in the IARU contest from New England. Our objective in the analysis, therefore, was to find sites that would be as close to flat as possible using this overall FOM as a figure-of-merit.

We performed a sensitivity analysis to see how the FOM would change if the WRTC stations worked a lot more US stations. The change in FOM was only a few tenths of a decibel for most sites and would have not affected our site selection.

The remaining issue was where to draw the line for rejecting locations as being too good or too poor. In 2012 I made a presentation at the New England Division Convention at Boxboro, Massachusetts, laying out the guidelines we wanted to follow:

• The FOM difference from flat terrain shall not exceed 3 dB on any one band to either the US or Europe.
• The maximum weighted FOM difference from flat terrain for either the US, Europe or both should not exceed 1.8 dB.
• Gain for a range of 4-5 elevation angles should not differ by ~8–10 dB from flat terrain on any band to either the US or Europe (ie, no big peaks or valleys in the antenna pattern).
• The overall FOM weighted difference from flat terrain shall not exceed 1.5 dB.

An easy way to translate these guidelines into more intuitive parameters is to convert them to power levels. For example, a 1.5 dB difference from flat terrain would be equivalent to a station running 140 W instead of the 100 W limit.

There were many sites along the way that exceeded these limits and were rejected.  Most of our final sites, but not all,  were within these self-imposed limts that we defined as objectives back in 2012.   It was either that or simply not have 65 sites for the contest.

Topographic Database Considerations

A step-by-step procedure and templates for the WRTC terrain analysis is available on http://konecc.wix.com/HFTA . A very nice tool for quickly visualizing terrain is at http://www.geocontext.org/publ/2010/04/profiler/en/ which uses the NED topographic database. The software can generate CSV files which can be converted for use with HFTA.

The first step in any terrain analysis is to have a good map of the area. We used http://mapper.acme.com/ for all of our site layouts. It is based on Google Maps, but offers a unique instant overlay of a US Geological Survey (USGS) topographic map (for the US only) at a mouse click. This makes it very easy to quickly “eyeball” locations without running a detailed terrain model.

Next, we needed to establish a credible topographic database for New England. N6BV suggested the digital elevation model (DEM) database is usually the most accurate within the US. The Reagan-era motto “trust but verify” led me to pick several test sites to give me confidence in the model. One of the Wrentham sites (10A) was a good one. The terrain has a very slight downslope toward Europe, is relatively flat to the southwest and flat to the west, but then dropping rapidly toward a lake. With Dean’s help, we generated National Elevation Dataset (NED), DEM, and shuttle radar topography mission (SRTM) databases. I painstakingly compared the profiles in various directions using USGS maps as a benchmark. The lake to the west of 10A was a good reference point. The terrain profiles did not always accurately define the edge of the lake, and in some of the terrain databases, the terrain over the lake was not flat(!). After due consideration, I selected the NED for our WRTC analysis.

Assumptions and Limitations

Our methodology incorporated multiple assumptions. One of the biggest unknowns is the arrival angle distribution used in the model. We used N6BV’s published arrival angle distributions for each target area (Europe and the US) which are based on VOACAP predictions. The supporting data for these elevation angle statistics is acknowledged to be weak but it’s the best we have (Ref. 2). In addition, arrival angles can be very different depending on the Solar Spot Number (SSN), geomagnetic disturbances, season, and time of day. The VOACAP does not calculate arrival angles for Es and double-hop Es propagation which frequently occurs in summertime propagation. Given the absence of any better data, this is what we used and it was a reasonable choice in predicting propagation two years prior to the event.

Here is a list of other considerations and observations made as a result of running HFTA hundreds of times over the course of 2½ years:

1. When examining HFTA plots, it is very important to realize that there are artifacts in the calculations. One must not try to interpret small peaks and valleys in the antenna response curve as necessarily being significant. A good example of that is the Mansfield airport location (12A), where small dips in the antenna response happen to coincide with peaks in the arrival angles, leading to an overall FOM of -0.3, making the site model appear worse than it actually is. Real-world peaks and valleys in antenna response are much “smoother” than the model shows. Any micro-analysis of these small perturbations is beyond the resolution of this model and the underlying terrain database.
2. The HFTA results can be very sensitive to terrain data across different databases (NED, DEM, etc). Running sensitivity tests with small perturbations in antenna location, terrain files and azimuth for some of the sites occasionally showed a significant variability in results, especially on hilly and bumpy terrain.
3. Intuition regarding the effects of terrain on antenna pattern is not very reliable. The effect of terrain on a 40 foot high antenna is much different than it is on a 100 foot antenna. Local terrain features (saddlebacks, humpbacks) within the ground reflection zone can sometimes result in very large gain differences over a narrow range of radiation angles. HFTA has been used very successfully by station designers, but it is not a rigorously validated model.
4. We used the default distance for terrain profiles (4400m) as recommended by N6BV for general use. However, my own empirical observation after making hundreds of runs, the ground reflection zone for a small 40 ft high antenna is no more than 1000m for all practical purposes. There may be secondary reflections occurring due to diffraction over the bumpy terrain at distances >1000m. Terrain beyond 1000m is of not much consequence at these low antenna heights (Ref. 3). Careful study of the terrain profiles and HFTA plots will bear this out.
5. There can be significant diffraction effects over bumpy terrain we encountered in many of our sites. Turning diffraction on and off in HFTA illustrates this vividly. This further complicates any intuitive predictions of how well a site will perform.
6. HFTA does not work with vertical polarization. The inverted-V’s on 80 and 40 have a vertical polarization component which is not accounted for in the model.
7. Man-made features such as fill for roads can distort the results somewhat. This was the case, for example, at Hollis (1A) where the actual antenna location on a recently constructed sports field is not reflected accurately in the terrain database.

Terrain Data and HFTA Plots

All terrain profiles and antenna patterns for each of the 59 teams are available at WRTC2014 terrain and HFTA data 13_Aug_2014 (Ref 4). For each site, the terrain profile is shown followed by HFTA plots, for both Europe and for the two US directions. For the US, the blue curves are for 270 deg and the red for 210 deg. Note that the Rice sites (11B, 11C and 11E) are labeled as “Wrentham”, which was our internal designation for this property, so this is not an error. All terrain plots have been adjusted to the same scale so they can be easily compared.

Comparison of Model to Final Results

A comparison of final scores with the overall FOM for each of the sites is shown in Figure 3.

There is no correlation to the final score as seen by the linear regression analysis (R² = 0.0056). A perfect correlation would give an R² of 1.0. The range of FOM was from -0.3 to 2.0. The highest FOM site was Twin Valley (FOM was 2.0) which was a spare site pressed into service at the last minute. The median FOM of all the sites was 0.6. The plot of just the top 20 scores and their station calls signs also shows no obvious terrain advantages.

See the map at http://www.wrtc2014.org/map-of-operating-sites/ for the location of all operating sites.

Site Comparisons

The tables below show each of the 59 sites and their calculated FOMs.

North Sites

Site	Call	Operators	Coordinates	Overall Weighted diff. from flat (dB)	Europe Weighted diff. from flat (dB)	US Weighted diff. from flat (dB)
01A	W1T	AD4Z W4UH	N 42.73222 W 71.59279	1.2	1.5	-0.4
02A	W1Z	N5DX N2IC	N 42.70303 W 71.58863	0.5	0.6	0.5
02B	W1U	LZ4AX LZ3FN	N 42.70127 W 71.57982	0.9	1.1	-0.2
03A	K1U	KF5EYY YT1AD	N 42.68956 W 71.59410	1.0	1.2	0.1
04A	N1M	K9VV VE3EJ	N 42.66727 W 71.62870	1.6	1.8	0.2
04B	K1Z	VE7CC VE7SV	N 42.67007 W 71.62269	1.4	1.6	0.3
05A	K1W	K6AM N6AN	N 42.68618 W 71.59022	2.0	2.3	0.2
05B	N1T	ES5TV ES2RR	N 42.68117 W 71.58812	0.9	1.2	-0.8
06B	K1C	KE3X K0DQ	N 42.52024 W 71.60899	0.9	0.9	0.9
06C	W1F	CT1ILT CT1BOH	N 42.54489 W 71.62133	0.0	0.0	0.0
06D	N1V	K7RL KL2A	N 42.54663 W 71.63347	1.2	1.4	0.3
06E	K1V	G0CKV M0DXR	N 42.53472 W 71.63042	0.4	0.3	0.9
06G	N1D	NR5M W2GD	N 42.53492 W 71.60117	1.0	1.1	0.3
06H	K1F	VY2ZM KK6ZM	N 42.54787 W 71.61504	0.7	0.7	0.5
06J	N1Z	PY1NX LZ3YY	N 42.52311 W 71.60373	1.4	1.5	0.9
06K	K1I	UU4JMG UU0JM	N 42.54299 W 71.63744	0.5	0.2	2.0
06Q	W1N	5B4WN 5B4AFM	N 42.54138 W 71.60629	0.8	0.8	0.5
07B	K1P	M0CFW GI0RTN	N 42.48372 W 71.77833	0.7	0.5	1.6
07C	N1P	CX6VM LU1FAM	N 42.48904 W 71.77934	0.3	-0.3	2.3

South Sites

Site	Call	Operators	Coordinates	Overall Weighted diff. from flat (dB)	Europe Weighted diff. from flat (dB)	US Weighted diff. from flat (dB)
08A	K1M	IK1HJS I4UFH	N 42.21484 W 71.33339	1.3	1.5	-0.1
08B	N1A	DL1QQ DL8DYL	N 42.20961 W 71.33097	1.3	1.6	-0.4
08C	W1R	OH2BH OH2MM	N 42.21174 W 71.33778	-0.3	-0.7	1.3
08D	W1M	4O3A HA1AG	N 42.20899 W 71.34627	1.3	1.6	-0.2
09A	N1O	RC9O UA9PM	N 42.21764 W 70.83317	0.7	0.9	-0.3
09B	W1W	OH2UA OH6KZP	N 42.21911 W 70.84024	0.2	0.2	0.3
09D	K1G	9A6XX 9A1UN	N 42.22343 W 70.84605	1.1	1.3	0.1
09F	K1O	JH5GHM JA1OJE	N 42.21499 W 70.85559	0.1	0.1	0.1
09G	N1L	KU1CW EA5GTQ	N 42.21324 W 70.84757	1.8	2.2	-0.5
09H	N1R	UA3DPX UA4FER	N 42.18258 W 70.84749	0.5	0.6	0.1
10A	W1B	OE2VEL OE5OHO	N 42.08580 W 71.32105	0.6	0.7	0.4
10F	W1D	K1LZ YT6W	N 42.08238 W 71.32759	0.4	0.4	0.5
10G	K1K	RL3FT RA3CO	N 42.09002 W 71.31741	1.6	1.8	0.3
11B	N1G	RX3APM RV1AW	N 42.07739 W 71.32641	0.3	0.1	1.0
11C	K1D	UR0MC VE3DZ	N 42.07470 W 71.32142	0.8	0.8	0.9
11E	W1S	F8DBF F1AKK	N 42.07891 W 71.31810	0.8	1.0	0.0
12A	N1F	RW1A RA1A	N 41.99903 W 71.19228	-0.3	-0.3	0.2
12D	K1N	OE3DIA E77DX	N 41.99997 W 71.20113	0.1	0.0	0.2
13A	K1R	N4YDU N3KS	N 41.88059 W 70.99020	-0.1	-0.5	1.3
13B	W1G	F4DXW F8CMF	N 41.87113 W 70.98153	0.5	0.5	0.5
14A	W1L	OM3BH OM3GI	N 41.78610 W 71.05130	1.3	1.6	0.0
14C	N1K	DK6XZ DK9IP	N 41.81255 W 71.10927	0.4	0.5	0.1
14D	W1A	LY9A LY4L	N 41.83188 W 71.11659	0.6	0.7	0.2
15A	W1C	9A5K 9A1TT	N 41.83721 W 70.65687	0.2	0.2	0.5
15B	W1O	OM2VL OM3RM	N 41.83365 W 70.65223	0.6	0.7	0.0
15C	W1K	BA5CW BA7IO	N 41.83503 W 70.64477	0.8	0.9	0.4
15D	W1I	W2RE WW2DX	N 41.83039 W 70.64612	0.4	0.4	0.4
15E	K1S	W2SC N2NL	N 41.83243 W 70.65804	0.4	0.4	0.3
15F	K1L	S50A S57AW	N 41.84834 W 70.69252	-0.2	-0.4	0.4
15G	K1T	IZ1LBG WQ2N	N 41.81350 W 70.67121	0.0	0.0	0.1
15H	N1N	KH6ND KH6SH	N 41.84928 W 70.68134	1.2	1.4	0.3
15I	N1U	K8MR K9NW	N 41.84859 W 70.67522	-0.2	-0.1	-0.4
15J	N1B	YL1ZF YL2GQT	N 41.81724 W 70.67938	0.6	0.7	0.2
15M	N1W	PY2YU PY2NDX	N 41.86035 W 70.66443	0.4	0.3	0.6
15N	K1B	W9RE N5OT	N 41.85969 W 70.67053	0.2	0.0	1.0
15O	W1V	R9DX UA9CDV	N 41.85283 W 70.67272	0.6	0.5	0.9
15P	N1S	LX2A YO3JR	N 41.85925 W 70.68597	0.1	0.1	0.0
15R	N1C	IK2NCJ IK2QEI	N 41.85886 W 70.67814	1.0	1.1	0.6
15U	K1A	N6MJ KL9A	N 41.82532 W 70.64618	0.4	0.3	0.9
15W	W1P	DJ5MW DL1IAO	N 41.82787 W 70.65786	0.2	0.1	0.4

REFERENCES

1. The software comes on a CD with purchase of the ARRL Antenna Book
2. See http://www.voacap.com/angle.html for an excellent discussion of arrival angles in VOACAP
3. See http://www.vk3um.com/Ground%20Gain_DUBUS%203-2011_ON4KHG.pdf for more detail on basic ground reflection theory
4. Terrain profiles and antenna elevation angle patterns for each of the 59 competition sites are available at WRTC2014 terrain and HFTA data 13_Aug_2014 Note: This pdf file is easier to view in Adobe Acrobat reader than in your browser. Right click on the link and select Save As… to copy to your computer. Then double click on the file to view.