PiAware & Automatic Gain Control

Odd fluctuations in my PiAware ADS-B receivers’ coverage graphs made me wonder how AGC in dump1090-fa really worked, because information on the subject is contradictory. A common technique for gaining insight into a process is by studying its anomalies, and the following announcement offered a promising opportunity.

     February 12th, 2019
For one of my receivers, air traffic over Belgium is the main source of ADS-B messages, especially in the 80-160 km range. So I was expecting to collect substantially less position broadcasts during the 24-hour strike. That proved to be the case, but I also noticed something else: for the first time since I permanently mounted the antenna outside, I was finally receiving a considerable amount of long distance (420+ km) positions again.

[Note that the bar graphs have different scales]

I recognized the above behaviour from the time when my antenna was still mobile. It always occurred immediately after moving the antenna from inside to outside the house. The effect would then gradually fade during the next couple of hours.

Conclusion so far is that many strong (relatively nearby) messages will make the AGC mechanism gradually reduce the receiver’s sensitivity. This may be beneficial for the total number of aircraft seen, but it comes at the cost of long distance reception. Once the strike was over, long distance reception dropped again within a couple of hours.

I’ve been experimenting with fixed gain values in order to receive more long distance aircraft, mainly to determine how far my receiver could reach. However, none of the allowed gain values gave results that resembled what I got during the 24-hour strike, or shortly after moving the antenna from inside to outside. Could it be that AGC does more than just adjust the receiver’s gain?

 

 

 

Both Sides Now…

With my FlightAware feeder now permanently mounted outside, I began thinking of installing a second ADS-B/Mode-S feeder at the opposite side of my apartment. Would the extra reception coverage be worth the costs, as the second antenna could only be placed inside, blocked by some nearby obstacles. Earlier tests with a NooElec NESDR SMArt dongle and a simple antenna had been quite disappointing, but after placing the antenna behind a solid wall instead of a window, reception suddenly became interesting enough to make me buy a second RasPi + FlightAware Pro Stick Plus + 66 cm antenna.

Antenna coverage graphs of both feeders

 

First I had the second RasPi run just dump1090-fa (not the complete Piaware suite) and forward its decoded messages to the first feeder. A picture on the FlightAware site illustrates how processes are linked together.

[not shown is port 30004, on which dump1090-fa is listening for incoming ADS-B messages from other feeders]

I used netcat for forwarding messages from one feeder to the other:

 

Forwarding messages from dump1090-fa to the FlightAware feeder boosted its detected aircraft figure, but (inevitably?) at the cost of its MLAT synchronisation/reception. So I ended up with both feeders running Piaware and showing up as two separate ‘sites’ under my flightware.com account. For visualizing detected traffic from both receivers I use Virtual Radar Server’s local webpage (http://127.0.0.1:88/VirtualRadar/desktop.html#) on a Windows laptop.

The image below shows my Virtual Radar, with combined coverage graphs of both receivers’ ADS-B & MLAT reception. The outer black circle is at 400 km from my location, so I’ve been receiving aircraft from well over 600 km distance. Occasionally, particularly when nearby traffic is low, the receivers pick up positions from even further distances (800+ km), but these are too irregular to be represented in VR’s coverage graph.

 

[ I could figure out which of the feeders gets most MLAT aircraft, and then forward that feeder’s ADS-B messages to the other feeder. This would stop that other feeder from receiving MLAT aircraft and make it feed FlightAware with ADS-B aircraft from both feeders. That would substantially raise its site ranking as well as my FA user ranking (ADS-B aircraft from the first feeder would be counted twice). I suspect that FA wouldn’t be pleased, so I’ll stick to operating two strictly separated feeders. ]

ADS-B experiences

After two months of experimenting with a generic SDR dongle, a dedicated ADS-B dongle and a couple of antennas, here’s my experience so far.

 

Used equipment:

  • FlightAware Pro Stick Plus
  • NooElec NESDR SMArt
  • FlightAware antenna (66 cm)
  • Magnetic ground plane antenna
  • Telescopic FM antenna
  • Raspberry Pi 3B+ (Piaware image)
 

In order of importance, expressed in an (approximated) percentage of influence on my ADS-B reception, this is what helped to improve my FlightAware feeder statistics:

[70%] – Antenna placement. Outside is better than inside (obviously), especially for houses with energy saving windows (metal coated glass reduces signal strength ~40%). Higher is better, in general, but it pays off to experiment:

  • Raising my outside antenne by 80 cm, as well as increasing its view angle by 2 degrees, made it receive more aircraft – but at the cost of most of its >400 km reception…
  • My inside antenna turned out to perform much better after moving it from the window to a wall, where an obstacle outside did no longer block a high air traffic area.
  • Since ADS-B is basically a ‘line of sight’ transmission, it’s clear that obstacles can block signals. But they can also act as rear view mirrors! A large apartment building that blocks my long range reception in one direction compensates for this by bouncing signals from an otherwise completely blocked direction.  

[20%] – Antenna type & quality. The FlightAware antenna isn’t cheap, but its 5 dB gain will extend your reception range compared to simple 1/4 or 1/2 wavelength antennas.

[5%] – Dongle. The FlightAware Pro Stick Plus squeezed a bit more out of all tested antennas. And it didn’t get as hot as the NooElec.

[5%] – Cables & connectors. Use quality cables and keep them as short as possible. Also: periodically check your SMA and/or N-type connections (they can get loose over time).

Tip 1: The amazing Virtual Radar Server (VRS) software is extremely useful for finding the best antenna position. It can create receiver range plots over time, even for different altitude ranges. This allows for a visual comparison of different antenna positions and choosing the one that fits your needs (i.e. best coverage of a particular area, best long range reception, etc.)

Tip 2: Experiment with your receiver’s gain. This is a setting in dump1090 (the demodulator/decoder software component). Default setting in the Piaware image is ‘AGC’ (Automatic Gain Control), that is supposed to dynamically adjust gain in order to avoid ‘clipping’ of the strongest signals. However, some reliable sources claim it will set gain to ‘max’ (49.6 dB) …

If you have ssh enabled on Piaware, here’s the commands to read the current gain setting:

sudo systemctl status dump1090-fa -l

To set gain to, for instance, 40.2 dB:

sudo piaware-config rtlsdr-gain 40.2

Make sure to restart the dump1090 process for your changes to come into effect:

sudo systemctl restart dump1090-fa

Possible gain values are: 0.0, 0.9, 1.4, 2.7, 3.7, 7.7, 8.7, 12.5, 14.4, 15.7, 16.6, 19.7, 20.7, 22.9, 25.4, 28.0, 29.7, 32.8, 33.8, 36.4, 37.2, 38.6, 40.2, 42.1, 43.4, 43.9, 44.5, 48.0 and 49.6

Note: supplied values will be rounded to the closest value in the list. A value of -10 will set the receiver in AGC mode.

 

 

 

Pi in the Sky

It had been waiting on the runway for some time, but now my tinkering with Raspberry Pi  has finally been cleared for takeoff. After getting familiar with my FlightAware ADS-B dongle on Windows (previous post), using a RasPi as a 24/7 FlightAware feeder was a logical choice. No Linux knowledge required: just download the PiAware image, burn it on an SD card, boot the RasPi with it and claim your device with flightaware.com.

The essential program here is dump1090, a clever piece of software for decoding raw Mode_S messages to human-readable flight information. Collected data is made accessible through its built in web server, via TCP, or in json format (default address: http://<local_ip_address_of_piaware>:8080/data/aircraft.json). The program is also highly configurable:

The PiAware package comes with Skyview, a web interface for displaying received flight data from your receiver. It connects to a local web server, so no Internet connection is required. I also discovered that the Virtual Radar program that I use on my Windows PC can be configured to retrieve its data from RasPi’s dump1090 instance:

Note: the above setting will only grab ADS-B aircraft. If your Piaware receiver has MLAT enabled, you can configure a second receiver, listening to port 30105 of the RasPi. Then you can combine both aircraft types in a merged feed (Tools – Options – Merged Feeds) and select ‘Merged Feed’ as receiver in the Virtual Radar web page.

The Virtual Radar web interface is similar to Skyview, but it shows some additional information in the right pane, like aircraft photos. It can use a local aircraft database for filling in missing data like registration and aircraft type, based on the received ICAO code.

Now that I’ve come this far with my close-to-zero Linux knowledge, it’s time for some new challenges:

  • Expand coverage by placing a second ADS-B receiver at the opposite side of my house, merging data from both receivers (= boost my FlightAware ranking) by having an additional instance of dump1090 acting as a hub (–net-only mode).
  • Feed other flight tracking services with the same (merged) data, like flightradar24.com and – the original goal of this whole excercise – adsbexchange.com. There are scripts for that, but I want to make sure that they will not mess up my current setup.
  • Make a local version of my What’s Up sketch, that gets its json data from the RasPi.

Some of this may require some (hopefully not too much) Linux knowledge, and at least SSH access to the RasPi(s), which is said to be easily achieved by putting an empty file named ‘ssh’ in the /root folder on the PiAware SD card.

So far, we’re on schedule. Stay on board and enjoy your flight.

Radar Love

My What’s Up? flight radar sketch for esp8266 relies on adsb-exchange.com for its data. In order to do something in return, I bought this ADS-B receiver and a decent 1090 MHz antenna*.

The idea is to become a feeder for the adsb-exchange.com and flightaware.com servers, sharing transponder data that’s picked up from planes within reach of my receiver.

The quickest way for me to test the device was by plugging it into a Windows 10 tablet. Everything turned out to be simple: within 10 minutes I was looking at “my airplanes”, quietly moving over a map. Even with the antenna placed indoors, the receiver managed to ‘see’ planes at a distance of up to 340 355 390 km! (even> 450 km after opening the window). If my rough calculations are correct, that’s close to the distance where an aitplane at 12 km altitude will disappear behind the antenna’s horizon. Impressive.

This is the (free) software that I used on Windows 10:

  • zadig : this magic tool lets you replace the default Windows driver for the dongle by an alternative driver that supports RTL/SDR functions. This needs to be done just once (per usb port?).
  • dump1090 : extracts and decodes the received ADS-B messages. Keep this program running because it provides the source information for virtual radar (see next).
  • virtual radar : starts a local web server and opens a console window with some statistics + the url of a local web page, displaying all planes within reach of the receiver.

After some experimenting in order to find the best antenna position, my receiver is now part of a Raspberry Pi-based FlightAware ADS-B feeder. Next step will be to feed the  adsb-exchange.com database simultaneously.

Piaware Skyview

*Surprisingly, this € 60 antenna performs only slightly better than a cheap 6.8 cm antenna that came with a Nooelec SRD dongle. Placement of the antenna is clearly more important.

 

What’s up? (finishing touch)

Today I finished my What’s up (flight monitor) project by adding touch functionality to the TFT display. Touching a plane on the map will temporarily turn its icon red and display its flight details in a popup text area. After a couple of seconds, the map will be restored to full size again.

[Background conversation is live air traffic control from Amsterdam Airport Schiphol; video quality suffers from interference between the TFT display and my smartphone’s camera]

I had to use a STMPE160 touch screen controller because an esp8266 board like the Wemos only has one analog input, whereas my display needs two. Adafruit’s STMPE160 library seems to lack reliable control over the touch buffer, resulting in old touch point data being returned when a new touch event occurs. Luckily, I found a way to flush the buffer and make the sketch work.

I learned a lot from this challenging project and I’m quite satisfied with the final result. Intermediate versions of this project are here and here.

What’s up? (graphical)

Going graphical was a logical step between my original text-based flight monitor and the final touch screen version. Here’s the result on a 480×320 TFT display, powered by an esp8266 based Wemos D1 R2. It shows spotted airplanes over the displayed area, updating their positions every 5 seconds.

The suddenly appearing fourth airplane obviously just switched on its transponder while standing on the runway of Eindhoven Airport, that’s why it doesn’t move.

Military airplanes will appear in a different color and I’m considering the use of different icons for helicopters, gliders, air balloons and UFOs…

The refresh of the airplane icons isn’t as fast as I had hoped for, especially if there are many (can’t do them one at the time because they can overlap). This HX8357D driven Adafruit 3.5″ display is great, but not exactly fast. Let’s hope someone optimizes the library one day. Anyway, I’m still quite satisfied with the overall result so far.

See also: the final touch screen version of this project. Clicking a plane temporarily shows its flight details in a popup text area.

 

 

What’s up?

Sitting on my balcony on a cloudless day, I can see airplanes passing by almost every single minute. Where do they all come from and what’s their destination? Sites like fligthradar24.com can tell, but wouldn’t it be great if real time air traffic information would be freely available on the Internet. It was after discovering global.adsbexchange.com that I realized that, in this respect, the sky is not the limit!

Sending a request to their REST API will return a JSON-formatted response, containing flight data on all planes within the scope of your request. Here you can find a full explanation of the request parameters, as well as the returned data.

[Text-based prototype; see also the graphical prototype and the final touch screen version]

Reading JSON on-the-fly

Because the JSON response string will contain full information on all planes within the requested scope, it’s likely to be too large to be stored in memory. That means that  information of interest has to be extracted on-the-fly (how appropriate!). This requires a parser that processes the JSON response as a stream.

First I tried this generic JSON streaming parser, ported from php to esp8266 by Daniel Eichhorn, who uses it for his far more ambitious PlaneSpotter project. Although that parser worked great, I ended up writing my own “ultralight” adsbexchange parser, basically just for fun. It uses an array to store the names of keys that you want to extract, making it easy to configure your sketch according to your needs.

 

Displaying values from the JSON response using the Adafruit_GFX library (not trivial…)

In order to properly display special characters in the names of airfields, airplane models or operators, you must first realize that the JSON response is utf-8 encoded, whereas the GFX library expects extended ascii compliant with “Code page 850” (aka Latin-1).

A commonly used solution, proposed in the Arduino Playground, still produced lots of wrong characters because it was written for a different code page, so I had to write my own print_utf8 function for translating utf-8 encoded characters to the correct extended ascii equivalents, if they exist.

Secondly, many characters can appear in the JSON response that don’t have an extended ascii equivalent (mostly Cyrillics). My function will ‘romanize’ them (i.e. remove their accents, preserving their case). A two dimensional translation table transtables is used, stored in PROGMEM (see sketch).

The image below shows the correct codes for all characters that can be displayed by the Adafruit_GFX library (Code page 850 – Latin-1 ).

[code table for the Adafruit_GFX library]

 

The sketch

The example sketch below is for esp8266 based boards (I used a Wemos D1 R2). It monitors up to ten airplanes within 35 km distance from a given location, showing up to five of those on a 480×320 TFT display. Flight information is updated every 5 seconds.

Just for this post, I’ve put everything in one long sketch that shouldn’t be too difficult to grasp though, due to the section layout and many comments. Don’t forget to change WiFi credentials and parameters of the url in the WiFi Stuff section of the sketch, as well as display specific lines.

Note: As a next step, I made a graphical version, showing planes moving over a map. Then I finished this nice project with a touch screen version.