Swinging Needles

…finally!

It took me some time and trouble, but I finally managed to have my ESP33/VS1053 based Internet Radio drive two vintage style analog VU meters!

Web browser controlled prototype without the matrix keyboard attached

 

Soon after I had purchased a rather obscure VU meter kit, it turned out that the driver board was completely useless because it was mono (Chinese for stereo?).

So I decided to have the ESP32 read the analog (DC-biased) line out signals from the VS1053, map the peak-to-peak readings to [0,255] scaled values and send these to the ESP’s DACs for driving the meters. However, I couldn’t figure out how to do that without having to connect the VS1053’s floating audio ground to the system ground, which, so I guessed, would either not work, cause noise or even damage something.

Then I remembered the VU meter plugin for the VS1053, enabling you to read dB levels from a memory register. But these plugin readings are leaking peak dB levels, so I started wondering if it was possible to use them for reconstructing instantaneous dB levels. Not suitable for audio purists, but I wouldn’t mind if the result was convincing. After all, these cheap Chinese meters will never behave like professional VU meters anyway.

So I wrote a simple algorithm with some tuning parameters and functions for scaling and smoothing, calculated resistor values for limiting current to my meters to 0.4 mA, added (probably unnecessary) flyback diodes to protect the ESP32 from negative voltage spikes, and just gave it a try. Here’s the wiring that I used.

The first result looked surprisingly good: the needles eagery followed every move and twist of Dylan’s Thunder on the Mountain. Since everything is controlled by an algorithm, any desired meter characteristics, like logarithmic scaling, responsiveness or needle damping, can be easily simulateded without electronics.

For now, further fine tuning seems unnecessary. Besides, a more audiophile approach already crossed my mind: have a FreeRTOS task listen in to the mp3 stream while it’s being sent to the VS1053, and programmatically decode it. That will allow me to extract true dB levels and could even become a replacement for the VS1053 board. Like I recently read in an article about Software Defined Radio (SDR), “software is the new hardware“.

Anyway, music finally looks almost as good again as it did in my youth. Let’s just hope for an analog VU meter revival soon. And how about a global rediscovery of reel-to-reel tapedecks (with analog VU meters, of course)? That would make much more sense to me than the somewhat puzzling comeback of vinyl.

Mini MP3 Player

This little €4 module filled my entire weekend, mainly spent on finding the right documentation and a library that will work on Arduino, ESP8266 and ESP32. It’s a DFPlayer Mini MP3 Player from DFRobot (or, more likely, one of its many clones).

It can play MP3 and WAV files, stored on an SD card in the onboard card reader or from an attached USB drive. So far, I only tested MP3 files from SD.

You can connect one (4 Ω) speaker to the (quite impressive) onboard mono amplifier. Stereo line output is said to be provided on the DAC_R and DAC_L pins.

The main control method of this player is via commands over the module’s 9600 baud serial port. A subset of most commonly used commands can also be executed by means of two GPIO pins and/or two resistor-sensitive ‘ADKEY’ pins.

I made a false start by testing it on an ESP32 over hardware serial. That didn’t seem to work, so I tried an ESP8266 over software serial. That didn’t work either, and I had reached the point of giving up, assuming it was just another Chinese fake, when I gave it one more try on my good old Arduino Uno. With a quick & dirty 1K resistor for level shifting the module’s RX pin, the example sketch of one of many libraries finally made the little guy play me a song! In the end, it was the power supply that made the difference.

Some important things that I learned so far:

  • The module works best when powered by a stable 5 Volt. My Arduino Uno and Wemos D1 R2 could deliver it from their 5V pin, smaller ESP8266 and ESP32 boards couldn’t. In that case: use a separate 5 Volt power supply.
  • Wiring as suggested here caused a shutdown of my laptop’s usb port! I’m still searching for a way to connect the module to ESP32.
  • The module operates on 3.3 V logic level, so make sure to level shift the connection to the module’s RX pin when using a 5 V microcontoller.
  • Startup noise disappeared after I connected both GND pins from the player to ground.
  • Although it’s for a Flyron FN-M16P module, this excellent manual/datasheet has more and better written information than DFRobot’s own summary. Apparently both modules use the same commands and probably have the same chip.
  • Getting stereo ouput from the DAC pins is still a problem. According to the manual, DAC is enabled on startup, but is it? Explicitely enabling it by sending 0x0000 to memory address 0x1A made no difference while sending 0x0001 instead (supposed to turn it off) started a terrible noise…
  • I tested quite some libraries from github, most of which have issues [update: starting sketches with delay(1000); solved most of them] and seem unnecessarily complicated to me. An exception is this simple and transparent library that is basically just a wrapper for sending serial commands. It can easily be extended with additional methods, even by unexperienced C++ programmers. You just have to lookup the command- and parameter bytes for the desired function in the datasheet and create a new function to send them (see example at the bottom of this page). [update: the author of the library has meanwhile added more commands]
  • The module has an SPK_1 and an SPK_2 pin, with a GND pin in between, which could suggest that it has a stereo amplifier. Although the documentation clearly says it’s mono, someone actually published a wiring diagram with two speakers connected. This person may own a different version of the module, but mine definitely has a mono amplifier. So the two wires of a single speaker should go to the SPK pins.

[ Wiring for Arduino Uno, with SoftSerial on pins 10 and 11; don’t forget the 1K resistor! ]

 

The location of files and their names is also very important because of the way commands address them. All information can be found in the datasheet, but here’s a summary:

  • The SD card’s maximum size is 32 GB. It must be FAT or FAT32 formatted.
  • There can be up to 3000 .mp3 (or .wav files) in the root directory of the SD card, named 0001.mp3 to 3000.mp3.
  • The root can have up to 99 subdirectories, named 01, 02, … 99. Each of these subdirectories can hold up to 255 files. Their names must start with 001, 002, … 255,  but you can append decriptive titles to these files, e.g. 007_James_Bond_Tune.mp3.
  • You can have a special subdirectory in the root named MP3. This can hold up to 3000 files, named 0001.mp3 to 3000.mp3.
  • You can also have a subdirecory ADVERT in the root. Special commands are available for making tracks in this folder interrupt currently playing tracks from the MP3 folder (for advertisement puposes). The rules for the MP3 subdirectory apply here as well.
  • Don’t create subdirectories inside subdirectories.
  • The module does not actually use file- or directory names, but the order of files and directories in the SD card’s FAT. The previously mentioned naming conventions just try to give you control over that order. If you properly organize your audio files in a folder on your PC first, and then copy that entire structure to a freshly formatted SD card, then (in theory) the FAT order should match the file- and directory numbering. See also this tool

Apart from adding some extra functions to the above mentioned library, I still have to find out how to reliably receive and read the module’s response to commands and queries. Once that I’m in full control over this (for its price) remarkably versatile module, it will be useful for a couple of ‘talking’ projects that I have in mind. Probably just mono and not ESP32 driven, but anyway: lots of fun for just €4.

 


Here’s an example of a function that I added to the DFPlayerMini_Fast library. Calling play_in_dir(2, 5) will play file 005.mp3 in subdirectory 02.

Using an existing function as a template, I first added my new function to the DFPlayerMini_Fast.cpp file:

Then I added these two accompanying lines to the DFPlayerMini_Fast.h header file:

and (inside the code for the class):

 

Explanation: according to the datasheet, the 10 (hexadecimal) bytes that have to be sent (by the sendData() function) to play track 005.mp3  in directory 02 are:

7E FF 06 0F 00 02 05 xx xx EF

7E FF 06 are the first three bytes for every command. The following 0F is de code for ‘play a file in a direcory’, the following 00 means ‘no confirmation feedback needed’ and is followed by the two parameters for the desired directory- and file number: 02 and 05. Then comes xx xx which are two checksum bytes that will be calculated and replaced by the findChecksum() function of the library before the command is sent by sendData(). Finally, every command has to be closed by EF.

As you can see, my newly added function play_in_dir() takes two parameters only, corresponding with two bytes in the command sequence. All other bytes of the sequence are either fixed or calculated by the findChecksum() function.

 

 

Dual Core Business

Almost a year after finishing my ESP8266-based Internet Radio project, I still have had no idea why running that sketch on the much more powerful ESP32 produces a heavily stuttering sound, resembling Michael Palin in “A Fish Called Wanda”.  So I wondered if I could solve this problem by dividing the audio streaming process between the ESP32’s two cores: filling the audio buffer on one core, and feeding the VS1053 decoder from that buffer on the other core.

Thanks to its builtin FreeRTOS, pinning a task to a specific core of the ESP32 is easy:

1. Put the code for the task in a function, wrapped in an endless for (;;) { } loop.

2. Create a task handle for the task in the main section of the sketch:

3. Create the task, specifying which function it should execute on which core:

 

(Since there was nothing in my loop() function (yet), I put a delay(1000); command in it. The loop() function runs on core 1, and I didn’t want it to claim too much of that core’s processor time.)

So my Radio sketch will have two independent tasks running on different cores: one that listens to the radio station’s audio stream for filling the circular buffer, and one for feeding the VS1053 decoder with bytes from that buffer. Both processes maintain their own pointer for accessing the circular buffer and the code guarantees that they will never catch up with the other process’ pointer. So there is no need to use a semaphore.

However, my first step into the dual core business produced an error. A ‘watchdog timer’ had become nervous and halted the ESP32. It turned out that including the following line in both functions solved at least that problem (bug?):

 

Now the ESP32 no longer crashed, and it actually produced sound. But alas, it was Michael Palin again…!

I can’t think of any reason for this behaviour. The serial monitor shows that the ESP32 has no problem filling the 30 Kbyte circular buffer, so feeding the VS1053 seems to be the bottleneck. I swapped cores for both tasks, but that didn’t help.

UPDATE: Meanwhile, I had become so desperate about finding a solution that I renamed the project “dESPeRadio“. But then, during a sleepless night, I wondered if the ESP32 might perhaps need a different policy for filling the circular buffer. The chunks it reads from the WiFi client per loop cycle have an upper size limit of 1024 bytes, in order to prevent the VS1053 from running dry during the process. But I didn’t specify a lower size limit.

For the ESP8266 this was OK, but the EP32 is so fast that, after it has rapidly filled the circular buffer, there will never be more than just a couple of free positions in the next loop cycle. All following cycles will therefore be very inefficient, causing so much overhead that the VS1053 actually runs dry in a very regular time pattern – the stuttering.

So I added just one single line for skipping the fill cycle if there’s less than 512 bytes of free buffer space available and uploaded the sketch without much hope, because so many ‘good’ ideas had failed before. What followed was complete silence…, which then turned out to be the pause between two movements of  Brahms’ fourth symphony! Then the music started in clean and, above all, uninterrupted sound!

The radio has been running non-stop now for a couple of hours. It looks like I finally have a solid and stable foundation for further development of my dESPeRadio on ESP32. Although using two cores isn’t necessary, it will be nice way to explore FreeRTOS.

In order to keep the code clean and readable, all future features and controls will be implemented as (optional) includible header files. I’ll make the full code available in my first github repository soon, so if you’re interested, stay tuned.

Plans for additional control options, apart from the already implemented keypad, are:

  • ☑ an embedded web server
  • ☐ the touch screen overlay of my display
  • ☐ IR remote
  • ☐ rotary encoders for volume, bass and treble

Other plans:

  • ☐ add VU meters (I love the analogue ones!)
  • ☐ use a FreeRTOS queue instead of a circular buffer

 

 

 

 

 

PiAware & Automatic Gain Control

Odd fluctuations in my PiAware ADS-B receivers’ coverage graphs made me wonder how AGC in dump1090-fa really worked, because information on the subject is contradictory. A common technique for gaining insight into a process is by studying its anomalies, and the following announcement offered a promising opportunity.

     February 12th, 2019
For one of my receivers, air traffic over Belgium is the main source of ADS-B messages, especially in the 80-160 km range. So I was expecting to collect substantially less position broadcasts during the 24-hour strike. That proved to be the case, but I also noticed something else: for the first time since I permanently mounted the antenna outside, I was finally receiving a considerable amount of long distance (420+ km) positions again.

[Note that the bar graphs have different scales]

I recognized the above behaviour from the time when my antenna was still mobile. It always occurred immediately after moving the antenna from inside to outside the house. The effect would then gradually fade during the next couple of hours.

Conclusion so far is that many strong (relatively nearby) messages will make the AGC mechanism gradually reduce the receiver’s sensitivity. This may be beneficial for the total number of aircraft seen, but it comes at the cost of long distance reception. Once the strike was over, long distance reception dropped again within a couple of hours.

I’ve been experimenting with fixed gain values in order to receive more long distance aircraft, mainly to determine how far my receiver could reach. However, none of the allowed gain values gave results that resembled what I got during the 24-hour strike, or shortly after moving the antenna from inside to outside. Could it be that AGC does more than just adjust the receiver’s gain?

 

 

 

Both Sides Now…

With my FlightAware feeder now permanently mounted outside, I began thinking of installing a second ADS-B/Mode-S feeder at the opposite side of my apartment. Would the extra reception coverage be worth the costs, as the second antenna could only be placed inside, blocked by some nearby obstacles. Earlier tests with a NooElec NESDR SMArt dongle and a simple antenna had been quite disappointing, but after placing the antenna behind a solid wall instead of a window, reception suddenly became interesting enough to make me buy a second RasPi + FlightAware Pro Stick Plus + 66 cm antenna.

Antenna coverage graphs of both feeders

 

First I had the second RasPi run just dump1090-fa (not the complete Piaware suite) and forward its decoded messages to the first feeder. A picture on the FlightAware site illustrates how processes are linked together.

[not shown is port 30004, on which dump1090-fa is listening for incoming ADS-B messages from other feeders]

I used netcat for forwarding messages from one feeder to the other:

 

Forwarding messages from dump1090-fa to the FlightAware feeder boosted its detected aircraft figure, but (inevitably?) at the cost of its MLAT synchronisation/reception. So I ended up with both feeders running Piaware and showing up as two separate ‘sites’ under my flightware.com account. For visualizing detected traffic from both receivers I use Virtual Radar Server’s local webpage (http://127.0.0.1:88/VirtualRadar/desktop.html#) on a Windows laptop.

The image below shows my Virtual Radar, with combined coverage graphs of both receivers’ ADS-B & MLAT reception. The outer black circle is at 400 km from my location, so I’ve been receiving aircraft from well over 600 km distance. Occasionally, particularly when nearby traffic is low, the receivers pick up positions from even further distances (800+ km), but these are too irregular to be represented in VR’s coverage graph.

 

[ I could figure out which of the feeders gets most MLAT aircraft, and then forward that feeder’s ADS-B messages to the other feeder. This would stop that other feeder from receiving MLAT aircraft and make it feed FlightAware with ADS-B aircraft from both feeders. That would substantially raise its site ranking as well as my FA user ranking (ADS-B aircraft from the first feeder would be counted twice). I suspect that FA wouldn’t be pleased, so I’ll stick to operating two strictly separated feeders. ]

LOLIN TFT-2.4 Shield

Here’s a well designed new display from Wemos Electronics. They call it a (D1 Mini) Shield, but instead of going on top of a Wemos D1 Mini (Pro), it lets you plug in the Mini at the back of the 2.4″ TFT touch screen display, leaving all GPIO pins accessible, thanks to double header rows. Very useful!

This table shows the internal pin mapping when used as a shield. For TFT_LED, TFT_RST and TS_CS, this mapping can be changed by closing solder bridges at the back of the display.

The default pin mapping leaves I2C pins D1 and D2, as well as D4, A0, TX and RX available for other purposes. With my Wemos Mini Pro v2 plugged in, this shield has become my first choice for battery powered esp8266 projects that require a display.

 

 

 

But there’s more. All pins are also broken out to a breadboard-friendly header and to a TFT connector. Connecting the display to a D32 Pro (that has the same TFT connector) now requires only one (special) cable! The picture shows the connector’s pinout.

 

The 320×240 display uses the well-proven ili9341 chip, for which I already wrote a lot of sketches, both for esp8266 and esp32 boards. The XPT2046 library can be used to drive the touch screen.

I tested several sketches on a plugged-in Mini Pro, as well as a TFT cable-connected D32 Pro, and everything worked fine. Example sketches on the Wemos site use Adafruit’s ili9341 library, but I prefer the much faster TFT_eSPI library.

This is a well-thought product that makes wiring very convenient and stable. Recommended!

 

 

ADS-B experiences

After two months of experimenting with a generic SDR dongle, a dedicated ADS-B dongle and a couple of antennas, here’s my experience so far.

 

Used equipment:

  • FlightAware Pro Stick Plus
  • NooElec NESDR SMArt
  • FlightAware antenna (66 cm)
  • Magnetic ground plane antenna
  • Telescopic FM antenna
  • Raspberry Pi 3B+ (Piaware image)
 

In order of importance, expressed in an (approximated) percentage of influence on my ADS-B reception, this is what helped to improve my FlightAware feeder statistics:

[70%] – Antenna placement. Outside is better than inside (obviously), especially for houses with energy saving windows (metal coated glass reduces signal strength ~40%). Higher is better, in general, but it pays off to experiment:

  • Raising my outside antenne by 80 cm, as well as increasing its view angle by 2 degrees, made it receive more aircraft – but at the cost of most of its >400 km reception…
  • My inside antenna turned out to perform much better after moving it from the window to a wall, where an obstacle outside did no longer block a high air traffic area.
  • Since ADS-B is basically a ‘line of sight’ transmission, it’s clear that obstacles can block signals. But they can also act as rear view mirrors! A large apartment building that blocks my long range reception in one direction compensates for this by bouncing signals from an otherwise completely blocked direction.  

[20%] – Antenna type & quality. The FlightAware antenna isn’t cheap, but its 5 dB gain will extend your reception range compared to simple 1/4 or 1/2 wavelength antennas.

[5%] – Dongle. The FlightAware Pro Stick Plus squeezed a bit more out of all tested antennas. And it didn’t get as hot as the NooElec.

[5%] – Cables & connectors. Use quality cables and keep them as short as possible. Also: periodically check your SMA and/or N-type connections (they can get loose over time).

Tip 1: The amazing Virtual Radar Server (VRS) software is extremely useful for finding the best antenna position. It can create receiver range plots over time, even for different altitude ranges. This allows for a visual comparison of different antenna positions and choosing the one that fits your needs (i.e. best coverage of a particular area, best long range reception, etc.)

Tip 2: Experiment with your receiver’s gain. This is a setting in dump1090 (the demodulator/decoder software component). Default setting in the Piaware image is ‘AGC’ (Automatic Gain Control), that is supposed to dynamically adjust gain in order to avoid ‘clipping’ of the strongest signals. However, some reliable sources claim it will set gain to ‘max’ (49.6 dB) …

If you have ssh enabled on Piaware, here’s the commands to read the current gain setting:

sudo systemctl status dump1090-fa -l

To set gain to, for instance, 40.2 dB:

sudo piaware-config rtlsdr-gain 40.2

Make sure to restart the dump1090 process for your changes to come into effect:

sudo systemctl restart dump1090-fa

Possible gain values are: 0.0, 0.9, 1.4, 2.7, 3.7, 7.7, 8.7, 12.5, 14.4, 15.7, 16.6, 19.7, 20.7, 22.9, 25.4, 28.0, 29.7, 32.8, 33.8, 36.4, 37.2, 38.6, 40.2, 42.1, 43.4, 43.9, 44.5, 48.0 and 49.6

Note: supplied values will be rounded to the closest value in the list. A value of -10 will set the receiver in AGC mode.

 

 

 

Pi in the Sky

It had been waiting on the runway for some time, but now my tinkering with Raspberry Pi  has finally been cleared for takeoff. After getting familiar with my FlightAware ADS-B dongle on Windows (previous post), using a RasPi as a 24/7 FlightAware feeder was a logical choice. No Linux knowledge required: just download the PiAware image, burn it on an SD card, boot the RasPi with it and claim your device with flightaware.com.

The essential program here is dump1090, a clever piece of software for decoding raw Mode_S messages to human-readable flight information. Collected data is made accessible through its built in web server, via TCP, or in json format (default address: http://<local_ip_address_of_piaware>:8080/data/aircraft.json). The program is also highly configurable:

The PiAware package comes with Skyview, a web interface for displaying received flight data from your receiver. It connects to a local web server, so no Internet connection is required. I also discovered that the Virtual Radar program that I use on my Windows PC can be configured to retrieve its data from RasPi’s dump1090 instance:

Note: the above setting will only grab ADS-B aircraft. If your Piaware receiver has MLAT enabled, you can configure a second receiver, listening to port 30105 of the RasPi. Then you can combine both aircraft types in a merged feed (Tools – Options – Merged Feeds) and select ‘Merged Feed’ as receiver in the Virtual Radar web page.

The Virtual Radar web interface is similar to Skyview, but it shows some additional information in the right pane, like aircraft photos. It can use a local aircraft database for filling in missing data like registration and aircraft type, based on the received ICAO code.

Now that I’ve come this far with my close-to-zero Linux knowledge, it’s time for some new challenges:

  • Expand coverage by placing a second ADS-B receiver at the opposite side of my house, merging data from both receivers (= boost my FlightAware ranking) by having an additional instance of dump1090 acting as a hub (–net-only mode).
  • Feed other flight tracking services with the same (merged) data, like flightradar24.com and – the original goal of this whole excercise – adsbexchange.com. There are scripts for that, but I want to make sure that they will not mess up my current setup.
  • Make a local version of my What’s Up sketch, that gets its json data from the RasPi.

Some of this may require some (hopefully not too much) Linux knowledge, and at least SSH access to the RasPi(s), which is said to be easily achieved by putting an empty file named ‘ssh’ in the /root folder on the PiAware SD card.

So far, we’re on schedule. Stay on board and enjoy your flight.

Radar Love

My What’s Up? flight radar sketch for esp8266 relies on adsb-exchange.com for its data. In order to do something in return, I bought this ADS-B receiver and a decent 1090 MHz antenna*.

The idea is to become a feeder for the adsb-exchange.com and flightaware.com servers, sharing transponder data that’s picked up from planes within reach of my receiver.

The quickest way for me to test the device was by plugging it into a Windows 10 tablet. Everything turned out to be simple: within 10 minutes I was looking at “my airplanes”, quietly moving over a map. Even with the antenna placed indoors, the receiver managed to ‘see’ planes at a distance of up to 340 355 390 km! (even> 450 km after opening the window). If my rough calculations are correct, that’s close to the distance where an aitplane at 12 km altitude will disappear behind the antenna’s horizon. Impressive.

This is the (free) software that I used on Windows 10:

  • zadig : this magic tool lets you replace the default Windows driver for the dongle by an alternative driver that supports RTL/SDR functions. This needs to be done just once (per usb port?).
  • dump1090 : extracts and decodes the received ADS-B messages. Keep this program running because it provides the source information for virtual radar (see next).
  • virtual radar : starts a local web server and opens a console window with some statistics + the url of a local web page, displaying all planes within reach of the receiver.

After some experimenting in order to find the best antenna position, my receiver is now part of a Raspberry Pi-based FlightAware ADS-B feeder. Next step will be to feed the  adsb-exchange.com database simultaneously.

Piaware Skyview

*Surprisingly, this € 60 antenna performs only slightly better than a cheap 6.8 cm antenna that came with a Nooelec SRD dongle. Placement of the antenna is clearly more important.

 

Quest for Fire

By no means a gamer myself, I’ve always been interested in the math behind the graphics in video games. A previous post showed how simple clouds can be produced by the Diamond-Square algorithm, running on an esp8266. That algorithm can generate maps and landscapes as well.

Another visual effect on my todo list was algorithmically simulated fire.

 

The video shows my first attempt on a small TFT display. I had to write a relatively simple algorithm, because it needs to run on a microcontroller. Some further experimenting with the color palette, weighting factors and randomization, as well as adding Perlin noise, will hopefully result in a more realistic fire. I also plan to add a rotary encoder for regulating the ‘fire’.