Stereoscopy on ESP32?

double trouble…?

I knew in advance that trying to make an ESP32 show moving stereoscopic images on a TFT display could be unsuccessful. But sometimes you just want to know how close you can get. Besides, it fits my lifelong interest in stereoscopy*.

I found suitable test material in the form of this animated gif (author unknown). Now I ‘only’ had to find a technique for hiding the left frames for the right eye, and vice versa.

 

The first technique that I tried used two of these Adafruit “LCD Light Valves“:

Source: Adafruit website

Together, they formed active shutter glasses for looking at a single TFT display. Looping over the original frames, the display alternately showed the left and right half of a frame, always closing the opposite eye’s valve. The speed of these shutters surprised me, but their accuracy proved insufficient. In order to make the left frame completely invisible to the right eye and vice versa, I had to build in short delays that led to flickering. Without those delays, there was too much ‘leakage’ from the opposite side, a recipe for headaches.

 

The only solution for the above problem was a physical separation of left and right eye-range (i.e. the classical View·Master© approach). Luckily, I had this 3D Virtual Reality Viewer for smartphones laying around. Instead of a smartphone, it can easily hold an ESP32, a battery and two TFT displays. The technical challenge of this method was to keep both displays in sync.

It’s easy for a single ESP32 to drive two identical TFT displays over SPI. The state of a display’s CS line decides whether it responds to the master or not. With CS for left pulled LOW and CS for right pulled HIGH, only the left display will respond, and vice versa. And when data needs to be sent to both displays, you simply pull both CS lines LOW.

The results of this approach were disappointing. Unfortunately, the time the SPI bus needs for switching between clients proved to be too long, resulting in even more flickering than with the shutter glasses technique.

 

As for now, I can see only one option left: using two ESP32 boards, each driving their own display. This will likely solve the flickering (this post shows that the ESP32 is fast enough), but now I’ll have to keep both ESP32 boards in sync.

Both boards are in the same housing and share a power supply, so the most simple, fast and accurate way to sync them seemed over two cross-wired IO pins. Unfortunately, and for reasons still unknown to me, when I run the sketch below on both boards, they keep waiting for each other forever because they will always read 1 on their inPin, even when the other board writes 0 to its outPin.

 

Now that I’ve come closer to a 3D movie viewer than expected, it would be a shame if this apparently simple synchronization problem would stop me.

To be continued (tips are welcome)

 


 

*The first 3D image that I saw as a kid was a “color anaglyph” map of a Dutch coal mine. It came with cardboard glasses holding a red and a cyan colored lens. The map looked something like this:

 

There were also these picture pairs, forcing your eyes to combine short-distance focus with long-distance alignment. It gave me some headaches until I invented a simple trick: swap left and right and watch them cross-eyed…

 

My aunt once gave me an antique wooden predecessor of the viewmaster, complete with dozens of black & white images of European capitals.

A newer ‘antique’ that I like for its ingenuity is Nintendo’s Virtual Boy with its rotating mirror technique. Instant headache, but so cool!

 

Somewhat eccentric newcomers were the “stereograms”. Fascinating how fast our brain manages to make 3D spaghetti out of their messy images, even if they move!

 

 

 

M5Stack Fire

 

It looked like the perfect hardware for a couple of gadgets that I have in mind, but my enthusiasm for the M5Stack Fire development board quickly cooled down after my first experiments.

Meanwhile, I’ve been able to tackle most software issues, but it’s the hardware flaws that worry me most. My verdict is at the bottom of this post.

 

M5Stack is a clever concept, built around an ESP32 core that can be expanded by stackable (magnetic) modules. Also, special purpose units can be connected via Grove connector cables. A growing number of modules and units are available and it all looks very appealing indeed. But then, this is definitely an expensive way of prototyping.

I chose the Fire variant because it houses an ESP32 WROVER module with 16 MB Flash and 4 MB PSRAM. This is what you get:

  • 320×240 ili9342 display (M5Stack library uses Bodmer’s unrivalled TFT_eSPI)
  • 3 buttons
  • microphone
  • speaker (1 Watt)
  • 10 Neopixels
  • accelerometer MPU6886 (not an MPU9250!)
  • magnetometer BMM150
  • SD card slot (max. 16 GB)
  • 600 MAh battery
  • stackable charger module
  • plastic storage box with USB-C cable, hexagonal wrench and some LEGO…

At first, it looked like I had received a defective device that did nothing more than producing a ticking noise. It turned out that this behaviour occurs when the battery is completely empty, even if the module is powered by USB. Fair enough (once you know it).

The device comes pre-loaded with UIFlow firmware, a graphical programming environment. Apart from the self-started demo, I didn’t test it. Instead, I added the board and its official library to the Arduino IDE and uploaded one of my most hardware demanding sketches (Perrier Loop). Everything worked fine, but all colors were inverted. In sketches that use the original TFT_eSPI library, adding the following lines in the setup() section of your sketch fixes the issue:

 

The module’s speaker is a bit noisy: it cracks on ESP32 resets and hisses while the power/reset button is being pressed. For its price, I would have expected a better hardware design. Besides, I personally would have liked a more ergonomic power button and a more consistent on/off/reset behaviour.

When I tried the official library’s example sketch for reading the IMU sensors, the ESP32 crashed. It turned out that my Fire did not have the MPU9250 sensor that the example had been written for, but a MPU6886 and a BMM150 instead. Wonder why they never updated their 2.5 years old example.

After this crash, I was no longer able to upload new sketches to the Fire (A fatal error occurred: Timed out waiting for packet content…). I managed to fix it, but unfortunately this behaviour keeps returning. And there’s more strange behaviour, like on-board neopixels being lit while the running sketch doesn’t even use them,  as well as spontaneous beeps.

Things got worse: the power/reset switch stopt working after I had uploaded the library example WiFiSetting.ino sketch (that worked OK, by the way). The module would no longer switch off, despite many double clicks, executed at different intervals….

Finally, after lots of randomly clicking the button, the device appeared to be off, but could no longer be switched on…. The Serial monitor then showed that the ESP32 was actually still running, but also continuously crashing. This also blocked uploading new sketches completely 🙁

In the end, I had to remove the battery module. After reconnecting it, I managed to erase ESP32’s flash memory. Then I used a tool called M5Burner for uploading the original UIFlow firmware.

The error messages when the device is crashing in a loop seem to suggest that not all ESP32 strapping pins are properly being pulled up or down during boot:

rst:0x10 (RTCWDT_RTC_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun 8 2016 00:22:57


My provisional verdict, based on an ongoing struggle with (my copy of) the M5Stack Fire:

 

Pros:

  • clever concept
  • nice look and mostly decent physical build quality
  • high quality 2″ display, supported by Bodmer’s TFT_eSPI library.

Cons:

  • seriously flawed reset/power circuit!
  • noisy speaker (poor audio isolation; use dacWrite(25, 0); if you don’t need audio)
  • SD card slot not properly aligned
  • The official library’s interrupt-based button functions can crash your WiFi (and worse)
  • Grove port C  appears to be useless by design (hard-wired to PSRAM pins 16 & 17)
  • 5V tolerancy of Grove ports A and B (pins 21, 22, 26 & 36) remains unclear

This nice looking ESP32 toy may need a few electronic design changes to enhance its usefulness and stability. Until then, I’ll keep my distance from the Fire….

 

 

Greed for Speed

One of my first Arduino peripherals was this 3.5″ touchscreen from Adafruit. It has a 480×320 resolution, an SD card reader and two interfaces (SPI and 8-bit parallel).

However, after first connecting it to my Arduino Uno, this absolutely beautiful display turned out to be rather slow, even in parallel mode. Back then, there were no fast libraries for its HX8357D chip, so I soon bought a smaller 320×240 ili9341 display for projects that required fast graphics (fractals, cellular automata etc.). But I never stopped looking for a faster way to drive my 480×320 ‘Ferrari’.

New hope arose when ESP8266 entered the market, especially after ‘Bodmer’ added support for the HX8357D chip to his amazing TFT_eSPI library on Github. With the ESP8266’s clock speed set to 180 MHz, cellular automata now grew considerably faster, but still noticeably slower than on ili9341 displays (with half the amount of pixels, that is).

Then came ESP32. The cpu-hungry algorithms that generate complex graphics proved to run much faster on this dual core board. Finally, some of my math visualisations started looking acceptable on my beloved 480×320 display.

A decisive breakthrough was the arrival of the WROVER variant of ESP32. It came with PSRAM, more than enough for (multi) buffering a 480×320 display (4 MB, or even 8 MB on model B). Writing pixel values to a buffer in PSRAM first, and then push (part of) this buffer to the display in one transaction, is much faster than writing to individual pixels.

A welcome dessert was the speed gain after increasing the SPI frequency from 27 MHz to 40 MHz (a setting in the TFT_eSPI library’s user_setup.h file), which finally made my Adafruit display fully competitive. And, as the proverbial cherry on the cake, controlling it over its 8-bit parallel interface made it even faster again!

This video shows the display at top speed. If you’ve ever seen this standard graphics test on an Arduino Uno, you’ll know that I’ve made some progress here. My smartphone’s camera had some trouble filming the display (it even turned yellow into white…).

Benchmark sketch on an ESP32, driving the 480×320 display over its 8-bit parallel interface

Guess this is where my Speed Quest has its happy ending. Below are the benchmark results (in microseconds) for the 320×240 ili9341 and the 480×320 HX8357D displays, both driven by an ESP32 (LOLIN D32 Pro). Output is from the library’s graphicstest.ino sketch, running on a single core, without the use of display buffering.

 

 

 

Real Virtuality (RV)

Just a thought that crossed my mind: in today’s Virtual Reality, computers are used to simulate reality in a virtual setting, but when the first computers became available in the 1940’s, they were used by mathematicians for exactly the opposite.

For centuries, they had been creating all kinds of virtual worlds inside their minds, supported by little more than paper or a blackboard. Mind games, where everything was allowed, as long as you could prove it from self-postulated axioms. The principle of fractals or the concept behind cellular automata, for instance, had already been developed long before their fascinating complexity could be visualized.

[Kurt Gödel*, wearing 2D glasses]

The arrival of computers offered previously unthinkable possibilities to visualize these virtual worlds ‘for real’ (Real Virtuality…?). By studying the results, scientists developed many new insights and ideas. Nevertheless, true geniuses obviously don’t need VR or RV. Kurt didn’t even use his blackboard…

 

* Kurt Gödel was an Austrian mathematician, most famous for his Incompleteness Theorem.

 

 

 

 

 

 

Larry’s Page

With our famous 5-minute installation, setting up WordPress for the first time is simple.

True, and within those 5 minutes I even managed to change the site’s theme to the clean and unobtrusive Libre theme.

Things weren’t that easy in 1998, when I wanted my CD database to be accessible via the (still young and innocent) Internet. At the same time when Larry Page was hatching his Google egg, I spent my entire Christmas holiday building my first database driven website. What if it had been the other way around..?

BTW, my meanwhile retro-looking CD site is still online (so is Google, I’ve been told).