Reversing the Scout ADS-B receiver (part 1)

TL;DR: the Scout ADS-B receiver transmits its data using a proprietary, undocumented format that I cannot decode, but perhaps with scrutiny it could be decoded and used for fun projects. Come for the story of my process; stay for the crappy responses from ForeFlight and uAvionix tech support.  This article is part 1, wouldn’t it be nice if there were a part 2…?

The uAvionix Scout is a dual-band ADS-B receiver designed for use in private aviation. For those not familiar, ADS-B is a technology used to communicate traffic and inflight weather information to pilots; a typical use-case is to enhance the pilot’s situational awareness by displaying this info on the pilot’s iPad which is running an “electronic flight bag” app like ForeFlight. (ADS-B data itself is free to use, is subscriptionless, and is transmitted from FAA stations, all of which is paid for with your tax dollar.)

Although the Scout device has been discontinued you can find them on eBay for less than $200 US. It’s a compact piece of gear, about the size of a pack of gum, and is powered by 5V micro-USB.

ForeFlight summarizes its feature set nicely on https://foreflight.com/support/scout/ :

Scout provides dual-band ADS-B In capabilities. With antennas for both the 978 MHz and 1090 MHz frequency bands, Scout enables the display of inflight weather and traffic on ForeFlight Mobile. Scout supports the full range of FIS-B weather and data, including animated regional and CONUS NEXRAD, METARs, TAFs, AIR/SIGMETs, PIREPs, winds and temps aloft, TFRs, NOTAMs, and SUA information. Scout can also receive air-to-air traffic information from ADS-B Out equipped aircraft and re-broadcast traffic information (ADS-R and TIS-B) from FAA ground towers. Keep in mind that if your aircraft is not equipped with ADS-B Out, then you will have a limited view of traffic on ForeFlight.

That’s a wealth of data that would be wonderful to read and display.

I acquired a Scout some time ago and used it happily prior to upgrading to a more featureful Stratux unit. Afterward the Scout languished in a drawer until I wondered if it would be possible to use the Scout data on the ground, outside the cockpit.¹

The typical way a ForeFlight user utilizes the Scout (or the Stratux) is over Wi-Fi: the Scout maintains a low-power open network which the iPad joins. Upon powering the unit the Scout advertises an unencrypted Wi-Fi access point with ESSID “Scout-A702” in my case – and of course any device can join, not just iPads.

I attached to the Wi-Fi with my computer and fired off nmap scans of TCP and UDP to see what ports might be available. The Scout was at 192.168.4.1, and it provided my computer the IP address 192.168.4.100 via DHCP.

$ nmap -sT 192.168.4.1
Starting Nmap 7.92 ( https://nmap.org ) at ___
Nmap scan report for 192.168.4.1
Host is up (0.046s latency).
Not shown: 999 closed tcp ports (conn-refused)
PORT STATE SERVICE
80/tcp open http
Nmap done: 1 IP address (1 host up) scanned in 0.80 seconds

$ sudo nmap -sU 192.168.4.1
Starting Nmap 7.92 ( https://nmap.org ) at ___
Nmap scan report for 192.168.4.1
Host is up (0.019s latency).
Not shown: 999 closed udp ports (port-unreach)
PORT STATE SERVICE
4000/udp open|filtered icq
MAC Address: a:b:c:d:e:f (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 1.85 seconds

I would come back to UDP 4000 in a bit; I wanted to examine that webserver first.

It turns out that there is a very tiny web page there:

Interesting. Those two Update hyperlinks point to /update and /pingUpdate. Those might be worth exploring later. ForeFlight notes, “You can update Scout’s firmware directly through ForeFlight.” Perhaps ForeFlight uses these two HTTP endpoints for this purpose.

Before proceeding I ran feroxbuster (a Rust-based utility similar to gobuster or dirbuster) to try to discover any hidden API endpoints on the unit. I was unable to find any additional endpoints using this technique.

feroxbuster -d 1 --url http://192.168.4.1 -w SecLists/Discovery/Web-Content/DirBuster-2007_directory-list-2.3-small.txt

The next logical step was to let ForeFlight and the Scout talk to each other and observe their network traffic. The observation computer was Linux-based, so it was a trivial matter to put its Wi-Fi adapter in monitor mode, set it to channel 1 (the channel of Scout-A702), and begin monitoring the 802.11 traffic between ForeFlight and the Scout.²

The first thing I noticed was ForeFlight advertising this JSON string as a UDP broadcast on port 63093 every five seconds:³

{"App":"ForeFlight","GDL90":{"port":4000}}

Interesting. ForeFlight was advertising some sort of service or API on UDP port 4000, the same open port that nmap revealed on the Scout. I also noticed ForeFlight regularly broadcasting a curious string:

i-want-to-play-ffm-udp

A little googling brought me to this page: https://www.foreflight.com/connect/spec/ , “GDL 90 Extended Specification”. Being unfamiliar with said spec I did more googling; it turns out that GDL 90 is a Garmin aviation ADS-B datalink transceiver as well as a data interchange specification apparently based on this device. 

The Specification website above said,

ForeFlight offers the industry-standard GDL 90 Data Interface Specification defined below for third-party devices to transmit live inflight data to ForeFlight Mobile. Properly configured devices will be able to display ADS-B weather and traffic, AHRS, device name information, and GPS data in the ForeFlight Mobile app.
…
ForeFlight expects data sent using UDP to port 4000 on the iOS device.

The website went on to describe the ForeFlight broadcast JSON on port 63093 that I noted with Wireshark, as well as the message specification. This was exciting! This might be the information I needed to decipher the messages flowing between ForeFlight and the Scout. On this site was a link to a FAA-hosted Garmin PDF titled “GDL 90 Data Interface Specification”. I saved this for later reference.

I continued to monitor the connection in Wireshark. The UDP traffic flowing from the Scout into ForeFlight port 4000 had evident UDP payloads but of differing lengths: 34 bytes, 35 bytes, and 52 bytes.

I referred to the Garmin PDF mentioned earlier and discovered information about decoding the payload in section 2.2.1:

The basic message structure is as follows (see Figure 1): 1) A Flag Byte character (0x7E). 2) A one-byte Message ID which specifies the type of message being transmitted. 3) The Message Data, which can be of variable lengths. 4) A message Frame Check Sequence (FCS). The FCS is a 16-bit CRC with the least significant byte first. 5) Another Flag Byte character (0x7E). 

Observing the captured payloads above it was clear that these adhered somewhat (but not totally) to the Garmin spec: they began and ended with 0x7e as specified, but the second byte (the identifying Message ID) was a puzzler: in each case it is 0x26 (decimal 38). The Garmin document stipulated a number of Message ID values in section 3, “Message Definitions.” None of these were 0x26 / decimal 38. 

I referred back to ForeFlight’s “Extended Specification” website discovered earlier. It too did not define any message with ID 0x26. (It did, by implication, indicate that ForeFlight was happy to extend the Garmin messaging spec with their own message types. Clearly I was observing undocumented extensions.)

I googled further and stumbled across an IEEE paper titled GDL90fuzz: Fuzzing – GDL-90 Data Interface Specification Within Aviation Software and Avionics Devices — A  Cybersecurity Pentesting Perspective available at https://ieeexplore.ieee.org/document/9709804 . This was an interesting analysis in its own right and I encourage you to read it; there’s some impressive work done therein.

The paper yielded this notable gem that was relevant to my Scout work:

Some vendors have their own interpretation of the protocol outside of the Garmin standard. For example, uAvionix’s SkyEcho2 mainly uses the standard messaging types, but it outputs its ownship message with the message type code 101. On the other hand, ForeFlight’s Sentry extends the protocol and does not communicate with the standard message types. Sentry transmits messages with IDs 37 and 38, which are longer than the standard heartbeat, ownship, and traffic messages and most likely contain multiple message types in a single packet.⁴

The IEEE researchers had seen exactly what I was seeing: message traffic using undocumented / nonstandard IDs.

I also belatedly realized that the 52-byte message above was in fact two concatenated messages in one payload: ID 38 followed by ID 37.

Further research turned over no new data on what these undocumented messages might contain. I decided to try a bit of a Hail Mary: I would ask the vendors if they had any documentation to share. (It seemed like a reasonable request; after all, ForeFlight maintains a nice public website with their “extended specification.”)

I emailed the support addresses of these two firms:

Hi. I’m trying to find documentation on the GDL90 protocol extensions embraced by the Scout and Sentry products — in particular I’m seeking documentation on the nonstandard message IDs 37 (0x25) and 38 (0x26). 

I’ve consulted Foreflight’s page here : https://www.foreflight.com/connect/spec/ but unfortunately it doesn’t mention these two message IDs.

Are you able to forward me some documentation on this, please?

The responses I got back seemed to be wary and dripping with suspicion. ForeFlight sulkily quizzed me on where I had found the nonstandard messages and chided me that their existing public documentation was complete. I replied with my Wireshark data; ForeFlight then shut down the conversation:

Thank you for your request. The messages you are inquiring about are part of the proprietary data sent between ForeFlight Mobile and the Sentry line of products that we co-developed with uAvionix. Unfortunately, we are unable to provide additional documentation regarding these UDP messages.

uAvionix was even less friendly, replying tersely:

Thank you for contacting uAvionix, unfortunately we are not able to provide information on those proprietary messages.

So there you have it. The Scout (and presumably Sentry) communicate with ForeFlight using a private messaging extension to the open GDL-90 spec. It’s a shame that neither firm will disclose the contents or formats of these messages, as it would be delightful to decode them and use them for other fun tasks. I would love insight into the byte layout of messages 0x25 and 0x26.

1. Yes, I understand that receiving ADS-B data on the ground can be challenging; ADS-B works best at altitude when in proximity to an ADS-B ground station. It is still possible to receive broadcasts from in-range aircraft and rebroadcast ground station data. It really just depends on location factors. ↩︎
2. Could I have just associated the computer to the Scout’s open network? Sure. But in my experience promiscuous-mode wifi capture doesn’t always work perfectly; monitor mode is a few extra steps but more reliable. ↩︎
3. Unlike TCP, UDP can broadcast since it is connectionless. ↩︎
4. The IEEE paper also mentions, “When the app is accepting traffic, it sends ‘i-want-to-play-ffm-udp’; and when it goes to sleep it sends ‘i-cannot-play-ffm-udp.’” I observed the former but not the latter. ↩︎

Monte Carlo Analysis of Chutes & Ladders with Python

If you spend time around young children, sooner or later you’ll end up playing the game Chutes and Ladders. This game actually has quite an interesting history, dating back to ancient India. The Milton-Bradley version dates from the 1940s.

The game’s history is probably the most interesting thing about the game, beause it is utterly devoid of strategy or technique. The players’ only interaction with the game is to spin a spinner that provides a random number from 1 through 6; there is no skill aspect, there are no decisions to be made. And certainly, while perfectly enjoyable as a family pastime that brings people together, in and of itself the game offers little intellectual distraction.

This intellectual idle-time led me to think about the nature of the board. I wondered if the game, being entirely stochastic, had any hidden patterns within. After all, it’s reasonable to assume that Milton-Bradley didn’t have access to any sort of automated analysis of their game back in 1943. I wondered things like:

• What’s the average number of spins to win?
• How many chutes and ladders will a user typically hit?
• Are the chutes and ladders balanced?
• Are some squares hit more often than others?

What’s neat is the nature of the game lends itself well to numerical analysis. There’s no human component that requires estimation or approximation. Literally the only action a player takes is to spin a spinner. Why not use Monte Carlo analysis to play bulk trials of the game and look for answers?

Coding this in Python was fairly straightforward. Although not the most performant language, Python’s excellent visualization tools make it the language of choice for this project. Using joblib to parallelize the task meant that I could simulate 500,000 single-player games in 7 seconds. (Why single-player? The game doesn’t change at all by the number of players. There are no exclusions or conflicts or indeed any alterations at all, from a numerical analysis perspective.)

The results are kind of interesting if you’re the sort of person who wonders about these sorts of things.

For half a million games, I saw this:

7.633320899999944 seconds elapsed time
Minimum number of spins to win: 7
Maximum number of spins to win: 446
Arithmetic mean number of ladders hit by player: 3
Arithmetic mean number of chutes hit by player: 4
Arithmetic mean of spins to win: 40.092362
Median of spins to win: 33.0
Standard deviation = 25.522641855046196

There’s more information to be found in the visualizations:

Noting the average number of spins to win the game is 40, the cumulative number of spins to win a game rises in a smooth curve. You can see, for example, that the probability of concluding a game between 50 and 60 spins is about 81-82%, and that most games are won under 40 spins (the median number-of-spins-to-win being 33).

The heatmaps of board positions are particularly interesting to me. There are two heatmaps of note:

1. a heatmap of where the user lands initially after spinning (i.e. before following any chute/ladder)
2. a heatmap of where the user ends up at the conclusion of their turn (i.e. after following any chute/ladder)

The former shows two notable darker/cooler bands across the board: along the bottom, and in the upper third. These cooler bands are areas of infrequent landing. Once chutes/ladders are taken into account, however, the picture becomes much noisier. The squares with ‘0’ values may seem counter-intuitive until you realize those are the heads of chutes or bottoms of ladders — no turn ever ‘ends’ on these squares. This heatmap shows some particular hotspots such as the top of the “big ladder” and the bottom of Square 48’s yellow chute. Also note how it’s very, very common to get stuck at square 99, waiting for an exact spin of ‘1’ to finally win the game.

If you care to try this out locally, the source code is on Github.

On the ground at CppCon 2014

I’ve just returned from the week-long CppCon 2014 in Bellevue, Washington. Here’s what I experienced.

I’ve absorbed a great deal from a variety of C++ developer conferences – CppNow, Going Native, C++ And Beyond – but always virtually, via video and webcast. This was an opportunity to jump into the thick of things and participate in person. With community heavyweights like Herb Sutter and Scott Meyers in attendance I knew the content would be stimulating and informative. (Honestly, the speaker list featured nearly every name in the “C++ royalty” that you could imagine. I smiled to myself seeing Bjarne Stroustrup standing in the registration line like he was just another attendee.) So when the conference’s early-bird admission opened in March, I eagerly sent in my hard-earned dollars and blocked off the week of September eighth on my calendar.

Unit test your private methods for great justice

The continuing dissent and confusion about unit testing of private class methods surprises me.

The access specifier is much like your choice of software license: it exists to limit consumers’ actions, not to limit yours. A method’s access specifier is completely irrelevant to testing, and only describes what you want the consumer to use; any code that takes inputs and produces outputs, private or not, should be tested.

The opponents of private-method testing tend to argue in quasi-religious terms: that private methods are mere hidden implementation details; that users of the class will only care about the public API; that testing of private methods breaks encapsulation. A typical unhelpful “solution”: private methods should be put into a different class and made public there.

To argue against granular testing of private methods is to mean well while being thoroughly unhelpful. The purpose of testing is more than just to guarantee the viability of your public interface – it is also to examine the inner machinery and support routines of your class to ensure that they themselves function correctly for a spectrum of inputs and edge cases. The private implementation will contain non-trivial complexities that are more readily and precisely tested directly than via the public API.

Beautiful logging for Ruby on Rails 4

In a previous post I showed you a simple way to get beautiful, easy-to-read logs in your Rails 3.2 application. Rails 4 changed the game again; for Rails 3.2 or earlier, refer to my earlier post; but for Rails 4 read on…

It’s really easy. Just make a new file in your ‘config/initializers’ directory called something like ‘log_formatting.rb’ and paste into it the following code. Restart your app, and voila: pretty logs again!

UPDATED. Konrad’s comment below was correct. I’ve altered this code to work with both the regular logger and the new tagged logger. Now you can configure your logger as

config.logger = ActiveSupport::Logger.new('your_app.log')

or

config.logger = ActiveSupport::TaggedLogging.new(Logger.new('your_app.log'))

… both will work. Here’s the updated monkey patch:

class ActiveSupport::Logger::SimpleFormatter
  SEVERITY_TO_TAG_MAP     = {'DEBUG'=>'meh', 'INFO'=>'fyi', 'WARN'=>'hmm', 'ERROR'=>'wtf', 'FATAL'=>'omg', 'UNKNOWN'=>'???'}
  SEVERITY_TO_COLOR_MAP   = {'DEBUG'=>'0;37', 'INFO'=>'32', 'WARN'=>'33', 'ERROR'=>'31', 'FATAL'=>'31', 'UNKNOWN'=>'37'}
  USE_HUMOROUS_SEVERITIES = true

  def call(severity, time, progname, msg)
    if USE_HUMOROUS_SEVERITIES
      formatted_severity = sprintf("%-3s",SEVERITY_TO_TAG_MAP[severity])
    else
      formatted_severity = sprintf("%-5s",severity)
    end

    formatted_time = time.strftime("%Y-%m-%d %H:%M:%S.") << time.usec.to_s[0..2].rjust(3)
    color = SEVERITY_TO_COLOR_MAP[severity]

    "\033[0;37m#{formatted_time}\033[0m [\033[#{color}m#{formatted_severity}\033[0m] #{msg.strip} (pid:#{$$})\n"
  end
end

Steering Behaviors for Ruby game dev now on Github, Rubygems

I just released my Steering Behaviors package to Github, and an accompanying Gem to Rubygems.

If you’re building a game, you’ll want your game agents and characters to exhibit realistic motion. A standard way of doing this is with ‘steering behaviors’.

The seminal paper by Craig Reynolds established a core set of steering behaviors that could be utilized for a variety of common movement tasks. These include such behaviors as predictive pursuit, fleeing, arrival, and wandering. This Ruby library can accomplish many/most of those tasks for your Ruby / JRuby game.

The basic behaviors can be layered for more complicated and advanced behaviors, such as flocking and crowd movement.

Embellishments and expansions are planned, but this is working software you can use to drive your own game’s characters. (I’m using it in my own game programming.) The Github repo includes working graphical examples, and you can install the Gem for easier and more direct use in your own game.

Pull requests are enthusiastically encouraged.

 

On Github: my Fuzzy Associative Memory (FAM) package

I’ve just released to Github my working fuzzy logic module, Fuzzy Associative Memory.

A Fuzzy Associative Memory (FAM for short) is a Fuzzy Logic tool for decision making. It uses Fuzzy Sets to establish a set of rules that are linguistic in nature; examples might include:

• “If the room is a bit warm, turn the fan up a little bit”
• “If the orc’s hit points are a little low, retreat from the enemy”
• “If the ship is off course by a little bit, correct just a little to the right”
• “If the bird is much slower than the flock, speed it up a lot”

As you can see, the rules are deliberately vague and use qualifiers like “a little” and “a lot”. This is the nature of fuzzy sets; they capture such human fuzziness in a way that extracts highly natural behavior from the fuzzy rules.

It has a wide range of applications:

• Industrial control, such as governing a fan to keep a room at the “just right” temperature
• Game AI, such as giving human-like behavior capabilities to NPCs
• Prediction systems

Status

This is working, functional software. It currently supports:

• Triangular fuzzy sets for input/output
• Larsen Implication (scaling)
• Atomic antecedent propositions (if A then Z)

To do:

• Trapezoidal (and other shapes) for fuzzy sets
• Hedges (‘very’ and ‘fairly’)
• Mamdani Implication (clipping)
• Composite antecedent propositions (if A or B, then Z)
• Additional examples

Examples

The bin directory contains the following examples:

• hvac_system_example illustrates how a FAM could govern an HVAC fan unit to maintain a constant, comfortable temperature

Get it here!

Entity-Component game programming using JRuby and libGDX – part 8

Introduction

Our Lunar Lander game is somewhat playable by this point but it still lacks some key features. After all, it would be nice if we could detect collisions and determine if the lander has safely landed on the pad. Let’s see how our flexible Entity-Component system permits us to expand our game with minimal fuss.

Collision Detection

First, a frank disclaimer: the following collision detection algorithm is entirely inefficient. It’s kept simple for our basic teaching purposes here but is probably undesirable in a game of any scale. But that’s OK: E-C will permit you to swap in a much more advanced collision detection system when you’re ready. 🙂

Entity-Component game programming using JRuby and libGDX – part 7

Introduction

Entity-Component systems, we’ve learned, are easy to implement and maintain; the elegance is basically “baked in” due to the way components and entities are married in the Entity Manager.

One particularly tidy aspect of an entity-component system is how well it lends itself to data persistence, or in practical terms: saving game state. Let’s take a look.

Where Is State?

In a conventional object oriented design, state is scattered all over the place, embedded in your far-flung object instances. But in E-C everything is neatly gathered together under one roof: the Entity Manager. This manager knows every entity “instance” along with every entity’s components, which are where the entity state data are stored.

Therefore, persist the entity manager to disk and you’ve saved the game in its entirety. Load from disk to memory and you’ve just loaded the game. It really is that easy.

Entity-Component game programming using JRuby and libGDX – part 6

Introduction

Now that we have laid the entity-component foundation and introduced some necessary libGDX concepts, we can finally get around to putting together a little game. Let’s make a “Lunar Lander” type game to illustrate all the concepts we’ve learned so far.

Remember that the source code for this Entity-Component Framework and the game we’re writing is all available at Github. The Github version is, of course, the “final version” that includes features I might not have addressed so far in the blog series, but if you’re eager to jump ahead…

Entities and Components

For this exercise let’s begin by defining our entities and some of their relevant components.

What are we going to need for this game?

• The lunar lander module
• A platform for it to try to land on
• Ground to collide with

That seems like a fair assessment of our initial entity needs. Now, moving on to components. What are some of the aspects / behaviors / features that we should provide to our entities?