HF Modem - Version 2.0

Matt Roberts - matt-at-kk5jy-dot-net

Published: 2017-07-25

Updated: 2017-07-27


Still QRP, But More Powerful

The first RTTY and CW modem projects were developed for the Arduino platform.  I was surprised to see that the Arduino UNO had enough CPU power to run real-time modulators and demodulators for both RTTY and CW.  With both modems, however, I quickly reached a point where I wanted to add more features, whether they be filters, better tuning indicators, or whatever.  The little 8-bit AVR processors can do one thing well, but when I tried to load them up with more items, I quickly ran out of memory or CPU cycles.  So I decided to upgrade the hardware to a device that has room for more features, but still consumes very low power while operating.  To accomplish this, I chose the Teensy.

The Teensy is actually an entire family of microcontroller boards, just like the Arduino, but the Teensy uses a newer 32-bit ARM as the CPU.  This allows it to exploit higher clock rates, doing more math per cycle, with more memory, peripherals, and so on.  However, the Teensy is still a very low-power board, easily running from a single USB connection.  Further, the Teensy offers an optional audio codec board and an associated signal processing library, which takes all of the hard work out of getting a quality, high resolution audio signal from a radio.  This turns out to be a great combination for a dedicated modem for amateur radio.  Even better is that the Teensy is available at a price point that is very competitive with the Arduino boards.  A 96MHz 32-bit Teensy can be had for less than $20, which is cheaper than a 16MHz 8-bit Arduino Uno.

I have now rebuilt both the RTTY and CW modems on Teensy-based hardware, and tested both of them in major contests.  A third project has been built to provide a cross-platform user interface that can drive the modems for either (or any) mode.  The software was used for the first time on a Raspberry Pi during Field Day 2017 and has been used subsequently for both CW and RTTY contests.


The Hardware

Modem Prototype Hardware
Figure 1: Prototype Hardware
for CW and RTTY
The hardware for the 2.0 modems is fairly simple.  There are four small boards in the kit:
The protype is shown in Figure 1.

The audio is read using the line in connection on the Teensy audio board.  The firmware can be configured for several different maximum audio levels, including typical line-level peak-to-peak voltages that are common with ham radio transceivers.

Since most modern radios have an FSK keying input, I chose to use the same hardware for both version 2.0 modems.  The PTT function is the same for both modes, but the keying line is used for CW straight-key control, or for RTTY FSK shifting, depending on the firmware used.  The only difference is the cable used to connect to the radio.  The audio codec could also support AFSK operation, and I may add that feature to it at a later time.  I prefer the hardware-keyed FSK and CW, so this arrangement is sufficient for now.

The firmware for CW and RTTY are separate for now, so changing modes requires changing firmware.  I bought two Teensy boards and two audio boards, so I'm covered for CW and RTTY. ;)  A future firmware release should be able to easily merge them into a single firmware image that can switch between CW and RTTY on the fly.  Such an arrangement might even have two dedicated keying outputs, one for RTTY and one for CW.


The Software

The modem devices are fully functional with nothing more than a simple serial terminal program to drive them.  This means that they can be operated from just about any modern device that has a screen, including Android, Rasberry Pi, or any tablet, notebook, or desktop computer.

SMP Application
Figure 2: SMP Application
In order to use the modems in a contesting environment, however, a more complete user interface is needed.  In order to support this, I did a side-project called SMP, that uses the firmware's common modem protocol for monitoring and changing settings.

The SMP software is shown in Figure 2, connected to the RTTY modem while it was decoding a recording of a CQ call with noise added.  The software was written in C# using GTK+ as the UI toolkit, so the application can run on any Windows or Linux computer, including the Raspberry Pi and other SBCs.

The application supports basic logging, and a simple, efficient set of keyboard and mouse actions that can allow the contest operator to operate the radio and the log with a minimum of effort.  The result is a contesting interface that gives you to do everything needed to run or to S&P without ever removing your hands from the keyboard.  Intuitive mouse control is also provided, so that the most natural action preferred by the operator can be used for any given contesting task, whether that be copying received text into log fields, editing log fields, running and editing macros, or moving between the QSO and the log functions.

For example, there are shortcuts for all of the buttons on the screen.  So operating a macro button, adding a log entry, or toggling the break-in state can be done from the keyboard regardless of where the cursor is located.  Similarly, there is a shortcut for each of the text fields, including the QSO window, regardless of where the cursor is currently located.  There are shortcuts for selecting any of the settings at the bottom of the screen.  Pressing TAB cycles between the QSO window, the CALL box, and the RX exchange box.  When running in a contest, these are the fields that are accessed in succession.  You enter the call, and the exchange, and you often need to send something to the station, such as KA5? or AGN? or other such things.  No matter where you are in the workflow, you can get to where you need to be next in two TAB presses or less.  If you select text in the QSO window, either with the keyboard or the mouse, you can use keyboard shortcuts to place that text into any of the log text fields.

Macro buttons are placed at the right-hand side of the screen.  This is for several reasons.  First, most screens these days, including on embedded devices, are wide.  Placing the buttons vertically along the side, rather than horizontally, makes better use of the available space on the screen.  Next, traditional macro buttons appear horizontally to correspond to function keys on the PC keyboard.  However, this also places the button text horizontally, resulting in two different sets of boundaries between macros.  To scan the buttons with your eyes, you have to scan across the entire button text to get to the next button, and the amount of text in each button is different, which makes horizontal visual scanning awkward and slow.  Aligning the buttons into a column, where the text is all left-aligned, allows you to scan vertically for key text in a button, without having to scan across the text of all the buttons above it.  This is a subtle difference in workflow, but every little bit helps.

What is also important is what is not in the window.  There are no entry boxes for time-on or time-off.  These are computed automatically as you interact with the other fields.  There is no noisy waterfall ticking away on the screen.  In fact, the only tuning indicators are on the modem.  So when you want to center a station, you turn the radio knob and watch the modem indicators.  When the station is tuned, you go back to the keyboard.  This provides clean separation between functions, and it also eliminates the delay experienced by all waterfall displays due to soundcard buffering.  Supporting information such as operating frequency and log count are placed out of the way and in lighter font weights than the main fields.  The entire focus of the user interface is on the few tasks and fields that are used repeatedly as a contest progresses.

SMP Application
Figure 3: JayLog ADIF Editor
There is only one field for received exchange, and one for sent exchange.  It doesn't make sense to spend time during a contest run trying to per-parse the various fields of an exchange.  The needed information can be captured in a single selection operation, and parsed later by appropriate ADIF software.  The log generated by SMP is ADIF, and all operations are done transactionally, so that the file can be used by SMP while it is being viewed in another application.  A minimal ADIF editing application was also part of this project, as shown in Figure 3.  Any ADIF editor can be used as long as it doesn't hold the active ADIF file open.

Both modems run in break-in mode.  When text is sent to the modem, it automatically asserts the PTT line and sends data.  When all data has been sent, and a short timeout occurs, the modem places the radio back into receive.  The SMP application leverages this by using a single QSO window for both transmit and receive.  There is no explicit command to place the radio into transmit.  When you type characters, they are automatically transmitted.  When you stop typing, you are listening to the other station.  Macros are essentially automated typing -- when you click a macro button, the macro variables, if any, are expanded, and the resulting text is "pasted" into the QSO window, just as if you had typed it live.  At any point, the ESC button will immediately put the radio back into receive mode.  This allows CW operators to use a keyboard for full- or semi-break-in operation that is every bit as responsive as using a paddle and keyer.  Each modem supports configuration of the PTT lead-in and lead-out times, so the difference between full- and semi-break-in is achieved by setting the PTT timings to one's taste.  Alternatively, one can elect to simply omit the PTT connection altogether.

The result is a simple, tight, unified interface to contesting that can both encode and decode RTTY or CW using dedicated decoder hardware and firmware that runs in real time.  This means that the user interface device need not have any significant amount of CPU power at all, and the modem's DSP performance is independent of the UI used or the hardware on which it runs.  The SMP architectures is deliberately left open and modular, so that a modem might be connected to any number of terminal devices or software packages.

The workflow achieved using this combination of hardware, firmware, and software was the least demanding of any kind of computer-based contest interface that I have ever used, including HRD, FlDigi, or any of the logger/control programs.  Using software that focuses on the few essential contesting tasks, and allowing easy transition between those tasks, minimizes the operator workload.  Minimizing the contest workload results in a faster pace, fewer errors, and less stress on the operator.  The current software feature set is sufficiently complete that I will likely not use any of the monolithic software packages for contesting again.  The experience improvement was that good.


Firmware Details

Both CW and RTTY firmware use keying code that essentially unchanged from the original v1.0 modems.  The PTT and KEY outputs are driven by open-drain FETs.

The Teensy audio shield captures 16-bit stereo audio, at a 44.1kHz sampling rate.  The firmware passes the left channel only through a single IIR band-pass filter (BPF), the width of which is selectable in the firmware.  In order to prevent ringing in the filter, a width of 500Hz was chosen by default.  This filter provides three different functions, all of which are important to optimal decoding of signals from the radio:
The output of the BPF is limited (clipped) so that all amplitude information is removed.  Users of FM are probably familiar of the concept of limiting used in FM analog radios.  This step is essentially the same in the modem firmware, and done for the same reasons.

This is where the two modems differ in their demodulation strategy:

The CW modem uses a PLL algorithm, which is similar to the LM567 tone decoder chip, but optimized for discrete-math operation in software.  The PLL settings have reasonable defaults, but can be adjusted by the user through the SMP interface, and the settings persisted in EEPROM.  The PLL uses boxcar integrators instead of the RC-style LPF used by the 567 and similar parts.  These are used to smoothe the I and Q detector outputs which drive the tone detection.  The I detector is used to drive the output state, while the Q detector is used to control the software VCO.

The blue LED is illuminated whenever there is CW tone present, allowing visual indication of the tone detection action.

The PLL Q-phase error level, similar to a loop control voltage in a hardware PLL, is separately integrated with an LPF, and then used to drive the tuning indicator.  The three tuning LEDs are lit to indicate the frequency of the received signal, with respect to the center frequency of the VCO.  If the signal is on-frequency, only the green LED is illuminated.  If the signal is slightly off-frequency, the green LED and one yellow LED will be illumuniated together.  If the signal is significantly off-frequency, only the yellow LED will illuminate.  One yellow LED is provided for each direction of possible frequency error.  This provides a quick visual tuning aid without requiring an elaborate waterfall or other such device.  Although the PLL jitter causes a small amount of display error at very low SNR, the indicator was reasonably accurate, and more than enough to ensure accurate decoding in a contest environment.  Even at low SNR, using the tuning indicator to zero beat another station will get you much closer to their frequency than most people bother to tune by ear.

The PLL approach allowed accurate detection of signals down to around -20dB SNR in wide-band AGWN.  This made the algorithm competitive with many of the popular software decoders in use, and was more than enough to serve as an extra set of ears as I worked CW stations.

In order to remain responsive, and with reasonable frequency accuracy, the PLL requires a sampling rate of at least 8kHz.  The default configuration in the firmware runs with 11.025kHz.

The RTTY modem uses a frequency discriminator that is driven by two very narrow resonant filters, each centered on one of the two expected FSK frequencies.  The output of each filter is individually rectified and run through individual low-pass filters, to measure the amount of signal within each of the filters.  These two levels are then continually compared with each other, to determine the net frequency deviation of an FSK signal that is reasonably zero-beat within the filters.  The frequency deviation is obtained by approximating the arctangent of the two signal levels as if they were X and Y coordinates on a unit circle.  Passing the resulting "angle" through an LPF and a simple hysteresis comparator provides the restored Baudot bit stream.

This approach is a bit brute-force, but when I compared it to the SNR performance of discriminators based on various forms of quadrature detector, the two-filter approach was superior by about 10dB, providing solid copy well below the -20dB SNR point in wide-band AGWN.  It did require a bit more adjustment to get the filter settings optimal, but in the end, it was able to copy truly weak RTTY signals in contest conditions.  I suspect the reason for its performance is the use of resonant filters, rather than band-pass filters, for each of the tone frequencies.  While no filter has zero width, the peaks of each of these filters has essentially zero width, making them ideal for frequency discrimination of individual tones.

The downside to using highly selective filters for detecting individual tones is that it becomes difficult to use any of the resulting information to drive a visual tuning indicator.  In essence, the signal is either aligned properly with the filters or it is not.  The amount of "offset" information available when the signal is off-frequency is quite limited.  What I was able to do was to use the error angle to drive the LED trigraph directly, so that the visual indication is that of the RTTY "diddle" on the yellow LEDs.  If the signal was within +/- 20Hz of zero beat, the yellow LEDs dance with the incoming tones.  If the signal drifts much beyond this, one LED will stop blinking, and with any further deviation, the other LED will stop blinnking.  The LED trigraph is essentially reflecting the hysteresis output, providing more of an alighment indicator than a large-offset tuning indicator.  As it turns out, this was good enough for contesting purposes, and it is accurate even at very low SNR.

Unlike with CW, FSK doesn't use key-up time as part of the signaling.  With CW, the empty space between key-down pulses is significant, and its length determines whether it is part of a letter, part of a word, or the space between words.  FSK uses 100% constant key-down to generate all of the signaling elements, with the two keying elements being the two different carriers.  This means that the RTTY demodulator can actually detect three distinct states from the received audio.  Since the discriminator output is independent of the signal strength, and depends only on the received frequency, the discriminator is either solidly left of center, right of center, or it is floating near the center.  This third state, where the deviation is near zero for an extended period of time is the output when there is no FSK signal present.  This allows the FSK demodulator to use a software squelch that operates properly even in the presence of fading, whether selective or full-signal.

The RTTY modem exploits this, and provides an optional squelch setting that will mute the received data stream if the discriminator spends a specific amount of time without achieving enough deviation.  The squelch operates with a configurable timeout and deviation limit, so that the details can be adjusted by the operator.  Since a single bit of sufficient deviation is enough to open the squelch, the de-squelch operation has essentially negative operating time, un-muting the received data stream before the initial byte from another station has completed transmission.  This avoids missing the initial characters of a transmission that arrive when the squelch is already closed.  The default settings allow the squelch to operate properly even with signals whose SNR is at the lower limit of the discriminator.

The handling of other tasks in the modems, such as T/R timing, command processing, etc., is quite similar between the modems.  Each firmware package has many options that can be configured, including center frequency, decimation divisor (which sets the sampling rate), the preselector filter width, and so on.  Many settings can be adjusted by the SMP application as the modem runs, and the new settings are committed to EEPROM within the modem.  This allows for field calibration of those settings.


Software Downloads

The firmware and SMP software will be made available in source form, although I want to do some more clean-up and packaging before I release it openly on the web.  If anyone is interested in tinkering with this setup, send me an email and I will send you a link to the source.

The software is still being fine-tuned, and the next major release I do may be a combined CW/RTTY modem.  The Teensy should have plenty of space and I/O for this, and it would make a more useful device with a single piece of hardware.

Links

Teensy - Open-source microcontroller and embedded development tools.
SparkFun - Supplier for Arduino boards and hardware.
AdaFruit - Another good source for Arduino hardware.

Copyright (C) 2017 by Matt Roberts, All Rights Reserved.