A Software Modem for FT8

Matt Roberts - matt-at-kk5jy-dot-net

Published: 2023-02-23

Updated: 2024-12-22

Have you ever wanted to run FT8 without WSJT-X? Would you like text-only FT8 software, to script or automate your contacts? Are you looking for a lightweight FT8 engine for your embedded, mobile, or remote operation?

That's what this project is all about. It offers a number of different programs for operating FT8 and related modes, including:

ft8modem - the core component, managing the sound card, decoding data frames from the receiver, and modulating the transmiter.
ft8cat - connects the modem to the CAT functions of a transceiver, for T/R switching, controlling frequencies, and recording decoded frames.
ft8report - optionally uploads received spots to the PSKreporter website.

The central program is a command-line software modem for Linux and similar operating systems. The input and output are newline-delimited lines of text, which will be familiar to Linux users. This turns FT8 into a pipeline utility, that can be scripted, embedded, and managed in any way desired, just like other text utilities.

The Building Blocks

The WSJT-X software package includes a few command-line tools of its own. The jt9 utility is the decoder for all of the supported modes. It can decode messages from WAV files or from shared memory. There are also utilities for encoding messages, such as ft8code and ft4code. Fortunately, that this is enough to build a working modem, by adding a small amount of management code.

The jt9 program isn't just a command-line decoding utility — it is the decoder used by WSJT-X. Using that program for decoding gives all the benefits of the latest decoder from the WSJT-X team, with the same performance as the full application.

The encoders are a different story. WSJT-X doesn't include a command-line modulator, only encoders. These convert text into the symbols corresponding to MFSK audio tones. E.g., for FT8, the ft8code utility maps a CQ message to MFSK tones like this:

CQ KK5JY EM16 --> 3140652 00000000111247246550507003371 3140652 16252542560162105621757762062 3140652

The resulting sequence of digits are the tones to be sent, one per bit. In FT8's 8-tone MFSK, the lowest tone is '0', while the highest tone is '7', sent at a rate of 6.25 bps. The modem uses the encoder to get the symbol sequence, but then custom code generates MFSK audio from them while transmitting.

Operation Description

The modem feeds received audio to the jt9 utility, one time slot at a time. The returned decodes from jt9 are cleaned up, then written to standard output. Transmitted frames are handled similarly, by writing the message information to the modem's standard input. The text is passed through the mode-specific command-line encoder, which generates the keying symbols. Those symbols are passed to the modulator, which generates the MFSK tones through the sound device.

The custom modulator in the ft8modem uses smoothed symbol keying, similar to GFSK, to reduce keying noise in the transmitted audio. The digital filter used to shape the keying waveform is hand-optimized to produce the best reported SNR in the stock jt9 decoder, across the widest range of noise levels the decoder can handle. In bench-testing, the ft8modem modulator produced a signal that, when measured by jt9 itself, reports 24dB better SNR than normal FSK, and 12dB better than the audio produced by the WSJT-X modulator.

The Interface

The ft8modem interface is simple — received decodes are read from the program's output, and transmitted frames are sent to the program's input. This is also how commands are sent to the modem, and status messages sent back. Received decode lines are prefixed with D:, while transmitted messages are reported with an E: prefix.

Transmitting is done by sending an audio frequency value, a space, and then the message itself, to standard input. The modem will pick an appropriate start time to begin transmission, depending on the mode. Other commands are similar, usually sending a single keyword, a space, and a value.

The ft8modem program takes two main arguments. These are the mode and the sound device number. The sound device must support a reasonable sampling rate, preferably 48kHz, which is decimated to 12kHz for the jt9 utility. Run the ft8modem program with no arguments to see a list of available options and sound devices.

For example, when running the modem by hand from the command line, a typical QSO might look like this, with commands shown in bold:

$ ft8modem ft8 133         (this starts the modem in FT8 mode on sound card 133)
...
D: 1724348415  -8  0.2 1870 ~  SP9EIA W6TK RR73      (these are decodes from the radio)
D: 1724348415 -12  0.2  677 ~  CT2GRV AB4JG R-05
D: 1724348415 -11  0.3  769 ~  IK1FJH CT3MD IM13
D: 1724348415 -15  0.2  750 ~  CQ KO1H FN67
D: 1724348415   0  0.5  452 ~  NZ5U WA6TMJ R-03
D: 1724348415 -18  0.2 2262 ~  CQ ZS1ABC KG43
D: 1724348415 -10  0.3  499 ~  IQ0AAI KA9FOX EN43
...
2262 ZS1ABC KK5JY EM16                               (this is a command to transmit at 2262 Hz)
E: 1724348430   0  0.0 2262 ~  ZS1ABC KK5JY EM16     (this is a transmit spot from the modem)
...
D: 1724348445 -12  0.1 2262 ~  KK5JY ZS1ABC -05      (more received decodes)
...
2262 ZS1ABC KK5JY R+01                               (another transmission)
E: 1724348400   0  0.0 2262 ~  ZS1ABC KK5JY R+01
...
D: 1724348415 -12  0.1 2262 ~  KK5JY ZS1ABC RR73     (another response)
...
2262 ZS1ABC KK5JY 73                                 (another transmission)
E: 1724348430   0  0.0 2262 ~  ZS1ABC KK5JY 73
...
D: 1724348445 -12  0.2  870 ~  DO1SGH XQ3SK -16      (and so on)
D: 1724348445 -13  0.2 2576 ~  V51CO SQ2OMK JO93
D: 1724348445 -21  0.2 1656 ~  V51CO DO9MO JN58
D: 1724348445   1  0.3 2496 ~  CQ DX KQ4EKW EL95
D: 1724348445  -2  0.4  641 ~  SP9RG KP2B R-17
...

There will be plenty of other status and diagnostic information mixed in, replaced by the ... in the example above, but this shows the QSO interactions themselves.

The decode (D:) lines show messages received from the radio. Each decode starts wth a number, which is the Unix time of the start of the time slot. The other columns should be familiar to FT8 users, including the SNR estimate, the time offset, and the audio frequency value. The encode (E:) lines show transmitted frames sent to the transmitter. These are included so that the caller can see when each transmission actually starts.

This line-by-line interaction makes FT8 and FT4 entirely scriptable. A script can run the ft8modem similar to the example above, and work stations by sending simple commands that transmit messages. This enables easy integration or automation, even when using very simple programs.

Similarly, a custom user interface need not concern itself with sound card code or radio control, just sending and receiving one-line messages. This is the advantage of pipeline designs.

Where is the radio control?

There are obviously some features not shown above — things like auto-sequence and QSO management. Since this is a pipeline tool, those are features to be implemented by other tools. That's the idea of a pipeline — you connect different utilities together to make them work the way you want them to work.

For example, to add radio (CAT) control and split operation, there is a program included called ft8cat. This is another pipeline tool that runs the modem, then connects to a local instance of the rigctld from the hamlib project, which connects to the radio to control it.

A sample command line, combining ft8modem and ft8cat might be:

   ./ft8cat -s ./ft8modem ft8 133

...which sets the mode to FT8, and begins decoding on sound device #133. To see a list of available sound devices, run ft8modem with no arguments.

The -s option tells ft8cat to enable split operation during transmission. This allows the modulator to generate tones near the higher end of the transmitter passband, even when working a station low in the receiver passband. The VFOB is carefully set to make the transmission happen on the radio frequency intended. This allows the filtering in the radio to eliminate distortion generated by overdriving the audio stage.

The ft8cat utility adds some status output messages, including FA: and FB:, that can be used in higher-level scripts to know the current dial frequencies. This is important for logging and remote monitoring.

The ft8cat script accepts commands, too. For example, it can change the BAND or the MODE to something reasonable, e.g.,

BAND 20m     (this changes the radio to 20m, and sets the VFO(s) appropriately)
MODE FT4     (this changes the modem to FT4, and sets the VFO(s) appropriately)

There is another example script in the installation, called ft8qso. This program does basic QSO management. By using the QSO command documented in the script, the computer will implement auto-sequencing to work another station without further user interaction.

The ft8modem itself also supports a couple of runtime commands, for setting the transmit LEVEL and the decoder DEPTH, e.g.,

LEVEL -20    (this sets the transmit level to -20dB, roughly 10% volume)
DEPTH 2      (this sets the decoder to medium search depth)

See the -d option from jt9(1) for more information about decoding depth. Both of these settings are saved to disk when the program exits, and restores their values the next time the program is run. This is also true of the sound device string passed to the program.

Since the ft8cat stacks on top of ft8modem, it passes any unused commands to the latter. So you can freely send commands meant for either utility, and they will route each command to the proper program.

Typical Applications

Typical Usage

Multi-SDR Example

There are utilities in the package for adding more features. For example, the ft8report program uploads spots to the very popular PSKreporter website. There are also programs for building multi-SDR receiver sites.

At right is a diagram of how the various pieces might fit together for a typical station. This example uses four utilities:

The ft8modem for interfacing with the sound hardware, providing the MFSK encode and decode functions.
The rigctld from hamlib, to communicate with the CAT control hardware of the radio.
The ft8cat which coordinates between all of the components, and provides the ALL.TXT log of spots received.
The ft8report for reporting spots from ALL.TXT to the PSKreporter website.

This example provides the most common functions that most users expect from WSJT-X, but in a command-line format that can be run completely from a script or other program. The script can do band changes, mode changes, send and receive messages, set the radio drive level, monitor the received audio level, etc., all using simple text command-response messages.

A second example is shown at right, for using two RTL-SDR receivers to update the PSKreporter website continuously, as well as record all the spots received. In this example:

Each ft8sdr runs one rtl_fm instance, to receive a specific frequency, and feed the virtual audio to an ft8modem via UDP.
Each ft8cat controls an ft8modem as in the previous example, but connects to the CAT port of the ft8sdr to read and control the frequency.
Each ft8cat also sends received spots to a single ft8collect via UDP, which writes a single, combined ALL.TXT log file.
A single ft8report reads the combined log, and updates the PSKreporter website.

The SDR example could easily scale to any number of receivers. I use four receiver dongles on a single PC, and it could easily host more. I also periodically change the frequency of each receiver, by sending a frequency command to each rtlsdr CAT port, which lets me scan through many more bands than I have receivers.

The modem and its support utilities form the DSP core of a complete FT8 station. This allows you to write custom FT8 software without having to do any DSP or sound card development. How it is used is up to you.

Current Status

The ft8modem is used daily with my automation engine, where it has made several thousand contacts between FT8 and FT4, with good signal reports from the modulator. I also use the modem with a handful of RTL-SDR v3 receivers to monitor multiple bands 24/7, and upload the results to a PSKreporter map for my call. These are the two example configurations shown above.

A README.txt document is included that shows the basics of using the various programs, along with build instructions.

The code should build without issue on Linux, including recent Raspberry Pi images. Building requires Gary Scavone's RtAudio library, version 6.0.1 or later, to be installed first. The core applications are written in C++, which is intended to keep them light and fast. The higher level utilities are written in Python, because it is universal in Linux distributions, and makes it easy to quickly write control logic.

The easiest way to get the needed command-line utilities from the WSJT-X package is to install WSJT-X on your system. For those who prefer not to build or install the full WSJT-X package just to use its command-line utilities, there is also a download called wsjtx-utils. This contains builds of the key utilities for several different Linux architectures, including x86 and three different ARM builds for RasPi. At a minimum, a working installation will need the jt9, ft8code, and ft4code utilities somewhere in the system path, most likely /usr/local/bin.

There are available projects for patching jt9 in a way that supports persistent hashing between each run of that program. If you install a patched version of jt9 on your system, see the README.txt for notes on how to pass hashing arguments. Thanks to Mike, KC5N, for the link.

Support for JT9, JT65, and WSPR has been added and bench-tested between PCs. Activity in those modes is increasingly rare, so it may be some time before I can test them on-air.

ft8modem Downloads

Click the link above to download software packages. The source is being released under the GPL version 3, which is also available on the download page.

Release History

2024-12-22 - Fix small bug in Makefile's install target; other minor tweaks.
2024-11-05 - More small feature additions and bug fixes.
2024-10-28 - More performance improvements, small feature additions and bug fixes.
2024-10-22 - Small performance and doco improvements.
2024-10-21 - A few more small improvements, add basic documentation for new utilities.
2024-10-14 - Fix small timing bug in ft8modem. Added the ft8collect and ft8sdr utilities for building multi-SDR monitor sites.
2024-10-07 - More small improvements; add new ft8report utility for PSKreporter support.
2024-10-02 - A few more minor improvements and bug fixes; add support for WSPR.
2024-09-29 - Minor changes to build environment.
2024-09-28 - Add support for JT9 and JT65, more MFSK modulator improvements, other fixes and improvements.
2024-09-25 - A few more customization options; retune the MFSK LPF to provide best decodes at the receiver across wide SNR range.
2024-09-13 - Several small improvements, mostly new options to allow customization.
2024-09-10 - Add option to ft8cat to compensate for reduced VFO resolution when using split mode.
2024-09-07 - Several small improvements, mostly to the ft8cat utility.
2024-09-03 - Add command-line options to ft8modem to allow customization; more testing and optimization.
2024-08-30 - Improve ft8cat to better track transactions, and respond to unusual cases.
2024-08-29 - More on-air testing with FT4; minor improvements and bug fixes.
2024-08-27 - Improvements to ft8cat and ft8modem; implement transmit spots.
2024-08-26 - ft8cat now changes frequency automatically when the mode is changed; more bug fixes.
2024-08-25 - Update BAND and MODE commands in ft8cat, update the README doc, fix some minor bugs.
2024-08-24 - Improve input decimator, add input level tracking, finish BAND command in ft8cat.
2024-08-23 - Improve decode latency, use default API in RtAudio.
2024-08-22 - Fix even more bugs; switch to RtAudio 6.0.1 and half-duplex audio; more on-air testing.
2024-08-20 - Fix a lot more bugs, and do some on-air testing.
2023-03-09 - Fix more bugs.
2023-03-07 - Fix bugs.
2023-02-25 - Added the ft8cat utility.
2023-02-23 - Initial beta release.

Copyright (C) 2023-2024 by Matt Roberts, KK5JY. All Rights Reserved.