A Software Modem for FT8
Matt Roberts - matt-at-kk5jy-dot-net
Published: 2023-02-23
Updated: 2024-11-12
Have you ever wanted to run FT8 without WSJT-X?
Would you like text-only FT8 software, to script or automate
your contacts? Are you looking for a lightweight FT8
engine for your embedded, mobile, or remote operation?
That's what this project is all about. It offers a number of
different programs for operating FT8 and related modes, including:
- ft8modem - the core component, managing the sound card,
decoding data frames from the receiver, and modulating the
transmiter.
- ft8cat - connects the modem to the CAT
functions of a transceiver, for T/R switching, controlling frequencies,
and recording decoded frames.
- ft8report - optionally uploads received spots to the PSKreporter
website.
The central program is a
command-line
software modem
for
Linux
and similar operating systems. The input and output are
newline-delimited
lines of text, which will be familiar to Linux users. This turns
FT8 into a
pipeline utility,
that can be scripted, embedded, and managed in any way desired, just
like other text utilities.
The Building Blocks
The WSJT-X
software package includes a few command-line tools of its own.
The jt9 utility is the decoder for all of the supported modes.
It can decode messages from WAV files or from shared memory. There
are also utilities for encoding messages, such as ft8code and
ft4code. Fortunately, that this is enough to build a working
modem, by adding a small amount of management code.
The jt9 program isn't just a command-line decoding utility —
it is the decoder used by WSJT-X. Using
that program for decoding gives all the benefits of the latest decoder
from the WSJT-X team, with the same performance as the
full application.
The encoders are a different story. WSJT-X doesn't include
a command-line
modulator,
only encoders. These convert text into the symbols corresponding to
MFSK
audio tones. E.g., for FT8, the ft8code utility maps a
CQ message to MFSK tones like this:
CQ KK5JY EM16 --> 3140652 00000000111247246550507003371 3140652 16252542560162105621757762062 3140652
The resulting sequence of digits are the tones to be sent, one
per bit. In FT8's 8-tone MFSK, the lowest tone is '0', while
the highest tone is '7', sent at a rate of 6.25
bps.
The modem uses the encoder to get the symbol sequence, but then
custom code generates MFSK audio from them while transmitting.
Operation Description
The modem feeds received audio to the jt9 utility, one time
slot at a time. The returned decodes from jt9 are
cleaned up, then written to
standard
output. Transmitted frames are handled similarly, by
writing the message information to the modem's standard
input. The text is passed through the mode-specific
command-line encoder, which generates the keying symbols.
Those symbols are passed to the modulator, which generates the
MFSK tones
through the sound device.
The custom modulator in the ft8modem uses smoothed symbol
keying, similar to
GFSK,
to reduce
keying noise
in the transmitted audio. The
digital filter
used to shape the keying waveform is hand-optimized to produce the
best reported
SNR
in the stock jt9 decoder, across the widest range of noise
levels the decoder can handle. In bench-testing, the
ft8modem modulator produced a signal that, when measured
by jt9 itself, reports 24dB better SNR than normal FSK, and
12dB better than the audio produced by the WSJT-X modulator.
The Interface
The ft8modem interface is simple — received decodes are
read from the program's output, and transmitted frames are sent to the
program's input. This is also how commands are sent to the modem,
and status messages sent back. Received decode lines are prefixed
with D:, while transmitted messages are reported with an E:
prefix.
Transmitting is done by sending an audio frequency value, a space,
and then the message itself, to standard input. The modem will
pick an appropriate start time to begin transmission, depending on
the mode. Other commands are similar, usually sending a single
keyword, a space, and a value.
The ft8modem program takes two main arguments. These
are the mode and the sound device number. The sound device
must support a reasonable sampling rate, preferably 48kHz, which
is decimated to 12kHz for the jt9 utility. Run the
ft8modem program with no arguments to see a list of
available options and sound devices.
For example, when running the modem by hand from the command line,
a typical QSO might look like this, with commands shown in bold:
$ ft8modem ft8 133 (this starts the modem in FT8 mode on sound card 133)
...
D: 1724348415 -8 0.2 1870 ~ SP9EIA W6TK RR73 (these are decodes from the radio)
D: 1724348415 -12 0.2 677 ~ CT2GRV AB4JG R-05
D: 1724348415 -11 0.3 769 ~ IK1FJH CT3MD IM13
D: 1724348415 -15 0.2 750 ~ CQ KO1H FN67
D: 1724348415 0 0.5 452 ~ NZ5U WA6TMJ R-03
D: 1724348415 -18 0.2 2262 ~ CQ ZS1ABC KG43
D: 1724348415 -10 0.3 499 ~ IQ0AAI KA9FOX EN43
...
2262 ZS1ABC KK5JY EM16 (this is a command to transmit at 2262 Hz)
E: 1724348430 0 0.0 2262 ~ ZS1ABC KK5JY EM16 (this is a transmit spot from the modem)
...
D: 1724348445 -12 0.1 2262 ~ KK5JY ZS1ABC -05 (more received decodes)
...
2262 ZS1ABC KK5JY R+01 (another transmission)
E: 1724348400 0 0.0 2262 ~ ZS1ABC KK5JY R+01
...
D: 1724348415 -12 0.1 2262 ~ KK5JY ZS1ABC RR73 (another response)
...
2262 ZS1ABC KK5JY 73 (another transmission)
E: 1724348430 0 0.0 2262 ~ ZS1ABC KK5JY 73
...
D: 1724348445 -12 0.2 870 ~ DO1SGH XQ3SK -16 (and so on)
D: 1724348445 -13 0.2 2576 ~ V51CO SQ2OMK JO93
D: 1724348445 -21 0.2 1656 ~ V51CO DO9MO JN58
D: 1724348445 1 0.3 2496 ~ CQ DX KQ4EKW EL95
D: 1724348445 -2 0.4 641 ~ SP9RG KP2B R-17
...
There will be plenty of other status and diagnostic information
mixed in, replaced by the ... in the example above, but this
shows the QSO interactions themselves.
The decode (D:) lines show messages received from the radio.
Each decode starts wth a number, which is the
Unix time
of the start of the time slot. The other columns should be
familiar to FT8 users, including the SNR estimate, the time offset,
and the audio frequency value. The encode (E:) lines show
transmitted frames sent to the transmitter. These are
included so that the caller can see when each transmission actually
starts.
This line-by-line interaction makes FT8 and FT4 entirely scriptable.
A script can run the ft8modem similar to the example above,
and work stations by sending simple commands that transmit messages.
This enables easy integration or automation, even when using very
simple programs.
Similarly, a custom user interface need not concern itself with
sound card code or radio control, just sending and receiving
one-line messages. This is the advantage of pipeline
designs.
Where is the radio control?
There are obviously some features not shown above — things
like auto-sequence and QSO management. Since this is a pipeline
tool, those are features to be implemented by other tools. That's
the idea of a pipeline — you connect different utilities together
to make them work the way you want them to work.
For example, to add radio (CAT) control and split operation, there
is a program included called ft8cat. This is another
pipeline tool that runs the modem, then connects to a local instance
of the rigctld from the
hamlib project,
which connects to the radio to control it.
A sample command line, combining ft8modem and ft8cat
might be:
./ft8cat -s ./ft8modem ft8 133
...which sets the mode to FT8, and begins decoding on sound device
#133. To see a list of available sound devices, run
ft8modem with no arguments.
The -s option tells ft8cat to
enable split operation during transmission. This allows the
modulator to generate tones near the higher end of the transmitter
passband, even when working a station low in the receiver passband.
The VFOB is carefully set to make the transmission happen on the
radio frequency intended. This allows the filtering in the
radio to eliminate distortion generated by overdriving the audio
stage.
The ft8cat utility adds some status output
messages, including FA: and FB:, that can be used
in higher-level scripts to know the current dial frequencies.
This is important for logging and remote monitoring.
The ft8cat script accepts commands, too. For example,
it can change the BAND or the MODE to something
reasonable, e.g.,
BAND 20m (this changes the radio to 20m, and sets the VFO(s) appropriately)
MODE FT4 (this changes the modem to FT4, and sets the VFO(s) appropriately)
There is another example script in the installation, called
ft8qso. This program does basic QSO management.
By using the QSO command documented in the script, the
computer will implement auto-sequencing to work another station
without further user interaction.
The ft8modem itself also supports a couple of runtime
commands, for setting the transmit LEVEL and the decoder
DEPTH, e.g.,
LEVEL -20 (this sets the transmit level to -20dB, roughly 10% volume)
DEPTH 2 (this sets the decoder to medium search depth)
See the -d option from jt9(1) for more information
about decoding depth. Both of these settings are saved to
disk when the program exits, and restores their values the next
time the program is run. This is also true of the sound
device string passed to the program.
Since the ft8cat stacks on top of ft8modem, it passes
any unused commands to the latter. So you can freely send
commands meant for either utility, and they will route each command
to the proper program.
Typical Applications
Typical Usage
Multi-SDR Example
There are utilities in the package for adding more
features. For example, the ft8report program
uploads spots to the very popular PSKreporter
website. There are also programs for building multi-SDR
receiver sites.
At right is a diagram of how the various pieces might fit together
for a typical station. This example uses four utilities:
- The ft8modem for interfacing with the sound hardware,
providing the MFSK encode and decode functions.
- The rigctld from hamlib, to communicate with the CAT
control hardware of the radio.
- The ft8cat which coordinates between all of the
components, and provides the ALL.TXT log of spots received.
- The ft8report for reporting spots from ALL.TXT to
the PSKreporter website.
This example provides the most common functions that most users
expect from WSJT-X, but in a command-line format that
can be run completely from a script or other program. The
script can do band changes, mode changes, send and receive messages,
set the radio drive level, monitor the received audio level, etc.,
all using simple text command-response messages.
A second example is shown at right, for using two RTL-SDR receivers
to update the PSKreporter website continuously, as well as record
all the spots received. In this example:
- Each ft8sdr runs one rtl_fm instance, to
receive a specific frequency, and feed the virtual audio to an
ft8modem via UDP.
- Each ft8cat controls an ft8modem as in the
previous example, but connects to the CAT port of the ft8sdr
to read and control the frequency.
- Each ft8cat also sends received spots to a single
ft8collect via UDP, which writes a single, combined
ALL.TXT log file.
- A single ft8report reads the combined log, and
updates the PSKreporter website.
The SDR example could easily scale to any number of receivers.
I use four receiver dongles on a single PC, and it could easily host
more. I also periodically change the frequency of each receiver,
by sending a frequency command to each rtlsdr CAT port, which
lets me scan through many more bands than I have receivers.
The modem and its support utilities form the DSP core of a complete
FT8 station. This allows you to write custom FT8 software
without having to do any DSP or sound card development. How
it is used is up to you.
Current Status
The ft8modem is used daily with my
automation engine,
where it has made several thousand contacts between FT8 and FT4,
with good signal reports from the modulator. I also use the
modem with a handful of
RTL-SDR v3 receivers
to monitor multiple bands 24/7, and upload the results to a
PSKreporter map for my call.
These are the two example configurations shown above.
A README.txt document is included that shows the basics of
using the various programs, along with build instructions.
The code should build without issue on Linux, including recent
Raspberry Pi images. Building requires Gary Scavone's
RtAudio
library, version 6.0.1 or later, to be installed first.
The core applications are written in C++, which is intended to keep
them light and fast. The higher level utilities are written
in Python, because it is
universal in Linux distributions, and makes it easy to quickly
write control logic.
The easiest way to get the needed command-line utilities from the
WSJT-X package is to install WSJT-X on your system.
For those who prefer not to build or install the full WSJT-X
package just to use its command-line utilities, there is also a
download called wsjtx-utils. This contains builds of
the key utilities for several different Linux architectures, including
x86 and three
different ARM builds for
RasPi.
At a minimum, a working installation will need the jt9, ft8code,
and ft4code utilities somewhere in the system path, most
likely /usr/local/bin.
There are
available projects
for patching jt9 in a way that supports persistent hashing
between each run of that program. If you install a patched
version of jt9 on your system, see the README.txt for notes
on how to pass hashing arguments. Thanks to Mike, KC5N, for
the link.
Support for JT9, JT65, and WSPR has been added and bench-tested
between PCs. Activity in those modes is increasingly rare,
so it may be some time before I can test them on-air.
ft8modem Downloads
Click the link above to download software packages. The source is being
released under the GPL version 3, which is also available on the download page.
Release History
2024-11-05 - More small feature additions and bug fixes.
2024-10-28 - More performance improvements, small feature additions and bug fixes.
2024-10-22 - Small performance and doco improvements.
2024-10-21 - A few more small improvements, add basic documentation for new utilities.
2024-10-14 - Fix small timing bug in ft8modem. Added the ft8collect and ft8sdr utilities for building multi-SDR monitor sites.
2024-10-07 - More small improvements; add new ft8report utility for PSKreporter support.
2024-10-02 - A few more minor improvements and bug fixes; add support for WSPR.
2024-09-29 - Minor changes to build environment.
2024-09-28 - Add support for JT9 and JT65, more MFSK modulator improvements, other fixes and improvements.
2024-09-25 - A few more customization options; retune the MFSK LPF to provide best decodes at the receiver across wide SNR range.
2024-09-13 - Several small improvements, mostly new options to allow customization.
2024-09-10 - Add option to ft8cat to compensate for reduced VFO resolution when using split mode.
2024-09-07 - Several small improvements, mostly to the ft8cat utility.
2024-09-03 - Add command-line options to ft8modem to allow customization; more testing and optimization.
2024-08-30 - Improve ft8cat to better track transactions, and respond to unusual cases.
2024-08-29 - More on-air testing with FT4; minor improvements and bug fixes.
2024-08-27 - Improvements to ft8cat and ft8modem; implement transmit spots.
2024-08-26 - ft8cat now changes frequency automatically when the mode is changed; more bug fixes.
2024-08-25 - Update BAND and MODE commands in ft8cat, update the README doc, fix some minor bugs.
2024-08-24 - Improve input decimator, add input level tracking, finish BAND command in ft8cat.
2024-08-23 - Improve decode latency, use default API in RtAudio.
2024-08-22 - Fix even more bugs; switch to RtAudio 6.0.1 and half-duplex audio; more on-air testing.
2024-08-20 - Fix a lot more bugs, and do some on-air testing.
2023-03-09 - Fix more bugs.
2023-03-07 - Fix bugs.
2023-02-25 - Added the ft8cat utility.
2023-02-23 - Initial beta release.
Copyright (C) 2023-2024 by Matt Roberts, KK5JY.
All Rights Reserved.
