Debugging the RS232 serial port

 

Debugging Serial Communication FT1000MP Win-Test CW-Skimmer

In this article we will investigate how various Contest programms (Win-Test, N1MM and CW-Skimmer) communicate with the FT1000MP shortwave transceiver over the serial port. We will see what happens when multiple programs try to talk to the transceiver at the same time, what problems appear and how we can successfully monitor and debug them.

Introduction

A couple of weeks ago I modified my Yaesu FT1000MP short wave transceiver in order to be able to use it with an external software defined radio (SDR). For decoding morse code (cw), Alex Shovkoplyas, VE3NEA wrote the neat software “CW-Skimmer” which is able decode up to 700 cw stations in conjunction with a wide band receiver.

In the “old days” the transceiver was connected exclusively to the logging software like Win-Test or N1MM. With the introduction of CW-Skimmer another application “announced” the access to the transceivers interface. Unfortunately, Windows allows natively only one application to use the serial (com)-port at any moment.

Win-Test & CW-Skimmer with FT1000MP

But nobody is wondering, that in the golden days of Computer Virtualization also virtual COM-Port drivers became available. Several companies offer a variety of virtual COM port/splitter/router applications. This is a list of the ones I found:

A Virtual Com-Port Splitter allows us to access the same physical Com-Port of a Computer multiple times simultaneously. My favorite one is Eterlogic’s VSPE. It’s intuitive, very reliable and freeware!

This is how the configuration looks like now:

Win-Test & CW-Skimmer with VSPE talking to FT1000MP

Now we have established the communication, we can start with the problems ;-)

Virtual Serial Splitter allow us to use the same com port with several applications, however they have no intelligence regarding packet routing. Let’s have a closer look what I mean with packet routing.

The virtual splitter sends the data requests from both applications to the transceiver:

CW-Skimmer & Win-Test using VSPESince serial communication was defined as a point to point communication, no address is included in the data packages. For this reason, VSPE sends all incoming data packages to both applications. Since this will include “answer” data packages which haven’t been requested the one or other application might get into some trouble.

Win-Test & CW Skimmer receiving all packets from FT1000MP

In this case, Win-Test has some trouble. With a polling ratio of 1:10 (Win-Test: 100ms and CW-Skimmer: 1000ms) the frequencies are displayed in both applications correct. Unfortunately Win-Test‘s Bandmap always toggles between the first and second VFO wrongly indicating “Split” Operation. During serious operations this get quite soon very annoying especially during crossband operation, when QSOs are accidentally logged on the wrong band.

See the following two screenshots when the bandmap is in “normal” and “split” operation.

Win-Test Screenshot Nosplit Win-Test Screenshot Split

However, N1MM doesn’t show this behavior at all. It works smoothly in conjunction with CW-Skimmer on the same COM Port. So, what is difference? Isn’t serial communication = serial communication ? No, it isn’t!

FT1000MP Serial Communication Protocol

Before we go into deeper analysis, we actually need to know what we are looking for. Therefor we should take a brief look at the FT1000MP communication protocol. I picked out the most important messages. Please note that the transceiver’s serial communication protocol is explained very well in the FT1000MP user manual (page 73 – 83). The manual can be downloaded from several websites.

For the FT1000MP we distinguish between “command (request) messages” and “answer messages”.

Command messages consists of 5 bytes (using HEX numbers).

The two most important commands are:

Request (HEX): 00 00 00 03 10 -> “Please send me MAIN VFO A & Sub VFO B Data” (i.e. Frequency, Band data, filter settings…etc)

Request (HEX): 00 00 00 00 FA -> “Please send me the status flags” (i.e. split operation, dual reception, Antenna tuning in progress…etc).

Reply on (HEX) 00 00 00 30 10:

FT1000MP protocol answer bits

FT1000MP protocol frequency bits

This is the description of status flag 1 (which is for our purpose the most important one, because it contains the “Split Mode” setting):

FT1000MP protocol status bits

Tracing the serial interface

Now that we know what we have to expect we’ll start tracing the serial interface.

For tracing, monitoring and debugging purpose I recommend the following tools:

  1. Portmon, part of Microsoft’s great Sysinternal Suite (freeware)
  2. HHDSoftware’s Free Serial Port Monitor (freeware)
  3. Realterm, Flexible Terminal Programm (open Source)

Lets begin with the communication between N1MM and the FT1000MP:

N1MM communicating with FT1000MPN1MM Serial Data Sniffer

We can see that N1MM is requesting all data (two data requests) within a single package:

Request (HEX):
00 00 00 03 10 00 00 00 00 FA

Answer (HEX):
11 01 55 FA 40 FF D0 02 B3 00 11 B3 11 11 11 00 (VFO1)
11 01 56 5C 00 00 00 02 B3 00 11 B3 11 11 11 00 (VFO2)
0A 20 00 03 93 (Status Flags)

Lets try Win-Test and the FT1000MP:

Win-Test talking to FT1000MPSniffing Win-Test Serial Communication

We can see that Win-Test is requesting the data in two consecutive packages:

Request (HEX):
00 00 00 03 10

Answer (HEX):
11 01 55 FA 40 FF D0 02 B3 00 11 B3 11 11 11 00 (VFO1)
11 01 56 5C 00 00 00 02 B3 00 11 B3 11 11 11 00 (VFO2)

Request (HEX):
00 00 00 00 FA

Answer (HEX):
0A 20 00 03 93 (Status Flags)

And now we will trace the communication between CW-Skimmer and the FT1000MP:

cw-skimmer with FT1000MPcwskimmer_trace

Well, and CW-Skimmer is, requesting quite a lot of additional information.

Request (HEX):
00 00 00 02 10

Answer (HEX):
11 01 55 FA 40 FF D0 02 B3 00 11 B3 11 11 11 00

Request (HEX):
00 00 00 03 10

Answer (HEX):
11 01 55 FA 40 FF D0 02 B3 00 11 B3 11 11 11 00 (VFO1)
11 01 56 5C 00 00 00 02 B3 00 11 B3 11 11 11 00 (VFO2)

Request (HEX):
00 00 00 00 FA

Answer (HEX):
0A 20 00 03 93 (Status Flags)

Request (HEX):
00 00 00 01 FA

Answer (HEX):
0A 20 00 00 09 00 (Status Flag)

Okay, honestly CW-Skimmer doesn’t need that much information. CW-Skimmer only needs the frequency of the VFOs. In order to reduce the possibility of message interferences on the com port it is advisable to reduce the traffic and length to a the amount as little as necessary. Due the the fantastic flexible interface of VE3NEA’s OMNI-RIG (that’s CW-Skimmers Transceiver Interface engine) I changed the communication/request pattern to “N1MM style”.

Multiple Software accessing a the single COM Port

Well, so far we haven’t seen any conflict. Let’s analyze the serial port while CW-Skimmer and Win-Test are accessing it simultaneously. The configuration looks like this:

CW-Skimmer, Win-Test Tracing communication with FT1000MPThis is the trace:

CW-Skimmer & Win-Test Trace of Serial Communication

We can see that some of the messages are mixed up and other produce junk. In the case of the “status request” (00 00 00 00 FA) we can see that answers (or the bytes that win-test thinks to receives) alters in the first byte (remember the #1 Status byte). This would mean the one time the #1 status byte is set to HEX: 0×11 -> 1111 1111 and the other time to Hex: 0x0A -> 0000 1010. This would toggle bit 0 “Split Mode” flag.

Tracing the connection with N1MM and CW-Skimmer (modified communication pattern) shows better answers. At least there seem to be no more clashed packages. Please note that FT1000MP seems to respond by repeating twice the requested information. Luckily, both applications can handle this without any problem.

CW-Skimmer & N1MM Trace RS232 Communication

Verification

At this stage we already have isolated the problem pretty good.

1. VFO Data and Status bytes should be requested in one “package”

2. If they are requested in two separated “packages” this can cause trouble and “misunderstanding”.

3. The aborted messages and package clashes cause Win-Test to misinterpret the first status byte (which contains the “Split-Mode” flag).

In order finally verify these theses we will simulate the FT1000MP by replacing it with the HEX-capable Terminal Program Realterm.

As mentioned, we connected logically through a virtual Com-Port (VSPE “Pair”) with our terminal program to the “other side”, where the FT1000MP should be. In Realterm we can see that Win-Test is trying to poll the FT1000MP. We will now feed the answers we traced earlier in order to find out if our theses are correct:

Realterm Configuration to Emulate FT1000MP

realterm Screenshot

1. Win-Test Request (HEX):
00 00 00 03 10

1. Realterm Answer (HEX):
11 01 55 FA 40 FF D0 02 B3 00 11 B3 11 11 11 00
11 01 56 5C 00 00 00 02 B3 00 11 B3 11 11 11 00

2. Win-Test Request (HEX):
00 00 00 00 FA

2. Realterm Answer (HEX):
0A 20 00 03 93 (Status Flags)

alternative second Realterm Answer (HEX):
11 20 00 03 93 (Status Flags)

And indeed, as soon as we feed the second message, Win-Test starts toggling the VFOs.

Conclusion

We have seen how it is possible to analyze and investigate a problem. In detail, we have investigated the behavior of various programs with a “non standard configuration” by allowing these programs to talk simultaneously over the same serial port with an HF transceiver. By using serial analyzing software we found out that there is in deed a difference how to obtain the same data from the transceiver. At the end we finally verified how a optimum “data request” should look in a virtual com port environment.

I’ll contact now the the producers of Win-Test and kindly ask them if it might be possible to change the pattern of the requesting command. Even better would be, if the Omni-Rig Engine would be directly supported by Win-Test.

During this investigation I used the following software versions:

Windows XP: SP2 (Administrative rights)

Win-Test: 3.21.0

N1MM: 8.0.0

CW-Skimmer: 1.2

VSPE: 0.80.2.385

Free Serial Port Monitor: 3.31

Realterm: 2.0.0.57

About Tobias (DH1TW)

Self-confessed Starbucks addict. Loves to travel around the globe. Enjoys the technical preparations of Amateur-Radio contests as much as the contests themselves. Engineer by nature. Entrepreneur. For more, follow him @DH1TW

Comments

  1. PAUL says:

    VSPE run ok and very easy to configure it.

    BUT I never a running right with 2 softwares if each have a waterfall…

    the maxi config.

    mixwin + dxbase+atlas+rttyrite ( from writelog)+spectravue (soft panoramique for SDR-IQ no rs232 used

    I read “langsam” your solution

    73 paul F5QE

  2. Hello Tobias!
    tnx for this nice page. I do like your clean design and the praezise writen explanations very much.
    Keep up the gus work!
    55 de Lup DJ7SW/DL0KF

  3. Sotiris SV1BDO says:

    Very well written article Tobias!
    Have you tried to add in the above configuration a software (Winrad or PowerSR-IQ) that will control an external SDR receiver, like PM-SDR? What will be the VSPE settings?
    Have you tried your debugging procedure with other transceivers, like Kenwoods or Icoms?

    73, Sotiris SV1BDO

    • Hey Sotiris,

      no, I just did the debugging on the Yaesu FT1000MP. Since the FT1000MP is one of the few radios with two real receivers, this problem might only occur on a few more Ham Radio Transceivers.

      In my FT1000MP I used a fixed frequency softrock clone. Since the IF frequency is always fixed (455kHz) there was no need to connect a tunable receiver to its Intermediate Frequency.

  4. HA1AG says:

    UFB analysis and troubleshooting. Toby! I’m experimenting with VSPE as my N8VB driver stopped functioning since the last XP update. 73 zoli ha1ag

Trackbacks

  1. [...] Dabar besikuisdamas internete uztikau dar daugiau ta COMM port dalinanciu programeliu dykai. Labai geras straipsniukas ( nuoroda) ta tema: http://www.dh1tw.de/debugging-serial-port [...]

  2. [...] This made it impossible to use this setup in an operational contest environment. Check out my Win-Test & CW Skimmer investigation if you like to go down as deep as bit & bytes [...]

Speak Your Mind

*