cool ideas and hardware

reverse engineering (USB) serial protocols on windows

This post is part of a series of posts on reverse-engineering Serial protocols using Wireshark.
Here's the other posts in this series:

  1. reverse engineering (USB) serial protocols on windows
  2. decoding CH340/CH341 USB messages in Wireshark

I’m trying to reverse engineer the serial protocol of a solder reflow oven since the manufacturers software is windows only, unresponsive and lacks some features. It also seems like a good project to practice reverse engineering. This will probably end up as a small series of posts about different parts of the process, this first post is about the various approaches I tried and how I ended up gathering data to analyze.

The reflow oven has an RS-232 port and comes with a CH340-based USB-to-Serial converter. There’s also a USB stick with drivers to install for pre- and post-Windows 7 PCs, and the oven’s software itself. My goal is to be able to see and record the bytes sent back and forth over the serial connection while I use the oven’s software. From my experience on Linux I figured it would be pretty easy to intercept the serial data stream in software, but it turned out quite a bit more complicated than I thought.

Here’s the various approaches I considered or attempted:

sniffing the COM port with portmon

portmon is a tool by microsoft that is made for the exact purpose of monitoring serial ports. Great, so we’re done, right? Well… Wrong. Apparently portmon stopped working starting from Windows 7 on and noone has bothered to fix it. Some people claim they’ve gotten it to work for 32-bit apps using various compatiblity modes etc but I’ve given up after trying this for days on a Windows 7 machine before so I decided to look for another way.

  • requires:
    • Windows XP, apparently?
  • result: skipped

sniffing the COM port with a third party serial monitor software

I had found a shareware tool that mostly worked last time I researched this but limited capture length and exporting options. It was kind of a pain. I thought there should be a better way so I skipped this too. - - result: skipped

  • requires:
    • patience or cash
  • result: skipped

using a logic analyzer on the RS-232 connection

I’ve tried this in the past for another project and it worked pretty well, but I don’t have access to the Logic Analyzer I had used anymore. The cheap LA I do have only supports 0-5V signals. I also couldn’t find the solderable DE-9 connectors I had bought last time around so I decided to move on to a different plan.

  • requires:
    • logic analyzer that can handle ±15V signals
    • DE-9 connectors or a sacrificial DE-9 cable
  • result: skipped

using a logic analyzer on the control board

The MCU on the control board presumably runs on 5V or 3.3V and uses uses an IC to handle the RS-232 communication. By attaching a LA to the lines between MCU and that IC we should be able to read the serial messages as well. That would require disassembling the oven, which is not only messy and prevents me from actually using the oven until I’m dome, but might also limit the kinds of tests I can do in the meantime. There’s also a small chance of breaking something.

  • requires:
    • cheapo logic analyzer
    • disassembling the oven
  • result: skipped

running the software on Linux with wine

The oven’s software looks like old windows software, and WINE presumably fares better with old windows software, so maybe it’d manage to run it fine? It’s at least equally likely that WINE wouldn’t manage to run it (properly), and instability or strange behaviours might take some time to discover. Getting something to run things in WINE is a major pain in the ass at the best of times, but getting chinese industrial control software I know almost nothing about just sounds like a really bad time. Pass. The oven is at work and my work PC runs Windows only at the moment, so it would’ve taken a little effort to try this.

  • requires:
    • patience
  • result: skipped

sniffing the COM port with com0com

com0com is a “virtual loopback serial driver” which allows you to redirect communication from software that is made to talk to hardware devices to other software. It creates pairs of virtual COM ports that forward data between each other. My plan was to hook the oven’s software up to one port of such a pair, then write a small script that logs and forwards messages between the other port of the pair and the hardware port. I was hoping that I could do that from within the WSL because I hate windows. It turns out that the newer WSL2 doesn’t allow sharing COM ports between Windows and Linux in any way, so I looked for some other way again.

  • requires:
    • com0com v3.* (official signed version available)
    • disabling secure boot
  • result: might have worked?

bridging the COM port to TCP with com2tcp, then logging the network traffic

While searching for tools I also found com2tcp which bridges Serial communication to TCP. It consist of two very 90s looking GUIs that you can move around on your desktop, one of which creates a TCP server bridged to a physical COM port, while the other creates a virtual COM port that can communicate to such a TCP server.

By running both parts on the same PC but having a server man-in-the-middle the TCP connection, in theory I should be able to intercept the traffic. I had this most of the way set up when for a brief moment of time the layers of yak shave curled back onto themselves and I caught a glimpse of reason. This was not the way. Here are some of the reasons why:

  • the TCP comms will have some type of framing to communicate flow control signals, so now I’d have two undocumented protocols to reverse engineer to get to my data.
  • setting this up requires knowing the baud rate and serial parameters. I assumed it would be either 9600 or 115200 but that turned out to be wrong
  • these tools need to be started and connected in a particular order, which makes the total connection process even more complex and potentially error prone

good riddance.

  • requires:
    • concentrated crystalline desperation
  • result: no

sniffing the USB-Serial communications with Wireshark

While getting ready to sniff network traffic I installed Wireshark and noticed that the installer offered to install USBPcap. To use USBPcap make sure secure boot is disabled, you’ve rebooted since the install. You also need to copy USBPcapCMD.exe to the wireshark extcap directory. Capturing the USB communication between the PC and the USB-to-Serial converter will reveal everything the oven software is doing, but the price is that now we have to reverse-engineer the USB communication between the driver and the CH340 serial chip. Luckily the CH340/341 chip’s protocol, although undocumented, is supported in the Linux kernel so there is prior art on it. Capturing USB traffic worked right away. It helps to disable the “Capture from all devices connected” and “Capture from newly connected devices” options and select only the device in question in the USBPcap capture settings.

The amount of messages I captured was overwhelming at first, but I soon got the hang of it. After the enumeration messages (GET DESCRIPTOR, SET CONFIGURATION), there’s a burst of URB_CONTROL messages going both ways and then only URB_BULK messages follow.

It seemed pretty clear that the URB_CONTROL was mostly the serial driver configuring the chipset for the serial parameters, while the URB_BULK messages all contained some “leftover capture data”. This turned out to just be exactly the actual serial data going both ways! I eventually also managed to put together a small Wireshark “dissector” to understand the control messages which helped me figure out the baud the serial communication uses, bu that’s for another blog post :)

  • requires:
    • Wireshark
    • USB-to-Serial converter
    • disabling secure boot
  • result: yay!

In conclusion, this took wayyyyy too much time to figure out, but I’m happy that I finally landed on a process that works, is comfortable, and powerful as well. I’m also very happy with the knowledge I’m gaining (how USB works, what linux driver code looks like) and the skills I’m building (debugging USB, writing Wireshark plugins) in the process. 15-to-20-year-old me would be very impressed!