Jul 21,2019 | Posted by Or Cyngiser, Senior Security Researcher
Today we will be presenting a mental framework for approaching IoT security analysis, giving our readers a sound overview of the different stages so that they may pursue their own research projects.
IoT devices are interesting attack targets for multiple reasons. They often allow attackers to cross the bridge from a pure network threat to physical compromise, via devices such as smart locks and doorbells. Other times they can be used in espionage, as by taking control of cameras or microphone-enabled devices attackers gain a great deal of information on their victims. With more and more fragments of our homes being replaced by their smart equivalent – lighting, air conditioning, thermostats, refrigerators and so on, we are increasingly vulnerable to attacks which leave us helpless in our own homes. Additionally, some of the greatest DDoS attacks in history were performed by botnets spread by IoT vulnerabilities.
Suppose we decided on a device to examine. The best way to begin is often by consuming as much openly available information as possible. Reading up the manufacturer’s user guide and specifications will help map all the device’s features and may already hint at their implementation. Wikipedia, tear-down articles and previous researches all go a long way to being familiarized with the device ecosystem.
The next step is a full hardware examination of the target. IoT devices come in a wide spectrum of shapes, sizes and designs, but schematically they are quite similar. By either unscrewing or slashing open the device you will get to a board (PCB) connecting electronic circuits of chips, resistors, capacitors and so on. There will be a main chip, either a microcontroller (MCU) or a system-on-chip (SoC), which is an integrated circuit containing a microprocessor, boot ROM, flash memory, RAM and in the case of SoC a variable number of additional components, such as a GPU and a connectivity processor (WiFi, Bluetooth, Zigbee etc). Frequently additional flash storage or RAM are required and are connected to the main chip on the PCB. The various buttons can often be traced onto GPIO pins, as well as common components like digital signal processors (DSP), LEDs and speaker modules. Importantly, chip debugging interfaces will often be exposed on the board in what are called “test points”. Sometimes they will even be labeled descriptively.
Main PCB of the Echo Dot
Packaging diagram from the DM3725 SoC datasheet
Staring down a new PCB may be discouraging but things usually get clearer swiftly. Almost always the significant chips will be labeled, and a quick Google lookup will lead to a detailed datasheet (unless it is only distributed under NDA). Tracing connected components on a board can range from being trivial to extremely difficult, depending on the number of copper layers above the substrate and whether you have an X-ray machine at your disposal (i.e. universities and three letter agencies).
The two main ways to proceed are firmware extraction and getting debugging functionality on the processing chip. Getting the firmware is different depending on the type of storage, but generally it is done by mounting the chip on a hardware device called a “programmer”. Programmers are either vendor-specific or universal, which are preprogrammed to interface with the respective chip pins via SPI (serial peripheral interface) or some proprietary protocol. The extracted binary contents are usually an ARM firmware image, consisting mostly of an interrupt vector and machine instructions. Various tools such as IDA Python Embedded Toolkit make reverse engineering of this binary blob much easier, identifying the main function and mapping data and string pointers to code.
TL866 Universal Programmer
On more complex systems, the file system is sometimes stored on eMMC cards, essentially an SDcard variant with ball grid array (BGA) packaging. Reading eMMCs is widely supported, you can identify the package type and purchase an appropriate eMMC to USB or SD adapter. When you connect the adapter to a PC with the matching file system support (usually ext4), it will automatically mount the discovered partitions. Note that after unsoldering eMMCs reinserting them on the board is a tedious technical process and usually not worth the effort (but still worth considering, for example for backdooring an operating system).
eMMC BGA100 reader
Interfacing with the SoC is more difficult but also more rewarding. The key idea is to use either leftovers from the development environment or purposely built backdoor functionality to debug and control the chip. The most well-known interfaces are UART and JTAG, but there are plenty of others like SPI, SWD, I2C and many proprietary ones. These are all specifications for a ruleset defining how electronic signals are transmitted between two components via wire sets and the governed state machines. For UART and SPI, it is up to the IoT device manufacturer to program their functionality, for example to expose a shell or memory read / write access. Dumping firmware may take you a long way to interact with those pins correctly, but just trying to get a serial shell over UART is worthwhile. JTAG, SWD and chip-proprietary protocols are usually handled by the permanent boot ROM itself (the very first code segment actually executed, usually read-only to prevent bricking). Sometimes the debugging ports are exposed on the PCB and are easy to hook to once identified, however in many cases you will need to solder wires onto the chip itself, using the datasheet as a reference for the pin layout. Identifying unmarked debugging ports is both an art and a science, beyond the scope of this article.
UART connection between two devices
After finding an interface, you will need to connect to it via hardware and software. One option is to use Swiss army knife products like the Bus Pirate which support many hardware protocols. A more recommended approach is a dedicated component for each one – for example J-Link for JTAG connections, FT232 for UART and so on. Make sure to check the specific chip is supported by the JTAG software suite that drives the JTAG probe. It is worth noting that JTAG does not define high-level operations such as reading or writing firmware, changing register values and stepping in / over. It is up to JTAG software running on both the client and the debugged device to implement them using the primitive framework supplied by JTAG operations. Also, cross your fingers that the JTAG fuse wasn’t blown on the assembly line disabling any JTAG activity.
JTAG state machine specification
Standard 10-pin JTAG
Moving on from hardware to software, it is worthwhile to discuss the code stack found in firmware. In a desktop or mobile environment, the software stack called, for example, by shelling a bird in “angry birds” crosses numerous layers of developer entities. Application specific code written by Angry Birds’ designers uses, for example, some Apache Java libraries, which in turn are processed by the Android runtime (basically a JVM). The runtime eventually makes system calls which are implemented by an Android kernel ported twice, first by the SoC vendor to support the chip in the HAL (hardware abstraction layer) level, and again by the phone manufacturer to accommodate for the exact device model. In contrast, IoT products have a varying number of software layers. The SoC code, usually referred to as original equipment manufacturer (OEM) code, is always at the bottom and provides APIs for handling IO ports such as UART, GPIO and SPI, by accessing memory-mapped IO. Additional services may be hardware interrupt handling and clock management. In more complex devices, OEM may supply a ported operating system, often a real-time OS such as FreeRTOS but possibly even a full-fledged Android. On top of OEM code, the vendor writes its application specific software which fulfills the product’s purpose.
The three topmost layers are optional in IoT
So finally, we reached software reverse engineering. Analyzing OEM code without any reference is extremely difficult. The SoC datasheet is essential for understanding the system’s memory map (roughly speaking, how all the components are mounted in memory) and thus how the chip interacts with the outside world. It will also specify the hardware architecture, the boot sequence and any debugging interfaces supported. The next step is obtaining the SoC SDK, usually comprising of header files and libraries implementing OEM code. Many manufacturers release their SDK on the internet, sometimes packaged within a dedicated IDE. Understanding the firmware API greatly helps reverse engineering product code as it will be much clearer what SoC pins are being used and how they are translated to SoC functionality. As an example, imagine there is a call to a function which contains a read operation from 0x02004000 and then the result is compared with a hardcoded ASCII string. Looking up this address in the SDK headers we may find it is the UART RX (receive) buffer and conclude the string is a pre-programmed UART command which we can use. It is worth mentioning the datasheet will also provide an IO memory map, but likely not in the same high degree of granularity as header files.
TI CC3200 start of memory map
GPIO addresses taken from TLSR8677 header files
After studying the API, including provided example programs, a good exercise is to compile some samples and study the compiled firmware. Comparing the compiled and source code of a controlled model helps a lot in understanding the general structure of the firmware and will eventually make studying the compiled firmware much more manageable.
At this stage the researcher should consider the merits of reversing the compiled OEM code and hunting for vulnerabilities. This avenue differs from product-specific vulnerabilities is multiple ways. Most importantly, bugs in OEM transitively affect every product using the chip, multiplying the scope of impact often by orders of magnitude. Secondly, vendors are more likely to update their own software than OEM code, so bugs may entrench for years. On the minus side, this code is usually particularly hard to reverse engineer. It deals with the lowest layers intentionally hidden away from developers to save bugs – clock synchronization between peripherals, MAC/PHY layer code of wireless protocols and so on. Chip manufacturers carefully review each line of code, recognizing the number of devices which will use it. Finding bugs in this region is probably beyond the skillset of most vulnerability researchers.
Telink only releases library files for some SoC code
After using the device, reading documentation and hopefully inspecting its firmware code, you should have a reasonably solid overview of the product. At this stage, prepare a list of as many interfaces or attack vectors to the device as possible. It should include obvious surfaces like UART, buttons and remote controls via an app, as well as more discreet ones such as a DHCP client, any open ports, BLE characteristics and so on. Note that many attack surfaces are only reachable in a specific flow, for example the device may host a vulnerable web server in the initial wireless configuration stage or open a DNS resolver in a “captive portal” environment. Any running code that processes some type of user input is a potential compromise and listing as many of them allows the researcher to assess and prioritize between different attack flows.
A great way to uncover more attack vectors is to study the device’s controller app. By reverse engineering the Android / iOS application, you can see the specific protocol where commands and responses are exchanged, usually much clearer than in the firmware. If there are any hard-coded authentication mechanisms, they will be found in the app data.
Reverse engineering .apk files using jadx
Now would be a good time to set up a networking lab for the device and mobile or desktop app. Open a Linux box and set up an access point via hostapd and dnsmasq and bridge it to the WAN interface. You should be able to connect to it with the application host and the smart device. Sniffing on the wireless interface will show the application and device traffic, which will help expose new attack vectors. For viewing HTTPS traffic and modifying HTTP/S requests (to the cloud or to the device), use a HTTP MitM (Man in the middle) suite like Burp. One consideration to watch out for is to make sure target traffic is popped off the bridge and passed to the network stack via ebtables and then redirected to the MitM listening port via iptables. Another caveat is to install Burp’s certificate as a trusted CA on the smartphone / PC. Some apps use a technique called certificate pinning which rejects even a trusted CA, however this can be bypassed using SSLUnpinning on a rooted Android and similar tools for other platforms. You can even try intercepting the IoT device’s SSL traffic, which will only work if it does not validate the chain of trust properly. For Bluetooth devices, tools like GATTack can help intercept and modify BLE traffic, useful for mapping requests and their implementation.
We are rapidly reaching the limit to what can be generalized and given as generic advise for IoT devices. At this stage, proceedings are similar to any other static vulnerability research - there are three main approaches: Firstly, an A to Z analysis of the target component or complete platform. Although it is infeasible for sufficiently large firmware, it is very educational and a great way to convincingly cross off vulnerabilities. Another approach is to locate where user input in received and perform branching analysis / taint analysis from there, hopefully finding a desirable program flow down the line. Such a flow may be a use-after-free, command injection, buffer overflow, insufficient access check or any other of the tens of common security pitfalls. The last main approach is finding dangerous fragments of code and traversing back the code graph looking for potential inputs that could lead to a flaw. Some examples are strcpy() and exec() variants, code segments parsing a shell and so on. Much of what makes a good researcher is adapting and choosing the correct techniques for the specific problem at hand.
Exploitation is a bit of a double-edged sword in IoT. On the one hand, mitigation is usually far behind the standard in mobile, desktop and server environments. Most embedded systems are simply too low on resources to implement harsh security policies like W^X, shadow stack and ASLR. However, the lack of debugging facilities in the event hardware hacking attempts fail makes exploitation very difficult. For a bare-metal or proprietary embedded OS, shellcode needs to be written to the specific environment, largely with help from the discussed SoC IDE. If the exploit fails it is next to impossible to troubleshoot. The best practice is to set up a device clone (with the dumped firmware) which is as similar as possible to the original device and use the IDE debugger or some compiled gdbserver to diagnose the exploit. This opens the door for many other opportunities like dynamic analysis and on-chip fuzzing, which we will save for another day.
To summarize, we have discussed a framework for performing a security analysis of IoT devices. The goal was to give just enough tools, details and guidance so that readers can start researching by themselves and when faced with obstacles, ask the right questions. Good luck and happy hacking!
Images courtesy of: