# Getting Ready for Debugging - Linux Embedded systems

The limited resources on an embedded system mean you need to begin thinking about application debugging while you’re doing the application development. This section contains some best practices that help make debugging easier when the target machine lacks a keyboard, mouse, or monitor. If you’re accustomed to debugging an operating system that included the application as a process within the OS (hint: this OS begins with the letters vx), Linux works differently, and this requires a change in practice. The project you created earlier performs one of the critical steps in preparing to debug: creating a build for the software that runs on the development target. If you can run and debug the code on the target, that’s the most effective way to root out gross defects. Finer defects, like those surrounding timing or handling the input from a device, must be done on the board. Intelligently configuring the project so that it’s isolated enough from its hardware offers a handsome return, because debugging on the development host is much more productive than remote debugging.

Next, no matter what the target, you should compile the program being built with no optimizations by the compiler; you do so by using -O0 (capital 0, number zero) to compile the program. High optimization levels make it so that the program flow doesn’t match that of the language; stepping through code with a debugger can be very confusing because there’s little association between a line of source code and the machine code generated by the compiler.
The most popular tool for debugging code running on a target is printf(). This unobtrusive, lowresource tool has the flexibility to handle most anything. But if you’re new to the craft, it seem stodgy and downright primitive. Because most of the debugging happens while the code is running on the host, using printf() is more reasonable that you may initially think.

When you use printf, the best practice is to print all debugging output on standard err (stderr) and leave standard out (stdout) for regular, nondebugging messages. When a program starts, both stderr and stdout are associated with the application’s console, so it appears that they’re the same.

When output appears on the console, you may assume it’s data from stdout, but it may be from either stdin or stdout. Take the classic “Hello World” program, and modify it a bit so that it prints to stderr:

Store this in a file (hello.c), compile, and run it. The following appears:

Now, try to direct the output into a file:

The > symbol results in stdout being sent to the file hello-out. To redirect stderr, do the following:
./hello 2> hello-out

This works because stderr is opened at file handle two. It also separates the diagnostic output from program output, as the program could be run to send all of the output into a file that the program will throw away:
$/hello 2> /dev/null The next approach is to use the system log, or syslog, as a place to write debugging data. The syslog is a buffer in memory that is monitored by a process that fetches the data and writes it to a file. On most systems, that file is /var/log/messages, but it can be any file. Consider this sample code, saved in log.c: Compile this with the following command. Cross-compiling this isn’t important right now, because this code behaves the same way on a target machine:$gcc log.c o log

Run the code; nothing appears on the console. However, look at the contents of
/var/log/messages, which has the following output:
May 10 12:04:23 sony-laptop ./log[32056]: Notice message
May 10 12:04:23 sony-laptop ./log[32056]: Informational notice

The setlogmast() function changes the messages that are sent to the syslog. This function accepts a bitmask for an acceptable messages, and the helper macro LOG_UPTO means that messages less than the argument to LOG_INFO aren’t displayed. Table lists the possible message codes and when they should be used, from highest to lowest.

Using this facility means you don’t need to change or recompile the code to change the chattiness of what’s kept in the log file. Using this approach also results in more flexibility, because you can change the log mask on the fly to track down a problem that can be produced in the field.