Instrumentation - Linux Embedded systems

Instrumenting code is something that almost everyone has done and is a very effective way of debugging on machines with limited resources; the code for printing out something to the console is probably already part of the application. Adding a few more lines of output doesn’t demand much in terms of memory or resources. Instrumentation is as simple as putting a print statement in the code, running the program, and watching the results. Some target machines don’t have the resources necessary to run even a small program like gdbserver to host a debugging session. Instrumentation is also the best way to find bugs that appear at odd intervals or to solve timing-related problems, where interactively debugging skews the results of the program.

The biggest drawback of this approach is that the code itself contains debugging statements that should be removed for production builds. There isn’t a “strip” command for removing debugging print statements like there is for removing debugging symbols. For this reason, many QA departments want to retest the code after instrumentation has been removed, because it does require a rebuild of the software. Minor drawbacks include the fact that you may mistakenly write instrumentation code that has a side effect on the program; for example:
printf("value of a=%u\n", a++); // changes the value of a!

When this is removed from the code, it will probably have an effect on how the program operates because it changes the value of a in the print statement. Most engineers know not to do something like this, but everyone makes mistakes.

Instrumentation in C is very simple because of the preprocessor. Consider the small example project that uses a symbol D to enable debugging. When the symbol D is set to 1, extra flags are set to enable debugging. You can use the same mechanism to define a preprocessor symbol that includes code in the C files when the symbol is defined. Defining directives means the symbol is set to some value; any value will suffice. For instance, the make file may look like the following:

CPPFLAGS aren’t additional flags for the C preprocessor C++, even though the CPP in the name may lead you to that conclusion. The source files in the project may contain source code like the following:

This sort of code is peppered through the project, which causes a readability problem if you don’t regularly remove the debug output that you know is no longer used. This is a matter of how the company manages the source code: some companies forbid checking in code containing any debugging print code but support the technique as a way for you to debug code on the desktop.

Notice that the output is sent to the stderr file. A program has two output handles, stdout and stderr. Reserve stdout for output that’s always intended for a user and stderr for errors (obviously) and debugging information; that way, you have a method of getting to debugging data in the field without rebuilding the application, You can also open an additional file handle just for printing debugging messages if the regular output to stderr contains information that’s used elsewhere and the instrumentation would interfere.

Debugging messages don’t usually go to the console; instead, you can send them to the system log, which is a buffer of memory that the kernel maintains. When the buffer fills, new data goes into the top, overwriting the oldest data. This structure is called a circular buffer and is used when the system can afford to lose old data in an overflow situation. Chapter contained code for producing system log messages that can also be wrapped in a macro so they aren’t included in the final product. For example:

This has the effect of logging the code in a location such that even if it’s accidentally sent out with debugging information, the result isn’t as bad because the debugging output is hidden from most users. On a low-resource system, the system logger is off and the buffer is small, so not much memory or processing time is wasted. The point isn't to plan on software going out with debugging instrumentation. If the code is being sent out even though it hasn’t been compiled properly, the consequences aren’t as dire as when you write a large amount of data to the console.

The output of syslog() can also be sent to another machine by the daemon handling the syslog buffer, if it’s running. By starting this program with the -R option, you can send the output to another machine that is configured for listening. For example, on the embedded device, BusyBox’s syslogd is started like this:
$ syslog -R <ip address>

On the machine at <ip address>, syslogd needs to be configured to accept remote syslog information; you do so by adding the argument -h to syslogd and restart the service. If you’re running Ubuntu, the file where this option is set is at /etc/default/syslogd. After this is configured, the syslog output appears in the development host log. When you’re testing a device that is memory constrained, being able to send the debugging data to a machine with more storage makes this debugging practical.

All rights reserved © 2020 Wisdom IT Services India Pvt. Ltd Protection Status

Linux Embedded systems Topics