Command Line Tools

CPIP has a number of tools run from the command line that can analyse source code. The main one is CPIPMain.py

CPIPMain

CPIPMain.py acts very much like a normal pre-processor but, instead of writing out a Translation Unit as test it emits a host of HTML and SVG pages about each file to be pre-processed. Here are Some Real Examples.

Usage

usage: CPIPMain.py [-h] [-c] [-d DUMP] [-g GLOB] [--heap] [-j JOBS] [-k]
                   [-l LOGLEVEL] [-o OUTPUT] [-p] [-r] [-t] [-G]
                   [-S PREDEFINES] [-C] [-D DEFINES] [-P PREINC] [-I INCUSR]
                   [-J INCSYS]
                   path

cpip.cpp -- Preprocess the file or the files in the directory.
  Created by Paul Ross on 2011-07-10.
  Copyright 2008-2015. All rights reserved.
  Licensed under GPL 2.0
USAGE

positional arguments:
  path                  Path to source file.

optional arguments:
  -h, --help            show this help message and exit
  -c                    Add conditionally included files to the plots.
                        [default: False]
  -d DUMP, --dump DUMP  Dump output, additive. Can be: C - Conditional
                        compilation graph. F - File names encountered and
                        their count. I - Include graph. M - Macro environment.
                        T - Token count. R - Macro dependencies as an input to
                        DOT. [default: []]
  -g GLOB, --glob GLOB  Pattern match to use when processing directories.
                        [default: *.*]
  --heap                Profile memory usage. [default: False]
  -j JOBS, --jobs JOBS  Max simultaneous processes when pre-processing
                        directories. Zero uses number of native CPUs [4]. 1
                        means no multiprocessing. [default: 0]
  -k, --keep-going      Keep going. [default: False]
  -l LOGLEVEL, --loglevel LOGLEVEL
                        Log Level (debug=10, info=20, warning=30, error=40,
                        critical=50) [default: 30]
  -o OUTPUT, --output OUTPUT
                        Output directory. [default: out]
  -p                    Ignore pragma statements. [default: False]
  -r, --recursive       Recursively process directories. [default: False]
  -t, --dot             Write an DOT include dependency table and execute DOT
                        on it to create a SVG file. [default: False]
  -G                    Support GCC extensions. Currently only #include_next.
                        [default: False]
  -S PREDEFINES, --predefine PREDEFINES
                        Add standard predefined macro definitions of the form
                        name<=definition>. They are introduced into the
                        environment before anything else. They can not be
                        redefined. __DATE__ and __TIME__ will be automatically
                        allocated in here. __FILE__ and __LINE__ are defined
                        dynamically. See ISO/IEC 9899:1999 (E) 6.10.8
                        Predefined macro names. [default: []]
  -C, --CPP             Sys call 'cpp -dM' to extract and use platform
                        specific macros. These are inserted after -S option
                        and before the -D option. [default: False]
  -D DEFINES, --define DEFINES
                        Add macro definitions of the form name<=definition>.
                        These are introduced into the environment before any
                        pre-include. [default: []]
  -P PREINC, --pre PREINC
                        Add pre-include file path, this file precedes the
                        initial translation unit. [default: []]
  -I INCUSR, --usr INCUSR
                        Add user include search path. [default: []]
  -J INCSYS, --sys INCSYS
                        Add system include search path. [default: []]

Note

Multiprocessing: The pre-processor, and information derived from it, can only be run as a single process but writing individual source files can take advantage of multiple processes. As the latter constitutes the bulk of the time CPIPMain.py takes then using the -j option on multi-processor machines can save a lot of time.

Options

Option	Description
--version	Show program’s version number and exit
-h, --help	Show this help message and exit.
-c	Even if a file is conditionally included then add it to the plot. This is experimental so use it at your own risk! [default False]
-d DUMP, --dump=DUMP	Dump various outputs to stdout (see below). [default: []]
-g GLOB, --glob=GLOB	Pattern to use when searching directories (ignored for #includes. [default: .]
--heap	Profile memory usage (requires guppy to be installed). [default: False]
-j JOBS, --jobs=JOBS	Max processes when multiprocessing. Zero uses number of native CPUs [8]. -1 disables multiprocessing. [default: -1]
-k	Keep going as far as sensible. [default: False]
-l LOGLEVEL, --loglevel=LOGLEVEL	Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 30]
-o OUTPUT, --output=OUTPUT	Output directory [default: “out”]
-p	Ignore pragma statements. [default: False]
-r	Recursively provesses directories. [default: False]
-t, --dot	Write an DOT include dependency table and execute DOT on it to create a SVG file. [default: False]
-C , --CPP	Sys call cpp -dM to extract and use platform specific macros. These are inserted after -S option and before the -D option. [default: False]
-G	Support GCC extensions. Currently only #include_next. [default: False]
-I INCUSR, --usr=INCUSR	Add user include search path (additive). [default: []]
-J INCSYS, --sys=INCSYS	Add system include search path (additive). [default: []]
-S PREDEFINES, --predefine=PREDEFINES	Add standard predefined macro defintions of the form name<=defintion>. These are introduced into the environment before anything else. These macros can not be redefined. __DATE__ and __TIME__ will be automatically defined. [default: []]
-D DEFINES, --define=DEFINES	Add macro definitions of the form name<=definition>. These are introduced into the environment before any pre-include. [default: []]
-P PREINC, --pre=PREINC	Add pre-include file, this will be included before any header (additive). [default: []]

The -d option can be repeated to generate multiple text outputs on stdout:

Output	Description
-d C	Conditional compilation graph.
-d F	File names encountered and their count.
-d I	Include graph.
-d M	Macro environment.
-d T	Token count.
-d R	Macro dependencies as an input to DOT.

Examples of these are shown below Using -d Option.

Arguments

One or more paths of file(s) to be preprocessed.

Examples

Here is a simple example of processing the demo code that is in the PpLexer tutorial here: Files to Pre-Process.

Here we set:

• l 20 sets logging to INFO
• -o sets the output to ../../demo/output_00/
• -C is used to get the platform specific macros.
• -J is used to set a single system include as ../../demo/sys/
• -I is used to set a single user include as ../../demo/usr/

We are processing ../../demo/src/main.cpp and stdout is something like this:

$ python3 CPIPMain.py -l 20 -C -o ../../demo/output_00/ -J ../../demo/sys/ -I ../../demo/usr/ ../../demo/src/main.cpp
2012-03-20 07:41:38,655 INFO     TU in HTML:
2012-03-20 07:41:38,655 INFO       ../../demo/output_00/main.cpp.html
2012-03-20 07:41:38,664 INFO     Processing TU done.
2012-03-20 07:41:38,665 INFO     Macro history to:
2012-03-20 07:41:38,665 INFO       ../../demo/output_00/main.cpp_macros.html
2012-03-20 07:41:38,668 INFO     Include graph (SVG) to:
2012-03-20 07:41:38,668 INFO       ../../demo/output_00/main.cpp.include.svg
2012-03-20 07:41:38,679 INFO     Writing include graph (TEXT) to:
2012-03-20 07:41:38,679 INFO       ../../demo/output_00/main.cpp.include.svg
2012-03-20 07:41:38,679 INFO     Writing include graph (DOT) to:
2012-03-20 07:41:38,679 INFO       ../../demo/output_00/main.cpp.include.svg
2012-03-20 07:41:38,679 INFO     Creating include Graph for DOT...
2012-03-20 07:41:38,692 INFO     dot returned 0
2012-03-20 07:41:38,693 INFO     Creating include Graph for DOT done.
2012-03-20 07:41:38,693 INFO     Conditional compilation graph in HTML:
2012-03-20 07:41:38,693 INFO       ../../demo/output_00/main.cpp.ccg.html
2012-03-20 07:41:38,698 INFO     Done: ../../demo/src/main.cpp
2012-03-20 07:41:38,698 INFO     ITU in HTML: ...\main.cpp
2012-03-20 07:41:38,708 INFO     ITU in HTML: ...\system.h
2012-03-20 07:41:38,711 INFO     ITU in HTML: ...\user.h
2012-03-20 07:41:38,716 INFO     All done.
CPU time =    0.051 (S)
Bye, bye!

In the output directory will be the HTML and SVG results.

Using -d Option

All these are using the following command where ? is replace with a letter:

$ python3 CPIPMain.py -d? -o ../../demo/output_00/ -J ../../demo/sys/ -I ../../demo/usr/ ../../demo/src/main.cpp

Multiple outputs are obtained with, for example, -dC -dF

-d C

Conditional compilation graph:

---------------------- Conditional Compilation Graph ----------------------
#ifndef __USER_H__ /* True "../../demo/usr/user.h" 1 0 */
    #ifndef __SYSTEM_H__ /* True "../../demo/sys/system.h" 1 4 */
    #endif /* True "../../demo/sys/system.h" 6 13 */
#endif /* True "../../demo/usr/user.h" 7 20 */
#if defined(LANG_SUPPORT) && defined(FRENCH) /* True "../../demo/src/main.cpp" 5 69 */
#elif defined(LANG_SUPPORT) && defined(AUSTRALIAN) /* False "../../demo/src/main.cpp" 7 110 */
#else /* False "../../demo/src/main.cpp" 9 117 */
#endif /* False "../../demo/src/main.cpp" 11 124 */
-------------------- END Conditional Compilation Graph --------------------

-d F

Files encountered and how many times processed:

------------------------ Count of files encountered -----------------------
   1  ../../demo/src/main.cpp
   1  ../../demo/sys/system.h
   1  ../../demo/usr/user.h
---------------------- END Count of files encountered ---------------------

-d I

The include graph:

------------------------------ Include Graph ------------------------------
../../demo/src/main.cpp [43, 21]:  True "" ""
000002: #include ../../demo/usr/user.h
        ../../demo/usr/user.h [10, 6]:  True "" "['"user.h"', 'CP=None', 'usr=../../demo/usr/']"
        000004: #include ../../demo/sys/system.h
                ../../demo/sys/system.h [10, 6]:  True "!def __USER_H__" "['<system.h>', 'sys=../../demo/sys/']"
---------------------------- END Include Graph ----------------------------

-d M

The macro environment and history:

---------------------- Macro Environment and History ----------------------
Macro Environment:
#define FRENCH /* ../../demo/usr/user.h#5 Ref: 1 True */
#define LANG_SUPPORT /* ../../demo/sys/system.h#4 Ref: 2 True */
#define __SYSTEM_H__ /* ../../demo/sys/system.h#2 Ref: 0 True */
#define __USER_H__ /* ../../demo/usr/user.h#2 Ref: 0 True */

Macro History (referenced macros only):
In scope:
#define FRENCH /* ../../demo/usr/user.h#5 Ref: 1 True */
    ../../demo/src/main.cpp 5 38
#define LANG_SUPPORT /* ../../demo/sys/system.h#4 Ref: 2 True */
    ../../demo/src/main.cpp 5 13
    ../../demo/src/main.cpp 7 15
-------------------- END Macro Environment and History --------------------

-d T

The token count:

------------------------------- Token count -------------------------------
       0  header-name
       8  identifier
       1  pp-number
       0  character-literal
       1  string-literal
      11  preprocessing-op-or-punc
       0  non-whitespace
      11  whitespace
       0  concat
      32  TOTAL
----------------------------- END Token count -----------------------------