.. moduleauthor:: Paul Ross <cpipdev@googlemail.com>
.. sectionauthor:: Paul Ross <cpipdev@googlemail.com>
.. Description of CPIP command line tools
.. _cpip.cmd_line_tools:
###################################
Command Line Tools
###################################
CPIP has a number of tools run from the command line that can analyse source code. The main one is :file:`CPIPMain.py`
***********************************
CPIPMain
***********************************
:file:`CPIPMain.py` acts very much like a normal pre-processor but, instead of writing out a Translation Unit as test it emits a host of HTML and SVG pages about each file to be pre-processed. Here are :ref:`cpip-examples-real`.
Usage
=======================
::
usage: CPIPMain.py [-h] [-c] [-d DUMP] [-g GLOB] [--heap] [-j JOBS] [-k]
[-l LOGLEVEL] [-o OUTPUT] [-p] [-r] [-t] [-G]
[-S PREDEFINES] [-C] [-D DEFINES] [-P PREINC] [-I INCUSR]
[-J INCSYS]
path
cpip.cpp -- Preprocess the file or the files in the directory.
Created by Paul Ross on 2011-07-10.
Copyright 2008-2015. All rights reserved.
Licensed under GPL 2.0
USAGE
positional arguments:
path Path to source file.
optional arguments:
-h, --help show this help message and exit
-c Add conditionally included files to the plots.
[default: False]
-d DUMP, --dump DUMP Dump output, additive. Can be: C - Conditional
compilation graph. F - File names encountered and
their count. I - Include graph. M - Macro environment.
T - Token count. R - Macro dependencies as an input to
DOT. [default: []]
-g GLOB, --glob GLOB Pattern match to use when processing directories.
[default: *.*]
--heap Profile memory usage. [default: False]
-j JOBS, --jobs JOBS Max simultaneous processes when pre-processing
directories. Zero uses number of native CPUs [4]. 1
means no multiprocessing. [default: 0]
-k, --keep-going Keep going. [default: False]
-l LOGLEVEL, --loglevel LOGLEVEL
Log Level (debug=10, info=20, warning=30, error=40,
critical=50) [default: 30]
-o OUTPUT, --output OUTPUT
Output directory. [default: out]
-p Ignore pragma statements. [default: False]
-r, --recursive Recursively process directories. [default: False]
-t, --dot Write an DOT include dependency table and execute DOT
on it to create a SVG file. [default: False]
-G Support GCC extensions. Currently only #include_next.
[default: False]
-S PREDEFINES, --predefine PREDEFINES
Add standard predefined macro definitions of the form
name<=definition>. They are introduced into the
environment before anything else. They can not be
redefined. __DATE__ and __TIME__ will be automatically
allocated in here. __FILE__ and __LINE__ are defined
dynamically. See ISO/IEC 9899:1999 (E) 6.10.8
Predefined macro names. [default: []]
-C, --CPP Sys call 'cpp -dM' to extract and use platform
specific macros. These are inserted after -S option
and before the -D option. [default: False]
-D DEFINES, --define DEFINES
Add macro definitions of the form name<=definition>.
These are introduced into the environment before any
pre-include. [default: []]
-P PREINC, --pre PREINC
Add pre-include file path, this file precedes the
initial translation unit. [default: []]
-I INCUSR, --usr INCUSR
Add user include search path. [default: []]
-J INCSYS, --sys INCSYS
Add system include search path. [default: []]
.. note::
Multiprocessing: The pre-processor, and information derived from it, can only be run as a single process but writing individual source files can take advantage of multiple processes. As the latter constitutes the bulk of the time :file:`CPIPMain.py` takes then using the ``-j`` option on multi-processor machines can save a lot of time.
Options
----------------
+--------------------------------------+---------------------------------------------------------------------------------+
| Option | Description |
+======================================+=================================================================================+
| ``--version`` | Show program's version number and exit |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-h, --help`` | Show this help message and exit. |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-c`` | Even if a file is conditionally included then add it to the plot. |
| | This is experimental so use it at your own risk! [default False] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-d DUMP, --dump=DUMP`` | Dump various outputs to stdout (see below). [default: []] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-g GLOB, --glob=GLOB`` | Pattern to use when searching directories (ignored for #includes. [default: *.*]|
+--------------------------------------+---------------------------------------------------------------------------------+
| ``--heap`` | Profile memory usage (requires ``guppy`` to be installed). [default: False] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-j JOBS, --jobs=JOBS`` | Max processes when multiprocessing. Zero uses number of native CPUs [8]. |
| | -1 disables multiprocessing. [default: -1] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-k`` | Keep going as far as sensible. [default: False] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-l LOGLEVEL, --loglevel=LOGLEVEL`` | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 30] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-o OUTPUT, --output=OUTPUT`` | Output directory [default: "out"] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-p`` | Ignore pragma statements. [default: False] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-r`` | Recursively provesses directories. [default: False] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-t, --dot`` | Write an DOT include dependency table and execute DOT on it to create a SVG |
| | file. [default: False] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-C , --CPP`` | Sys call ``cpp -dM`` to extract and use platform specific macros. These are |
| | inserted after -S option and before the -D option. [default: False] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-G`` | Support GCC extensions. Currently only #include_next. [default: False] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-I INCUSR, --usr=INCUSR`` | Add user include search path (additive). [default: []] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-J INCSYS, --sys=INCSYS`` | Add system include search path (additive). [default: []] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-S PREDEFINES,`` | Add standard predefined macro defintions of the form ``name<=defintion>``. |
| ``--predefine=PREDEFINES`` | These are introduced into the environment before anything else. These macros |
| | can not be redefined. ``__DATE__`` and ``__TIME__`` will be automatically |
| | defined. [default: []] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-D DEFINES, --define=DEFINES`` | Add macro definitions of the form name<=definition>. These are introduced into |
| | the environment before any pre-include. [default: []] |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-P PREINC, --pre=PREINC`` | Add pre-include file, this will be included before any header (additive). |
| | [default: []] |
+--------------------------------------+---------------------------------------------------------------------------------+
The `-d` option can be repeated to generate multiple text outputs on stdout:
+--------------------------------------+---------------------------------------------------------------------------------+
| Output | Description |
+======================================+=================================================================================+
| ``-d C`` | Conditional compilation graph. |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-d F`` | File names encountered and their count. |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-d I`` | Include graph. |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-d M`` | Macro environment. |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-d T`` | Token count. |
+--------------------------------------+---------------------------------------------------------------------------------+
| ``-d R`` | Macro dependencies as an input to DOT. |
+--------------------------------------+---------------------------------------------------------------------------------+
Examples of these are shown below :ref:`cmdline.CPIPMain.dump`.
Arguments
----------------
One or more paths of file(s) to be preprocessed.
Examples
=======================
Here is a simple example of processing the demo code that is in the ``PpLexer`` tutorial here: :ref:`pplexer.tutorial.files`.
Here we set:
* ``l 20`` sets logging to ``INFO``
* ``-o`` sets the output to :file:`../../demo/output_00/`
* ``-C`` is used to get the platform specific macros.
* ``-J`` is used to set a single system include as :file:`../../demo/sys/`
* ``-I`` is used to set a single user include as :file:`../../demo/usr/`
We are processing :file:`../../demo/src/main.cpp` and stdout is something like this::
$ python3 CPIPMain.py -l 20 -C -o ../../demo/output_00/ -J ../../demo/sys/ -I ../../demo/usr/ ../../demo/src/main.cpp
2012-03-20 07:41:38,655 INFO TU in HTML:
2012-03-20 07:41:38,655 INFO ../../demo/output_00/main.cpp.html
2012-03-20 07:41:38,664 INFO Processing TU done.
2012-03-20 07:41:38,665 INFO Macro history to:
2012-03-20 07:41:38,665 INFO ../../demo/output_00/main.cpp_macros.html
2012-03-20 07:41:38,668 INFO Include graph (SVG) to:
2012-03-20 07:41:38,668 INFO ../../demo/output_00/main.cpp.include.svg
2012-03-20 07:41:38,679 INFO Writing include graph (TEXT) to:
2012-03-20 07:41:38,679 INFO ../../demo/output_00/main.cpp.include.svg
2012-03-20 07:41:38,679 INFO Writing include graph (DOT) to:
2012-03-20 07:41:38,679 INFO ../../demo/output_00/main.cpp.include.svg
2012-03-20 07:41:38,679 INFO Creating include Graph for DOT...
2012-03-20 07:41:38,692 INFO dot returned 0
2012-03-20 07:41:38,693 INFO Creating include Graph for DOT done.
2012-03-20 07:41:38,693 INFO Conditional compilation graph in HTML:
2012-03-20 07:41:38,693 INFO ../../demo/output_00/main.cpp.ccg.html
2012-03-20 07:41:38,698 INFO Done: ../../demo/src/main.cpp
2012-03-20 07:41:38,698 INFO ITU in HTML: ...\main.cpp
2012-03-20 07:41:38,708 INFO ITU in HTML: ...\system.h
2012-03-20 07:41:38,711 INFO ITU in HTML: ...\user.h
2012-03-20 07:41:38,716 INFO All done.
CPU time = 0.051 (S)
Bye, bye!
In the output directory will be the HTML and SVG results.
.. _cmdline.CPIPMain.dump:
Using ``-d`` Option
------------------------
All these are using the following command where ``?`` is replace with a letter::
$ python3 CPIPMain.py -d? -o ../../demo/output_00/ -J ../../demo/sys/ -I ../../demo/usr/ ../../demo/src/main.cpp
Multiple outputs are obtained with, for example, ``-dC -dF``
``-d`` C
^^^^^^^^^^^^^^^^^^^^^^
Conditional compilation graph::
---------------------- Conditional Compilation Graph ----------------------
#ifndef __USER_H__ /* True "../../demo/usr/user.h" 1 0 */
#ifndef __SYSTEM_H__ /* True "../../demo/sys/system.h" 1 4 */
#endif /* True "../../demo/sys/system.h" 6 13 */
#endif /* True "../../demo/usr/user.h" 7 20 */
#if defined(LANG_SUPPORT) && defined(FRENCH) /* True "../../demo/src/main.cpp" 5 69 */
#elif defined(LANG_SUPPORT) && defined(AUSTRALIAN) /* False "../../demo/src/main.cpp" 7 110 */
#else /* False "../../demo/src/main.cpp" 9 117 */
#endif /* False "../../demo/src/main.cpp" 11 124 */
-------------------- END Conditional Compilation Graph --------------------
``-d`` F
^^^^^^^^^^^^^^^^^^^^^^
Files encountered and how many times processed::
------------------------ Count of files encountered -----------------------
1 ../../demo/src/main.cpp
1 ../../demo/sys/system.h
1 ../../demo/usr/user.h
---------------------- END Count of files encountered ---------------------
``-d`` I
^^^^^^^^^^^^^^^^^^^^^^
The include graph::
------------------------------ Include Graph ------------------------------
../../demo/src/main.cpp [43, 21]: True "" ""
000002: #include ../../demo/usr/user.h
../../demo/usr/user.h [10, 6]: True "" "['"user.h"', 'CP=None', 'usr=../../demo/usr/']"
000004: #include ../../demo/sys/system.h
../../demo/sys/system.h [10, 6]: True "!def __USER_H__" "['<system.h>', 'sys=../../demo/sys/']"
---------------------------- END Include Graph ----------------------------
``-d`` M
^^^^^^^^^^^^^^^^^^^^^^
The macro environment and history::
---------------------- Macro Environment and History ----------------------
Macro Environment:
#define FRENCH /* ../../demo/usr/user.h#5 Ref: 1 True */
#define LANG_SUPPORT /* ../../demo/sys/system.h#4 Ref: 2 True */
#define __SYSTEM_H__ /* ../../demo/sys/system.h#2 Ref: 0 True */
#define __USER_H__ /* ../../demo/usr/user.h#2 Ref: 0 True */
Macro History (referenced macros only):
In scope:
#define FRENCH /* ../../demo/usr/user.h#5 Ref: 1 True */
../../demo/src/main.cpp 5 38
#define LANG_SUPPORT /* ../../demo/sys/system.h#4 Ref: 2 True */
../../demo/src/main.cpp 5 13
../../demo/src/main.cpp 7 15
-------------------- END Macro Environment and History --------------------
``-d`` T
^^^^^^^^^^^^^^^^^^^^^^
The token count::
------------------------------- Token count -------------------------------
0 header-name
8 identifier
1 pp-number
0 character-literal
1 string-literal
11 preprocessing-op-or-punc
0 non-whitespace
11 whitespace
0 concat
32 TOTAL
----------------------------- END Token count -----------------------------