PpDefine

This handles definition, undefinition, redefintion, replacement and rescaning of macro declarations

It implements: ISO/IEC 9899:1999(E) section 6 (aka ‘C99’) and/or: ISO/IEC 14882:1998(E) section 16 (aka ‘C++98’)

exception cpip.core.PpDefine.ExceptionCpipDefine

Exception when handling PpDefine object.

exception cpip.core.PpDefine.ExceptionCpipDefineBadArguments

Exception when scanning an argument list for a function style macro fails. NOTE: This is only raised during replacement not during initialisation.

exception cpip.core.PpDefine.ExceptionCpipDefineBadWs

Exception when calling bad whitespace is in a define statement. See: ISO/IEC 9899:1999(E) Section 6.10-f and ISO/IEC 14882:1998(E) 16-2

exception cpip.core.PpDefine.ExceptionCpipDefineDupeId

Exception for a function-like macro has duplicates in the identifier-list.

exception cpip.core.PpDefine.ExceptionCpipDefineInit

Exception when creating PpDefine object fails.

exception cpip.core.PpDefine.ExceptionCpipDefineInitBadLine

Exception for a bad line number given as argument.

exception cpip.core.PpDefine.ExceptionCpipDefineInvalidCmp

Exception for a redefinition where the identifers are different.

exception cpip.core.PpDefine.ExceptionCpipDefineMissingWs

Exception when calling missing ws between identifier and replacement tokens.

See: ISO/IEC 9899:1999(E) Section 6.10.3-3 and ISO/IEC 14882:1998(E) Section ???

Note: cpp says for “#define PLUS+”: src.h:1:13: warning: ISO C requires whitespace after the macro name

exception cpip.core.PpDefine.ExceptionCpipDefineReplace

Exception when replacing a macro definition fails.

class cpip.core.PpDefine.PpDefine(theTokGen, theFileId, theLine)

Represents a single #define directive and performs ISO/IEC 9899:1999 (E) 6.10.3 Macro replacement.

theTokGen is a PpToken generator that is expected to generate pp-tokens that appear after the start of the #define directive from the first non-whitespace token onwards i.e. the __init__ will, itself, consume leading whitespace.

theFileId is a string that represents the file ID.

theLine is a positive integer that represents the line in theFile that the #define statement occurred.

Definition example, object-like macros:

[identifier, [replacement-list (opt)], new-line, ...]

Or function-like macros:

[
    identifier,
    lparen,
    [identifier-list(opt),],
    ')',
    replacement-list,
    new-line,
    ...
]

NOTE: No whitespace is allowed between the identifier and the lparen of function-like macros.

The identifier-list of parameters is stored as a list of names. The replacement-list is stored as a list of preprocessor tokens. Leading and trailing whitespace in the replacement list is removed to facilitate redefinition comparison.

CPP_CONCAT_OP = '##'

C standard definition of concatenation operator

CPP_STRINGIZE_OP = '#'

C standard definition of string’izing operator

IDENTIFIER_SEPERATOR = ','

C standard definition of identifier separator in function-like macros

INITIAL_REF_COUNT = 0

This is what the reference count is set to on construction

LPAREN = '('

C standard definition of left parenthesis

PLACEMARKER = None

Our representation of a placemarker token

RPAREN = ')'

C standard definition of right parenthesis

STRINGIZE_WHITESPACE_CHAR = ' '

Whitespace runs are replaced by a single space ISO/IEC 9899:1999 (E) 6.10.3.2-2

VARIABLE_ARGUMENT_IDENTIFIER = '...'

Variable argument (variadic) macro definitions

VARIABLE_ARGUMENT_SUBSTITUTE = '__VA_ARGS__'

Variable argument (variadic) macro substitution

assertReplListIntegrity()

Tests that any identifier tokens in the replacement list are actually replaceable. This will raise an assertion failure if not. It is really an integrity tests to see if an external entity has grabbed a reference to the replacement list and set a token to be not replaceable.

consumeFunctionPreamble(theGen)

This consumes tokens to the preamble of a Function style macro invocation. This really means consuming whitespace and the opening LPAREN.

This will return either:

• None - Tokens including the leading LPAREN have been consumed.
• List of (token, token_type) if the LPAREN is not found.

For example given this:

#define t(a) a+2
t   (21) - t  ;

For the first t this would consume '   (' and return None leaving the next token to be (‘21’, ‘pp-number’).

For the second t this would consume '  ;' and return:

[
    ('  ', 'whitespace'),
    (';',   'preprocessing-op-or-punc'),
]

This allows the MacroReplacementEnv to generate the correct result:

21 +2 - t ;

expandArguments

The flag that says whether arguments should be expanded. For object like macros this will be False. For function like macros this will be False if there is a stringize (‘#’) or a token pasting operator (‘##’). True otherwise.

fileId

The file ID given as an argument in the constructor.

identifier

The macro identifier i.e. the name as a string.

incRefCount(theFileLineCol=None)

Increment the reference count. Typically callers do this when replacement is certain of in the event of definition testing e.g. #ifdef SPAM or defined(SPAM) etc. Or if the macro is expanded e.g. #define SPAM_N_EGGS spam and eggs The menu is SPAM_N_EGGS. theFileLineCol is a FileLocation.FileLineCol object.

isCurrentlyDefined

Returns True if the current instance is a valid definition i.e. it has not been #undef’d.

isObjectTypeMacro

True if this is an object type macro and False if it is a function type macro.

isReferenced

Returns True if the reference count has been incremented since construction.

isSame(other)

Tests ‘sameness’. Returns: -1 if the identifiers are different. 1 if the identifiers are the same but redefinition is NOT allowed. 0 if the identifiers are the same but redefinition is allowed i.e. the macros are equivelent.

isValidRefefinition(other)

Returns True if this is a valid redefinition of other, False otherwise. Will raise an ExceptionCpipDefineInvalidCmp if the identifiers are different. Will raise an ExceptionCpipDefine if either is not currently defined.

From: ISO/IEC 9899:1999 (E) 6.10.3:

1. Two replacement lists are identical if and only if the preprocessing

   tokens in both have the same number, ordering, spelling, and white-space separation, where all white-space separations are considered identical.

2. An identifier currently defined as a macro without use of lparen

   (an object-like macro) may be redefined by another #define preprocessing directive provided that the second definition is an object-like macro definition and the two replacement lists are identical, otherwise the program is ill-formed.

3. An identifier currently defined as a macro using lparen (a

   function-like macro) may be redefined by another #define preprocessing directive provided that the second definition is a function-like macro definition that has the same number and spelling of parameters, and the two replacement lists are identical, otherwise the program is ill-formed.

See also: ISO/IEC 14882:1998(E) 16.3 Macro replacement [cpp.replace]

line

The line number given as an argument in the constructor.

parameters

The list of parameter names as strings for a function like macros or None if this is an object type Macro.

refCount

Returns the current reference count as an integer less its initial value on construction.

refFileLineColS

Returns the list of FileLineCol objects where this macro was referenced.

replaceArgumentList(theArgList)

Given an list of arguments this does argument substitution and returns the replacement token list. The argument list is of the form given by retArgumentListTokens(). The caller must have replaced any macro invocations in theArgList before calling this method. NOTE: For function style macros only.

replaceObjectStyleMacro()

Returns a list of [(token, token_type), ...] from the replacement of an object style macro.

replacementTokens

The list of zero or more replacement token i.e. [class PpToken, ...].

replacements

The list of zero or more replacement tokens as strings.

retArgumentListTokens(theGen)

For a function macro this reads the tokens following a LPAREN and returns a list of arguments where each argument is a list of PpToken objects.

Thus this function returns a list of lists of PpToken objects, for example given this:

#define f(x,y) ...
f(a,b)

This function, then passed “a,b)” returns:

[
    [
        PpToken.PpToken('a', 'identifier'),
    ],
    [
        PpToken.PpToken('b', 'identifier'),
    ],
]

And an invocation of: f(1(,)2,3) i.e. this gets passed via the generator "1(,)2,3)" and returns two argunments:

[
    [
        PpToken('1', 'pp-number'),
        PpToken('(', 'preprocessing-op-or-punc'),
        PpToken(',', 'preprocessing-op-or-punc'),
        PpToken(')', 'preprocessing-op-or-punc'),
        PpToken('2', 'pp-number'),
    ],
    [
        PpToken('3', 'pp-number'),
    ],
]

So this function supports two cases:

1. Parsing function style macro declarations.
2. Interpreting function style macro invocations where the argument list is subject to replacement before invoking the macro.

In the case that an argument is missing a PpDefine.PLACEMARKER token is inserted. For example:

#define FUNCTION_STYLE(a,b,c) ...
FUNCTION_STYLE(,2,3)

Gives:

[
    PpDefine.PLACEMARKER,
    [
        PpToken.PpToken('2',       'pp-number'),
    ],
    [
        PpToken.PpToken('3',       'pp-number'),
    ],
]

Placemarker tokens are not used if the macro is defined with no arguments. This might raise a ExceptionCpipDefineBadArguments if the list does not match the prototype or a StopIteration if the token list is too short. This ignores leading and trailing whitespace for each argument.

TODO: Raise an ExceptionCpipDefineBadArguments if there is a #define statement. e.g.:

#define f(x) x x
f (1
#undef f
#define f 2
f)

strIdentPlusParam()

Returns the identifier name and parameters if a function-like macro as a string.

strReplacements()

Returns the replacements tokens with minimised whitespace as a string.

tokenCounter

The PpTokenCount object that counts tokens that have been consumed from the input.

tokensConsumed

The total number of tokens consumed by the class.

undef(theFileId, theLineNum)

Records this instance of a macro #undef’d at a particular file and line number. May raise an ExceptionCpipDefine if already undefined of the line number is bad.

undefFileId

The file ID where this macro was undef’d or None.

undefLine

The line number where this macro was undef’d or None.