Boost C++ Libraries

First, let's cover some terminology that we'll be using throughout the docs:

A semantic action is an arbitrary bit of logic associated with a parser, that is only executed when the parser matches.

Simpler parsers can be combined to form more complex parsers. Given some combining operation C, and parsers P0, P1, ... PN, C(P0, P1, ... PN) creates a new parser Q. This creates a parse tree. Q is the parent of P1, P2 is the child of Q, etc. The parsers are applied in the top-down fashion implied by this topology. When you use Q to parse a string, it will use P0, P1, etc. to do the actual work. If P3 is being used to parse the input, that means that Q is as well, since the way Q parses is by dispatching to its children to do some or all of the work. At any point in the parse, there will be exactly one parser without children that is being used to parse the input; all other parsers being used are its ancestors in the parse tree.

A subparser is a parser that is the child of another parser.

The top-level parser is the root of the tree of parsers.

The current parser or bottommost parser is the parser with no children that is currently being used to parse the input.

A rule is a kind of parser that makes building large, complex parsers easier. A subrule is a rule that is the child of some other rule. The current rule or bottommost rule is the one rule currently being used to parse the input that has no subrules. Note that while there is always exactly one current parser, there may or may not be a current rule — rules are one kind of parser, and you may or may not be using one at a given point in the parse.

The top-level parse is the parse operation being performed by the top-level parser. This term is necessary because, though most parse failures are local to a particular parser, some parse failures cause the call to parse() to indicate failure of the entire parse. For these cases, we say that such a local failure "causes the top-level parse to fail".

Throughout the Boost.Parser documentation, I will refer to "the call to parse()". Read this as "the call to any one of the functions described in The parse() API". That includes prefix_parse(), callback_parse(), and callback_prefix_parse().

There are some special kinds of parsers that come up often in this documentation.

One is a sequence parser; you will see it created using operator>>, as in p1 >> p2 >> p3. A sequence parser tries to match all of its subparsers to the input, one at a time, in order. It matches the input iff all its subparsers do.

Another is an alternative parser; you will see it created using operator|, as in p1 | p2 | p3. An alternative parser tries to match all of its subparsers to the input, one at a time, in order; it stops after matching at most one subparser. It matches the input iff one of its subparsers does.

Finally, there is a permutation parser; it is created using operator||, as in p1 || p2 || p3. A permutation parser tries to match all of its subparsers to the input, in any order. So the parser p1 || p2 || p3 is equivalent to (p1 >> p2 >> p3) | (p1 >> p3 >> p2) | (p2 >> p1 >> p3) | (p2 >> p3 >> p1) | (p3 >> p1 >> p2) | (p3 >> p2 >> p1). Hopefully its terseness is self-explanatory. It matches the input iff all of its subparsers do, regardless of the order they match in.

Boost.Parser parsers each have an attribute associated with them, or explicitly have no attribute. An attribute is a value that the parser generates when it matches the input. For instance, the parser double_ generates a double when it matches the input. ATTR() is a notional macro that expands to the attribute type of the parser passed to it; ATTR(double_) is double. This is similar to the attribute type trait.

Next, we'll look at some simple programs that parse using Boost.Parser. We'll start small and build up from there.

This is just about the most minimal example of using Boost.Parser that one could write. We take a string from the command line, or "World" if none is given, and then we parse it:

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


namespace bp = boost::parser;

int main(int argc, char const * argv[])
{
    std::string input = "World";
    if (1 < argc)
        input = argv[1];

    std::string result;
    bp::parse(input, *bp::char_, result);
    std::cout << "Hello, " << result << "!\n";
}

The expression *bp::char_ is a parser-expression. It uses one of the many parsers that Boost.Parser provides: char_. Like all Boost.Parser parsers, it has certain operations defined on it. In this case, *bp::char_ is using an overloaded operator* as the C++ version of a Kleene star operator. Since C++ has no postfix unary * operator, we have to use the one we have, so it is used as a prefix.

So, *bp::char_ means "any number of characters". In other words, it really cannot fail. Even an empty string will match it.

The parse operation is performed by calling the parse() function, passing the parser as one of the arguments:

bp::parse(input, *bp::char_, result);

The arguments here are: input, the range to parse; *bp::char_, the parser used to do the parse; and result, an out-parameter into which to put the result of the parse. Don't get too caught up on this method of getting the parse result out of parse(); there are multiple ways of doing so, and we'll cover all of them in subsequent sections.

Also, just ignore for now the fact that Boost.Parser somehow figured out that the result type of the *bp::char_ parser is a std::string. There are clear rules for this that we'll cover later.

The effects of this call to parse() is not very interesting — since the parser we gave it cannot ever fail, and because we're placing the output in the same type as the input, it just copies the contents of input to result.

Let's look at a slightly more complicated example, even if it is still trivial. Instead of taking any old chars we're given, let's require some structure. Let's parse one or more doubles, separated by commas.

The Boost.Parser parser for double is double_. So, to parse a single double, we'd just use that. If we wanted to parse two doubles in a row, we'd use:

boost::parser::double_ >> boost::parser::double_

operator>> in this expression is the sequence-operator; read it as "followed by". If we combine the sequence-operator with Kleene star, we can get the parser we want by writing:

boost::parser::double_ >> *(',' >> boost::parser::double_)

This is a parser that matches at least one double — because of the first double_ in the expression above — followed by zero or more instances of a-comma-followed-by-a-double. Notice that we can use ',' directly. Though it is not a parser, operator>> and the other operators defined on Boost.Parser parsers have overloads that accept character/parser pairs of arguments; these operator overloads will create the right parser to recognize ','.

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


namespace bp = boost::parser;

int main()
{
    std::cout << "Enter a list of doubles, separated by commas.  No pressure. ";
    std::string input;
    std::getline(std::cin, input);

    auto const result = bp::parse(input, bp::double_ >> *(',' >> bp::double_));

    if (result) {
        std::cout << "Great! It looks like you entered:\n";
        for (double x : *result) {
            std::cout << x << "\n";
        }
    } else {
        std::cout
            << "Good job!  Please proceed to the recovery annex for cake.\n";
    }
}

The first example filled in an out-parameter to deliver the result of the parse. This call to parse() returns a result instead. As you can see, the result is contextually convertible to bool, and *result is some sort of range. In fact, the return type of this call to parse() is std::optional<std::vector<double>>. Naturally, if the parse fails, std::nullopt is returned. We'll look at how Boost.Parser maps the type of the parser to the return type, or the filled in out-parameter's type, a bit later.

If I run it in a shell, this is the result:

$ example/trivial
Enter a list of doubles, separated by commas.  No pressure. 5.6,8.9
Great! It looks like you entered:
5.6
8.9
$ example/trivial
Enter a list of doubles, separated by commas.  No pressure. 5.6, 8.9
Good job!  Please proceed to the recovery annex for cake.

It does not recognize "5.6, 8.9". This is because it expects a comma followed immediately by a double, but I inserted a space after the comma. The same failure to parse would occur if I put a space before the comma, or before or after the list of doubles.

One more thing: there is a much better way to write the parser above. Instead of repeating the double_ subparser, we could have written this:

bp::double_ % ','

That's semantically identical to bp::double_ >> *(',' >> bp::double_). This pattern — some bit of input repeated one or more times, with a separator between each instance — comes up so often that there's an operator specifically for that, operator%. We'll be using that operator from now on.

Let's modify the trivial parser we just saw to ignore any spaces it might find among the doubles and commas. To skip whitespace wherever we find it, we can pass a skip parser to our call to parse() (we don't need to touch the parser passed to parse()). Here, we use ws, which matches any Unicode whitespace character.

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


namespace bp = boost::parser;

int main()
{
    std::cout << "Enter a list of doubles, separated by commas.  No pressure. ";
    std::string input;
    std::getline(std::cin, input);

    auto const result = bp::parse(input, bp::double_ % ',', bp::ws);

    if (result) {
        std::cout << "Great! It looks like you entered:\n";
        for (double x : *result) {
            std::cout << x << "\n";
        }
    } else {
        std::cout
            << "Good job!  Please proceed to the recovery annex for cake.\n";
    }
}

The skip parser, or skipper, is run between the subparsers within the parser passed to parse(). In this case, the skipper is run before the first double is parsed, before any subsequent comma or double is parsed, and at the end. So, the strings "3.6,5.9" and " 3.6 , \t 5.9 " are parsed the same by this program.

Skipping is an important concept in Boost.Parser. You can skip anything, not just whitespace; there are lots of other things you might want to skip. The skipper you pass to parse() can be an arbitrary parser. For example, if you write a parser for a scripting language, you can write a skipper to skip whitespace, inline comments, and end-of-line comments.

We'll be using skip parsers almost exclusively in the rest of the documentation. The ability to ignore the parts of your input that you don't care about is so convenient that parsing without skipping is a rarity in practice.

Like all parsing systems (lex & yacc, Boost.Spirit, etc.), Boost.Parser has a mechanism for associating semantic actions with different parts of the parse. Here is nearly the same program as we saw in the previous example, except that it is implemented in terms of a semantic action that appends each parsed double to a result, instead of automatically building and returning the result. To do this, we replace the double_ from the previous example with double_[action]; action is our semantic action:

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


namespace bp = boost::parser;

int main()
{
    std::cout << "Enter a list of doubles, separated by commas. ";
    std::string input;
    std::getline(std::cin, input);

    std::vector<double> result;
    auto const action = [&result](auto & ctx) {
        std::cout << "Got one!\n";
        result.push_back(_attr(ctx));
    };
    auto const action_parser = bp::double_[action];
    auto const success = bp::parse(input, action_parser % ',', bp::ws);

    if (success) {
        std::cout << "You entered:\n";
        for (double x : result) {
            std::cout << x << "\n";
        }
    } else {
        std::cout << "Parse failure.\n";
    }
}

Run in a shell, it looks like this:

$ example/semantic_actions
Enter a list of doubles, separated by commas. 4,3
Got one!
Got one!
You entered:
4
3

In Boost.Parser, semantic actions are implemented in terms of invocable objects that take a single parameter to a parse-context object. The parse-context object represents the current state of the parse. In the example we used this lambda as our invocable:

auto const action = [&result](auto & ctx) {
    std::cout << "Got one!\n";
    result.push_back(_attr(ctx));
};

We're both printing a message to std::cout and recording a parsed result in the lambda. It could do both, either, or neither of these things if you like. The way we get the parsed double in the lambda is by asking the parse context for it. _attr(ctx) is how you ask the parse context for the attribute produced by the parser to which the semantic action is attached. There are lots of functions like _attr() that can be used to access the state in the parse context. We'll cover more of them later on. The Parse Context defines what exactly the parse context is and how it works.

Note that you can't write an unadorned lambda directly as a semantic action. Otherwise, the compile will see two '[' characters and think it's about to parse an attribute. Parentheses fix this:

p[([](auto & ctx){/*...*/})]

Before you do this, note that the lambdas that you write as semantic actions are almost always generic (having an auto & ctx parameter), and so are very frequently re-usable. Most semantic action lambdas you write should be written out-of-line, and given a good name. Even when they are not reused, named lambdas keep your parsers smaller and easier to read.

Semantic actions inside rules

There are some other forms for semantic actions, when they are used inside of rules. See More About Rules for details.

So far we've seen examples that parse some text and generate associated attributes. Sometimes, you want to find some subrange of the input that contains what you're looking for, and you don't want to generate attributes at all.

There are two directives that affect the attribute type of any parser, raw[] and string_view[]. (We'll get to directives in more detail in the Directives section later. For now, you just need to know that a directive wraps a parser, and changes some aspect of how it functions.)

raw[]

raw[] changes the attribute of its parser to be a subrange whose begin() and end() return the bounds of the sequence being parsed that match p.

namespace bp = boost::parser;
auto int_parser = bp::int_ % ',';            // ATTR(int_parser) is std::vector<int>
auto subrange_parser = bp::raw[int_parser];  // ATTR(subrange_parser) is a subrange

// Parse using int_parser, generating integers.
auto ints = bp::parse("1, 2, 3, 4", int_parser, bp::ws);
assert(ints);
assert(*ints == std::vector<int>({1, 2, 3, 4}));

// Parse again using int_parser, but this time generating only the
// subrange matched by int_parser.  (prefix_parse() allows matches that
// don't consume the entire input.)
auto const str = std::string("1, 2, 3, 4, a, b, c");
auto first = str.begin();
auto range = bp::prefix_parse(first, str.end(), subrange_parser, bp::ws);
assert(range);
assert(range->begin() == str.begin());
assert(range->end() == str.begin() + 10);

static_assert(std::is_same_v<
              decltype(range),
              std::optional<bp::subrange<std::string::const_iterator>>>);

Note that the subrange has the iterator type std::string::const_iterator, because that's the iterator type passed to prefix_parse(). If we had passed char const * iterators to prefix_parse(), that would have been the iterator type. The only exception to this comes from Unicode-aware parsing (see Unicode Support). In some of those cases, the iterator being used in the parse is not the one you passed. For instance, if you call prefix_parse() with char8_t * iterators, it will create a UTF-8 to UTF-32 transcoding view, and parse the iterators of that view. In such a case, you'll get a subrange whose iterator type is a transcoding iterator. When that happens, you can get the underlying iterator — the one you passed to prefix_parse() — by calling the .base() member function on each transcoding iterator in the returned subrange.

auto const u8str = std::u8string(u8"1, 2, 3, 4, a, b, c");
auto u8first = u8str.begin();
auto u8range = bp::prefix_parse(u8first, u8str.end(), subrange_parser, bp::ws);
assert(u8range);
assert(u8range->begin().base() == u8str.begin());
assert(u8range->end().base() == u8str.begin() + 10);

string_view[]

string_view[] has very similar semantics to raw[], except that it produces a std::basic_string_view<CharT> (where CharT is the type of the underlying range begin parsed) instead of a subrange. For this to work, the underlying range must be contiguous. Contiguity of iterators is not detectable before C++20, so this directive is only available in C++20 and later.

namespace bp = boost::parser;
auto int_parser = bp::int_ % ',';              // ATTR(int_parser) is std::vector<int>
auto sv_parser = bp::string_view[int_parser];  // ATTR(sv_parser) is a string_view

auto const str = std::string("1, 2, 3, 4, a, b, c");
auto first = str.begin();
auto sv1 = bp::prefix_parse(first, str.end(), sv_parser, bp::ws);
assert(sv1);
assert(*sv1 == str.substr(0, 10));

static_assert(std::is_same_v<decltype(sv1), std::optional<std::string_view>>);

Since string_view[] produces string_views, it cannot return transcoding iterators as described above for raw[]. If you parse a sequence of CharT with string_view[], you get exactly a std::basic_string_view<CharT>. If the parse is using transcoding in the Unicode-aware path, string_view[] will decompose the transcoding iterator as necessary. If you pass a transcoding view to parse() or transcoding iterators to prefix_parse(), string_view[] will still see through the transcoding iterators without issue, and give you a string_view of part of the underlying range.

auto sv2 = bp::parse("1, 2, 3, 4" | bp::as_utf32, sv_parser, bp::ws);
assert(sv2);
assert(*sv2 == "1, 2, 3, 4");

static_assert(std::is_same_v<decltype(sv2), std::optional<std::string_view>>);

Now would be a good time to describe the parse context in some detail. Any semantic action that you write will need to use state in the parse context, so you need to know what's available.

The parse context is an object that stores the current state of the parse — the current- and end-iterators, the error handler, etc. Data may seem to be "added" to or "removed" from it at different times during the parse. For instance, when a parser p with a semantic action a succeeds, the context adds the attribute that p produces to the parse context, then calls a, passing it the context.

Though the context object appears to have things added to or removed from it, it does not. In reality, there is no one context object. Contexts are formed at various times during the parse, usually when starting a subparser. Each context is formed by taking the previous context and adding or changing members as needed to form a new context object. When the function containing the new context object returns, its context object (if any) is destructed. This is efficient to do, because the parse context has only about a dozen data members, and each data member is less than or equal to the size of a pointer. Copying the entire context when mutating the context is therefore fast. The context does no memory allocation.

Accessors for data that are always available

By convention, the names of all Boost.Parser functions that take a parse context, and are therefore intended for use inside semantic actions, contain a leading underscore.

_pass()

_pass() returns a reference to a bool indicating the success of failure of the current parse. This can be used to force the current parse to pass or fail:

[](auto & ctx) {
    // If the attribute fails to meet this predicate, fail the parse.
    if (!necessary_condition(_attr(ctx)))
        _pass(ctx) = false;
}

Note that for a semantic action to be executed, its associated parser must already have succeeded. So unless you previously wrote _pass(ctx) = false within your action, _pass(ctx) = true does nothing; it's redundant.

_begin(), _end() and _where()

_begin() and _end() return the beginning and end of the range that you passed to parse(), respectively. _where() returns a subrange indicating the bounds of the input matched by the current parse. _where() can be useful if you just want to parse some text and return a result consisting of where certain elements are located, without producing any other attributes. _where() can also be essential in tracking where things are located, to provide good diagnostics at a later point in the parse. Think mismatched tags in XML; if you parse a close-tag at the end of an element, and it does not match the open-tag, you want to produce an error message that mentions or shows both tags. Stashing _where(ctx).begin() somewhere that is available to the close-tag parser will enable that. See Error Handling and Debugging for an example of this.

_error_handler()

_error_handler() returns a reference to the error handler associated with the parser passed to parse(). Using _error_handler(), you can generate errors and warnings from within your semantic actions. See Error Handling and Debugging for concrete examples.

Accessors for data that are only sometimes available

_attr()

_attr() returns a reference to the value of the current parser's attribute. It is available only when the current parser's parse is successful. If the parser has no semantic action, no attribute gets added to the parse context. It can be used to read and write the current parser's attribute:

[](auto & ctx) { _attr(ctx) = 3; }

If the current parser has no attribute, a none is returned.

_val()

_val() returns a reference to the value of the attribute of the current rule being used to parse (if any), and is available even before the rule's parse is successful. It can be used to set the current rule's attribute, even from a parser that is a subparser inside the rule. Let's say we're writing a parser with a semantic action that is within a rule. If we want to set the current rule's value to some function of subparser's attribute, we would write this semantic action:

[](auto & ctx) { _val(ctx) = some_function(_attr(ctx)); }

If there is no current rule, or the current rule has no attribute, a none is returned.

You need to use _val() in cases where the default attribute for a rule's parser is not directly compatible with the attribute type of the rule. In these cases, you'll need to write some code like the example above to compute the rule's attribute from the rule's parser's generated attribute. For more info on rules, see the next page, and More About Rules.

_globals()

_globals() returns a reference to a user-supplied object that contains whatever data you want to use during the parse. The "globals" for a parse is an object — typically a struct — that you give to the top-level parser. Then you can use _globals() to access it at any time during the parse. We'll see how globals get associated with the top-level parser in The parse() API later. As an example, say that you have an early part of the parse that needs to record some black-listed values, and that later parts of the parse might need to parse values, failing the parse if they see the black-listed values. In the early part of the parse, you could write something like this.

[](auto & ctx) {
    // black_list is a std::unordered_set.
    _globals(ctx).black_list.insert(_attr(ctx));
}

Later in the parse, you could then use black_list to check values as they are parsed.

[](auto & ctx) {
    if (_globals(ctx).black_list.contains(_attr(ctx)))
        _pass(ctx) = false;
}

_locals()

_locals() returns a reference to one or more values that are local to the current rule being parsed, if any. If there are two or more local values, _locals() returns a reference to a boost::parser::tuple. Rules with locals are something we haven't gotten to yet (see More About Rules), but for now all you need to know is that you can provide a template parameter (LocalState) to rule, and the rule will default construct an object of that type for use within the rule. You access it via _locals():

[](auto & ctx) {
    auto & local = _locals(ctx);
    // Use local here.  If 'local' is a hana::tuple, access its members like this:
    using namespace hana::literals;
    auto & first_element = local[0_c];
    auto & second_element = local[1_c];
}

If there is no current rule, or the current rule has no locals, a none is returned.

_params()

_params(), like _locals(), applies to the current rule being used to parse, if any (see More About Rules). It also returns a reference to a single value, if the current rule has only one parameter, or a boost::parser::tuple of multiple values if the current rule has multiple parameters. If there is no current rule, or the current rule has no parameters, a none is returned.

Unlike with _locals(), you do not provide a template parameter to rule. Instead you call the rule's with() member function (again, see More About Rules).

_no_case()

_no_case() returns true if the current parse context is inside one or more (possibly nested) no_case[] directives. I don't have a use case for this, but if I didn't expose it, it would be the only thing in the context that you could not examine from inside a semantic action. It was easy to add, so I did.

This example is very similar to the others we've seen so far. This one is different only because it uses a rule. As an analogy, think of a parser like char_ or double_ as an individual line of code, and a rule as a function. Like a function, a rule has its own name, and can even be forward declared. Here is how we define a rule, which is analogous to forward declaring a function:

bp::rule<struct doubles, std::vector<double>> doubles = "doubles";

This declares the rule itself. The rule is a parser, and we can immediately use it in other parsers. That definition is pretty dense; take note of these things:

Ok, so if doubles is a parser, what does it do? We define the rule's behavior by defining a separate parser that by now should look pretty familiar:

auto const doubles_def = bp::double_ % ',';

This is analogous to writing a definition for a forward-declared function. Note that we used the name doubles_def. Right now, the doubles rule parser and the doubles_def non-rule parser have no connection to each other. That's intentional — we want to be able to define them separately. To connect them, we declare functions with an interface that Boost.Parser understands, and use the tag type struct doubles to connect them together. We use a macro for that:

BOOST_PARSER_DEFINE_RULES(doubles);

This macro expands to the code necessary to make the rule doubles and its parser doubles_def work together. The _def suffix is a naming convention that this macro relies on to work. The tag type allows the rule parser, doubles, to call one of these overloads when used as a parser.

BOOST_PARSER_DEFINE_RULES expands to two overloads of a function called parse_rule(). In the case above, the overloads each take a struct doubles parameter (to distinguish them from the other overloads of parse_rule() for other rules) and parse using doubles_def. You will never need to call any overload of parse_rule() yourself; it is used internally by the parser that implements rules, rule_parser.

Here is the definition of the macro that is expanded for each rule:

#define BOOST_PARSER_DEFINE_IMPL(_, rule_name_)                                \
    template<                                                                  \
        typename Iter,                                                         \
        typename Sentinel,                                                     \
        typename Context,                                                      \
        typename SkipParser>                                                   \
    decltype(rule_name_)::parser_type::attr_type parse_rule(                   \
        decltype(rule_name_)::parser_type::tag_type *,                         \
        Iter & first,                                                          \
        Sentinel last,                                                         \
        Context const & context,                                               \
        SkipParser const & skip,                                               \
        boost::parser::detail::flags flags,                                    \
        bool & success,                                                        \
        bool & dont_assign)                                                    \
    {                                                                          \
        auto const & parser = BOOST_PARSER_PP_CAT(rule_name_, _def);           \
        using attr_t =                                                         \
            decltype(parser(first, last, context, skip, flags, success));      \
        using attr_type = decltype(rule_name_)::parser_type::attr_type;        \
        if constexpr (boost::parser::detail::is_nope_v<attr_t>) {              \
            dont_assign = true;                                                \
            parser(first, last, context, skip, flags, success);                \
            return {};                                                         \
        } else if constexpr (std::is_same_v<attr_type, attr_t>) {              \
            return parser(first, last, context, skip, flags, success);         \
        } else if constexpr (std::is_constructible_v<attr_type, attr_t>) {     \
            return attr_type(                                                  \
                parser(first, last, context, skip, flags, success));           \
        } else {                                                               \
            attr_type attr{};                                                  \
            parser(first, last, context, skip, flags, success, attr);          \
            return attr;                                                       \
        }                                                                      \
    }                                                                          \
                                                                               \
    template<                                                                  \
        typename Iter,                                                         \
        typename Sentinel,                                                     \
        typename Context,                                                      \
        typename SkipParser,                                                   \
        typename Attribute>                                                    \
    void parse_rule(                                                           \
        decltype(rule_name_)::parser_type::tag_type *,                         \
        Iter & first,                                                          \
        Sentinel last,                                                         \
        Context const & context,                                               \
        SkipParser const & skip,                                               \
        boost::parser::detail::flags flags,                                    \
        bool & success,                                                        \
        bool & dont_assign,                                                    \
        Attribute & retval)                                                    \
    {                                                                          \
        auto const & parser = BOOST_PARSER_PP_CAT(rule_name_, _def);           \
        using attr_t =                                                         \
            decltype(parser(first, last, context, skip, flags, success));      \
        if constexpr (boost::parser::detail::is_nope_v<attr_t>) {              \
            parser(first, last, context, skip, flags, success);                \
        } else {                                                               \
            parser(first, last, context, skip, flags, success, retval);        \
        }                                                                      \
    }

Now that we have the doubles parser, we can use it like we might any other parser:

auto const result = bp::parse(input, doubles, bp::ws);

The full program:

#include <boost/parser/parser.hpp>

#include <deque>
#include <iostream>
#include <string>


namespace bp = boost::parser;


bp::rule<struct doubles, std::vector<double>> doubles = "doubles";
auto const doubles_def = bp::double_ % ',';
BOOST_PARSER_DEFINE_RULES(doubles);

int main()
{
    std::cout << "Please enter a list of doubles, separated by commas. ";
    std::string input;
    std::getline(std::cin, input);

    auto const result = bp::parse(input, doubles, bp::ws);

    if (result) {
        std::cout << "You entered:\n";
        for (double x : *result) {
            std::cout << x << "\n";
        }
    } else {
        std::cout << "Parse failure.\n";
    }
}

All this is intended to introduce the notion of rules. It still may be a bit unclear why you would want to use rules. The use cases for, and lots of detail about, rules is in a later section, More About Rules.

So far, we've seen only simple parsers that parse the same value repeatedly (with or without commas and spaces). It's also very common to parse a few values in a specific sequence. Let's say you want to parse an employee record. Here's a parser you might write:

namespace bp = boost::parser;
auto employee_parser = bp::lit("employee")
    >> '{'
    >> bp::int_ >> ','
    >> quoted_string >> ','
    >> quoted_string >> ','
    >> bp::double_
    >> '}';

The attribute type for employee_parser is boost::parser::tuple<int, std::string, std::string, double>. That's great, in that you got all the parsed data for the record without having to write any semantic actions. It's not so great that you now have to get all the individual elements out by their indices, using get(). It would be much nicer to parse into the final data structure that your program is going to use. This is often some struct or class. Boost.Parser supports parsing into arbitrary aggregate structs, and non-aggregates that are constructible from the tuple at hand.

Aggregate types as attributes

If we have a struct that has data members of the same types listed in the boost::parser::tuple attribute type for employee_parser, it would be nice to parse directly into it, instead of parsing into a tuple and then constructing our struct later. Fortunately, this just works in Boost.Parser. Here is an example of parsing straight into a compatible aggregate type.

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


struct employee
{
    int age;
    std::string surname;
    std::string forename;
    double salary;
};

namespace bp = boost::parser;

int main()
{
    std::cout << "Enter employee record. ";
    std::string input;
    std::getline(std::cin, input);

    auto quoted_string = bp::lexeme['"' >> +(bp::char_ - '"') >> '"'];
    auto employee_p = bp::lit("employee")
        >> '{'
        >> bp::int_ >> ','
        >> quoted_string >> ','
        >> quoted_string >> ','
        >> bp::double_
        >> '}';

    employee record;
    auto const result = bp::parse(input, employee_p, bp::ws, record);

    if (result) {
        std::cout << "You entered:\nage:      " << record.age
                  << "\nsurname:  " << record.surname
                  << "\nforename: " << record.forename
                  << "\nsalary  : " << record.salary << "\n";
    } else {
        std::cout << "Parse failure.\n";
    }
}

Unfortunately, this is taking advantage of the loose attribute assignment logic; the employee_parser parser still has a boost::parser::tuple attribute. See The parse() API for a description of attribute out-param compatibility.

For this reason, it's even more common to want to make a rule that returns a specific type like employee. Just by giving the rule a struct type, we make sure that this parser always generates an employee struct as its attribute, no matter where it is in the parse. If we made a simple parser P that uses the employee_p rule, like bp::int >> employee_p, P's attribute type would be boost::parser::tuple<int, employee>.

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


struct employee
{
    int age;
    std::string surname;
    std::string forename;
    double salary;
};

namespace bp = boost::parser;

bp::rule<struct quoted_string, std::string> quoted_string = "quoted name";
bp::rule<struct employee_p, employee> employee_p = "employee";

auto quoted_string_def = bp::lexeme['"' >> +(bp::char_ - '"') >> '"'];
auto employee_p_def = bp::lit("employee")
    >> '{'
    >> bp::int_ >> ','
    >> quoted_string >> ','
    >> quoted_string >> ','
    >> bp::double_
    >> '}';

BOOST_PARSER_DEFINE_RULES(quoted_string, employee_p);

int main()
{
    std::cout << "Enter employee record. ";
    std::string input;
    std::getline(std::cin, input);

    static_assert(std::is_aggregate_v<std::decay_t<employee &>>);

    auto const result = bp::parse(input, employee_p, bp::ws);

    if (result) {
        std::cout << "You entered:\nage:      " << result->age
                  << "\nsurname:  " << result->surname
                  << "\nforename: " << result->forename
                  << "\nsalary  : " << result->salary << "\n";
    } else {
        std::cout << "Parse failure.\n";
    }
}

Just as you can pass a struct as an out-param to parse() when the parser's attribute type is a tuple, you can also pass a tuple as an out-param to parse() when the parser's attribute type is a struct:

// Using the employee_p rule from above, with attribute type employee...
boost::parser::tuple<int, std::string, std::string, double> tup;
auto const result = bp::parse(input, employee_p, bp::ws, tup); // Ok!

General class types as attributes

Many times you don't have an aggregate struct that you want to produce from your parse. It would be even nicer than the aggregate code above if Boost.Parser could detect that the members of a tuple that is produced as an attribute are usable as the arguments to some type's constructor. So, Boost.Parser does that.

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


namespace bp = boost::parser;

int main()
{
    std::cout << "Enter a string followed by two unsigned integers. ";
    std::string input;
    std::getline(std::cin, input);

    constexpr auto string_uint_uint =
        bp::lexeme[+(bp::char_ - ' ')] >> bp::uint_ >> bp::uint_;
    std::string string_from_parse;
    if (parse(input, string_uint_uint, bp::ws, string_from_parse))
        std::cout << "That yields this string: " << string_from_parse << "\n";
    else
        std::cout << "Parse failure.\n";

    std::cout << "Enter an unsigned integer followed by a string. ";
    std::getline(std::cin, input);
    std::cout << input << "\n";

    constexpr auto uint_string = bp::uint_ >> +bp::char_;
    std::vector<std::string> vector_from_parse;
    if (parse(input, uint_string, bp::ws, vector_from_parse)) {
        std::cout << "That yields this vector of strings:\n";
        for (auto && str : vector_from_parse) {
            std::cout << "  '" << str << "'\n";
        }
    } else {
        std::cout << "Parse failure.\n";
    }
}

Let's look at the first parse.

constexpr auto string_uint_uint =
    bp::lexeme[+(bp::char_ - ' ')] >> bp::uint_ >> bp::uint_;
std::string string_from_parse;
if (parse(input, string_uint_uint, bp::ws, string_from_parse))
    std::cout << "That yields this string: " << string_from_parse << "\n";
else
    std::cout << "Parse failure.\n";

Here, we use the parser string_uint_uint, which produces a boost::parser::tuple<std::string, unsigned int, unsigned int> attribute. When we try to parse that into an out-param std::string attribute, it just works. This is because std::string has a constructor that takes a std::string, an offset, and a length. Here's the other parse:

constexpr auto uint_string = bp::uint_ >> +bp::char_;
std::vector<std::string> vector_from_parse;
if (parse(input, uint_string, bp::ws, vector_from_parse)) {
    std::cout << "That yields this vector of strings:\n";
    for (auto && str : vector_from_parse) {
        std::cout << "  '" << str << "'\n";
    }
} else {
    std::cout << "Parse failure.\n";
}

Now we have the parser uint_string, which produces boost::parser::tuple<unsigned int, std::string> attribute — the two chars at the end combine into a std::string. Those two values can be used to construct a std::vector<std::string>, via the count, T constructor.

Just like with using aggregates in place of tuples, non-aggregate class types can be substituted for tuples in most places. That includes using a non-aggregate class type as the attribute type of a rule.

However, while compatible tuples can be substituted for aggregates, you can't substitute a tuple for some class type T just because the tuple could have been used to construct T. Think of trying to invert the substitution in the second parse above. Converting a std::vector<std::string> into a boost::parser::tuple<unsigned int, std::string> makes no sense.

Frequently, you need to parse something that might have one of several forms. operator| is overloaded to form alternative parsers. For example:

namespace bp = boost::parser;
auto const parser_1 = bp::int_ | bp::eps;

parser_1 matches an integer, or if that fails, it matches epsilon, the empty string. This is equivalent to writing:

namespace bp = boost::parser;
auto const parser_2 = -bp::int_;

However, neither parser_1 nor parser_2 is equivalent to writing this:

namespace bp = boost::parser;
auto const parser_3 = bp::eps | bp::int_; // Does not do what you think.

The reason is that alternative parsers try each of their subparsers, one at a time, and stop on the first one that matches. Epsilon matches anything, since it is zero length and consumes no input. It even matches the end of input. This means that parser_3 is equivalent to eps by itself.

It is very common to need to parse quoted strings. Quoted strings are slightly tricky, though, when using a skipper (and you should be using a skipper 99% of the time). You don't want to allow arbitrary whitespace in the middle of your strings, and you also don't want to remove all whitespace from your strings. Both of these things will happen with the typical skipper, ws.

So, here is how most people would write a quoted string parser:

namespace bp = boost::parser;
const auto string = bp::lexeme['"' >> *(bp::char_ - '"') > '"'];

Some things to note:

This is a very common pattern. I have written a quoted string parser like this dozens of times. The parser above is the quick-and-dirty version. A more robust version would be able to handle escaped quotes within the string, and then would immediately also need to support escaped escape characters.

Boost.Parser provides quoted_string to use in place of this very common pattern. It supports quote- and escaped-character-escaping, using backslash as the escape character.

namespace bp = boost::parser;

auto result1 = bp::parse("\"some text\"", bp::quoted_string, bp::ws);
assert(result1);
std::cout << *result1 << "\n"; // Prints: some text

auto result2 =
    bp::parse("\"some \\\"text\\\"\"", bp::quoted_string, bp::ws);
assert(result2);
std::cout << *result2 << "\n"; // Prints: some "text"

As common as this use case is, there are very similar use cases that it does not cover. So, quoted_string has some options. If you call it with a single character, it returns a quoted_string that uses that single character as the quote-character.

auto result3 = bp::parse("!some text!", bp::quoted_string('!'), bp::ws);
assert(result3);
std::cout << *result3 << "\n"; // Prints: some text

You can also supply a range of characters. One of the characters from the range must quote both ends of the string; mismatches are not allowed. Think of how Python allows you to quote a string with either '"' or '\'', but the same character must be used on both sides.

auto result4 = bp::parse("'some text'", bp::quoted_string("'\""), bp::ws);
assert(result4);
std::cout << *result4 << "\n"; // Prints: some text

Another common thing to do in a quoted string parser is to recognize escape sequences. If you have simple escape sequencecs that do not require any real parsing, like say the simple escape sequences from C++, you can provide a symbols object as well. The template parameter T to symbols<T> must be char or char32_t. You don't need to include the escaped backslash or the escaped quote character, since those always work.

// the c++ simple escapes
bp::symbols<char> const escapes = {
    {"'", '\''},
    {"?", '\?'},
    {"a", '\a'},
    {"b", '\b'},
    {"f", '\f'},
    {"n", '\n'},
    {"r", '\r'},
    {"t", '\t'},
    {"v", '\v'}};
auto result5 =
    bp::parse("\"some text\r\"", bp::quoted_string('"', escapes), bp::ws);
assert(result5);
std::cout << *result5 << "\n"; // Prints (with a CRLF newline): some text

Now that you've seen some examples, let's see how parsing works in a bit more detail. Consider this example.

namespace bp = boost::parser;
auto int_pair = bp::int_ >> bp::int_;         // Attribute: tuple<int, int>
auto int_pairs_plus = +int_pair >> bp::int_;  // Attribute: tuple<std::vector<tuple<int, int>>, int>

int_pairs_plus must match a pair of ints (using int_pair) one or more times, and then must match an additional int. In other words, it matches any odd number (greater than 1) of ints in the input. Let's look at how this parse proceeds.

auto result = bp::parse("1 2 3", int_pairs_plus, bp::ws);

At the beginning of the parse, the top level parser uses its first subparser (if any) to start parsing. So, int_pairs_plus, being a sequence parser, would pass control to its first parser +int_pair. Then +int_pair would use int_pair to do its parsing, which would in turn use bp::int_. This creates a stack of parsers, each one using a particular subparser.

Step 1) The input is "1 2 3", and the stack of active parsers is int_pairs_plus -> +int_pair -> int_pair -> bp::int_. (Read "->" as "uses".) This parses "1", and the whitespace after is skipped by bp::ws. Control passes to the second bp::int_ parser in int_pair.

Step 2) The input is "2 3" and the stack of parsers looks the same, except the active parser is the second bp::int_ from int_pair. This parser consumes "2" and then bp::ws skips the subsequent space. Since we've finished with int_pair's match, its boost::parser::tuple<int, int> attribute is complete. It's parent is +int_pair, so this tuple attribute is pushed onto the back of +int_pair's attribute, which is a std::vector<boost::parser::tuple<int, int>>. Control passes up to the parent of int_pair, +int_pair. Since +int_pair is a one-or-more parser, it starts a new iteration; control passes to int_pair again.

Step 3) The input is "3" and the stack of parsers looks the same, except the active parser is the first bp::int_ from int_pair again, and we're in the second iteration of +int_pair. This parser consumes "3". Since this is the end of the input, the second bp::int_ of int_pair does not match. This partial match of "3" should not count, since it was not part of a full match. So, int_pair indicates its failure, and +int_pair stops iterating. Since it did match once, +int_pair does not fail; it is a zero-or-more parser; failure of its subparser after the first success does not cause it to fail. Control passes to the next parser in sequence within int_pairs_plus.

Step 4) The input is "3" again, and the stack of parsers is int_pairs_plus -> bp::int_. This parses the "3", and the parse reaches the end of input. Control passes to int_pairs_plus, which has just successfully matched with all parser in its sequence. It then produces its attribute, a boost::parser::tuple<std::vector<boost::parser::tuple<int, int>>, int>, which gets returned from bp::parse().

Something to take note of between Steps #3 and #4: at the beginning of #4, the input position had returned to where is was at the beginning of #3. This kind of backtracking happens in alternative parsers when an alternative fails. The next page has more details on the semantics of backtracking.

Parsers in detail

So far, parsers have been presented as somewhat abstract entities. You may be wanting more detail. A Boost.Parser parser P is an invocable object with a pair of call operator overloads. The two functions are very similar, and in many parsers one is implemented in terms of the other. The first function does the parsing and returns the default attribute for the parser. The second function does exactly the same parsing, but takes an out-param into which it writes the attribute for the parser. The out-param does not need to be the same type as the default attribute, but they need to be compatible.

Compatibility means that the default attribute is assignable to the out-param in some fashion. This usually means direct assignment, but it may also mean a tuple -> aggregate or aggregate -> tuple conversion. For sequence types, compatibility means that the sequence type has insert or push_back with the usual semantics. This means that the parser +boost::parser::int_ can fill a std::set<int> just as well as a std::vector<int>.

Some parsers also have additional state that is required to perform a match. For instance, char_ parsers can be parameterized with a single code point to match; the exact value of that code point is stored in the parser object.

No parser has direct support for all the operations defined on parsers (operator|, operator>>, etc.). Instead, there is a template called parser_interface that supports all of these operations. parser_interface wraps each parser, storing it as a data member, adapting it for general use. You should only ever see parser_interface in the debugger, or possibly in some of the reference documentation. You should never have to write it in your own code.

As described in the previous page, backtracking occurs when the parse attempts to match the current parser P, matches part of the input, but fails to match all of P. The part of the input consumed during the parse of P is essentially "given back".

This is necessary because P may consist of subparsers, and each subparser that succeeds will try to consume input, produce attributes, etc. When a later subparser fails, the parse of P fails, and the input must be rewound to where it was when P started its parse, not where the latest matching subparser stopped.

Alternative parsers will often evaluate multiple subparsers one at a time, advancing and then restoring the input position, until one of the subparsers succeeds. Consider this example.

namespace bp = boost::parser;
auto const parser = repeat(53)[other_parser] | repeat(10)[other_parser];

Evaluating parser means trying to match other_parser 53 times, and if that fails, trying to match other_parser 10 times. Say you parse input that matches other_parser 11 times. parser will match it. It will also evaluate other_parser 21 times during the parse.

The attributes of the repeat(53)[other_parser] and repeat(10)[other_parser] are each std::vector<ATTR(other_parser)>; let's say that ATTR(other_parser) is int. The attribute of parser as a whole is the same, std::vector<int>. Since other_parser is busy producing ints — 21 of them to be exact — you may be wondering what happens to the ones produced during the evaluation of repeat(53)[other_parser] when it fails to find all 53 inputs. Its std::vector<int> will contain 11 ints at that point.

When a repeat-parser fails, and attributes are being generated, it clears its container. This applies to parsers such as the ones above, but also all the other repeat parsers, including ones made using operator+ or operator*.

So, at the end of a successful parse by parser of 10 inputs (since the right side of the alternative only eats 10 repetitions), the std::vector<int> attribute of parser would contain 10 ints.

Expectation points

Ok, so if parsers all try their best to match the input, and are all-or-nothing, doesn't that leave room for all kinds of bad input to be ignored? Consider the top-level parser from the Parsing JSON example.

auto const value_p_def =
    number | bp::bool_ | null | string | array_p | object_p;

What happens if I use this to parse "\""? The parse tries number, fails. It then tries bp::bool_, fails. Then null fails too. Finally, it starts parsing string. Good news, the first character is the open-quote of a JSON string. Unfortunately, that's also the end of the input, so string must fail too. However, we probably don't want to just give up on parsing string now and try array_p, right? If the user wrote an open-quote with no matching close-quote, that's not the prefix of some later alternative of value_p_def; it's ill-formed JSON. Here's the parser for the string rule:

auto const string_def = bp::lexeme['"' >> *(string_char - '"') > '"'];

Notice that operator> is used on the right instead of operator>>. This indicates the same sequence operation as operator>>, except that it also represents an expectation. If the parse before the operator> succeeds, whatever comes after it must also succeed. Otherwise, the top-level parse is failed, and a diagnostic is emitted. It will say something like "Expected '"' here.", quoting the line, with a caret pointing to the place in the input where it expected the right-side match.

Choosing to use > versus >> is how you indicate to Boost.Parser that parse failure is or is not a hard error, respectively.

When writing a parser, it often comes up that there is a set of strings that, when parsed, are associated with a set of values one-to-one. It is tedious to write parsers that recognize all the possible input strings when you have to associate each one with an attribute via a semantic action. Instead, we can use a symbol table.

Say we want to parse Roman numerals, one of the most common work-related parsing problems. We want to recognize numbers that start with any number of "M"s, representing thousands, followed by the hundreds, the tens, and the ones. Any of these may be absent from the input, but not all. Here are three symbol Boost.Parser tables that we can use to recognize ones, tens, and hundreds values, respectively:

bp::symbols<int> const ones = {
    {"I", 1},
    {"II", 2},
    {"III", 3},
    {"IV", 4},
    {"V", 5},
    {"VI", 6},
    {"VII", 7},
    {"VIII", 8},
    {"IX", 9}};

bp::symbols<int> const tens = {
    {"X", 10},
    {"XX", 20},
    {"XXX", 30},
    {"XL", 40},
    {"L", 50},
    {"LX", 60},
    {"LXX", 70},
    {"LXXX", 80},
    {"XC", 90}};

bp::symbols<int> const hundreds = {
    {"C", 100},
    {"CC", 200},
    {"CCC", 300},
    {"CD", 400},
    {"D", 500},
    {"DC", 600},
    {"DCC", 700},
    {"DCCC", 800},
    {"CM", 900}};

A symbols maps strings of char to their associated attributes. The type of the attribute must be specified as a template parameter to symbols — in this case, int.

Any "M"s we encounter should add 1000 to the result, and all other values come from the symbol tables. Here are the semantic actions we'll need to do that:

int result = 0;
auto const add_1000 = [&result](auto & ctx) { result += 1000; };
auto const add = [&result](auto & ctx) { result += _attr(ctx); };

add_1000 just adds 1000 to result. add adds whatever attribute is produced by its parser to result.

Now we just need to put the pieces together to make a parser:

using namespace bp::literals;
auto const parser =
    *'M'_l[add_1000] >> -hundreds[add] >> -tens[add] >> -ones[add];

We've got a few new bits in play here, so let's break it down. 'M'_l is a literal parser. That is, it is a parser that parses a literal char, code point, or string. In this case, a char 'M' is being parsed. The _l bit at the end is a UDL suffix that you can put after any char, char32_t, or char const * to form a literal parser. You can also make a literal parser by writing lit(), passing an argument of one of the previously mentioned types.

Why do we need any of this, considering that we just used a literal ',' in our previous example? The reason is that 'M' is not used in an expression with another Boost.Parser parser. It is used within *'M'_l[add_1000]. If we'd written *'M'[add_1000], clearly that would be ill-formed; char has no operator*, nor an operator[], associated with it.

On to the next bit: -hundreds[add]. By now, the use of the index operator should be pretty familiar; it associates the semantic action add with the parser hundreds. The operator- at the beginning is new. It means that the parser it is applied to is optional. You can read it as "zero or one". So, if hundreds is not successfully parsed after *'M'[add_1000], nothing happens, because hundreds is allowed to be missing — it's optional. If hundreds is parsed successfully, say by matching "CC", the resulting attribute, 200, is added to result inside add.

Here is the full listing of the program. Notice that it would have been inappropriate to use a whitespace skipper here, since the entire parse is a single number, so it was removed.

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


namespace bp = boost::parser;

int main()
{
    std::cout << "Enter a number using Roman numerals. ";
    std::string input;
    std::getline(std::cin, input);

    bp::symbols<int> const ones = {
        {"I", 1},
        {"II", 2},
        {"III", 3},
        {"IV", 4},
        {"V", 5},
        {"VI", 6},
        {"VII", 7},
        {"VIII", 8},
        {"IX", 9}};

    bp::symbols<int> const tens = {
        {"X", 10},
        {"XX", 20},
        {"XXX", 30},
        {"XL", 40},
        {"L", 50},
        {"LX", 60},
        {"LXX", 70},
        {"LXXX", 80},
        {"XC", 90}};

    bp::symbols<int> const hundreds = {
        {"C", 100},
        {"CC", 200},
        {"CCC", 300},
        {"CD", 400},
        {"D", 500},
        {"DC", 600},
        {"DCC", 700},
        {"DCCC", 800},
        {"CM", 900}};

    int result = 0;
    auto const add_1000 = [&result](auto & ctx) { result += 1000; };
    auto const add = [&result](auto & ctx) { result += _attr(ctx); };

    using namespace bp::literals;
    auto const parser =
        *'M'_l[add_1000] >> -hundreds[add] >> -tens[add] >> -ones[add];

    if (bp::parse(input, parser) && result != 0)
        std::cout << "That's " << result << " in Arabic numerals.\n";
    else
        std::cout << "That's not a Roman number.\n";
}

Diagnostic messages

Just like with a rule, you can give a symbols a bit of diagnostic text that will be used in error messages generated by Boost.Parser when the parse fails at an expectation point, as described in Error Handling and Debugging. See the symbols constructors for details.

The previous example showed how to use a symbol table as a fixed lookup table. What if we want to add things to the table during the parse? We can do that, but we need to do so within a semantic action. First, here is our symbol table, already with a single value in it:

bp::symbols<int> const symbols = {{"c", 8}};
assert(parse("c", symbols));

No surprise that it works to use the symbol table as a parser to parse the one string in the symbol table. Now, here's our parser:

auto const parser = (bp::char_ >> bp::int_)[add_symbol] >> symbols;

Here, we've attached the semantic action not to a simple parser like double_, but to the sequence parser (bp::char_ >> bp::int_). This sequence parser contains two parsers, each with its own attribute, so it produces two attributes as a tuple.

auto const add_symbol = [&symbols](auto & ctx) {
    using namespace bp::literals;
    // symbols::insert() requires a string, not a single character.
    char chars[2] = {_attr(ctx)[0_c], 0};
    symbols.insert(ctx, chars, _attr(ctx)[1_c]);
};

Inside the semantic action, we can get the first element of the attribute tuple using UDLs provided by Boost.Hana, and boost::hana::tuple::operator[](). The first attribute, from the char_, is _attr(ctx)[0_c], and the second, from the int_, is _attr(ctx)[1_c] (if boost::parser::tuple aliases to std::tuple, you'd use std::get or boost::parser::get instead). To add the symbol to the symbol table, we call insert().

auto const parser = (bp::char_ >> bp::int_)[add_symbol] >> symbols;

During the parse, ("X", 9) is parsed and added to the symbol table. Then, the second 'X' is recognized by the symbol table parser. However:

assert(!parse("X", symbols));

If we parse again, we find that "X" did not stay in the symbol table. The fact that symbols was declared const might have given you a hint that this would happen.

The full program:

#include <boost/parser/parser.hpp>

#include <iostream>
#include <string>


namespace bp = boost::parser;

int main()
{
    bp::symbols<int> const symbols = {{"c", 8}};
    assert(parse("c", symbols));

    auto const add_symbol = [&symbols](auto & ctx) {
        using namespace bp::literals;
        // symbols::insert() requires a string, not a single character.
        char chars[2] = {_attr(ctx)[0_c], 0};
        symbols.insert(ctx, chars, _attr(ctx)[1_c]);
    };
    auto const parser = (bp::char_ >> bp::int_)[add_symbol] >> symbols;

    auto const result = parse("X 9 X", parser, bp::ws);
    assert(result && *result == 9);
    (void)result;

    assert(!parse("X", symbols));
}

It is possible to add symbols to a symbols permanently. To do so, you have to use a mutable symbols object s, and add the symbols by calling s.insert_for_next_parse(), instead of s.insert(). These two operations are orthogonal, so if you want to both add a symbol to the table for the current top-level parse, and leave it in the table for subsequent top-level parses, you need to call both functions.

It is also possible to erase a single entry from the symbol table, or to clear the symbol table entirely. Just as with insertion, there are versions of erase and clear for the current parse, and another that applies only to subsequent parses. The full set of operations can be found in the symbols API docs.

[mpte There are two versions of each of the symbols *_for_next_parse() functions — one that takes a context, and one that does not. The one with the context is meant to be used within a semantic action. The one without the context is for use outside of any parse.]

Boost.Parser comes with all the parsers most parsing tasks will ever need. Each one is a constexpr object, or a constexpr function. Some of the non-functions are also callable, such as char_, which may be used directly, or with arguments, as in char_('a', 'z'). Any parser that can be called, whether a function or callable object, will be called a callable parser from now on. Note that there are no nullary callable parsers; they each take one or more arguments.

Each callable parser takes one or more parse arguments. A parse argument may be a value or an invocable object that accepts a reference to the parse context. The reference parameter may be mutable or constant. For example:

struct get_attribute
{
    template<typename Context>
    auto operator()(Context & ctx)
    {
        return _attr(ctx);
    }
};

This can also be a lambda. For example:

[](auto const & ctx) { return _attr(ctx); }

The operation that produces a value from a parse argument, which may be a value or a callable taking a parse context argument, is referred to as resolving the parse argument. If a parse argument arg can be called with the current context, then the resolved value of arg is arg(ctx); otherwise, the resolved value is just arg.

Some callable parsers take a parse predicate. A parse predicate is not quite the same as a parse argument, because it must be a callable object, and cannot be a value. A parse predicate's return type must be contextually convertible to bool. For example:

struct equals_three
{
    template<typename Context>
    bool operator()(Context const & ctx)
    {
        return _attr(ctx) == 3;
    }
};

This may of course be a lambda:

[](auto & ctx) { return _attr(ctx) == 3; }

The notional macro RESOLVE() expands to the result of resolving a parse argument or parse predicate. You'll see it used in the rest of the documentation.

An example of how parse arguments are used:

namespace bp = boost::parser;
// This parser matches one code point that is at least 'a', and at most
// the value of last_char, which comes from the globals.
auto last_char = [](auto & ctx) { return _globals(ctx).last_char; }
auto subparser = bp::char_('a', last_char);

Don't worry for now about what the globals are for now; the take-away is that you can make any argument you pass to a parser depend on the current state of the parse, by using the parse context:

namespace bp = boost::parser;
// This parser parses two code points.  For the parse to succeed, the
// second one must be >= 'a' and <= the first one.
auto set_last_char = [](auto & ctx) { _globals(ctx).last_char = _attr(x); };
auto parser = bp::char_[set_last_char] >> subparser;

Each callable parser returns a new parser, parameterized using the arguments given in the invocation.

This table lists all the Boost.Parser parsers. For the callable parsers, a separate entry exists for each possible arity of arguments. For a parser p, if there is no entry for p without arguments, p is a function, and cannot itself be used as a parser; it must be called. In the table below:

If you have an integral type IntType that is not covered by any of the Boost.Parser parsers, you can use a more verbose declaration to declare a parser for IntType. If IntType were unsigned, you would use uint_parser. If it were signed, you would use int_parser. For example:

constexpr parser_interface<int_parser<IntType>> hex_int;

uint_parser and int_parser accept three more non-type template parameters after the type parameter. They are Radix, MinDigits, and MaxDigits. Radix defaults to 10, MinDigits to 1, and MaxDigits to -1, which is a sentinel value meaning that there is no max number of digits.

So, if you wanted to parse exactly eight hexadecimal digits in a row in order to recognize Unicode character literals like C++ has (e.g. \Udeadbeef), you could use this parser for the digits at the end:

constexpr parser_interface<uint_parser<unsigned int, 16, 8, 8>> hex_int;