Parsing "true" and "false" using Boost.Spirit.Lex and Boost.Spirit.Qi
- by Andrew Ross
As the first stage of a larger grammar using Boost.Spirit I'm trying to parse "true" and "false" to produce the corresponding bool values, true and false.
I'm using Spirit.Lex to tokenize the input and have a working implementation for integer and floating point literals (including those expressed in a relaxed scientific notation), exposing int and float attributes.
Token definitions
#include <boost/spirit/include/lex_lexertl.hpp>
namespace lex = boost::spirit::lex;
typedef boost::mpl::vector<int, float, bool> token_value_type;
template <typename Lexer>
struct basic_literal_tokens : lex::lexer<Lexer> {
basic_literal_tokens() {
this->self.add_pattern("INT", "[-+]?[0-9]+");
int_literal = "{INT}";
// To be lexed as a float a numeric literal must have a decimal point
// or include an exponent, otherwise it will be considered an integer.
float_literal = "{INT}(((\\.[0-9]+)([eE]{INT})?)|([eE]{INT}))";
literal_true = "true";
literal_false = "false";
this->self = literal_true | literal_false | float_literal | int_literal;
}
lex::token_def<int> int_literal;
lex::token_def<float> float_literal;
lex::token_def<bool> literal_true, literal_false;
};
Testing parsing of float literals
My real implementation uses Boost.Test, but this is a self-contained example.
#include <string>
#include <iostream>
#include <cmath>
#include <cstdlib>
#include <limits>
bool parse_and_check_float(std::string const & input, float expected) {
typedef std::string::const_iterator base_iterator_type;
typedef lex::lexertl::token<base_iterator_type,
token_value_type
> token_type;
typedef lex::lexertl::lexer<token_type> lexer_type;
basic_literal_tokens<lexer_type> basic_literal_lexer;
base_iterator_type input_iter(input.begin());
float actual;
bool result = lex::tokenize_and_parse(input_iter, input.end(),
basic_literal_lexer,
basic_literal_lexer.float_literal,
actual);
return result && std::abs(expected - actual) < std::numeric_limits<float>::epsilon();
}
int main(int argc, char *argv[]) {
if (parse_and_check_float("+31.4e-1", 3.14)) {
return EXIT_SUCCESS;
} else {
return EXIT_FAILURE;
}
}
Parsing "true" and "false"
My problem is when trying to parse "true" and "false". This is the test code I'm using (after removing the Boost.Test parts):
bool parse_and_check_bool(std::string const & input, bool expected) {
typedef std::string::const_iterator base_iterator_type;
typedef lex::lexertl::token<base_iterator_type,
token_value_type
> token_type;
typedef lex::lexertl::lexer<token_type> lexer_type;
basic_literal_tokens<lexer_type> basic_literal_lexer;
base_iterator_type input_iter(input.begin());
bool actual;
lex::token_def<bool> parser = expected ? basic_literal_lexer.literal_true
: basic_literal_lexer.literal_false;
bool result = lex::tokenize_and_parse(input_iter, input.end(),
basic_literal_lexer, parser,
actual);
return result && actual == expected;
}
but compilation fails with:
boost/spirit/home/qi/detail/assign_to.hpp: In function ‘void boost::spirit::traits::assign_to(const Iterator&, const Iterator&, Attribute&) [with Iterator = __gnu_cxx::__normal_iterator<const char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, Attribute = bool]’:
boost/spirit/home/lex/lexer/lexertl/token.hpp:434: instantiated from ‘static void boost::spirit::traits::assign_to_attribute_from_value<Attribute, boost::spirit::lex::lexertl::token<Iterator, AttributeTypes, HasState>, void>::call(const boost::spirit::lex::lexertl::token<Iterator, AttributeTypes, HasState>&, Attribute&) [with Attribute = bool, Iterator = __gnu_cxx::__normal_iterator<const char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, AttributeTypes = boost::mpl::vector<int, float, bool, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, HasState = mpl_::bool_<true>]’
... backtrace of instantiation points ....
boost/spirit/home/qi/detail/assign_to.hpp:79: error: no matching function for call to ‘boost::spirit::traits::assign_to_attribute_from_iterators<bool, __gnu_cxx::__normal_iterator<const char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, void>::call(const __gnu_cxx::__normal_iterator<const char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >&, const __gnu_cxx::__normal_iterator<const char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >&, bool&)’
boost/spirit/home/qi/detail/construct.hpp:64: note: candidates are: static void boost::spirit::traits::assign_to_attribute_from_iterators<bool, Iterator, void>::call(const Iterator&, const Iterator&, char&) [with Iterator = __gnu_cxx::__normal_iterator<const char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >]
My interpretation of this is that Spirit.Qi doesn't know how to convert a string to a bool - surely that's not the case? Has anyone else done this before? If so, how?
