YACC/FLEX and reporting my errors

[Benny the Guard]

I am writing a parser to handle input commands and process them. There are some builtin in commands that it will recognize one of which is TOK_CMD with another keyword after it (all tokenized based on supported commands).

The processing is working great for real commands, so moving on to error detection. I have the rule:

| TOK_CMD error TOK_EOL
  {
  printf ("illegal command dummy\n");
  }
| error TOK_EOL
  {
  printf ("this is not a command\n");
  }

at the end of my grammar. The idea is that I want to treat "unknown commands" differently then other unknown text. The trap seems to work perfectly, but to make it better at reporting I want to spit out the text of 'error' which could be any number of unmatched tokens or unmatched code. Note I do not want the entire line, just the unmatched part (ie missing the TOK_CMD part). Its been years since I used YACC so hoping this is easy and I just forgot.

[JosAH]

Can you use '$1' as a string (i.e. a char*)? As in:

  printf ("illegal command: %s\n", $1);
 

kind regards,

Jos

[Benny the Guard]

Yes, the TOK_CMD can be used as a string. But the part i want is what 'error' represents not the value of TOK_CMD (although that may be important as well).

I have the whole line in a char* buffer. So in theory maybe I could do:

| TOK_CMD error TOK_EOL
{
 
const char *begin_err_str = strstr (MyLineBuffer, $1);
if (begin_err_str)
     // advance to the end of TOK_CMD
     begin_err_str += strlen ($1);
 else
     // Should never happen but just in case
     begin_err_str = My_Line_Buffer;
 
printf ("%s is not a supported command dummy\n", begin_err_str);
}

Of course ther newline is still in begin-err_string but I can remove that easily as well. A bit clunky but if you cannot use the $2 on error then its all I have to go on.

[JosAH]

Yes, the TOK_CMD can be used as a string. But the part i want is what 'error' represents not the value of TOK_CMD (although that may be important as well).

I have the whole line in a char* buffer. So in theory maybe I could do:

| TOK_CMD error TOK_EOL
{
 
const char *begin_err_str = strstr (MyLineBuffer, $1);
if (begin_err_str)
     // advance to the end of TOK_CMD
     begin_err_str += strlen ($1);
 else
     // Should never happen but just in case
     begin_err_str = My_Line_Buffer;
 
printf ("%s is not a supported command dummy\n", begin_err_str);
}

Of course ther newline is still in begin-err_string but I can remove that easily as well. A bit clunky but if you cannot use the $2 on error then its all I have to go on.

Hm, indeed it's a bit 'mechanical' but it's all you can do (as far as I can tell now).
Yacc pops states until it finds your (error) state and even can skip further tokens
to work its way out of the error state: everything could've happened in the mean
time. If you got that MyLineBuffer available I think the way you outlined it is the
way to go.

Or use PCCTS or its successor ANTLR of course ;-)

kind regards,

Jos

[Benny the Guard]

Well it was a good idea, but seems to have been for naught. What happennns is the $1 does not return the full text for the token. Just part of it.

Here is my FLEX file (portion):
ID            [[:alnum:]_\-\.\*,()#^\:\/\$\+\[\]\|\{\}@&\%\<\>]+
WHITESPACE    " "+|\t+
CMD                  "command"|"cmd"
CMD_EXEC        ({ID}{WHITESPACE}{CMD})
 
%%
{CMD}                {return TOK_CMD;    }
{CMD_EXEC}           {return TOK_CMD;    }

and here is my YACC code:

| TOK_CMD error TOK_NEWLINE
    {
        char err_str [4098];
 
        char *cmd_str_ptr = strstr (Line_Buffer, $1);
    if (cmd_str_ptr)
          cmd_str_ptr += strlen ($1);
    else
          cmd_str_ptr = Line_Buffer;
 
        // Now remove the NEWLINE and copy locally
    strncpy (err_str, cmd_str_ptr, strlen (cmd_str_ptr) - strlen ($3));
        printf ("Invalid command exec: %s\n", err_str);
        }

I do have TOK_CMD rules above this that have a valid/known string after it. This is the catch for CMD_EXEC that is unknown. But when I see:

command run
command run1
#PC>command run
#PC>command run2

In my test file ('run is supported') I see:

Invalid command exec: command run1
Invalid command exec: command run2

So the $1 only includes the part of the string not explicitly defined in the lexer, such as the '#PC>' part. Why does it not include the "command" portion as that is definitly part of TOK_CMD

[JosAH]

So the $1 only includes the part of the string not explicitly defined in the lexer, such as the '#PC>' part. Why does it not include the "command" portion as that is definitly part of TOK_CMD

Because the other tokens were 'previous' tokens and all that is returned is TOK_CMD
with only the last lexical token present in (f)lex's buffer, i.e. the CMD part.
The {ID} and {WHITESPACE} tokens are already forgotten (but correctly scanned).

kind regards,

Jos