returns true if this is multiline
returns token type
returns file info for source
returns 1-based source line number of token start
returns 1-based source line position of token start
returns token text
returns opcode ID - for opcode tokens
returns keyword ID - for keyword tokens
returns true if this is documentation comment token
returns true if this is multiline
returns true if it's invalid token (can be returned in error tolerant mode of tokenizer)
returns error message if it's invalid token (can be returned in error tolerant mode of tokenizer)
returns error code if it's invalid token (can be returned in error tolerant mode of tokenizer)
returns type of token parsing of which has been failed - if it's invalid token (can be returned in error tolerant mode of tokenizer)
set start position for token (line is 1-based, pos is 0-based)
set source file information for token
set start position for token (line is 1-based, pos is 0-based)
comment token