Token

Token.

Constructors

this
this(TokenType type)
Undocumented in source.
this
this(TokenType type, SourceFile file, int line, int pos)
Undocumented in source.

Members

Functions

clone
Token clone()
Undocumented in source.
setFile
void setFile(SourceFile file)

set source file information for token

setPos
void setPos(SourceFile file, int line, int pos)

set start position for token (line is 1-based, pos is 0-based)

setPos
void setPos(int line, int pos)

set start position for token (line is 1-based, pos is 0-based)

Properties

doubleValue
double doubleValue [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
errorCode
int errorCode [@property getter]

returns error code if it's invalid token (can be returned in error tolerant mode of tokenizer)

errorMessage
string errorMessage [@property getter]

returns error message if it's invalid token (can be returned in error tolerant mode of tokenizer)

filename
SourceFile filename [@property getter]

returns file info for source

floatValue
float floatValue [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
intValue
ulong intValue [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
invalidTokenType
TokenType invalidTokenType [@property getter]

returns type of token parsing of which has been failed - if it's invalid token (can be returned in error tolerant mode of tokenizer)

isBracket
bool isBracket [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
isCloseBracket
bool isCloseBracket [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
isDocumentationComment
bool isDocumentationComment [@property getter]

returns true if this is documentation comment token

isEof
bool isEof [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
isError
bool isError [@property getter]

returns true if it's invalid token (can be returned in error tolerant mode of tokenizer)

isImaginary
bool isImaginary [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
isLong
ulong isLong [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
isMultilineComment
bool isMultilineComment [@property getter]

returns true if this is multiline

isOpenBracket
bool isOpenBracket [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
isUnsigned
bool isUnsigned [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
keyword
Keyword keyword [@property getter]

returns keyword ID - for keyword tokens

line
int line [@property getter]

returns 1-based source line number of token start

literalType
dchar literalType [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
opCode
OpCode opCode [@property getter]

returns opcode ID - for opcode tokens

pos
int pos [@property getter]

returns 1-based source line position of token start

precision
byte precision [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
realValue
real realValue [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
text
dstring text [@property getter]

returns token text

toString
string toString [@property getter]
Undocumented in source. Be warned that the author may not have intended to support it.
type
TokenType type [@property getter]

returns token type

Variables

_file
SourceFile _file;
Undocumented in source.
_line
int _line;
Undocumented in source.
_pos
int _pos;
Undocumented in source.
_type
TokenType _type;
Undocumented in source.

Meta