Contains information regarding the tokens around a cursor position. Mostly for internally use by autosuggest .
Notes:
prev, at, next properties that contain tokens are set looking only at the list of tokens extracted from the ast by extractTokens. This is why there is an extra whitespace property to tell us whether there is whitespace (i.e. a hole) between the cursor and the next/prev valid tokens or if it can't find any, the start/end of the input.let i = tokens.findIndex(t => t === info.next)
while (tokens[i] instanceof ErrorToken) {...}
Examples:
Ctrl+| A // tokens: ["Ctrl", "+", TokenError(Key), "A"]
^
{
index: 5, // cursor position
at: undefined, // it's not inside a token
prev: "+",
next: TokenError, // key is missing to the right
// closest valid tokens to either side
valid: {
prev: "+",
next: "A",
},
// whether there is whitespace between the cursor and the next/prev valid tokens or start/end of input
whitespace: {
prev: false,
next: true,
}
}
Ctrl+Sh|ift+A
^
{
index: 7,
at: "Shift", // it's inside a token
prev: "+",
next: "+",
valid: {
prev: "+",
next: "-",
},
whitespace: {
prev: false,
next: false,
}
}
The token the cursor is inside of. By "inside", we mean the ends of the token are before/after the cursor respectively (e.g. a|a, but NOT |aa or aa|). This token, if defined, is always a valid token, since error tokens have no length.
The first token, valid or invalid, that starts at or after the index position.
The first token (going backwards), valid or invalid, that ends at or before the index position.
Closest valid tokens.
Closest prev valid token.
Closest next valid token.
Whether there is whitespace between the cursor and the next/prev valid tokens or start/end of the input.
Whether there is whitespace between the cursor and the next valid token or the end of the input.
Whether there is whitespace between the cursor and the prev valid token or the start of the input.
Color used to highlight the hints in parens that indicate how the node is being used (e.g. a variable node might be a property, or alone as a variable, etc)
Color used to highlight the extra information some nodes contain in their headers for a quick overview (e.g. which operator for expression nodes, if a condition/group value is true, how long an array value is etc).
Color used to reset highlights.
Color used to highlight the actual text content of the token nodes.
Keys can have "notes" attached which are stored in a key's note property. To enable them, you can just pass true or {}. Options are merged over the default { left: "(", right: ")", space: true } will be used.
They are useful for specifying things like holds, toggle states, multiple clicks or whatever other extra states you can think of: Capslock(on), Capslock(off), SomeKey(Hold:5000) (hold for 5 seconds), RButton(2:200) (2 clicks within 200ms).
Note the lack of a space between the key and the start delimiter. Capslock (on) would NOT be parsed like you might expect, but like [KEY] [MISSING KEY][...NOTE] because a space always starts a new chain combo/chord.
You can parse the note contents to something else by passing a parser function. You can additional properties to valid tokens by extending the ValidToken and ErrorToken classes respectively.
class NoteToken extends ValidToken<TOKEN_TYPE.NOTE_CONTENT> {
constructor(
// for ValidToken
token: { type: TOKEN_TYPE.NOTE_CONTENT, value: string, start: number, end: number }
// extra properties you might want to add
{ noteType, state }: { noteType: "toggle"|"hold", state: number }
) {
super(token)
this.noteType = noteType
this.state = state
}
// optionally also specify a custom stringify function (this does not need to escape any characters but the right delimiter, even though more options than that are passed)
stringify(opts) {
//...
}
}
const parser = new Parser<{}, NoteToken | ErrorToken<TOKEN_TYPE.NOTE_CONTENT>>({
keyNote: {
left:"(", right:")",
parser(token) {
if (token instanceof ValidToken) {
let match
const input = token.value.toLowercase()
if (match = input.match(/toggle:(?<type>on|off)/i)) {
return new NoteToken(token, {noteType: "toggle", state: match.groups.type === "on" ? 1 : 0})
} else if (match = input.match(/hold:(?<amount>[0-9]+)/)) {
return new NoteToken(token, {noteType: "hold", state: parseInt(match.groups.amount)})
}
return new ErrorToken({start, end, expected: [TOKEN_TYPE.NOTE_CONTENT]})
} else {
return token
}
}
}
})
// ... later when accessing some key:
key.note.contents.noteType // hold | toggle
key.note.contents.state // number
key.note.contents.parent // key
A list of allowed key separators (["+", "-"] by default.)
They can be escaped to be used as part of a key by escaping them (e.g. key\+ means a key with the name key+).
The first separator is used to stringify the separator token (e.g. given separators = ["+", "-"] and input like key+key-key, it will get stringified like key+key+key).
For validating the ast before it's evaluated using Parser.validate (e.g. for syntax highlighting purposes). For the moment the ast must be valid (without syntax errors) to be validated.
The function is passed a Token and should return any positions of interest. These are collected and are returned as the result of the validate method. Basically it makes it easy to "tag" ranges. You're not restricted to returning just the position and you can use the generic argument to type additional properties you might return.
const allowedKeys = ["a", "b", "c"]
// ...
tokenValidator(token) {
let res = []
if (token instanceof ErrorToken) {
res.push({start:token.start, end:token.end, type: "MISSING " + (token.type === TOKEN_TYPE.KEY ? "KEY" : "SEPARATOR")})
}
if (token.type === TOKEN_TYPE.KEY) {
if (!allowedKeys.includes(token.value)) {
res.push({start:token.start, end:token.end, type: "INVALID_KEY"})
}
}
if (token.note) {
// validate token notes
}
// more complicated things:
if (token.parent.lastKey === token) {// if is last key
// check key is not modifier
}
}
// later
const errors => parser.validate(ast)
// ... use the information to highlight those locations accordingly
A suggestion entry that describes a type of suggestion.
Whether the suggestion was created because there was an error token there and it would fix it.
The range the suggestion should replace / be inserted at.
When key notes are enabled, whether the suggestion would require inserting the delimiters and which ones.
Examples:
key| require inserting both delimiters.
key|) requires inserting the left delimiter.
Whether the suggestion would require a separator be inserted immediately before.
For example, if a cursor is here Ctrl|, suggestions both to replace the Ctrl key and to insert a key would be provided, but the insert would require a separator. i.e. Replaced| vs Ctrl+Insert
Generated using TypeDoc
For more easily typing tokens that might or might not be valid.
Using Token does not work well in certain situations and is also more complex because it has so many generics.