s
regex flag.matchToToken
function now have a closed
property. It is set to undefined
for the tokens where
“closed” doesn’t make sense. This means that all tokens objects have the
same shape, which might improve performance.These are the breaking changes:
'/a/s'.match(jsTokens)
no longer returns
['/', 'a', '/', 's']
, but ['/a/s']
. (There are
of course other variations of this.)closed
property could now behave differently.This release contains one breaking change, that should improve performance in V8:
So how can you, as a JavaScript developer, ensure that your RegExps are fast? If you are not interested in hooking into RegExp internals, make sure that neither the RegExp instance, nor its prototype is modified in order to get the best performance:
var re = /./g; .exec(''); // Fast path. re.new_property = 'slow'; re
This module used to export a single regex, with
.matchToToken
bolted on, just like in the above example.
This release changes the exports of the module to avoid this issue.
Before:
import jsTokens from "js-tokens"
// or:
var jsTokens = require("js-tokens")
var matchToToken = jsTokens.matchToToken
After:
import jsTokens, {matchToToken} from "js-tokens"
// or:
var jsTokens = require("js-tokens").default
var matchToToken = require("js-tokens").matchToToken
**
exponentiation operator.These are the breaking changes:
'**'.match(jsTokens)
no longer returns
['*', '*']
, but ['**']
.'**='.match(jsTokens)
no longer returns
['*', '*=']
, but ['**=']
.-
followed by a number is now correctly
matched as a punctuator followed by a number. It used to be matched as
just a number, but there is no such thing as negative number literals.
(Possibly backwards-incompatible change.)u
flag.jsTokens.matchToToken
performance.jsTokens.names
array has been replaced
with the jsTokens.matchToToken
function. The capturing
groups of jsTokens
are no longer part of the public API;
instead use said function. See this gist for
an example. (Backwards-incompatible change.)=>
) as an
operator, instead of its own category (“functionArrow”), for simplicity.
(Backwards-incompatible change.)...
) are now matched as an operator
(instead of three punctuations). (Backwards-incompatible change.)