Automattic\WooCommerce\EmailEditorVendor\Symfony\Component\CssSelector\Parser\Tokenizer

Tokenizer::tokenize │ public │ WC 1.0

Tokenize selector source code.

Метод класса: Tokenizer{}

Хуков нет.

Возвращает

null. Ничего (null).

Использование

$Tokenizer = new Tokenizer();
$Tokenizer->tokenize( $reader ): TokenStream;

$reader(Reader) (обязательный)

.

Код Tokenizer::tokenize() Tokenizer::tokenize ^{WC 11.2.2}

public function tokenize(Reader $reader): TokenStream
{
    $stream = new TokenStream();

    while (!$reader->isEOF()) {
        foreach ($this->handlers as $handler) {
            if ($handler->handle($reader, $stream)) {
                continue 2;
            }
        }

        $stream->push(new Token(Token::TYPE_DELIMITER, $reader->getSubstring(1), $reader->getPosition()));
        $reader->moveForward(1);
    }

    return $stream
        ->push(new Token(Token::TYPE_FILE_END, null, $reader->getPosition()))
        ->freeze();
}