QStringTokenizer Class
template <typename Haystack, typename Needle> class QStringTokenizerThe QStringTokenizer class splits strings into tokens along given separators. More...
Header: | #include <QStringTokenizer> |
CMake: | find_package(Qt6 REQUIRED COMPONENTS Core) target_link_libraries(mytarget PRIVATE Qt6::Core) |
qmake: | QT += core |
Since: | Qt 6.0 |
Inherits: | QtPrivate::Tok::HaystackPinning (private), QtPrivate::Tok::NeedlePinning (private), and |
- List of all members, including inherited members
- QStringTokenizer is part of Classes for String Data.
Note: All functions in this class are reentrant.
Public Types
const_iterator | |
const_pointer | |
const_reference | |
difference_type | |
iterator | |
pointer | |
reference | |
sentinel | |
size_type | |
value_type |
Public Functions
QStringTokenizer(Haystack haystack, Needle needle, Qt::CaseSensitivity cs, Qt::SplitBehavior sb = Qt::KeepEmptyParts) | |
QStringTokenizer(Haystack haystack, Needle needle, Qt::SplitBehavior sb = Qt::KeepEmptyParts, Qt::CaseSensitivity cs = Qt::CaseSensitive) | |
QStringTokenizer::iterator | begin() const |
QStringTokenizer::iterator | cbegin() const |
QStringTokenizer::sentinel | cend() const |
QStringTokenizer::sentinel | end() const |
LContainer | toContainer(LContainer &&c = {}) const & |
RContainer | toContainer(RContainer &&c = {}) const && |
Related Non-Members
decltype(QtPrivate::Tok::TokenizerResult<Haystack, Needle>({std::forward<Haystack>(h), std::forward<Needle>(n), flags...})) | qTokenize(Haystack &&haystack, Needle &&needle, Flags... flags) |
Detailed Description
Splits a string into substrings wherever a given separator occurs, returning a (lazily constructed) list of those strings. If the separator does not match anywhere in the string, produces a single-element list containing this string. If the separator is empty, QStringTokenizer produces an empty string, followed by each of the string's characters, followed by another empty string. The two enumerations Qt::SplitBehavior and Qt::CaseSensitivity further control the output.
QStringTokenizer drives QStringView::tokenize(), but you can use it directly, too:
for (auto it : QStringTokenizer{string, separator}) use(*it);
Note: You should never, ever, name the template arguments of a QStringTokenizer explicitly. You may write QStringTokenizer{string, separator}
(without template arguments), or use the qTokenize() function, or the QStringView::split() or QLatin1StringView::split() member functions and store the return value only in auto
variables:
auto result = string.split(sep);
This is because the template arguments of QStringTokenizer have a very subtle dependency on the specific string and separator types from with which they are constructed, and they don't usually correspond to the actual types passed.
Lazy Sequences
QStringTokenizer acts as a so-called lazy sequence, that is, each next element is only computed once you ask for it. Lazy sequences have the advantage that they only require O(1) memory. They have the disadvantage that, at least for QStringTokenizer, they only allow forward, not random-access, iteration.
The intended use-case is that you just plug it into a ranged for loop:
for (auto it : QStringTokenizer{string, separator}) use(*it);
or a C++20 ranged algorithm:
std::ranges::for_each(QStringTokenizer{string, separator}, [] (auto token) { use(token); });
End Sentinel
The QStringTokenizer iterators cannot be used with classical STL algorithms, because those require iterator/iterator pairs, while QStringTokenizer uses sentinels. That is, it uses a different type, QStringTokenizer::sentinel, to mark the end of the range. This improves performance, because the sentinel is an empty type. Sentinels are supported from C++17 (for ranged for) and C++20 (for algorithms using the new ranges library).
Temporaries
QStringTokenizer is very carefully designed to avoid dangling references. If you construct a tokenizer from a temporary string (an rvalue), that argument is stored internally, so the referenced data isn't deleted before it is tokenized:
auto tok = QStringTokenizer{widget.text(), u','}; // return value of `widget.text()` is destroyed, but content was moved into `tok` for (auto e : tok) use(e);
If you pass named objects (lvalues), then QStringTokenizer does not store a copy. You are responsible to keep the named object's data around for longer than the tokenizer operates on it:
auto text = widget.text(); auto tok = QStringTokenizer{text, u','}; text.clear(); // destroy content of `text` for (auto e : tok) // ERROR: `tok` references deleted data! use(e);
See also QStringView::split(), QString::split(), and QRegularExpression.
Member Type Documentation
[alias]
QStringTokenizer::const_iterator
This typedef provides an STL-style const iterator for QStringTokenizer.
See also iterator.
[alias]
QStringTokenizer::const_pointer
Alias for value_type *
.
[alias]
QStringTokenizer::const_reference
Alias for value_type &
.
[alias]
QStringTokenizer::difference_type
Alias for qsizetype.
[alias]
QStringTokenizer::iterator
This typedef provides an STL-style const iterator for QStringTokenizer.
QStringTokenizer does not support mutable iterators, so this is the same as const_iterator.
See also const_iterator.
[alias]
QStringTokenizer::pointer
Alias for value_type *
.
QStringTokenizer does not support mutable iterators, so this is the same as const_pointer.
[alias]
QStringTokenizer::reference
Alias for value_type &
.
QStringTokenizer does not support mutable references, so this is the same as const_reference.
[alias]
QStringTokenizer::sentinel
This typedef provides an STL-style sentinel for QStringTokenizer::iterator and QStringTokenizer::const_iterator.
See also const_iterator.
[alias]
QStringTokenizer::size_type
Alias for qsizetype.
[alias]
QStringTokenizer::value_type
Alias for const QStringView
or const QLatin1StringView
, depending on the tokenizer's Haystack
template argument.
Member Function Documentation
[explicit constexpr]
QStringTokenizer::QStringTokenizer(Haystack haystack, Needle needle, Qt::CaseSensitivity cs, Qt::SplitBehavior sb = Qt::KeepEmptyParts)
[explicit constexpr]
QStringTokenizer::QStringTokenizer(Haystack haystack, Needle needle, Qt::SplitBehavior sb = Qt::KeepEmptyParts, Qt::CaseSensitivity cs = Qt::CaseSensitive)
Constructs a string tokenizer that splits the string haystack into substrings wherever needle occurs, and allows iteration over those strings as they are found. If needle does not match anywhere in haystack, a single element containing haystack is produced.
cs specifies whether needle should be matched case sensitively or case insensitively.
If sb is Qt::SkipEmptyParts, empty entries don't appear in the result. By default, empty entries are included.
See also QStringView::split(), QString::split(), Qt::CaseSensitivity, and Qt::SplitBehavior.
QStringTokenizer::iterator QStringTokenizer::begin() const
QStringTokenizer::iterator QStringTokenizer::cbegin() const
Returns a const STL-style iterator pointing to the first token in the list.
[constexpr]
QStringTokenizer::sentinel QStringTokenizer::cend() const
Same as end().
[constexpr]
QStringTokenizer::sentinel QStringTokenizer::end() const
Returns a const STL-style sentinel pointing to the imaginary token after the last token in the list.
template <typename LContainer> LContainer QStringTokenizer::toContainer(LContainer &&c = {}) const &
Converts the lazy sequence into a (typically) random-access container of type LContainer
.
This function is only available if Container
has a value_type
matching this tokenizer's value_type.
If you pass in a named container (an lvalue) for c, then that container is filled, and a reference to it is returned. If you pass in a temporary container (an rvalue, incl. the default argument), then that container is filled, and returned by value.
// assuming tok's value_type is QStringView, then... auto tok = QStringTokenizer{~~~}; // ... rac1 is a QList: auto rac1 = tok.toContainer(); // ... rac2 is std::pmr::vector<QStringView>: auto rac2 = tok.toContainer<std::pmr::vector<QStringView>>(); auto rac3 = QVarLengthArray<QStringView, 12>{}; // appends the token sequence produced by tok to rac3 // and returns a reference to rac3 (which we ignore here): tok.toContainer(rac3);
This gives you maximum flexibility in how you want the sequence to be stored.
template <typename RContainer> RContainer QStringTokenizer::toContainer(RContainer &&c = {}) const &&
This is an overloaded function.
Converts the lazy sequence into a (typically) random-access container of type RContainer
.
In addition to the constraints on the lvalue-this overload, this rvalue-this overload is only available when this QStringTokenizer does not store the haystack internally, as this could create a container full of dangling references:
auto tokens = QStringTokenizer{widget.text(), u','}.toContainer(); // ERROR: cannot call toContainer() on rvalue // 'tokens' references the data of the copy of widget.text() // stored inside the QStringTokenizer, which has since been deleted
To fix, store the QStringTokenizer in a temporary:
auto tokenizer = QStringTokenizer{widget.text90, u','}; auto tokens = tokenizer.toContainer(); // OK: the copy of widget.text() stored in 'tokenizer' keeps the data // referenced by 'tokens' alive.
You can force this function into existence by passing a view instead:
func(QStringTokenizer{QStringView{widget.text()}, u','}.toContainer()); // OK: compiler keeps widget.text() around until after func() has executed
If you pass in a named container (an lvalue)for c, then that container is filled, and a reference to it is returned. If you pass in a temporary container (an rvalue, incl. the default argument), then that container is filled, and returned by value.
Related Non-Members
[constexpr, since 6.0]
template <typename Haystack, typename Needle, typename Flags> decltype(QtPrivate::Tok::TokenizerResult<Haystack, Needle>({std::forward<Haystack>(h), std::forward<Needle>(n), flags...})) qTokenize(Haystack &&haystack, Needle &&needle, Flags... flags)
Factory function for a QStringTokenizer that splits the string haystack into substrings wherever needle occurs, and allows iteration over those strings as they are found. If needle does not match anywhere in haystack, a single element containing haystack is produced.
Pass values from Qt::CaseSensitivity and Qt::SplitBehavior enumerators as flags to modify the behavior of the tokenizer.
This function was introduced in Qt 6.0.
© 2024 The Qt Company Ltd. Documentation contributions included herein are the copyrights of their respective owners. The documentation provided herein is licensed under the terms of the GNU Free Documentation License version 1.3 as published by the Free Software Foundation. Qt and respective logos are trademarks of The Qt Company Ltd. in Finland and/or other countries worldwide. All other trademarks are property of their respective owners.