c - VS 2015 中的 std::wcstok

Question

我已经简化了用过的代码案例，当使用 VS 2015 C++ 编译器编译时会产生警告。

#include <cwchar>
#include <iostream>

int main()
{
    wchar_t input[100] = L"A bird came down the walk";
    wchar_t* token = std::wcstok(input, L" ");
    while (token) {
        std::wcout << token << '\n';
        token = std::wcstok(nullptr, L" ");
    }
}

这产生了以下警告。

warning C4996: 'wcstok': wcstok has been changed to conform with the ISO C standard, adding an extra context parameter. To use the legacy Microsoft wcstok, define _CRT_NON_CONFORMING_WCSTOK.
1>  c:\program files (x86)\windows kits\10\include\10.0.10240.0\ucrt\corecrt_wstring.h(254): note: see declaration of 'wcstok'

warning C4996: 'wcstok': wcstok has been changed to conform with the ISO C standard, adding an extra context parameter. To use the legacy Microsoft wcstok, define _CRT_NON_CONFORMING_WCSTOK.
1>  c:\program files (x86)\windows kits\10\include\10.0.10240.0\ucrt\corecrt_wstring.h(254): note: see declaration of 'wcstok'

在网上查找，我读到 std::wcstok和 breaking changes in VS 2015其中提到 C 标准引入了第三个参数，

It used an internal, per-thread context to track state across calls, as is done for strtok. The function now has the signature wchar_t* wcstok(wchar_t*, wchar_t const*, wchar_t**), and requires the caller to pass the context as a third argument to the function.

以听起来天生愚蠢为代价，我仍然会继续问， 谁能用简单的术语解释这第三个参数的用途，以及它如何改变 std::wcstok 从其早期版本？

Answer 1

旧版本类似于 strtok 并使用全局线程本地存储来存储最后一个标记末尾之后的位置。

使用方法的问题在于它不允许嵌套函数，如 strtok/wcstok。

假设我们有一个像 "r0c0;r0c1\nr1c0;r1c1" 这样的字符串(一个 2 行 2 列的表)，我们想先将它拆分成行，然后将每一行拆分成列。

为此，我们需要 2 个循环。使用旧方法这是不可能的，因为嵌套循环会覆盖外部循环的状态。使用新方法，每个循环都可以将单独的状态存储在单独的变量中:

#include <cwchar>
#include <iostream>

int main()
{
    wchar_t input[] = L"r0c0;r0c1\n"
                      L"r1c0;r1c1";
    wchar_t *rowstate;
    wchar_t *row = std::wcstok(input, L"\n", &rowstate);

    while (row != nullptr) {
        std::wcout << L"Row: " << row << std::endl;

        wchar_t *colstate;
        wchar_t *col = std::wcstok(row, L";", &colstate);

        while (col != nullptr) {
            std::wcout << "  Col: " << col << std::endl;
            col = std::wcstok(nullptr, L" ", &colstate);
        }

        row = std::wcstok(nullptr, L" ", &rowstate);
    }
}

输出是:

Row: r0c0;r0c1
  Col: r0c0
  Col: r0c1
Row: r1c0;r1c1
  Col: r1c0
  Col: r1c1