Nested strtok() calls to tokenize given string does not work as expected

Question

I want to tokenize a provided input string using strtok(). In the first step, I want to tokenize the given string by "|" symbol. Then, I tokenize each substring by ";". Finally, I tokenize the resulting substrings by " "(empty char). At the end, I aim to obtain an array of string arrays.

However, my while loop iterates just one time so I cannot process the whole string. All I get is:

I duplicate substrings before I pass them to strtok() but it does not work. I have spent almost 3 days but I could not achieve it yet. Any help will be greatly appreciated. For more detail, I provide the code and the output I receive.

Output: cat my_books.txt[enter image description here

Answer 1

The function strtok has a static local variable (pointer) that stores the last position of the parsed string within the function.

For example after the inner most loop

while (token3 != NULL) {
    printf("%s\n", token3);
    token3 = strtok(NULL, " ");
}

the static variable will be equal to NULL. So all outer loops will stop their iterations.

Instead use function strtok_s for which you can specify an argument that will keep the last position in the parsed string.

The function is declared like

char *strtok_s(char * restrict s1,
               rsize_t * restrict s1max,
               const char * restrict s2,
               char ** restrict ptr);

So for each loop you can declare a pointer like for example

char *pos;

and pass it to the function as expression &pos.