advanced patterns when using `strtok` for tokenizing strings in C with multiple delimiters
I'm working with an scenario when using `strtok` to tokenize a string that contains multiple delimiters... The function seems to skip tokens unexpectedly when the delimiters are adjacent. Here's the relevant code snippet: ```c #include <stdio.h> #include <string.h> int main() { char str[] = "hello,,world;this:is:a:test"; const char *delimiters = ",;:"; char *token; token = strtok(str, delimiters); while (token != NULL) { printf("%s\n", token); token = strtok(NULL, delimiters); } return 0; } ``` When I run this code, I expect to get the tokens `hello`, `world`, `this`, `is`, `a`, and `test`. However, the output is: ``` hello world this is a test ``` The comma between `hello` and `world` should be treated as a single delimiter, but it seems like `strtok` is skipping over it and considering `world` as a separate token. I've checked the documentation for `strtok`, and it seems like it should correctly handle multiple consecutive delimiters. I've also tested with different combinations of delimiters, and the behavior is consistent. Is there a reason why `strtok` behaves this way? Should I be using a different approach or function for this kind of tokenization? I would appreciate any insights or recommendations on how to handle this situation effectively.