We often deal with strings in a C program and we want to split the strings into small tokens on the basis of delimiter. We can do this easily. Lets scan each character of the string from the beginning and put NULL whenever it matches the delimiter character. We have a static variable to store the last token position. This advances as we obtain a token and move to the next posotion. Below is a stepping diagram and C code of this process.

Input String:
"A Quick Brown Fox Jumps over the lazy dog."

Tokenize process:

Token 1: "A\0Quick Brown Fox Jumps over the lazy dog."
Position^
Token 2: "A\0Quick\0Brown Fox Jumps over the lazy dog."
Position^
Token 3: "A\0Quick\0Brown\0Fox Jumps over the lazy dog."
Position^
Token 4: "A\0Quick\0Brown\0Fox\0Jumps over the lazy dog."
Position^
Token 5: "A\0Quick\0Brown\0Fox\0Jumps\0over the lazy dog."
Position^
Token 6: "A\0Quick\0Brown\0Fox\0Jumps\0over\0the lazy dog."
Position^
Token 7: "A\0Quick\0Brown\0Fox\0Jumps\0over\0the\0lazy dog."
Position^
Token 8: "A\0Quick\0Brown\0Fox\0Jumps\0over\0the\0lazy\0dog."
Position^
Token 9: "A\0Quick\0Brown\0Fox\0Jumps\0over\0the\0lazy\0dog\0"
Position^
 1 char * strtok(char *str, const char * delim) 2 { 3 int i = 0; 4 int j, len; 5 static char * last_pos; 6 char *token; 7 if (str) { 8 last_pos = str; 9 } 10 11 token = last_pos; 12 len = strlen(delim); 13 14 while(*last_pos){ 15 for(j = 0; j < len; j++) { 16 if (delim[j] == *last_pos) { 17 *last_pos = ''; 18 last_pos++; 19 return token; 20 } 21 } 22 last_pos++; 23 } 24 return NULL; 25 } 26

We do not require to do all the steps as C runtime library has strtok() function to do all these for us. Here is a sample example program.

 1 #include  2 #include  3 int main(int argc, char* argv[]) 4 { 5 int i = 0; 6 char *str = "A Quick Brown Fox Jumps over the lazy dog."; 7 char *tokens, *token; 8 tokens = strdup (str); 9 token = strtok (tokens, " ."); 10 while(token) { 11 printf("Token %d: %s\n", i + 1, token); 12 token = strtok(NULL, " ."); 13 i++; 14 } 15 free(tokens); 16 return 0; 17 } 18 Output: 19 Token 1: A 20 Token 2: Quick 21 Token 3: Brown 22 Token 4: Fox 23 Token 5: Jumps 24 Token 6: over 25 Token 7: the 26 Token 8: lazy 27 Token 9: dog 28

Note: Buffer allocated by strdup should be released with free(). strtok(), strrev() etc are some utility functions modify its input buffer and produce the result. This is why we should not pass a constant string. CPU will produce a access violation while execution. strdup() is usedful for these cases.

You have viewed 1 page out of 248. Your C learning is 0.00% complete. Login to check your learning progress.

0

Similar topics related to this section