verify whether an id passes through all stages

Question

id  current stage  previous stages
1      06              05
1      06              03

2     04               03
2     04               02

suppose there are 5 stages of an id.(02,03 etc) An id should goes through each of the stages. Here in example Id num 1 skips 04 and 02 stage but id num 2 passes through all.so it should be current stage -1 and -2 etc...

i have to identify such ids which skips stages. need to do it R or hadoop query.

Answer 1

If I understood the question correctly then you can try below dplyr solution.

library(dplyr)

df %>%
  group_by(id, current_stage) %>%
  summarise(all_prev_stages = paste(sort(previous_stages, decreasing = T), collapse = ",")) %>%
  mutate(posible_prev_stages = paste(seq(current_stage-1, 2), collapse = ",")) %>%
  filter(all_prev_stages != posible_prev_stages) %>%
  select(id)

This gives the list of ids which skip stages (i.e. id = 1 in your sample data):

     id
1     1

Sample data:

df <- structure(list(id = c(1L, 1L, 2L, 2L), current_stage = c(6L, 
6L, 4L, 4L), previous_stages = c(5L, 3L, 3L, 2L)), .Names = c("id", 
"current_stage", "previous_stages"), class = "data.frame", row.names = c(NA, 
-4L))