Actually, just the first two measures form a very conventional and familiar Bossa Nova rhythm, but he extends it to make the unbroken dotted quarter note pulse. He plays that with his left hand on the snare drum and toms, so the complete groove looks like this:
It's a familiar feature of Elvin's playing that he will emphasize the dotted quarter note, essentially playing in 3 while the rest of the band is in 4, but it's still rather striking that he does it so aggressively here. Just so we're clear, the tune has normal eight-measure phrases— if you're going to practice this groove, you have to emphasize the eight-measure phrases of the tune, not the three-measure phrase of the drum pattern.
You can move your left hand to the toms where the music builds. There's nothing significant going on with the left foot— you can play it conventionally on 2 and 4 if you want. At a few points Elvin breaks up the bass drum part like this, accenting the bass drum strongly, further emphasizing the cross rhythm. Listen for this in the middle of the saxophone solo:
He maintains this groove pretty consistently, but he does break it up to fit the tune, like with these phrase-ending fills early in the track— these are each measures 7 and 8 of their phrase:
At the beginning of a new phrase, after playing a fill, he usually does start the pattern at the beginning, with the rim click on 2. But not always— he begins the piano solo with a rim click on the & of 2, continuing the dotted-quarter rhythm from there. If you're going to use this in your playing, don't over-think it— use your ears and play instinctively. We want to know what Elvin is doing, but in real playing just learn the patterns, and then put all your focus on playing the tune at hand.