Cavendish made contact with Sagan in the final sprint, being forced into barriers, crashing to the ground and suffering a to-be-diagnosed shoulder injury.
Sagan was initially docked 30 seconds after nudging Cavendish into the barriers at the end of stage four, which was won by Frenchman Arnaud Demare in Vittel.
The race's yellow jersey holder, Britain's Geraint Thomas, hit the deck in a separate pile-up near the end, but as the incident occurred within the final three kilometers he will be credited with the same time as the victor. The crash left Cavendish needing treatment for his injuries and further examinations in a hospital.
"If he came across that is one thing but I am not a fan of him putting his elbow out like that", Cavendish vented after the stage had finished.
Cavendish falls back into the road and is crashed into by other riders Christophe Ena AP
Australian Michael Matthews was second on the stage with Dan Martin of Ireland third, just ahead of Olympic champion Greg Van Avermaet in fourth. "I'd just like to speak to him about it".
After the crash, Sagan went over to see how Cavendish was and patted him on the back, while the British rider showed him his wounds.
The 30-year-old has one Tour stage win to his name, taken in Bagneres-de-Bigorre in 2013, but that was at a time when he came to grand tours targeting individual stages.
Thomas scraped his knee but said it was OK.
Michigan Gas Prices Fall For July 4th Holiday
In Connecticut, the state's average of $2.43 is a penny less compared to last week; and is the same as this time past year . Hawaii at $3.05 and California at $2.94 are the two states with the highest average prices in the nation.
The world time trial champion, who had hoped to win the Stage 1 time trial in Düsseldorf and sport the yellow jersey, instead started Stage 4 sitting almost 13 minutes down on the general classification.
Sagan, who has since apologised to Cavendish, said: "Mark was coming pretty fast from the back".
"The guys look after me incredibly well, it's not like I'm on my own all the time", he said. The Quick-Step Floors rider's brilliant effort in Longwy - which returned in the Tour de France after a 35-year hiatus - saw him take a total of six seconds on all the other race favourites, a welcomed bonus that helped him move up to 15th place in the overall rankings. "Winning in the Tour is the best".
Jeff Horn shocks Manny Pacquiao
But boxing experts were almost unanimous in their opinion that Horn received a gift, and Pacquiao was the bout's true victor . Meanwhile, trainer Freddie Roach said he and Pacquiao will soon discuss whether to continue fighting or call it quits.
Wednesday's lumpy 160.5km stage from Vittel to La Planche des Belles Filles concludes with the first mountain-top finish of the race - a rendez-vous which is bound to spice up the battle for yellow.
Information from The Associated Press was used in this story.
Trump talks unity with Gulf state leaders
Qatar has been under unprecedented diplomatic and economic sanctions for weeks from Saudi Arabia and its allies. Qatar denied all allegations, but the country's authorities stated that they were open to negotiations.