Convicted killer Kenneth Eugene Smith violently convulsed during his nitrogen gas execution for 4 painfully long minutes, and the founder of a leading euthanasia group says he knew it wouldn't go smoothly when he previewed the prison set-up.

Dr. Philip Nitschke, founder of pro-euthanasia group "Exit International", spoke to "TMZ Live" Friday about the first-ever nitrogen gas execution ... saying he was concerned to see a gas mask set-up for Smith's Thursday night execution at the William C. Holman Correctional Facility in Atmore, Alabama.

He tells us he's long helped people wanting to end their lives with nitrogen in a quick, peaceful, and reliable way -- but ditched face masks 15 years earlier, because he found them to be unreliable and dangerous.

In layman's terms ... he says if the mask loses its seal on someone's face it alters the amount of air flowing in and slows the process, potentially leading to muscle convulsions.

Also, he tells us there was the obvious factor of Smith fighting his own death, which made his execution particularly excruciating.

Nitschke says when people choose to die on their own terms via nitrogen gas, they're encouraged to take long, deep breaths to make it quick -- but Smith, who he met in the prison during his visit -- made it clear he didn't want to die, took short shallow breaths, which added to his suffering.

Nitschke says he warned prison officials they were going about his execution the wrong way, but, clearly, that fell on deaf ears.

Smith became the first-ever inmate to be put to death utilizing nitrogen hypoxia ... basically, suffocating him.

Tormenting scenes played out across the board ... his wife Deanna cried in horror as her husband took his last breaths, prison officials gasped out loud in shock and Smith's Pastor Jeff Hood said it was the worst thing he's ever witnessed in his life.