Madonna had some words for Andy Cohen ... playfully airing her grievances with him after spotting him at her "Celebration" tour in NYC.

The Queen of Pop advised the talk show host to steer clear of any future digs on "Watch What Happens Live" ... with a clip showing her walking up to him with a bottle in hand during Saturday's show, asking, "How f****** lucky am I, Andy?"

Waiting for your permission to load the Instagram Media.

"If you say one more bad thing about me on your show," she began jolting her drink forward enough for some spillage near Andy -- cautioning: "Oh yeah, you're gonna be in so much trouble. You little troublemaking queen!!"

A public talking to from Madonna would make many want to dissipate into a deep, dark hole -- but Andy was delighted by the scolding -- mouthing "I love you" to the star.

In fact, he was so thrilled he shared the clip on IG, writing that as a lifelong Madonna fan, he was blown away to be called a "troublemaking queen" -- and couldn't wait to go back.

The duo's paths have crossed before IRL ... with them posing together for a snap as far back as 2009 ... though it looks like Andy's consistently found himself in the fan zone with her.

As for what jibes Madonna was referring to ... we don't know for sure, though earlier this year, Andy said he was so over the Material Girl not materializing on his show.

At this rate ... it seems like only a prayer can help make that happen.