Nigel Lythgoe is in the crosshairs of an investigation by the studio that produces his long-running show "So You Think You Can Dance" ... with his job possibly hanging in the balance.

Sources with direct knowledge tell TMZ ... the embattled EP of the hit reality competition show is at the center of an internal probe being conducted by Sony Pictures Television -- which co-produces 'SYTYCD' under its subsidiary 19 Entertainment, alongside Dick Clark Productions.

We're told Sony's looking into recent claims of sexual assault and harassment made against Nigel -- who created 'So You Think' with Simon Fuller, and has served as an executive producer since its inception, not to mention a consistent judge until he stepped away in '22.

Of course, what Sony's investigating here is Paula Abdul's allegations -- which she made in an explosive lawsuit ... claiming she'd been sexually assaulted by Lythgoe twice, once in the early 2000s during her 'Idol' stint and again in the 2010s while she judged on 'SYTYCD.'

Remember, as it pertains to the latter allegation -- which is what Sony's concerned with here -- she alleged Lythgoe invited her to his pad to discuss work opportunities ... only to allegedly pounce on her in an attempt to kiss her. PA also claimed Lythgoe groped one of her assistants during her time on the dance show.

Lythgoe denies Paula's claims, saying they're "deeply offensive" to everything he stands for and totally false.

Our sources say that while Sony digs into this matter, no decision has been made on whether Nigel will be allowed to stay on as an EP. It also remains to be seen if he'll serve as a judge -- this despite the fact he was announced to return to his chair last month for Season 18 ... which will start airing in March. He wasn't a judge during this past season.

Probably worth noting ... Paula named 19 Entertainment as a defendant in her suit too -- alleging sexual assault and battery, sexual harassment, gender violence and negligence.