I can relate in the opposite way. I started on Medium writing humor. And when I added fiction, it initially got ignored. People wanted me to be funny.

But I didn't want to be pigeonholed as a humor writer, so I persevered writing fiction and added poetry and essays, LOL!

If the humor readers didn't like it, that was their problem (and loss), not mine.

I add category kickers above my stories so readers can ignore the genres they don't want to read. I think I have more readers now than if I had stuck to a niche, which I find boring! And most of my loyal readers have come to accept that I write many things, not just humor. And they read them.

Of course, I don't write for a living, so I have that freedom.

Stick to it. People will come around.