As you know, I'm right there with you in the music department. "Headhunters" is one of my favorite albums of all time, right along with "B-----s Brew" and "A Love Supreme." Keep in mind that "A Love Supreme" was, for Coltrane, a religious (albeit fairly new-age) piece.
Still, there's no denying the close relationship between sex and both jazz and funk music. Jazz itself was born in the brothels of my neighbor city, New Orleans. As recounted by one musician, the term itself originated when a client was pushing the clock and the mistress would instruct the house pianist to "jazz it up," or in other words to "spice" the music up, to grab the attention of the customer and notify him that his time was almost up.
As for the drug connection with jazz, yes, many of the early innovators and stars of jazz were drug addicts (you forgot to mention heroin, which was an epidemic problem in the early jazz community, Charlie Parker being its most fabled victim. John Coltrane was also an addict, and credits God with bringing him out of his addiction.) But there is no connection between that and the music. Every man is sinful in some way. If we are to avoid the products of every sinful person, where would we be? :)
Funk has also been closely associated with sexuality. Exhibit A: James Brown. The rhythms are heavily syncopated which creates an innate urge to move the body. While many funk artists and fans use this as an aural aphrodisiac, it is not necessary to do so. It is perfectly possible to dance in a very modest way to funk music.
Ultimately, both art forms have evolved significantly from their inception to the point that in some cases, they bear little resemblance to their origins. Jazz fusion is a prime example of this. Regardless, I don't believe the art form itself is sinful. It depends on how it is used.
The disordered drives of mankind can (and WILL) pervert anything. For instance, there has been a longstanding trend in the adult industry of "nun porn." Just a thought to consider.
Now, for a last bit of encouragement and reassurance, I give you Fr. John Moulder, Catholic priest and jazz guitar virtuoso:
Catholic Priest Uses Jazz to Feed the Hungry