Normally the Prime Directive would forbid contact between Starfleet from open contact with a species incapable of interstellar communication. It is generally taken for granted that a similar rule or law must be in force for Federation civilians. It is hard to imagine the Horta could have warp drive, or even something like subspace radio. The Horta are almost obligate subterranean dwellers, finding even an open cavern uncomfortable compared to their own narrow tunnels. It is unlikely any Horta has ever deliberately traveled to Janus VI's surface. Even if one were to, they give every indication of being blind in the conventional sense-- or at least poor enough of vision that it took a mind meld for the Mother Horta to even understand what her unexpected guests even looked like. The Horta simply cannot look up to the stars and wonder, as other species can. They live in a world where the idea of flight could barely occur to anybody. It's debatable whether even the idea of tool use would be meaningful in such an environment (though the Mother Horta clearly understood the idea enough to understand how to sabotage the miners' life support.) And yet they are clearly an intelligent and sophisticated people, if alien. Extracanonical materials see the Horta join the Federation and become a vital part of its mining infrastructure, and even on a few occasions show mature Horta who have joined Starfleet, serving on such prestigious vessels as Kirk's Enterprise and Riker's Titan.
There is no indication that either the Federation miners nor the crew of the Enterprise encountered any reprisals for establishing a formal relationship with the Horta, even despite the fact that in the former case, their actions so interfered in the future of the species and their indigenous culture as to actively jeopardize their survival! Granted the miners had no way of assuming that the Mother Horta was sapient until long after the deed was done, but it was certainly clear to Spock before he attempted the Mind Meld.
This raises the question: why? Does Starfleet carve out some sort of exception for a species that may be incapable of ever meeting the standards for first contact? Did the fact that the contact by civilians was completely accidental create a justifiable breach, as it seems to have in "A Private Little War", and if so does that mean that in Picard's era Starfleet would have been obligated to ignore the Horta as the interpretation of the Prime Directive grew stricter?