Sponges are engaged in intimate symbioses with a diversity of microorganisms from all three domains of life, namely Bacteria, Archaea and Eukarya. Sponges have been well studied and categorized for their bacterial communities, some displaying a high microbial abundance (HMA), while others show low microbial abundance (LMA). However, the associated Archaea and Eukarya have remained relatively understudied. We assessed the bacterial, archaeal and eukaryotic diversities in the LMA sponge species Dysidea avara and Dysidea etheria by deep amplicon sequencing, and compared the results to those in the HMA sponges Aplysina aerophoba and Aplysina cauliformis. D. avara and A. aerophoba are sympatric in the Mediterranean Sea, while D. etheria and A. cauliformis are sympatric in the Caribbean Sea. The bacterial communities followed a host-specific pattern, with host species identity explaining most of the variation among samples. We identified OTUs shared by the Aplysina species that support a more ancient association of these microbes, before the split of the two species studied here. These shared OTUs are suitable targets for future studies of the microbial traits that mediate interactions with their hosts. Even though the archaeal communities were not as rich as the bacterial ones, we found a remarkable diversification and specificity of OTUs of the family Cenarchaeaceae and the genus Nitrosopumilus in all four sponge species studied. Similarly, the differences in fungal communities were driven by sponge identity. The structures of the communities of small eukaryotes such as dinophytes and ciliophores (alveolates), and stramenopiles, could not be explained by either sponge host, sponge genus or geographic location. Our analyses suggest that the host specificity that was previously described for sponge bacterial communities also extends to the archaeal and fungal communities, but not to other microbial eukaryotes.