Metazoan germ cells develop as sperm or oocytes, depending on chromosomal sex, extrinsic signaling from somatic tissue and intrinsic factors within the germ cells. Gamete fate regulatory networks have been analyzed in nematodes, flies and mammals, but only in C. elegans have terminal intrinsic regulators been identified, which include a Tob/BTG protein family member, FOG-3. Canonical Tob/BTG proteins function as monomeric adaptor proteins that link RNA binding proteins to deadenylases. To ask if FOG-3 functions similarly, we first determined its crystal structure. FOG-3 harbors a classical Tob/BTG fold, but unlike other Tob/BTG proteins, FOG-3 dimerizes and these FOG-3 dimers assemble into polymers. The importance of FOG-3 polymers to sperm fate specification was confirmed in vivo using CRISPR/Cas9 gene editing to create mutations designed to disrupt the polymer interface. The FOG-3 surface potential is highly basic, suggesting binding to nucleic acid. We find that FOG-3 binds RNA directly with a strong preference for 3'UTRs of oogenic mRNAs. Our results reveal a divergent but striking molecular assembly for proteins with a Tob/BTG fold, make key advances in understanding the mechanism of sperm fate specification and highlight the potential for undiscovered protein polymers in biology.