Extensive studies on HIV-1 have led to the discovery of a variety of structurally and functionally diverse innate defense factors that target various steps of the retroviral replication cycle. Some of them, such as APOBEC3, tetherin, and SERINC5, are well established. Their importance is evident from the fact that HIV-1 uses its accessory proteins Vif, Vpu, and Nef to counteract them. However, the list of antiviral factors is constantly increasing, and accumulating evidence suggests that innate defense mechanisms, which restrict HIV-1 and/or are counteracted by viral proteins, remain to be discovered. These antiviral factors are relevant to diseases other than HIV/AIDS, since they are commonly active against various viral pathogens. In this review, we provide an overview of recently reported antiretroviral factors and viral countermeasures, present the evidence suggesting that more innate defense mechanisms remain to be discovered, and discuss why this is a challenging but rewarding task.