Primary and foreign keys are fundamental characteristics of relational databases, as originally noted in E.F. Codd’s paper, “A Relational Model of Data for Large Shared Data Banks”, published in 1970. The quote often repeated is, "The key, the whole key, and nothing but the key, so help me Codd."
In this post, Erin Stellato (@erinstellato) discusses the benefits of adding indexes on top of declared referential constraints.