A group of Conway's leading citizens met on the south edge of town in a grove of oak trees on March 9, 1880.  They established Oak Grove Cemetery and elected Colonel George W. Bruce as the first president of the Oak Grove Cemetery Association.  The first burial ceremony at the cemetery was for little Maude E. Ingram, 8 year old daughter of William and Agnes Ingram (July 11, 1881).  Shortly after its establishment, a few persons who had been buried on the McCulloch property near Hendrix College were moved to Oak Grove Cemetery.  Formal rites for the dedication of Oak Grove Cemetery were held on September 26, 1881, with the Rev. William M. Lea and the Rev. B. Williams officiating the consecration.  To provide water for flowers and shrubs, a well was dug and covered in April 1893. 

By 1945, all the lots within the cemetery's original boundary had been sold. The cemetery association began acquiring acreage on the east side. In 1955, approximately ten acres were purchased on the south edge of the cemetery.  The Robinette addition, a previously private cemetery of 3.5 acres was added on the west side.  Today, Oak Grove Cemetery covers 24 acres.

Most of Conway's leading citizens from its earliest era are buried here, making Oak Grove historically significant for the community. About 100 Civil War soldiers, both Union and Confederate, are interred along with veterans of other wars. Bronze stars on grave markers identify soldiers of the Confederate Army. 

A white iron gate was constructed at the entrance to Oak Grove in 1996.  A flag pole flying an American flag was installed near the center of the cemetery in 2002.  A gazebo was built nearby by the Robinette family and dedicated on Memorial Day, 2004.  A bronze plaque with a history of the cemetery was added in 2007.  In November 2009, the 10 acre Historic Section was officially listed on the National Register of Historic Places.