Captain Hunter and James Davey took up pastoral leases along the coast. The Government of the Port Phillip District commissioned a survey of the area in 1854 and land sales followed. Land along the coastline was purchased by wealthy businessmen who built mansions, such as Richard Grice of Manyung,
followed by his son James of Moondah, Captain Lintot of Earimil and J.T. Smith, Lord Mayor of Melbourne who owned the land where the Ranelagh Estate was established. Small landholders also settled here, Cahills, Woods, Hobsons, Moselys and Martins.
The community was small but active. They built St. James The Less Church in 1863/4 through local effort after Captain Lintot donated land for a primary school school and church to the Church of England. The school struggled and would close for several years at a time until the end of the 19th Century when regular numbers made it viable. The current site of the primary school was occupied in the 1920s.
Mt. Eliza’s population rose from the late 1950s to become the thriving village it is today.