The first settlement in the Town of Elba was in 1843. The town is bounded on the west by Columbia County, on the north by Town of Calamus, east by Town of Lowell and south by Town of Portland. Also included within the Town of Elba are the unincorporated communities of Astico and Danville. Miles Burnham along with is his cousins, Morris Burnham and Samuel Hasey, were the first settlers, they were natives of New York. Miles purchased 40 acres of land where Danville is currently located. Miles was the first township clerk as well as being a justice of the peace, serving on the board of county of supervisors and in the general assembly of the state. The first settlers were generally from the east. Later the Irish predominated and in the northern part there was a large settlement of Welsh.

The first election was held at the cabin of a Mr. Robinson at a place called Ox Bow, on account of the river presenting that appearance near his house. There were two tickets in the field for town officers, one representing the landowners and the other the "squatters." The result of the ballot was a tie. In consequence thereof the business of the town for that year was transacted at Lowell. In the next election the landowners were victorious, and the town was fully organized. The secretary of state, having been consulted, decided upon the name of Elba, after Mr. Thompson and Mr. Robinson had failed to agree on what the new town should be called.

The current population of the Town of Elba is approximately 998 and remains predominately agricultural.