The island originally hosted a significant city, although it was abandoned sometime in the Second Era. Tiber Septim used Olenveld as his own personal graveyard, lining its abandoned streets with gravestones. Eventually the island city was forgotten, and Olenveld disappeared from the maps.
After Hannibal Traven became the Arch-Mage of the Mages Guild circa 3E 431, he introduced a ban on necromancy in the guild. Forced to go into hiding, a group of necromancers set out to find Olenveld and its plentiful supply of bodies. After spending a week searching for the island, they set up shop in the old Mages Guild building and became somewhat well-known for their ability to preserve sickly children by turning them undead.
Necromancers still inhabited Olenveld several centuries into the Fourth Era. In 4E 201, two necromancers set out from Olenveld in order to open a portal to the Soul Cairn. They established themselves within Brittleshin Pass in mainland Skyrim, where they managed to acquire the Staff of Hasedoki via the local smugglers. They planned to release Hasedoki's soul from the staff to earn the favor of the Ideal Masters, although they were both killed by the Last Dragonborn before this could occur.