The Iowa Court of Appeals has affirmed District Court Judge Michael Schilling's ruling in March 2012 that Fort Madison was fully warranted and justified in firing William Bowker in February 2010.
The city had said Bowker was fired for various acts of misconduct.
The acts included sleeping on the job and missing calls while serving on the Lee County Narcotics Task Force and having an affair with the wife of Police Chief Bruce Niggemeyer.
The opinion from the Iowa Court of Appeals, which was released Wednesday morning, states that "the district court appropriately considered Bowker's incidents of past misconduct in affirming his termination from the Fort Madison Police Department."
Bowker had argued that the incidents occuring during his time on the task force should not result in penalties related to his employment in the city.
He also said he was already punished by being removed from the task force prior to his termination.
The three-judge panel also said Schilling's ruling was not "arbitrary or capricious", as Bowker claimed, because of how the affair impacted the department's chain of command and the willingness of fellow officers to work with him.
Bowker can appeal the ruling to the Iowa Supreme Court.
You can read the full ruling by the Iowa Court of Appeals here.