However you look at it, the key fact is insulation and heat input.
As far as heating is concerned the cheapest heat is still gas and probably the best form of heating is the standard radiator, quick to warm up and quick to cool down - probably cheapest to fit as an extension to your existing heating.
The quick to cool down is important when the sun comes out, thermostatic valves are a great help.
The insulation side is harder, double glazing as far as possible, insulated floor and walls.
By its very nature a conservatory will always loose heat.