If you have a horse, it's bugged. I highly doubt you'll be able to remove it. Back when I was experimenting with the Bioware horse scripts, I noted characters could be permanently bugged if the horse wasn't properly removed.
Personally? I despise the horse scripting. There's a very noticeable lag/delay for everyone on the server when a horse spawns, and they corrupted my set bonus scripting by creating a player skin with a slew of variables (so you can cart stuff around in your horse saddlebags).
If I could redo horses with just the visuals and none of the variables/lag, I'd consider it. I'd keep it for the visuals, but it's just not a high priority.