The area isn't zoned for horses and my property didn't say Horse Property on it., Like if I have a property that's big enough for the horse with proper covering and space, what kind of trouble would I end up in if I brought my horse to live on my property?
You need to investigate the details of ownership of your property. That usually means hiring a lawyer to do a title search and prepare a digest of laws that apply to your property. This might be a simple matter, and it might be a lot of legalese. For instance, "not being horse zoned" means nothing by itself. There has to be a law restricting horses on that property, and then you can talk about whether you can get an exception to that law.
If you do something, nothing happens until a neighbor complains. Then the city will send a letter demanding that you get rid of the horse. If you have had a horse for some time and nobody complained, then you have the legal high ground and should fight the order.