I semi feel the same way. Although I'm studying graphic design. It seems like a lot of jobs in my industry as well are asking for people with a BA or BS in GD or "other related fields". Even though there's really nothing special after our Associates, it's just working on portfolio pieces in your spare time. This makes me think that the people asking for these positions have no clue what they're really asking for, they just need someone to "draw the pretty pictures" when they feel like they need something new to stay ahead of their competition. I went to a 4 year university that has the same accreditation as my 2 year college does and looked at their student portfolios. It was the exact same skill level as ours. On some projects, our stuff was better anyways. The 4 year students seems to be trained to just do and not trained to think. The only advantage they had was they obviously had more time to work on projects, but didn't come out with a higher quantity.
I'm somewhat at an advantage because I studied engineering prior so I learned some Solidworks, Rhino3d and some small amount of manufacturing processes. So what I've been doing lately is instead of "rebranding" or just simply coming up with lets say a packaging label design, I can come up and model a conceptual bottle and put my design on it. I'm also teaching myself some Alias (which is actually not that hard once you get used to it and stop trying to treat it like a solid modeling program).
In short, do what you want, but since the market is asking for 4 year degrees, might as well just go get it. In your spare time, dabble in some other forms of design and try to incorporate your ID skills into it, or maybe learn some Adobe products. InDesign could help you make a physical portfolio and learning how to make a Wordpress site could be your online presence. You could be the greatest designer in the world, but nobody would know about it unless you get yourself out there.
Sorry for the rambling.