Join the club!
I have swivel at the top, and if you shop around you should be able to get an anchor swivel for less http://www.marinepartdepot.com/new316ststan.html
I wish I could tell you what you're doing wrong, but I'm still trying to figure that out myself witrh my limited experience using the Jennaker. One issue has been getting a consistant number of wraps as it furls. I've found myself with no line left on the drum, or too much line on the drum, and plenty of sail still to be furled among oter issues similar to yours. I know it makes no sense as it's just an oversized Jib, but they sure seem to have a mind of their own.
You could wrap a bungee around it to keep it furled until it's ready to use, but this is really a band aid, rather than a cure.
If you figure it out...let us all know!