I can't say that I've experienced exactly what you are going through, so my first and primary suggestion would be to visit a therapist or doctor to see if they have some pro tips on dealing with it without potentially placing yourself in harms way too much.
That being said, I've had a couple somewhat similar instances, triggered by a crash of my own. In 2012, at the track, I had what I call my "big crash." I was back riding 5 weeks later (as soon as I could operate a clutch), and while it slowed me down a bit (particularly in the corner I crashed in), it didn't stop me or stop my overall progress.
Then, almost a year later, while assisting at a crash scene that my group and I rolled up on, I had a flash back where some of the lost memories of my own crash filled in, but along with that I had a physical reaction as well. Not full on panic, I doubt anyone else noticed, but I sure did! Didn't think much of it until a few weeks later while sitting at a stop light the same thing happened where more memories kicked in and the physical reaction - almost like my body going into shock. Thankfully I was just sitting still and it passed after a minute, but that made me nervous about getting back to the track for fear that this could happen while riding in general.
In this case, I made an effort to remember and fill in those lost minutes since they were coming back on their own (I had a pretty good concussion at the time of the accident), and managed to fill in most of the blanks. I didn't have another incident after that, so I feel like it was almost more of a physical reaction to the memory than a "normal" panic attack.
This doesn't apply too much to your situation other than this: perhaps taking time to think about the things that scare you, imagine it in your mind (when you're NOT on a bike) and think of possible scenarios and means of dealing with it. Knowing your options is HUGE in overcoming fears, and realizing that you CAN control the situation may help prevent you from having a full panic attack. just a suggestion, again, not a professional in those fields at all!
Special thanks to my 2015 sponsors!
RiderzLaw; Dunlop Race Tires; Catalyst Reaction Suspension Tuning; MotoSport; Woodcraft; Shoei Helmets; Moto Shop; Precision Trackdays; Erich Stiegler Farmer's Insurance; Speed Mob; Cycle Gear; AXO; Driven Racing; City Bike; Chicken Hawk Racing
"Four wheels move the body, two wheels move the soul"
"Try not, do or do not. There is no try"