I've had a few whale encounters a lot like this and yes, they are frightening. I've also had close encounters with sharks, including a great white. I can tell you fur sure that having a fully grown whale a couple of feet away from the kayak is much scarier than having a GW shark similarly close.
Here's a photo of the closest call I had with a whale on a kayak:
In this shot you can see my rod holder in the foreground, just to demonstrate how close it was:
And this is a great white:
In this vid you can see footage of both the shark and the whale encounters (both at the start). You have to look close to see the whale fin pop up next to my kayak, but if you catch it, you'll see how close it was.
[youtube]http://www.youtube.com/watch?v=tz085AeGHNs[/youtube]